Python: Not able to read attached files when carryout simple analysis with csv files

[nicogarciaara]

Description

I have a chatbot application, with Foundry agents that are already created (and I don't want or shouldn't create agents on the fly).

If a user uploads a csv file, I want to be able to make simple data analysis questions (for example, what is the average of column x)

However, when uploading things to the Agents Client in Azure AI and using that file in the codeInterpreter I still get a: "Please upload the file, and I will analyze the data for you." answer, I have looked at multiple documentation pages and this seems to be the best approach but I cannot make it work

Code Sample

import asyncio
import os

from dotenv import load_dotenv
from azure.identity.aio import AzureCliCredential

from agent_framework import ChatAgent, ChatMessage, TextContent, DataContent
from agent_framework.azure import AzureAIAgentClient
from agent_framework import HostedCodeInterpreterTool
import os
import asyncio
from azure.ai.agents.aio import AgentsClient
from azure.ai.agents.models import FilePurpose
from azure.identity.aio import DefaultAzureCredential


import base64

async def check_models_functioning():
    load_dotenv()  # Load variables from .env into environment

    foundry_project_endpoint = os.getenv('FOUNDRY_PROJECT_ENDPOINT')
    if not foundry_project_endpoint:
        raise ValueError("Missing FOUNDRY_PROJECT_ENDPOINT in environment/.env")


    filename = os.getenv('SAMPLE_CSV_FILE')
    
    async with AzureCliCredential() as cred:
        agents_client = AgentsClient(endpoint=foundry_project_endpoint, credential=cred)
        async with agents_client:
            uploaded = await agents_client.files.upload_and_poll(
                file_path=filename,
                purpose=FilePurpose.AGENTS,
            )
            print("file_id:", uploaded.id)

    # AzureCliCredential is async and should be closed properly
    async with AzureCliCredential() as credential:
        agent = ChatAgent(
            chat_client=AzureAIAgentClient(
                agent_id=os.getenv('GPT-4O-AGENT'),
                project_endpoint=foundry_project_endpoint,
                model_deployment_name="gpt-4o",
                credential=credential,
            ),
            instructions="You are a helpful assistant that can analyze and read attached data files using Python code.",
            tools=HostedCodeInterpreterTool(inputs=[{"file_id": uploaded.id}])
        )
        message = ChatMessage(role='user', contents=[TextContent(f'Hello! Analyze the attached file How many records do you find here? What is the average number of X?')])
        result = await agent.run(message)
        print(result.text)


def main():
    asyncio.run(check_models_functioning())


if __name__ == "__main__":
    main()

Error Messages / Stack Traces

Package Versions

agent-framework-core==1.0.0b251218, agent-framework-azure-ai==1.0.0b251218, agent-framework-anthropic==1.0.0b251223

Python Version

Python 3.10

Additional Context

No response

[giles17]

Hi @nicogarciaara, thanks for reporting this!

A fix has been opened in PR #4201. Here are a few things to note:

1. Update to the latest version

HostedCodeInterpreterTool has been removed from the framework. Hosted tools like code interpreter are now accessed through factory methods on the client classes. Please update your packages to the latest version

2. Recommended: Use AzureAIClient (V2 client)

If possible, we recommend using AzureAIClient instead of AzureAIAgentClient. It already supports file_ids and provides a more complete experience:

   from agent_framework import Agent
   from agent_framework.azure import AzureAIClient, AzureAIProvider
   from azure.identity.aio import AzureCliCredential

   async with (
       AzureCliCredential() as credential,
       AzureAIProvider(credential=credential) as provider,
   ):
       client = AzureAIClient(credential=credential)
       tool = client.get_code_interpreter_tool(file_ids=["your-file-id"])

       agent = await provider.create_agent(
           name="DataAnalyst",
           instructions="You are a helpful assistant that analyzes data files using Python code.",
           tools=[tool],
       )
       response = await agent.run("Analyze the attached CSV. What is the average of column X?")
       print(response.text)

3. Using AzureAIAgentClient (V1 client)

If you need to stay on AzureAIAgentClient, the fix in #4201 adds file_ids support to its factory method as well. This code sample should work when the PR is merged:

   from agent_framework.azure import AzureAIAgentClient, AzureAIAgentsProvider
   from azure.ai.agents.aio import AgentsClient
   from azure.ai.agents.models import FilePurpose
   from azure.identity.aio import AzureCliCredential

   async with (
       AzureCliCredential() as credential,
       AgentsClient(endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"], credential=credential) as agents_client,
       AzureAIAgentsProvider(agents_client=agents_client) as provider,
   ):
       # Upload the file
       uploaded = await agents_client.files.upload_and_poll(
           file_path="your_data.csv",
           purpose=FilePurpose.AGENTS,
       )

       # Create code interpreter tool with the uploaded file
       client = AzureAIAgentClient(credential=credential)
       tool = client.get_code_interpreter_tool(file_ids=[uploaded.id])

       agent = await provider.create_agent(
           name="DataAnalyst",
           instructions="You are a helpful assistant that analyzes data files using Python code.",
           tools=[tool],
       )
       response = await agent.run("Analyze the attached CSV. What is the average of column X?")
       print(response.text)

[nicogarciaara]

Hi @giles17 thanks for your answer!

I still have a question there, because in the code you are describing, files are being attached to the agent, and if two users are using the same agent, content should be shared (which is not desired). Is there a way that the file is only attached to a message sent to an LLM, similar to what happens in AzureAI:

import os
from azure.identity import DefaultAzureCredential
from azure.ai.agents import AgentsClient
from azure.ai.agents.models import (
    CodeInterpreterTool,
    FilePurpose,
    MessageAttachment,
)

from dotenv import load_dotenv

load_dotenv()

endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]            # mismo endpoint del proyecto
model = "gpt-4o"
file_path = "sample_ventas_tests.csv"

with AgentsClient(endpoint=endpoint, credential=DefaultAzureCredential()) as agents_client:
    # 1) Create agent with code interpreter
    agent = agents_client.create_agent(
        model=model,
        name="ci-agent-test",
        instructions="Uses the code interpreter to analize uploaded files when needed.",
        tools=CodeInterpreterTool().definitions,
    )

    # 2) Thread
    thread = agents_client.threads.create()

    # 3) Upload file
    f = agents_client.files.upload_and_poll(file_path=file_path, purpose=FilePurpose.AGENTS)

    # 4) Attach file ONLY to this message
    attachment = MessageAttachment(file_id=f.id, tools=CodeInterpreterTool().definitions)
    agents_client.messages.create(
        thread_id=thread.id,
        role="user",
        content="Analize the attached CSV file.",
        attachments=[attachment],
    )

    # 5) Run + polling
    run = agents_client.runs.create_and_process(thread_id=thread.id, agent_id=agent.id)
    print("Run status:", run.status)

    # 6) Read Answer
    msgs = agents_client.messages.list(thread_id=thread.id)
    for m in msgs:
        if m.role == "assistant":
            print(m)

    # 7) “One-shot” interaction: delete file
    agents_client.files.delete(f.id)