Skip to content

Instrument tool calling to track tool invocations, arguments, and results #474

@psschwei

Description

@psschwei

Description:
Instrument tool calling to track tool invocations, arguments, and results.

Detailed Requirements:

  1. Add spans for tool operations:
    • tool.call - Tool invocation
  2. Add attributes:
    • tool.name - Name of the tool being called
    • tool.arguments_hash - Hash of arguments (for privacy)
    • tool.status - success/error
  3. Add span events for:
    • Tool call start with argument summary (redacted)
    • Tool call completion with result summary
    • Tool call error with exception details
  4. Handle tool call extraction from model responses
  5. Track multiple tool calls in single response

Files to Modify:

  • mellea/stdlib/tools/interpreter.py - Tool execution instrumentation
  • mellea/backends/tools.py - Tool call extraction
  • mellea/helpers/openai_compatible_helpers.py - OpenAI tool handling

Span Hierarchy Example:

backend.request
├── [model returns tool calls]
└── tool.call (tool_1)
    └── [tool execution]
└── tool.call (tool_2)
    └── [tool execution]

Acceptance Criteria:

  • Each tool invocation has a span
  • Tool name and status captured
  • Arguments handled with privacy in mind
  • Multiple tool calls tracked separately

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions