Skip to content

Instrument formatter operations to provide visibility into template rendering and response parsing #471

@psschwei

Description

@psschwei

Description:
Instrument formatter operations to provide visibility into template rendering and response parsing.

Detailed Requirements:

  1. Add spans for formatter operations:
    • formatter.apply - Overall formatter application
    • formatter.render_prompt - Prompt template rendering
    • formatter.parse_response - Response parsing/extraction
  2. Add attributes:
    • formatter_type - Class name of formatter
    • template_name - Template identifier if applicable
    • input_length - Input character count
    • output_length - Output character count
  3. Handle multiple formatter types in codebase
  4. Capture parsing errors as span events

Files to Modify:

  • mellea/stdlib/formatters/ - Instrument formatter classes
  • mellea/core/formatter.py - Base formatter instrumentation

Acceptance Criteria:

  • Formatter spans created during prompt/response processing
  • Format type and sizes captured
  • Parse errors recorded as span events
  • Works with all formatter implementations

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions