-
Notifications
You must be signed in to change notification settings - Fork 572
fix(litellm): fix gen_ai.request.messages to be as expected
#5255
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
fix(litellm): fix gen_ai.request.messages to be as expected
#5255
Conversation
…inal messages and handle data URLs correctly
Semver Impact of This PR🟢 Patch (bug fixes) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨
Bug Fixes 🐛Litellm
Other
Documentation 📚
Internal Changes 🔧Release
Other
🤖 This preview updates automatically when you update the PR. |
sentry_sdk/integrations/litellm.py
Outdated
| return { | ||
| "type": "blob", | ||
| "modality": "image", | ||
| "mime_type": mime_type, | ||
| "content": content, | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
litellm accepts input which follows OpenAI format, is there an opportunity to deduplicate with the openai-python input handling here?
…AI messages Add transform_content_part() and transform_message_content() functions to standardize content part handling across all AI integrations. These functions transform various SDK-specific formats (OpenAI, Anthropic, Google, LangChain) into a unified format: - blob: base64-encoded binary data - uri: URL references (including file URIs) - file: file ID references Also adds get_modality_from_mime_type() helper to infer content modality (image/audio/video/document) from MIME types.
Replace local _convert_message_parts implementation with the shared transform_message_content function, removing ~50 lines of duplicated code.
Add dedicated transform functions for each AI SDK: - transform_openai_content_part() for OpenAI/LiteLLM image_url format - transform_anthropic_content_part() for Anthropic image/document format - transform_google_content_part() for Google GenAI inline_data/file_data - transform_generic_content_part() for LangChain-style generic format Refactor transform_content_part() to be a heuristic dispatcher that detects the format and delegates to the appropriate specific function. This allows integrations to use the specific function directly for better performance and clarity, while maintaining backward compatibility through the dispatcher for frameworks that can receive any format. Added 38 new unit tests for the SDK-specific functions.
LiteLLM uses OpenAI's message format, so use the OpenAI-specific transform_openai_content_part function directly for better performance. Also update test to reflect correct behavior: data URIs without base64 encoding are still inline data and should be treated as blobs, not URIs.
Issues
Closes https://linear.app/getsentry/issue/TET-1635/redact-images-litellm