Skip to content

Conversation

@constantinius
Copy link
Contributor

@constantinius constantinius requested a review from a team as a code owner December 17, 2025 16:24
@linear
Copy link

linear bot commented Dec 17, 2025

…inal messages and handle data URLs correctly
Base automatically changed from constantinius/fix/redact-message-parts-type-blob to master January 13, 2026 09:56
@github-actions
Copy link
Contributor

github-actions bot commented Jan 13, 2026

Semver Impact of This PR

🟢 Patch (bug fixes)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


New Features ✨

  • feat(ai): add parse_data_uri function to parse a data URI by constantinius in #5311
  • feat(asyncio): Add on-demand way to enable AsyncioIntegration by sentrivana in #5288
  • feat(openai-agents): Inject propagation headers for HostedMCPTool by alexander-alderman-webb in #5297
  • feat: Support array types for logs and metrics attributes by alexander-alderman-webb in #5314

Bug Fixes 🐛

Litellm

  • fix(litellm): fix gen_ai.request.messages to be as expected by constantinius in #5255
  • fix(litellm): Guard against module shadowing by alexander-alderman-webb in #5249

Other

  • fix(ai): redact message parts content of type blob by constantinius in #5243
  • fix(clickhouse): Guard against module shadowing by alexander-alderman-webb in #5250
  • fix(gql): Revert signature change of patched gql.Client.execute by alexander-alderman-webb in #5289
  • fix(grpc): Derive interception state from channel fields by alexander-alderman-webb in #5302
  • fix(pure-eval): Guard against module shadowing by alexander-alderman-webb in #5252
  • fix(ray): Guard against module shadowing by alexander-alderman-webb in #5254
  • fix(threading): Handle channels shadowing by sentrivana in #5299
  • fix(typer): Guard against module shadowing by alexander-alderman-webb in #5253
  • fix: Stop suppressing exception chains in AI integrations by alexander-alderman-webb in #5309
  • fix: Send client reports for span recorder overflow by sentrivana in #5310

Documentation 📚

  • docs(metrics): Remove experimental notice by alexander-alderman-webb in #5304
  • docs: Update Python versions banner in README by sentrivana in #5287

Internal Changes 🔧

Release

  • ci(release): Bump Craft version to fix issues by BYK in #5305
  • ci(release): Switch from action-prepare-release to Craft by BYK in #5290

Other

  • chore(gen_ai): add auto-enablement for google genai by shellmayr in #5295
  • chore: add unlabeled trigger to changelog-preview by BYK in #5315
  • chore: Add type for metric units by sentrivana in #5312
  • ci: Update tox and handle generic classifiers by sentrivana in #5306

🤖 This preview updates automatically when you update the PR.

Comment on lines 90 to 95
return {
"type": "blob",
"modality": "image",
"mime_type": mime_type,
"content": content,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

litellm accepts input which follows OpenAI format, is there an opportunity to deduplicate with the openai-python input handling here?

https://docs.litellm.ai/docs/completion/input#common-params

…AI messages

Add transform_content_part() and transform_message_content() functions
to standardize content part handling across all AI integrations.

These functions transform various SDK-specific formats (OpenAI, Anthropic,
Google, LangChain) into a unified format:
- blob: base64-encoded binary data
- uri: URL references (including file URIs)
- file: file ID references

Also adds get_modality_from_mime_type() helper to infer content modality
(image/audio/video/document) from MIME types.
Replace local _convert_message_parts implementation with the shared
transform_message_content function, removing ~50 lines of duplicated code.
Add dedicated transform functions for each AI SDK:
- transform_openai_content_part() for OpenAI/LiteLLM image_url format
- transform_anthropic_content_part() for Anthropic image/document format
- transform_google_content_part() for Google GenAI inline_data/file_data
- transform_generic_content_part() for LangChain-style generic format

Refactor transform_content_part() to be a heuristic dispatcher that
detects the format and delegates to the appropriate specific function.

This allows integrations to use the specific function directly for
better performance and clarity, while maintaining backward compatibility
through the dispatcher for frameworks that can receive any format.

Added 38 new unit tests for the SDK-specific functions.
LiteLLM uses OpenAI's message format, so use the OpenAI-specific
transform_openai_content_part function directly for better performance.

Also update test to reflect correct behavior: data URIs without base64
encoding are still inline data and should be treated as blobs, not URIs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants