Problem
Providers like xAI, Mistral, Nebius, Ollama, and other custom OpenAI-compatible endpoints route through pkg/model/provider/oaistream with whatever Provider string is set in the model config (e.g. "xai", "mistral", "ollama"). The modelcaps system looks these up in models.dev, but the provider IDs in models.dev may not match the provider strings used in docker-agent configs.
For example:
- An xAI model config with
provider: xai, model: grok-2-vision-1212 produces a lookup for "xai/grok-2-vision-1212"
- An Ollama config with
provider: ollama, model: llava produces "ollama/llava"
If models.dev doesn't have those exact keys, capability detection silently falls back to text-only and image/PDF attachments are dropped.
Additionally, user-configured aliases (via display_model) are already handled correctly by ModelCapsID() (which uses Model, not DisplayOrModel()), but the underlying provider identifier mapping to models.dev still depends on the config's provider field matching models.dev exactly.
Possible fixes
- Maintain a mapping table from docker-agent provider strings to models.dev provider IDs
- Allow per-model capability overrides in the agent config YAML
- Warn (rather than silently drop) when a model ID is not found in models.dev, so users can diagnose the issue
Context
Identified during review of PR #2738 (fix: pass fully-qualified provider/model ID to modelcaps.Load).
Problem
Providers like xAI, Mistral, Nebius, Ollama, and other custom OpenAI-compatible endpoints route through
pkg/model/provider/oaistreamwith whateverProviderstring is set in the model config (e.g."xai","mistral","ollama"). Themodelcapssystem looks these up in models.dev, but the provider IDs in models.dev may not match the provider strings used in docker-agent configs.For example:
provider: xai, model: grok-2-vision-1212produces a lookup for"xai/grok-2-vision-1212"provider: ollama, model: llavaproduces"ollama/llava"If models.dev doesn't have those exact keys, capability detection silently falls back to text-only and image/PDF attachments are dropped.
Additionally, user-configured aliases (via
display_model) are already handled correctly byModelCapsID()(which usesModel, notDisplayOrModel()), but the underlying provider identifier mapping to models.dev still depends on the config'sproviderfield matching models.dev exactly.Possible fixes
Context
Identified during review of PR #2738 (fix: pass fully-qualified provider/model ID to modelcaps.Load).