Skip to content

DMR local models: vision/PDF attachments always degrade to text-only #2739

@simonferquel-clanker

Description

@simonferquel-clanker

Problem

When using DMR-hosted local models, pkg/model/provider/dmr calls oaistream.ConvertMessages with modelcaps.NewID("dmr", modelName). The modelcaps.Load function looks up capability data in models.dev, but DMR is not a known provider there — so the lookup always misses and attachments fall back to the conservative text-only strategy (StrategyDrop for images/PDFs).

Root cause

models.dev does not contain entries for DMR-hosted models. The modelcaps system depends entirely on the models.dev database for capability detection, so any model not registered there silently loses multimodal support.

Possible fixes

  • Maintain a local override/allowlist for known DMR models that support vision/PDF
  • Support a provider-level opt-in (e.g. a capability hint in the model config)
  • Query DMR's own capability API if one exists

Context

Identified during review of PR #2738 (fix: pass fully-qualified provider/model ID to modelcaps.Load).

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions