Background
Literal constraints on tool fields are silently dropped on the way to the
backend and on the way to validation. Two independent gaps combine to produce
the same end-user symptom — strict=True isn't actually strict for
Literal[...] fields.
This isn't introduced by #1104; both gaps predate it. PR #1104 made the
validator-side gap visible because discriminated-union tools now reach the
validator with Literal tag constraints in their schema for the first time.
End-user impact
strict=True isn't truly strict. Two reproducers, one for each gap:
Gap A — discriminated-union branches
class Cat(BaseModel):
kind: Literal["cat"]
name: str
class Dog(BaseModel):
kind: Literal["dog"]
name: str
breed: str
def act(pet: Annotated[Cat | Dog, Field(discriminator="kind")]) -> str:
...
mt = MelleaTool.from_callable(act)
validate_tool_arguments(
mt,
{"pet": {"kind": "horse", "name": "Bob", "breed": "lab"}},
strict=True,
)
# returns successfully — "horse" is not a valid kind
After #1104, the schema produced for pet correctly includes
{"const": "cat", "type": "string"} on each branch's kind field. The
validator receives that constraint but ignores it: _build_pydantic_type_from_schema
in mellea/backends/tools.py:549 maps everything with type: "string" to
plain str, never inspecting const or enum.
Gap B — plain Literal fields
def file_op(path: str, mode: Literal["read", "write"]) -> str:
...
mt = MelleaTool.from_callable(file_op)
validate_tool_arguments(
mt,
{"path": "/etc/passwd", "mode": "delete"},
strict=True,
)
# returns successfully — "delete" is not in the Literal
For plain Literal fields the enum keyword is also lost in the schema
itself: Pydantic emits {"enum": ["read", "write"], "type": "string"} for
mode, but the simple-type flattening near the bottom of
convert_function_to_ollama_tool keeps only the type and discards the
enum array. So the constraint is invisible to both the backend (the
LLM doesn't see the allowed values either) and the validator.
When this matters in practice
Most of the time, LLMs emit values that are in the schema, so this isn't a
frequent runtime problem. It does matter when:
- smaller models may hallucinate a tag value the schema forbids
- you want defensive validation against malformed or hostile payloads
- a test suite expects
strict=True to actually reject every malformed shape
- the model never saw the allowed values at all (Gap B), so it has no chance
of knowing them
Workaround
Attach a field_validator to the Pydantic model, or post-validate arguments
manually before invoking the tool.
Success criteria
- The schema produced by
convert_function_to_ollama_tool preserves
enum and const keywords on simple-type fields. Plain Literal[...]
fields should arrive at backends with the allowed values intact.
validate_tool_arguments(..., strict=True) rejects any payload where a
field constrained by Literal[...], Enum, or schema-level const /
enum carries a value outside the allowed set. Both reproducers above
should raise ValidationError.
- A negative test for each reproducer should land alongside the fix.
Suggested direction
Two small changes, ideally in one PR:
-
In the simple-type flattening branch of convert_function_to_ollama_tool,
preserve enum and const keywords on the output property when present
on the input.
-
In _build_pydantic_type_from_schema, recognise:
{"const": V} → typing.Literal[V]
{"enum": [V1, V2, ...]} → typing.Literal[V1, V2, ...]
before falling through to the plain-type mapping.
Together this closes both gaps with a few branches of new logic.
Background
Literalconstraints on tool fields are silently dropped on the way to thebackend and on the way to validation. Two independent gaps combine to produce
the same end-user symptom —
strict=Trueisn't actually strict forLiteral[...]fields.This isn't introduced by #1104; both gaps predate it. PR #1104 made the
validator-side gap visible because discriminated-union tools now reach the
validator with
Literaltag constraints in their schema for the first time.End-user impact
strict=Trueisn't truly strict. Two reproducers, one for each gap:Gap A — discriminated-union branches
After #1104, the schema produced for
petcorrectly includes{"const": "cat", "type": "string"}on each branch'skindfield. Thevalidator receives that constraint but ignores it:
_build_pydantic_type_from_schemain
mellea/backends/tools.py:549maps everything withtype: "string"toplain
str, never inspectingconstorenum.Gap B — plain
LiteralfieldsFor plain
Literalfields theenumkeyword is also lost in the schemaitself: Pydantic emits
{"enum": ["read", "write"], "type": "string"}formode, but the simple-type flattening near the bottom ofconvert_function_to_ollama_toolkeeps only thetypeand discards theenumarray. So the constraint is invisible to both the backend (theLLM doesn't see the allowed values either) and the validator.
When this matters in practice
Most of the time, LLMs emit values that are in the schema, so this isn't a
frequent runtime problem. It does matter when:
strict=Trueto actually reject every malformed shapeof knowing them
Workaround
Attach a
field_validatorto the Pydantic model, or post-validate argumentsmanually before invoking the tool.
Success criteria
convert_function_to_ollama_toolpreservesenumandconstkeywords on simple-type fields. PlainLiteral[...]fields should arrive at backends with the allowed values intact.
validate_tool_arguments(..., strict=True)rejects any payload where afield constrained by
Literal[...],Enum, or schema-levelconst/enumcarries a value outside the allowed set. Both reproducers aboveshould raise
ValidationError.Suggested direction
Two small changes, ideally in one PR:
In the simple-type flattening branch of
convert_function_to_ollama_tool,preserve
enumandconstkeywords on the output property when presenton the input.
In
_build_pydantic_type_from_schema, recognise:{"const": V}→typing.Literal[V]{"enum": [V1, V2, ...]}→typing.Literal[V1, V2, ...]before falling through to the plain-type mapping.
Together this closes both gaps with a few branches of new logic.