Skip to content

fix: add ollama async timeout#1101

Open
psschwei wants to merge 1 commit into
generative-computing:mainfrom
psschwei:ollama-hang
Open

fix: add ollama async timeout#1101
psschwei wants to merge 1 commit into
generative-computing:mainfrom
psschwei:ollama-hang

Conversation

@psschwei
Copy link
Copy Markdown
Member

Pull Request

Issue

Fixes #650

Description

Adds an optional timeout parameter to OllamaModelBackend that's forwarded to both the sync and async Ollama clients (including the per-event-loop async clients created lazily by _async_client). When the Ollama server stalls under load, requests now fail fast instead of hanging indefinitely.

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

Signed-off-by: Paul S. Schweigert <paul@paulschweigert.com>
@psschwei psschwei requested a review from a team as a code owner May 20, 2026 01:35
@github-actions github-actions Bot added the bug Something isn't working label May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: generate_from_raw hanging when ollama under load

1 participant