feature: Adds queue backlog management and queue position visibility

[JoeGaffney]

No description provided.

[Copilot]

Pull request overview

Adds queue backlog management and queue position visibility across API task polling, plus introduces worker-level global task timeouts and extends text responses to include logs.

Changes:

• Add Redis-backed helpers to report pending-task queue position via logs in GET endpoints (images/texts/videos/workflows).
• Replace per-key rate limiting with a global backlog limit based on Redis queue lengths.
• Add global Celery hard/soft task timeouts and extend TextResponse schemas/OpenAPI to include logs.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
workers/worker.py	Adds global Celery soft/hard time limits for tasks.
workers/texts/schemas.py	Adds logs field to text response schema for workers.
workers/common/schemas.py	Adds QueuePosition model to worker shared schemas.
clients/openapi.json	Regenerates OpenAPI to reflect new logs field in TextResponse.
api/workflows/router.py	Injects queue position logs for PENDING workflow tasks.
api/videos/router.py	Injects queue position logs for PENDING video tasks.
api/texts/schemas.py	Adds logs field to text response schema for API.
api/texts/router.py	Injects queue position logs for PENDING text tasks.
api/images/router.py	Injects queue position logs for PENDING image tasks.
api/common/task_helpers.py	Adds helper to format queue-position information into logs.
api/common/schemas.py	Adds QueuePosition model to API shared schemas.
api/common/redis_manager.py	Introduces Redis-backed queue length + queue position lookup (Lua).
api/common/config.py	Adds task_backlog_limit setting to cap waiting tasks.
api/common/auth.py	Enforces backlog limit on POST task creation (replaces per-key rate limit).
api/admin/router.py	Switches admin key management over to redis_manager.

Comments suppressed due to low confidence (4)

api/common/redis_manager.py:137

• Avoid using a mutable default argument for queues. If this list is ever mutated (even accidentally), it will affect all future calls. Prefer a tuple constant or queues: Sequence[str] | None = None and set a default inside the function.
  api/common/redis_manager.py:140
• Same mutable-default issue for queues here; prefer an immutable default (tuple) or None + assignment inside the function.
  api/common/redis_manager.py:142
• Docstring typo: "task logic" reads like a copy/paste error; it should refer to the task "id" (or "task message") being searched for in Redis.
  api/common/redis_manager.py:5
• Import of 'Any' is not used.