-
Notifications
You must be signed in to change notification settings - Fork 15
Description
Summary
Add a new --queues CLI option to the worker that lets operators divide
worker capacity into slot groups, each with its own queue preference chain.
This issue covers configuration parsing and validation only — the runtime
behavior (per-group workloops) is a separate issue.
Background
Currently the worker has a single --capacity flag (default 5) that controls
the total number of concurrent runs. All runs are claimed from a single global
queue with no differentiation.
The fast lanes feature requires workers to dedicate specific slots to specific
queues. For example: 1 slot pinned to sync webhook work, 4 slots for general
work.
Configuration syntax
--queues "<preference>:<count> <preference>:<count> ..."
| Delimiter | Meaning |
|---|---|
, |
Queue preference separator (left = highest preference) |
* |
Wildcard: accept runs from any queue |
:N |
Slot count for this preference group |
(space) |
Slot group separator |
Examples:
# 1 slot pinned to fast_lane, 4 slots preferring manual then anything
--queues "fast_lane:1 manual,*:4"
# 5 generic slots (pure FIFO across all queues)
--queues "*:5"
# 2 fast lane (strict), 3 with manual preference
--queues "fast_lane:2 manual,*:3"Environment variable: WORKER_QUEUES
Slot group data structure
Each parsed group should produce something like:
interface SlotGroup {
queues: string[]; // e.g., ["fast_lane"] or ["manual", "*"]
maxSlots: number; // e.g., 1
}Backward compatibility
--queuesand--capacityare mutually exclusive — specifying both is
an error- If neither is provided, default to
--capacity 5behavior - When
--queuesis not provided, derive from--capacity:
manual,*:<capacity>(e.g., capacity 5 →"manual,*:5") - This preserves current behavior where manual/immediate runs get preference
Validation rules
- Each slot group must have count >= 1
- Queue names must be non-empty strings (alphanumeric + underscore)
*can only appear as the last element in a preference chain- Total slot count across all groups becomes the effective capacity
--queuesand--capacityare mutually exclusive
Protocol type update
Update ClaimPayload in the lexicon types to include the optional queues
field:
interface ClaimPayload {
demand?: number;
worker_name?: string;
queues?: string[]; // NEW: queue preference chain
}Testing
- Parse
"fast_lane:1 manual,*:4"→ two slot groups with correct queues and counts - Parse
"*:5"→ one slot group with["*"]and 5 slots - Parse
"fast_lane,manual,*:1 *:4"→ correct multi-preference chain - Reject
"*,fast_lane:1"(wildcard not at end) - Reject
":0"or":abc"(invalid counts) - Reject conflicting
--queuesand--capacityflags - Default derivation: capacity 5 with no
--queues→[{ queues: ["manual", "*"], maxSlots: 5 }]
Metadata
Metadata
Assignees
Labels
Type
Projects
Status