Add dual-segment progress bar for in-progress vs completed rollouts

[mikasenghaas]

Summary

• Adds a dual-segment progress bar that visually distinguishes between completed rollouts (green) and actively executing rollouts (amber), with remaining rollouts shown in dark gray
• Adds an on_acquire callback to with_sem() that fires when a rollout acquires the semaphore and starts executing, enabling accurate tracking of in-flight work
• Wires a new RolloutStartCallback through the generate() → evaluate() → run_evaluation() callback chain so the display knows how many rollouts are currently active
• Shows an "active" count in the progress text (e.g., (5/100 rollouts, 8 active)) when rollouts are in flight

How it works

The bar has three visual segments:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  completed (green)   active (amber)   remaining (gray)

in_progress = started - completed, where started is incremented each time a rollout acquires the concurrency semaphore and completed comes from the existing on_progress callback.

Files changed

File	Change
verifiers/utils/async_utils.py	Add optional on_acquire callback to with_sem()
verifiers/types.py	Add RolloutStartCallback type alias
verifiers/envs/environment.py	Wire on_rollout_start through generate() and evaluate()
verifiers/utils/eval_utils.py	Wire on_rollout_start through run_evaluation() and run_with_progress()
verifiers/utils/eval_display.py	Add DualBarColumn / _DualBar renderable, track started count in EnvEvalState

Test plan

• All existing tests pass (uv run pytest tests/ - 608 tests)
• uv run pre-commit run --all-files passes (ruff check + format)
• Manual verification: run prime eval run with a real environment and observe the dual-segment bar during execution
• Verify the bar correctly shows active rollouts ramping up when the evaluation starts, and draining to zero as it finishes

Generated with Claude Code

---

Note

Medium Risk
Changes the public callback surface (on_progress -> on_task_done plus new on_task_start) and adds semaphore-acquire hooks, which could affect integrations and progress accounting under concurrency/resume scenarios.

Overview
Adds rollout in-flight tracking and a new Rich UI to visualize it: the evaluation display now renders a three-segment progress bar (completed vs running vs remaining) and maintains a started counter to compute in_progress.

Refactors the generation/evaluation callback API by replacing on_progress with on_task_done and introducing on_task_start; Environment.generate() now triggers on_task_start when a task acquires the concurrency semaphore via a new with_sem(..., on_acquire=...) hook, and this signal is threaded through evaluate(), run_evaluation(), and the GEPA adapter.

^{Written by Cursor Bugbot for commit ad1f673. This will update automatically on new commits. Configure here.}

[cursor]

Dual-segment bar overflows width by one character

Medium Severity

The _DualBar rendering can overflow the declared width by one character when both the completed and in-progress segments have an odd number of half-chars that sum to total_halves. Each half-char (╸) occupies a full terminal cell, so two trailing halves (one per segment) consume 2 cells but the halves-based clamping only budgets for 1 cell. For example, with width=40, total=100, completed=99, in_progress=1: c_halves=79, p_halves=1 yields used = 39+1+0+1 = 41 > 40. The character-position total needs to be checked and adjusted after divmod, not just the halves sum.

 

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

[cursor]

Renamed parameter breaks existing test caller

High Severity

Renaming the on_progress parameter to on_task_done in generate() breaks an existing test in tests/test_environment_extra.py (line 327) that still calls generate(on_progress=no_op). This will raise a TypeError at runtime since on_progress is no longer a valid keyword argument. The refactoring is incomplete — all callers need to be updated to use the new name.