Skip to content

perf: force condense ended orphaned ended sessions#599

Open
peyton-alt wants to merge 17 commits intomainfrom
perf/force-condense-ended-sessions
Open

perf: force condense ended orphaned ended sessions#599
peyton-alt wants to merge 17 commits intomainfrom
perf/force-condense-ended-sessions

Conversation

@peyton-alt
Copy link
Contributor

@peyton-alt peyton-alt commented Mar 4, 2026

Fixes O(N) commit-time overhead caused by stale ENDED sessions accumulating and being re-processed on every
PostCommit (~73-103ms per session, indefinitely). Tracked as issue #591.

Two complementary fixes:

  • Eager condense on session stop (CondenseAndMarkFullyCondensed): When a session ends via the stop hook and
    has no FilesTouched, its checkpoint data is immediately condensed to entire/checkpoints/v1 and the session
    is marked FullyCondensed. PostCommit skips these sessions entirely on all future commits.
  • Force-condense safety net in PostCommit: ENDED sessions that are older than 1 hour and still have
    FilesTouched with no commit overlap are now force-condensed instead of being carried forward forever. This
    catches sessions the eager path couldn't handle (older CLI versions, crashes, or sessions that had
    FilesTouched at stop time — the primary subagent accumulation shape from Orphaned sessions cause 3GB+ RAM usage and 2m44s commit times #591).

Key design choices:

  • CondenseAndMarkFullyCondensed intentionally keeps Phase = ENDED (unlike CondenseSessionByID which resets
    to IDLE) and skips sessions with FilesTouched so PostCommit's carry-forward tracking still works for those
  • The 1-hour forceCondenseThreshold gives users a window to commit a session's files in a separate commit
    before the safety net fires
  • Force-condensed sessions skip carry-forward computation entirely — FilesTouched is cleared to nil

Test plan

  • mise run test — unit tests including new
    TestPostCommit_EndedSessionCarryForward_ForceCondensedWithoutOverlap,
    TestPostCommit_RecentEndedSession_NotForceCondensed, TestPostCommit_EmptyEndedSession_MarkedFullyCondensed,
    TestCondenseAndMarkFullyCondensed_Guards, TestCondenseAndMarkFullyCondensed_WithDataNoFiles
  • mise run test:integration — TestCarryForward_NewSessionCommitDoesNotCondenseOldSession,
    TestSubagentAccumulation_Issue591
  • mise run test:ci — full suite including E2E canary
  • Verify accumulation_bench_test.go regression: second PostCommit is >2x faster than first once sessions
    are FullyCondensed

Copilot AI review requested due to automatic review settings March 4, 2026 05:23
@cursor
Copy link

cursor bot commented Mar 4, 2026

PR Summary

Cursor Bugbot is generating a summary for commit 091a9a4. Configure here.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a PostCommit performance issue where ENDED sessions with carry-forward FilesTouched but no overlap with later commits could be re-processed indefinitely, causing cumulative per-commit overhead (issue #591). It introduces a “force-condense” path to ensure those sessions get condensed once and then skipped on subsequent commits.

Changes:

  • Add a force-condensation path for ENDED sessions with FilesTouched and new content even when there’s no commit overlap.
  • Update PostCommit phase tests to assert the new force-condense behavior (cleared FilesTouched, reset counters, FullyCondensed set).
  • Add benchmarks to measure PostCommit overhead with accumulated ENDED sessions and verify the “second commit after fix” performance.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
cmd/entire/cli/strategy/manual_commit_hooks.go Adds forceCondensed flow to condense ENDED sessions without overlap and skip carry-forward.
cmd/entire/cli/strategy/phase_postcommit_test.go Updates test expectations to confirm ENDED sessions are force-condensed and marked FullyCondensed.
cmd/entire/cli/strategy/accumulation_bench_test.go Adds benchmarks and repo/session setup helpers to quantify accumulation overhead and validate the fix.

@peyton-alt peyton-alt force-pushed the perf/force-condense-ended-sessions branch 2 times, most recently from 3fc0364 to 7da585a Compare March 4, 2026 05:40
@peyton-alt peyton-alt force-pushed the perf/force-condense-ended-sessions branch from 7da585a to 4324e69 Compare March 17, 2026 04:37
@peyton-alt peyton-alt changed the title perf: force condence ended orphaned ended sessions perf: force condense ended orphaned ended sessions Mar 18, 2026
@peyton-alt peyton-alt force-pushed the perf/force-condense-ended-sessions branch from 7ef6fae to ef991a8 Compare March 18, 2026 01:17
@peyton-alt peyton-alt marked this pull request as ready for review March 18, 2026 01:18
@peyton-alt peyton-alt requested a review from a team as a code owner March 18, 2026 01:18
peyton-alt and others added 2 commits March 17, 2026 18:58
Entire-Checkpoint: 4af77a52ab04
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@peyton-alt peyton-alt enabled auto-merge (squash) March 18, 2026 17:59
Soph and others added 5 commits March 18, 2026 21:24
…ap commits

ENDED sessions do not need a carry-forward shadow branch — the session is
done and no future turn will resume it. Creating one anyway caused the branch
to linger indefinitely when no subsequent commit overlapped with the session's
files, breaking AssertNoShadowBranches in E2E tests.

IDLE sessions still get a carry-forward shadow branch so that if the agent
resumes, sessionHasNewContent returns true for future commits. Only
BaseCommit is advanced for ENDED sessions (sufficient for Case b discovery
in findSessionsFromShadowTree).

Fixes TestMidTurnCommit_DifferentFilesThanPreviousTurn and
TestPartialCommitStashNewPrompt vogon canary failures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Entire-Checkpoint: dbd222ef1cea
96d1bce stopped creating shadow branches for ENDED carry-forward sessions,
but three code paths needed updating to handle the no-shadow-branch case:

1. listAllSessionStates (session.go): add len(FilesTouched)==0 guard to
   orphan cleanup, preventing deletion of ENDED carry-forward state files
   that have no shadow branch by design.

2. sessionHasNewContent (hooks.go): for ENDED carry-forward with no shadow
   branch, return true when staged files overlap with FilesTouched (PrepareCommitMsg
   context), and unconditionally in PostCommit context (no staged files). This
   allows PrepareCommitMsg to add a trailer when the user commits carry-forward
   files, enabling the full PostCommit condensation path.

3. HandleCondenseIfFilesTouched (hooks.go): for ENDED carry-forward with no
   shadow branch when shouldCondense=true, consume the carry-forward state
   (clear FilesTouched, reset StepCount) without calling CondenseSession.
   The session transcript was already condensed on the previous commit;
   no new checkpoint entry is needed.

Fixes TestCarryForward_NewSessionCommitDoesNotCondenseOldSession.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants