Evals TUI tree traversal

[miguelg719]

why

what changed

test plan

---

Summary by cubic

Unifies the Evals TUI/CLI behind a shared command tree with path-style traversal and contextual help. Supports Linear STG-1880 and fixes root fallback parsing by stripping a leading evals sigil to improve onboarding and command discovery.

• New Features

  • Single buildCommandTree powers both REPL and argv dispatch.
  • > works as a path separator in REPL and argv; REPL preserves quoted >, argv re-splits any arg containing >.
  • Context navigation and prompt: .. goes up; bare evals jumps to root; invoking a node with children auto-enters that context in REPL; prompt shows the current path; banner mentions ...
  • Help routing: --help after a leaf calls that node’s help; help shows help for the current context.
  • Root fallback: unknown tokens at root call run and now strip a leading evals before forwarding (e.g., evals act); unknown at depth errors for clarity.
  • REPL quality of life: Esc pops context when idle; Esc still aborts in-flight runs (cooperative/double-press aggressive).

• Refactors

  • cli.ts and repl.ts now resolve and execute via dispatch from the shared command tree; added tokenizeArgv for argv > splitting and renderPrompt for context-aware prompts.
  • Moved line tokenization to tui/tokenize.ts; added tui/commandTree.ts with resolution, prompt rendering, and argv tokenization.
  • Kept command handlers lazy-loaded; added focused tests for the tree, > chaining, argv sigil handling, and CLI edge cases.

^{Written for commit 2f5c929. Summary will update on new commits.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 2f5c929

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

2 issues found across 7 files

Confidence score: 3/5

• There is a concrete regression risk in packages/evals/tui/commandTree.ts: for inputs like evals act, the fallback forwards tokens including the "evals" sigil to the run handler, which can misparse arguments and break expected command handling.
• A second medium-severity concern is in packages/evals/tui/commandTree.ts, where user-facing failures are thrown as generic Error values and echoed with raw token interpolation, increasing the chance of confusing or unsanitized REPL error output.
• Given one high-confidence parsing bug (7/10) plus one additional medium issue (6/10), this is not merge-blocking by itself but carries meaningful user-impact risk if shipped without a fix.
• Pay close attention to packages/evals/tui/commandTree.ts - command fallback token forwarding and REPL error sanitization need verification.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/evals/tui/commandTree.ts">

<violation number="1" location="packages/evals/tui/commandTree.ts:242">
P2: Custom agent: **Exception and error message sanitization**

User-facing command errors are thrown as generic `Error` and echoed to the REPL with raw token interpolation, rather than using sanitized typed errors.</violation>

<violation number="2" location="packages/evals/tui/commandTree.ts:275">
P1: When the `evals` sigil is present and the token after it is unknown (e.g., `evals act`), the root fallback passes the full `tokens` array — including the leading `"evals"` — to the run handler. This causes `parseRunArgs` to misinterpret `"evals"` as a target/flag. Strip the sigil before delegating.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant Shell as Shell/argv
    participant CLI as cli.ts
    participant REPL as repl.ts
    participant CmdTree as commandTree.ts
    participant Tok as tokenize.ts
    participant Handler as Command Handlers
    participant Disc as discovery.ts
    participant Run as run command

    Note over Shell,Run: NEW: Unified command tree traversal

    alt argv mode (shell)
        Shell->>CLI: node evals [args...]
        CLI->>CmdTree: buildCommandTree()
        CmdTree-->>CLI: root CommandNode
        CLI->>CLI: tokenizeArgv(args) — split on >
        CLI->>CmdTree: dispatch(root, tokens, ctx)
    else REPL mode (interactive)
        Shell->>REPL: node evals (no args)
        REPL->>REPL: printBanner()
        REPL->>CmdTree: buildCommandTree()
        CmdTree-->>REPL: root CommandNode
        loop Each input line
            REPL->>Tok: tokenize(line) — split on whitespace or >
            Tok-->>REPL: tokens[]
            REPL->>CmdTree: dispatch(root, tokens, ctx)
        end
    end

    Note over CmdTree: Resolution logic (resolveCommand)

    CmdTree->>CmdTree: Check first token for meta commands (..,help,exit,clear)
    alt Meta command
        CmdTree->>REPL: Pop context / print help / exit
    else Leading "evals" sigil
        CmdTree->>CmdTree: Strip sigil, resolve from root
    else Relative path (REPL context)
        CmdTree->>CmdTree: walkPath(contextPath) → current node
        CmdTree->>CmdTree: matchPath(current, tokens) → deepest match
        alt Full match
            CmdTree->>CmdTree: Check for --help/help/-h on first arg
            opt Help requested
                CmdTree->>Handler: node.printHelp(absolutePath)
            end
            CmdTree->>Handler: node.handler(args, ctx)
            alt REPL mode + node has children + no args
                CmdTree->>REPL: setContextPath(absolutePath)
                REPL->>REPL: renderPrompt(contextPath)
            end
        else Partial match at root (unknown token)
            CmdTree->>Run: fallback: run.handler(tokens, ctx)
            Run->>Disc: discoverTasks(getRuntimeTasksRoot())
            Disc-->>Run: registry
            Run->>Run: execute run with registry
        else Partial match at depth (unknown)
            CmdTree-->>REPL: throw Error (unknown subcommand)
        end
    end

    Note over REPL: Context-aware prompt
    REPL->>REPL: renderPrompt(contextPath) → "evals > config > core > "

    Note over REPL: Esc key handling (CHANGED)
    REPL->>REPL: On Esc press
    alt abortRef.current is null (idle)
        alt contextPath.length > 0
            REPL->>REPL: pop context level
            REPL->>REPL: setPrompt(renderPrompt(contextPath))
        end
    else abortRef.current exists (run in flight)
        REPL->>REPL: Cooperative/aggressive abort logic
        REPL->>REPL: abortRef.current.abort(reason)
    end

    Note over CLI: Root fallback for unknown tokens
    CmdTree->>Ctx: unknown at root → call run handler with all tokens
    Ctx->>CmdTree: runNode.handler(tokens, ctx)

    Note over CmdTree: Registry lazy loading
    CmdTree->>REPL: getRegistry()
    REPL->>Disc: discoverTasks(resolvedTasksRoot)
    Disc-->>REPL: TaskRegistry
    REPL->>REPL: cache in ctx.setRegistry()

    Note over Shell,Run: Banner changed to mention .. context navigation

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}

[miguelg719]

@cubic-dev-ai

[cubic-dev-ai]

@cubic-dev-ai

@miguelg719 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

No issues found across 7 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.