Skip to content

[fix] Anthropic CUA triple_click mapping#2104

Open
chromiebot wants to merge 5 commits intobrowserbase:mainfrom
chromiebot:chromie/chore-make-a-pr-to-address-the-gap-i
Open

[fix] Anthropic CUA triple_click mapping#2104
chromiebot wants to merge 5 commits intobrowserbase:mainfrom
chromiebot:chromie/chore-make-a-pr-to-address-the-gap-i

Conversation

@chromiebot
Copy link
Copy Markdown
Contributor

@chromiebot chromiebot commented May 11, 2026

Why

AnthropicCUAClient.convertToolUseToAction() handled actions like click, double_click, and left_click, but did not explicitly support triple_click.

Two issues:

  1. The action type remained in snake_case (triple_click) instead of being normalized to tripleClick — the format expected by v3CuaAgentHandler.executeAction().
  2. Coordinates provided via Anthropic’s coordinate: [x, y] format were not converted into the separate x / y fields required by the handler.

As a result, Anthropic triple-click actions were silently ignored by the handler switch.

What Changed

  • AnthropicCUAClient.ts

    • Added explicit handling for triple_click / tripleClick in convertToolUseToAction()
    • Mirrors the existing double_click implementation
    • Normalizes action type to tripleClick
    • Extracts coordinates from both:
      • coordinate: [x, y]
      • direct x / y fields
  • v3CuaAgentHandler.ts

    • Added "triple_click" as a switch-case alias in executeAction()

Test Plan

Added anthropic-cua-triple-click.test.ts (Vitest) covering:

  • triple_click with coordinate: [x, y]
  • triple_click with direct x / y fields

All existing unit tests continue to pass.


Summary by cubic

Fix incorrect handling of Anthropic CUA triple_click so triple-click actions run and coordinates are parsed correctly. Adds unit tests for both coordinate-array and x/y formats, and a changeset for a patch release.

  • Bug Fixes

    • Normalize triple_click/tripleClick to internal tripleClick in AnthropicCUAClient, extracting x/y from coordinate or explicit fields.
    • Add "triple_click" case to V3CuaAgentHandler so both naming styles execute consistently.
  • Dependencies

    • Add changeset to publish a patch for @browserbasehq/stagehand.

Written for commit 7a54548. Summary will update on new commits.

claude added 2 commits May 11, 2026 07:16
Verifies that the AnthropicCUAClient correctly converts triple_click
actions (with both coordinate-array and x/y formats) into tripleClick
actions consumed by v3CuaAgentHandler.
AnthropicCUAClient.convertToolUseToAction() had no explicit handler for
the triple_click action that Claude can emit via the computer tool. The
action fell through to the generic handler which:
1. Kept the snake_case type "triple_click" instead of normalizing to
   "tripleClick" expected by v3CuaAgentHandler.executeAction().
2. Did not extract coordinates from Anthropic's coordinate array format.

This adds:
- Explicit triple_click/tripleClick mapping in the Anthropic client,
  following the same pattern as the existing double_click handler.
- A "triple_click" case alias in v3CuaAgentHandler.executeAction(),
  matching how "double_click" and "doubleClick" are both handled.
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 11, 2026

🦋 Changeset detected

Latest commit: 7a54548

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@browserbasehq/stagehand Patch
@browserbasehq/stagehand-evals Patch
@browserbasehq/stagehand-server-v3 Patch
@browserbasehq/stagehand-server-v4 Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

This PR is from an external contributor and must be approved by a stagehand team member with write access before CI can run.
Approving the latest commit mirrors it into an internal PR owned by the approver.
If new commits are pushed later, the internal PR stays open but is marked stale until someone approves the latest external commit and refreshes it.

@github-actions github-actions Bot added external-contributor Tracks PRs mirrored from external contributor forks. external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. labels May 11, 2026
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.
Architecture diagram
sequenceDiagram
    participant AnthropicAPI as Anthropic API
    participant CUA as AnthropicCUAClient
    participant Handler as V3CuaAgentHandler

    Note over AnthropicAPI,Handler: Triple Click Action Flow

    AnthropicAPI->>CUA: tool_use (action: "triple_click", coordinate: [x,y])
    anthropicAPI->>CUA: tool_use (action: "tripleClick", x: number, y: number)

    CUA->>CUA: Normalize action to internal "tripleClick"
    alt coordinate array format
        CUA->>CUA: Extract x = coordinate[0], y = coordinate[1]
    else x/y fields format
        CUA->>CUA: Use x, y directly
    end
    CUA->>CUA: Map to { type: "tripleClick", x, y }

    CUA->>Handler: Execute action (type: "tripleClick", x, y)

    alt Recording mode
        Handler->>Handler: Record action for replay
    end

    Handler->>Handler: Perform triple click at (x, y)
    Handler-->>CUA: { success: true }

    CUA-->>AnthropicAPI: Tool result sent back (via next API call)
Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. external-contributor Tracks PRs mirrored from external contributor forks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants