Skip to content

👷 use BrowserStack plan endpoint to gate test runs#4576

Closed
thomas-lebeau wants to merge 1 commit intomainfrom
thomas.lebeau/bs-wrapper-use-plan-endpoint
Closed

👷 use BrowserStack plan endpoint to gate test runs#4576
thomas-lebeau wants to merge 1 commit intomainfrom
thomas.lebeau/bs-wrapper-use-plan-endpoint

Conversation

@thomas-lebeau
Copy link
Copy Markdown
Collaborator

Motivation

While investigating why our CI was blocked from launching new BrowserStack runs, we found that a previous CI job (unit-bs on a now-cancelled pipeline) had left 5 sessions stuck in the running state on BrowserStack even though the job had failed over an hour earlier. The driver client never connected (sessions are internally queued), so neither DELETE /automate/sessions/{id} nor DELETE /automate/builds/{id} can clear them — only BS support can.

waitForAvailability() in scripts/test/bs-wrapper.ts queries /automate/builds.json?status=running, which counts these zombie builds and would block local test runs indefinitely until BS support clears them.

The BrowserStack dashboard, however, correctly showed "Parallels 0/5 running | 0 queued" — because zombie builds don't actually consume parallel slots. That state is exposed via /automate/plan.json (parallel_sessions_running + queued_sessions), which is what BS itself gates new sessions on.

Changes

  • Replace BS_BUILD_URL (/automate/builds.json?status=running) with BS_PLAN_URL (/automate/plan.json) in scripts/test/bs-wrapper.ts
  • Rename hasRunningBuildhasParallelSessionsInUse
  • Use parallel_sessions_running + queued_sessions > 0 as the availability gate
  • Add a comment explaining the zombie-build edge case

Test instructions

  1. Set BS_USERNAME / BS_ACCESS_KEY env vars
  2. Run yarn typecheck (passes)
  3. Manual sanity check:
    curl -s -u "$BS_USERNAME:$BS_ACCESS_KEY" https://api.browserstack.com/automate/plan.json
    should return { parallel_sessions_running, queued_sessions, ... }
  4. Run a BS test job (e.g. yarn test:unit:bs) and verify it proceeds when no other parallel sessions are active

Checklist

  • Tested locally
  • Tested on staging
  • Added unit tests for this change.
  • Added e2e/integration tests for this change.
  • Updated documentation and/or relevant AGENTS.md file

The previous availability check listed running builds, but zombie
builds (driver never connected) stay reported as running indefinitely
without consuming parallel slots. Switch to /automate/plan.json which
reflects the real slot usage that BrowserStack gates on.
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Bundles Sizes Evolution

📦 Bundle Name Base Size Local Size 𝚫 𝚫% Status
Rum 179.65 KiB 179.65 KiB 0 B 0.00%
Rum Profiler 6.17 KiB 6.17 KiB 0 B 0.00%
Rum Recorder 27.03 KiB 27.03 KiB 0 B 0.00%
Logs 56.78 KiB 56.78 KiB 0 B 0.00%
Rum Slim 135.50 KiB 135.50 KiB 0 B 0.00%
Worker 23.63 KiB 23.63 KiB 0 B 0.00%
🚀 CPU Performance
Action Name Base CPU Time (ms) Local CPU Time (ms) 𝚫%
RUM - add global context 0.0039 0.0069 +76.92%
RUM - add action 0.0131 0.0199 +51.91%
RUM - add error 0.0117 0.0182 +55.56%
RUM - add timing 0.0025 0.004 +60.00%
RUM - start view 0.0119 0.0175 +47.06%
RUM - start/stop session replay recording 0.0006 0.0011 +83.33%
Logs - log message 0.0138 0.0224 +62.32%
🧠 Memory Performance
Action Name Base Memory Consumption Local Memory Consumption 𝚫
RUM - add global context 31.78 KiB 32.89 KiB +1.11 KiB
RUM - add action 56.94 KiB 55.92 KiB -1.02 KiB
RUM - add timing 32.72 KiB 31.59 KiB -1.13 KiB
RUM - add error 60.80 KiB 61.76 KiB +990 B
RUM - start/stop session replay recording 32.10 KiB 31.47 KiB -651 B
RUM - start view 484.29 KiB 484.03 KiB -271 B
Logs - log message 95.66 KiB 102.95 KiB +7.29 KiB

🔗 RealWorld

@datadog-official
Copy link
Copy Markdown

Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 77.03% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 69e63ad | Docs | Datadog PR Page | Give us feedback!

@github-actions github-actions Bot locked and limited conversation to collaborators May 8, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant