Skip to content

feat: support incremental diffs for GitHub merge queue#768

Draft
danadajian wants to merge 5 commits into
mainfrom
merge-queue-support
Draft

feat: support incremental diffs for GitHub merge queue#768
danadajian wants to merge 5 commits into
mainfrom
merge-queue-support

Conversation

@danadajian
Copy link
Copy Markdown
Contributor

@danadajian danadajian commented Apr 29, 2026

Summary

GitHub's merge queue combines each queued PR with all preceding positions into a single commit, which causes visual regression checks to surface false positives — position N appears to have diffs from N-1's changes even when it didn't touch those components.

This PR introduces a merge-queue-diff workflow that eliminates that noise. It splits the merge queue check into two jobs: Job 1 runs visual tests and uploads screenshots as usual (skipping commit status and PR comments), and Job 2 waits for the preceding queue position to finish, then computes pixel-isolated diffs by comparing N's renders against N-1's renders rather than the shared base images. Job 2 sets the final commit status, so only N's own visual changes are reported. When N-1 had no diffs or failed, Job 2 falls back to base-image comparison directly.

The pr workflow is also hardened for merge queue runs: if Job 1 crashes, it now sets a failure commit status so Job 2 doesn't poll until timeout with no signal. Pixelmatch utilities are refactored from comparadise-utils into shared/ to support the new workflow.

How it works

Consumers add two jobs to their merge queue workflow:

on:
  merge_group:

jobs:
  visual-tests:        # Job 1: run tests, upload images, skip status/comment
    steps:
      - uses: ExpediaGroup/comparadise@main
        with:
          workflow: pr
          commit-hash: ${{ github.event.merge_group.head_sha }}

  visual-diff:         # Job 2: wait for N-1, compute isolated diffs, set status
    needs: visual-tests
    steps:
      - uses: ExpediaGroup/comparadise@main
        with:
          workflow: merge-queue-diff
          commit-hash: ${{ github.event.merge_group.head_sha }}

Job 2 polls N-1's commit status, then runs pixelmatch against N-1's uploaded renders. If N-1 had no diffs (success) or failed, it falls back to comparing against base-images/ directly.

Test plan

  • Queue two PRs with overlapping visual changes — N should only show its own diffs, not N-1's
  • Queue a PR with no visual changes behind one that has changes — N should show success
  • Crash Job 1 during a merge queue run — Job 2 should see failure status and not poll until timeout
  • Queue a PR at position 1 (no N-1) — should fall back to base-image comparison directly
  • Accept diffs via the Comparadise UI using the commit status target URL — status should flip to success

Generated with Claude Code

@danadajian danadajian marked this pull request as draft April 29, 2026 22:26
@danadajian danadajian changed the title feat: add merge-queue-diff workflow for GitHub merge queue support feat: support incremental diffs for GitHub merge queue Apr 29, 2026
danadajian and others added 2 commits April 29, 2026 17:35
…ils build

Add --dts.eager flag to tsdown build so it can resolve type exports from
the shared workspace package without needing pre-built .d.ts files.
Also use `import type` for the PixelMatchOptions import.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant