ci(repo): dogfood snapi api checks#8621
Conversation
🦋 Changeset detectedLatest commit: 7b226d2 The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughThis PR adds SNAPI-based API change detection: a new snapi.config.json lists four packages to scan and snapshot settings; a GitHub Actions workflow (api-changes.yml) now runs on push and pull_request for main and release branches, adds a publish-baseline job for push that builds declarations and caches baseline snapshots, and reworks the check-api job for PRs to restore or build baseline snapshots, run snapi detect (failing on breaking changes), and upload api-changes-report.md. The UI package gains a declarations-only tsconfig and a build:declarations script. A changeset entry was updated. Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/api-changes.yml:
- Line 67: The workflow generates current snapshots to ".api-snapshots-current"
but snapi.config.json uses snapshotDir ".api-snapshots", so update the pnpm dlx
snapi snapshot invocation or the snapi.detect invocation to use the same
directory: either change the snapshot command (the run line that calls "pnpm dlx
--package \"$SNAPI_PACKAGE\" snapi snapshot --output .api-snapshots-current") to
output to ".api-snapshots" or pass the explicit --current flag/--output value to
the "snapi detect" step so it points at ".api-snapshots-current", ensuring the
snapshot generation and detection commands reference the same directory.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: 9f43cc8f-f375-42fc-a605-7463dfc82b40
📒 Files selected for processing (3)
.changeset/dogfood-snapi.md.github/workflows/api-changes.ymlsnapi.config.json
@clerk/astro
@clerk/backend
@clerk/chrome-extension
@clerk/clerk-js
@clerk/dev-cli
@clerk/expo
@clerk/expo-passkeys
@clerk/express
@clerk/fastify
@clerk/hono
@clerk/localizations
@clerk/nextjs
@clerk/nuxt
@clerk/react
@clerk/react-router
@clerk/shared
@clerk/tanstack-react-start
@clerk/testing
@clerk/ui
@clerk/upgrade
@clerk/vue
commit: |
Publish the API baseline to a GHA cache on push to main and the release branches, keyed by commit sha. PR runs restore from that cache with a prefix-match fallback, and only rebuild the baseline via worktree on a full miss. Add @clerk/ui to the watched package set with a tsc-based build:declarations task matching @clerk/shared's pattern.
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/api-changes.yml:
- Around line 124-127: The fallback fetch uses a moving ref
(origin/${GITHUB_BASE_REF}) which can drift; change the git fetch invocation(s)
that currently use GITHUB_BASE_REF to fetch the immutable PR base SHA
(github.event.pull_request.base.sha) instead so the fallback baseline matches
the PR base commit; update both occurrences of the fetch command in the workflow
to use the PR base SHA when populating refs/remotes/origin/${GITHUB_BASE_REF}.
- Around line 57-61: Update the workflow steps that use actions to pin their
versions to specific commit SHAs and disable persisted credentials: replace both
uses of actions/checkout@v4 (the "Checkout Repo" steps) with the corresponding
actions/checkout@<commit-sha> and add persist-credentials: false under their
with blocks; likewise replace actions/cache/save@v4 and actions/cache/restore@v4
with their pinned commit SHAs (actions/cache/save@<commit-sha>,
actions/cache/restore@<commit-sha>). Ensure you only change the version
specifiers and add the persist-credentials: false key to the checkout steps
while keeping existing keys like filter and show-progress intact.
- Around line 112-120: The baseline cache restore step (id baseline-cache using
actions/cache/restore@v4) must gate rebuilds on an exact cache hit, not partial
matches: change the rebuild condition to check
steps.baseline-cache.outputs.cache-hit != 'true' so any non-exact hit (including
partial matches) triggers the rebuild; when rebuilding, ensure the checkout used
to create the baseline worktree checks out the immutable PR base SHA (use ref:
${{ github.event.pull_request.base.sha }} or GITHUB event value) instead of
GITHUB_BASE_REF; set persist-credentials: false on both actions/checkout steps
to avoid leaking tokens; and pin external actions (actions/cache,
actions/checkout, etc.) to specific commit SHAs instead of floating `@v4` tags.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: c19b2a18-6732-4ac4-a1ee-d17aff1504ca
📒 Files selected for processing (4)
.github/workflows/api-changes.ymlpackages/ui/package.jsonpackages/ui/tsconfig.declarations.jsonsnapi.config.json
✅ Files skipped from review due to trivial changes (3)
- packages/ui/package.json
- packages/ui/tsconfig.declarations.json
- snapi.config.json
| - name: Checkout Repo | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| filter: 'blob:none' | ||
| show-progress: false |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the workflow file
ls -la .github/workflows || true
sed -n '1,200p' .github/workflows/api-changes.yml | nl -ba | sed -n '40,140p'
# Find all uses of actions/checkout in workflows
fd -t f ".github/workflows" -e yml -e yaml | xargs -r rg -n "uses:\s*actions/checkout@"
rg -n "persist-credentials\s*:" .github/workflows || true
# Also check if checkout is pinned anywhere (by SHA)
rg -n "uses:\s*actions/checkout@[^v][0-9a-f]{7,40}|uses:\s*actions/checkout@[0-9a-f]{7,40}" .github/workflows || trueRepository: clerk/javascript
Length of output: 1112
🏁 Script executed:
#!/bin/bash
set -euo pipefail
wf=".github/workflows/api-changes.yml"
echo "=== api-changes.yml (lines 1-140) ==="
awk 'NR>=1 && NR<=140 {printf "%d:%s\n", NR, $0}' "$wf"
echo "=== All checkout usages in .github/workflows ==="
rg -n "uses:\s*actions/checkout@" .github/workflows/**/*.yml .github/workflows/**/*.yaml 2>/dev/null || true
echo "=== All persist-credentials settings in .github/workflows ==="
rg -n "persist-credentials\s*:" .github/workflows/**/*.yml .github/workflows/**/*.yaml 2>/dev/null || true
echo "=== checkout pinned to SHA? (quick check) ==="
rg -n "uses:\s*actions/checkout@([0-9a-f]{7,40})\b" .github/workflows/**/*.yml .github/workflows/**/*.yaml 2>/dev/null || trueRepository: clerk/javascript
Length of output: 4977
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Collect workflow files
mapfile -t wfs < <(fd -t f ".github/workflows" -e yml -e yaml)
echo "Found ${`#wfs`[@]} workflow files"
# Show all checkout usages with some context
printf "%s\n" "${wfs[@]}" | xargs -r rg -n -C 1 "uses:\s*actions/checkout@" || true
# Show any persist-credentials settings
printf "%s\n" "${wfs[@]}" | xargs -r rg -n "persist-credentials\s*:" || true
# Show whether checkout is pinned to a SHA (very rough check)
printf "%s\n" "${wfs[@]}" | xargs -r rg -n "uses:\s*actions/checkout@([0-9a-f]{7,40})\b" || trueRepository: clerk/javascript
Length of output: 117
🏁 Script executed:
#!/bin/bash
set -euo pipefail
tmp="$(mktemp)"
fd -t f ".github/workflows" -e yml -e yaml > "$tmp" || true
mapfile -t wfs < "$tmp"
echo "Found ${`#wfs`[@]} workflow files"
# Checkout usages
if [ ${`#wfs`[@]} -gt 0 ]; then
printf "%s\n" "${wfs[@]}" | xargs -r rg -n -C 1 "uses:\s*actions/checkout@" || true
# persist-credentials settings
printf "%s\n" "${wfs[@]}" | xargs -r rg -n "persist-credentials\s*:" || true
# checkout pinned to SHA (quick heuristic)
printf "%s\n" "${wfs[@]}" | xargs -r rg -n "uses:\s*actions/checkout@([0-9a-f]{7,40})\b" || true
fi
rm -f "$tmp"Repository: clerk/javascript
Length of output: 475
🏁 Script executed:
#!/bin/bash
set -euo pipefail
dir=".github/workflows"
echo "Listing $dir"
ls -ლა "$dir"
echo
echo "=== actions/checkout usages ==="
rg -n --glob '*.{yml,yaml}' "uses:\s*actions/checkout@" "$dir" || true
echo
echo "=== persist-credentials settings ==="
rg -n --glob '*.{yml,yaml}' "persist-credentials\s*:" "$dir" || true
echo
echo "=== checkout pinned to SHA (heuristic) ==="
rg -n --glob '*.{yml,yaml}' "uses:\s*actions/checkout@([0-9a-f]{7,40})\b" "$dir" || true
echo
echo "=== actions/cache usages (for the 80/114 lines context) ==="
rg -n --glob '*.{yml,yaml}' "uses:\s*actions/cache/(save|restore)@" "$dir" || trueRepository: clerk/javascript
Length of output: 168
🏁 Script executed:
#!/bin/bash
set -euo pipefail
dir=".github/workflows"
echo "Listing $dir"
ls -la "$dir"
echo
echo "=== actions/checkout usages ==="
rg -n --glob '*.{yml,yaml}' "uses:\s*actions/checkout@" "$dir" || true
echo
echo "=== persist-credentials settings ==="
rg -n --glob '*.{yml,yaml}' "persist-credentials\s*:" "$dir" || true
echo
echo "=== checkout pinned to SHA (heuristic) ==="
rg -n --glob '*.{yml,yaml}' "uses:\s*actions/checkout@([0-9a-f]{7,40})\b" "$dir" || true
echo
echo "=== actions/cache save/restore usages ==="
rg -n --glob '*.{yml,yaml}' "uses:\s*actions/cache/(save|restore)@" "$dir" || trueRepository: clerk/javascript
Length of output: 3054
🏁 Script executed:
#!/bin/bash
set -euo pipefail
wf=".github/workflows/api-changes.yml"
echo "=== api-changes.yml lines 50-130 ==="
awk 'NR>=50 && NR<=130 {printf "%d:%s\n", NR, $0}' "$wf"
echo
echo "=== api-changes.yml lines 130-170 ==="
awk 'NR>=130 && NR<=170 {printf "%d:%s\n", NR, $0}' "$wf"
echo
echo "=== persist-credentials in all workflows ==="
rg -n "persist-credentials\s*:" .github/workflows -S --glob '*.{yml,yaml}' || true
echo
echo "=== checkout usages with persist-credentials close-by (±6 lines) ==="
for f in .github/workflows/*.{yml,yaml}; do
rg -n -C 6 "uses:\s*actions/checkout@v4" "$f" || true
done | head -n 120Repository: clerk/javascript
Length of output: 8024
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check for checkout pinned to a full SHA
rg -n --glob '.github/workflows/*.{yml,yaml}' "uses:\s*actions/checkout@([0-9a-f]{40})\b" .github/workflows || true
echo "----"
# Show persist-credentials: false context
rg -n --glob '.github/workflows/*.{yml,yaml}' -C 6 "persist-credentials\s*:\s*false" .github/workflows || trueRepository: clerk/javascript
Length of output: 821
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n --glob '.github/workflows/*.{yml,yaml}' "uses:\s*actions/cache/(save|restore)@[0-9a-f]{40}\b" .github/workflows || trueRepository: clerk/javascript
Length of output: 42
Pin checkout/cache actions to commit SHAs and disable persisted credentials in api-changes.yml.
- name: Checkout Repo
uses: actions/checkout@v4
with:
filter: 'blob:none'
show-progress: falseactions/checkout@v4in.github/workflows/api-changes.ymlis unpinned and lackspersist-credentials: false(also present on the second checkout step in the same workflow).actions/cache/save@v4(line 80) andactions/cache/restore@v4(line 114) are also unpinned; pin them to commit SHAs.
🧰 Tools
🪛 zizmor (1.25.2)
[warning] 57-61: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 58-58: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/api-changes.yml around lines 57 - 61, Update the workflow
steps that use actions to pin their versions to specific commit SHAs and disable
persisted credentials: replace both uses of actions/checkout@v4 (the "Checkout
Repo" steps) with the corresponding actions/checkout@<commit-sha> and add
persist-credentials: false under their with blocks; likewise replace
actions/cache/save@v4 and actions/cache/restore@v4 with their pinned commit SHAs
(actions/cache/save@<commit-sha>, actions/cache/restore@<commit-sha>). Ensure
you only change the version specifiers and add the persist-credentials: false
key to the checkout steps while keeping existing keys like filter and
show-progress intact.
| - name: Restore baseline from cache | ||
| id: baseline-cache | ||
| uses: actions/cache/restore@v4 | ||
| with: | ||
| path: .api-snapshots-baseline | ||
| key: snapi-baseline-${{ github.event.pull_request.base.sha }} | ||
| restore-keys: | | ||
| snapi-baseline- | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
For actions/cache/restore@v4, what are the exact semantics of cache-hitandcache-matched-key for: (1) exact key hit, (2) restore-keys partial hit, and (3) full miss?
💡 Result:
For actions/cache/restore@v4, the output variables cache-hit and cache-matched-key function as follows: (1) Exact Key Hit - cache-hit: Set to 'true' [1][2][3]. - cache-matched-key: Set to the exact key that was used [4]. (2) Restore-Keys Partial Hit - cache-hit: Set to 'false' [1][2][3]. - cache-matched-key: Set to the specific key that was found and restored (i.e., the partial match) [4]. (3) Full Miss - cache-hit: Set to an empty string [1][2][3]. - cache-matched-key: Set to an empty string (or not present/undefined, as no cache was found) [5]. Important Semantics - The cache-hit output is a string value, not a boolean, despite its name; it is typically 'true', 'false', or an empty string [1][2][3]. - Relying on cache-hit to determine if any cache was restored can be misleading in scenarios involving partial hits. Because cache-hit is 'false' during a partial hit, developers often use cache-matched-key to verify if any cache was restored [4]. - When checking cache-hit in conditional expressions (if:), ensure you compare against the string value (e.g., if: steps.cache.outputs.cache-hit!= 'true') rather than a boolean [1][4].
Citations:
- 1: https://github.com/actions/cache/tree/v4
- 2: https://github.com/actions/cache
- 3: https://github.com/actions/cache/blob/main/README.md
- 4: https://stackoverflow.com/questions/72752961/github-actions-why-cache-restored-successfully-but-cache-hit-got-false-pr
- 5: https://www.npmjs.com/package/@actions/cache
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/api-changes.yml"
echo "== File: $FILE =="
stat "$FILE" || true
python3 - <<'PY'
import itertools,sys
path=".github/workflows/api-changes.yml"
start=80
end=180
with open(path,'r',encoding='utf-8') as f:
for i,line in enumerate(f,1):
if start<=i<=end:
print(f"{i:4d}: {line.rstrip()}")
PYRepository: clerk/javascript
Length of output: 3726
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/api-changes.yml"
echo "== File: $FILE =="
python3 - <<'PY'
path=".github/workflows/api-changes.yml"
ranges=[(1,90),(90,170)]
with open(path,'r',encoding='utf-8') as f:
lines=f.readlines()
for a,b in ranges:
for i in range(a,b+1):
if i<=0 or i>len(lines):
continue
print(f"{i:4d}: {lines[i-1].rstrip()}")
PYRepository: clerk/javascript
Length of output: 6252
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/api-changes.yml"
echo "== Grep: persist-credentials =="
rg -n "persist-credentials" "$FILE" || true
echo
echo "== Grep: actions/checkout =="
rg -n "uses:\s*actions/checkout@" "$FILE" || true
echo
echo "== Grep: actions/cache/restore/save =="
rg -n "uses:\s*actions/cache/(restore|save)@" "$FILE" || true
echo
echo "== Show checkout blocks (approx) =="
python3 - <<'PY'
import re
path=".github/workflows/api-changes.yml"
lines=open(path,'r',encoding='utf-8').read().splitlines()
for i,l in enumerate(lines,1):
if "uses: actions/checkout@" in l:
start=max(1,i-3); end=min(len(lines), i+8)
print(f"\n-- lines {start}-{end} around checkout at {i} --")
for j in range(start,end+1):
print(f"{j:4d}: {lines[j-1]}")
PYRepository: clerk/javascript
Length of output: 1117
Fix baseline cache gating + ensure rebuild uses the PR base SHA in api-changes workflow
actions/cache/restore@v4: exact key hit setscache-hit: 'true'; partial matches viarestore-keysleavecache-hitas'false'while still populatingcache-matched-key. Currentif: cache-matched-key == ''only rebuilds on full miss, so partial restores can skip rebuild and compare against the wrong baseline.- On cache miss/rebuild, baseline worktree is created from
${GITHUB_BASE_REF}(branch tip) instead of the immutable${{ github.event.pull_request.base.sha }}used for the cache key. - Hardening: both
actions/checkout@v4instances don’t setpersist-credentials: false, andactions/*@v4`` uses aren’t pinned to commit SHAs.
Proposed fix
- name: Fetch base branch
- if: steps.baseline-cache.outputs.cache-matched-key == ''
+ if: steps.baseline-cache.outputs.cache-hit != 'true'
run: git fetch origin "${GITHUB_BASE_REF}:refs/remotes/origin/${GITHUB_BASE_REF}" --depth=100
- name: Create baseline worktree
- if: steps.baseline-cache.outputs.cache-matched-key == ''
+ if: steps.baseline-cache.outputs.cache-hit != 'true'
run: |
mkdir -p .worktrees
git worktree add .worktrees/snapi-baseline "origin/${GITHUB_BASE_REF}"
cp snapi.config.json .worktrees/snapi-baseline/snapi.config.json
- name: Install baseline dependencies
- if: steps.baseline-cache.outputs.cache-matched-key == ''
+ if: steps.baseline-cache.outputs.cache-hit != 'true'
run: pnpm --dir .worktrees/snapi-baseline install --frozen-lockfile
- name: Build baseline declarations
- if: steps.baseline-cache.outputs.cache-matched-key == ''
+ if: steps.baseline-cache.outputs.cache-hit != 'true'
run: pnpm --dir .worktrees/snapi-baseline turbo build:declarations $TURBO_ARGS $SNAPI_FILTERS
- name: Generate baseline API snapshots
- if: steps.baseline-cache.outputs.cache-matched-key == ''
+ if: steps.baseline-cache.outputs.cache-hit != 'true'
run: |
+ rm -rf "$GITHUB_WORKSPACE/.api-snapshots-baseline"
pnpm --dir .worktrees/snapi-baseline dlx --package "$SNAPI_PACKAGE" snapi snapshot \
--output "$GITHUB_WORKSPACE/.api-snapshots-baseline"🧰 Tools
🪛 zizmor (1.25.2)
[error] 114-114: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/api-changes.yml around lines 112 - 120, The baseline cache
restore step (id baseline-cache using actions/cache/restore@v4) must gate
rebuilds on an exact cache hit, not partial matches: change the rebuild
condition to check steps.baseline-cache.outputs.cache-hit != 'true' so any
non-exact hit (including partial matches) triggers the rebuild; when rebuilding,
ensure the checkout used to create the baseline worktree checks out the
immutable PR base SHA (use ref: ${{ github.event.pull_request.base.sha }} or
GITHUB event value) instead of GITHUB_BASE_REF; set persist-credentials: false
on both actions/checkout steps to avoid leaking tokens; and pin external actions
(actions/cache, actions/checkout, etc.) to specific commit SHAs instead of
floating `@v4` tags.
| - name: Fetch base branch | ||
| if: steps.baseline-cache.outputs.cache-matched-key == '' | ||
| run: git fetch origin "${GITHUB_BASE_REF}:refs/remotes/origin/${GITHUB_BASE_REF}" --depth=100 | ||
|
|
There was a problem hiding this comment.
Build fallback baseline from immutable PR base SHA.
Line 126 and Line 132 use origin/${GITHUB_BASE_REF} (moving branch head). If the base branch advances, the fallback baseline can drift from github.event.pull_request.base.sha, producing incorrect API diffs.
Proposed fix
- name: Fetch base branch
if: steps.baseline-cache.outputs.cache-hit != 'true'
- run: git fetch origin "${GITHUB_BASE_REF}:refs/remotes/origin/${GITHUB_BASE_REF}" --depth=100
+ run: git fetch origin "${{ github.event.pull_request.base.sha }}" --depth=1
- name: Create baseline worktree
if: steps.baseline-cache.outputs.cache-hit != 'true'
run: |
mkdir -p .worktrees
- git worktree add .worktrees/snapi-baseline "origin/${GITHUB_BASE_REF}"
+ git worktree add --detach .worktrees/snapi-baseline "${{ github.event.pull_request.base.sha }}"
cp snapi.config.json .worktrees/snapi-baseline/snapi.config.jsonAlso applies to: 132-133
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/api-changes.yml around lines 124 - 127, The fallback fetch
uses a moving ref (origin/${GITHUB_BASE_REF}) which can drift; change the git
fetch invocation(s) that currently use GITHUB_BASE_REF to fetch the immutable PR
base SHA (github.event.pull_request.base.sha) instead so the fallback baseline
matches the PR base commit; update both occurrences of the fetch command in the
workflow to use the PR base SHA when populating
refs/remotes/origin/${GITHUB_BASE_REF}.
Not ready for review. Dogfooding the snapi API-changes workflow against this repo to see what the report looks like on a real PR before we open it up for review.
Pins snapi to clerk/snapi@8c021a2 and wires a non-blocking workflow for clerk-js, react, and shared. Empty changeset since this is repo tooling only.