fix(ci): use locked mode to eliminate GitHub API calls from mise#1118
Open
palootcenas-outreach wants to merge 9 commits intomainfrom
Open
fix(ci): use locked mode to eliminate GitHub API calls from mise#1118palootcenas-outreach wants to merge 9 commits intomainfrom
palootcenas-outreach wants to merge 9 commits intomainfrom
Conversation
Enable strict lockfile mode for mise in CI so all tools resolve from pre-resolved URLs in lockfiles instead of hitting api.github.com for version resolution. This is safe because: - mise already ignores .tool-versions in CI (MISE_OVERRIDE_TOOL_VERSIONS_FILENAMES=none) - All devbase tools have entries in mise.devbase.lock - The setting only applies to mise install calls, not to mise settings/activate Repos can opt out by setting MISE_LOCKED=0 in their CircleCI project env. Relates-to: DT-5157
Replace MISE_LOCKED=1 global env var which validated ALL config sources (global, conf.d, .tool-versions) against a single project lockfile. Instead: - Set MISE_CONFIG_DIR to a temp dir during install to hide global config - Pass --locked per-invocation: guarded by mise.lock existence for downstream repos, always-on for devbase (mise.devbase.lock) - Restore MISE_CONFIG_DIR after install for devbase_configure_global_tools
palootcenas-outreach
changed the title
The previous approach exported MISE_CONFIG_DIR globally, which broke wait-for-gh-rate-limit (a mise shim) because the shim couldn't resolve its tool version against the empty config dir. Instead, pass _MISE_INSTALL_CONFIG_DIR through run_mise() and apply MISE_CONFIG_DIR as an inline env var only on the mise binary invocation. Shims and other helper tools still see the real global config.
go: backend tools (goimports, dlv, goveralls) compile from source via `go install` and cannot produce lockfile URLs. --locked always fails for them. Split devbase tool install into two passes: 1) MISE_DISABLE_BACKENDS=go with --locked for all lockable tools 2) Normal install for go: tools (they use Go module proxy, not GitHub API)
MISE_DISABLE_BACKENDS=go doesn't work for explicitly declared tools (mise bug: get() in backend/mod.rs bypasses the disable_backends filter by re-creating backends on demand). Use MISE_DISABLE_TOOLS with dynamically extracted go: tool names from mise.devbase.toml.
Contributor
|
Link to code coverage report (posted by coverbot 🤖) |
Pass 2 runs without --locked for go: backend tools (they compile from source and can't use lockfiles). Without MISE_LOCKFILE=false, mise could still write to mise.lock during this pass, causing dirty-tree failures in the pre-release dry run step. Pass 1 (--locked) is already read-only by definition. This ensures neither pass can mutate lockfiles in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Eliminate GitHub API calls from mise tool resolution in CI by using
--lockedmode withMISE_CONFIG_DIRisolation. This is the single highest-impact fix for CI GitHub rate limit exhaustion.Related fix
#1117
Problem
mise's
github:backend tools (orc, ci, lintroller) callapi.github.com/repos/.../releaseson everymise installto resolve versions. With orc declared as"latest", this happens on every CI build. These calls consume the shared 15,000 req/hr GitHub App token pool across all concurrent CI builds org-wide.devbase already has lockfiles (
mise.lock,mise.devbase.lock) with pre-resolved download URLs and checksums. But without--locked, mise treats these as advisory — it still hits the API.ℹ️ Failed attempt: MISE_LOCKED=1 as a global env var 👎
Setting
MISE_LOCKED=1as a global env var failed because locked mode validates all tools from all config sources against a single project lockfile:.tool-versionstools (go, node, protoc) — not in any mise lockfile~/.config/mise/config.toml) — no lockfile exists for these~/.config/mise/conf.d/devbase.toml) — in CI image from prior runsThis is confirmed as intentional behavior by the mise maintainer.
Solution
Three-part fix:
1.
MISE_CONFIG_DIRisolationSet
MISE_CONFIG_DIRto a temp directory duringmise installcalls. This hides all global config (config.toml,conf.d/,~/.tool-versions) from discovery. Project-level configs (mise.toml,mise.devbase.toml) are unaffected since they're found by directory walking, not via the config dir.Restored after install so
devbase_configure_global_toolswrites to the real~/.config/mise/for shim setup.2. Per-invocation
--lockedwith existence guard--lockedonly ifmise.lockexists. ~60%+ of repos already have lockfiles; repos without them get plainmise install(same as today).--locked—mise.devbase.lockis maintained by thestencil:post:misehook.3. Two-pass install for
go:backend toolsgo:backend tools (goimports, dlv, goveralls) compile from source viago installand cannot produce lockfile URLs — their lockfile entries contain only version and backend, with no platform-specific download URLs. This is a known mise limitation.--lockedalways fails for these tools. Fix: two-pass install indevbase_install_mise_tools():MISE_DISABLE_BACKENDS=go mise install --locked— installs all lockable tools (zero GitHub API calls)MISE_DISABLE_TOOLStill mise#8789 gets resolvedmise install— the lockable tools are already installed so mise skips them; only installs the 3go:tools via Go module proxy (proxy.golang.org), not GitHub APIWhat this fixes
"latest": Resolves from lockfile (pinned with direct URLs) — zero API callsgithub:/aqua:backend tools: Noapi.github.comcalls for version resolutiongo:backend tools: Handled separately — they never used GitHub API anyway--lockedprevents lockfile mutation in CIFailure modes
mise.tomlbut nomise.lock: Falls back to normalmise installwithout--locked— no change from todaymise install --lockedfails with clear error ("tool X not found in lockfile"). Fix: runmise locklocally and commitRelated
locked=true(Backlog)locked=true)locked=true(merged then reverted)--lockeddoesn't work with all backendsMISE_DISABLE_BACKENDS