docs: ARM64 LiteLLM hardening fixes for spark-02 bare-metal setup#3
Open
cmdlabtech wants to merge 1 commit into
Open
docs: ARM64 LiteLLM hardening fixes for spark-02 bare-metal setup#3cmdlabtech wants to merge 1 commit into
cmdlabtech wants to merge 1 commit into
Conversation
Four issues discovered during production setup on ARM64 Ubuntu 24.04 that silently break spark-02 (bare-metal systemd) but not spark-01 (Docker Compose). All fixes documented with root-cause explanations. Step 3a (install): - Uninstall litellm-proxy-extras to prevent prisma migrate deploy conflicts on fresh ARM64 databases (TableNotFoundError on startup) - Fetch linux-arm64-openssl-3.0.x prisma query engine binary explicitly; auto-discovery fails on aarch64/GB10, causing dead-reconnect loops Step 3b (systemd service): - Add DATABASE_URL to override.conf drop-in; prisma query engine subprocess reads env directly, not from litellm-config.yaml Step 3d (Postgres + Prisma schema): - Switch postgres container to --network host so localhost:5432 resolves correctly from the systemd service - Replace prisma db push with --force-reset --skip-generate to avoid migration history conflicts from the now-removed proxy-extras package - Add journalctl verify step and Admin UI database-access check - Add post-upgrade reminder to re-run prisma fetch + push Step 5 (Client Open WebUI): - Add callout: OPENAI_API_KEY must exactly match master_key in litellm-config.yaml; drift causes 401 on every chat request Known issues: add four ARM64-specific entries covering all four fixes https://claude.ai/code/session_013uGhhRTYyquQXYrbiWPavA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
$(cat <<'EOF'
Summary
Four production issues discovered on ARM64 Ubuntu 24.04 (spark-02 bare-metal systemd) that don't affect spark-01 (Docker Compose). Each fix is documented with root-cause explanation and verified expected output.
litellm-proxy-extras(preventsprisma migrate deployconflicts on fresh ARM64 DB); explicitly fetchlinux-arm64-openssl-3.0.xprisma query engine binary (auto-discovery fails on aarch64/GB10, causingprisma-query-engine PID N is deadloops)DATABASE_URLtooverride.confdrop-in — prisma query engine subprocess reads env directly, not fromlitellm-config.yaml--network host; replaceprisma db pushwith--force-reset --skip-generateto avoid migration history conflicts; add journalctl verify and Admin UI check; add post-upgrade reminderOPENAI_API_KEYmust exactly matchmaster_key— drift causes401 Unauthorizedon every chat requestTest plan
prisma py fetch/prisma generatecommandsoverride.confcreation withDATABASE_URLbeforedaemon-reloaddocker runuses--network host(not-p 5432:5432),--force-reset --skip-generateon prisma push, and includes journalctl + curl verify blocksdocker runblock with the key-mismatch check commandshttps://claude.ai/code/session_013uGhhRTYyquQXYrbiWPavA
EOF
)
Generated by Claude Code