Skip to content

Claude/add llm api support wc mv e#738

Open
pinglanyi wants to merge 10 commits intogoogle:mainfrom
pinglanyi:claude/add-llm-api-support-wcMvE
Open

Claude/add llm api support wc mv e#738
pinglanyi wants to merge 10 commits intogoogle:mainfrom
pinglanyi:claude/add-llm-api-support-wcMvE

Conversation

@pinglanyi
Copy link

Description

Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.

List which issues are fixed by this PR. For larger changes, raising an issue first helps reduce redundant work.

Pre-launch Checklist

If you need help, consider asking for advice on the discussion board.

claude and others added 5 commits February 26, 2026 06:54
- tools/editor: Replace @google/genai with openai SDK in middleware/gemini.ts.
  The middleware now reads OPENAI_API_KEY, OPENAI_BASE_URL and AI_MODEL from
  .env, supports any OpenAI-compatible endpoint (including vLLM), and retains
  backward compatibility with GEMINI_API_KEY by automatically routing to
  Google's OpenAI-compatible endpoint.

- tools/editor/package.json: Add openai ^4.0.0 dependency.

- tools/editor/.env.example: New file documenting all supported env vars.

- samples/agent/adk (all agents): Update API key validation to accept
  OPENAI_API_KEY in addition to GEMINI_API_KEY. Add AI_MODEL env var
  support (takes precedence over legacy LITELLM_MODEL). Update all
  .env.example files with OpenAI/vLLM configuration instructions.

https://claude.ai/code/session_01UA8Dx6GkhaeJNAnJhwS1ty
…7U3V

Add OpenAI-compatible API support (OpenAI, vLLM, Ollama, etc.)
Two root-cause bugs prevented OPENAI_API_KEY from working:

1. Default model was always "gemini/gemini-2.5-flash", so LiteLLM would
   call the Gemini API even when only OPENAI_API_KEY was set.
   Fix: auto-detect the default model — use "openai/gpt-4o" when
   OPENAI_API_KEY is set and GEMINI_API_KEY is not.

2. LiteLLM reads OPENAI_API_BASE (not OPENAI_BASE_URL) for custom
   endpoints (vLLM, Ollama, etc.).
   Fix: in each __main__.py, after load_dotenv(), map
   OPENAI_BASE_URL → OPENAI_API_BASE if the latter is not already set.

https://claude.ai/code/session_01UA8Dx6GkhaeJNAnJhwS1ty
…7U3V

Fix: use correct default model and base URL for OpenAI-compatible APIs
Align personalized_learning agent with restaurant_finder's approach for
supporting multiple LLM API backends (OpenAI, DeepSeek, vLLM, Gemini).

Changes:
- agent.py: Replace hardcoded Vertex AI client with LiteLLM in both the
  ADK Agent (model=LiteLlm) and the _generate_a2ui_content helper
  (litellm.acompletion). Auto-detect model from available API keys with
  priority AI_MODEL > LITELLM_MODEL > GENAI_MODEL > auto-detect.
- agent.py: Map OPENAI_BASE_URL → OPENAI_API_BASE at startup so vLLM /
  DeepSeek / Ollama endpoints work out of the box with LiteLLM.
- pyproject.toml: Add litellm dependency.
- .env.template: Document all three API options (OpenAI-compatible,
  Gemini API, Vertex AI) with DeepSeek and vLLM examples.

https://claude.ai/code/session_01V5sNhR4nbocU1btT1fekRw
@google-cla
Copy link

google-cla bot commented Feb 27, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for OpenAI-compatible LLM APIs across various sample agents, which is a valuable enhancement. The implementation is mostly sound, but there are several areas for improvement regarding code duplication, robustness, and adherence to Python style conventions. Specifically, the logic for selecting a default model could be more intuitive, and there's a recurring pattern of unsafe environment variable access. Furthermore, significant portions of configuration and model selection logic are duplicated across multiple agents, which will make future maintenance challenging. The repository's style guide also requires tests for new code changes, which appear to be missing for the new logic introduced.

Comment on lines +98 to +102
_default_model = (
"openai/gpt-4o"
if os.getenv("OPENAI_API_KEY") and not os.getenv("GEMINI_API_KEY")
else "gemini/gemini-2.5-flash"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The logic for selecting the default model appears to prioritize Gemini if both OPENAI_API_KEY and GEMINI_API_KEY are set. Given that the documentation in .env.example recommends OpenAI, it would be more intuitive to prioritize OpenAI whenever its key is available.

Additionally, this block of model selection logic is duplicated across several sample agents (e.g., in contact_multiple_surfaces, orchestrator, restaurant_finder). This makes maintenance difficult. Consider refactoring this logic into a shared utility function to improve maintainability and ensure consistency.

Suggested change
_default_model = (
"openai/gpt-4o"
if os.getenv("OPENAI_API_KEY") and not os.getenv("GEMINI_API_KEY")
else "gemini/gemini-2.5-flash"
)
_default_model = (
"openai/gpt-4o"
if os.getenv("OPENAI_API_KEY")
else "gemini/gemini-2.5-flash"
)

Comment on lines +32 to +33
if os.getenv("OPENAI_BASE_URL") and not os.getenv("OPENAI_API_BASE"):
os.environ["OPENAI_API_BASE"] = os.environ["OPENAI_BASE_URL"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Accessing the environment variable with os.environ["OPENAI_BASE_URL"] can raise a KeyError if the variable is not set. Although the if condition checks for its existence, using os.getenv("OPENAI_BASE_URL") for the assignment is safer against future refactoring and more consistent with the check. This pattern is repeated across several __main__.py files in this PR.

Suggested change
if os.getenv("OPENAI_BASE_URL") and not os.getenv("OPENAI_API_BASE"):
os.environ["OPENAI_API_BASE"] = os.environ["OPENAI_BASE_URL"]
if os.getenv("OPENAI_BASE_URL") and not os.getenv("OPENAI_API_BASE"):
os.environ["OPENAI_API_BASE"] = os.getenv("OPENAI_BASE_URL")

if os.getenv("OPENAI_API_KEY") and not os.getenv("GEMINI_API_KEY")
else "gemini/gemini-2.5-flash"
)
LITELLM_MODEL = os.getenv("AI_MODEL") or os.getenv("LITELLM_MODEL", _default_model)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to PEP 8, variable names should be in lowercase_with_underscores. The name LITELLM_MODEL suggests it's a constant, but its value is determined dynamically from environment variables. Renaming it to litellm_model would improve readability and adherence to Python's style guide. Please note that you will also need to update its usage on line 119. This naming issue is also present in other files (e.g., MODEL_ID in samples/personalized_learning/agent/agent.py).

Suggested change
LITELLM_MODEL = os.getenv("AI_MODEL") or os.getenv("LITELLM_MODEL", _default_model)
litellm_model = os.getenv("AI_MODEL") or os.getenv("LITELLM_MODEL", _default_model)

…-server

Update api-server.ts to support OpenAI / DeepSeek / vLLM alongside the
existing Google Gemini and Vertex AI backends.

Changes:
- Remove hard requirement for GOOGLE_CLOUD_PROJECT; now only needed for
  Vertex AI or Agent Engine. Server starts with just OPENAI_API_KEY.
- Add OPENAI_BASE_URL / OPENAI_API_BASE and AI_MODEL / LITELLM_MODEL
  constants for OpenAI-compatible endpoint configuration.
- Update initGenAI() to auto-detect backend: OpenAI-compatible >
  Gemini API > Vertex AI.
- Add callOpenAI() helper that calls any OpenAI-compatible endpoint via
  fetch (no extra npm dependency needed).
- Update generateLocalQuiz(), handleChatRequest(), and
  handleCombinedChatRequest() to route through callOpenAI() when the
  OpenAI backend is selected, falling back to google-genai otherwise.

Usage examples in .env:
  DeepSeek : OPENAI_API_KEY=... OPENAI_BASE_URL=https://api.deepseek.com/v1 AI_MODEL=openai/deepseek-chat
  vLLM     : OPENAI_API_KEY=any OPENAI_BASE_URL=http://localhost:8000/v1 AI_MODEL=openai/<model>
  OpenAI   : OPENAI_API_KEY=sk-...

https://claude.ai/code/session_01V5sNhR4nbocU1btT1fekRw
…api-server

Allows the server to start and run without any GCP credentials/gcloud CLI.

Problems fixed:
1. initializeApp({ credential: applicationDefault() }) was called
   unconditionally at module load, crashing the process when GCP ADC /
   gcloud is not available (ECONNREFUSED on /api/chat endpoints).
2. queryAgentEngine() always called getAccessToken() → gcloud CLI even
   when Agent Engine was not configured.

Changes:
- Move IS_LOCAL_DEV_MODE check before Firebase init so it can guard it.
- Wrap initializeApp() in try/catch, only run when VITE_FIREBASE_API_KEY
  is set. Store getAuth() result in firebaseAuth (null in local dev mode).
- Update authenticateRequest() to accept null firebaseAuth (local dev).
- Add generateLocalFlashcards(), getStaticAudioContent(),
  getStaticVideoContent(), generateLocalContent() helpers.
- /a2ui-agent/a2a/query: when Agent Engine is not configured, generate
  all content formats locally using the configured LLM backend instead of
  calling Agent Engine / gcloud.

Result: server starts with just OPENAI_API_KEY (or GEMINI_API_KEY).
No gcloud, no GCP project, no Firebase required for local development.

https://claude.ai/code/session_01V5sNhR4nbocU1btT1fekRw
…d LLM

Restructure the notebook so that Google Cloud / gcloud are no longer
required for basic local use. The new flow is:

  Step 1 – LLM API Configuration  (OpenAI / DeepSeek / vLLM / Gemini / Vertex AI)
  Step 2 – Install deps & write .env
  Step 3 – npm run dev  (fully local, no Agent Engine needed)
  Optional – Deploy to Vertex AI Agent Engine
  Optional – Production deployment (Cloud Run + Firebase Hosting)

Key changes:
- Prerequisites: replace "GCP project + gcloud required" with "LLM API
  key required; GCP optional".
- Step 1 config cell: three options (OpenAI-compatible / Gemini / Vertex AI)
  instead of just PROJECT_ID.
- Step 2 auth cell: runs npm install instead of gcloud auth.
- Step 2 setup cell: validates selected backend and writes .env with the
  correct env vars (OPENAI_API_KEY, OPENAI_BASE_URL, AI_MODEL, GEMINI_API_KEY,
  or GOOGLE_GENAI_USE_VERTEXAI + GOOGLE_CLOUD_PROJECT).
- Step 3: now just "npm run dev" instructions; Agent Engine deploy
  moved to "Optional" section.
- configure cell: handles empty AGENT_RESOURCE_ID gracefully (skip).
- Architecture section: documents the LLM backend env vars table.

https://claude.ai/code/session_01V5sNhR4nbocU1btT1fekRw
Vite's watcher crashes with UNKNOWN error when it tries to watch NFS
files inside the Python .venv directory. Add server.watch.ignored to
exclude .venv and node_modules from watching.

https://claude.ai/code/session_01V5sNhR4nbocU1btT1fekRw
The "Agent Engine not configured, generating content locally" message was
misleading users into thinking their LLM API key wasn't being used.

- Replace with "✅ Local mode: generating <format> content via <backend>"
  so it's clear the configured API IS being used.
- Fix "Error calling Gemini" log that appeared even when using OpenAI.

https://claude.ai/code/session_01V5sNhR4nbocU1btT1fekRw
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

2 participants