Skip to content

Add image hard gate to bypass router for image requests#4203

Closed
aashna wants to merge 1 commit intomicrosoft:mainfrom
aashna:aashnagarg/image-hard-gate
Closed

Add image hard gate to bypass router for image requests#4203
aashna wants to merge 1 commit intomicrosoft:mainfrom
aashna:aashnagarg/image-hard-gate

Conversation

@aashna
Copy link
Copy Markdown
Contributor

@aashna aashna commented Mar 4, 2026

When a chat request contains an image attachment, bypass the model router entirely and use the heuristic-based path (_resolveWithoutRouterModel) which already handles vision-capable model selection via _applyVisionFallback.

The router model is text-only and cannot reason about image content, so sending image requests through the router would produce unreliable routing decisions. Instead, the existing vision fallback heuristic is the correct path.

Changes in automodeService.ts:

  • Check hasImage(chatRequest) before deciding to use router
  • When image detected and router configured, emit automode.imageHardGate telemetry event and fall through to heuristic path
  • usingRouterModel = routerConfigured && !requestHasImage

Changes in routerDecisionFetcher.ts:

  • Add optional hasImage parameter to getRoutedModel()
  • Pass has_image: true in request body when set (defense-in-depth for server-side gating in auto-intent-service)

Tests:

  • Add test: should send has_image flag when provided
  • Add test: should not send has_image when not provided

When a chat request contains an image attachment, bypass the model router
entirely and use the heuristic-based path (_resolveWithoutRouterModel) which
already handles vision-capable model selection via _applyVisionFallback.

The router model is text-only and cannot reason about image content, so sending
image requests through the router would produce unreliable routing decisions.
Instead, the existing vision fallback heuristic is the correct path.

Changes in automodeService.ts:
- Check hasImage(chatRequest) before deciding to use router
- When image detected and router configured, emit automode.imageHardGate
  telemetry event and fall through to heuristic path
- usingRouterModel = routerConfigured && !requestHasImage

Changes in routerDecisionFetcher.ts:
- Add optional hasImage parameter to getRoutedModel()
- Pass has_image: true in request body when set (defense-in-depth for
  server-side gating in auto-intent-service)

Tests:
- Add test: should send has_image flag when provided
- Add test: should not send has_image when not provided
Copilot AI review requested due to automatic review settings March 4, 2026 20:31
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an "image hard gate" to the AutomodeService so that chat requests containing image attachments bypass the router model entirely and use the existing heuristic-based path (_resolveWithoutRouterModel/_applyVisionFallback). The router is text-only and would produce unreliable routing decisions for image requests. As a secondary change, a hasImage flag is added to the RouterDecisionFetcher.getRoutedModel() API as a defense-in-depth mechanism to inform the server-side router of image presence.

Changes:

  • automodeService.ts: Introduces requestHasImage check before routing; when an image is detected and the router is configured, emits an automode.imageHardGate telemetry event and routes via the heuristic path instead.
  • routerDecisionFetcher.ts: Adds an optional hasImage parameter to getRoutedModel() that includes has_image: true in the POST request body when set.
  • routerDecisionFetcher.spec.ts: Adds two tests for the new hasImage parameter behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
src/platform/endpoint/node/automodeService.ts Adds image hard gate logic: detects images in the request, bypasses router, and emits telemetry
src/platform/endpoint/node/routerDecisionFetcher.ts Adds optional hasImage parameter to getRoutedModel() to pass has_image in the request body
src/platform/endpoint/node/test/routerDecisionFetcher.spec.ts New tests for has_image flag behavior in getRoutedModel()

Two issues were identified:

  1. Dead code in the defense-in-depth path (moderate): The hasImage parameter added to getRoutedModel is never passed as true from production code. The hard gate (usingRouterModel = routerConfigured && !requestHasImage) guarantees that _resolveWithRouterModel — the only caller of getRoutedModel — is reached only when requestHasImage is false. As a result, has_image: true is never included in any real router request body. For the stated defense-in-depth to function, requestHasImage needs to be threaded through _resolveWithRouterModel and passed to getRoutedModel. The two new tests in routerDecisionFetcher.spec.ts verify this unreachable scenario.

  2. Missing test coverage for the image hard gate in automodeService.spec.ts (moderate): The existing resolveAutoModeEndpoint test suite covers several bypass scenarios (inline chat, terminal, no router URL), but no test verifies the new image hard gate path: a panel chat request with an image reference that has mimeType starting with "image/" should bypass the router fetch and emit the automode.imageHardGate telemetry event.

Comment thread src/platform/endpoint/node/automodeService.ts
Comment thread src/platform/endpoint/node/automodeService.ts
@lramos15
Copy link
Copy Markdown
Member

I believe we've decided this one isn't needed as we will fallback to the client's image safety logic which seems to be working well

@lramos15 lramos15 closed this Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants