feat(sdk): default merge events in SDK#155
Conversation
…re/59787-merge-events
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
lan17
left a comment
There was a problem hiding this comment.
I think there are still two blockers before we merge this. I left inline notes on the partial sink integration and the unconditional reconstruction work on the evaluation hot path.
lan17
left a comment
There was a problem hiding this comment.
Blocking issue:
server/src/agent_control_server/endpoints/evaluation.py:242-263now trustsX-Agent-Control-Merge-Eventsfrom the caller and skips server-side observability ingestion solely on that header. A direct API caller can set the header and suppress control-execution events without reconstructing/re-emitting them anywhere. The server needs to treat merged mode as a trusted SDK flow, or otherwise keep the default ingestion path for untrusted callers.
|
@lan17 Fixed. The server no longer trusts X-Agent-Control-Merge-Events by itself. Merged mode is now established during init(..., merge_events=True) / initAgent, and the server records that enablement for the initialized (authenticated client, agent) session. On /evaluation, the server only skips observability ingestion if that same client previously initialized that same agent with merge-events enabled; otherwise it keeps the default ingestion path. Since merge_events defaults to False, existing behavior remains unchanged unless merged mode is explicitly turned on. |
lan17
left a comment
There was a problem hiding this comment.
Re-reviewed current head 9af1755.
The original header-trust issue looks fixed, but there is still one blocker:
server/src/agent_control_server/merge_event_sessions.py:28-40scopes merge-enable state by(client.api_key, agent_name). For session-cookie auth,_authenticate_via_cookie()returnsAuthenticatedClient(api_key="", ...)inserver/src/agent_control_server/auth.py:118-121, so every cookie-authenticated browser session collapses to the same key("", agent_name). If one logged-in UI session initializes an agent with merge events enabled, any other logged-in UI session can sendX-Agent-Control-Merge-Events: trueand suppress server-side ingestion for that agent. The trusted-session key needs to use a per-session or otherwise user-specific identifier, not the emptyapi_keysentinel.
|
Stepping back from the line-level issues, I think the direction here is reasonable, but the architecture is taking on too much implicit session and state coupling too quickly. Separating evaluation semantics from observability event delivery is the right idea, and extracting shared event reconstruction logic is a real improvement over duplicating emission behavior in multiple places. Where this starts to feel brittle is that the behavior of a single evaluation request now depends on prior For a v1, I would be comfortable with a narrower version of this. I would keep merged event creation scoped only to the local-first SDK path, keep What I do not think we should do yet is spread this across multiple entry points, multiple trust mechanisms, and alternate delivery sinks at the same time. Once merge mode works in a way that is explicit, robust across auth modes and deployment topologies, and easy to reason about end to end, then adding a configurable sink is a straightforward follow-on. Right now So my recommendation would be: land a smaller, clearer version of merged creation first, prove that the ownership model is correct, and only then generalize it. I think the core idea is good. I just do not think we should lock in this much flexibility before the underlying contract is simpler and more stable. |
Summary
This PR makes the SDK the single owner of
ControlExecutionEventcreation and emission, moving all observability event reconstruction from the server to the SDK. The/api/v1/evaluationendpoint now returns only evaluation semantics, keeping evaluation and observability concerns cleanly separated.Key changes:
EvaluationResponse/evaluationendpoint is now evaluation-only — no longer builds or ingests observability events/api/v1/observability/events→ OSS storage)ControlDefinition.observability_identity()for composite conditionsBehavior
When SDK observability is enabled
check_evaluation_with_local()reconstructs local + server events in the SDK and enqueues one combined batchcheck_evaluation()reconstructs server events in the SDK and enqueues through the built-in pipelineWhen SDK observability is disabled
Error handling
What changed
SDK (
sdks/python/src/agent_control/):evaluation_events.pyevaluation.pyto support:EvaluationResponse+ cached control definitionscheck_evaluation()reconstruction when observability enabledmerge_eventsmode from Python SDK surfaceServer (
server/src/agent_control_server/):endpoints/evaluation.pyto be evaluation-onlyEvaluationResponsewithout observability payloads/evaluationendpointTypeScript client:
Testing
Tests updated to cover:
/evaluationbehavior with no server-side observability emissionMigration notes
/api/v1/evaluationno longer get observability automatically unless they also emit events through/api/v1/observability/events