feat(sdk): expose user-provided idempotency key and scope in task context #2903
Conversation
🦋 Changeset detectedLatest commit: c21d103 The changes in this PR will be included in the next version bump. This PR includes changesets to release 27 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
WalkthroughAdds first-class support for user-provided idempotency key options across the stack. A new optional JSON field TaskRun.idempotencyKeyOptions (Prisma) and ClickHouse columns idempotency_key_user / idempotency_key_scope were introduced. RunEngine, runAttemptSystem, triggering/SDK plumbing, presenters, replication service, and API schemas were updated to accept, persist, propagate, and prefer user-provided idempotency key and scope. The idempotency-keys module gained types and helpers to attach/extract options and to create/reset scope-aware keys. Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
📜 Recent review detailsConfiguration used: Repository UI Review profile: CHILL Plan: Pro ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (6)
🚧 Files skipped from review as they are similar to previous changes (3)
🧰 Additional context used📓 Path-based instructions (9)**/*.{ts,tsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
{packages/core,apps/webapp}/**/*.{ts,tsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
**/*.{ts,tsx,js,jsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
**/*.ts📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)
Files:
**/*.{js,ts,jsx,tsx,json,md,yaml,yml}📄 CodeRabbit inference engine (AGENTS.md)
Files:
{packages,integrations}/**/*📄 CodeRabbit inference engine (CLAUDE.md)
Files:
apps/webapp/app/**/*.{ts,tsx}📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Files:
apps/webapp/app/services/**/*.server.{ts,tsx}📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Files:
apps/webapp/**/*.{ts,tsx}📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Files:
🧠 Learnings (16)📓 Common learnings📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2026-01-15T11:50:06.044ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2026-01-15T11:50:06.044ZApplied to files:
📚 Learning: 2025-07-12T18:06:04.133ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2025-11-27T16:26:37.432ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
📚 Learning: 2025-11-27T16:27:35.304ZApplied to files:
🧬 Code graph analysis (2)packages/core/src/v3/serverOnly/idempotencyKeys.ts (2)
apps/webapp/app/services/runsReplicationService.server.ts (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
🔇 Additional comments (6)
✏️ Tip: You can disable this entire section by setting Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review CompleteYour review story is ready! Comment !reviewfast on this PR to re-generate the story. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@apps/webapp/app/services/runsReplicationService.server.ts`:
- Around line 894-895: The `#extractIdempotencyKeyScope` method currently returns
whatever is stored in idempotencyKeyOptions without validation; update it to
defensively validate the extracted scope is a string and one of the allowed
values "run" | "attempt" | "global" (matching the check used in
runAttemptSystem.ts) and if not, fall back to a safe default (e.g., "run") or
handle it the same way runAttemptSystem.ts does; locate and modify the
`#extractIdempotencyKeyScope` function to perform this whitelist check before
returning the scope.
In `@internal-packages/clickhouse/src/taskRuns.ts`:
- Around line 96-97: TASK_RUN_COLUMNS has its entries out of order causing
writes to map to wrong ClickHouse columns; update the TASK_RUN_COLUMNS array so
that columns added by migrations are appended in the actual table order: ensure
after the existing columns you place concurrency_key, bulk_action_group_ids,
worker_queue, max_duration_in_seconds, idempotency_key_user,
idempotency_key_scope (in that exact sequence) so the array order matches the
ClickHouse schema and insert mapping is correct.
🧹 Nitpick comments (1)
apps/webapp/app/presenters/v3/SpanPresenter.server.ts (1)
713-726: Consider adding Zod validation for the parsed JSON options.The type assertion on
run.idempotencyKeyOptionsassumes the stored JSON matches the expected shape. If the data is malformed (e.g., from a migration issue or schema evolution), this could cause subtle runtime issues when accessingoptions.key.As per coding guidelines, Zod should be used for validation in
apps/webapp.♻️ Suggested validation approach
+import { z } from "zod"; + +const IdempotencyKeyOptionsSchema = z.object({ + key: z.string().optional(), + scope: z.string().optional(), +}).nullable(); + getUserProvidedIdempotencyKey( run: Pick<FindRunResult, "idempotencyKey" | "idempotencyKeyOptions"> ): string | null { - const options = run.idempotencyKeyOptions as { - key?: string; - scope?: string; - } | null; + const parsed = IdempotencyKeyOptionsSchema.safeParse(run.idempotencyKeyOptions); + const options = parsed.success ? parsed.data : null; if (options?.key) { return options.key; } return run.idempotencyKey; }
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
references/hello-world/src/trigger/idempotency.tsis excluded by!references/**
📒 Files selected for processing (15)
apps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/services/runsReplicationService.server.tsinternal-packages/clickhouse/schema/013_add_task_runs_v2_idempotency_key_options.sqlinternal-packages/clickhouse/src/taskRuns.tsinternal-packages/database/prisma/migrations/20260116154810_add_idempotency_key_options_to_task_run/migration.sqlinternal-packages/database/prisma/schema.prismainternal-packages/run-engine/src/engine/index.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsinternal-packages/run-engine/src/engine/types.tspackages/core/src/v3/idempotencyKeys.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/schemas/common.tspackages/trigger-sdk/src/v3/shared.ts
🧰 Additional context used
📓 Path-based instructions (12)
internal-packages/database/prisma/migrations/**/*.sql
📄 CodeRabbit inference engine (CLAUDE.md)
internal-packages/database/prisma/migrations/**/*.sql: When editing the Prisma schema, remove extraneous migration lines related to specific tables:_BackgroundWorkerToBackgroundWorkerFile,_BackgroundWorkerToTaskQueue,_TaskRunToTaskRunTag,_WaitpointRunConnections,_completedWaitpoints,SecretStore_key_idx, and unrelatedTaskRunindexes
Database indexes must use CONCURRENTLY to avoid table locks and must be in their own separate migration file
Files:
internal-packages/database/prisma/migrations/20260116154810_add_idempotency_key_options_to_task_run/migration.sql
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead
**/*.{ts,tsx}: Always import tasks from@trigger.dev/sdk, never use@trigger.dev/sdk/v3or deprecatedclient.defineJobpattern
Every Trigger.dev task must be exported and have a uniqueidproperty with no timeouts in the run function
Files:
internal-packages/run-engine/src/engine/index.tsapps/webapp/app/services/runsReplicationService.server.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/clickhouse/src/taskRuns.tsapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tspackages/core/src/v3/schemas/common.tspackages/core/src/v3/idempotencyKeys.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/trigger-sdk/src/v3/shared.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use function declarations instead of default exports
Import from
@trigger.dev/coreusing subpaths only, never import from root
Files:
internal-packages/run-engine/src/engine/index.tsapps/webapp/app/services/runsReplicationService.server.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/clickhouse/src/taskRuns.tsapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tspackages/core/src/v3/schemas/common.tspackages/core/src/v3/idempotencyKeys.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/trigger-sdk/src/v3/shared.ts
**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)
**/*.ts: When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs
Do not use high-cardinality attributes in OTEL metrics such as UUIDs/IDs (envId, userId, runId, projectId, organizationId), unbounded integers (itemCount, batchSize, retryCount), timestamps (createdAt, startTime), or free-form strings (errorMessage, taskName, queueName)
When exporting OTEL metrics via OTLP to Prometheus, be aware that the exporter automatically adds unit suffixes to metric names (e.g., 'my_duration_ms' becomes 'my_duration_ms_milliseconds', 'my_counter' becomes 'my_counter_total'). Account for these transformations when writing Grafana dashboards or Prometheus queries
Files:
internal-packages/run-engine/src/engine/index.tsapps/webapp/app/services/runsReplicationService.server.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/clickhouse/src/taskRuns.tsapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tspackages/core/src/v3/schemas/common.tspackages/core/src/v3/idempotencyKeys.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/trigger-sdk/src/v3/shared.ts
**/*.{js,ts,jsx,tsx,json,md,yaml,yml}
📄 CodeRabbit inference engine (AGENTS.md)
Format code using Prettier before committing
Files:
internal-packages/run-engine/src/engine/index.tsapps/webapp/app/services/runsReplicationService.server.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/clickhouse/src/taskRuns.tsapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tspackages/core/src/v3/schemas/common.tspackages/core/src/v3/idempotencyKeys.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/trigger-sdk/src/v3/shared.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use zod for validation in packages/core and apps/webapp
Files:
apps/webapp/app/services/runsReplicationService.server.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tspackages/core/src/v3/schemas/common.tspackages/core/src/v3/idempotencyKeys.ts
apps/webapp/app/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access all environment variables through the
envexport ofenv.server.tsinstead of directly accessingprocess.envin the Trigger.dev webapp
Files:
apps/webapp/app/services/runsReplicationService.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.ts
apps/webapp/app/services/**/*.server.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Separate testable services from configuration files; follow the pattern of
realtimeClient.server.ts(testable service) andrealtimeClientGlobal.server.ts(configuration) in the webapp
Files:
apps/webapp/app/services/runsReplicationService.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
apps/webapp/**/*.{ts,tsx}: When importing from@trigger.dev/corein the webapp, use subpath exports from the package.json instead of importing from the root path
Follow the Remix 2.1.0 and Express server conventions when updating the main trigger.dev webappAccess environment variables via
envexport fromapps/webapp/app/env.server.ts, never useprocess.envdirectly
Files:
apps/webapp/app/services/runsReplicationService.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.ts
{packages,integrations}/**/*
📄 CodeRabbit inference engine (CLAUDE.md)
Add a changeset when modifying any public package in
packages/*orintegrations/*usingpnpm run changeset:add
Files:
packages/core/src/v3/schemas/api.tspackages/core/src/v3/schemas/common.tspackages/core/src/v3/idempotencyKeys.tspackages/trigger-sdk/src/v3/shared.ts
internal-packages/clickhouse/schema/**/*.sql
📄 CodeRabbit inference engine (CLAUDE.md)
internal-packages/clickhouse/schema/**/*.sql: ClickHouse migrations must use Goose format with-- +goose Upand-- +goose Downmarkers
Follow ClickHouse naming conventions:raw_prefix for input tables,_v1,_v2suffixes for versioning,_mv_v1suffix for materialized views
Files:
internal-packages/clickhouse/schema/013_add_task_runs_v2_idempotency_key_options.sql
packages/trigger-sdk/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code
Files:
packages/trigger-sdk/src/v3/shared.ts
🧠 Learnings (25)
📓 Common learnings
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Scope idempotency keys globally or to current run using the scope parameter
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeys.create()` to create idempotency keys for preventing duplicate task executions
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Export tasks with unique IDs within the project to enable proper task discovery and execution
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeyTTL` option to define a time window during which duplicate triggers return the original run
📚 Learning: 2026-01-15T11:50:06.044Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-15T11:50:06.044Z
Learning: Applies to internal-packages/database/prisma/migrations/**/*.sql : When editing the Prisma schema, remove extraneous migration lines related to specific tables: `_BackgroundWorkerToBackgroundWorkerFile`, `_BackgroundWorkerToTaskQueue`, `_TaskRunToTaskRunTag`, `_WaitpointRunConnections`, `_completedWaitpoints`, `SecretStore_key_idx`, and unrelated `TaskRun` indexes
Applied to files:
internal-packages/database/prisma/migrations/20260116154810_add_idempotency_key_options_to_task_run/migration.sqlinternal-packages/database/prisma/schema.prismainternal-packages/clickhouse/src/taskRuns.tsinternal-packages/clickhouse/schema/013_add_task_runs_v2_idempotency_key_options.sql
📚 Learning: 2026-01-15T11:50:06.044Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-15T11:50:06.044Z
Learning: Applies to internal-packages/database/prisma/migrations/**/*.sql : Database indexes must use CONCURRENTLY to avoid table locks and must be in their own separate migration file
Applied to files:
internal-packages/database/prisma/migrations/20260116154810_add_idempotency_key_options_to_task_run/migration.sql
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeys.create()` to create idempotency keys for preventing duplicate task executions
Applied to files:
internal-packages/run-engine/src/engine/index.tsapps/webapp/app/services/runsReplicationService.server.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/clickhouse/src/taskRuns.tsapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tspackages/core/src/v3/schemas/common.tspackages/core/src/v3/idempotencyKeys.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeyTTL` option to define a time window during which duplicate triggers return the original run
Applied to files:
internal-packages/run-engine/src/engine/index.tsapps/webapp/app/services/runsReplicationService.server.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/database/prisma/schema.prismaapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tspackages/core/src/v3/idempotencyKeys.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Scope idempotency keys globally or to current run using the scope parameter
Applied to files:
internal-packages/run-engine/src/engine/index.tsapps/webapp/app/services/runsReplicationService.server.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/database/prisma/schema.prismainternal-packages/clickhouse/src/taskRuns.tsinternal-packages/clickhouse/schema/013_add_task_runs_v2_idempotency_key_options.sqlapps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tspackages/core/src/v3/schemas/common.tspackages/core/src/v3/idempotencyKeys.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Attach metadata to task runs using the metadata option when triggering, and access/update it inside runs using metadata functions
Applied to files:
internal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/clickhouse/src/taskRuns.tspackages/core/src/v3/schemas/common.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Export tasks with unique IDs within the project to enable proper task discovery and execution
Applied to files:
internal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/clickhouse/src/taskRuns.tspackages/core/src/v3/schemas/common.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use the `task()` function from `trigger.dev/sdk/v3` to define tasks with id and run properties
Applied to files:
internal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tspackages/core/src/v3/schemas/common.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2026-01-15T11:50:06.044Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-15T11:50:06.044Z
Learning: Applies to **/*.{ts,tsx} : Every Trigger.dev task must be exported and have a unique `id` property with no timeouts in the run function
Applied to files:
internal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/types.tspackages/core/src/v3/schemas/common.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `tasks.batchTrigger()` to trigger multiple runs of a single task with different payloads
Applied to files:
internal-packages/run-engine/src/engine/index.tsapps/webapp/app/runEngine/services/triggerTask.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata methods (set, del, replace, append, remove, increment, decrement, stream, flush) to update metadata during task execution
Applied to files:
internal-packages/run-engine/src/engine/index.tsinternal-packages/run-engine/src/engine/types.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `schemaTask()` from `trigger.dev/sdk/v3` with Zod schema for payload validation
Applied to files:
internal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/api.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `runs.subscribeToRunsWithTag()` to subscribe to all runs with a specific tag
Applied to files:
packages/core/src/v3/schemas/api.ts
📚 Learning: 2025-01-13T18:31:48.160Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 1608
File: apps/webapp/app/v3/services/triggerTask.server.ts:418-418
Timestamp: 2025-01-13T18:31:48.160Z
Learning: The `MachinePresetName` schema is used to validate machine preset values in the trigger.dev codebase, ensuring type safety and validation of machine preset options.
Applied to files:
packages/core/src/v3/schemas/api.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.trigger()` to trigger a task from inside another task with specified payload
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `tasks.trigger()` with type-only imports to trigger tasks from backend code without importing the task implementation
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Configure OpenTelemetry instrumentations and exporters in trigger.config.ts for enhanced logging
Applied to files:
internal-packages/run-engine/src/engine/types.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Use build extensions in trigger.config.ts (additionalFiles, additionalPackages, aptGet, prismaExtension, etc.) to customize the build
Applied to files:
internal-packages/run-engine/src/engine/types.ts
📚 Learning: 2026-01-15T11:50:06.044Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-15T11:50:06.044Z
Learning: Applies to internal-packages/clickhouse/schema/**/*.sql : Follow ClickHouse naming conventions: `raw_` prefix for input tables, `_v1`, `_v2` suffixes for versioning, `_mv_v1` suffix for materialized views
Applied to files:
internal-packages/clickhouse/schema/013_add_task_runs_v2_idempotency_key_options.sql
📚 Learning: 2026-01-15T11:50:06.044Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-15T11:50:06.044Z
Learning: Applies to internal-packages/clickhouse/schema/**/*.sql : ClickHouse migrations must use Goose format with `-- +goose Up` and `-- +goose Down` markers
Applied to files:
internal-packages/clickhouse/schema/013_add_task_runs_v2_idempotency_key_options.sql
📚 Learning: 2025-07-12T18:06:04.133Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2264
File: apps/webapp/app/services/runsRepository.server.ts:172-174
Timestamp: 2025-07-12T18:06:04.133Z
Learning: In apps/webapp/app/services/runsRepository.server.ts, the in-memory status filtering after fetching runs from Prisma is intentionally used as a workaround for ClickHouse data delays. This approach is acceptable because the result set is limited to a maximum of 100 runs due to pagination, making the performance impact negligible.
Applied to files:
apps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.ts
📚 Learning: 2025-08-14T12:13:20.455Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 2392
File: packages/cli-v3/src/utilities/gitMeta.ts:195-218
Timestamp: 2025-08-14T12:13:20.455Z
Learning: In the GitMeta schema (packages/core/src/v3/schemas/common.ts), all fields are intentionally optional to handle partial data from various deployment contexts (local, GitHub Actions, GitHub App). Functions like getGitHubAppMeta() are designed to work with missing environment variables rather than validate their presence.
Applied to files:
internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts
📚 Learning: 2025-11-27T16:26:37.432Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-27T16:26:37.432Z
Learning: Applies to packages/trigger-sdk/**/*.{ts,tsx} : In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `trigger.dev/sdk/v3` for all imports in Trigger.dev tasks
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
🧬 Code graph analysis (5)
apps/webapp/app/services/runsReplicationService.server.ts (2)
internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts (2)
run(1922-1928)run(1930-1939)packages/core/src/v3/schemas/common.ts (2)
TaskRun(209-237)TaskRun(239-239)
apps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.ts (3)
apps/webapp/app/presenters/v3/SpanPresenter.server.ts (4)
getUserProvidedIdempotencyKey(713-726)run(608-614)run(616-681)run(683-685)apps/webapp/app/services/runsReplicationService.server.ts (4)
run(846-905)run(907-916)run(957-960)run(962-965)internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts (2)
run(1922-1928)run(1930-1939)
packages/core/src/v3/idempotencyKeys.ts (3)
packages/trigger-sdk/src/v3/idempotencyKeys.ts (1)
IdempotencyKey(8-8)packages/core/src/v3/apiClient/index.ts (1)
resetIdempotencyKey(665-680)packages/core/src/v3/apiClient/core.ts (1)
ZodFetchOptions(31-39)
internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts (4)
apps/webapp/app/presenters/v3/SpanPresenter.server.ts (3)
run(608-614)run(616-681)run(683-685)apps/webapp/app/services/runsReplicationService.server.ts (4)
run(846-905)run(907-916)run(957-960)run(962-965)apps/webapp/app/v3/services/completeAttempt.server.ts (1)
run(419-517)internal-packages/run-engine/src/run-queue/index.ts (1)
options(1126-1142)
packages/trigger-sdk/src/v3/shared.ts (1)
packages/core/src/v3/idempotencyKeys.ts (2)
makeIdempotencyKey(90-104)getIdempotencyKeyOptions(29-33)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (39)
internal-packages/database/prisma/migrations/20260116154810_add_idempotency_key_options_to_task_run/migration.sql (1)
1-2: LGTM — straightforward nullable JSONB column addition.No extra indexes or extraneous lines introduced.
internal-packages/clickhouse/schema/013_add_task_runs_v2_idempotency_key_options.sql (1)
1-16: LGTM — Goose markers and column additions look correct.Defaults align with non-null String columns.
internal-packages/database/prisma/schema.prisma (1)
592-594: Good addition of idempotencyKeyOptions with clear doc comment.This aligns with the new persistence layer for user-provided idempotency metadata.
internal-packages/run-engine/src/engine/types.ts (1)
127-128: Nice addition of idempotencyKeyOptions to TriggerParams.This cleanly exposes the user-provided key and scope in the engine API.
internal-packages/run-engine/src/engine/index.ts (1)
398-399: LGTM — idempotencyKeyOptions is correctly threaded into TaskRun creation.This ensures the new options are persisted alongside the run.
Also applies to: 548-548
apps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.ts (3)
21-39: LGTM! Helper function correctly extracts user-provided idempotency key.The implementation is consistent with the pattern used in
SpanPresenter.server.ts. One minor observation: SpanPresenter's version returnsstring | nullwhile this returnsstring | undefined. Both work correctly but the slight inconsistency could be unified in a future refactor.
61-61: LGTM!Correctly adds
idempotencyKeyOptionsto the select object, enabling the helper function to access user-provided key options.
466-466: LGTM!Correctly uses the helper to prefer user-provided idempotency keys over the stored hash, ensuring API consumers receive the original key they provided.
packages/core/src/v3/schemas/common.ts (2)
217-220: LGTM! Schema additions for idempotency key metadata.The fields are correctly added as optional for backwards compatibility. The JSDoc comments clearly document the purpose. The scope enum values align with the
IdempotencyKeyOptionsSchemadefined inapi.ts.
380-383: LGTM! Consistent schema extension in V3TaskRun.The additions mirror the
TaskRunschema changes, maintaining consistency across both schemas.packages/core/src/v3/schemas/api.ts (4)
152-158: Schema design consideration: Bothkeyandscopeare required.The schema requires both
keyandscopefields. This aligns with the design whereidempotencyKeyOptionsis only populated when the user explicitly provides both. If partial options should be supported (e.g., key without scope), consider makingscopeoptional with a default.Based on learnings, the scope parameter is used to "scope idempotency keys globally or to current run." The current required design seems intentional to ensure explicit scoping when user-provided keys are used.
202-203: LGTM!Correctly adds
idempotencyKeyOptionsas an optional field toTriggerTaskRequestBody.options, allowing users to provide their original key and scope when triggering tasks.
253-254: LGTM!Consistently adds
idempotencyKeyOptionstoBatchTriggerTaskItem.options, maintaining parity with single-task triggers.
360-361: LGTM!Correctly adds
idempotencyKeyOptionstoCreateBatchRequestBodyfor the 2-phase batch API.internal-packages/clickhouse/src/taskRuns.ts (2)
158-159: LGTM! Type definitions updated consistently.Both
TaskRunFieldTypesandTaskRunInsertArrayare updated with the new string fields at the correct positions matchingTASK_RUN_COLUMNS.Also applies to: 291-292
43-44: LGTM! ClickHouse schema extension for idempotency key metadata.The fields use
default("")which aligns with the extraction helpers inrunsReplicationService.server.tsthat return empty strings when options are missing. The ClickHouse migration (013_add_task_runs_v2_idempotency_key_options.sql) exists with matching column definitions and proper Goose format, and all four schema locations intaskRuns.tsare updated consistently with correct column ordering.apps/webapp/app/runEngine/services/triggerTask.server.ts (1)
307-307: LGTM!Correctly propagates
idempotencyKeyOptionsfrom the request body to the engine's trigger method, enabling the run engine to persist the user-provided key and scope.internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts (5)
197-197: LGTM!Correctly adds
idempotencyKeyOptionsto the Prisma select to enable the helper functions.
265-266: LGTM!Correctly populates
idempotencyKeyandidempotencyKeyScopein theTaskRunContextusing the new helper methods.
427-427: LGTM!Correctly includes
idempotencyKeyOptionsin the update query's select clause.
576-577: LGTM!Correctly applies the helpers to populate idempotency key fields in the execution context returned to workers.
1921-1939: LGTM! Well-implemented helper methods.The private helpers follow the same pattern established across the codebase:
#getUserProvidedIdempotencyKeycorrectly returns user-provided key if available, falling back to the hash#getIdempotencyKeyScopeproperly validates that scope is one of the allowed values before returningThe explicit scope validation on lines 1935-1936 is a good defensive check that ensures only valid enum values propagate to consumers.
apps/webapp/app/presenters/v3/SpanPresenter.server.ts (2)
232-232: LGTM!The refactoring to use
getUserProvidedIdempotencyKeyproperly prioritizes user-provided keys over stored hashes, which aligns with the PR objective.Also applies to: 648-648
358-358: LGTM!The addition of
idempotencyKeyOptionsto the select query ensures the field is available for the helper method.packages/trigger-sdk/src/v3/shared.ts (10)
14-14: LGTM!Import of
getIdempotencyKeyOptionsis correctly added to support the new feature.
2118-2135: LGTM!The idempotency key processing in
trigger_internalcorrectly extracts options and passes both the stringified key and options to the API.
2065-2094: InconsistentidempotencyKeyOptionspropagation across transform functions.This function correctly extracts and propagates
idempotencyKeyOptions, but the other five similar transform functions do not:
transformBatchItemsStream(lines 1791-1835)transformBatchItemsStreamForWait(lines 1843-1887)transformBatchByTaskItemsStream(lines 1894-1938)transformBatchByTaskItemsStreamForWait(lines 1945-1989)transformSingleTaskBatchItemsStream(lines 1996-2042)This means user-provided idempotency key options won't be preserved when using streaming batches via
batchTriggerById,batchTriggerByIdAndWait,batchTriggerTasks, orbatchTriggerAndWaitTasks.Is this intentional, or should the same pattern be applied to all transform functions for consistency?
630-654: Similar inconsistency in array-based batch paths.The public
batchTriggerById,batchTriggerByIdAndWait,batchTriggerTasks, andbatchTriggerAndWaitTasksfunctions don't extractidempotencyKeyOptionsfrom individual items in their array paths, whilebatchTrigger_internalandbatchTriggerAndWait_internaldo.This creates an inconsistent user experience where the same feature works for some batch APIs but not others.
Also applies to: 886-910, 1146-1170, 1407-1430
1544-1545: LGTM!The
executeBatchTwoPhaseoptions type correctly includesidempotencyKeyOptionswith the proper scope union type.Also applies to: 1559-1560
2239-2254: LGTM!The batch-level idempotency key options are correctly processed and passed to
executeBatchTwoPhasefor the array path.
2301-2316: LGTM!The streaming path correctly extracts and passes batch-level idempotency key options to
executeBatchTwoPhaseStreaming.
2374-2403: LGTM!The
triggerAndWait_internalcorrectly processes the idempotency key and extracts options for storage.
2512-2529: LGTM!The batch-level idempotency key processing in
batchTriggerAndWait_internalarray path is correct.
2581-2598: LGTM!The streaming path in
batchTriggerAndWait_internalcorrectly handles batch-level idempotency key options.packages/core/src/v3/idempotencyKeys.ts (5)
7-12: LGTM!New types correctly define the idempotency key scope and options structure. Using
typeinstead ofinterfacealigns with coding guidelines.
14-46: LGTM!The symbol-based metadata attachment pattern is well-implemented:
- Uses
Symbol.for()for consistent cross-realm identity- Creates String objects via
new String()to allow property attachment while maintaining string semantics- Internal helper is properly marked with
@internalJSDoc
48-61: LGTM!The updated
isIdempotencyKeycorrectly handles both primitive strings and String objects created byattachIdempotencyKeyOptions.
141-153: LGTM!The
createIdempotencyKeyfunction now correctly attaches the original user key and scope as metadata, enabling downstream extraction viagetIdempotencyKeyOptions.
192-279: LGTM!The
resetIdempotencyKeyfunction handles multiple input formats correctly:
- Direct 64-char hashes pass through efficiently
- IdempotencyKey objects with attached options have their metadata extracted
- Raw strings/arrays are hashed with proper scope handling
- Clear error messages when required context is missing outside task runs
The documentation is thorough with good examples.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
… also added a test for the resetting changes
|
@claude could you review this PR please? |
|
Claude finished @ericallam's task in 2m 14s —— View job PR Review: feat(sdk): expose user-provided idempotency key and scope in task contextOverviewThis PR solves an important usability problem: previously, idempotency keys were hashed (SHA-256) before storage, making debugging difficult since users couldn't see the value they originally set. The solution elegantly stores the original key and scope alongside the hash, exposing them via Architecture Assessment ✅The implementation approach is well-designed:
Strengths 💪
Minor Observations 📝
SummaryThis is a well-implemented feature that:
The code is clean, well-organized, and follows the repository's patterns. LGTM 👍 | Branch: |
|
@ericallam on the run page can we update the "Details" panel? If the user idempotency key is set show that, and the scope |
Summary
ctx.run.idempotencyKeyas the user-provided key (not the hash)ctx.run.idempotencyKeyScopeto show the scope ("run", "attempt", or "global")Problem
Idempotency keys were hashed (SHA-256) before storage, making debugging difficult since users couldn't see the value they originally set or search for runs by idempotency key.
Solution
Attach metadata to the
Stringobject returned byidempotencyKeys.create()using a Symbol, extract it in the SDK before the API call, and store it in the database alongside the hash.Test plan