|
1 | 1 | --- |
2 | 2 | title: "AI Agents" |
3 | 3 | sidebarTitle: "Overview" |
4 | | -description: "Run AI SDK chat completions as durable Trigger.dev agents with built-in realtime streaming, multi-turn conversations, and message persistence." |
| 4 | +description: "Durable multi-turn AI chats — one Trigger.dev task per conversation, surviving refreshes, deploys, and crashes." |
5 | 5 | --- |
6 | 6 |
|
7 | 7 | import RcBanner from "/snippets/ai-chat-rc-banner.mdx"; |
8 | 8 |
|
9 | 9 | <RcBanner /> |
10 | 10 |
|
11 | | -## Overview |
12 | | - |
13 | | -The `@trigger.dev/sdk` provides a custom [ChatTransport](https://sdk.vercel.ai/docs/ai-sdk-ui/transport) for the Vercel AI SDK's `useChat` hook. This lets you run chat completions as **durable Trigger.dev agents** instead of fragile API routes — with automatic retries, observability, and realtime streaming built in. |
14 | | - |
15 | | -**How it works:** |
16 | | -1. The frontend sends messages via `useChat` through `TriggerChatTransport` |
17 | | -2. The first message triggers a Trigger.dev agent; subsequent messages resume the **same run** via input streams |
18 | | -3. The agent streams `UIMessageChunk` events back via Trigger.dev's realtime streams |
19 | | -4. The AI SDK's `useChat` processes the stream natively — text, tool calls, reasoning, etc. |
20 | | -5. Between turns, the run stays idle briefly then suspends (freeing compute) until the next message |
21 | | - |
22 | | -No custom API routes needed. Your chat backend is a Trigger.dev agent. |
23 | | - |
24 | | -<Accordion title="How it works (sequence diagrams)"> |
25 | | - |
26 | | -### First message flow |
27 | | - |
28 | | -```mermaid |
29 | | -sequenceDiagram |
30 | | - participant User |
31 | | - participant useChat as useChat + Transport |
32 | | - participant API as Trigger.dev API |
33 | | - participant Task as chat.agent Worker |
34 | | - participant LLM as LLM Provider |
35 | | -
|
36 | | - User->>useChat: sendMessage("Hello") |
37 | | - useChat->>useChat: No session for chatId → trigger new run |
38 | | - useChat->>API: triggerTask(payload, tags: [chat:id]) |
39 | | - API-->>useChat: { runId, publicAccessToken } |
40 | | - useChat->>useChat: Store session, subscribe to SSE |
41 | | -
|
42 | | - API->>Task: Start run with ChatTaskWirePayload |
43 | | - Task->>Task: onChatStart({ chatId, messages, clientData }) |
44 | | - Task->>Task: onTurnStart({ chatId, messages }) |
45 | | - Task->>LLM: streamText({ model, messages, abortSignal }) |
46 | | - LLM-->>Task: Stream response chunks |
47 | | - Task->>API: Write chunks to session.out |
48 | | - API-->>useChat: SSE: UIMessageChunks |
49 | | - useChat-->>User: Render streaming text |
50 | | - Task->>API: Write turn-complete control record |
51 | | - API-->>useChat: SSE: turn complete + refreshed token |
52 | | - useChat->>useChat: Close stream, update session |
53 | | - Task->>Task: onTurnComplete({ messages, stopped: false }) |
54 | | - Task->>Task: Wait for next message (idle → suspend) |
55 | | -``` |
56 | | - |
57 | | -### Multi-turn flow |
58 | | - |
59 | | -```mermaid |
60 | | -sequenceDiagram |
61 | | - participant User |
62 | | - participant useChat as useChat + Transport |
63 | | - participant API as Trigger.dev API |
64 | | - participant Task as chat.agent Worker |
65 | | - participant LLM as LLM Provider |
66 | | -
|
67 | | - Note over Task: Suspended, waiting for message |
68 | | -
|
69 | | - User->>useChat: sendMessage("Tell me more") |
70 | | - useChat->>useChat: Session exists → send via input stream |
71 | | - useChat->>API: sendInputStream(runId, "chat-messages", payload) |
72 | | - Note right of useChat: Only sends new message (not full history) |
73 | | -
|
74 | | - API->>Task: Deliver to messagesInput |
75 | | - Task->>Task: Wake from suspend |
76 | | - Task->>Task: Append to accumulated messages |
77 | | - Task->>Task: onTurnStart({ turn: 1 }) |
78 | | - Task->>LLM: streamText({ messages: [all accumulated] }) |
79 | | - LLM-->>Task: Stream response |
80 | | - Task->>API: Write chunks to session.out |
81 | | - API-->>useChat: SSE: UIMessageChunks |
82 | | - useChat-->>User: Render streaming text |
83 | | - Task->>API: Write turn-complete control record |
84 | | - Task->>Task: onTurnComplete({ turn: 1 }) |
85 | | - Task->>Task: Wait for next message (idle → suspend) |
86 | | -``` |
87 | | - |
88 | | -### Stop signal flow |
89 | | - |
90 | | -```mermaid |
91 | | -sequenceDiagram |
92 | | - participant User |
93 | | - participant useChat as useChat + Transport |
94 | | - participant API as Trigger.dev API |
95 | | - participant Task as chat.agent Worker |
96 | | - participant LLM as LLM Provider |
97 | | -
|
98 | | - Note over Task: Streaming response... |
99 | | -
|
100 | | - User->>useChat: Click "Stop" |
101 | | - useChat->>API: sendInputStream(runId, "chat-stop", { stop: true }) |
102 | | - API->>Task: Deliver to stopInput |
103 | | - Task->>Task: stopController.abort() |
104 | | - LLM-->>Task: Stream ends (AbortError) |
105 | | - Task->>Task: cleanupAbortedParts(responseMessage) |
106 | | - Note right of Task: Remove partial tool calls,<br/>mark streaming parts as done |
107 | | - Task->>API: Write trigger:turn-complete |
108 | | - API-->>useChat: SSE: turn complete |
109 | | - Task->>Task: onTurnComplete({ stopped: true }) |
110 | | - Task->>Task: Wait for next message |
111 | | -``` |
112 | | - |
113 | | -</Accordion> |
114 | | - |
115 | | -## How multi-turn works |
116 | | - |
117 | | -### One conversation, many runs |
118 | | - |
119 | | -Each chat is backed by a durable Session row — the unit of state that owns the chat's runs across their full lifecycle. The conversation's identity stays keyed on `chatId` across run boundaries; messages flow through the session's `.in` channel; responses stream on `.out`. |
120 | | - |
121 | | -Within a session, a single run handles many turns. After each AI response, the run waits for the next message via the session's `.in` channel. The frontend transport handles this automatically — triggers a new run on the session for the first message, and sends subsequent messages into the existing run. |
122 | | - |
123 | | -Every turn is a span inside the same run in the Trigger.dev dashboard. The Agents dashboard view also lets you inspect the session directly — all runs that have ever touched it, filterable and resumable. |
124 | | - |
125 | | -### Warm and suspended states |
126 | | - |
127 | | -After each turn, the run goes through two phases of waiting: |
128 | | - |
129 | | -1. **Warm phase** (default 30s) — The run stays active and responds instantly to the next message. Uses compute. |
130 | | -2. **Suspended phase** (default up to 1h) — The run suspends, freeing compute. It wakes when the next message arrives. There's a brief delay as the run resumes. |
| 11 | +An AI chat isn't a request — it's a session. `chat.agent` runs every conversation as a single long-lived Trigger.dev task: you write the loop, it wakes up when a message arrives, freezes when none do, and the same in-memory state and on-disk workspace survive across page refreshes, deploys, idle gaps, and crashes. The substrate handles the parts most teams stitch together by hand — turn lifecycle, mid-stream resume, recovery from cancel/crash/OOM, HITL approvals, deploy upgrades — so your code is the loop you'd write anyway: messages in, `streamText` out. |
131 | 12 |
|
132 | | -If no message arrives within the turn timeout, the run ends gracefully. The session stays open. The next message from the frontend automatically starts a fresh run **on the same session** — chat history and identity persist across the run boundary. |
| 13 | +## A minimal example |
133 | 14 |
|
134 | | -<Info> |
135 | | - You are not charged for compute during the suspended phase. Only the idle phase uses compute resources. |
136 | | -</Info> |
| 15 | +A `chat.agent` task takes `messages`, calls `streamText`, and returns the result. The frontend wires the [Vercel AI SDK's `useChat`](https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-chat) to a `TriggerChatTransport`. No API routes. |
137 | 16 |
|
138 | | -### Resume and inbox |
| 17 | +```ts trigger/chat.ts |
| 18 | +import { chat } from "@trigger.dev/sdk/ai"; |
| 19 | +import { streamText } from "ai"; |
| 20 | +import { openai } from "@ai-sdk/openai"; |
139 | 21 |
|
140 | | -Because the session outlives the run, a chat you were in yesterday resumes against the same session today — even after the original run has idle-timed out or crashed. Pass `resume: true` to `useChat` on page load and the transport reconnects via `sessionId` + `lastEventId`, kicking off a new run only if the user sends a message. |
141 | | - |
142 | | -You can also enumerate every chat in your environment with [`sessions.list`](/ai-chat/sessions#sessions-list-options-requestoptions): |
143 | | - |
144 | | -```ts |
145 | | -import { sessions } from "@trigger.dev/sdk"; |
| 22 | +export const myChat = chat.agent({ |
| 23 | + id: "my-chat", |
| 24 | + run: async ({ messages, signal }) => |
| 25 | + streamText({ model: openai("gpt-4o"), messages, abortSignal: signal }), |
| 26 | +}); |
| 27 | +``` |
146 | 28 |
|
147 | | -for await (const s of sessions.list({ type: "chat.agent", tag: "user:user-456" })) { |
148 | | - console.log(s.id, s.externalId, s.createdAt, s.closedAt); |
| 29 | +```tsx app/components/Chat.tsx |
| 30 | +import { useChat } from "@ai-sdk/react"; |
| 31 | +import { useTriggerChatTransport } from "@trigger.dev/sdk/chat/react"; |
| 32 | + |
| 33 | +export function Chat() { |
| 34 | + const transport = useTriggerChatTransport<typeof myChat>({ |
| 35 | + task: "my-chat", |
| 36 | + accessToken: ({ chatId }) => mintChatAccessToken(chatId), |
| 37 | + startSession: ({ chatId, taskId, clientData }) => |
| 38 | + startChatSession({ chatId, taskId, clientData }), |
| 39 | + }); |
| 40 | + const { messages, sendMessage } = useChat({ transport }); |
| 41 | + // ... render UI |
149 | 42 | } |
150 | 43 | ``` |
151 | 44 |
|
152 | | -This powers inbox-style UIs (your own chat list page) without maintaining a separate index. |
153 | | - |
154 | | -### What the backend accumulates |
155 | | - |
156 | | -The backend automatically accumulates the full conversation history across turns. After the first turn, the frontend transport only sends the new user message — not the entire history. This is handled transparently by the transport and agent. |
157 | | - |
158 | | -The accumulated messages are available in: |
159 | | -- `run()` as `messages` (`ModelMessage[]`) — for passing to `streamText` |
160 | | -- `onTurnStart()` as `uiMessages` (`UIMessage[]`) — for persisting before streaming |
161 | | -- `onTurnComplete()` as `uiMessages` (`UIMessage[]`) — for persisting after the response |
162 | | - |
163 | | -<Warning> |
164 | | - **Always spread `chat.toStreamTextOptions()` into every `streamText` call.** It wires up the `prepareStep` callback that drives compaction, steering, and background injection. Skipping the spread silently disables those features. See [Backend → chat.agent()](/ai-chat/backend#chat-agent). |
165 | | -</Warning> |
166 | | - |
167 | | -Agents appear in the **Agents** section of the dashboard (not Tasks) and can be tested via the **Playground**. |
168 | | - |
169 | | -## Three approaches |
170 | | - |
171 | | -There are three ways to build the backend, from most opinionated to most flexible: |
172 | | - |
173 | | -| Approach | Use when | What you get | |
174 | | -|----------|----------|--------------| |
175 | | -| [chat.agent()](/ai-chat/backend#chat-agent) | Most apps | Auto-piping, lifecycle hooks, message accumulation, stop handling | |
176 | | -| [chat.createSession()](/ai-chat/backend#chat-createsession) | Need a loop but not hooks | Async iterator with per-turn helpers, message accumulation, stop handling | |
177 | | -| [Raw task + primitives](/ai-chat/backend#raw-task-with-primitives) | Full control | Manual control of every step — use `chat.messages`, `chat.createStopSignal()`, etc. | |
178 | | - |
179 | | -## Related |
180 | | - |
181 | | -- [Quick Start](/ai-chat/quick-start) — Get a working chat in 3 steps |
182 | | -- [Database persistence](/ai-chat/patterns/database-persistence) — Conversation + session state across hooks (ORM-agnostic) |
183 | | -- [Code execution sandbox](/ai-chat/patterns/code-sandbox) — Warm/teardown pattern for E2B (or similar) with `onWait` / `chat.local` |
184 | | -- [Backend](/ai-chat/backend) — Backend approaches in detail |
185 | | -- [Frontend](/ai-chat/frontend) — Transport setup, sessions, client data |
186 | | -- [Types](/ai-chat/types) — TypeScript patterns, including custom `UIMessage` with `chat.withUIMessage` |
187 | | -- [`chat.local`](/ai-chat/chat-local) — Per-run typed state across hooks, run, tools, subtasks |
188 | | -- [Sub-agents pattern](/ai-chat/patterns/sub-agents) — Subtask-as-tool, `target: "root"` streaming, `ai.toolExecute` helpers |
189 | | -- [Background injection](/ai-chat/background-injection) — `chat.inject()` and `chat.defer()` for between-turn work |
190 | | -- [API Reference](/ai-chat/reference) — Complete reference tables |
| 45 | +See [Quick Start](/ai-chat/quick-start) for the matching server actions and a runnable project. |
| 46 | + |
| 47 | +## Why use AI Agents on Trigger.dev |
| 48 | + |
| 49 | +- **Resume across refreshes, deploys, and crashes.** A chat in progress when you redeploy keeps streaming on the new version. Mid-stream refreshes pick up where they left off. |
| 50 | +- **Native AI SDK support.** Text, tool calls, reasoning, and custom `data-*` parts all flow through `useChat` over a custom `ChatTransport`. No custom protocol to maintain. |
| 51 | +- **Multi-turn for free.** Each turn is a step inside the same durable task; conversation history accumulates server-side, so clients only ship the new message. |
| 52 | +- **Fast cold starts.** Opt-in [Head Start](/ai-chat/fast-starts#head-start) runs the first `streamText` step in your warm Next.js / Hono / SvelteKit server while the agent boots in parallel — cuts time-to-first-chunk roughly in half. |
| 53 | +- **Production primitives ship in the box.** Stop generation, steering, edits, branching, sub-agents, HITL tool approvals, version upgrades, recovery from cancel/crash/OOM — all first-class. |
| 54 | +- **Observable.** Every turn is a span in the Trigger.dev dashboard. Sessions are queryable via `sessions.list` for inbox-style UIs. |
| 55 | + |
| 56 | +## How it fits together |
| 57 | + |
| 58 | +Three primitives, related but distinct: |
| 59 | + |
| 60 | +- **Chat agents** — the SDK surface you define with [`chat.agent()`](/ai-chat/backend#chat-agent). Owns the turn loop, lifecycle hooks, and the response stream. |
| 61 | +- **Sessions** — the durable, bi-directional channel keyed on `chatId` that holds the conversation across run boundaries. A chat agent runs *on top of* a [Session](/ai-chat/sessions). |
| 62 | +- **Sub-agents** — Delegate work from one agent to another via [`AgentChat`](/ai-chat/patterns/sub-agents). The sub-agent runs as its own durable agent on its own session; its response streams back through the parent as preliminary tool results, so the frontend sees the sub-agent working inside the parent's tool card. |
| 63 | + |
| 64 | +## Next steps |
| 65 | + |
| 66 | +<CardGroup cols={2}> |
| 67 | + <Card title="Quick Start" icon="rocket" href="/ai-chat/quick-start"> |
| 68 | + Get a working chat in three steps — agent, token, frontend. |
| 69 | + </Card> |
| 70 | + <Card title="How it works" icon="diagram-project" href="/ai-chat/how-it-works"> |
| 71 | + Sessions, the turn loop, durable streams, and what survives a refresh. |
| 72 | + </Card> |
| 73 | + <Card title="Backend" icon="server" href="/ai-chat/backend"> |
| 74 | + `chat.agent` options, lifecycle hooks, and the raw-task primitives. |
| 75 | + </Card> |
| 76 | + <Card title="Patterns" icon="puzzle-piece" href="/ai-chat/patterns/sub-agents"> |
| 77 | + HITL approvals, branching, sub-agents, OOM/crash recovery. |
| 78 | + </Card> |
| 79 | +</CardGroup> |
0 commit comments