Skip to content

Commit 76f391c

Browse files
committed
docs(ai-chat): rewrite overview to lead with the session framing
The overview was carrying internals (sequence diagrams, suspend timing, inbox patterns, "what the backend accumulates") before a reader saw a runnable line of code. Rewritten to match how the product is actually pitched: - Opener leads with "An AI chat isn't a request — it's a session" - Minimal example (chat.agent + useChat) appears in the first H2, not after three screens of mechanics - Outcome bullets cover durability, native AI SDK support, multi-turn, Head Start, production primitives, and observability - "How it fits together" names the three primitives (chat agents, Sessions, sub-agents via AgentChat) without explaining internals - CardGroup CTAs to Quick Start / How it works / Backend / Patterns Length: ~520 words (was ~1180). Internals that were duplicated here already live on how-it-works, sessions, and backend.
1 parent 6ceb79a commit 76f391c

1 file changed

Lines changed: 62 additions & 173 deletions

File tree

docs/ai-chat/overview.mdx

Lines changed: 62 additions & 173 deletions
Original file line numberDiff line numberDiff line change
@@ -1,190 +1,79 @@
11
---
22
title: "AI Agents"
33
sidebarTitle: "Overview"
4-
description: "Run AI SDK chat completions as durable Trigger.dev agents with built-in realtime streaming, multi-turn conversations, and message persistence."
4+
description: "Durable multi-turn AI chats — one Trigger.dev task per conversation, surviving refreshes, deploys, and crashes."
55
---
66

77
import RcBanner from "/snippets/ai-chat-rc-banner.mdx";
88

99
<RcBanner />
1010

11-
## Overview
12-
13-
The `@trigger.dev/sdk` provides a custom [ChatTransport](https://sdk.vercel.ai/docs/ai-sdk-ui/transport) for the Vercel AI SDK's `useChat` hook. This lets you run chat completions as **durable Trigger.dev agents** instead of fragile API routes — with automatic retries, observability, and realtime streaming built in.
14-
15-
**How it works:**
16-
1. The frontend sends messages via `useChat` through `TriggerChatTransport`
17-
2. The first message triggers a Trigger.dev agent; subsequent messages resume the **same run** via input streams
18-
3. The agent streams `UIMessageChunk` events back via Trigger.dev's realtime streams
19-
4. The AI SDK's `useChat` processes the stream natively — text, tool calls, reasoning, etc.
20-
5. Between turns, the run stays idle briefly then suspends (freeing compute) until the next message
21-
22-
No custom API routes needed. Your chat backend is a Trigger.dev agent.
23-
24-
<Accordion title="How it works (sequence diagrams)">
25-
26-
### First message flow
27-
28-
```mermaid
29-
sequenceDiagram
30-
participant User
31-
participant useChat as useChat + Transport
32-
participant API as Trigger.dev API
33-
participant Task as chat.agent Worker
34-
participant LLM as LLM Provider
35-
36-
User->>useChat: sendMessage("Hello")
37-
useChat->>useChat: No session for chatId → trigger new run
38-
useChat->>API: triggerTask(payload, tags: [chat:id])
39-
API-->>useChat: { runId, publicAccessToken }
40-
useChat->>useChat: Store session, subscribe to SSE
41-
42-
API->>Task: Start run with ChatTaskWirePayload
43-
Task->>Task: onChatStart({ chatId, messages, clientData })
44-
Task->>Task: onTurnStart({ chatId, messages })
45-
Task->>LLM: streamText({ model, messages, abortSignal })
46-
LLM-->>Task: Stream response chunks
47-
Task->>API: Write chunks to session.out
48-
API-->>useChat: SSE: UIMessageChunks
49-
useChat-->>User: Render streaming text
50-
Task->>API: Write turn-complete control record
51-
API-->>useChat: SSE: turn complete + refreshed token
52-
useChat->>useChat: Close stream, update session
53-
Task->>Task: onTurnComplete({ messages, stopped: false })
54-
Task->>Task: Wait for next message (idle → suspend)
55-
```
56-
57-
### Multi-turn flow
58-
59-
```mermaid
60-
sequenceDiagram
61-
participant User
62-
participant useChat as useChat + Transport
63-
participant API as Trigger.dev API
64-
participant Task as chat.agent Worker
65-
participant LLM as LLM Provider
66-
67-
Note over Task: Suspended, waiting for message
68-
69-
User->>useChat: sendMessage("Tell me more")
70-
useChat->>useChat: Session exists → send via input stream
71-
useChat->>API: sendInputStream(runId, "chat-messages", payload)
72-
Note right of useChat: Only sends new message (not full history)
73-
74-
API->>Task: Deliver to messagesInput
75-
Task->>Task: Wake from suspend
76-
Task->>Task: Append to accumulated messages
77-
Task->>Task: onTurnStart({ turn: 1 })
78-
Task->>LLM: streamText({ messages: [all accumulated] })
79-
LLM-->>Task: Stream response
80-
Task->>API: Write chunks to session.out
81-
API-->>useChat: SSE: UIMessageChunks
82-
useChat-->>User: Render streaming text
83-
Task->>API: Write turn-complete control record
84-
Task->>Task: onTurnComplete({ turn: 1 })
85-
Task->>Task: Wait for next message (idle → suspend)
86-
```
87-
88-
### Stop signal flow
89-
90-
```mermaid
91-
sequenceDiagram
92-
participant User
93-
participant useChat as useChat + Transport
94-
participant API as Trigger.dev API
95-
participant Task as chat.agent Worker
96-
participant LLM as LLM Provider
97-
98-
Note over Task: Streaming response...
99-
100-
User->>useChat: Click "Stop"
101-
useChat->>API: sendInputStream(runId, "chat-stop", { stop: true })
102-
API->>Task: Deliver to stopInput
103-
Task->>Task: stopController.abort()
104-
LLM-->>Task: Stream ends (AbortError)
105-
Task->>Task: cleanupAbortedParts(responseMessage)
106-
Note right of Task: Remove partial tool calls,<br/>mark streaming parts as done
107-
Task->>API: Write trigger:turn-complete
108-
API-->>useChat: SSE: turn complete
109-
Task->>Task: onTurnComplete({ stopped: true })
110-
Task->>Task: Wait for next message
111-
```
112-
113-
</Accordion>
114-
115-
## How multi-turn works
116-
117-
### One conversation, many runs
118-
119-
Each chat is backed by a durable Session row — the unit of state that owns the chat's runs across their full lifecycle. The conversation's identity stays keyed on `chatId` across run boundaries; messages flow through the session's `.in` channel; responses stream on `.out`.
120-
121-
Within a session, a single run handles many turns. After each AI response, the run waits for the next message via the session's `.in` channel. The frontend transport handles this automatically — triggers a new run on the session for the first message, and sends subsequent messages into the existing run.
122-
123-
Every turn is a span inside the same run in the Trigger.dev dashboard. The Agents dashboard view also lets you inspect the session directly — all runs that have ever touched it, filterable and resumable.
124-
125-
### Warm and suspended states
126-
127-
After each turn, the run goes through two phases of waiting:
128-
129-
1. **Warm phase** (default 30s) — The run stays active and responds instantly to the next message. Uses compute.
130-
2. **Suspended phase** (default up to 1h) — The run suspends, freeing compute. It wakes when the next message arrives. There's a brief delay as the run resumes.
11+
An AI chat isn't a request — it's a session. `chat.agent` runs every conversation as a single long-lived Trigger.dev task: you write the loop, it wakes up when a message arrives, freezes when none do, and the same in-memory state and on-disk workspace survive across page refreshes, deploys, idle gaps, and crashes. The substrate handles the parts most teams stitch together by hand — turn lifecycle, mid-stream resume, recovery from cancel/crash/OOM, HITL approvals, deploy upgrades — so your code is the loop you'd write anyway: messages in, `streamText` out.
13112

132-
If no message arrives within the turn timeout, the run ends gracefully. The session stays open. The next message from the frontend automatically starts a fresh run **on the same session** — chat history and identity persist across the run boundary.
13+
## A minimal example
13314

134-
<Info>
135-
You are not charged for compute during the suspended phase. Only the idle phase uses compute resources.
136-
</Info>
15+
A `chat.agent` task takes `messages`, calls `streamText`, and returns the result. The frontend wires the [Vercel AI SDK's `useChat`](https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-chat) to a `TriggerChatTransport`. No API routes.
13716

138-
### Resume and inbox
17+
```ts trigger/chat.ts
18+
import { chat } from "@trigger.dev/sdk/ai";
19+
import { streamText } from "ai";
20+
import { openai } from "@ai-sdk/openai";
13921

140-
Because the session outlives the run, a chat you were in yesterday resumes against the same session today — even after the original run has idle-timed out or crashed. Pass `resume: true` to `useChat` on page load and the transport reconnects via `sessionId` + `lastEventId`, kicking off a new run only if the user sends a message.
141-
142-
You can also enumerate every chat in your environment with [`sessions.list`](/ai-chat/sessions#sessions-list-options-requestoptions):
143-
144-
```ts
145-
import { sessions } from "@trigger.dev/sdk";
22+
export const myChat = chat.agent({
23+
id: "my-chat",
24+
run: async ({ messages, signal }) =>
25+
streamText({ model: openai("gpt-4o"), messages, abortSignal: signal }),
26+
});
27+
```
14628

147-
for await (const s of sessions.list({ type: "chat.agent", tag: "user:user-456" })) {
148-
console.log(s.id, s.externalId, s.createdAt, s.closedAt);
29+
```tsx app/components/Chat.tsx
30+
import { useChat } from "@ai-sdk/react";
31+
import { useTriggerChatTransport } from "@trigger.dev/sdk/chat/react";
32+
33+
export function Chat() {
34+
const transport = useTriggerChatTransport<typeof myChat>({
35+
task: "my-chat",
36+
accessToken: ({ chatId }) => mintChatAccessToken(chatId),
37+
startSession: ({ chatId, taskId, clientData }) =>
38+
startChatSession({ chatId, taskId, clientData }),
39+
});
40+
const { messages, sendMessage } = useChat({ transport });
41+
// ... render UI
14942
}
15043
```
15144

152-
This powers inbox-style UIs (your own chat list page) without maintaining a separate index.
153-
154-
### What the backend accumulates
155-
156-
The backend automatically accumulates the full conversation history across turns. After the first turn, the frontend transport only sends the new user message — not the entire history. This is handled transparently by the transport and agent.
157-
158-
The accumulated messages are available in:
159-
- `run()` as `messages` (`ModelMessage[]`) — for passing to `streamText`
160-
- `onTurnStart()` as `uiMessages` (`UIMessage[]`) — for persisting before streaming
161-
- `onTurnComplete()` as `uiMessages` (`UIMessage[]`) — for persisting after the response
162-
163-
<Warning>
164-
**Always spread `chat.toStreamTextOptions()` into every `streamText` call.** It wires up the `prepareStep` callback that drives compaction, steering, and background injection. Skipping the spread silently disables those features. See [Backend → chat.agent()](/ai-chat/backend#chat-agent).
165-
</Warning>
166-
167-
Agents appear in the **Agents** section of the dashboard (not Tasks) and can be tested via the **Playground**.
168-
169-
## Three approaches
170-
171-
There are three ways to build the backend, from most opinionated to most flexible:
172-
173-
| Approach | Use when | What you get |
174-
|----------|----------|--------------|
175-
| [chat.agent()](/ai-chat/backend#chat-agent) | Most apps | Auto-piping, lifecycle hooks, message accumulation, stop handling |
176-
| [chat.createSession()](/ai-chat/backend#chat-createsession) | Need a loop but not hooks | Async iterator with per-turn helpers, message accumulation, stop handling |
177-
| [Raw task + primitives](/ai-chat/backend#raw-task-with-primitives) | Full control | Manual control of every step — use `chat.messages`, `chat.createStopSignal()`, etc. |
178-
179-
## Related
180-
181-
- [Quick Start](/ai-chat/quick-start) — Get a working chat in 3 steps
182-
- [Database persistence](/ai-chat/patterns/database-persistence) — Conversation + session state across hooks (ORM-agnostic)
183-
- [Code execution sandbox](/ai-chat/patterns/code-sandbox) — Warm/teardown pattern for E2B (or similar) with `onWait` / `chat.local`
184-
- [Backend](/ai-chat/backend) — Backend approaches in detail
185-
- [Frontend](/ai-chat/frontend) — Transport setup, sessions, client data
186-
- [Types](/ai-chat/types) — TypeScript patterns, including custom `UIMessage` with `chat.withUIMessage`
187-
- [`chat.local`](/ai-chat/chat-local) — Per-run typed state across hooks, run, tools, subtasks
188-
- [Sub-agents pattern](/ai-chat/patterns/sub-agents) — Subtask-as-tool, `target: "root"` streaming, `ai.toolExecute` helpers
189-
- [Background injection](/ai-chat/background-injection)`chat.inject()` and `chat.defer()` for between-turn work
190-
- [API Reference](/ai-chat/reference) — Complete reference tables
45+
See [Quick Start](/ai-chat/quick-start) for the matching server actions and a runnable project.
46+
47+
## Why use AI Agents on Trigger.dev
48+
49+
- **Resume across refreshes, deploys, and crashes.** A chat in progress when you redeploy keeps streaming on the new version. Mid-stream refreshes pick up where they left off.
50+
- **Native AI SDK support.** Text, tool calls, reasoning, and custom `data-*` parts all flow through `useChat` over a custom `ChatTransport`. No custom protocol to maintain.
51+
- **Multi-turn for free.** Each turn is a step inside the same durable task; conversation history accumulates server-side, so clients only ship the new message.
52+
- **Fast cold starts.** Opt-in [Head Start](/ai-chat/fast-starts#head-start) runs the first `streamText` step in your warm Next.js / Hono / SvelteKit server while the agent boots in parallel — cuts time-to-first-chunk roughly in half.
53+
- **Production primitives ship in the box.** Stop generation, steering, edits, branching, sub-agents, HITL tool approvals, version upgrades, recovery from cancel/crash/OOM — all first-class.
54+
- **Observable.** Every turn is a span in the Trigger.dev dashboard. Sessions are queryable via `sessions.list` for inbox-style UIs.
55+
56+
## How it fits together
57+
58+
Three primitives, related but distinct:
59+
60+
- **Chat agents** — the SDK surface you define with [`chat.agent()`](/ai-chat/backend#chat-agent). Owns the turn loop, lifecycle hooks, and the response stream.
61+
- **Sessions** — the durable, bi-directional channel keyed on `chatId` that holds the conversation across run boundaries. A chat agent runs *on top of* a [Session](/ai-chat/sessions).
62+
- **Sub-agents** — Delegate work from one agent to another via [`AgentChat`](/ai-chat/patterns/sub-agents). The sub-agent runs as its own durable agent on its own session; its response streams back through the parent as preliminary tool results, so the frontend sees the sub-agent working inside the parent's tool card.
63+
64+
## Next steps
65+
66+
<CardGroup cols={2}>
67+
<Card title="Quick Start" icon="rocket" href="/ai-chat/quick-start">
68+
Get a working chat in three steps — agent, token, frontend.
69+
</Card>
70+
<Card title="How it works" icon="diagram-project" href="/ai-chat/how-it-works">
71+
Sessions, the turn loop, durable streams, and what survives a refresh.
72+
</Card>
73+
<Card title="Backend" icon="server" href="/ai-chat/backend">
74+
`chat.agent` options, lifecycle hooks, and the raw-task primitives.
75+
</Card>
76+
<Card title="Patterns" icon="puzzle-piece" href="/ai-chat/patterns/sub-agents">
77+
HITL approvals, branching, sub-agents, OOM/crash recovery.
78+
</Card>
79+
</CardGroup>

0 commit comments

Comments
 (0)