diff --git a/docs/ai-chat/custom-agents.mdx b/docs/ai-chat/custom-agents.mdx index 54c0461a75..fed9857d70 100644 --- a/docs/ai-chat/custom-agents.mdx +++ b/docs/ai-chat/custom-agents.mdx @@ -114,10 +114,11 @@ Each turn yielded by the iterator provides: | `continuation` | `boolean` | Whether this is a continuation run | | `previousTurnUsage` | `LanguageModelUsage \| undefined` | Token usage from the previous turn (undefined on turn 0) | | `totalUsage` | `LanguageModelUsage` | Cumulative token usage across all completed turns | +| `handover` | `{ isFinal: boolean } \| null` | The [`chat.headStart`](/ai-chat/fast-starts#handover-with-custom-agents) handover for this turn (turn 0 only); `null` otherwise | | Method | Description | | ----------------------------- | ---------------------------------------------------------------------------------------------------------- | -| `turn.complete(source)` | Pipe stream, capture response, accumulate, and signal turn-complete | +| `turn.complete(source?)` | Pipe stream, capture response, accumulate, and signal turn-complete. Call with no source on a final head-start handover (`turn.handover.isFinal`), where the warm step-1 partial is already the response | | `turn.done()` | Signal turn-complete only (when you have piped manually) | | `turn.addResponse(response)` | Add a response to the accumulator manually | | `turn.setMessages(uiMessages)`| Replace the accumulated messages — continuation seeding and on-demand compaction | diff --git a/docs/ai-chat/fast-starts.mdx b/docs/ai-chat/fast-starts.mdx index 6910dbb8b3..0217d703e8 100644 --- a/docs/ai-chat/fast-starts.mdx +++ b/docs/ai-chat/fast-starts.mdx @@ -108,7 +108,7 @@ if (payload.trigger === "preload") { ## Head Start -Head Start runs step 1's LLM call in your warm server process while the chat.agent run boots in parallel. The user sees one continuous turn: text first from your server, then a clean handover to the agent for tool execution and any further steps. +Head Start runs step 1's LLM call in your warm server process while the agent run boots in parallel. The user sees one continuous turn: text first from your server, then a clean handover to the agent for tool execution and any further steps. The agent you hand off to can be a `chat.agent`, a `chat.customAgent`, or a `chat.createSession` loop (see [Handover with custom agents](#handover-with-custom-agents)). `chat.headStart` returns a standard [Web Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) handler — `(req: Request) => Promise` — so it slots into any runtime that speaks Web Fetch. @@ -545,16 +545,86 @@ Head Start composes with [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemes Your hydrate hook shapes **model context**, not the transcript — dropping reasoning-only entries or unresolved tool rows from the returned chain is fine and does not affect what `onTurnComplete` persists or what the UI renders. +### Handover with custom agents + +The route handler is backend-agnostic: `agentId` can point at a `chat.agent`, a [`chat.customAgent`](/ai-chat/custom-agents), or a [`chat.createSession`](/ai-chat/custom-agents#managed-loop-chatcreatesession) loop. With `chat.agent` the handover is consumed for you (the steps above). The two hand-rolled backends consume it explicitly on turn 0. + +#### chat.createSession + +The turn iterator surfaces the handover as `turn.handover`. On a final (pure-text) handover, call `turn.complete()` with no source to finalize the warm partial without streaming; otherwise stream as usual. The iterator threads the spliced partial as `originalMessages` for you, so a resumed tool round merges into the handed-over assistant. + +```ts trigger/chat.ts +for await (const turn of session) { + // Pure-text handover (isFinal): step 1 already IS the response. + const result = turn.handover?.isFinal + ? undefined + : streamText({ + model: anthropic("claude-sonnet-4-6"), + messages: turn.messages, + abortSignal: turn.signal, + stopWhen: stepCountIs(10), + }); + + await turn.complete(result); // no source on a final handover +} +``` + +#### chat.customAgent + +In a hand-rolled loop, call `conversation.consumeHandover({ payload })` at the top of turn 0. It waits for the handover signal, seeds prior history from `payload.headStartMessages`, splices the warm step-1 partial into the accumulator, and returns `{ isFinal, skipped }`. + +```ts trigger/chat.ts +// Turn 0, gated on a head-start run: +if (turn === 0 && payload.trigger === "handover-prepare") { + const { isFinal, skipped } = await conversation.consumeHandover({ payload }); + if (skipped) return; // not a head-start run, or the warm handler aborted — exit + if (!isFinal) { + // The partial carries a pending tool call. Run step 2 to execute it, + // passing originalMessages so the tool output merges into the + // handed-over assistant instead of starting a new message. + const result = streamText({ + model: anthropic("claude-sonnet-4-6"), + messages: conversation.modelMessages, + stopWhen: stepCountIs(10), + }); + const response = await chat.pipeAndCapture(result, { + originalMessages: conversation.uiMessages, + }); + if (response) await conversation.addResponse(response); + } + await chat.writeTurnComplete(); // on isFinal the warm partial is already the response + return; +} +``` + +Gate the call on `trigger === "handover-prepare"` — `consumeHandover` consumes the warm handover, not a normal first message. See [Custom agents](/ai-chat/custom-agents) for the full loop (continuation seeding, stop handling, persistence). The lower-level `chat.waitForHandover({ payload })` and `accumulator.applyHandover(signal)` are exported if you need to wait and splice in separate steps. + + + Always pass `originalMessages: conversation.uiMessages` to `pipeAndCapture` in a custom loop. It keeps assistant message IDs stable across turns and lets a tool-approval or handover resume merge into the trailing assistant — the same threading `chat.agent` does internally. + + ### The `chat.headStart` API ```ts chat.headStart({ - agentId: string, // The chat.agent({ id }) you're handing off to + agentId: string, // The chat.agent / chat.customAgent id you're handing off to run: (args: HeadStartRunArgs) => Promise>, idleTimeoutInSeconds?: number, // How long the agent waits for the handover signal. Default: 60 + triggerConfig?: Partial, // Run options for the handover-prepare run }): (req: Request) => Promise ``` +`triggerConfig` sets run options on the auto-triggered handover-prepare run: `tags`, `queue`, `machine`, `maxAttempts`, `maxDuration`, `region`, and `lockToVersion`. The `chat:{chatId}` tag is prepended automatically. Because the session is created once on the first head-start turn (idempotent on the chat id), this is the only place to set those options for a head-start chat's lifetime, mirroring what [`chat.createStartSessionAction`](/ai-chat/sessions) sets for the direct-trigger path. + +```ts lib/chat-handler.ts +export const chatHandler = chat.headStart({ + agentId: "my-chat", + triggerConfig: { tags: ["org:acme"], queue: "chat", machine: "small-2x" }, + run: async ({ chat: helper }) => + streamText({ ...helper.toStreamTextOptions({ tools: headStartTools }), model, system }), +}); +``` + The `run` callback receives: - `messages: UIMessage[]` — user messages parsed from the request body. @@ -599,3 +669,4 @@ This is **not** a stock `useChat` `endpoint` — it's not the canonical request - [`chat.headStart` factory and types](/ai-chat/reference) — full signatures for `HeadStartRunArgs`, `HeadStartChatHelper`, `HeadStartSession`, `HeadStartHandlerOptions`. - [`headStart` transport option](/ai-chat/reference#triggerchattransport-options) — alongside `accessToken`, `startSession`, etc. - [`onPreload` hook](/ai-chat/lifecycle-hooks#onpreload) — the backend hook that fires when a run is preloaded. +- [Custom agents](/ai-chat/custom-agents) — the `chat.customAgent` and `chat.createSession` loops that `consumeHandover` / `turn.handover` plug into. diff --git a/docs/ai-chat/reference.mdx b/docs/ai-chat/reference.mdx index dabcbb44e8..e7fa3e2804 100644 --- a/docs/ai-chat/reference.mdx +++ b/docs/ai-chat/reference.mdx @@ -475,15 +475,29 @@ Each turn yielded by `chat.createSession()`. | `continuation` | `boolean` | Whether this is a continuation run | | `previousTurnUsage` | `LanguageModelUsage \| undefined` | Token usage from the previous turn (undefined on turn 0) | | `totalUsage` | `LanguageModelUsage` | Cumulative token usage across all completed turns | +| `handover` | `{ isFinal: boolean } \| null` | The [`chat.headStart`](/ai-chat/fast-starts#handover-with-custom-agents) handover for this turn (turn 0 only); `null` otherwise | | Method | Returns | Description | | ------------------------ | --------------------------------- | ------------------------------------------------------------------------------------------------------ | -| `complete(source)` | `Promise` | Pipe, capture, accumulate, cleanup, and signal turn-complete | +| `complete(source?)` | `Promise` | Pipe, capture, accumulate, cleanup, and signal turn-complete. Call with no source on a final head-start handover (`handover.isFinal`), where the warm partial is already the response | | `done()` | `Promise` | Signal turn-complete (when you've piped manually) | | `addResponse(response)` | `Promise` | Add response to accumulator manually | | `setMessages(uiMessages)`| `Promise` | Replace the accumulated messages (continuation seeding, compaction) | | `prepareStep()` | `function \| undefined` | `prepareStep` callback wiring compaction + injection — pass to `streamText` when not using `chat.toStreamTextOptions()` | +## HeadStartHandlerOptions + +Options for [`chat.headStart()`](/ai-chat/fast-starts#head-start), the warm-server first-turn handler (`@trigger.dev/sdk/chat-server`). + +| Option | Type | Default | Description | +| ---------------------- | ------------------------------------------------------- | ----------- | ---------------------------------------------------------------------------------------------------- | +| `agentId` | `string` | required | The `chat.agent` / `chat.customAgent` id to hand off to | +| `run` | `(args: HeadStartRunArgs) => Promise` | required | First-turn callback. Call `streamText` and spread `chat.toStreamTextOptions({ tools })` | +| `idleTimeoutInSeconds` | `number` | `60` | How long the agent waits for the handover signal | +| `triggerConfig` | `Partial` | `undefined` | Run options (tags, queue, machine, maxAttempts, maxDuration, region, lockToVersion) for the auto-triggered handover-prepare run. The `chat:{chatId}` tag is prepended automatically | + +`chat.headStart(options)` returns the handler `(req: Request) => Promise`. The `run` callback receives `HeadStartRunArgs`: `{ messages: UIMessage[], signal: AbortSignal, chat: HeadStartChatHelper }`, where the helper exposes `chat.toStreamTextOptions({ tools })` and a `chat.session` escape hatch. See [Head Start](/ai-chat/fast-starts#head-start) for the full guide. + ## chat namespace All methods available on the `chat` object from `@trigger.dev/sdk/ai`. @@ -499,6 +513,7 @@ All methods available on the `chat` object from `@trigger.dev/sdk/ai`. | `chat.messages` | Input stream for incoming messages — use `.waitWithIdleTimeout()` | | `chat.local({ id })` | Create a per-run typed local (see [`chat.local`](/ai-chat/chat-local)) | | `chat.createStartSessionAction(taskId, options?)` | Returns a server action that creates a chat Session + triggers the first run + returns a session-scoped PAT. Idempotent on `(env, externalId)`. | +| `chat.waitForHandover(options)` | Wait for a [`chat.headStart`](/ai-chat/fast-starts#handover-with-custom-agents) handover signal in a custom loop. Returns the signal or `null`. `chat.MessageAccumulator` wraps this as `consumeHandover()` / `applyHandover()` | | `chat.requestUpgrade()` | End the current run after this turn so the next message starts on the latest agent version. Server-orchestrated handoff. | | `chat.setTurnTimeout(duration)` | Override turn timeout at runtime (e.g. `"2h"`) | | `chat.setTurnTimeoutInSeconds(seconds)` | Override turn timeout at runtime (in seconds) |