The Chat Loop

Turn lifecycle

A turn starts when a user message arrives at the chat handler. The handler routes to a skill, picks a model tier, exposes a scoped tool catalog, and runs the model in a tool calling loop until the model stops, the budget runs out, or a pause point is written.

Updated today

Overview

At a glance

Entry: POST /api/chat/v2
Handler: src/app/api/chat/v2/route.ts
Engine: src/lib/skills/engine.ts
Default model: Allegro, escalates to Forte for reasoning-heavy skills
Turn budget: Declared per skill in the manifest (3 to 16)
Streaming: Server-sent events to the chat surface

A turn is the smallest unit of work in Ultron. It is one round trip of input to the model, one set of tool calls executed, and one set of outputs persisted. A skill almost always spans more than one turn because each tool call costs a round trip.

The handler is structured to make the path explicit. Read the file top to bottom and you see the same five steps every request takes: authenticate, route, pick a tier, build tools, run the loop. Each step is its own function and each one is testable in isolation.

End to end trace

What runs between the user pressing send and the assistant streaming back. Times are typical for an Allegro turn with two tool calls.

trace.logtext

1+0ms POST /api/chat/v2 accept request

2+8ms auth.session supabase user resolved

3+22ms conversation.load id=8f2a, state=idle -> running

4+30ms router.select skill=cortex.competitive-analysis

5+34ms models.tier tier=allegro, maxTurns=6

6+48ms tools.build 14 tools exposed

7+62ms cascade.compress history compressed to 4.1k tokens

8+88ms loop.turn 1 -> model prompt sent

9+1.4s loop.turn 1 <- model tool_use: web_search

10+1.4s tools.exec web_search query="kyc fintech competitors"

11+3.1s loop.turn 2 -> model tool result appended

12+4.2s loop.turn 2 <- model tool_use: canvas.comparison_table

13+4.2s tools.exec canvas.comparison_table rendered 9 rows

14+5.0s loop.turn 3 -> model tool result appended

15+6.4s loop.turn 3 <- model text response, stop_reason=end_turn

16+6.5s persist.assistant_message id=msg_77c1

17+6.5s conversation.state running -> idle

18+6.5s stream.close done

Router

The router decides which skill runs and which persona owns the turn. It runs once per turn at the very top of the handler.

src/app/api/chat/v2/route.tsts

97// Route the incoming message to a skill

98const route = await routeSkill({

99 message,

100 history: lastFew(messages, 4),

101 slashCommand: parseSlash(message), // /cortex, /specter, etc.

102 hint: conversation.persona, // sticky persona for the session

103})

104

105// route.skillId is one of 57 ids in the registry

106// route.persona is the UI label shown to the user

Routing is cheap and explicit. If the user types a slash command, the router shortcuts to the persona it names. Otherwise it asks a quick Minuet-tier call to classify the message. If the conversation has a sticky persona from a prior turn, the router prefers a skill inside that persona before looking elsewhere.

Model tier selection

Each skill declares a default tier. The router may override based on message length, attachment count, or a difficulty hint.

Tier	When it runs	Latency budget	Cost
Minuet	Routing decisions, memory judging, short responses	200ms p50	Cheapest
Allegro	Default skill execution	1.4s p50 first turn	Mid
Forte	Reasoning heavy skills, long planning	3s p50 first turn	Highest

Note

The model name is not hardcoded in the skill. The manifest declares a tier and the chat handler resolves the tier to a concrete model id at request time. Swapping models across the fleet is a one-line change.

Tool catalog

Every turn exposes a scoped set of tools. The catalog is built at the start of the turn and frozen until the turn ends.

src/lib/tools/registry.tsts

19export function buildToolCatalog(ctx: ToolContext): ToolFunctionSchema[] {

20 // 1. Provider modules first (sim-style modular tools)

21 const moduleTools = MODULES.flatMap((m) => m.schemas)

23 // 2. Native catalog from chat-tools.ts, filtered by skill allow-list

24 const native = getNativeTools().filter((t) => ctx.skill.tools.includes(t.name))

26 // 3. External servers (MCP) discovered per-user, cached 10 minutes

27 const external = getExternalServerTools(ctx.userId)

29 return [...moduleTools, ...native, ...external]

30}

Warning

A skill cannot call a tool that is not in its tools manifest field. The chat handler enforces the allow-list before dispatching, even if the model attempts to call an unlisted tool.

The tool calling loop

The core loop runs in src/lib/skills/engine.ts. Each iteration is one turn.

01
Send the prompt
Build the message array from compressed history plus the current user message. Attach the tool catalog. Stream the response from the model.
02
Inspect stop reason
The model returns one of end_turn, tool_use, max_tokens, or stop_sequence. Only tool_use continues the loop.
03
Execute the tool calls
For each tool use, dispatch into the registry. Provider modules win first. Native handlers second. External servers third. Errors are caught and returned as tool error results so the model can self-correct.
04
Check for pause
If any tool returned the __APPROVAL_REQUIRED__ marker, write a pause point row, set the conversation state to paused, and exit the loop. The turn resumes when the user approves.
05
Loop or finalize
Append tool results to the message array and send the next request. Repeat until stop reason is end_turn or the turn budget reaches zero.

Stop conditions

Four ways the loop can terminate. The state at exit is recorded on the assistant message.

Condition	Trigger	Conversation state after
end_turn	Model returned a final assistant message	idle
pause	wait_for_approval tool was called	paused
budget_exhausted	Turn count reached maxTurns	idle, with a warning attached
error	Unrecoverable tool or model error	failed

Persistence

What gets written to the database at the end of the turn.

The chat handler persists every assistant message, every tool call, and every tool result. Tool results are persisted as their own message rows with role tool so the next turn can reconstruct the exact context the model saw. Streaming is purely a UI affordance; the source of truth is the database.

src/app/api/chat/v2/route.tsts

280// At the bottom of the loop, after stop_reason is final

281await supabase.from('messages').insert([

282 ...toolMessages, // role='tool', tool_call_id set

283 {

284 conversation_id: conversation.id,

285 role: 'assistant',

286 content: finalContent,

287 tool_calls: aggregatedToolCalls,

288 turn_index: nextTurnIndex,

289 },

290])

291

292await supabase.from('conversations').update({

293 state: nextState,

294 last_message_at: new Date().toISOString(),

295}).eq('id', conversation.id)

Tip

To reproduce a session offline, dump the messages table for the conversation and replay them through the same chat handler. The handler is deterministic given the same inputs and tool catalog.

File map

src/app/api/chat

v2/route.tsmain handler

resume/route.tsresume from pause

src/lib/skills

engine.tstool calling loop

registry.ts57 skills

router.tsskill selection

memory.tscontext cascade

src/lib/tools

registry.tsdispatcher

types.ts

providers

hitl.ts

stripe.ts

src/lib/chat-tools.ts91 native tools

Glossary

Turn: One iteration of the tool calling loop. Begins with a model request and ends with a stop reason.
Stop reason: How a model response ended. One of end_turn, tool_use, max_tokens, stop_sequence.
Tool catalog: The list of tool-call schemas exposed to the model on this turn. Built from provider modules, native handlers, and external servers.
Turn budget: The maximum number of turns a skill can take before the loop exits. Declared per skill.
Tier: A model class: Minuet, Allegro, Forte, or Plinth. Resolved to a concrete model id at request time.
Stream: Server sent events sent to the chat surface while the loop runs. Streaming is presentation only.