Dispatch-scoped pipelines

TL;DR

Think of these two as the sandwich around the executor. Every iteration of the dispatch loop fires them in order: dispatchInputPipeline runs, then the executor runs, then dispatchOutputPipeline runs. If the loop continues, the sandwich gets remade for the next iteration.

The cost rule: anything you put here runs once per iteration. Ten iterations means ten executions. Retrieval, history packing, memory scoring — none of that belongs here. It belongs in turn-scoped pipelines, which run once.

What does belong here: bounding the loop (otherwise it runs forever — the runner imposes no default cap), detecting when the model is stuck, deciding whether the dispatch should signal ack and finish.

Both dispatch-scoped pipelines see the DispatchContext — everything the TurnContext exposes plus the dispatch primitives.

dispatchInputPipeline

Dispatch-scoped. Fires once per iteration, before the executor. Context: DispatchContext — inherits the turn collections, adds the dispatch-only primitives listed below.

The dispatch loop has started. On iteration 0 the collections look as turnInputPipeline left them. On iteration 1+ the previous iteration's executor has appended a new Message and any new ToolCall records (results land only if the executor or a dispatchOutputPipeline middleware ran the tool — turnOutputPipeline has not run yet). Shape what changes before the next executor call.

• Bound the loop. if (ctx.iteration >= 10) ctx.nack(new Error('iteration cap')). The runner imposes no default cap. Skip this and your dispatch can run forever — the runner will not stop it, and the model will not volunteer.
• Reshape on retry. If the previous iteration is worth retrying, dispatchOutputPipeline leaves the dispatch unsignalled (no ack, no nack) and the runner loops; this pipeline can inject a hint into ctx.turnMessages before the executor sees it again. nack is terminal — retry is "don't signal yet"; nack is "give up."
• Per-iteration policy keyed on accumulated state — quota counters in ctx.stash, rate-limiting at dispatch granularity.

A throw wraps as E_DISPATCH_PIPELINE_ERROR; the dispatch nacks.

Between this pipeline and the next: the executor runs

The runner calls the DispatchExecutorFn you registered. It streams deltas through helpers.report*, populates records via ctx.store*, and either signals ctx.ack() / ctx.nack(error) itself or leaves the decision to dispatchOutputPipeline. The executor is not a middleware — it is the work the middlewares sandwich.

dispatchOutputPipeline

Dispatch-scoped. Fires once per iteration, after the executor. Same DispatchContext.

The executor returned. Its persistence calls have already mutated the context — every ctx.storeMessage / ctx.storeToolCall / ctx.storeThought the executor made during the iteration has already landed by the time this pipeline runs. What it produced is now in the dispatch collections; whether the loop continues is still up for grabs. This pipeline decides — and because persistence is already done, the work here is to inspect, mutate, or delete records that already exist, not to "postprocess output before it persists."

• Call ctx.ack() when done. A common pattern: a middleware that detects "no tool calls this iteration" and signals completion on the executor's behalf.
• Detect tool-call loops. if (ctx.toolCallCount(checksum) >= 3) ctx.nack(...). The model proposing the same call three times in a row is its way of telling you it is stuck. The third attempt is not the one that will work.
• Mutate or delete already-persisted records before the loop continues — refusal filtering, format normalisation, redaction. Use the matching ctx.mutate* / ctx.delete* callbacks; the record exists in the collection by the time you read it.
• Register onAck cleanup with DispatchContext.onAck. Does not fire on nack — see Signalling: ctx.iteration, ctx.toolCallCount, ctx.onAck.

A throw wraps as E_DISPATCH_PIPELINE_ERROR; the dispatch nacks. (Same code as dispatchInputPipeline — the runner does not split input vs. output error classes at this layer.)

Three exits: neither signal → loop continues with ctx.iteration incremented; ack → turnOutputPipeline runs; nack or abort → turnOutputPipeline is skipped (turnEnd still fires either way). The only path to turnOutputPipeline is ack. See Signalling for the full terminal-state semantics.

Dispatch-only primitives on DispatchContext

Middlewares in the two dispatch pipelines see everything TurnContext exposes plus the dispatch-loop primitives: DispatchContext.iteration, DispatchContext.toolCallCount, DispatchContext.ack, DispatchContext.nack, DispatchContext.onAck. Signalling and bounds covers ack/nack/aborted in depth, the iteration-boundary rule for early vs. late signals, and the rule that signalling is not silently idempotent.

Where to go next

• Turn-scoped pipelines — the once-per-turn sibling pipelines.
• LLM Dispatch, Signalling, Tools.