The Agent, In Full
This is not a walkthrough. Punching Above Its Weights already did the walkthrough — the planner, the gate cascade, the subtractive pass, the recovery ladder, all named and quoted in place. This page is the 31 source files those excerpts were pulled from: the complete, unredacted source of the agent running live on this site (the floating Ask ADK launcher), viewable end to end and fetchable whole by whatever coding agent you point at it. No excerpts, no "left as an exercise," no synthetic cleanup pass before publishing. What ran on the wire is what's below.
The 31 files split into two groups, and the split tracks one question: did we write this, or did we import it from ADK? The split is deliberate and it is not decoration. The agent is the scaffolding this showcase is actually about — code this project wrote on top of ADK, not shipped by it: the planner that commits to a turn's shape before generation, the gate cascade that owns turn completion, the subtractive context pass, the tool registry, the RAG and keyword-search retrieval, the classifiers and prose heuristics that back the gates, the citation and artifact-handle machinery, and the system prompt that ties it together. Every file in this group is portable — read it as reference for building the equivalent scaffolding around your own agent, on your own model, in your own runtime. None of it depends on this repo being a VitePress docs site.
The deployment glue is the other half — the wiring that made this specific deployment run, not scaffolding meant to be ported: the LiteRT-LM Web Worker and its proxy, the GPU-loss recovery policy, the LiteRT battery adapter wiring, the SQLite/Kysely persistence stack (the worker, the dialect, the swarm store, the store facade), the embedder, and the small state/route plumbing that hosts all of it inside a docs page. This is the part that is legitimately specific to "small model, in a browser tab, on WebGPU" — read it if you're deploying the same way, skim it if you're not.
Four files are excluded on purpose, and it's worth saying why rather than just quietly dropping them. agent_adk.ts is a thin app-side binding over @nhtio/adk/shims's already-public API — there's nothing to learn from it that the shims documentation and createAdkShim TSDoc don't already teach better, and consumers should build against the shipped module, not against this showcase's private call site. agent_wiretap.ts is a dev-only observability tap that doesn't exist in the released build. agent_dialog.vue and agent_dialog_parts.ts are UI chrome — the chat window rendering this agent's output — which is not what this page is for.
A porting note: the source below is MIT-licensed with the rest of this repository, so copying it wholesale is fine. What it depends on is not this showcase — it's the shipped API surface: the Context Batteries (thrift, the subtractive pass every dispatch here runs) and @nhtio/adk/shims (the async-resolver seam, documented in full on Runtime Loading) are both public, versioned, and exist independently of this agent. Build against those, not against this page's private wiring.
The agent
The deployment glue
// #region four_pipelines
// The flagship agent harness — the orchestration core, built on the ADK TurnRunner.
//
// WHY TurnRunner (not DispatchRunner directly): the ADK splits work across FOUR pipelines for
// a cost reason (see the LLM-Dispatch / "what each pipeline owns" docs). TurnRunner is the seam
// that owns all four; DispatchRunner only exposes the per-iteration two. The flagship needs the
// turn-scoped seam, so it uses TurnRunner:
//
// - turnInputPipeline (once / turn) → hydrate the TURN CONSTANTS: standing instructions.
// (Tools are turn-constant too, supplied via config.)
// - dispatchInputPipeline (per iteration, before the model) → hydrate the CONVERSATIONAL
// state the loop reasons over (messages — re-drained
// each iteration so a MID-TURN interrupt folds in —
// plus memories, thoughts, tool-calls, and the
// pre-fetched retrievables), then the subtractive pass
// and the force-answer guard.
// - dispatchOutputPipeline (per iteration, after the model) → loop telemetry (which tool ran).
// - persistence callbacks → the SQLite facade. The transformers.js adapter is a well-behaved
// executor: it calls ctx.storeMessage / storeThought /
// storeToolCall and ctx.ack() itself, so those
// callbacks ARE the persistence wiring — no bespoke
// store loop.
//
// LOOP CONTROL is by SIGNALS, using the ADK primitives (ctx.iteration, ctx.toolCallCount) — the
// policy thresholds are ours. We force an answer only on a GENUINE repeat (same tool+args ⇒ same
// checksum, REPEAT_LIMIT+ times) or the hard iteration cap — never on "any tool was used", so a
// healthy multi-step turn (search → catalog → call_a_tool → answer) runs freely.
//
// CHAT HISTORY vs CONTEXT WINDOW: the user turn is persisted in run(); the adapter persists the
// final assistant turn via the storeMessage callback. The model-facing context (turnMessages and
// the rest) is rebuilt each turn and never rendered. See memory chat_history_vs_context_window.
// #endregion four_pipelines
import { pendingGate } from './agent_state'
import { GpuLossPolicy } from './gpu_loss_policy'
import { requestIsOverScope } from './agent_citations'
import { searchSemantic, type DocHit } from './agent_rag'
import { buildSystemPrompt } from './agent_system_prompt'
import { MODELS, type ModelChoice } from './agent_models'
import { makeDispatchOutput } from './agent_gate_cascade'
import { makeDispatchInput } from './agent_context_assembly'
import { currentPath, navigateInternal } from './agent_route'
import { makePlan, renderPlanThought } from './agent_planner'
import { makeTurnCallbacks } from './agent_persistence_callbacks'
import { type ThriftTrace, type WorkingImage } from './agent_subtractive_pass'
import { createWorkerEngine, asWorkerEngineProxy } from './litert_lm_worker_proxy'
import { readModelFacingPreview, readToolResultPreview } from './agent_artifact_handles'
import { makeEmitCheckerThought, makeEmitThought, type TurnState } from './agent_turn_state'
import {
makeDefaultSpoolStore,
makeLiteRtAdapter,
ensureEncoderWiring,
} from './agent_litert_adapter'
import {
isContextOverflowError,
isGpuOomError,
isWindowTooSmallError,
unwrapToRootCause,
} from './agent_errors'
import {
buildToolRegistry,
type CapturedAnswer,
type PlanAnswerKind,
type PlanAnswerScope,
} from './agent_tools'
import {
looksLikeDirectiveAcknowledgment,
looksLikeRawDocSource,
stripFalseAbstentionPreamble,
} from './agent_prose_heuristics'
import {
contentTokens,
assembleCompactedTurns,
COMPACTION_SYSTEM_PROMPT,
type ThriftHistoryTurnType,
type CompactSummarizeFnType,
} from './agent_adk'
import {
classifyIsAbstention,
classifyIsDocAnswer,
classifyNeedsToolVerification,
classifyPlanViolation,
cleanInlinePathsIfAny,
} from './agent_classifiers'
import {
type AdapterFactory,
type AgentEvents,
type AgentGateKind,
type AgentThoughtKind,
type HarnessAdapter,
type RunInput,
type RunResult,
} from './agent_harness_types'
import {
storeMessage,
fetchMessages,
ensureDefaultConversation,
DEFAULT_CONVERSATION_ID,
storeMedia,
storeThought,
type ConversationMessage,
type ToolCallRecord,
} from './agent_store_facade'
import {
TurnRunner,
DispatchRunner,
Message,
Tokenizable,
Media,
inMemoryMediaReader,
isError,
probeGpuBudget,
type TurnContext,
type TurnRunnerConfig,
type TurnPipelineMiddlewareFn,
type DispatchExecutorFn,
} from './agent_adk'
import {
CAPACITY_MIN_CONTENT_WORDS,
CAPACITY_RAG_SCORE_FLOOR,
CHECKER_THOUGHT_ID,
CONTEXT_OVERFLOW_RETRY_FLOOR,
DYNAMIC_OUTPUT_BUDGET_ENABLED,
IMAGE_TOKEN_COST,
LITERT_ENGINE_MAX_TOKENS,
MIN_ANSWER_TOKENS,
RAG_TOPK,
SCOPE_OUTPUT_FRACTION,
} from './agent_turn_policy'
export { DEFAULT_MODEL, listModels, modelSupportsThinking } from './agent_models'
export type { ModelChoice, ModelConfig } from './agent_models'
export type {
AdapterFactory,
AgentEvents,
AgentGateKind,
AgentPhase,
AgentThoughtKind,
HarnessAdapter,
RunInput,
RunResult,
} from './agent_harness_types'
export class AgentHarness {
#adapter: HarnessAdapter | null = null
#model: ModelChoice
readonly #events: AgentEvents
// Injected LLM-adapter factory (Node harness supplies an Ollama-backed one); undefined → built-in LiteRt.
readonly #adapterFactory: AdapterFactory | null
// Injected spool store (Node harness supplies an in-memory one); undefined → OpfsSpoolStore (browser).
readonly #spoolStoreFactory: (() => unknown) | null
// Sliding-window device-loss escalation decider for the worker-hosted engine (1st loss → respawn the
// worker; 2nd inside the ~2-min window → page reload). Wired ONCE into the injected createEngine factory
// and consulted on every ev:deviceLost from the worker. See gpu_loss_policy.ts.
readonly #gpuLossPolicy = new GpuLossPolicy()
// Re-entrancy guard: a device loss can fire while another recovery is already respawning the worker.
// Prevents stacking dispose+preload cycles on the same loss burst.
#respawningAfterLoss = false
// The engine ceiling is large; the slider drives the thrift budget below it.
#contextWindow = 8192
// Output ceiling (the max-output slider). Default within the window's output reserve. The dialog
// clamps it to [enough-to-call-a-tool, contextWindow/4] before setting it.
#maxTokens = 2048
// Whether to GENERATE the model's own reasoning/thinking channel. Default OFF (thinking burns the tool
// loop). Controls GENERATION only, NOT display — the synthetic plan + nudge thoughts emit via onThought
// regardless and are always shown; this only adds the model's own CoT to that stream. Applies on the
// next adapter build (the dialog requires unload→load to change models/settings anyway).
#enableThinking = false
// OPT-IN: skip injecting the harness's synthetic THIS-turn thoughts (plan / nudge / cite / checker) into
// the MODEL prompt (ctx.turnThoughts). Default false = inject them (Gemma relies on them as its own
// reasoning-continuation, the book-end plan discipline). Set true for models whose chat template treats a
// TRAILING ASSISTANT message as a completed turn and emits an immediate stop — synthetic thoughts render as
// assistant messages and, being generated in response to the user turn, land LAST, so the prompt ends
// assistant. Gemma/deepseek continue past that; qwen3-coder-next stops (observed: eval_count:1, empty
// content, 300+ dispatch spin). This flag keeps them OUT of the prompt (they STILL emit to the UI via
// onThought — display is unaffected) so the prompt ends on the user/tool turn the model expects. The plan
// still governs tool routing + gates; only its in-prompt THOUGHT surface is suppressed.
#injectSyntheticThoughts = true
// CONTEXT-MANAGEMENT STRATEGY — the head-to-head experiment knob (confirmation-bias check). Which
// cross-turn history-assembly + per-dispatch fit strategy this run uses. Default 'thrift' = the real
// system (relevance-select ALL history via selectRelevantTurns + subtractive pass). The two BASELINES are
// swapped in ONLY for the comparison run, same model/corpus/window — the thrift claim is model-agnostic
// and tested head-to-head:
// 'compact' — faithful Claude Code auto-compaction: keep the most-recent turns verbatim + a running
// structured summary of everything older, re-summarised by a SAME-MODEL dispatch when the
// kept history approaches the window. Lossy-but-bounded; the realistic competitor.
// 'naive' — the floor: keep the most-recent turns that fit, FIFO-drop the oldest, no relevance/summary.
// Only the history-selection funnel (getSelectedTurns) branches on this; the per-dispatch subtractive pass
// still runs for ALL strategies (so a baseline can't "silently overflow" — it just loses DETAIL, which is
// exactly what the quality judge measures). Set via setContextStrategy (env ADK_CONTEXT_STRATEGY).
#contextStrategy: 'thrift' | 'compact' | 'naive' = 'thrift'
// COMPACT-arm running summary of older turns + the count of older turns it already covers (so we only
// re-summarise when NEW turns have aged out past the keep-verbatim window). Reset on dispose/clear.
#compactSummary: string | null = null
#compactCoveredOlder = 0
// Auto-resolution for the navigate-permission TurnGate (ctx.waitFor in navigate_to_page). In the browser
// a human clicks allow/deny; a HEADLESS harness (the Node stress corpus) has no dialog, so an opened gate
// never resolves and the whole run DEADLOCKS on that one dispatch (observed: kimi-k2.5 reached for
// navigate_to_page as if it were a research step and wedged turn 1 forever — no socket, cputime frozen).
// When set, the turnGateOpen observer auto-answers with this verdict instead of waiting on pendingGate.
// null = browser behavior (surface to the dialog, wait for the human). false = auto-DENY (the faithful
// headless default: the tool returns "user declined navigation", the turn continues, no real navigation
// happens in a non-DOM environment anyway).
#autoGateVerdict: boolean | null = null
constructor(
model: ModelChoice,
events: AgentEvents = {},
// Optional injection seams for the portable Node harness (see tests/agent/_harness/*). Both default to the
// browser implementations, so the live app constructs `new AgentHarness(model, events)` exactly as before.
opts?: {
adapterFactory?: AdapterFactory
spoolStoreFactory?: () => unknown
// Headless auto-ack for the navigate TurnGate. Pass false (or true) in the Node harness so a model that
// calls navigate_to_page doesn't deadlock waiting on a dialog that isn't there. See #autoGateVerdict.
autoGateVerdict?: boolean
}
) {
this.#model = model
this.#events = events
this.#adapterFactory = opts?.adapterFactory ?? null
this.#spoolStoreFactory = opts?.spoolStoreFactory ?? null
this.#autoGateVerdict = opts?.autoGateVerdict ?? null
// Kick off @nhtio/encoder wiring (registers ADK encodables + resolves the OPFS root for the
// spool-reader resolver). Awaited before the first decode in fetchToolCallsCallback.
void ensureEncoderWiring()
}
get model(): ModelChoice {
return this.#model
}
setContextWindow(n: number): void {
this.#contextWindow = n
}
/** Set the output-token ceiling. Applies to the NEXT adapter build (call before preload, or it
* takes effect after the next unload→load). The dialog clamps the value before calling. */
setMaxTokens(n: number): void {
this.#maxTokens = n
}
/** The ACTUAL output-token cap to apply for a turn, given the planner's committed answer_scope. The
* user's slider (`#maxTokens`) is the hard ceiling; `scope` tightens BELOW it (never above) so a
* `brief` answer is asked to finish inside a small budget. Undefined scope (no plan, or a non-answer
* dispatch) returns the full ceiling — today's behavior, the safe default. Floored at
* MIN_ANSWER_TOKENS so a too-low slider can't starve the answer below "enough to finish". */
#resolveOutputTokens(scope?: PlanAnswerScope): number {
// Flag off → the lever is inert (full ceiling), so both the stash write and the scoped plan-thought
// words collapse to pre-change behavior. Lets the incomplete-answer gate be evaluated on its own.
if (!DYNAMIC_OUTPUT_BUDGET_ENABLED || !scope) return this.#maxTokens
const scaled = Math.round(this.#maxTokens * SCOPE_OUTPUT_FRACTION[scope])
return Math.max(Math.min(scaled, this.#maxTokens), Math.min(MIN_ANSWER_TOKENS, this.#maxTokens))
}
/** Output budget expressed as an approximate word count, for telling the planner/worker how long the
* answer may be before the max-output cap truncates it. ~0.75 English words per token (conservative),
* floored and rounded to a friendly figure so it reads as a soft target. Single source of truth so
* the planner prompt and the plan-thought agree. Pass the turn's `scope` so the words target reflects
* the SCOPED cap actually applied (see {@link #resolveOutputTokens}); omit it for the ceiling (the
* planner prompt, which describes the max BEFORE a scope is chosen). */
#budgetWords(scope?: PlanAnswerScope): number {
return Math.max(40, Math.round((this.#resolveOutputTokens(scope) * 0.75) / 10) * 10)
}
/** Turn the model's reasoning/thinking channel on/off. Applies on the next adapter build. */
setEnableThinking(on: boolean): void {
this.#enableThinking = on
}
/**
* Opt out of injecting the harness's synthetic this-turn thoughts (plan/nudge/cite/checker) into the model
* prompt. Pass `false` for a model whose chat template stops on a trailing-assistant turn (e.g.
* qwen3-coder-next). The thoughts still emit to the UI; only their in-prompt surface is suppressed. Default
* on. Takes effect on the next turn.
*/
setInjectSyntheticThoughts(on: boolean): void {
this.#injectSyntheticThoughts = on
}
/**
* Set the context-management strategy for the head-to-head comparison (see #contextStrategy).
* 'thrift' (default, the real system) | 'compact' (faithful Claude Code auto-compaction baseline) |
* 'naive' (FIFO-drop floor). Only affects cross-turn history assembly; the per-dispatch subtractive pass
* always runs. Takes effect on the next turn.
*/
setContextStrategy(strategy: 'thrift' | 'compact' | 'naive'): void {
this.#contextStrategy = strategy
}
/** Swap the model — disposes the old adapter (frees ONNX/WebGPU) and forces a reload. */
async setModel(model: ModelChoice): Promise<void> {
if (model === this.#model && this.#adapter) return
await this.dispose()
this.#model = model
}
#ensureAdapter(): HarnessAdapter {
if (!this.#adapter) {
// Injected factory path (Node harness → Ollama battery). Bypasses the LiteRt-specific config below.
if (this.#adapterFactory) {
this.#adapter = this.#adapterFactory(this.#model, {
maxTokens: this.#maxTokens,
enableThinking: this.#enableThinking,
contextWindowMax: LITERT_ENGINE_MAX_TOKENS,
contextWindow: this.#contextWindow,
})
return this.#adapter
}
const cfg = MODELS[this.#model]
this.#adapter = makeLiteRtAdapter(
cfg,
{
model: cfg.model,
createEngine: createWorkerEngine({
onDeviceLost: ({ reason }) => {
void this.#onEngineDeviceLost(reason)
},
onProgress: () => {
this.#events.onLoadProgress?.({ file: '', pct: null })
},
}),
enableThinking: this.#enableThinking,
maxTokens: this.#maxTokens,
maxNumTokens: LITERT_ENGINE_MAX_TOKENS,
contextWindow: Math.max(2048, this.#contextWindow - this.#maxTokens),
} as never,
this.#events,
() => (this.#spoolStoreFactory?.() ?? makeDefaultSpoolStore()) as never
)
}
return this.#adapter
}
/** Preload the model (the ~2GB cold load) without running a turn. RAG is NOT seeded here —
* it is lazy (only when search runs / pre-fetch fires), so model-load progress is the only
* thing the user waits on. */
async preload(): Promise<void> {
this.#events.onPhase?.('loading')
const adapter = this.#ensureAdapter()
// The WebGPU budget is DEVICE-level (not model-level) — known the moment WebGPU is available, so
// probe it now and surface it immediately, rather than waiting for the first generate's `ready`
// lifecycle. Lets the user size the context window against real hardware before sending a turn.
try {
const b = await probeGpuBudget()
if (b.available) {
this.#events.onGpuBudget?.({
maxBufferMB: Math.round(b.maxBufferBytes / (1024 * 1024)),
available: true,
})
}
} catch {
/* budget is observability-only; never block load on it */
}
// ACTUALLY load the engine here (the ~2GB fetch + WebGPU compile). #ensureAdapter only CONSTRUCTS the
// adapter; the heavy load is otherwise deferred to the first generate — which made "Load" return
// instantly and the loading banner flash away, then the real wait happened (mislabelled "Thinking…")
// on the first turn. Awaiting adapter.preload() keeps phase='loading' for the whole download/compile,
// so the banner reflects reality. The onInitProgress hook (wired in #ensureAdapter) drives the bar.
try {
// Ollama (Node harness) has no cold-load, so its adapter omits preload() — skip when absent.
await adapter.preload?.()
} finally {
this.#events.onPhase?.('idle')
}
}
async dispose(): Promise<void> {
const a = this.#adapter as unknown as {
dispose?: () => Promise<void> | void
} | null
if (a?.dispose) {
try {
await a.dispose()
} catch {
/* best-effort */
}
}
this.#adapter = null
}
/**
* Free the WebGPU device memory the loaded engine retains, WITHOUT switching models. Delegates to the
* adapter's `recycle()` (`dispose()` then `preload()` the same model).
*
* @remarks
* With the disposable-Worker engine injected (browser path), this is now a TOTAL teardown: the adapter's
* `dispose()` → the worker-proxy engine's `delete()` → `worker.terminate()` (destroys the whole WebGPU
* context + WASM heap + every dead ref in one OS-level op), and `preload()` spawns a FRESH worker that
* calls requestAdapter() from a clean context. This is what lets recycling actually recover from a
* WebGPU device loss — a main-thread `dispose()` only nulls JS refs and reuses the dead GPUAdapter. It is
* also the consumer-facing lever surfaced after an OOM (reclaim the retained working-set, then Retry on a
* smaller window). Re-incurs the cold load (download cached; WebGPU graph/shader compile is not), so we
* drive the load phase.
*/
async recycle(): Promise<void> {
const a = this.#adapter as unknown as {
recycle?: () => Promise<void>
} | null
if (!a?.recycle) {
// No adapter loaded yet — nothing retained to free.
return
}
this.#events.onPhase?.('loading')
this.#events.onActivity?.('Freeing GPU memory…')
try {
await a.recycle()
} finally {
this.#events.onActivity?.('')
this.#events.onPhase?.('idle')
}
}
/**
* Handle a WebGPU device loss reported by the worker-hosted engine (driver TDR / GPU-process crash). The
* engine is dead — no in-flight or future turn can complete on it. Runs the escalation ladder:
*
* 1st loss in the ~2-min window → RESPAWN: terminate the dead worker + spawn a fresh one (adapter
* `recycle()`), so a transient TDR recovers with a clean GPU context. History is untouched (SQLite).
* 2nd loss inside the window → RELOAD: emit `onDeviceLost({action:'reload'})` and let the dialog do a
* `window.location.reload()`. Chrome blocks new GPU adapters after 2 GPU-process crashes in ~2 min;
* only a reload clears it. The reload is lossless — the conversation lives in SQLite.
*
* Re-entrancy-guarded so a burst of loss events on one crash doesn't stack recovery cycles.
*/
async #onEngineDeviceLost(reason: string): Promise<void> {
const action = this.#gpuLossPolicy.record()
// Always tell the app first (transient "recovering" banner for respawn; reload trigger for reload).
this.#events.onDeviceLost?.({ action, reason })
if (action === 'reload') {
// The dialog owns the actual reload (it can show a banner first). Nothing more to do here — do NOT
// respawn, the reload supersedes it.
return
}
if (this.#respawningAfterLoss) return
this.#respawningAfterLoss = true
this.#events.onPhase?.('loading')
this.#events.onActivity?.('Recovering the GPU engine…')
try {
await this.recycle()
} catch {
// A respawn that itself fails escalates the same way a repeat loss would — force a reload.
this.#events.onDeviceLost?.({ action: 'reload', reason })
} finally {
this.#respawningAfterLoss = false
this.#events.onActivity?.('')
this.#events.onPhase?.('idle')
}
}
/**
* DEV/TEST-ONLY: simulate a WebGPU device loss on the worker-hosted engine to exercise the
* respawn→reload escalation ladder end-to-end (the worker destroys its sentinel device, which fires the
* same `ev:deviceLost` path a real driver TDR takes). Reaches the worker proxy via the adapter's already-
* resolved engine. No-op when no worker engine is loaded (Node/Ollama path, or before preload). Returns
* true when the force was dispatched. Gated by the dialog behind `__ADK_WIRETAP__` so it never ships.
*/
async forceDeviceLostForTest(): Promise<boolean> {
const a = this.#adapter as unknown as {
preload?: () => Promise<unknown>
} | null
if (!a?.preload) return false
let engine: unknown
try {
engine = await a.preload()
} catch {
return false
}
const proxy = asWorkerEngineProxy(engine)
if (!proxy) return false
await proxy.forceDeviceLost()
return true
}
/**
* COMPACT-arm summariser (head-to-head baseline only; see #contextStrategy === 'compact'). The shipped
* `@nhtio/adk/batteries/context/compact` battery owns the summarise/assemble ALGORITHM (keep-verbatim
* window, summarise-at-tokens threshold, request assembly, the faithful Claude Code compaction prompt) —
* this is now just the injected {@link CompactSummarizeFnType} model-call seam the battery cannot bundle
* itself. Same dispatch shape as #classifyBinary (all-noop callbacks, output captured via storeMessage),
* maxTokens ~1200, toolCallParser 'none'. A failed summariser degrades to the prior summary (or empty) —
* the baseline just keeps less, matching the original inline behavior.
*
* CRITICAL BEHAVIORAL NOTE (reported per the "consume the shipped battery" mission, not silently
* adapted): the battery's own `summariseTurns` measures `onCost`'s `inTok` over `` `${system}\n\n${text}` ``
* (system prompt INCLUDED), whereas this showcase's prior local copy measured `inTok` over the user-content
* text ALONE (no system prompt). Delegating to the battery's `onCost` as-is would inflate the recorded
* compact-arm token cost by the (fixed, ~400-token) system-prompt size on every summarisation call,
* changing the honest head-to-head token-cost comparison the demo reports. Preserved the ORIGINAL
* (narrower) measurement here by NOT using the battery's built-in `onCost` plumbing for the ring counter —
* instead this method now measures `inTok`/`outTok` itself, the same way the pre-refactor code did, and
* pushes to `__agentCompactionCost` directly. The battery's `summariseTurns` char-cap
* (`HISTORY_TEXT_CHAR_CAP`) also differs subtly in SCOPE from the pre-refactor local slice (see this
* method's caller, {@link CompactAssembleCompactedTurnsOptionsType}'s wiring below, for the corresponding
* note) — both are flagged in the final report as CRITICAL drift, not silently reconciled beyond what's
* done here.
*/
#buildCompactSummarizer(): CompactSummarizeFnType {
return async ({ system, text }: { system: string; text: string }): Promise<string> => {
const adapter = this.#ensureAdapter()
let out = ''
const noop = async (): Promise<void> => undefined
const noopArr = async (): Promise<never[]> => []
try {
await DispatchRunner.dispatch({
raw: {
systemPrompt: new Tokenizable(system),
standingInstructions: [],
messages: [
new Message({
id: `__compact-${Date.now()}`,
role: 'user',
content: text,
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
}),
],
fetchMemories: noopArr,
fetchRetrievables: noopArr,
fetchMessages: noopArr,
fetchThoughts: noopArr,
fetchToolCalls: noopArr,
fetchTools: noopArr,
refreshStandingInstructions: noopArr,
storeStandingInstruction: noop,
mutateStandingInstruction: noop,
deleteStandingInstruction: noop,
storeMemory: noop,
mutateMemory: noop,
deleteMemory: noop,
storeRetrievable: noop,
mutateRetrievable: noop,
deleteRetrievable: noop,
storeMessage: async (_c: unknown, v: { content?: { toString(): string } }) => {
out += v.content?.toString?.() ?? ''
},
mutateMessage: noop,
deleteMessage: noop,
storeThought: noop,
mutateThought: noop,
deleteThought: noop,
storeToolCall: noop,
mutateToolCall: noop,
deleteToolCall: noop,
storeMediaBytes: noop,
storeRetrievableBytes: noop,
} as never,
executor: await adapter.executor({
maxTokens: 1200,
toolCallParser: 'none',
stream: false,
autoAck: true,
enableThinking: false,
}),
})
} catch (err) {
if (isGpuOomError(err)) throw err
// A failed summariser degrades to empty text — assembleCompactedTurns's own `|| priorSummary`
// fallback (mirroring the pre-refactor inline degrade-to-prior-summary behavior) takes it from here.
return ''
}
// Record the summariser token cost (input estimate + output) for the token-cost comparison — this is
// the overhead the compact baseline pays and thrift does not. Measured over `text` ALONE (no system
// prompt), matching the pre-refactor local behavior exactly (see this method's TSDoc, CRITICAL note).
const ring = ((
globalThis as unknown as {
__agentCompactionCost?: Array<{ inTok: number; outTok: number }>
}
).__agentCompactionCost ??= [])
const summary = out.trim()
ring.push({
inTok: Tokenizable.estimateTokens(text, 'gemma'),
outTok: Tokenizable.estimateTokens(summary, 'gemma'),
})
return summary
}
}
/**
* COMPACT-arm history assembler (baseline; #contextStrategy === 'compact'). Delegates the keep-verbatim /
* summarise-at-tokens / rolling-summary algorithm to the shipped
* `@nhtio/adk/batteries/context/compact` battery's `assembleCompactedTurns` (via the agent_adk.ts shim),
* threading `#compactSummary`/`#compactCoveredOlder` through as `priorState` in and writing the returned
* `{ summary, coveredOlder }` back — the battery is stateless between calls by design (see its own TSDoc),
* so this class's private fields are now the caller-side persistence the battery expects.
*
* CRITICAL BEHAVIORAL NOTE (char-cap scope — reported, not silently adapted): the battery's
* `summariseTurns` slices `` (priorSummaryPrefix + historyText) `` TOGETHER at a combined 12,000-char cap,
* whereas the pre-refactor local `#summariseTurns` — due to operator precedence in
* `` `CONVERSATION TO SUMMARISE:\n${historyText}`.slice(0, 12000) `` — capped ONLY the historyText
* template literal, leaving a prepended `priorSummary` UNBOUNDED. Delegating to the battery therefore
* changes the cap's scope: after several rounds of growth, `priorSummary` could now consume some/most of
* the 12k budget that previously went entirely to fresh historyText. Not reconciled here (doing so would
* require bypassing the battery's own internal request assembly, defeating the "consume the shipped API"
* goal) — flagged as CRITICAL drift in the final report per the mission's explicit instruction.
*/
async #assembleCompactedTurns(
turns: ReadonlyArray<ThriftHistoryTurnType<ConversationMessage, ToolCallRecord>>
): Promise<Array<ThriftHistoryTurnType<ConversationMessage, ToolCallRecord>>> {
// keepVerbatim/summariseAtTokens match the battery's own defaults (DEFAULT_KEEP_VERBATIM=2,
// DEFAULT_SUMMARISE_AT_TOKENS=2500) — confirmed byte-identical to the pre-refactor local constants
// they replace; systemPrompt is passed explicitly (even though it also matches the battery default)
// for self-documentation, mirroring agent_subtractive_pass.ts's convention.
const result = await assembleCompactedTurns<ConversationMessage, ToolCallRecord>(turns, {
summarize: this.#buildCompactSummarizer(),
estimateTokens: (v, enc, ctx) => Tokenizable.estimateTokens(v, enc, ctx as never),
encoding: 'gemma',
systemPrompt: COMPACTION_SYSTEM_PROMPT,
priorState: { summary: this.#compactSummary, coveredOlder: this.#compactCoveredOlder },
})
this.#compactSummary = result.summary
this.#compactCoveredOlder = result.coveredOlder
return result.turns
}
/**
* Surface a context overflow through the SAME observability channels the other rails use: an onGate
* `context-overflow` event (so it lands in the instrumentation / "what happened" stream alongside the
* behavioral gates), plus onPhase('error') + onActivity(''). This is what "emit through the appropriate
* observability channels" means — the overflow is a first-class, observed event, not a swallowed
* string. Idempotent-safe to call from both the nack and error seams (the dialog dedupes by content).
*/
#emitContextOverflow(err: unknown): void {
const msg = isError(err) ? err.message : String(err)
this.#events.onGate?.({ gate: 'context-overflow', detail: msg })
this.#events.onPhase?.('error')
this.#events.onActivity?.('')
}
/**
* Run one turn, with a bounded ONE-SHOT context-overflow recovery. Defense-in-depth layer 3 (after
* thrift + the typed guard): if a dispatch still overflows the engine cap, tighten this turn's effective
* context window a step and re-run the SAME turn ONCE so the subtractive pass sheds harder. If it
* overflows again, surface the typed banner (layer 4). Never loops — a single retry, then give up
* cleanly. Any non-overflow outcome (success, OOM, refusal) returns immediately without a retry.
*/
async run(input: RunInput): Promise<RunResult> {
const first = await this.#runTurnOnce(input)
// Two recovery signals share ONE machinery — shrink the effective window a step so the subtractive pass
// sheds harder, then re-run the SAME turn:
// • contextOverflow — the prompt exceeded the engine cap (one-shot: shed once, then surface the banner).
// • contextPoisoned — the model surfaced an internal trust-tier fence twice this turn; its context is
// poisoned by accumulated thrash. Per the user's spec, ESCALATE: shed and retry PROGRESSIVELY until
// the model answers clean (result no longer poisoned) or we hit the window floor — at which point
// #runTurnOnce's own LEAK_GATE_LIMIT/backstop ships a clean gracefulFallback, never the raw fence.
if (!first.contextOverflow && !first.contextPoisoned) return first
const savedWindow = this.#contextWindow
try {
let result = first
// Bounded by the floor: 8192 → 6144 → 4608 → 3456 → 2592 → 2048 is a handful of steps, so this cannot
// spin. Overflow only ever takes the first step (its own recovery is one-shot by contract); poison
// keeps stepping while it remains poisoned.
while (result.contextOverflow || result.contextPoisoned) {
const tightened = Math.max(
CONTEXT_OVERFLOW_RETRY_FLOOR,
Math.floor(this.#contextWindow * 0.75)
)
const atFloor = tightened >= this.#contextWindow
if (atFloor) {
// Can't shed any further. Overflow → surface the too-long banner (return as-is). Poison → do ONE
// final attempt that FORCES a clean fallback on any further leak, so the turn resolves with a real
// reply instead of returning an empty poisoned result.
if (result.contextOverflow) return result
this.#events.onActivity?.('Refocusing — shedding and retrying…')
return await this.#runTurnOnce(input, {
isOverflowRetry: true,
forceFallbackOnLeak: true,
})
}
this.#events.onActivity?.(
result.contextOverflow
? 'Context too long — shedding and retrying…'
: 'Refocusing — shedding and retrying…'
)
this.setContextWindow(tightened)
const wasOverflow = result.contextOverflow === true
result = await this.#runTurnOnce(input, { isOverflowRetry: true })
// Overflow recovery is one-shot (existing contract): after one shed, return whatever we got so the
// too-long banner can surface if it overflowed again. Poison recovery keeps escalating.
if (wasOverflow && !result.contextPoisoned) return result
}
return result
} finally {
this.setContextWindow(savedWindow)
}
}
/**
* Run one turn end-to-end through a TurnRunner. Persists the user message first (so the
* turn's history fetch includes it), wires hydration + loop control into the right pipelines,
* streams the answer, and lets the adapter persist the final assistant turn via the callback.
*/
async #runTurnOnce(
input: RunInput,
opts?: { isOverflowRetry?: boolean; forceFallbackOnLeak?: boolean }
): Promise<RunResult> {
// Immediate sign of life: a cold turn includes the ~2GB load — surface it BEFORE any await.
this.#events.onPhase?.(this.#adapter ? 'generating' : 'loading')
this.#events.onActivity?.(this.#adapter ? 'Thinking…' : 'Loading the model…')
const convId = await ensureDefaultConversation()
const stamp = Date.now()
// The dialog persists the user message before the (slow) turn so the human sees it instantly,
// then passes its id here. When it does, reuse that id and DON'T re-store (no duplicate row).
// When it doesn't (programmatic caller), persist the user message ourselves.
const userMessageId = input.userMessageId ?? `u-${stamp}`
const mediaId = `mm-${stamp}`
// On an OVERFLOW RETRY the user message + media are ALREADY persisted from the first attempt — the
// retry only re-runs the model with a tighter window, so skip re-persisting (a fresh Date.now()
// mediaId would otherwise duplicate the media row, and the message row is upserted by id but there's
// no reason to touch it). The retry reuses the same conversation so fetchMessages still sees them.
const isOverflowRetry = opts?.isOverflowRetry === true
// Set by run() ONLY on the final poison retry at the window floor: if the model STILL leaks a fence,
// commit a clean gracefulFallback instead of raising contextPoisoned again (which would return an empty
// answer). Guarantees the escalation always terminates with a real, clean reply — never the plumbing,
// never blank.
const forceFallbackOnLeak = opts?.forceFallbackOnLeak === true
if (!input.userMessageId && !isOverflowRetry) {
// Persist the user message NOW (HUMAN plane) so the turn's fetchMessages includes it. Use a
// stable id so an uploaded image can be linked to this exact message row.
await storeMessage(convId, {
id: userMessageId,
role: 'user',
content: input.text,
})
}
// Persist uploaded media bytes durably (survives reload; rendered in history). This is the
// user-upload path — the adapter's storeMediaBytes callback covers TOOL-generated media.
if (input.image && !isOverflowRetry) {
await storeMedia({
id: mediaId,
conversationId: convId,
messageId: userMessageId,
kind: 'image',
mimeType: input.image.mimeType,
filename: 'attachment.png',
origin: 'user-upload',
bytes: input.image.bytes,
})
}
const adapter = this.#ensureAdapter()
const path = currentPath()
// BASELINE PRE-RETRIEVAL — run ONCE, memoized, and reused by BOTH the planner (which counts the
// high-confidence hits to decide capacity_scoped) and fetchRetrievablesCallback (which seeds the turn's
// retrievables). It depends only on the user text + current path, so it's safe to run before the
// planner. Memoized so the turn never double-searches. Fail-open to [] (retrieval must never block a
// turn). See RAG_TOPK / CAPACITY_RAG_* for the breadth signal.
let prefetchedHitsPromise: Promise<DocHit[]> | null = null
const getPrefetchedHits = (): Promise<DocHit[]> => {
if (!prefetchedHitsPromise) {
prefetchedHitsPromise = searchSemantic(input.text, {
topK: RAG_TOPK,
currentPath: path,
}).catch(() => [] as DocHit[])
}
return prefetchedHitsPromise
}
// The image working-set entry (token weight, for the subtractive pass) + the image-bearing
// user Message (the multimodal carrier the adapter decodes). On-thesis: the image is decoded
// for the MODEL only on its upload turn; the subtractive pass may still shed it (media is the
// biggest hog). Past images stay persisted + rendered for the human, not re-fed to the model.
const image: WorkingImage | undefined = input.image
? { label: 'attached image', tokenCost: IMAGE_TOKEN_COST }
: undefined
const imageMessage: Message | null = input.image
? new Message({
id: userMessageId,
role: 'user',
content: input.text,
attachments: [
Media.userAttachment({
id: mediaId,
kind: 'image',
mimeType: input.image.mimeType,
filename: 'attachment.png',
reader: inMemoryMediaReader(input.image.bytes),
}),
],
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
})
: null
// Turn-local mutable state shared across pipelines.
const state: TurnState = {
lastTrace: null,
refused: false,
answer: '',
lastReportedTool: null,
dispatchError: null,
dispatchErrorIsOom: false,
dispatchErrorIsOverflow: false,
dispatchErrorIsWindowTooSmall: false,
turnPoisoned: false,
sawSourcelessAnswer: false,
captured: null,
proseAnswer: '',
lastRejectedProse: '',
repeatedProseCount: 0,
noProgressProseCount: 0,
bestOwnWordsProse: '',
incompleteAnswerCount: 0,
fakeCitationCount: 0,
leakedEnvelopeCount: 0,
emptyGenCount: 0,
retrievalAbstentionCount: 0,
tooLargeHandles: new Set<string>(),
turnPlan: null,
turnThoughtRecords: new Map<string, { kind: AgentThoughtKind; text: string }>(),
activeNudges: new Map<AgentGateKind, string>(),
activeNudgeThoughts: new Map<AgentGateKind, string>(),
groundingSources: [],
selectedTurnsPromise: null,
seededIds: new Set<string>(),
priorTurnToolCallIds: new Set<string>(),
priorThoughtIds: new Set<string>(),
prevToolCallCount: 0,
}
const emitThought = makeEmitThought({ state, events: this.#events })
// A CHECKER announces — in the worker's own first-person voice — the criterion it is about to weigh
// the latest generation against, BEFORE it accepts/rejects. Per the user: "give the checker a
// synthetic thought of 'I must evaluate if this XXXX' where XXXX is the criteria". Surfaces the rails
// *deciding* in the Thinking panel (showcase), and doubles as channel-2 self-talk the 2B is likelier
// to align with than to argue against. `criteria` is the bar in plain words (e.g. "answer actually
// came from the provide_answer tool, not just inline prose"). One stable id → the panel always shows
// the most recent criterion under evaluation. Turn-scoped; never replayed into the model next turn.
const emitCheckerThought = makeEmitCheckerThought({
emitThought,
checkerThoughtId: CHECKER_THOUGHT_ID,
})
const { registry, defaultVisible } = buildToolRegistry({
currentPath: () => path,
navigateInternal,
onAnswer: (a) => {
state.captured = a
},
groundSources: () => state.groundingSources,
})
// Build the catalog blurb (name — description) the planner sees, from the full registry. Hidden
// tools are included — the planner's job is to pick the RIGHT tool, including ones the worker must
// reach via call_a_tool. This is the list that stops the planner inventing tool names.
const toolBlurbs = registry
.all()
.map((t) => `- ${t.name} — ${t.description ?? ''}`)
.join('\n')
// PLANNER BOOK-END (open). Decide how to answer up front, against the real catalog. Fail-open:
// a null plan leaves the loop unplanned (prior behaviour). The plan seeds a synthetic this-turn
// thought (iteration 0) and is validated at close. An OOM here rethrows like any generate OOM.
this.#events.onActivity?.('Planning…')
// Give the planner the IMMEDIATELY-PRIOR turn (last assistant answer + the user question that
// prompted it) so it can tell a fresh library question from a REACTION to what was just said. Without
// this, a coreferent follow-up like "so it's not much of a kit, is it?" has no signal it's a follow-up
// and misclassifies as doc_cited. The current user message is already persisted (above), so it is the
// LAST row — the prior turn is the assistant/user pair before it. Best-effort: a load failure just
// yields no prior context (the planner falls back to its default classification).
let priorContext: string | undefined
let priorAnswer: string | undefined
try {
const rows = await fetchMessages(convId)
const priorAssistantIdx = (() => {
// Walk back from the end, skipping the just-persisted current user message, to the most recent
// assistant answer.
for (let i = rows.length - 1; i >= 0; i--) {
if (rows[i].role === 'assistant') return i
}
return -1
})()
if (priorAssistantIdx >= 0) {
const prevAssistant = rows[priorAssistantIdx]
priorAnswer = prevAssistant.content
// The user question that prompted that answer = the nearest user row before it.
let prevUser: (typeof rows)[number] | undefined
for (let i = priorAssistantIdx - 1; i >= 0; i--) {
if (rows[i].role === 'user') {
prevUser = rows[i]
break
}
}
const clip = (s: string, n = 500): string => (s.length > n ? s.slice(0, n) + '…' : s)
priorContext =
'The immediately-PRIOR turn of this conversation (for context — the user may be reacting ' +
'to it):\n' +
(prevUser ? ` · The user previously asked: "${clip(prevUser.content)}"\n` : '') +
` · You previously answered: "${clip(prevAssistant.content)}"`
}
} catch {
/* best-effort — no prior context on load failure */
}
// DETERMINISTIC capacity signal: how many pre-retrieved passages are strongly on-topic. Many ⇒ the
// request spans a lot of material ⇒ capacity_scoped (won't fit). Computed from the SAME memoized
// pre-retrieval the turn reuses. Fed to the planner (drives its capacity synthetic thought) AND used
// for the post-plan override below.
const prefetchedHits = await getPrefetchedHits()
const highConfidenceHits = prefetchedHits.filter(
(h) => h.score >= CAPACITY_RAG_SCORE_FLOOR
).length
// NEVER-SHED citation set. The subtractive pass can evict ctx.turnRetrievables at high util (observed:
// an extreme capacity turn hit 100% and the RAG was shed, so the iteration backstop had nothing to
// cite → uncited). These pre-fetched hits are captured HERE, before the loop, and never shed — so the
// backstop / capacity accept / dup-stop rescue can always cite real pages even when the working-set
// retrievables are gone. Deduped by page path, highest-score first, capped for chip sanity.
const prefetchedSources: Array<{ path: string; title: string }> = (() => {
const seen = new Set<string>()
const out: Array<{ path: string; title: string; score: number }> = []
for (const h of prefetchedHits) {
const srcPath = (h.pageUrl || '').replace(/#.*$/, '')
if (!srcPath.startsWith('/') || seen.has(srcPath)) continue
seen.add(srcPath)
out.push({ path: srcPath, title: h.title || srcPath, score: h.score })
}
return out
.sort((a, b) => b.score - a.score)
.slice(0, 6)
.map((s) => ({ path: s.path, title: s.title }))
})()
// Publish the never-shed real pages to the tool-deps grounding holder, so a good-but-sourceless
// provide_answer is delivered grounded from these instead of hard-erroring (Thread A+B fix).
state.groundingSources = prefetchedSources
state.turnPlan = await makePlan(adapter, {
userText: input.text,
toolBlurbs,
priorContext,
priorAnswer,
highConfidenceHits,
budgetWords: this.#budgetWords(),
model: this.#model,
maxTokens: this.#maxTokens,
contextWindow: this.#contextWindow,
})
// POST-PLAN DETERMINISTIC OVERRIDE — the reliability guarantee. The 2B classifies a genuinely-broad
// "give me everything" request as doc_cited most of the time (docs DO answer it; it just can't be
// rendered in FULL), and neither the prompt bullet nor the synthetic thought flips it reliably. So when
// the breadth signal is unmistakable — explicit-exhaustive phrasing OR many high-confidence hits — and
// the planner still committed doc_cited, we rewrite the kind to capacity_scoped here. Safe because
// capacity_scoped is delivery-compatible with doc_cited (both search + cite via provide_answer, same
// close gates); we only change the VOICE (honest decline + pointer) and ensure a search tool is present.
if (
state.turnPlan &&
state.turnPlan.answerKind === 'doc_cited' &&
requestIsOverScope(input.text, highConfidenceHits)
) {
const tools = state.turnPlan.toolsToUse.includes('search_docs_semantic')
? state.turnPlan.toolsToUse
: [...state.turnPlan.toolsToUse, 'search_docs_semantic']
state.turnPlan = {
...state.turnPlan,
answerKind: 'capacity_scoped',
answerScope: 'brief',
toolsToUse: tools,
}
this.#events.onActivity?.('Planning…')
}
// PLANNER AS TOOL ROUTER. The planner saw the WHOLE catalog and picked the tools for this turn;
// surface those directly to the worker (on top of the always-visible defaults) so it can call them
// NATIVELY — no tool_catalog → call_a_tool browse step, which a small model cannot reliably perform
// (observed: it narrates "I reviewed the catalog" without ever issuing an artifact_* read). The
// thrift lever survives: only this shortlist is visible, not all ~60 tools. Plan names are
// validated against the registry so a hallucinated name can't widen the set. This is ADDITIVE to
// the gates — the plan-conformance / unread-handle / unverified-claim rails all still apply.
const known = new Set(registry.all().map((t) => t.name))
const routedTools = (state.turnPlan?.toolsToUse ?? []).filter((t) => known.has(t))
// When the plan committed to a CONVERSATIONAL reply, expose NO tools at all — a greeting / small-talk
// turn must be answered in plain prose, and leaving tool_catalog / provide_answer visible is exactly
// what lured the 2B into browsing the catalog (and dumping it) or firing provide_answer with an
// invented source. With nothing to reach for, the only move left is to answer. (The close gate +
// 'answer in prose' prompt back this up; this removes the temptation at the source.) Any other plan
// gets the always-visible defaults plus the planner's routed shortlist.
const visibleToolNames =
state.turnPlan?.answerKind === 'conversational' || state.turnPlan?.answerKind === 'rhetorical'
? []
: [...new Set([...defaultVisible, ...routedTools])]
// The navigate-permission gate (ctx.waitFor) is opened by the navigate tool and wired by
// TurnRunner to its TurnGate; we surface it to the dialog below via the `turnGateOpen`
// observability event (no bespoke opener needed — TurnRunner owns the gate lifecycle).
// ── Persistence + retrieval callbacks (the real integration surface) ───────
const turnCallbacks = makeTurnCallbacks({
convId,
state,
contextWindow: this.#contextWindow,
maxTokens: this.#maxTokens,
contextStrategy: this.#contextStrategy,
inputText: input.text,
ensureEncoderWiring,
assembleCompactedTurns: (turns) => this.#assembleCompactedTurns(turns),
getPrefetchedHits,
readToolResultPreview,
readModelFacingPreview,
})
const {
noop2,
storeMediaBytesCallback,
fetchMessagesCallback,
fetchMemoriesCallback,
fetchToolCallsCallback,
fetchThoughtsCallback,
fetchRetrievablesCallback,
refreshStandingInstructionsCallback,
storeMemoryCallback,
mutateMemoryCallback,
deleteMemoryCallback,
storeStandingInstructionCallback,
deleteStandingInstructionCallback,
storeMessageCallback,
storeToolCallCallback,
} = turnCallbacks
// ── Pipelines ──────────────────────────────────────────────────────────────
// turnInputPipeline (ONCE / turn): hydrate the turn constants — standing instructions.
const hydrateStanding: TurnPipelineMiddlewareFn = (async (
ctx: TurnContext,
next: () => Promise<void>
) => {
for (const si of await ctx.refreshStandingInstructions()) {
ctx.standingInstructions.add(typeof si === 'string' ? new Tokenizable(si) : si)
}
await next()
}) as TurnPipelineMiddlewareFn
// turnInputPipeline (ONCE / turn): hydrate the baseline RAG as a TURN CONSTANT. Retrievables belong to the
// turn, not a single dispatch — like the tool registry (kept whole; the subtractive pass only HIDES tools
// for a dispatch, never unregisters them). Seeding here (not in dispatchInput's iteration-0 block) makes
// `source.turnRetrievables` a stable set: every dispatch gets a FRESH COPY of it (dispatch_runner builds
// `[...source.turnRetrievables]` per iteration), so the per-dispatch subtractive shed is non-destructive —
// eject passages to fit THIS window, and the NEXT dispatch rebuilds the full set. The old iteration-0 seed
// (in dispatchInput) let step-4 shed passages to zero and never re-hydrated them, so on a multi-dispatch
// doc turn RAG evaporated after the first iteration and the RAG-tools eject (subtractive Step 3c) could no
// longer fire. Gate on the SAME `wantsDocContext` predicate as before (doc_cited/capacity_scoped plan, or a
// substantive query) so a greeting/small-talk turn still seeds NO passages. `fetchRetrievables` reuses the
// memoized pre-retrieval (getPrefetchedHits), so this is the same single search the planner already ran.
const hydrateRetrievables: TurnPipelineMiddlewareFn = (async (
ctx: TurnContext,
next: () => Promise<void>
) => {
const wantsDocContext =
state.turnPlan?.answerKind === 'doc_cited' ||
state.turnPlan?.answerKind === 'capacity_scoped' ||
(contentTokens(input.text).size >= CAPACITY_MIN_CONTENT_WORDS &&
state.turnPlan?.answerKind !== 'conversational' &&
state.turnPlan?.answerKind !== 'rhetorical' &&
state.turnPlan?.answerKind !== 'tool_computed')
if (wantsDocContext) {
for (const r of await ctx.fetchRetrievables()) ctx.turnRetrievables.add(r)
}
await next()
}) as TurnPipelineMiddlewareFn
const dispatchInput = makeDispatchInput({
state,
events: this.#events,
input,
image,
imageMessage,
visibleToolNames,
adapter: this.#adapter,
contextWindow: this.#contextWindow,
maxTokens: this.#maxTokens,
model: this.#model,
injectSyntheticThoughts: this.#injectSyntheticThoughts,
resolveOutputTokens: (scope) => this.#resolveOutputTokens(scope),
budgetWords: (scope) => this.#budgetWords(scope),
emitThought,
makeWindowTooSmallError,
renderPlanThought,
})
const dispatchOutput = makeDispatchOutput({
state,
events: this.#events,
input,
contextWindow: this.#contextWindow,
maxTokens: this.#maxTokens,
forceFallbackOnLeak,
prefetchedSources,
emitCheckerThought,
emitContextOverflow: (err) => this.#emitContextOverflow(err),
classifyIsAbstention: (userQuestion, reply) =>
classifyIsAbstention(adapter, userQuestion, reply),
classifyPlanViolation: (plannedKind, uncalledTools, reply) =>
classifyPlanViolation(adapter, plannedKind, uncalledTools, reply),
classifyIsDocAnswer: (text) => classifyIsDocAnswer(adapter, text),
classifyNeedsToolVerification: (userQuestion, reply) =>
classifyNeedsToolVerification(adapter, userQuestion, reply),
cleanInlinePathsIfAny: (cap, kind) => cleanInlinePathsIfAny(adapter, this.#events, cap, kind),
gracefulFallback,
})
// #region thrift_window_guard
// GUARD = THRIFT WINDOW (Option A). The battery's pre-dispatch overflow guard must enforce the turn's
// ACTUAL input budget — the thrift window minus the output reserve — NOT the fixed engine cap the
// adapter was constructed with. Passing it per-dispatch is load-bearing twice over: (1) it collapses
// the silent overshoot zone (constructor guard = engineMax−maxTokens = 11776 while thrift targets
// #contextWindow = 8192, so a heavy turn thrift COULDN'T fit its 8192 target rode un-caught up to 11776
// then tipped over the engine limit — the multi-model overflow bug); (2) the overflow-retry ladder
// mutates #contextWindow (8192→6144→…), and only a per-dispatch override reflects that, so the guard
// tightens WITH the shed target instead of staying frozen at construction. maxNumTokens (the real
// engine hard-limit) is a separate concern and stays at the constructor value.
const executor: DispatchExecutorFn = await adapter.executor({
contextWindow: Math.max(2048, this.#contextWindow - this.#maxTokens),
} as never)
// #endregion thrift_window_guard
const config: TurnRunnerConfig = {
executorCallback: executor,
tools: registry.all(),
fetchToolsCallback: async (_ctx: unknown) => registry.all(),
fetchMessagesCallback,
fetchMemoriesCallback,
fetchThoughtsCallback,
fetchToolCallsCallback,
fetchRetrievablesCallback,
refreshStandingInstructionsCallback,
storeStandingInstructionCallback,
mutateStandingInstructionCallback: noop2,
deleteStandingInstructionCallback,
storeMemoryCallback,
mutateMemoryCallback,
deleteMemoryCallback,
storeRetrievableCallback: noop2,
mutateRetrievableCallback: noop2,
deleteRetrievableCallback: noop2,
storeMessageCallback,
mutateMessageCallback: noop2,
deleteMessageCallback: noop2,
storeThoughtCallback: noop2,
mutateThoughtCallback: noop2,
deleteThoughtCallback: noop2,
storeToolCallCallback,
mutateToolCallCallback: noop2,
deleteToolCallCallback: noop2,
storeMediaBytesCallback,
// Retrievable bytes: the flagship's retrievables are pre-fetched in-memory each turn (not
// byte-spooled), so this conduit is unused. A throwing arity-3 stub satisfies the
// reader-returning signature; it is never invoked.
storeRetrievableBytesCallback: (async (
_a: unknown,
_b: unknown,
_c: unknown
): Promise<never> => {
throw new Error('retrievable byte spooling is not used by the flagship agent')
}) as unknown as TurnRunnerConfig['storeRetrievableBytesCallback'],
// #region pipeline_wiring
turnInputPipeline: [hydrateStanding, hydrateRetrievables],
dispatchInputPipeline: [dispatchInput as never],
dispatchOutputPipeline: [dispatchOutput as never],
// #endregion pipeline_wiring
}
const runner = new TurnRunner(config)
// Stream the assistant message to the dialog as it generates.
runner.on('message', (content: unknown) => {
const c = content as { full?: string; aDelta?: string }
if (typeof c?.full === 'string') state.answer = c.full
else if (typeof c?.aDelta === 'string') state.answer += c.aDelta
this.#events.onDelta?.(state.answer)
})
// Stream the model's REAL chain-of-thought to the dialog (only generated when the user enabled the
// model's thinking channel). The synthetic plan/nudge thoughts are emitted separately, from
// dispatchInput where they're injected. `full` is the accumulated text, so the UI replaces by id.
runner.on('thought', (content: unknown) => {
const c = content as {
id?: string
full?: string
aDelta?: string
isComplete?: boolean
}
const text = typeof c?.full === 'string' ? c.full : (c?.aDelta ?? '')
emitThought({
id: c?.id ?? 'reasoning',
kind: 'reasoning',
text,
isComplete: c?.isComplete === true,
})
})
// The navigate gate: when a tool opens it via ctx.waitFor, surface it to the dialog.
runner.observe('turnGateOpen', (gate: unknown) => {
const g = gate as {
id: string
reason: string
payload: unknown
resolve: (v: unknown) => void
reject: (e: Error) => void
}
// Headless auto-ack: no dialog to answer this gate, so resolve it immediately with the configured
// verdict instead of parking it on pendingGate (which nothing in a Node run ever reads → deadlock).
// Still surface it through onGate for observability so the run row records that the gate fired.
if (this.#autoGateVerdict !== null) {
this.#events.onGate?.({ gate: 'navigate', detail: g.reason })
g.resolve({ allow: this.#autoGateVerdict })
return
}
// expiresAt (the gate's auto-decline deadline) rides the payload — surface it so the dialog renders a
// live countdown next to Allow/Deny. Undefined for a gate without a timeout (waits indefinitely).
const expiresAt = (g.payload as { expiresAt?: unknown } | null | undefined)?.expiresAt
pendingGate.value = {
id: g.id,
reason: g.reason,
payload: g.payload,
resolve: (v) => g.resolve(v),
reject: (e) => g.reject(e),
...(typeof expiresAt === 'number' ? { expiresAt } : {}),
}
})
runner.observe('turnGateClosed', () => {
pendingGate.value = null
})
// Non-fatal warnings the runner recovered from (today: token-estimation degraded to a char guesstimate
// when a tokenizer threw — e.g. special-token text). Surface via the SAME onGate instrumentation stream
// the rails use, so the degrade is visible in the "what happened" panel instead of silent. Does NOT set
// dispatchError — the turn continues normally.
runner.observe('warning', (w: unknown) => {
const evt = w as { kind?: string; message?: string } | null | undefined
this.#events.onGate?.({
gate: (evt?.kind as AgentGateKind) ?? 'token-estimation-degraded',
detail: evt?.message ?? 'a non-fatal degradation occurred',
})
})
runner.observe('error', (err: unknown) => {
// A nacked dispatch (e.g. a WebGPU/WASM OOM during generate) REJECTS DispatchRunner.dispatch,
// which TurnRunner catches and re-emits here as 'error' — and CRUCIALLY the dispatch-output
// pipeline is skipped on a nack (the runner breaks the iteration loop before it runs), so the
// dispatchOutput nackError capture never fires. This observer is therefore the real seam for a
// generation OOM. Capture the first one; run() surfaces it as the turn error + oom flag. We only
// OVERWRITE a prior non-OOM error with an OOM (the capacity signal is the one worth showing).
//
// The TurnRunner emits errors already WRAPPED in a generic container (E_LLM_EXECUTION_EXECUTOR_ERROR
// from a thrown executor, E_OUTPUT_PIPELINE_ERROR from an output-pipeline throw, etc.) whose
// `.message` is a useless "…callback threw an error." and whose OOM/overflow signature lives on the
// CAUSE. Unwrap to the root cause FIRST so both classification and the surfaced message are the real
// failure — otherwise a genuine OOM/overflow (or any diagnosable error) reads as the generic wrapper.
const root = unwrapToRootCause(err)
const isOom = isGpuOomError(root)
const isOverflow = isContextOverflowError(root)
const isWindowSmall = isWindowTooSmallError(root)
// The window-too-small WALL nacks from dispatchInput, so the dispatch REJECTS and TurnRunner
// re-emits it HERE (the dispatchOutput nackError capture is skipped on a nack). Classify it so
// run() surfaces the windowTooSmall flag (the budget-math banner) — WITHOUT this the wall's nack
// was captured as a generic error and the flag never set (the 4K stall). Treat it like overflow:
// it always wins the capture (it's the actionable signal), and rides the same observability channel.
if (!state.dispatchError || isOom || isOverflow || isWindowSmall) {
state.dispatchError = isError(root) ? root.message : String(root)
state.dispatchErrorIsOom = isOom
state.dispatchErrorIsOverflow = isOverflow
state.dispatchErrorIsWindowTooSmall = isWindowSmall
if (isOverflow || isWindowSmall) this.#emitContextOverflow(root)
}
})
try {
await runner.run({
turnAbortController: new AbortController(),
// Grade the system prompt to the turn's INPUT budget (declared window − output reserve). A tight
// edge window (e.g. 4096/1024 → 3072 input) renders the dense floor; a generous window renders
// the full verbose prompt. Uses #contextWindow (the current thrift window, which the overflow
// ladder may have tightened) so the prompt shrinks in lockstep with the wall.
systemPrompt: buildSystemPrompt(
{
currentPath: path,
hasImage: !!input.image,
},
Math.max(2048, this.#contextWindow - this.#maxTokens)
),
standingInstructions: [],
})
} catch (err) {
this.#events.onPhase?.('error')
this.#events.onActivity?.('')
// A thrown OOM (e.g. rethrown from the answer classifier) is the same capacity signal as a nacked
// one — flag it so the dialog shows the capacity banner + Retry, not a generic error. A thrown
// context overflow (e.g. rethrown from a classifier's own dispatch) likewise gets the too-long
// banner via contextOverflow. Unwrap the generic ADK container to the root cause first, so the
// classification and the message the banner shows reflect the REAL failure (not "…callback threw").
const root = unwrapToRootCause(err)
const overflow = isContextOverflowError(root)
const windowTooSmall = isWindowTooSmallError(root)
// The window-too-small WALL rides the same observability channel as the engine overflow (same
// user-facing class: "your window can't hold this turn"), but stays a DISTINCT RunResult flag so
// run() won't retry it and the dialog shows the budget-math banner, not the too-long one.
if (overflow || windowTooSmall) this.#emitContextOverflow(root)
return {
answer: state.answer,
trace: state.lastTrace,
refused: state.refused,
error: isError(root) ? root.message : String(root),
oom: isGpuOomError(root),
contextOverflow: overflow,
windowTooSmall,
}
}
// A NACK (e.g. WebGPU OOM mid-generate) ends the turn without throwing. Surface it as the turn's
// error so the dialog can show the friendly "reduce the context window + Retry" banner — and do
// NOT persist a fake assistant message for a turn the model never actually answered.
if (state.dispatchError && !state.captured) {
this.#events.onPhase?.('error')
this.#events.onActivity?.('')
return {
answer: state.answer,
trace: state.lastTrace,
refused: state.refused,
error: state.dispatchError,
oom: state.dispatchErrorIsOom,
contextOverflow: state.dispatchErrorIsOverflow,
windowTooSmall: state.dispatchErrorIsWindowTooSmall,
}
}
// POISONED-CONTEXT early return. The model surfaced an XML-shaped trust-tier fence a second time this
// turn — its context is poisoned (accumulated thrash). Do NOT persist anything; signal `run()` to shed
// the window and re-run. No user-facing error (unlike overflow/OOM) — this is a silent internal retry.
if (state.turnPoisoned && !state.captured) {
this.#events.onActivity?.('')
return { answer: '', trace: state.lastTrace, refused: state.refused, contextPoisoned: true }
}
// Persist the FINAL answer. Either a CITED answer captured from provide_answer (sources become
// message references / chips), or a plain-prose reply (conversational, general-knowledge, or an
// honest "I don't know" — abstention is prose now, not a tool). Rejected attempts are not persisted.
const cap = state.captured as CapturedAnswer | null
let assistantMsgId: string
if (cap) {
// Cited answer via provide_answer — persist with its sources as references. (Inline-path cleanup runs
// earlier, in the dispatchOutput pipeline where the answer is captured — see #rewriteRemoveInlinePaths.)
state.answer = cap.text
const stored = await storeMessage(convId, {
role: 'assistant',
content: cap.text,
references: cap.sources.map((s) => ({
id: s.path,
pageUrl: s.path,
anchor: '',
title: s.title ?? s.path,
headingPath: [],
excerpt: '',
})),
})
assistantMsgId = stored.id
} else {
// A plain-prose reply (no answer tool) — conversational, general-knowledge, an honest IDK, or a
// dup-stop/hard-stop rescue. Persist as-is, no citations. Strip any FALSE-ABSTENTION preamble at
// this single commit point so a good synthesis the 2B prefixed with "I couldn't find … however,"
// is delivered as the real answer (covers every accept path, not just the dup-stop rescue).
const cleaned = stripFalseAbstentionPreamble(state.proseAnswer || '')
// FINAL SAFETY NET: never commit a verbatim doc-source QUOTE (raw {@link …}, ::: containers, //
// comments) NOR a directive-ACKNOWLEDGMENT ("I understand the directive, I will ensure…") as the
// answer. If either slipped through every gate, surface the graceful fallback instead — defense in
// depth so neither documentation source nor gate-meta-chatter ever reaches the human plane.
state.answer =
looksLikeRawDocSource(cleaned) || looksLikeDirectiveAcknowledgment(cleaned)
? gracefulFallback(state.turnPlan?.answerKind)
: cleaned || "I wasn't able to produce an answer for that."
const stored = await storeMessage(convId, {
role: 'assistant',
content: state.answer,
})
assistantMsgId = stored.id
}
// Persist this turn's reasoning, tied to the assistant message just stored, so the Thinking panel
// rehydrates on reload (durable turn history). Insertion order is the reasoning order. Best-effort:
// a thought-store failure must never sink an answer that already landed.
// CRITICAL: do NOT persist with the in-turn key (`__plan-thought` / `__nudge:*`). Those stable ids are
// ONLY for de-duping within a turn (live UI + record map). If they were written to the store, next
// turn's fetchThoughts would return id `__plan-thought`, dispatchInput would mark it seeded, and the
// injection guard (`!seededIds.has(PLAN_THOUGHT_ID)`) would SUPPRESS the new turn's plan — so the
// planner silently stopped running from turn 2 on. Let storeThought mint a fresh unique id instead.
try {
for (const [, t] of state.turnThoughtRecords) {
// Never persist a blank-text thought. A streaming reasoning partial (or any emitThought fired
// before its text landed) can record an empty string; persisting it plants a row that the NEXT
// turn's fetchThoughts would feed to `new Thought`, which REJECTS empty content
// (E_INVALID_INITIAL_THOUGHT_VALUE) inside dispatchInput — ending that turn in an error banner.
// Skip at the source; the fetch side also filters (belt-and-suspenders). [[no_pathological_cases_dont_cheat]]
if (typeof t.text !== 'string' || t.text.trim().length === 0) continue
await storeThought(convId, {
messageId: assistantMsgId,
kind: t.kind,
text: t.text,
})
}
} catch {
/* reasoning is non-critical history — never fail the turn over it */
}
this.#events.onPhase?.('complete')
this.#events.onActivity?.('')
this.#events.onTurnComplete?.()
return { answer: state.answer, trace: state.lastTrace, refused: state.refused }
}
}
/**
* The WALL. Build the typed error the loop nacks with when the subtractive pass could not fit the user's
* context-window budget even after shedding everything sheddable (`ThriftTrace.refused`). This is a
* docs-app / flagship-loop POLICY error — NOT a battery concern — so it lives here, not in `src/` (the
* battery keeps its own engine-cap `E_LITERT_LM_CONTEXT_OVERFLOW`). The `code` is cross-realm-safe like
* the others so {@link isWindowTooSmallError} can classify it from the catch/nack seams. The message names
* the real math so the user knows the exact lever: raise the window, or reduce the tool shortlist. The
* distinction from a context overflow matters: overflow = the ENGINE's fixed cap was exceeded (recovery
* shrinks the window and retries); window-too-small = the USER's chosen budget can't hold this turn's
* unsheddable floor (retrying at the same window would refuse identically — so `run()` does NOT retry).
*/
function makeWindowTooSmallError(trace: ThriftTrace): Error & { code: string } {
const over = trace.totalAfter - trace.budget
// AGENT VOICE. This message is shown to the user VERBATIM (agent_dialog reportTurnError → errorMsg). The
// agent has just discovered, mid-task, that it has gathered more than its working memory can hold and
// still leave room to reply — so it should SAY that, in first person, the way an assistant would when it
// hits its own limit: own it, say what it CAN still do, and name the levers the user actually controls
// (the context-window slider raises the budget; a smaller max-output reserve frees input room; a more
// focused question needs fewer sources). NOT a system diagnostic. The exact token math is kept as a short
// trailing parenthetical for the curious/devs, not the lead. No unreachable levers ("reduce tools" — the
// planner picks those internally). ([[adk_ethos_surface_not_impose]])
const err = new Error(
`I've pulled together more material on this than I can hold in memory at once and still have room to ` +
`write you a proper answer — I ran out of working space partway through. Try narrowing the question ` +
`to the part you care about most, or give me more room to think by raising the context-window slider ` +
`(or lowering the reply-length reserve). ` +
`(Working memory: needed ${trace.totalAfter.toLocaleString()} tokens, ${over.toLocaleString()} over ` +
`the ${trace.budget.toLocaleString()}-token budget for this ${trace.contextWindow.toLocaleString()}-token window.)`
) as Error & { code: string }
err.code = 'E_ADK_WINDOW_TOO_SMALL'
err.name = 'E_ADK_WINDOW_TOO_SMALL'
return err
}
/**
* The graceful fallback string when every gate rejected the model's output and there's no usable prose to
* surface — routed by the plan's answer_kind so the message actually fits the request. Before this was
* kind-aware, a conversational/rhetorical turn (e.g. "so it's not much of a kit, is it?") that fell back
* got the doc-citation string ("I found relevant documentation… open the cited pages"), which is nonsense
* for a turn that neither searched nor cited. Each kind now gets a fallback that matches what it was
* actually doing.
*/
function gracefulFallback(kind: PlanAnswerKind | undefined): string {
switch (kind) {
case 'conversational':
case 'rhetorical':
// No docs, no tool — a re-engage prompt, never doc framing.
return "I'm not quite sure I follow — could you say a bit more about what you mean?"
case 'tool_computed':
return "I couldn't get that value just now — mind trying again?"
case 'capacity_scoped':
// The honest capacity-decline turn wedged before it could compose — keep the on-brand message
// (small on-device model, narrow it or read the docs), never the doc-abstention.
return "That's a broad, exhaustive ask — more than I can lay out in full as a small on-device model. Try narrowing it to one area, or browse the documentation directly for the complete picture."
case 'doc_cited':
default:
// The only place the doc-flavored abstention is apt: a doc question that searched but couldn't
// compose. Kept as the default too (a null plan defaulting to a doc lookup is the safe assumption).
return "I found relevant documentation but couldn't compose a clear answer from it. Try asking a more specific question, or open the cited pages in the tool details."
}
}
void DEFAULT_CONVERSATION_ID