Punching Above Its Weights
This showcase supersedes an earlier agent design
This site's live agent used to be the tool-less, synthetic-RAG 3B documented on /showcase/ask-adk. That page is not wrong and it is not being walked back — every technique it documents shipped and works. What changed is completeness. This build replaces injected retrieval with a real tool loop, adds a planner that commits to a turn's shape before the model ever generates, adds a cascade of gates that own turn completion instead of a single citation check, and adds durable SQLite/Kysely persistence where the 3B had none. Ask ADK remains on the site as a study of the design it replaced — read it for the case for synthetic RAG on a tool-less model. This page is the case for a real tool-using agent on a hostile budget.
This agent runs a Gemma-4 E2B — an effective-2B model, engineered to run at a two-billion-parameter footprint — in a browser tab, on your GPU, driving a real multi-step tool loop (search, read, compute, cite) inside a context window you can shrink live, mid-conversation. No backend. No API key. That is the agent. This page is how it is built.
Here is what happens if you build that agent the way every tutorial implies you can — hand the model some tools, loop until it answers. You ask it what day it is. It calls get_current_time, a perfectly good tool, and gets back a perfectly good result. Then it calls get_current_time again. Same arguments. Then again. The result it needs is sitting right there in its context, and the model reads straight past it, forever, because nothing in a two-billion-parameter checkpoint knows that having an answer and using one are supposed to be connected. Or it stops looping — and confidently tells you it "doesn't have access to the current date," with the date literally in front of it. Or it answers a documentation question with a page path it invented, formatted exactly like a real one. None of this is exotic. If you have ever wired a small model to tools, you have watched some version of it, probably within the first ten minutes.
The standard responses are to prompt harder ("You MUST use the tool result…") or to rent a bigger model and hope. Prompting harder fails because a small model under context pressure drops instructions exactly the way you drop the fourth thing on a mental to-do list. The bigger model mostly works — which is why almost nobody builds the machinery this page describes — but "mostly" is doing heavy lifting, and now every message costs real money for failure modes that never actually went away. They just got rarer and better-worded.
This build takes the third option: treat every one of those failures as a missing mechanism, and build the mechanism. Not asked for in the prompt — enforced in code, outside the model, where it can be read, tested, and fixed.
TL;DR — the five mechanisms
Nothing here works because the model got smarter. Five pieces of scaffolding do the actual work:
- The model answers before deciding what it's answering → so a planner runs first, committing to the turn's shape — what kind of answer, which tools, how long — before a single token of the real answer is generated.
- The context window fills with stale, low-value tokens → so a subtractive pass starts every dispatch from an oversized working set and cuts, by measured token count, until it fits the window — best content survives, never a mid-sentence truncation.
- The model emits answers that shouldn't count (unread results, echoed corrections, plan violations) → so a gate cascade inspects every candidate answer and refuses to let the turn end until the output is actually acceptable — a mechanism, not a system prompt asking nicely.
- One tool result can eat the whole budget → so results arrive as artifact handles: a 500KB search result costs a few dozen tokens until the model explicitly reads the part it needs.
- The GPU dies mid-conversation (this is WebGPU on real hardware) → so a recovery ladder respawns the model in a fresh worker and the conversation survives, losing at most the turn in flight.
None of this is the model being disciplined. It is scaffolding, in code, that makes a small, cheap, undisciplined model behave like it has adult supervision.
And the size of that lever is measured, not asserted. In the head-to-head evaluation at the bottom of this page, this same E2B on an 8k window — context managed subtractively — scores 0.82 on a blind-judged 94-turn corpus, while a far larger open-weights reasoning model on a 128k window, fed by naive append-everything context, collapses to 0.08 on the same corpus, same judges. The small disciplined agent out-answers the large undisciplined one outright. It does not close the raw-capability gap — the larger model still wins arm-for-arm when both are disciplined — it proves something more useful: the scaffolding is worth more than the parameters the moment the parameters are left to fend for themselves. The model sets the ceiling; the scaffolding decides whether you get anywhere near it. The rails are not a nice-to-have layered on top of a good agent. They are the agent.
How to read this page: the live demo below lets you watch the context discipline work with one slider — start there. The sections after it walk each mechanism in the order a turn actually executes, with the real source embedded, and close with the judged numbers. Everything is a pointer to code you can read, not a promise.
What this agent is mechanically prevented from doing
- Cannot commit an uncited doc answer — within the gate's retry budget. Prose classified as a documentation answer without a real citation is not acked; the turn continues, nudged to re-issue via
provide_answer(the four-case terminator, case 3). Past that budget (STUCK_PROSE_LIMIT), the gate stops re-looping and accepts the prose as-is — a real, honestly-uncited answer on the record, never a fabricated citation bolted on to make the gate happy. - Cannot answer past an unread tool result. The read-consume invariant blocks any answer — tool call or prose — while an artifact handle from this turn sits unopened.
- Cannot finish a turn that violates its own plan. Plan-conformance tracks which planned tools are never successfully called and keeps correcting until they are, or the plan itself dissolves because the window shed them.
- Cannot silently overflow the window. The subtractive pass enforces the thrift budget every dispatch; what doesn't fit is shed by a fixed order, never truncated mid-token.
- Cannot re-emit its own rejected prose indefinitely. The parroting gate excludes the model's own prior this-turn generations from the false-positive set, but still catches it echoing a nudge, a tool result, or a directive as if that were an answer — up to
NO_PROGRESS_PROSE_LIMITre-loops, past which the gate accepts the repeat rather than nudge forever with nothing changing. - Cannot survive a GPU device loss silently. A lost WebGPU device is escalated through a fixed ladder — respawn, then reload — never swallowed and never left hanging.
Every item above is a named mechanism, in code, further down this page — not a promise, a pointer.
See it: one model, a window you control
You do not have to take any of this on faith, and you do not need a lab: the live agent is on this page, right now — the Ask ADK bubble floating in the bottom-right corner, or just
open it from here. Load the model (a one-time ~2GB download, straight to your GPU — no server, no API key, nothing leaving your machine), and ask it something real about this library. It runs a real gemma-4-E2B-it — Gemma-4's effective-2B build, engineered to run at a ~2B footprint, so "small" is if anything generous — fully in your browser on WebGPU through the ADK's LiteRtLmAdapter.
Then do the thing no other agent demo lets you do: shrink its world while you talk to it. The dialog's context-window slider is live — drag it down mid-conversation and watch the utilization meter at the bottom. Every segment is one bucket of the working set (system prompt, tools, tool results, retrieved docs, memories, conversation, thoughts), and as the budget tightens you can watch the subtractive pass make its cuts in real time: the retrieval tail sheds first — gets cut, lowest-ranked chunks first, best stay — prior-turn thoughts are stripped, ejected items are struck through rather than silently vanished, and the tool registry collapses to what the turn actually needs. Then it answers — correctly — from what survived. Your conversation, your questions, a real shrinking window: no scripted fixture, no rehearsed happy path.
One deliberate choice worth naming: there is exactly one agent on this site, and everything this page demonstrates happens inside it. A page about running models on WebGPU that casually loads a second multi-gigabyte model next to the first is a page that crashes your GPU to make a point — VRAM is finite, device loss is real (see Surviving the GPU), and that applies to us too.
Field note: a working set that looks long can still fit a small window
A dozen chat turns and a stack of policy snippets look long but never trigger shedding — English runs about four characters per token, so a chunky paragraph is only ~80 tokens and the whole thing fits a 2K window with room to spare. A demo that never sheds proves nothing. What actually blows a small window is realistic agent context: full policy sections, multi-message histories, real tool results. Ask the agent a few real questions, let the tool results and retrieval stack up, then drag the window down — that is when the meter starts showing real cuts. If your context "feels big" but fits comfortably, you are not measuring, you are guessing.
The shape of the turn
A multi-step tool loop — search, then read, then compute, then answer — has an obvious way to build it: one big loop that reloads everything on every pass, then calls the model, then repeats. It works, but a five-step turn now repeats four passes of setup that only needed to happen once — re-hydrating (reloading from storage) the same chat history and memories every iteration, exactly what gets expensive as history grows.
@nhtio/adk's TurnRunner exists to stop that duplication: turn-scoped work runs once, dispatch-scoped work runs every iteration — where a dispatch is one model call within a turn. This agent uses all four pipelines, and the split isn't decorative — it's why a five-iteration tool loop doesn't re-hydrate history five times:
// The flagship agent harness — the orchestration core, built on the ADK TurnRunner.
//
// WHY TurnRunner (not DispatchRunner directly): the ADK splits work across FOUR pipelines for
// a cost reason (see the LLM-Dispatch / "what each pipeline owns" docs). TurnRunner is the seam
// that owns all four; DispatchRunner only exposes the per-iteration two. The flagship needs the
// turn-scoped seam, so it uses TurnRunner:
//
// - turnInputPipeline (once / turn) → hydrate the TURN CONSTANTS: standing instructions.
// (Tools are turn-constant too, supplied via config.)
// - dispatchInputPipeline (per iteration, before the model) → hydrate the CONVERSATIONAL
// state the loop reasons over (messages — re-drained
// each iteration so a MID-TURN interrupt folds in —
// plus memories, thoughts, tool-calls, and the
// pre-fetched retrievables), then the subtractive pass
// and the force-answer guard.
// - dispatchOutputPipeline (per iteration, after the model) → loop telemetry (which tool ran).
// - persistence callbacks → the SQLite facade. The transformers.js adapter is a well-behaved
// executor: it calls ctx.storeMessage / storeThought /
// storeToolCall and ctx.ack() itself, so those
// callbacks ARE the persistence wiring — no bespoke
// store loop.
//
// LOOP CONTROL is by SIGNALS, using the ADK primitives (ctx.iteration, ctx.toolCallCount) — the
// policy thresholds are ours. We force an answer only on a GENUINE repeat (same tool+args ⇒ same
// checksum, REPEAT_LIMIT+ times) or the hard iteration cap — never on "any tool was used", so a
// healthy multi-step turn (search → catalog → call_a_tool → answer) runs freely.
//
// CHAT HISTORY vs CONTEXT WINDOW: the user turn is persisted in run(); the adapter persists the
// final assistant turn via the storeMessage callback. The model-facing context (turnMessages and
// the rest) is rebuilt each turn and never rendered. See memory chat_history_vs_context_window.turnInputPipeline loads the turn constants once — standing instructions; tools are turn-constant too, supplied via config. dispatchInputPipeline reloads the conversational state every iteration instead — messages (re-drained each time so a mid-turn interrupt folds straight in), memories, thoughts, tool-calls, and pre-fetched retrievables — then runs the subtractive pass and the force-answer guard. dispatchOutputPipeline is where the gate cascade lives. Persistence callbacks are the SQLite facade; the adapter is a well-behaved executor that calls ctx.storeMessage/storeThought/storeToolCall and ctx.ack() itself, so those callbacks are the entire persistence layer — no bespoke store loop bolted on the side.
The pre-dispatch overflow guard is passed per dispatch, not fixed once at construction, because the active thrift target moves — both with the user's slider and with the overflow-retry ladder tightening it mid-turn:
// GUARD = THRIFT WINDOW (Option A). The battery's pre-dispatch overflow guard must enforce the turn's
// ACTUAL input budget — the thrift window minus the output reserve — NOT the fixed engine cap the
// adapter was constructed with. Passing it per-dispatch is load-bearing twice over: (1) it collapses
// the silent overshoot zone (constructor guard = engineMax−maxTokens = 11776 while thrift targets
// #contextWindow = 8192, so a heavy turn thrift COULDN'T fit its 8192 target rode un-caught up to 11776
// then tipped over the engine limit — the multi-model overflow bug); (2) the overflow-retry ladder
// mutates #contextWindow (8192→6144→…), and only a per-dispatch override reflects that, so the guard
// tightens WITH the shed target instead of staying frozen at construction. maxNumTokens (the real
// engine hard-limit) is a separate concern and stays at the constructor value.
const executor: DispatchExecutorFn = await adapter.executor({
contextWindow: Math.max(2048, this.#contextWindow - this.#maxTokens),
} as never)turnInputPipeline: [hydrateStanding, hydrateRetrievables],
dispatchInputPipeline: [dispatchInput as never],
dispatchOutputPipeline: [dispatchOutput as never],In plain terms: the guard re-reads its target on every call, so tightening the window mid-turn takes effect on the very next one.
Loop control is by signal, not by vibes: ctx.iteration and ctx.toolCallCount are ADK primitives, the thresholds are this agent's own policy — the ADK core ships no maxIterations at all. There is no single global iteration cap; a plan-conformance failure is allowed to re-loop unbounded, deliberately, because an observed raw failure beats a silently truncated one. What actually stops a turn is a family of narrow, named counters, each bounded to the one failure mode it watches: a genuine repeat — same tool, same arguments, same checksum — force-answers past DUPLICATE_CALL_HARD_STOP; a near-identical uncited answer gets accepted past STUCK_PROSE_LIMIT; a stalled iteration with no tool progress gets accepted past NO_PROGRESS_PROSE_LIMIT. Never on "a tool was used," which is why a healthy multi-step turn (search → read the result → call a computed tool → answer) runs to completion without the harness getting nervous halfway through.
Field note: hiding a tool is not the same as removing it
Hiding a tool's schema (ctx.tools.setHidden(...)) strips it from the prompt — the model still emits tool-call syntax anyway, on a hallucinated argument, and the adapter parses and executes it, because nothing told the executor to refuse. What worked here, on a genuine repeat: force the answer by folding the prior result into an ephemeral user turn added to the live turnMessages set — never storeMessage'd, so a corrective nudge this-turn can never poison durable history or replay wrong on reload.
The planner book-end Showcase policy · built on ADK
Showcase policy, implemented using ADK
The planner, the gate cascade, and the GPU recovery ladder below are this agent's application policy — decisions we made for a 2B on a browser GPU. ADK is the capability provider: it supplies the dispatch loop, the pipeline seams, the artifact handles, and the context primitives these are built on. It does not ship a planner, these gates, or this recovery ladder. That is the point — the seams are the ADK; the policy is yours. The Showcase policy · built on ADK badge marks each section that is ours to decide, not inherent ADK behavior.
Ask a small model a question and watch what it does first: it starts answering. Not deciding what kind of answer this should be — conversational aside, a number the tools can compute, a cited doc lookup, an exhaustive list scoped to what fits — just answering, in whatever register and length the first few tokens happened to commit to. Get that wrong and everything downstream follows: a tool it didn't need, a tool it skipped, four sentences where the question wanted four words.
Our solution is a book-end: a dedicated dispatch, run before the real answer, whose only job is to decide the turn's shape and write it down. The why matters more than the shape: a frontier model does this step internally — its reasoning phase settles what kind of answer it's producing before committing tokens to it. This build doesn't skip that deliberation; it restructures it. Reasoning is spent exactly where it pays: the planner dispatch runs with thinking enabled — classifying the turn is the one decision this model most often gets wrong in a single greedy shot, and letting it think first is worth the tokens. But instead of letting that chain-of-thought ramble free through the whole turn (where it would burn the tool loop and drift), the deliberation is distilled into a deterministically generatable, testable, token-dense plan — a validated make_plan tool call against the real catalog, not free prose — which is then injected back into the worker as the model's own synthetic "thought." The other end of the book-end closes the loop: a conformance gate validates that the model actually stuck to the plan it committed to. Freeform reasoning in, structured commitment out, held to account at the close. The make_plan call — up to three retries on a parse failure, fail-open (an unplanned turn beats a blocked one) on anything that isn't a GPU OOM — commits three things: what kind of answer this is, which tools it intends to call, and how long the answer should run.
That book-end dispatch needs its own way to end, though. autoAck — the setting that lets the harness close a dispatch automatically — can't close this one: the LiteRT adapter only auto-acks a generation with zero tool calls, and the planner's entire job is to emit exactly one (make_plan). Without an explicit terminator, the model re-plans forever, burning the turn on "call make_plan now" repeated at the model that already called it:
// TERMINATOR (the fix for the planner thrash). autoAck on the LiteRT adapter only acks a
// generation that emitted NO tool call (adapter: `if (calls.length === 0) ctx.ack()`). But the
// planner's whole job is to emit ONE tool call — make_plan — so autoAck NEVER fires, the dispatch
// never acks, and it loops on "Call make_plan now" until the abort signal — re-planning 6–12× and
// burning the turn (a contradictory-directive thrash: "call make_plan now" persisting after
// make_plan was already called). So own termination explicitly: after each planner generation,
// ack — one generation per dispatch. If a plan was captured we're done; if make_plan failed to
// parse (plan still null), the ack ends THIS dispatch so the outer `for attempt<=3` mints a fresh
// one (the intended "3 fresh single-shot retries" design, which the missing terminator defeated).
turnOutputPipeline: [
(async (ctx: DispatchContext, next: () => Promise<void>): Promise<void> => {
if (!ctx.isSignalled) ctx.ack()
await next()
}) as never,
],In plain terms: the planner dispatch acks itself the moment make_plan lands, instead of waiting on a rule that never fires for it.
The committed plan doesn't get frozen into a static instruction. It is rendered as an evaluatableThought — a small function, not a fixed string, re-run every iteration against the live dispatch context — so it only ever claims a planned tool is "directly available" when that tool actually survived this dispatch's subtractive pass:
export function renderPlanThought(
plan: TurnPlan,
budgetWords: number,
capacityNote?: string
): (ectx?: { tools?: { visible: () => { name?: string }[] } }) => string {
return (ectx?: { tools?: { visible: () => { name?: string }[] } }): string => {
// Which planned tools are ACTUALLY visible in THIS dispatch? (No ctx → assume all planned tools are
// present: the no-ctx fallback is the pre-shed plan, matching how the thought reads outside assembly.)
const visibleNames = ectx?.tools ? new Set(ectx.tools.visible().map((t) => t.name)) : undefined
const availablePlannedTools =
visibleNames === undefined
? plan.toolsToUse
: plan.toolsToUse.filter((name) => visibleNames.has(name))
// A planned tool that is NOT directly visible is usually still REACHABLE via the catalog capstone
// (call_a_tool) — the flagship hides utility tools (get_current_time, calculate, …) behind
// tool_catalog/call_a_tool to save prompt budget; they are callable, just not in the direct list.
// This is the THIRD state the old binary (visible ⇒ "call it" / else ⇒ "answer from docs") missed:
// the proven tool_computed failure was a planned get_current_time landing in the "not available →
// answer from the documentation" branch, so the 2B abstained ("I don't have the date") instead of
// reaching the tool through the capstone. Only treat a planned tool as genuinely unreachable when the
// capstone ITSELF is also gone (a tight-window shed of everything).
const capstoneVisible = visibleNames === undefined || visibleNames.has('call_a_tool')
const plannedViaCapstone =
visibleNames === undefined ? [] : plan.toolsToUse.filter((name) => !visibleNames.has(name))That guard is not academic. A tight window can shed a planned tool between the moment the plan is made and the moment the model reads it, and a frozen "you MUST call search_docs_semantic, it is directly available" directive contradicts a dispatch where that tool is gone — a 2B flounders in exactly that contradiction without the re-check. The re-check adds a third state a naive binary misses: a planned tool absent from the direct list can still be reachable through the call_a_tool catalog capstone, and the plan-thought now says so explicitly, rather than reading as "unavailable" and prompting an honest-sounding but wrong "I don't have access to that."
Field note: the correction that works is the one in the model's own voice
Every re-looping gate on this page fires a first-person __nudge: thought ("My answer leaked an internal tag; I'll rewrite it clean") alongside a [SYSTEM DIRECTIVE] message, never the directive alone. A 2B converges in fewer iterations on something it apparently just decided than on an external order — audited across every gate that loops, and every one of them carries the first-person form. Terminal hard-stops skip it deliberately: the turn is already over, so there is no next generation left to read it.
The subtractive pass: shrink to fit, every dispatch
Leave context assembly unmanaged and one of two things happens: the window overflows and the dispatch errors outright, or — worse, because it's silent — it fills with stale junk and the answer just gets quietly worse.
This is the mechanism behind the slider you just dragged in the live agent. Every dispatch, the agent runs the @nhtio/adk/batteries/context/thrift battery (documented in full on the Token Thrift context battery page): hold a working set — everything that could go in the prompt — far larger than any one window, then subtract to fit, lowest-signal first, in a fixed shed order (the same category always cut before the same other, never at random), by measured token weight — never truncate mid-token, never guess. If even the unsheddable floor won't fit, the pass returns a bounded refusal: a typed error instead of a truncated dispatch.
// Tool-shed priority (LAST-RESORT bucket). Tools are the model's ability to ACT, so they shed only after
// every cheaper bucket (image, prior-turn results, RAG, memories, messages, stale thoughts) is exhausted
// and the dispatch STILL won't fit. Order = shed order (first listed sheds first).
//
// KEY POLICY (per the user): at a tight window the RIGHT move is to shed the ACTING tools — including
// `provide_answer` — and let the model answer in PROSE from the prefetched RAG that is ALREADY in context
// (baseline retrieval runs before any tool call). Shedding the answer tool frees its schema tokens for MORE
// RAG (higher-quality grounded prose) AND removes the citation pressure the model can't satisfy or render
// at that budget. The prose auto-capture path (dispatchOutput) commits the answer — no answer tool needed.
// So `provide_answer` is NOT kept-to-the-end; it sheds alongside the gather tools it can no longer cite
// from. The ONE thing to preserve is the ability to DELIVER, and prose auto-capture is that path.
//
// When `provide_answer` is shed, the loop drops all citation instructions/nudges (see agent_runtime.ts,
// keyed on `provide_answer` not being visible) so we never prompt a citation the window can't hold.
// Sentinel placeholder in TOOL_SHED_ORDER marking where UNLISTED tools (routed compute/battery tools, custom
// downstream tools) rank. Never a real tool name; shedRank uses its index as the fallback rank.
const COMPUTE_TIER_MARKER = ' compute-tier'
// TOTAL shed ranking: EVERY tool that can be visible gets a tier, shed-first → shed-last. A turn's visible
// set is defaultVisible (provide_answer, search_docs_*, navigate_to_page, store_memory, tool_catalog,
// call_a_tool) + the planner's routedTools (any of the ~80 battery tools — calculate, date_*, stats_*, …) +
// the forged artifact_* readers. Tools listed here take their explicit index; any UNLISTED tool (a routed
// battery/compute tool, or a downstream consumer's custom tool) falls into the COMPUTE tier via shedRank's
// fallback, so the ranking is total and robust to tools it has never seen. Protection (protectedToolNames)
// sits ABOVE this: a planned-but-uncalled tool is excluded from the shed queue entirely, so a tool_computed
// turn's routed `calculate` is unsheddable until called even though the compute tier would shed it early.
const TOOL_SHED_ORDER: readonly string[] = [
// Tier 1 — exotic artifact readers (highest volume, lowest utility for basic doc Q&A)
'artifact_json_type',
'artifact_json_keys',
'artifact_json_length',
'artifact_json_filter',
'artifact_json_slice',
'artifact_json_pluck',
'artifact_byte_length',
'artifact_line_count',
'artifact_estimate_tokens',
'artifact_tail',
'artifact_grep',
'artifact_cat',
// release_artifact — a pure budget luxury (frees a read result's body). Sheds among the first: if the
// window is tight enough to shed tools at all, the model should spend its schema budget on reading/answering,
// not on the "I'm done" bookkeeping tool. The N-cap backstop sheds old bodies without it.
'release_artifact',
// Tier 2 — routed compute/battery tools: NOT listed by name (there are ~80 and consumers add their own).
// shedRank maps any unlisted non-core tool here, between the exotic readers and the browse/meta tools.
COMPUTE_TIER_MARKER,
// Tier 3 — heavy browse / meta / action tools
'tool_catalog', // biggest browse schema, least essential once RAG is present
'call_a_tool',
'navigate_to_page',
// Tier 4 — memory / CRUD
'store_memory',
'update_memory',
// Tier 5 — gather tools
'search_docs_keyword',
'search_docs_semantic',
// Tier 6 — delivery
'provide_answer', // sheds with the gather tools → prose auto-capture commits, citations dissolve
// Tier 7 — core readers (kept LAST — a handle the model already has is its cheapest path to real grounding)
'artifact_json_get',
'artifact_head',
]
/** Ranks a tool by last-resort shed priority (step 9), built from {@link TOOL_SHED_ORDER}. A tool listed
* there takes its explicit index. Any UNLISTED tool — a routed compute/battery tool (calculate, date_*,
* …) or a downstream consumer's custom tool — falls into the COMPUTE tier (the marker's index), so it
* sheds AFTER the exotic artifact readers but BEFORE the browse/gather/deliver/core tools. An unlisted
* `artifact_*` reader (a subclass's bespoke reader not known by name) is treated as core-ish and kept
* near the end. This is the flagship's own domain knowledge — the battery's own default `shedRank`
* treats every tool equally, having no opinion on tool names. */
const shedRank = (name: string): number => {
const idx = TOOL_SHED_ORDER.indexOf(name)
if (idx >= 0) return idx
const computeTierRank = TOOL_SHED_ORDER.indexOf(COMPUTE_TIER_MARKER)
// Unlisted artifact_* reader → keep near the end (after everything listed), like the core readers.
if (name.startsWith('artifact_')) return TOOL_SHED_ORDER.length
// Every other unlisted tool → the compute tier (shed early, after the exotic readers).
return computeTierRank
}/**
* The subtractive pass. Start wide; measure every bucket; then shed lowest-signal first
* until the dispatch fits the active window's budget — the image (the single biggest hog)
* goes first when it doesn't fit, then tools the turn doesn't need, the tail of the
* retrieval ranking, low-value memories, the oldest conversation turns. Each cut is by
* EVIDENCE (a measured bucket), never a guess. If even the floor (system prompt + newest
* turn) won't fit, REFUSE — a bounded refusal beats a truncated, incoherent dispatch.
*
* THE ALGORITHM ITSELF now lives in `@nhtio/adk/batteries/context/thrift`'s `subtractToFit` (loaded via
* the agent_adk.ts shim). This wrapper preserves the flagship's OLD 9-positional-argument call
* signature (unchanged since before the battery extraction — every existing call site and cross-spec
* test depends on it) and adapts positionals → the battery's real options-object form internally,
* binding the showcase's own policy: `TOOL_SHED_ORDER`-derived `shedRank`, `ENCODING = 'gemma'`, and the
* `__eph-`/`__compact-summary` id predicates (passed explicitly even though they match the battery's own
* defaults, for self-documentation at this call site).
*/
export const subtractToFit = (
ws: WorkingSet,
contextWindow: number,
relevantToolNames: string[],
outputReserve?: number,
/** Thought ids to PRESERVE through the prior-turn strip (the planner's this-turn plan thought). */
keepThoughtIds?: ReadonlySet<string>,
/**
* The battery's tool-declaration renderer. When supplied, the tools bucket is measured against the
* REAL rendered `<tool_definitions>` block (full JSON-Schema per tool) instead of a `name: description`
* proxy — the load-bearing fix for the context-overflow crash, where the proxy undercounted the tool
* block by ~10× and let a doc turn blow the engine cap. Omit to fall back to the proxy.
*/
renderTools?: RenderToolsFn,
/**
* Thought ids that must NEVER be shed for budget even when over — the this-turn scaffolding the worker
* needs to answer at all (e.g. the planner's plan thought + the citation reinforcement). Everything else
* in the surviving keep-set (the per-iteration nudge thoughts, prior synthetic guidance) is sheddable
* oldest-first when the dispatch still doesn't fit. Omit to make every surviving thought sheddable.
* This is a SUBSET of `keepThoughtIds`: keepThoughtIds decides what survives the Gemma §3 prior-turn
* strip; protectThoughtIds decides what additionally survives the budget shed.
*/
protectThoughtIds?: ReadonlySet<string>,
/**
* The live dispatch context, so a DYNAMIC (evaluatable) thought's token count reflects the string it
* will resolve to for THIS dispatch — `t.content.estimateTokens(ENCODING, renderCtx)` counts exactly
* what the battery's `.render(renderCtx)` will assemble. Without it a dynamic thought would be counted
* as its no-ctx fallback, and the budget could disagree with the shipped prompt. Optional; static
* content counts identically with or without it.
*/
renderCtx?: unknown,
/**
* Tool names the CURRENT PLAN committed to that have NOT yet been called this turn. These are
* UNSHEDDABLE by the last-resort tool-shed: the plan-thought tells the model to call them, so removing
* them from the rendered `<tool_definitions>` block leaves the model instructed to call a tool it cannot
* see — the proven tool_computed failure (planner chose get_current_time, thrift shed it, the 2B could
* only see provide_answer and abstained "I don't have the date"). The protection is bounded: once the
* tool has been called (its result is in context) it drops off this set and sheds normally, so it is
* never a permanent floor. Omit to make every visible tool sheddable (prior behaviour).
*/
protectedToolNames?: ReadonlySet<string>
): ThriftTrace => {
const options: ThriftSubtractToFitOptionsType = {
estimateTokens: (v, enc, ctx) => Tokenizable.estimateTokens(v, enc, ctx as never),
encoding: ENCODING,
outputReserve,
keepThoughtIds,
renderTools: renderTools as unknown as ThriftSubtractToFitOptionsType['renderTools'],
protectThoughtIds,
renderCtx,
protectedToolNames,
shedRank,
// Passed explicitly even though these match the battery's own defaults — self-documentation of the
// flagship's id conventions at this call site (see the module header comment).
isEphemeralMessage: (m) => m.id.startsWith('__eph-'),
isSummaryMessage: (m) => m.id === '__compact-summary',
}
const trace: ThriftTraceResult = adk.subtractToFit(
ws as unknown as ThriftWorkingSetType,
contextWindow,
relevantToolNames,
options
)
return trace
}The battery's own selectRelevantTurns — invoked here with keepRecent newest turns always kept, plus a calibrated relevance floor over everything older — replaces this agent hand-rolling its own recency logic. The floor itself (RELEVANCE_FLOOR_MIN/MAX/CURVE, calibrated against a three-judge oracle over a 94-turn corpus) lives inside the battery, not this showcase's code — this agent is a consumer of that calibration, exactly the way an application is supposed to sit on top of a battery instead of re-deriving its constants. See the battery reference for the exact numbers and the calibration story.
Gemma's own model card is the cleanest proof this belongs in the pipeline, not behind a battery flag. Section 3, "No Thinking Content in History," is unambiguous: prior-turn reasoning must not be re-added before the next user turn, or the model's own multi-turn behavior degrades. @nhtio/adk doesn't need a thoughtSurfacing: 'none' option for this — it hands you the working set and the trim points, and you enforce a model-specific correctness rule exactly where the model documentation says to:
/**
* Gemma 4 model card, §3 Multi-Turn Conversations — "No Thinking Content in
* History": thoughts from previous model turns MUST NOT be re-added before the
* next user turn. This is a CORRECTNESS rule, enforced HERE in the pipeline, not
* with a battery flag — you decide what each dispatch sees. It is also pure
* token-thrift: prior-turn reasoning is the highest-volume, lowest-reuse content
* there is. The current turn's reasoning still surfaces live; it just never
* re-enters history.
*/
export const stripPriorTurnThoughts = (ws: WorkingSet): { dropped: number; tokens: number } => {
const tokens = ws.thoughts.reduce((n, t) => n + tok(t.content.toString()), 0)
const dropped = ws.thoughts.length
ws.thoughts = [] // none of last turn's thinking goes back into the window
return { dropped, tokens }
}Field note: the pass shed the one result the model just asked for
An early version measured every tool result's age off a single "oldest first" ranking, no this-turn exception — so a search result the model had requested seconds earlier, easily the highest-signal thing in the window, got evicted for being merely old, and the model immediately re-requested it. What looked like thrash was the subtractive pass sawing off the branch it was sitting on. The fix excludes this-turn tool results from the shed entirely; only prior-turn results are ever eligible.
The gate cascade owns the turn Showcase policy · built on ADK
Let the model decide for itself when an answer is done and it will call plenty of bad ones finished: an answer built on an unopened tool result, an invented citation, a silently abandoned plan, the same rejected sentence retried word for word. Every one of those is a documented failure this exact build produced before it had a check for it.
So the model doesn't get the final word. autoAck is off, and dispatchOutputPipeline decides instead, via a terminator with exactly four cases per iteration:
// dispatchOutputPipeline (PER iteration, after the model): the TURN TERMINATOR (autoAck is
// off — the harness owns when the turn ends). Four cases per iteration:
// 1. An answer tool fired (captured) → ack: cited answer / honest IDK ends the turn.
// 2. A non-answer tool ran this iteration (search, etc.) → keep looping to act on it.
// 3. Prose, classified as a doc ANSWER (1) but UNCITED → set needsCitation; do NOT ack —
// the next iteration nudges the model to re-issue via provide_answer.
// 4. Prose, classified conversational (0) → ack: a chat reply is a valid response.
// Backstop: at the hard cap, ack regardless so a stubborn model can't loop forever.An answer tool firing ends the turn. A non-answer tool (search, a computed lookup) keeps the loop running to act on it. Prose classified as a documentation answer but missing a real citation is held open and nudged — sent back with a corrective message, not just rejected — back toward provide_answer. Conversational prose is accepted as a valid reply on its own. There is no single hard iteration cap backstopping all of it — a stubborn model that keeps failing the same check for too long trips that check's own bounded limit (a duplicate-call hard stop, a stuck-prose limit, a no-progress limit), and each of those is the mechanism doing the actual work, not a generic last resort.
That bounded-vs-unbounded split is deliberate: a check stays unbounded — never gives up — when accepting its failure means shipping a fabrication. A check gets a bound — a hard limit past which the output is accepted as-is — when the alternative is looping forever with nothing changing, which helps nobody. Ahead of the citation/abstention ladder sit two deterministic, unbounded checks that fire before any classifier gets a vote. The first: did the model claim to answer while leaving a tool result — an artifact handle (a tool result the model has to explicitly open before it can use it) it requested this very turn — unread?
// READ-CONSUME INVARIANT (deterministic, unbounded). Whenever the model is trying to ANSWER (bare
// prose OR an answer tool fired), does it leave an unread artifact handle it produced this turn? If so,
// it must open it via an artifact_* reader before answering. We do NOT guard on artifact_* tool
// VISIBILITY: the forged readers are ephemeral + adapter-local (rendered into the prompt from the
// adapter's merged registry, never onto ctx.tools.visible()), so checking visible() was structurally
// always-false and permanently DISABLED this gate — the model was handed a readable handle it was never
// forced to read, and fabricated the value instead (the get_current_time "It is Wednesday" hallucination,
// caught only by the parroting canary → apology loop). The presence of an unread SpooledArtifact handle
// IMPLIES its readers were forged into the prompt this iteration; that is the availability signal. The
// tight-window concern (readers shed) is already covered: if the HANDLE itself was shed,
// collectUnreadResultHandles finds nothing → null → no demand → no deadlock (this-turn results are never
// budget-shed and artifact_* readers shed LAST; see agent_subtractive_pass.ts).
const unreadHandles = answering ? await collectUnreadResultHandles(calls) : null
// Shared unread-handle gate emission (used by BOTH the answer-tool path below and the prose path) — one
// first-person "read it now" thought, so a 2B commits to reading the handle as its next action.
const emitUnreadHandleGate = (handles: NonNullable<typeof unreadHandles>): void => {
const refs = handles.map((h) => `- ${h.tool} (callId=${h.callId})`).join('\n')
emitGate(
'unread-handle',
'Answered without reading a tool result it requested; sent the model back to read the handle',
'You answered without reading a tool result you already requested THIS TURN. It was not ' +
'inlined — it is waiting as an artifact handle, and you have not opened it. Do NOT describe ' +
'or guess its contents, and do NOT reply "I understand"/"I apologize" — that is not an answer. ' +
'READ it now with an artifact_* tool (artifact_head, or artifact_json_get with a path like ' +
'"$[*].excerpt" for a JSON result) using the callId below, THEN deliver the answer as a ' +
'provide_answer TOOL CALL citing it (not ' +
`chat prose, not "<call:…" text):\n${refs}`,
`I have not actually opened the tool result yet (${handles[0]?.tool}, callId=${handles[0]?.callId}). My NEXT output is an artifact_* call to read it — not prose, not an apology. Then I emit a provide_answer TOOL CALL citing it.`
)
}This is not a guess. collectUnreadResultHandles checks structurally which handles were opened via an artifact_* reader and which weren't — no classifier judgment call, no bound to hit and give up on. It catches the exact failure a plain citation check misses: the model requests a search, gets back a handle, and answers from parametric memory without ever opening what it asked for. Deliberately unbounded, because "you're allowed to fake it after N tries" is not a real fix for fabricating a value.
Field note: a model answering from a real result while it sits unread — proven false, not assumed
On a "what day of the week is it?" turn, get_current_time returned a real result and the model answered "I don't have the current date and time information" anyway — a false abstention, since the answer had arrived that same turn. The unread-handle gate runs first for a reason: an unopened handle is the one legitimate case where the model genuinely never saw the result, so it's checked and cleared before the false-abstention gate runs. By the time the abstention gate fires, the result is provably in context — the refusal has no honest excuse left.
The second: plan-conformance. The plan named tools the model committed to at turn-open — did it actually call them?
// The plan names tools the worker committed to. DETERMINISTIC signal: which planned tools were
// never SUCCESSFULLY called this turn. `planToolsSatisfied` (none uncalled) retires the
// plan-violation correction mid-loop; `uncalledPlanTools` is fed to the semi-deterministic
// conformance classifier at close to make a real violation far more likely to be caught.
const calledToolNames = new Set(
calls.filter((c) => !(c as { isError?: boolean }).isError).map((c) => c.tool)
)
const uncalledPlanTools = turnPlan
? turnPlan.toolsToUse.filter((t) => !calledToolNames.has(t))
: []
const planToolsSatisfied = uncalledPlanTools.length === 0And the parroting check — comparing a prose reply against everything the model is actually shown this turn — carries two deliberate, documented exemptions, because a naive version of this check creates its own failure mode:
// PARROTING CHECK (deterministic). When the model replied in PROSE, compare that reply against how
// every piece of context was rendered TO THE MODEL — each prior message + each tool result (the
// model-facing preview, the exact text it saw) — via Levenshtein similarity. A reply that closely
// mirrors any single context entry is a parrot, not an answer (the observed leak: echoing
// provide_answer's "Answer delivered…" confirmation, a rejection notice, or a search blob). Gathered
// only on the prose path; skipped otherwise (a tool call is never a parrot).
// TOOL_COMPUTED EXEMPTION. For a tool_computed turn that already has a successful tool result, the
// model's job is to STATE that computed VALUE — which necessarily resembles the tool-result text it
// just read (e.g. get_current_time → artifact_head → "It is Sunday, July 5, 2026" mirrors the
// "UTC: Sunday, July 5, 2026 …" the reader returned). Comparing the answer against the TOOL RESULT
// previews therefore mislabels the correct readback as parroting, and the loop the whole flagship was
// built to fix reappears: the model states the right value, parroting rejects it, it re-emits, ~50×.
// (PROVEN in the Node/Ollama real-loop dump: turn "what day is it?" = 104 dispatches, get_current_time
// → artifact_head → "It is Sunday" at dispatch 7, then ~50 rejected restatements.) So on a
// tool_computed turn WITH a successful result, do NOT compare against the tool-result previews. We STILL
// compare against prior MESSAGES — echoing a nudge / system directive is parroting on any turn kind.
const parrotExemptToolResults =
turnPlan?.answerKind === 'tool_computed' && hasSuccessfulResult
// SELF-ECHO EXEMPTION. The parroting check must compare the answer against EXTERNAL context the model
// might be echoing (a nudge/directive, a tool result, seeded prior-turn history) — NOT against the
// model's OWN this-turn prior generations. PROVEN on the wire (Node/Ollama T7 parroting-match tap):
// 26/28 parroting matches were the model's own earlier prose answer THIS turn, 0 were RAG. Mechanism:
// an answer gets rejected by some OTHER gate (incomplete/needs-citation) → flushed into turnMessages →
// the next attempt at the SAME question is naturally similar → parroting matches the prior attempt →
// rejected → compounds into the doc-turn loop (23-58 dispatches). Answering a question similarly to your
// own prior attempt is not parroting. Exclude assistant-role messages that are THIS-turn generations
// (assistant role AND not in seededIds — the same discriminator the self-echo cap uses above).
// [[fence_nonce_id_miscitation]] residual; [[tool_computed_loop_is_self_apology_accumulation]] class.
const parrotedContext =
proseNoTool && unreadHandles === null
? await (async (): Promise<boolean> => {
const ctxStrings: string[] = []
for (const m of ctx.turnMessages) {
if (
(m as { role?: string }).role === 'assistant' &&
!seededIds.has((m as { id?: string }).id ?? '')
) {
continue // the model's own this-turn re-attempt — not external context to parrot
}
const c = (m as { content?: { toString(): string } }).content
if (c) ctxStrings.push(c.toString())
}
if (!parrotExemptToolResults) {
for (const c of calls) {
const preview = await readModelFacingPreview(
c as Parameters<typeof readModelFacingPreview>[0]
)
if (preview) ctxStrings.push(preview)
}
}
return looksLikeParroting(proseAnswer, ctxStrings)
})()
: falseWithout the tool_computed exemption, a model correctly stating a value it just read — "it is Sunday, July 5th" after reading a get_current_time result that says almost the same thing — gets rejected for resembling the very result it was told to state, and re-emits the correct answer roughly fifty times before the turn gives up. Without the self-echo exemption, a rejected answer that gets a second, similar-sounding attempt reads as parroting itself — proven on the wire at 26 of 28 observed parroting matches being the model's own prior this-turn answer, not leaked context. Both exist because the naive gate manufactured the exact loop it was built to prevent.
Field note: a 2B invents citation markup we never asked it for
A conversational follow-up once ended with a fabricated <cite>Assembly</cite> tag rendered as literal text — <cite> is almost certainly something the model picked up in pretraining, but nothing in this system asked for it or handed it that form; a real citation only ever travels through provide_answer's sources[], never inline XML. There was no detector for a failure mode that had never been named, so nothing caught it. Our fix is a looksLikeFakeCitation check plus a bounded gate that rejects and re-nudges in the model's own voice to re-answer through the real citation path — the same doctrine as every gate in this build: name the failure precisely, then build the one check that catches exactly that shape.
Artifact handles: the tool loop's actual currency
Call a real search tool and the result can be 500KB of JSON. Paste that into the context window as-is and it isn't "a big tool result" anymore — it's your entire budget, gone, on one call.
So a tool result never lands in the window as a body. The battery spool-wraps it — stores the full result off to the side, keeping only a reference in context — into a SpooledArtifact, and renders a compact handle plus a set of forged artifact_* reader tools (generated on the fly for this one result, not part of the agent's fixed tool list) the model calls to pull only the slice it needs. A search result that would cost the whole budget as inline JSON costs a few dozen tokens as a handle.
The library's default reader order leads with a metadata-only tool — artifact_json_type, which returns just the string "json" and teaches the model nothing — so a model that reaches for the first tool listed flails, concludes it "couldn't find" the content, and abstains on a result that was sitting right there. This agent doesn't change the library default; it reorders the presentation through the battery's own render override, so a content-returning reader leads instead:
export const ARTIFACT_READER_PRIORITY = [
'artifact_json_get',
'artifact_json_keys',
'artifact_head',
'artifact_cat',
'artifact_json_filter',
'artifact_json_pluck',
'artifact_json_slice',
'artifact_grep',
'artifact_tail',
'artifact_line_count',
'artifact_byte_length',
'artifact_estimate_tokens',
'artifact_json_length',
'artifact_json_type',
]Same tools, same callId, same metadata — only the line order the model reads changes. And the dialog's "what did the model actually see" chip isn't an approximation: it calls the exact renderer the executor used, so what you inspect is byte-identical to the prompt, not a reconstruction that might quietly drift from the truth:
export async function readModelFacingPreview(tc: {
id?: string
tool?: string
inline?: boolean
fromArtifactTool?: boolean
results?: unknown
}): Promise<string | undefined> {
const r = tc.results
// Handle-rendered: spooled body the model did NOT get inline (matches the battery's render gate).
// looksLikeSpooledArtifact is the battery's own structural check (cross-realm-safe).
if (looksLikeSpooledArtifact(r) && tc.inline === false && !tc.fromArtifactTool) {
let byteLength = 0
let lineCount = 0
try {
byteLength = await (r as { byteLength: () => Promise<number> }).byteLength()
} catch {
/* best-effort metadata */
}
try {
lineCount = await (r as { lineCount: () => Promise<number> }).lineCount()
} catch {
/* best-effort metadata */
}
// The EXACT text the executor put in front of the model — same renderer (the json-first reorder
// this agent passes to the adapter via helpers.renderArtifactHandleBody), same fields.
return renderArtifactHandleBodyJsonFirst({
callId: tc.id ?? '',
artifact: r,
byteLength,
lineCount,
})
}
return readToolResultPreview(r)
}That equivalence is a discipline, not a convenience. A showcase that shows you a reconstruction of what the model saw is a showcase that can lie to you the moment the real renderer diverges from the approximation. If you're going to read the wire, read the actual wire.
Field note: "RAG is poison" was never RAG — it was a spool footgun
Every tool result once came back as a literal Failed to execute 'getFileHandle' … Name is not allowed string, and the model dutifully "answered" from it — reading like the retrieval layer itself was garbage in, garbage out. The actual defect was one call site: the OPFS-backed spool store took a keyPrefix it treated as a directory path, but a / is illegal in an OPFS filename, so every single spooled result failed identically. Retrieval itself was fine the whole time — top hit scored 0.58. The lesson generalizes: when a tool-using agent gives a confident wrong answer, read the actual tool result before blaming the model or the retrieval layer for it.
Surviving the GPU Showcase policy · built on ADK
This runs on real WebGPU hardware, which means the device gets lost — not a bug here, just what happens on real drivers: a TDR (the driver's own timeout, killing and resetting a GPU task it assumes has hung) or an outright GPU-process crash. Either way, that's device loss — the GPU handle this agent was using is gone, the engine is dead, nothing in flight will complete. Our fix is structural: the LiteRT-LM engine runs inside a disposable classic Web Worker (an older, simpler background-thread style, not the newer "module worker" kind, because the Emscripten glue it depends on calls importScripts(), which a module worker forbids), so recycle() becomes worker.terminate() plus a fresh worker, instead of a same-thread dispose() that can't fully claw back a wedged GPU context:
// Host-side proxy for the LiteRT-LM engine running in a disposable Web Worker.
//
// This is the injected `createEngine` factory (LiteRtLmAdapterOptions.createEngine seam). It returns an
// object that STRUCTURALLY implements the interface the adapter calls — `engine.createConversation(config)`
// → conversation with `sendMessageStreaming()`/`sendMessage()`/`cancel()`, and `engine.delete()` — but
// forwards every call over postMessage to the real engine in litert_lm_worker.ts. ZERO changes to the
// battery: the adapter body is untouched; it just drives this proxy instead of a same-thread Engine.
//
// The recycle-as-total-teardown property falls out for free: adapter.dispose() awaits engine.delete()
// (which here posts `delete` AND terminates the worker), and adapter.preload() calls this factory again
// (which spawns a FRESH worker). So the existing `recycle() = dispose()+preload()` becomes
// `worker.terminate()` + respawn with no adapter change — the whole point of the migration.
//
// Envelope: id-correlated one-shot promises for init/createConversation/send/cancel/delete (mirrors
// agent_sqlite_worker.ts), PLUS a PERSISTENT streamId-keyed map for streaming generation (which is
// high-frequency, not one-shot). Errors cross as STRINGS and are reconstructed into Error objects here so
// the adapter's toLiteRtGenerationError classification (E_LITERT_LM_CONTEXT_OVERFLOW / GPU-OOM) still works
// on the message signature.case 'ev:deviceLost': {
if (this.#deviceLost) return
this.#deviceLost = true
// Fail every in-flight call + stream — the engine is dead; nothing will complete.
const lostErr = `WebGPU device lost: ${event.reason}`
for (const [, p] of this.#pending) p.reject(new Error(lostErr))
this.#pending.clear()
for (const [, s] of this.#streams) s.error(lostErr)
this.#streams.clear()
this.#hooks.onDeviceLost?.({ reason: event.reason })
return
}Field note: the OOM is an honest memory signal, not a bug to hide
Failed to allocate memory for buffer mapping means exactly what it says: this context size doesn't fit the memory actually available, full stop. Even on LiteRT-LM, which clears the harder software ceiling that drove the runtime switch below, a large enough context still doesn't fit — the wall moved and got machine-dependent, it didn't vanish. The lesson is engine-agnostic: a smaller weight format buys headroom, not immunity, so the durable fix isn't a smaller number, it's product honesty. Surface the OOM as a plain-language banner ("not enough GPU for this context-window size — reduce it and retry") with a real retry path, instead of pretending a ceiling isn't there.
A single lost device is recoverable — a fresh worker, a fresh engine, the conversation resumes because history lives in SQLite, not in the dead worker's memory. But Chrome blocks new GPU adapters for a stretch after two GPU-process crashes close together, so a naive "always respawn" policy would hang forever on the second loss pretending it's still trying. The escalation ladder is a sliding window, not a single flag:
// GPU device-loss escalation policy for the worker-hosted LiteRT engine.
//
// When the worker's sentinel WebGPU device is lost (driver TDR / GPU-process crash), the engine is dead
// and no in-flight or future turn can complete on that worker. Recovery is a LADDER:
//
// 1st loss in the window → RESPAWN the worker (worker.terminate() + a fresh worker/engine). A single
// transient TDR is recoverable this way — a clean GPU context from scratch.
// 2nd loss in the window → full PAGE RELOAD. Chrome blocks new GPU adapters after 2 GPU-process crashes
// in ~2 min ("device lost" persists), so only a reload clears it. Conversation
// history lives in SQLite, so a reload is LOSSLESS — the user resumes the thread.
//
// The window is a sliding 120s: two losses spaced far apart are each treated as a first (transient) loss,
// not an escalation, so an occasional TDR over a multi-hour session never forces a reload.
//
// This module is pure policy (no DOM beyond the injected reload fn) so it is unit-testable. The host
// (agent_runtime.ts) owns the respawn action and passes reload; the dialog surfaces the reload with a banner.
/** The escalation the policy decided for a single device-loss event. */
export type GpuLossAction = 'respawn' | 'reload'
/** How long two losses must fall within to count as an escalation (Chrome's ~2-min GPU-crash window). */
export const GPU_LOSS_WINDOW_MS = 120_000
export interface GpuLossPolicyOptions {
/** Sliding window; a loss older than this no longer counts toward escalation. Default 120_000. */
windowMs?: number
/** Injected clock (for tests). Default `Date.now`. */
now?: () => number
}
/**
* Sliding-window device-loss escalation decider.
*
* @remarks
* Call {@link GpuLossPolicy.record} once per `ev:deviceLost`. It returns `'respawn'` for the first loss in
* the window and `'reload'` for a second (or later) loss still inside the window. Prunes losses older than
* the window so the counter reflects only the recent burst.
*/
export class GpuLossPolicy {
readonly #windowMs: number
readonly #now: () => number
#timestamps: number[] = []
constructor(options: GpuLossPolicyOptions = {}) {
this.#windowMs = options.windowMs ?? GPU_LOSS_WINDOW_MS
this.#now = options.now ?? Date.now
}
/**
* Record a device-loss event and return the escalation action.
*
* @returns `'respawn'` for the first loss in the current window, `'reload'` for a repeat inside it.
*/
record(): GpuLossAction {
const t = this.#now()
// Drop losses that have aged out of the window.
this.#timestamps = this.#timestamps.filter((ts) => t - ts < this.#windowMs)
this.#timestamps.push(t)
return this.#timestamps.length >= 2 ? 'reload' : 'respawn'
}
/** Number of losses currently inside the window (observability/tests). */
get recentCount(): number {
const t = this.#now()
return this.#timestamps.filter((ts) => t - ts < this.#windowMs).length
}
/** Clear the loss history (e.g. after a clean recovery + a settled session). */
reset(): void {
this.#timestamps = []
}
}One loss inside the 120-second window respawns; a second loss inside that same window escalates straight to a full page reload — lossless, because the conversation is durable. Two losses far apart are each treated as a first, transient loss, so an occasional TDR over a multi-hour session never forces a reload it didn't need. This is the least glamorous code on the page and the one most likely to be the reason a demo either recovers gracefully or silently dies on stage.
The runtime we didn't keep: maturity lost to fit
Device loss is the GPU being hostile in real time. There is a second kind of hostility, quieter, that shaped the single biggest low-level decision in this build: how much the engine can hold before it falls over — and it is the reason this agent runs on the runtime it does.
This agent runs on LiteRT-LM, an experimental Google runtime that, in the browser, supports exactly two models: Gemma-4 E2B and E4B. That is the whole allow-list. It is younger, narrower, and less battle-tested than the obvious alternative — transformers.js on ONNX Runtime Web, which drives thirteen model families in this same codebase, has years of production use behind it, and was where this agent actually started.
We moved off the mature one on purpose. Here is the receipt, because this was not a taste call.
ONNX Runtime Web executes its graph inside a wasm32 module, and a 32-bit address space caps that module's linear memory at 4 GiB — a software ceiling that has nothing to do with your GPU. The dev machine has ~39 GiB of usable Metal buffer space; the runtime advertised 4. That heap stages everything the loop touches that isn't resident weights: I/O tensors, kernel dispatch, shape metadata, the growing KV cache, every tool-result string folded back in. On a heavy multi-tool turn — search, read the result, compute, cite — that traffic accumulates across iterations and floods the 4 GiB heap. The agent OOM'd at an 8k window. It also OOM'd at 4k. The wall was never the context size; it was cumulative per-loop traffic through a 32-bit heap.
We pulled the real levers first, because switching runtimes is not where you start. Dropping weights from q4 to q4f16 recovered memory. Pinning the KV cache off the wasm heap onto GPU buffers recovered more. Both helped; neither was enough. The honest A/B is in the field note below. What those levers bought was a later OOM on the same turn — headroom, not immunity. For a light single-tool turn that might have been the end of it; for the heavy multi-tool agent we were actually shipping, "later" still means "on stage."
Field note: the KV-cache lever was necessary but not sufficient
By default, transformers.js parks the Gemma-4 KV cache on the wasm32 heap — the single largest growing consumer across a dispatch loop. There is an auto-pin path meant to move it onto GPU buffers, but it no-ops on Gemma-4's config shape, so it silently did nothing until we passed the preferredOutputLocation map explicitly (a live probe confirmed the tensors moving from cpu to gpu-buffer). We shipped it as a battery default. Then we ran the real agent, at the 8k window that broke it, pin off vs. pin on: both still OOM'd. Emptying the KV cache off the heap was correct and free, and it was not the wall. The rest of the per-loop traffic floods 4 GiB on its own. That result — a good, shipped optimization that does not move the wall far enough — is exactly why the decision came down to the runtime and not to one more knob.
LiteRT-LM does not route through ORT's wasm32 chokepoint. Multi-turn Gemma that reliably OOM'd on ONNX simply ran. So the trade was stated plainly: give up thirteen model families for two, and years of maturity for an experimental engine, in exchange for the one thing the mature stack could not give us — the heavy agent completing turns consistently, where we intended to run it, a browser tab on real WebGPU. Maturity is a real asset. It is not the asset that mattered here. Not every runtime is the right runtime for your application, and the most mature option is not automatically the right tool for the job — a narrow engine that works where you deploy beats a broad one that doesn't.
And here is the part that should have been expensive and wasn't: the switch was cheap. The model runtime is an executor behind ADK's adapter contract, and both engines ship as batteries implementing it — TransformersJsAdapter and LiteRtLmAdapter. Swapping one for the other is swapping which adapter the TurnRunner is constructed with; the planner, the gate cascade, the subtractive pass, the artifact handles, the persistence layer — none of it knew or cared which engine generated the tokens. The cost was not zero — LiteRT's stashed session state needed one adapter-specific hook in context assembly — but a runtime migration that would sink a tightly-coupled agent came down to a battery swap plus one seam. ADK does not marry you to a model runtime; that is the entire point of the boundary. We changed the hardest, lowest-level dependency in the stack late in the build, and everything above it held.
The verdict: a judged matrix
TL;DR — five findings from five cells
- The crossover is the thesis: this E2B with thrift at 8k (0.82) out-scores a much larger open-weights reasoning model on naive recency at 128k (0.08).
- Naive recency is the only strategy that ever collapses — and also the most expensive from 32k up, because re-sending everything every dispatch is the cost model (176.3M tokens at 1M vs. thrift's 65.9M, for tied quality).
- Thrift never collapses and is the cheapest or near-cheapest arm on every small-model cell — zero extra model calls, fewest dispatches everywhere but kimi.
- Compact genuinely wins twice (both 128k cells, real context pressure + a summarizer budget that pays off) and its summary undercuts even thrift's token bill on the reasoning cells — paid compression is a real strategy, not a strawman.
- No strategy wins everywhere. The arms trade places by cell; picking one is a workload decision, which is why both ship as batteries and neither is a default.
What this matrix is, and isn't: five cells don't cover every workload, and they're not meant to. The matrix exists to answer one question — is the thrift method worth shipping as a battery? — and the answer it gives is yes, not as a silver bullet, but as a competent generalist: never the strategy that collapses, paying benefits at both ends of the spectrum — the capacity-starved edge (an E2B at 8k) and the capability-rich cloud (a 1M window at 2.7× less cost for tied quality).
This agent's context strategy — thrift, the subtractive pass documented above — is evaluated head-to-head against a paid summarizing alternative (compact) and a naive-recency baseline across five model/window cells, on a shared 94-turn stress corpus, scored blind by an LLM committee against a 0–3 doc-verified rubric.
Here's how to read the numbers below: each turn is scored 0–3 by an LLM judge that never sees which strategy produced the answer, checked against the actual docs (0 = wrong or unsupported, 3 = fully correct and cited), then averaged across the 94-turn run — so 0.82 means "most turns landed between partial and full credit," not a percentage. Quality alone is half the bill, so each cell also carries the effort it took to earn that score: Σ is the arm's total token bill for the whole 94-turn run (input + output, summed from the engine's own per-dispatch counts in the run dumps, dedup-swept and reconciled against the harness reports — compact's Σ includes its summarizer dispatches), d/t is model dispatches per answered turn (from the per-turn harness reports; all cells 94/94 settled, zero errored turns), and compact's cells name its summarizer overhead separately. Score higher-is-better, Σ and dispatches lower-is-better, same corpus, same judges, every column:
| model / window | thrift | compact | naive | committee |
|---|---|---|---|---|
| gemma e2b @ 8k | 0.82 Σ 3.5M · 5.8 d/t | 0.51 Σ 4.1M · 7.1 d/t +85 summ · ~375k tok | 0.75 Σ 3.6M · 5.9 d/t | 3-judge |
| gemma 31b @ 32k | 1.55 Σ 12.2M · 5.3 d/t | 1.49 Σ 14.5M · 7.8 d/t +86 summ · ~412k tok | 1.65 Σ 16.5M · 6.2 d/t | 2-judge (final) |
| gemma 31b @ 128k | 1.35 Σ 38.4M · 6.4 d/t | 1.48 Σ ≥39.6M† · 10.7 d/t +80 summ · ~394k tok | 1.15 Σ 65.6M · 6.8 d/t | 3-judge |
| kimi-k2.5 @ 128k | 1.13 Σ 46.6M · 12.3 d/t | 1.48 Σ 23.3M · 11.5 d/t +89 summ · ~546k tok | 0.08 Σ 58.1M · 11.4 d/t | 3-judge |
| gemini-flash @ 1M | 1.14 Σ 65.9M · 5.1 d/t | 1.34 Σ 41.0M · 11.9 d/t 0 summ (never fired) | 1.13 Σ 176.3M · 9.7 d/t | 3-judge |
† Lower bound: this arm's dump lost 74 of 1,006 dispatch records to a killed harness wedge mid-run; the sum covers the 932 recovered records. Everything else reconciles against its harness report within 1%.
One row is a 2-judge pool
Four rows are 3-judge panels (gpt-5.5, opus, deepseek — three full 94-turn solo passes per row, per-turn scores merged, same blind rubric). The gemma-31b @ 32k row pools two judges and is final as-is: its thrift run transcript aged out of temporary storage before the third pass could run, and because judges score each turn as a blind comparison across all three arms, re-running the arm would invalidate the existing scores and force a full three-judge re-evaluation of the cell. No row mixes committees — every number in a row comes from the same judges.
🎯 The whole argument, in one number
Read the crossover first: naive recency scores 0.08 on kimi-k2.5 @ 128k — a reasoning model trapped in its own scaffolding by bloated, unfiltered context — while the Gemma-4 E2B, an effective-2B model at an 8k window with thrift, scores 0.82. A disciplined effective-2B out-answers a much larger open-weights reasoning model left to naive recency. That is not a fluke of one cell; it is the entire thesis of this page rendered as a number.
Ceiling honesty matters too: the 31b model beats the 2b model at every single arm, every window. Nothing here claims a small model matches a large one on raw capability — it claims a small model, disciplined, earns more than an undisciplined large one, a different and more useful claim. Compact tops two cells outright — kimi-k2.5 @ 128k (1.48 vs. thrift's 1.13) and the 128k gemma-31b control (1.48 vs. 1.35) — real context pressure with a paid summarizer budget, exactly the profile where paying for compression earns its keep. Naive even takes the 32k control outright (1.65 vs. thrift's 1.55) — but it's still the only strategy that ever collapses, cratering to 0.08 on kimi-k2.5 @ 128k. Thrift never collapses anywhere; it just doesn't always win.
Now read the effort columns against the score they buy, because that's where the comparison actually bites. Thrift is the lightest arm on every cell but kimi (where all three arms thrash roughly equally — reasoning-model dispatch counts are model personality, not strategy). Compact's two quality wins are not free wins: on the 128k control it pays 10.7 dispatches per turn against thrift's 6.4 plus 80 summarizer calls to earn its 1.48-vs-1.35 edge — though on the reasoning cells its summary does real token work, undercutting thrift's Σ outright (23.3M vs. 46.6M on kimi) by replacing long verbatim history with a short recap. The headline total from the biggest cell: at gemini-flash @ 1M across the full 94-turn run, thrift moves 65.9M tokens against naive's 176.3M — 2.7× less, for the same quality (1.14 vs. 1.13, a statistical tie). And naive's Σ is the accumulation story told in money: it never pays a summarizer and runs midfield on dispatches, yet posts the biggest bill on every cell from 32k up, because re-sending everything every dispatch is the cost model. For the cloud cell, these sums were cross-checked against the serving gateway's own per-request accounting: request counts match the dump within retry noise, and request bytes track the token sums at a consistent bytes-per-token ratio across all three arms — two independent ledgers telling the same story.
Field note: the run where every strategy scored the same — because they secretly were the same
An earlier phase of this matrix was much less interesting than it should have been: on the 31b @ 32k cell, thrift, compact, and naive all scored within about a tenth of a point of each other (gpt judge, 90 turns: thrift 1.46, compact 1.33, naive 1.43). A dead heat. For a moment that read as "the strategy barely matters."
It was a bug, and the bug is the lesson. All three arms shed through the same subtractive machinery — they are supposed to differ only in what they cut first (thrift by relevance, naive by age, compact by keeping a running summary and shedding the rest). But compact's summary was timestamped at the epoch so it would render at the head of history — which also made it the oldest turn, so the relevance pass evicted it first under pressure. Compact wasn't compacting. It had quietly degraded into thrift-with-recent-turns: compaction wearing thrift's clothes. The margins were narrow because we were partly measuring thrift against itself.
Fixing it — making the summary inviolable, so compact either fits it or honestly crashes the turn — is what produced the real numbers in the table above, where compact and thrift finally diverge. The honest takeaway is not "combining strategies is always better," which this run does not prove. It is narrower and more durable: there is no single right context strategy, the arms genuinely trade places by cell, and the one moment compaction stopped losing was the moment it accidentally got relevance-shedding underneath it. That is a pointer, not a proof — a well-built agent probably wants a bounded summary layered on top of relevance shedding, which is a hybrid we have not measured and did not ship. The receipt here is what the bug cost us: a result that looked like parity and was really an apparatus measuring its own reflection.
Two cells were run informally and dropped from the table rather than reported as data: a single-credential provider substitution and one alternate-hosting arm for the tightest cell, both excluded as noise rather than signal. Full methodology, the complete corpus, and the reasoning behind every threshold live on the Context Batteries hub — this table is the headline, that page is the receipts.
Use it yourself
Everything above composes from shipped, importable pieces, not showcase-only glue:
@nhtio/adk/batteries/context/thrift— the subtractive pass this agent runs every dispatch. See the battery reference for the full API and the calibrated relevance floor.@nhtio/adk/batteries/context/compact— the paid summarizing alternative this agent is evaluated against. See the battery reference for when it actually earns its cost.@nhtio/adk/shims— the async-resolver seam that lets this agent bind an ADK build without importing ADK's source into the docs site's own module graph; the same seam any consumer uses to load a fetched or worker-hosted ADK build.
The full source behind every embed on this page — the planner, the gate cascade, the artifact-handle renderer, the worker proxy, the loss policy, and the harness that wires all four pipelines together — is walked end to end on the source companion page.