---
url: 'https://adk.nht.io/showcase/punching-above-its-weights.md'
description: >-
  A real docs agent running entirely in the browser on Gemma-4 E2B/E4B: a tool
  loop, a planner, a gate cascade, SQLite persistence, per-dispatch context
  shedding. The point isn't that the small model got smarter — it's that the
  harness stopped letting it freelance.
---

# Punching Above Its Weights

## LLM summary — Punching Above Its Weights showcase

* The live agent on this site (the floating Ask ADK launcher, bottom-right on every page) is this build: Gemma-4 E2B/E4B via LiteRT-LM on WebGPU, real tool-calling, a planner book-end, a cascade of deterministic output gates, artifact-handle tool results, SQLite/Kysely persistence, and a subtractive per-dispatch context pass. It supersedes the earlier tool-less, synthetic-RAG Ask ADK design (`/showcase/ask-adk`) — not by being "more honest" (Ask ADK already shipped its receipts), but by being a genuinely more complete agent: a real tool loop instead of injected retrieval, a planner that commits to a shape before generating, and a full persistence layer.
* Built on `@nhtio/adk`'s [`TurnRunner`](https://adk.nht.io/api/@nhtio/adk/turn_runner/classes/TurnRunner): four pipelines (`turnInputPipeline`, `dispatchInputPipeline`, `dispatchOutputPipeline`, persistence callbacks) split turn-scoped work from per-iteration work.
* A dedicated planner dispatch runs at turn open, commits `answer_kind`/`tools_to_use`/`answer_scope` as one `make_plan` tool call, and renders that commitment into an evaluatable [`Thought`](https://adk.nht.io/api/@nhtio/adk/common/classes/Thought) the worker reads every iteration — re-evaluated live against whichever tools the subtractive pass actually kept, so a plan never demands a tool the budget shed.
* The subtractive per-dispatch pass — the same `@nhtio/adk/batteries/context/thrift` battery from the Token Thrift showcase — shrinks a large working set to the active window, in a fixed shed order, before every generation.
* `dispatchOutputPipeline` owns turn completion (`autoAck: false`): a four-case terminator plus a ladder of deterministic and semi-deterministic gates — unread-handle, parroting, plan-conformance, duplicate-call, false-abstention, incomplete-answer, and more — decide whether an iteration's output is allowed to end the turn.
* Tool results are spool-wrapped [`SpooledArtifact`](https://adk.nht.io/api/@nhtio/adk/spooled_artifact/classes/SpooledArtifact) handles the model reads with forged `artifact_*` readers, reordered so a content reader (not a metadata reader) is listed first.
* The model runtime is an ADK executor behind the adapter contract. This build started on transformers.js/ONNX Runtime Web (mature, thirteen model families) but ONNX's wasm32 4 GiB heap OOM'd the heavy multi-tool agent at both 4k and 8k windows; `q4f16` weights and pinning the KV cache to GPU buffers bought headroom but not immunity. It moved to the less mature, Gemma-only LiteRT-LM because that engine avoids the wasm32 chokepoint and ran the same turns consistently in-browser — choosing the runtime that fit the deployment over the more mature one. The swap was a battery change ([`TransformersJsAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/transformers_js/adapter/classes/TransformersJsAdapter) → [`LiteRtLmAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/litert_lm/adapter/classes/LiteRtLmAdapter)) plus one adapter-specific seam; the planner, gates, context pass, and persistence were untouched.
* WebGPU device loss (TDR, driver crash) is handled by running the engine in a disposable classic Web Worker: `terminate()` + respawn recovers a first loss; a second loss inside a sliding 2-minute window escalates to a full page reload, because Chrome blocks new GPU adapters after two crashes that close together.
* Evaluated head-to-head against a compacting strategy and a naive-recency baseline across five model/window cells on a shared 94-turn corpus, judged blind by an LLM committee (four cells 3-judge; one cell a final 2-judge gpt-5.5+opus pool — its thrift run transcript aged out of temporary storage before a third pass could run), plus per-cell dispatch counts and summarizer overhead from the same runs. Full numbers below.

::: info This showcase supersedes an earlier agent design
This site's live agent used to be the tool-less, synthetic-RAG 3B documented on
[`/showcase/ask-adk`](/showcase/ask-adk). That page is not wrong and it is not being walked back — every
technique it documents shipped and works. What changed is completeness. This build replaces injected
retrieval with a real tool loop, adds a planner that commits to a turn's shape before the model ever
generates, adds a cascade of gates that own turn completion instead of a single citation check, and adds
durable SQLite/Kysely persistence where the 3B had none. Ask ADK remains on the site as a study of the
design it replaced — read it for the case for synthetic RAG on a tool-less model. This page is the case
for a real tool-using agent on a hostile budget.
:::

This agent runs a Gemma-4 **E2B** — an *effective-2B* model, engineered to run at a two-billion-parameter
footprint — in a browser tab, on your GPU, driving a real multi-step tool loop (search, read, compute, cite)
inside a context window you can shrink live, mid-conversation. No backend. No API key. That is the agent.
This page is how it is built.

Here is what happens if you build that agent the way every tutorial implies you can — hand the model some
tools, loop until it answers. You ask it what day it is. It calls `get_current_time`, a perfectly good tool,
and gets back a perfectly good result. Then it calls `get_current_time` again. Same arguments. Then again.
The result it needs is sitting right there in its context, and the model reads straight past it, forever,
because nothing in a two-billion-parameter checkpoint knows that *having* an answer and *using* one are
supposed to be connected. Or it stops looping — and confidently tells you it "doesn't have access to the
current date," with the date literally in front of it. Or it answers a documentation question with a page
path it invented, formatted exactly like a real one. None of this is exotic. If you have ever wired a small
model to tools, you have watched some version of it, probably within the first ten minutes.

The standard responses are to prompt harder ("You MUST use the tool result…") or to rent a bigger model and
hope. Prompting harder fails because a small model under context pressure drops instructions exactly the way
you drop the fourth thing on a mental to-do list. The bigger model mostly works — which is why almost nobody
builds the machinery this page describes — but "mostly" is doing heavy lifting, and now every message costs
real money for failure modes that never actually went away. They just got rarer and better-worded.

This build takes the third option: treat every one of those failures as a *missing mechanism*, and build the
mechanism. Not asked for in the prompt — enforced in code, outside the model, where it can be read, tested,
and fixed.

::: tip TL;DR — the five mechanisms
Nothing here works because the model got smarter. Five pieces of scaffolding do the actual work:

* **The model answers before deciding what it's answering** → so a **planner** runs first, committing to the
  turn's shape — what kind of answer, which tools, how long — before a single token of the real answer is
  generated.
* **The context window fills with stale, low-value tokens** → so a **subtractive pass** starts every dispatch
  from an oversized working set and cuts, by measured token count, until it fits the window — best content
  survives, never a mid-sentence truncation.
* **The model emits answers that shouldn't count** (unread results, echoed corrections, plan violations) → so
  a **gate cascade** inspects every candidate answer and refuses to let the turn end until the output is
  actually acceptable — a mechanism, not a system prompt asking nicely.
* **One tool result can eat the whole budget** → so results arrive as **artifact handles**: a 500KB search
  result costs a few dozen tokens until the model explicitly reads the part it needs.
* **The GPU dies mid-conversation** (this is WebGPU on real hardware) → so a **recovery ladder** respawns the
  model in a fresh worker and the conversation survives, losing at most the turn in flight.

None of this is the model being disciplined. It is scaffolding, in code, that makes a small, cheap,
undisciplined model behave like it has adult supervision.
:::

And the size of that lever is measured, not asserted. In the head-to-head evaluation at the bottom of this
page, this same E2B on an 8k window — context managed subtractively — scores **0.82** on a blind-judged
94-turn corpus, while a far larger open-weights reasoning model on a 128k window, fed by naive
append-everything context, collapses to **0.08** on the same corpus, same judges. The small disciplined
agent out-answers the large undisciplined one outright. It does not close the raw-capability gap — the
larger model still wins arm-for-arm when both are disciplined — it proves something more useful: the
scaffolding is worth more than the parameters the moment the parameters are left to fend for themselves. The
model sets the ceiling; the scaffolding decides whether you get anywhere near it. The rails are not a
nice-to-have layered on top of a good agent. They *are* the agent.

How to read this page: the [live demo](#see-it-one-model-a-window-you-control) below lets you watch the
context discipline work with one slider — start there. The sections after it walk each mechanism in the
order a turn actually executes, with the real source embedded, and close with the
[judged numbers](#the-verdict-a-judged-matrix). Everything is a pointer to code you can read, not a promise.

::: warning What this agent is mechanically prevented from doing

* **Cannot commit an uncited doc answer — within the gate's retry budget.** Prose classified as a
  documentation answer without a real citation is not acked; the turn continues, nudged to re-issue via
  `provide_answer` (the four-case terminator, case 3). Past that budget (`STUCK_PROSE_LIMIT`), the gate
  stops re-looping and accepts the prose as-is — a real, honestly-uncited answer on the record, never a
  fabricated citation bolted on to make the gate happy.
* **Cannot answer past an unread tool result.** The read-consume invariant blocks any answer — tool call or
  prose — while an artifact handle from this turn sits unopened.
* **Cannot finish a turn that violates its own plan.** Plan-conformance tracks which planned tools are never
  successfully called and keeps correcting until they are, or the plan itself dissolves because the window
  shed them.
* **Cannot silently overflow the window.** The subtractive pass enforces the thrift budget every dispatch;
  what doesn't fit is shed by a fixed order, never truncated mid-token.
* **Cannot re-emit its own rejected prose indefinitely.** The parroting gate excludes the model's own prior
  this-turn generations from the false-positive set, but still catches it echoing a nudge, a tool result, or
  a directive as if that were an answer — up to `NO_PROGRESS_PROSE_LIMIT` re-loops, past which the gate
  accepts the repeat rather than nudge forever with nothing changing.
* **Cannot survive a GPU device loss silently.** A lost WebGPU device is escalated through a fixed ladder —
  respawn, then reload — never swallowed and never left hanging.
  :::

Every item above is a named mechanism, in code, further down this page — not a promise, a pointer.

## See it: one model, a window you control

You do not have to take any of this on faith, and you do not need a lab: the live agent is on this page,
right now — the **Ask ADK** bubble floating in the bottom-right corner, or just

open it from here.
Load the model (a one-time ~2GB download, straight to your GPU — no server, no API key, nothing leaving
your machine), and ask it something real about this library.
It runs a real `gemma-4-E2B-it` — Gemma-4's *effective-2B* build, engineered to run at a ~2B footprint, so
"small" is if anything generous — fully in your browser on WebGPU through the ADK's [`LiteRtLmAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/litert_lm/adapter/classes/LiteRtLmAdapter).

Then do the thing no other agent demo lets you do: **shrink its world while you talk to it.** The dialog's
context-window slider is live — drag it down mid-conversation and watch the utilization meter at the bottom.
Every segment is one bucket of the working set (system prompt, tools, tool results, retrieved docs,
memories, conversation, thoughts), and as the budget tightens you can watch the subtractive pass make its
cuts in real time: the retrieval tail **sheds** first — gets cut, lowest-ranked chunks first, best stay —
prior-turn thoughts are stripped, ejected items are struck through rather than silently vanished, and the
tool registry collapses to what the turn actually needs. Then it answers — correctly — from what survived.
Your conversation, your questions, a real shrinking window: no scripted fixture, no rehearsed happy path.

One deliberate choice worth naming: there is exactly **one** agent on this site, and everything this page
demonstrates happens inside it. A page about running models on WebGPU that casually loads a second
multi-gigabyte model next to the first is a page that crashes your GPU to make a point — VRAM is finite,
device loss is real (see [Surviving the GPU](#surviving-the-gpu)), and that applies to us too.

::: info Field note: a working set that looks long can still fit a small window
A dozen chat turns and a stack of policy snippets *look* long but never trigger shedding — English runs
about four characters per token, so a chunky paragraph is only ~80 tokens and the whole thing fits a 2K
window with room to spare. A demo that never sheds proves nothing. What actually blows a small window is
realistic agent context: full policy sections, multi-message histories, real tool results. Ask the agent a
few real questions, let the tool results and retrieval stack up, then drag the window down — that is when
the meter starts showing real cuts. If your context "feels big" but fits comfortably, you are not
measuring, you are guessing.
:::

## The shape of the turn

A multi-step tool loop — search, then read, then compute, then answer — has an obvious way to build it: one
big loop that reloads everything on every pass, then calls the model, then repeats. It works, but a
five-step turn now repeats four passes of setup that only needed to happen once — re-hydrating (reloading
from storage) the same chat history and memories every iteration, exactly what gets expensive as history
grows.

`@nhtio/adk`'s [`TurnRunner`](https://adk.nht.io/api/@nhtio/adk/turn_runner/classes/TurnRunner) exists to stop that duplication: turn-scoped work runs once, dispatch-scoped
work runs every iteration — where a **dispatch** is one model call within a turn. This agent uses all four
pipelines, and the split isn't decorative — it's why a five-iteration tool loop doesn't re-hydrate history
five times:

::: code-group

```ts \[why TurnRunner, not DispatchRunner]
// The flagship agent harness — the orchestration core, built on the ADK TurnRunner.
//
// WHY TurnRunner (not DispatchRunner directly): the ADK splits work across FOUR pipelines for
// a cost reason (see the LLM-Dispatch / "what each pipeline owns" docs). TurnRunner is the seam
// that owns all four; DispatchRunner only exposes the per-iteration two. The flagship needs the
// turn-scoped seam, so it uses TurnRunner:
//
//   - turnInputPipeline      (once / turn) → hydrate the TURN CONSTANTS: standing instructions.
//                                            (Tools are turn-constant too, supplied via config.)
//   - dispatchInputPipeline  (per iteration, before the model) → hydrate the CONVERSATIONAL
//                                            state the loop reasons over (messages — re-drained
//                                            each iteration so a MID-TURN interrupt folds in —
//                                            plus memories, thoughts, tool-calls, and the
//                                            pre-fetched retrievables), then the subtractive pass
//                                            and the force-answer guard.
//   - dispatchOutputPipeline (per iteration, after the model) → loop telemetry (which tool ran).
//   - persistence callbacks → the SQLite facade. The transformers.js adapter is a well-behaved
//                                            executor: it calls ctx.storeMessage / storeThought /
//                                            storeToolCall and ctx.ack() itself, so those
//                                            callbacks ARE the persistence wiring — no bespoke
//                                            store loop.
//
// LOOP CONTROL is by SIGNALS, using the ADK primitives (ctx.iteration, ctx.toolCallCount) — the
// policy thresholds are ours. We force an answer only on a GENUINE repeat (same tool+args ⇒ same
// checksum, REPEAT_LIMIT+ times) or the hard iteration cap — never on "any tool was used", so a
// healthy multi-step turn (search → catalog → call_a_tool → answer) runs freely.
//
// CHAT HISTORY vs CONTEXT WINDOW: the user turn is persisted in run(); the adapter persists the
// final assistant turn via the storeMessage callback. The model-facing context (turnMessages and
// the rest) is rebuilt each turn and never rendered. See memory chat_history_vs_context_window.
```

:::

`turnInputPipeline` loads the turn constants once — standing instructions; tools are turn-constant too,
supplied via config. `dispatchInputPipeline` reloads the conversational state every iteration instead —
messages (re-drained each time so a mid-turn interrupt folds straight in), memories, thoughts, tool-calls,
and pre-fetched retrievables — then runs the subtractive pass and the force-answer guard.
`dispatchOutputPipeline` is where the gate cascade lives. Persistence callbacks are the SQLite facade; the
adapter is a well-behaved executor that calls `ctx.storeMessage`/`storeThought`/`storeToolCall` and
`ctx.ack()` itself, so those callbacks are the entire persistence layer — no bespoke store loop bolted on
the side.

The pre-dispatch overflow guard is passed **per dispatch**, not fixed once at construction, because the
active thrift target moves — both with the user's slider and with the overflow-retry ladder tightening it
mid-turn:

::: code-group

```ts \[the four pipelines, wired]
turnInputPipeline: [hydrateStanding, hydrateRetrievables],
dispatchInputPipeline: [dispatchInput as never],
dispatchOutputPipeline: [dispatchOutput as never],
```

:::

In plain terms: the guard re-reads its target on every call, so tightening the window mid-turn takes effect
on the very next one.

Loop control is by signal, not by vibes: `ctx.iteration` and `ctx.toolCallCount` are ADK primitives, the
thresholds are this agent's own policy — the ADK core ships no `maxIterations` at all. There is no single
global iteration cap; a plan-conformance failure is allowed to re-loop unbounded, deliberately, because an
observed raw failure beats a silently truncated one. What actually stops a turn is a family of narrow,
named counters, each bounded to the one failure mode it watches: a genuine repeat — same tool, same
arguments, same checksum — force-answers past `DUPLICATE_CALL_HARD_STOP`; a near-identical uncited answer
gets accepted past `STUCK_PROSE_LIMIT`; a stalled iteration with no tool progress gets accepted past
`NO_PROGRESS_PROSE_LIMIT`. Never on "a tool was used," which is why a healthy multi-step turn (search → read
the result → call a computed tool → answer) runs to completion without the harness getting nervous halfway
through.

::: info Field note: hiding a tool is not the same as removing it
Hiding a tool's schema (`ctx.tools.setHidden(...)`) strips it from the prompt — the model still emits
tool-call *syntax* anyway, on a hallucinated argument, and the adapter parses and executes it, because
nothing told the executor to refuse. What worked here, on a genuine repeat: force the answer by folding the
prior result into an **ephemeral** user turn added to the live `turnMessages` set — never `storeMessage`'d,
so a corrective nudge this-turn can never poison durable history or replay wrong on reload.
:::

## The planner book-end  {#the-planner-book-end}

::: info Showcase policy, implemented using ADK
The planner, the gate cascade, and the GPU recovery ladder below are **this agent's application policy** — decisions *we* made for a 2B on a browser GPU. ADK is the capability provider: it supplies the dispatch loop, the pipeline seams, the artifact handles, and the context primitives these are built on. It does **not** ship a planner, these gates, or this recovery ladder. That is the point — the seams are the ADK; the policy is yours. The `Showcase policy · built on ADK` badge marks each section that is ours to decide, not inherent ADK behavior.
:::

Ask a small model a question and watch what it does first: it starts answering. Not deciding what *kind* of
answer this should be — conversational aside, a number the tools can compute, a cited doc lookup, an
exhaustive list scoped to what fits — just answering, in whatever register and length the first few tokens
happened to commit to. Get that wrong and everything downstream follows: a tool it didn't need, a tool it
skipped, four sentences where the question wanted four words.

Our solution is a **book-end**: a dedicated dispatch, run before the real answer, whose only job is to
decide the turn's shape and write it down. The *why* matters more than the shape: a frontier model does this
step internally — its reasoning phase settles what kind of answer it's producing before committing tokens to
it. This build doesn't skip that deliberation; it **restructures** it. Reasoning is spent exactly where it
pays: the planner dispatch runs with thinking *enabled* — classifying the turn is the one decision this
model most often gets wrong in a single greedy shot, and letting it think first is worth the tokens. But
instead of letting that chain-of-thought ramble free through the whole turn (where it would burn the tool
loop and drift), the deliberation is distilled into a **deterministically generatable, testable, token-dense
plan** — a validated `make_plan` tool call against the real catalog, not free prose — which is then injected
back into the worker as the model's own synthetic "thought." The other end of the book-end closes the loop:
a conformance gate validates that the model actually stuck to the plan it committed to. Freeform reasoning
in, structured commitment out, held to account at the close. The `make_plan` call — up to three retries on a
parse failure, fail-open (an unplanned turn beats a blocked one) on anything that isn't a GPU OOM — commits
three things: what *kind* of answer this is, which tools it intends to call, and how long the answer should
run.

That book-end dispatch needs its own way to end, though. `autoAck` — the setting that lets the harness close
a dispatch automatically — can't close this one: the LiteRT adapter only auto-acks a generation with zero
tool calls, and the planner's entire job is to emit exactly one (`make_plan`). Without an explicit
terminator, the model re-plans forever, burning the turn on "call make\_plan now" repeated at the model that
already called it:

::: code-group

```ts \[the fix for planner thrash: own the ack]
// TERMINATOR (the fix for the planner thrash). autoAck on the LiteRT adapter only acks a
// generation that emitted NO tool call (adapter: `if (calls.length === 0) ctx.ack()`). But the
// planner's whole job is to emit ONE tool call — make_plan — so autoAck NEVER fires, the dispatch
// never acks, and it loops on "Call make_plan now" until the abort signal — re-planning 6–12× and
// burning the turn (a contradictory-directive thrash: "call make_plan now" persisting after
// make_plan was already called). So own termination explicitly: after each planner generation,
// ack — one generation per dispatch. If a plan was captured we're done; if make_plan failed to
// parse (plan still null), the ack ends THIS dispatch so the outer `for attempt<=3` mints a fresh
// one (the intended "3 fresh single-shot retries" design, which the missing terminator defeated).
turnOutputPipeline: [
  (async (ctx: DispatchContext, next: () => Promise<void>): Promise<void> => {
    if (!ctx.isSignalled) ctx.ack()
    await next()
  }) as never,
],
```

:::

In plain terms: the planner dispatch acks itself the moment `make_plan` lands, instead of waiting on a rule
that never fires for it.

The committed plan doesn't get frozen into a static instruction. It is rendered as an **evaluatable**
[`Thought`](https://adk.nht.io/api/@nhtio/adk/common/classes/Thought) — a small function, not a fixed string, re-run every iteration against the live dispatch
context — so it only ever claims a planned tool is "directly available" when that tool actually survived
this dispatch's subtractive pass:

::: code-group

```ts \[the plan re-checks tool visibility at assembly time, not at commit time]
export function renderPlanThought(
  plan: TurnPlan,
  budgetWords: number,
  capacityNote?: string
): (ectx?: { tools?: { visible: () => { name?: string }[] } }) => string {
  return (ectx?: { tools?: { visible: () => { name?: string }[] } }): string => {
    // Which planned tools are ACTUALLY visible in THIS dispatch? (No ctx → assume all planned tools are
    // present: the no-ctx fallback is the pre-shed plan, matching how the thought reads outside assembly.)
    const visibleNames = ectx?.tools ? new Set(ectx.tools.visible().map((t) => t.name)) : undefined
    const availablePlannedTools =
      visibleNames === undefined
        ? plan.toolsToUse
        : plan.toolsToUse.filter((name) => visibleNames.has(name))
    // A planned tool that is NOT directly visible is usually still REACHABLE via the catalog capstone
    // (call_a_tool) — the flagship hides utility tools (get_current_time, calculate, …) behind
    // tool_catalog/call_a_tool to save prompt budget; they are callable, just not in the direct list.
    // This is the THIRD state the old binary (visible ⇒ "call it" / else ⇒ "answer from docs") missed:
    // the proven tool_computed failure was a planned get_current_time landing in the "not available →
    // answer from the documentation" branch, so the 2B abstained ("I don't have the date") instead of
    // reaching the tool through the capstone. Only treat a planned tool as genuinely unreachable when the
    // capstone ITSELF is also gone (a tight-window shed of everything).
    const capstoneVisible = visibleNames === undefined || visibleNames.has('call_a_tool')
    const plannedViaCapstone =
      visibleNames === undefined ? [] : plan.toolsToUse.filter((name) => !visibleNames.has(name))
```

:::

That guard is not academic. A tight window can shed a planned tool between the moment the plan is made and
the moment the model reads it, and a frozen "you MUST call `search_docs_semantic`, it is directly available"
directive contradicts a dispatch where that tool is gone — a 2B flounders in exactly that contradiction
without the re-check. The re-check adds a third state a naive binary misses: a planned tool absent from the
direct list can still be *reachable* through the `call_a_tool` catalog capstone, and the plan-thought now says
so explicitly, rather than reading as "unavailable" and prompting an honest-sounding but wrong "I don't have
access to that."

::: info Field note: the correction that works is the one in the model's own voice
Every re-looping gate on this page fires a first-person `__nudge:` thought ("My answer leaked an internal
tag; I'll rewrite it clean") alongside a `[SYSTEM DIRECTIVE]` message, never the directive alone. A 2B
converges in fewer iterations on something it apparently just decided than on an external order — audited
across every gate that loops, and every one of them carries the first-person form. Terminal hard-stops
skip it deliberately: the turn is already over, so there is no next generation left to read it.
:::

## The subtractive pass: shrink to fit, every dispatch

Leave context assembly unmanaged and one of two things happens: the window overflows and the dispatch errors
outright, or — worse, because it's silent — it fills with stale junk and the answer just gets quietly worse.

This is the mechanism behind the slider you just dragged in the live agent. Every dispatch, the agent runs
the `@nhtio/adk/batteries/context/thrift` battery (documented in full on the
[Token Thrift context battery page](/batteries/context/thrift)): hold a working set — everything that could
go in the prompt — far larger than any one window, then subtract to fit, lowest-signal first, in a fixed
**shed order** (the same category always cut before the same other, never at random), by measured token
weight — never truncate mid-token, never guess. If even the unsheddable floor won't fit, the pass returns a
**bounded refusal**: a typed error instead of a truncated dispatch.

::: code-group

```ts \[subtract to fit — measured, ordered, never truncated]
/**
 * The subtractive pass. Start wide; measure every bucket; then shed lowest-signal first
 * until the dispatch fits the active window's budget — the image (the single biggest hog)
 * goes first when it doesn't fit, then tools the turn doesn't need, the tail of the
 * retrieval ranking, low-value memories, the oldest conversation turns. Each cut is by
 * EVIDENCE (a measured bucket), never a guess. If even the floor (system prompt + newest
 * turn) won't fit, REFUSE — a bounded refusal beats a truncated, incoherent dispatch.
 *
 * THE ALGORITHM ITSELF now lives in `@nhtio/adk/batteries/context/thrift`'s `subtractToFit` (loaded via
 * the agent_adk.ts shim). This wrapper preserves the flagship's OLD 9-positional-argument call
 * signature (unchanged since before the battery extraction — every existing call site and cross-spec
 * test depends on it) and adapts positionals → the battery's real options-object form internally,
 * binding the showcase's own policy: `TOOL_SHED_ORDER`-derived `shedRank`, `ENCODING = 'gemma'`, and the
 * `__eph-`/`__compact-summary` id predicates (passed explicitly even though they match the battery's own
 * defaults, for self-documentation at this call site).
 */
export const subtractToFit = (
  ws: WorkingSet,
  contextWindow: number,
  relevantToolNames: string[],
  outputReserve?: number,
  /** Thought ids to PRESERVE through the prior-turn strip (the planner's this-turn plan thought). */
  keepThoughtIds?: ReadonlySet<string>,
  /**
   * The battery's tool-declaration renderer. When supplied, the tools bucket is measured against the
   * REAL rendered `<tool_definitions>` block (full JSON-Schema per tool) instead of a `name: description`
   * proxy — the load-bearing fix for the context-overflow crash, where the proxy undercounted the tool
   * block by ~10× and let a doc turn blow the engine cap. Omit to fall back to the proxy.
   */
  renderTools?: RenderToolsFn,
  /**
   * Thought ids that must NEVER be shed for budget even when over — the this-turn scaffolding the worker
   * needs to answer at all (e.g. the planner's plan thought + the citation reinforcement). Everything else
   * in the surviving keep-set (the per-iteration nudge thoughts, prior synthetic guidance) is sheddable
   * oldest-first when the dispatch still doesn't fit. Omit to make every surviving thought sheddable.
   * This is a SUBSET of `keepThoughtIds`: keepThoughtIds decides what survives the Gemma §3 prior-turn
   * strip; protectThoughtIds decides what additionally survives the budget shed.
   */
  protectThoughtIds?: ReadonlySet<string>,
  /**
   * The live dispatch context, so a DYNAMIC (evaluatable) thought's token count reflects the string it
   * will resolve to for THIS dispatch — `t.content.estimateTokens(ENCODING, renderCtx)` counts exactly
   * what the battery's `.render(renderCtx)` will assemble. Without it a dynamic thought would be counted
   * as its no-ctx fallback, and the budget could disagree with the shipped prompt. Optional; static
   * content counts identically with or without it.
   */
  renderCtx?: unknown,
  /**
   * Tool names the CURRENT PLAN committed to that have NOT yet been called this turn. These are
   * UNSHEDDABLE by the last-resort tool-shed: the plan-thought tells the model to call them, so removing
   * them from the rendered `<tool_definitions>` block leaves the model instructed to call a tool it cannot
   * see — the proven tool_computed failure (planner chose get_current_time, thrift shed it, the 2B could
   * only see provide_answer and abstained "I don't have the date"). The protection is bounded: once the
   * tool has been called (its result is in context) it drops off this set and sheds normally, so it is
   * never a permanent floor. Omit to make every visible tool sheddable (prior behaviour).
   */
  protectedToolNames?: ReadonlySet<string>
): ThriftTrace => {
  const options: ThriftSubtractToFitOptionsType = {
    estimateTokens: (v, enc, ctx) => Tokenizable.estimateTokens(v, enc, ctx as never),
    encoding: ENCODING,
    outputReserve,
    keepThoughtIds,
    renderTools: renderTools as unknown as ThriftSubtractToFitOptionsType['renderTools'],
    protectThoughtIds,
    renderCtx,
    protectedToolNames,
    shedRank,
    // Passed explicitly even though these match the battery's own defaults — self-documentation of the
    // flagship's id conventions at this call site (see the module header comment).
    isEphemeralMessage: (m) => m.id.startsWith('__eph-'),
    isSummaryMessage: (m) => m.id === '__compact-summary',
  }
  const trace: ThriftTraceResult = adk.subtractToFit(
    ws as unknown as ThriftWorkingSetType,
    contextWindow,
    relevantToolNames,
    options
  )
  return trace
}
```

:::

The battery's own `selectRelevantTurns` — invoked here with `keepRecent` newest turns always kept, plus a
calibrated relevance floor over everything older — replaces this agent hand-rolling its own recency logic.
The floor itself (`RELEVANCE_FLOOR_MIN`/`MAX`/`CURVE`, calibrated against a three-judge oracle over a 94-turn
corpus) lives inside the battery, not this showcase's code — this agent is a *consumer* of that calibration,
exactly the way an application is supposed to sit on top of a battery instead of re-deriving its constants.
See [the battery reference](/batteries/context/thrift) for the exact numbers and the calibration story.

Gemma's own model card is the cleanest proof this belongs in the pipeline, not behind a battery flag. Section
3, "No Thinking Content in History," is unambiguous: prior-turn reasoning must not be re-added before the
next user turn, or the model's own multi-turn behavior degrades. `@nhtio/adk` doesn't need a
`thoughtSurfacing: 'none'` option for this — it hands you the working set and the trim points, and you
enforce a model-specific correctness rule exactly where the model documentation says to:

::: code-group

```ts \[Gemma §3, enforced in the pipeline, not a battery flag]
/**
 * Gemma 4 model card, §3 Multi-Turn Conversations — "No Thinking Content in
 * History": thoughts from previous model turns MUST NOT be re-added before the
 * next user turn. This is a CORRECTNESS rule, enforced HERE in the pipeline, not
 * with a battery flag — you decide what each dispatch sees. It is also pure
 * token-thrift: prior-turn reasoning is the highest-volume, lowest-reuse content
 * there is. The current turn's reasoning still surfaces live; it just never
 * re-enters history.
 */
export const stripPriorTurnThoughts = (ws: WorkingSet): { dropped: number; tokens: number } => {
  const tokens = ws.thoughts.reduce((n, t) => n + tok(t.content.toString()), 0)
  const dropped = ws.thoughts.length
  ws.thoughts = [] // none of last turn's thinking goes back into the window
  return { dropped, tokens }
}
```

:::

::: info Field note: the pass shed the one result the model just asked for
An early version measured every tool result's age off a single "oldest first" ranking, no this-turn
exception — so a search result the model had requested seconds earlier, easily the highest-signal thing in
the window, got evicted for being merely old, and the model immediately re-requested it. What looked like
thrash was the subtractive pass sawing off the branch it was sitting on. The fix excludes this-turn tool
results from the shed entirely; only prior-turn results are ever eligible.
:::

## The gate cascade owns the turn  {#the-gate-cascade-owns-the-turn}

Let the model decide for itself when an answer is done and it will call plenty of bad ones finished: an
answer built on an unopened tool result, an invented citation, a silently abandoned plan, the same rejected
sentence retried word for word. Every one of those is a documented failure this exact build produced before
it had a check for it.

So the model doesn't get the final word. `autoAck` is off, and `dispatchOutputPipeline` decides instead, via
a terminator with exactly four cases per iteration:

::: code-group

```ts \[ack is a decision, not a default]
// dispatchOutputPipeline (PER iteration, after the model): the TURN TERMINATOR (autoAck is
// off — the harness owns when the turn ends). Four cases per iteration:
//   1. An answer tool fired (captured) → ack: cited answer / honest IDK ends the turn.
//   2. A non-answer tool ran this iteration (search, etc.) → keep looping to act on it.
//   3. Prose, classified as a doc ANSWER (1) but UNCITED → set needsCitation; do NOT ack —
//      the next iteration nudges the model to re-issue via provide_answer.
//   4. Prose, classified conversational (0) → ack: a chat reply is a valid response.
// Backstop: at the hard cap, ack regardless so a stubborn model can't loop forever.
```

:::

An answer tool firing ends the turn. A non-answer tool (search, a computed lookup) keeps the loop running to
act on it. Prose classified as a documentation answer but missing a real citation is held open and
**nudged** — sent back with a corrective message, not just rejected — back toward `provide_answer`.
Conversational prose is accepted as a valid reply on its own. There is no single hard iteration cap
backstopping all of it — a stubborn model that keeps failing the *same* check for too long trips that
check's own bounded limit (a duplicate-call hard stop, a stuck-prose limit, a no-progress limit), and each of
those is the mechanism doing the actual work, not a generic last resort.

That bounded-vs-unbounded split is deliberate: a check stays **unbounded** — never gives up — when accepting
its failure means shipping a fabrication. A check gets a **bound** — a hard limit past which the output is
accepted as-is — when the alternative is looping forever with nothing changing, which helps nobody. Ahead of
the citation/abstention ladder sit two deterministic, unbounded checks that fire before any classifier gets a
vote. The first: did the model claim to answer while leaving a tool result — an artifact handle (a tool
result the model has to explicitly open before it can use it) it requested this very turn — unread?

::: code-group

```ts \[read before you answer, or you don't answer]
// READ-CONSUME INVARIANT (deterministic, unbounded). Whenever the model is trying to ANSWER (bare
// prose OR an answer tool fired), does it leave an unread artifact handle it produced this turn? If so,
// it must open it via an artifact_* reader before answering. We do NOT guard on artifact_* tool
// VISIBILITY: the forged readers are ephemeral + adapter-local (rendered into the prompt from the
// adapter's merged registry, never onto ctx.tools.visible()), so checking visible() was structurally
// always-false and permanently DISABLED this gate — the model was handed a readable handle it was never
// forced to read, and fabricated the value instead (the get_current_time "It is Wednesday" hallucination,
// caught only by the parroting canary → apology loop). The presence of an unread SpooledArtifact handle
// IMPLIES its readers were forged into the prompt this iteration; that is the availability signal. The
// tight-window concern (readers shed) is already covered: if the HANDLE itself was shed,
// collectUnreadResultHandles finds nothing → null → no demand → no deadlock (this-turn results are never
// budget-shed and artifact_* readers shed LAST; see agent_subtractive_pass.ts).
const unreadHandles = answering ? await collectUnreadResultHandles(calls) : null
// Shared unread-handle gate emission (used by BOTH the answer-tool path below and the prose path) — one
// first-person "read it now" thought, so a 2B commits to reading the handle as its next action.
const emitUnreadHandleGate = (handles: NonNullable<typeof unreadHandles>): void => {
  const refs = handles.map((h) => `- ${h.tool} (callId=${h.callId})`).join('\n')
  emitGate(
    'unread-handle',
    'Answered without reading a tool result it requested; sent the model back to read the handle',
    'You answered without reading a tool result you already requested THIS TURN. It was not ' +
      'inlined — it is waiting as an artifact handle, and you have not opened it. Do NOT describe ' +
      'or guess its contents, and do NOT reply "I understand"/"I apologize" — that is not an answer. ' +
      'READ it now with an artifact_* tool (artifact_head, or artifact_json_get with a path like ' +
      '"$[*].excerpt" for a JSON result) using the callId below, THEN deliver the answer as a ' +
      'provide_answer TOOL CALL citing it (not ' +
      `chat prose, not "<call:…" text):\n${refs}`,
    `I have not actually opened the tool result yet (${handles[0]?.tool}, callId=${handles[0]?.callId}). My NEXT output is an artifact_* call to read it — not prose, not an apology. Then I emit a provide_answer TOOL CALL citing it.`
  )
}
```

:::

This is not a guess. `collectUnreadResultHandles` checks structurally which handles were opened via an
`artifact_*` reader and which weren't — no classifier judgment call, no bound to hit and give up on. It
catches the exact failure a plain citation check misses: the model requests a search, gets back a handle,
and answers from parametric memory without ever opening what it asked for. Deliberately unbounded, because
"you're allowed to fake it after N tries" is not a real fix for fabricating a value.

::: info Field note: a model answering from a real result while it sits unread — proven false, not assumed
On a "what day of the week is it?" turn, `get_current_time` returned a real result and the model answered
"I don't have the current date and time information" anyway — a false abstention, since the answer had
arrived that same turn. The unread-handle gate runs first for a reason: an unopened handle is the one
legitimate case where the model genuinely never saw the result, so it's checked and cleared before the
false-abstention gate runs. By the time the abstention gate fires, the result is provably in context —
the refusal has no honest excuse left.
:::

The second: plan-conformance. The plan named tools the model committed to at turn-open — did it actually
call them?

::: code-group

```ts \[which planned tools are never successfully called]
// The plan names tools the worker committed to. DETERMINISTIC signal: which planned tools were
// never SUCCESSFULLY called this turn. `planToolsSatisfied` (none uncalled) retires the
// plan-violation correction mid-loop; `uncalledPlanTools` is fed to the semi-deterministic
// conformance classifier at close to make a real violation far more likely to be caught.
const calledToolNames = new Set(
  calls.filter((c) => !(c as { isError?: boolean }).isError).map((c) => c.tool)
)
const uncalledPlanTools = turnPlan
  ? turnPlan.toolsToUse.filter((t) => !calledToolNames.has(t))
  : []
const planToolsSatisfied = uncalledPlanTools.length === 0
```

:::

And the parroting check — comparing a prose reply against everything the model is actually shown this turn
— carries two deliberate, documented exemptions, because a naive version of this check creates its own
failure mode:

::: code-group

```ts \[two exemptions: a correct readback isn't a parrot, and neither is your own retry]
// PARROTING CHECK (deterministic). When the model replied in PROSE, compare that reply against how
// every piece of context was rendered TO THE MODEL — each prior message + each tool result (the
// model-facing preview, the exact text it saw) — via Levenshtein similarity. A reply that closely
// mirrors any single context entry is a parrot, not an answer (the observed leak: echoing
// provide_answer's "Answer delivered…" confirmation, a rejection notice, or a search blob). Gathered
// only on the prose path; skipped otherwise (a tool call is never a parrot).
// TOOL_COMPUTED EXEMPTION. For a tool_computed turn that already has a successful tool result, the
// model's job is to STATE that computed VALUE — which necessarily resembles the tool-result text it
// just read (e.g. get_current_time → artifact_head → "It is Sunday, July 5, 2026" mirrors the
// "UTC: Sunday, July 5, 2026 …" the reader returned). Comparing the answer against the TOOL RESULT
// previews therefore mislabels the correct readback as parroting, and the loop the whole flagship was
// built to fix reappears: the model states the right value, parroting rejects it, it re-emits, ~50×.
// (PROVEN in the Node/Ollama real-loop dump: turn "what day is it?" = 104 dispatches, get_current_time
// → artifact_head → "It is Sunday" at dispatch 7, then ~50 rejected restatements.) So on a
// tool_computed turn WITH a successful result, do NOT compare against the tool-result previews. We STILL
// compare against prior MESSAGES — echoing a nudge / system directive is parroting on any turn kind.
const parrotExemptToolResults =
  turnPlan?.answerKind === 'tool_computed' && hasSuccessfulResult
// SELF-ECHO EXEMPTION. The parroting check must compare the answer against EXTERNAL context the model
// might be echoing (a nudge/directive, a tool result, seeded prior-turn history) — NOT against the
// model's OWN this-turn prior generations. PROVEN on the wire (Node/Ollama T7 parroting-match tap):
// 26/28 parroting matches were the model's own earlier prose answer THIS turn, 0 were RAG. Mechanism:
// an answer gets rejected by some OTHER gate (incomplete/needs-citation) → flushed into turnMessages →
// the next attempt at the SAME question is naturally similar → parroting matches the prior attempt →
// rejected → compounds into the doc-turn loop (23-58 dispatches). Answering a question similarly to your
// own prior attempt is not parroting. Exclude assistant-role messages that are THIS-turn generations
// (assistant role AND not in seededIds — the same discriminator the self-echo cap uses above).
// [[fence_nonce_id_miscitation]] residual; [[tool_computed_loop_is_self_apology_accumulation]] class.
const parrotedContext =
  proseNoTool && unreadHandles === null
    ? await (async (): Promise<boolean> => {
        const ctxStrings: string[] = []
        for (const m of ctx.turnMessages) {
          if (
            (m as { role?: string }).role === 'assistant' &&
            !seededIds.has((m as { id?: string }).id ?? '')
          ) {
            continue // the model's own this-turn re-attempt — not external context to parrot
          }
          const c = (m as { content?: { toString(): string } }).content
          if (c) ctxStrings.push(c.toString())
        }
        if (!parrotExemptToolResults) {
          for (const c of calls) {
            const preview = await readModelFacingPreview(
              c as Parameters<typeof readModelFacingPreview>[0]
            )
            if (preview) ctxStrings.push(preview)
          }
        }
        return looksLikeParroting(proseAnswer, ctxStrings)
      })()
    : false
```

:::

Without the `tool_computed` exemption, a model correctly stating a value it just read — "it is Sunday, July
5th" after reading a `get_current_time` result that says almost the same thing — gets rejected for
resembling the very result it was told to state, and re-emits the correct answer roughly fifty times before
the turn gives up. Without the self-echo exemption, a rejected answer that gets a second, similar-sounding
attempt reads as parroting *itself* — proven on the wire at 26 of 28 observed parroting matches being the
model's own prior this-turn answer, not leaked context. Both exist because the naive gate manufactured the
exact loop it was built to prevent.

::: info Field note: a 2B invents citation markup we never asked it for
A conversational follow-up once ended with a fabricated `<cite>Assembly</cite>` tag rendered as literal
text — `<cite>` is almost certainly something the model picked up in pretraining, but *nothing in this
system* asked for it or handed it that form; a real citation only ever travels through `provide_answer`'s
`sources[]`, never inline XML. There was no detector for a failure mode that had never been named, so
nothing caught it. Our fix is a `looksLikeFakeCitation` check plus a bounded gate that rejects and re-nudges
in the model's own voice to re-answer through the real citation path — the same doctrine as every gate in
this build: name the failure precisely, then build the one check that catches exactly that shape.
:::

## Artifact handles: the tool loop's actual currency

Call a real search tool and the result can be 500KB of JSON. Paste that into the context window as-is and
it isn't "a big tool result" anymore — it's your entire budget, gone, on one call.

So a tool result never lands in the window as a body. The battery **spool-wraps** it — stores the full
result off to the side, keeping only a reference in context — into a [`SpooledArtifact`](https://adk.nht.io/api/@nhtio/adk/spooled_artifact/classes/SpooledArtifact), and renders a
compact handle plus a set of **forged** `artifact_*` reader tools (generated on the fly for this one result,
not part of the agent's fixed tool list) the model calls to pull only the slice it needs. A search result
that would cost the whole budget as inline JSON costs a few dozen tokens as a handle.

The library's default reader order leads with a metadata-only tool — `artifact_json_type`, which returns
just the string `"json"` and teaches the model nothing — so a model that reaches for the first tool listed
flails, concludes it "couldn't find" the content, and abstains on a result that was sitting right there. This
agent doesn't change the library default; it reorders the *presentation* through the battery's own render
override, so a content-returning reader leads instead:

::: code-group

```ts \[content readers first, metadata readers last]
export const ARTIFACT_READER_PRIORITY = [
  'artifact_json_get',
  'artifact_json_keys',
  'artifact_head',
  'artifact_cat',
  'artifact_json_filter',
  'artifact_json_pluck',
  'artifact_json_slice',
  'artifact_grep',
  'artifact_tail',
  'artifact_line_count',
  'artifact_byte_length',
  'artifact_estimate_tokens',
  'artifact_json_length',
  'artifact_json_type',
]
```

:::

Same tools, same callId, same metadata — only the line order the model reads changes. And the dialog's
"what did the model actually see" chip isn't an approximation: it calls the exact renderer the executor used,
so what you inspect is byte-identical to the prompt, not a reconstruction that might quietly drift from the
truth:

::: code-group

```ts \[the debug chip renders the SAME function the executor used — no approximation]
export async function readModelFacingPreview(tc: {
  id?: string
  tool?: string
  inline?: boolean
  fromArtifactTool?: boolean
  results?: unknown
}): Promise<string | undefined> {
  const r = tc.results
  // Handle-rendered: spooled body the model did NOT get inline (matches the battery's render gate).
  // looksLikeSpooledArtifact is the battery's own structural check (cross-realm-safe).
  if (looksLikeSpooledArtifact(r) && tc.inline === false && !tc.fromArtifactTool) {
    let byteLength = 0
    let lineCount = 0
    try {
      byteLength = await (r as { byteLength: () => Promise<number> }).byteLength()
    } catch {
      /* best-effort metadata */
    }
    try {
      lineCount = await (r as { lineCount: () => Promise<number> }).lineCount()
    } catch {
      /* best-effort metadata */
    }
    // The EXACT text the executor put in front of the model — same renderer (the json-first reorder
    // this agent passes to the adapter via helpers.renderArtifactHandleBody), same fields.
    return renderArtifactHandleBodyJsonFirst({
      callId: tc.id ?? '',
      artifact: r,
      byteLength,
      lineCount,
    })
  }
  return readToolResultPreview(r)
}
```

:::

That equivalence is a discipline, not a convenience. A showcase that shows you a reconstruction of what the
model saw is a showcase that can lie to you the moment the real renderer diverges from the approximation. If
you're going to read the wire, read the actual wire.

::: info Field note: "RAG is poison" was never RAG — it was a spool footgun
Every tool result once came back as a literal `Failed to execute 'getFileHandle' … Name is not allowed`
string, and the model dutifully "answered" from it — reading like the retrieval layer itself was garbage
in, garbage out. The actual defect was one call site: the OPFS-backed spool store took a `keyPrefix` it
treated as a directory path, but a `/` is illegal in an OPFS filename, so every single spooled result
failed identically. Retrieval itself was fine the whole time — top hit scored 0.58. The lesson generalizes:
when a tool-using agent gives a confident wrong answer, read the actual tool result before blaming the
model or the retrieval layer for it.
:::

## Surviving the GPU  {#surviving-the-gpu}

This runs on real WebGPU hardware, which means the device gets lost — not a bug here, just what happens on
real drivers: a **TDR** (the driver's own timeout, killing and resetting a GPU task it assumes has hung) or an
outright GPU-process crash. Either way, that's **device loss** — the GPU handle this agent was using is gone,
the engine is dead, nothing in flight will complete. Our fix is structural: the LiteRT-LM engine runs inside
a disposable **classic Web Worker** (an older, simpler background-thread style, not the newer "module worker"
kind, because the Emscripten glue it depends on calls `importScripts()`, which a module worker forbids), so
`recycle()` becomes `worker.terminate()` plus a fresh worker, instead of a same-thread `dispose()` that can't
fully claw back a wedged GPU context:

::: code-group

```ts \[every in-flight call and stream fails cleanly on device loss]
case 'ev:deviceLost': {
  if (this.#deviceLost) return
  this.#deviceLost = true
  // Fail every in-flight call + stream — the engine is dead; nothing will complete.
  const lostErr = `WebGPU device lost: ${event.reason}`
  for (const [, p] of this.#pending) p.reject(new Error(lostErr))
  this.#pending.clear()
  for (const [, s] of this.#streams) s.error(lostErr)
  this.#streams.clear()
  this.#hooks.onDeviceLost?.({ reason: event.reason })
  return
}
```

:::

::: info Field note: the OOM is an honest memory signal, not a bug to hide
`Failed to allocate memory for buffer mapping` means exactly what it says: this context size doesn't fit the
memory actually available, full stop. Even on LiteRT-LM, which clears the harder software ceiling that
[drove the runtime switch below](#the-runtime-we-didnt-keep), a large enough context still doesn't fit — the
wall moved and got machine-dependent, it didn't vanish. The lesson is engine-agnostic: a smaller weight format
buys headroom, not immunity, so the durable fix isn't a smaller number, it's product honesty. Surface the OOM
as a plain-language banner ("not enough GPU for this context-window size — reduce it and retry") with a real
retry path, instead of pretending a ceiling isn't there.
:::

A single lost device is recoverable — a fresh worker, a fresh engine, the conversation resumes because
history lives in SQLite, not in the dead worker's memory. But Chrome blocks new GPU adapters for a stretch
after two GPU-process crashes close together, so a naive "always respawn" policy would hang forever on the
second loss pretending it's still trying. The escalation ladder is a sliding window, not a single flag:

::: code-group

```ts \[respawn once, reload on the second loss inside the window]
// GPU device-loss escalation policy for the worker-hosted LiteRT engine.
//
// When the worker's sentinel WebGPU device is lost (driver TDR / GPU-process crash), the engine is dead
// and no in-flight or future turn can complete on that worker. Recovery is a LADDER:
//
//   1st loss in the window  → RESPAWN the worker (worker.terminate() + a fresh worker/engine). A single
//                             transient TDR is recoverable this way — a clean GPU context from scratch.
//   2nd loss in the window  → full PAGE RELOAD. Chrome blocks new GPU adapters after 2 GPU-process crashes
//                             in ~2 min ("device lost" persists), so only a reload clears it. Conversation
//                             history lives in SQLite, so a reload is LOSSLESS — the user resumes the thread.
//
// The window is a sliding 120s: two losses spaced far apart are each treated as a first (transient) loss,
// not an escalation, so an occasional TDR over a multi-hour session never forces a reload.
//
// This module is pure policy (no DOM beyond the injected reload fn) so it is unit-testable. The host
// (agent_runtime.ts) owns the respawn action and passes reload; the dialog surfaces the reload with a banner.

/** The escalation the policy decided for a single device-loss event. */
export type GpuLossAction = 'respawn' | 'reload'

/** How long two losses must fall within to count as an escalation (Chrome's ~2-min GPU-crash window). */
export const GPU_LOSS_WINDOW_MS = 120_000

export interface GpuLossPolicyOptions {
  /** Sliding window; a loss older than this no longer counts toward escalation. Default 120_000. */
  windowMs?: number
  /** Injected clock (for tests). Default `Date.now`. */
  now?: () => number
}

/**
 * Sliding-window device-loss escalation decider.
 *
 * @remarks
 * Call {@link GpuLossPolicy.record} once per `ev:deviceLost`. It returns `'respawn'` for the first loss in
 * the window and `'reload'` for a second (or later) loss still inside the window. Prunes losses older than
 * the window so the counter reflects only the recent burst.
 */
export class GpuLossPolicy {
  readonly #windowMs: number
  readonly #now: () => number
  #timestamps: number[] = []

  constructor(options: GpuLossPolicyOptions = {}) {
    this.#windowMs = options.windowMs ?? GPU_LOSS_WINDOW_MS
    this.#now = options.now ?? Date.now
  }

  /**
   * Record a device-loss event and return the escalation action.
   *
   * @returns `'respawn'` for the first loss in the current window, `'reload'` for a repeat inside it.
   */
  record(): GpuLossAction {
    const t = this.#now()
    // Drop losses that have aged out of the window.
    this.#timestamps = this.#timestamps.filter((ts) => t - ts < this.#windowMs)
    this.#timestamps.push(t)
    return this.#timestamps.length >= 2 ? 'reload' : 'respawn'
  }

  /** Number of losses currently inside the window (observability/tests). */
  get recentCount(): number {
    const t = this.#now()
    return this.#timestamps.filter((ts) => t - ts < this.#windowMs).length
  }

  /** Clear the loss history (e.g. after a clean recovery + a settled session). */
  reset(): void {
    this.#timestamps = []
  }
}

```

:::

One loss inside the 120-second window respawns; a second loss inside that same window escalates straight to
a full page reload — lossless, because the conversation is durable. Two losses far apart are each treated as
a first, transient loss, so an occasional TDR over a multi-hour session never forces a reload it didn't
need. This is the least glamorous code on the page and the one most likely to be the reason a demo either
recovers gracefully or silently dies on stage.

## The runtime we didn't keep: maturity lost to fit {#the-runtime-we-didnt-keep}

Device loss is the GPU being hostile in real time. There is a second kind of hostility, quieter, that shaped
the single biggest low-level decision in this build: how much the engine can hold before it falls over — and
it is the reason this agent runs on the runtime it does.

This agent runs on **LiteRT-LM**, an experimental Google runtime that, in the browser, supports exactly two
models: Gemma-4 E2B and E4B. That is the whole allow-list. It is younger, narrower, and less battle-tested
than the obvious alternative — **transformers.js on ONNX Runtime Web**, which drives thirteen model families
in this same codebase, has years of production use behind it, and was where this agent actually started.

We moved off the mature one on purpose. Here is the receipt, because this was not a taste call.

ONNX Runtime Web executes its graph inside a **wasm32** module, and a 32-bit address space caps that module's
linear memory at **4 GiB** — a software ceiling that has nothing to do with your GPU. The dev machine has
\~39 GiB of usable Metal buffer space; the runtime advertised 4. That heap stages everything the loop touches
that isn't resident weights: I/O tensors, kernel dispatch, shape metadata, the growing KV cache, every
tool-result string folded back in. On a heavy multi-tool turn — search, read the result, compute, cite — that
traffic accumulates across iterations and floods the 4 GiB heap. The agent OOM'd at an 8k window. It also
OOM'd at 4k. The wall was never the context size; it was cumulative per-loop traffic through a 32-bit heap.

We pulled the real levers first, because switching runtimes is not where you start. Dropping weights from `q4`
to `q4f16` recovered memory. Pinning the KV cache off the wasm heap onto GPU buffers recovered more. Both
helped; neither was enough. The honest A/B is in the field note below. What those levers bought was a *later*
OOM on the same turn — headroom, not immunity. For a light single-tool turn that might have been the end of
it; for the heavy multi-tool agent we were actually shipping, "later" still means "on stage."

::: info Field note: the KV-cache lever was necessary but not sufficient
By default, transformers.js parks the Gemma-4 KV cache on the wasm32 heap — the single largest growing
consumer across a dispatch loop. There is an auto-pin path meant to move it onto GPU buffers, but it no-ops
on Gemma-4's config shape, so it silently did nothing until we passed the `preferredOutputLocation` map
explicitly (a live probe confirmed the tensors moving from `cpu` to `gpu-buffer`). We shipped it as a battery
default. Then we ran the real agent, at the 8k window that broke it, pin off vs. pin on: **both still OOM'd.**
Emptying the KV cache off the heap was correct and free, and it was not the wall. The rest of the per-loop
traffic floods 4 GiB on its own. That result — a good, shipped optimization that does not move the wall far
enough — is exactly why the decision came down to the runtime and not to one more knob.
:::

LiteRT-LM does not route through ORT's wasm32 chokepoint. Multi-turn Gemma that reliably OOM'd on ONNX simply
ran. So the trade was stated plainly: give up thirteen model families for two, and years of maturity for an
experimental engine, in exchange for the one thing the mature stack could not give us — the heavy agent
completing turns **consistently, where we intended to run it**, a browser tab on real WebGPU. Maturity is a
real asset. It is not the asset that mattered here. **Not every runtime is the right runtime for your
application, and the most mature option is not automatically the right tool for the job** — a narrow engine
that works where you deploy beats a broad one that doesn't.

And here is the part that should have been expensive and wasn't: the switch was cheap. The model runtime is an
**executor** behind ADK's adapter contract, and both engines ship as batteries implementing it —
[`TransformersJsAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/transformers_js/adapter/classes/TransformersJsAdapter) and [`LiteRtLmAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/litert_lm/adapter/classes/LiteRtLmAdapter). Swapping one for the other is swapping which
adapter the `TurnRunner` is constructed with; the planner, the gate cascade, the subtractive pass, the
artifact handles, the persistence layer — none of it knew or cared which engine generated the tokens. The
cost was not zero — LiteRT's stashed session state needed one adapter-specific hook in context assembly — but
a runtime migration that would sink a tightly-coupled agent came down to a battery swap plus one seam. ADK
does not marry you to a model runtime; that is the entire point of the boundary. We changed the hardest,
lowest-level dependency in the stack late in the build, and everything above it held.

## The verdict: a judged matrix

::: tip TL;DR — five findings from five cells

* **The crossover is the thesis**: this E2B with thrift at 8k (0.82) out-scores a much larger open-weights
  reasoning model on naive recency at 128k (0.08).
* **Naive recency is the only strategy that ever collapses** — and also the most expensive from 32k up,
  because re-sending everything every dispatch *is* the cost model (176.3M tokens at 1M vs. thrift's 65.9M,
  for tied quality).
* **Thrift never collapses and is the cheapest or near-cheapest arm on every small-model cell** — zero extra
  model calls, fewest dispatches everywhere but kimi.
* **Compact genuinely wins twice** (both 128k cells, real context pressure + a summarizer budget that pays
  off) and its summary undercuts even thrift's token bill on the reasoning cells — paid compression is a
  real strategy, not a strawman.
* **No strategy wins everywhere.** The arms trade places by cell; picking one is a workload decision, which
  is why both ship as batteries and neither is a default.

**What this matrix is, and isn't**: five cells don't cover every workload, and they're not meant to. The
matrix exists to answer one question — is the thrift method worth shipping as a battery? — and the answer it
gives is yes, not as a silver bullet, but as a **competent generalist**: never the strategy that collapses,
paying benefits at both ends of the spectrum — the capacity-starved edge (an E2B at 8k) and the
capability-rich cloud (a 1M window at 2.7× less cost for tied quality).
:::

This agent's context strategy — thrift, the subtractive pass documented above — is evaluated head-to-head
against a paid summarizing alternative (compact) and a naive-recency baseline across five model/window cells,
on a shared 94-turn stress corpus, scored blind by an LLM committee against a 0–3 doc-verified rubric.

Here's how to read the numbers below: each turn is scored 0–3 by an LLM judge that never sees which strategy
produced the answer, checked against the actual docs (0 = wrong or unsupported, 3 = fully correct and cited),
then averaged across the 94-turn run — so 0.82 means "most turns landed between partial and full credit," not
a percentage. Quality alone is half the bill, so each cell also carries the effort it took to earn that score:
**Σ** is the arm's total token bill for the whole 94-turn run (input + output, summed from the engine's own
per-dispatch counts in the run dumps, dedup-swept and reconciled against the harness reports — compact's Σ
*includes* its summarizer dispatches), **d/t** is model dispatches per answered turn (from the per-turn
harness reports; all cells 94/94 settled, zero errored turns), and compact's cells name its summarizer
overhead separately. Score higher-is-better, Σ and dispatches lower-is-better, same corpus, same judges,
every column:

| model / window | thrift | compact | naive | committee |
| --- | --- | --- | --- | --- |
| gemma e2b @ 8k | **0.82**Σ 3.5M · 5.8 d/t | 0.51Σ 4.1M · 7.1 d/t+85 summ · ~375k tok | 0.75Σ 3.6M · 5.9 d/t | 3-judge |
| gemma 31b @ 32k | 1.55Σ 12.2M · 5.3 d/t | 1.49Σ 14.5M · 7.8 d/t+86 summ · ~412k tok | **1.65**Σ 16.5M · 6.2 d/t | 2-judge (final) |
| gemma 31b @ 128k | 1.35Σ 38.4M · 6.4 d/t | **1.48**Σ ≥39.6M† · 10.7 d/t+80 summ · ~394k tok | 1.15Σ 65.6M · 6.8 d/t | 3-judge |
| kimi-k2.5 @ 128k | 1.13Σ 46.6M · 12.3 d/t | **1.48**Σ 23.3M · 11.5 d/t+89 summ · ~546k tok | 0.08Σ 58.1M · 11.4 d/t | 3-judge |
| gemini-flash @ 1M | 1.14Σ 65.9M · 5.1 d/t | **1.34**Σ 41.0M · 11.9 d/t0 summ (never fired) | 1.13Σ 176.3M · 9.7 d/t | 3-judge |

† Lower bound: this arm's dump lost 74 of 1,006 dispatch records to a killed harness wedge mid-run; the sum
covers the 932 recovered records. Everything else reconciles against its harness report within 1%.

::: warning One row is a 2-judge pool
Four rows are 3-judge panels (gpt-5.5, opus, deepseek — three full 94-turn solo passes per row, per-turn
scores merged, same blind rubric). The gemma-31b @ 32k row pools two judges and is final as-is: its thrift
run transcript aged out of temporary storage before the third pass could run, and because judges score each
turn as a blind comparison across all three arms, re-running the arm would invalidate the existing scores
and force a full three-judge re-evaluation of the cell. No row mixes committees — every number in a row
comes from the same judges.
:::

::: tip 🎯 The whole argument, in one number
Read the crossover first: naive recency scores **0.08** on kimi-k2.5 @ 128k — a reasoning model trapped in
its own scaffolding by bloated, unfiltered context — while the Gemma-4 E2B, an effective-2B model at an 8k
window with thrift, scores **0.82**. **A disciplined effective-2B out-answers a much larger open-weights
reasoning model left to naive recency.** That is not a fluke of one cell; it is the entire thesis of this page
rendered as a number.
:::

Ceiling honesty matters too: the 31b model beats the 2b model at every single arm, every window. Nothing
here claims a small model matches a large one on raw capability — it claims a small model, disciplined,
earns more than an undisciplined large one, a different and more useful claim. Compact tops two cells
outright — kimi-k2.5 @ 128k (1.48 vs. thrift's 1.13) and the 128k gemma-31b control (1.48 vs. 1.35) — real
context pressure with a paid summarizer budget, exactly the profile where paying for compression earns its
keep. Naive even takes the 32k control outright (1.65 vs. thrift's 1.55) — but it's still the only strategy
that ever collapses, cratering to 0.08 on kimi-k2.5 @ 128k. Thrift never collapses anywhere; it just doesn't
always win.

Now read the effort columns against the score they buy, because that's where the comparison actually bites.
Thrift is the lightest arm on every cell but kimi (where all three arms thrash roughly equally —
reasoning-model dispatch counts are model personality, not strategy). Compact's two quality wins are not
free wins: on the 128k control it pays 10.7 dispatches per turn against thrift's 6.4 *plus* 80 summarizer
calls to earn its 1.48-vs-1.35 edge — though on the reasoning cells its summary does real token work,
undercutting thrift's Σ outright (23.3M vs. 46.6M on kimi) by replacing long verbatim history with a short
recap. The headline total from the biggest cell: at gemini-flash @ 1M across the full 94-turn run, thrift
moves **65.9M** tokens against naive's **176.3M** — 2.7× less, for the same quality (1.14 vs. 1.13, a
statistical tie). And naive's Σ is the accumulation story told in money: it never pays a summarizer and runs
midfield on dispatches, yet posts the biggest bill on every cell from 32k up, because re-sending everything
every dispatch *is* the cost model. For the cloud cell, these sums were cross-checked against the serving
gateway's own per-request accounting: request counts match the dump within retry noise, and request bytes
track the token sums at a consistent bytes-per-token ratio across all three arms — two independent ledgers
telling the same story.

::: info Field note: the run where every strategy scored the same — because they secretly were the same
An earlier phase of this matrix was much less interesting than it should have been: on the 31b @ 32k cell,
thrift, compact, and naive all scored within about a tenth of a point of each other (gpt judge, 90 turns:
thrift 1.46, compact 1.33, naive 1.43). A dead heat. For a moment that read as "the strategy barely matters."

It was a bug, and the bug is the lesson. All three arms shed through the *same* subtractive machinery — they
are supposed to differ only in *what* they cut first (thrift by relevance, naive by age, compact by keeping a
running summary and shedding the rest). But compact's summary was timestamped at the epoch so it would render
at the head of history — which also made it the *oldest* turn, so the relevance pass evicted it **first** under
pressure. Compact wasn't compacting. It had quietly degraded into thrift-with-recent-turns: compaction wearing
thrift's clothes. The margins were narrow because we were partly measuring thrift against itself.

Fixing it — making the summary inviolable, so compact either fits it or honestly crashes the turn — is what
produced the real numbers in the table above, where compact and thrift finally diverge. The honest takeaway is
not "combining strategies is always better," which this run does not prove. It is narrower and more durable:
**there is no single right context strategy, the arms genuinely trade places by cell, and the one moment
compaction stopped losing was the moment it accidentally got relevance-shedding underneath it.** That is a
pointer, not a proof — a well-built agent probably wants a *bounded* summary layered on top of relevance
shedding, which is a hybrid we have not measured and did not ship. The receipt here is what the bug cost us: a
result that looked like parity and was really an apparatus measuring its own reflection.
:::

Two cells were run informally and dropped from the table rather than reported as data: a single-credential
provider substitution and one alternate-hosting arm for the tightest cell, both excluded as noise rather than
signal. Full methodology, the complete corpus, and the reasoning behind every threshold live on the
[Context Batteries hub](/batteries/context/) — this table is the headline, that page is the receipts.

## Use it yourself

Everything above composes from shipped, importable pieces, not showcase-only glue:

* **`@nhtio/adk/batteries/context/thrift`** — the subtractive pass this agent runs every dispatch. See
  [the battery reference](/batteries/context/thrift) for the full API and the calibrated relevance floor.
* **`@nhtio/adk/batteries/context/compact`** — the paid summarizing alternative this agent is evaluated
  against. See [the battery reference](/batteries/context/compact) for when it actually earns its cost.
* **`@nhtio/adk/shims`** — the async-resolver seam that lets this agent bind an ADK build without importing
  ADK's source into the docs site's own module graph; the same seam any consumer uses to load a fetched or
  worker-hosted ADK build.

The full source behind every embed on this page — the planner, the gate cascade, the artifact-handle
renderer, the worker proxy, the loss policy, and the harness that wires all four pipelines together — is
walked end to end on [the source companion page](/showcase/punching-above-its-weights-source).