Token Thrift

Every dispatch starts from a working set — messages, memories, retrieved chunks, thoughts, tools — that is routinely bigger than the window it has to fit in. Token Thrift's answer is purely subtractive: measure everything, then cut by measured relevance in a fixed, lowest-signal-first order until it fits, paying zero extra model calls to decide what to drop.

Token Thrift ships with @nhtio/adk — no separate install, no peer dependency. Import it from the battery subpath:

import { subtractToFit, type WorkingSet } from '@nhtio/adk/batteries/context/thrift'

This page is the API surface: what each export does, what it needs from you, and the real option shapes. For why subtractive beats additive, the research behind it, and when this policy is the wrong one, see Token Thrift (The Loop). For the evaluated numbers (thrift vs. compact vs. naive across five models), see the domain hub.

The one thing thrift cannot do itself: count tokens

Every entry point that measures anything requires an injected estimateTokens — a function you pass in, not a bundled default, because thrift has no tokenizer of its own and refuses to guess at one:

export type EstimateTokensFn = (value: string, encoding: string, ctx?: unknown) => number

There is no default and no bundled tokenizer — a caller who omits it gets an immediate, clearly-named thrown error, never a silent guess. If you already hold core's Tokenizable, the recipe is one line, and it gets you byte-for-byte parity with what your own overflow guard counts (including ctx-resolved dynamic content):

import { Tokenizable } from '@nhtio/adk'

const estimateTokens = (value: string, encoding: string, ctx?: unknown) =>
  Tokenizable.estimateTokens(value, encoding, ctx)

Measuring under a custom (non-built-in) encoding? Register it ONCE with Tokenizable's own registerTokenEstimator, then keep passing Tokenizable.estimateTokens exactly as above — the registered encoding becomes a drop-in peer of the built-ins, so thrift never needs to know it's non-standard.

If you have no tokenizer at all, a plain chars/4 estimator is a legitimate starting point (it's what English runs at, roughly) — just know it's a rough proxy, not a real count:

const estimateTokens = (value: string) => Math.ceil(value.length / 4)

subtractToFit — the whole pass

function subtractToFit(
  ws: WorkingSet,
  contextWindow: number,
  relevantToolNames: string[],
  options: SubtractToFitOptions
): ThriftTrace

ws is your working set — everything reasonable to send, before any shedding:

interface WorkingSet {
  systemPrompt: ContentLike | string
  standingInstructions?: Array<ContentLike | string>
  messages: WorkingMessage[]
  memories: WorkingMemory[]
  retrievables: WorkingRetrievable[]
  thoughts: WorkingThought[]
  tools: WorkingToolRegistry
  toolCalls?: WorkingToolCall[]
  image?: WorkingImage
}

subtractToFit mutates ws in place — thoughts, retrievables, memories, messages, toolCalls, image.kept, and the tool registry's hidden set may all change — and returns a ThriftTrace describing what happened:

interface ThriftTrace {
  contextWindow: number
  reserve: number
  budget: number // contextWindow - reserve
  totalBefore: number
  totalAfter: number
  fits: boolean
  refused: boolean // !fits — even the irreducible floor exceeded the window
  buckets: BucketTrace[] // one entry per step, before/after tokens + counts
}

refused: true means even the floor (system prompt + standing instructions + the newest turn + the output reserve) doesn't fit — a bounded refusal, not a truncated, incoherent dispatch.

import { subtractToFit, type WorkingSet } from '@nhtio/adk/batteries/context/thrift'
import { Tokenizable } from '@nhtio/adk'

const ws: WorkingSet = {
  systemPrompt: mySystemPrompt,
  messages: myMessages,
  memories: myMemories,
  retrievables: myRagChunks,
  thoughts: myThoughts,
  tools: myToolRegistry,
}

const trace = subtractToFit(ws, model.contextWindow, myShortlistedToolNames, {
  estimateTokens: (value, encoding, ctx) => Tokenizable.estimateTokens(value, encoding, ctx),
  outputReserve: model.maxOutputTokens,
})

if (!trace.fits) {
  // Refuse the dispatch — even the irreducible floor exceeds this window.
}
// ws.messages / ws.memories / ws.retrievables / ws.thoughts / ws.toolCalls are now the surviving,
// budget-fitting subset. Render the dispatch from ws, not from your original working set.

The shed order

The pass subtracts in a fixed, lowest-signal-first order — full narrative on Token Thrift (The Loop) and in the source's own step-by-step comments (src/batteries/context/thrift/subtractive_pass.ts); the short version:

1. Prior-turn thoughts stripped (Gemma model-card §3 — "No Thinking Content in History"; also just pure thrift, since prior reasoning is the highest-volume, lowest-reuse content there is). Gated by stripPriorTurnThoughts (default true); keepThoughtIds preserves e.g. a fresh this-turn plan thought.
2. Tools hidden except the turn's shortlist (relevantToolNames) — hidden tools cost 0 schema tokens but stay callable via a catalog.
3. Image — the single biggest token hog, dropped first if over budget.
   • 3b. Prior-turn tool results, oldest first; this-turn results are protected (evicting one the model just fetched causes an immediate re-request), then capped to the newest thisTurnResultKeep (default 3) as a last-resort backstop for a deep read-loop turn.
4. RAG tail — lowest score first, keeping the best-ranked chunks.
5. Low-importance memories — lowest first.
6. Stale __eph-* ephemeral messages — only the latest survives.
7. Oldest conversation turns — protects any message matching isSummaryMessage (compact's running summary) and always keeps the newest turn.
8. Surviving guidance thoughts, oldest first — protectThoughtIds never shed.
9. Visible tools, last resort — ranked by shedRank (lower sheds first), protectedToolNames shed last, driven toward zero rather than left as an unsheddable floor.

SubtractToFitOptions

Only estimateTokens has no default. Everything else is a calibrated default you can override:

Option	Default	What it does
estimateTokens	— (required)	Token measurement, see above.
encoding	'cl100k_base'	Opaque string threaded to estimateTokens.
outputReserve	—	Exact tokens held back for output; pass real maxTokens.
reserveFraction	0.35	Fallback fraction of the window when outputReserve is unknown.
stripPriorTurnThoughts	true	Gemma §3 strip (step 1).
keepThoughtIds	—	Thought ids preserved through the strip.
renderTools	—	Real tool-declaration renderer; falls back to a name: description proxy (can undercount schema-heavy tools by an order of magnitude).
protectThoughtIds	—	Thought ids never shed in step 9 (the this-turn scaffolding the model needs to answer at all).
renderCtx	—	Live dispatch context, so a dynamic/evaluatable value measures at its resolved (larger) size, matching your own guard.
protectedToolNames	—	Plan-committed, not-yet-called tools — unsheddable until every other tool is already gone.
shedRank	every tool ranks 0	Last-resort tool shed priority; lower sheds first.
isEphemeralMessage	(m) => m.id.startsWith('__eph-')	Ephemeral control-message predicate (step 7).
isSummaryMessage	(m) => m.id === '__compact-summary'	Protects compact's running summary from step 8's oldest-turn shed.
thisTurnResultKeep	3	Newest-N this-turn tool-result backstop.

stripPriorTurnThoughts — standalone

function stripPriorTurnThoughts(
  ws: Pick<WorkingSet, 'thoughts'>,
  options: EstimatorOptions,
  keepIds?: ReadonlySet<string>
): { dropped: number; tokens: number }

The Gemma §3 rationale, in the source's own words: thoughts from previous model turns must not be re-added before the next user turn. subtractToFit calls this internally as step 1 (gated by stripPriorTurnThoughts), but it's also exported standalone for a caller who wants the strip without the rest of the pass — e.g. applying it once at turn-assembly time, ahead of a separate budget-fit step.

Relevance-based turn selection

subtractToFit sheds messages oldest-first once the budget blows (step 7/8 above) — a purely recency-based policy for WHICH turns get sent at all. selectRelevantTurns is a smarter alternative for that upstream decision: instead of "the last N turns," it walks the entire history and keeps anything lexically relevant to the current query, regardless of age.

function groupHistoryIntoTurns<M extends RelevanceMessage, TC extends RelevanceToolCall>(
  messages: readonly M[],
  toolCalls: readonly TC[]
): Array<HistoryTurn<M, TC>>

function selectRelevantTurns<M extends RelevanceMessage, TC extends RelevanceToolCall>(
  turns: ReadonlyArray<HistoryTurn<M, TC>>,
  queryText: string,
  options: SelectRelevantTurnsOptions
): Array<HistoryTurn<M, TC>>

function selectNaiveTurns<M extends RelevanceMessage, TC extends RelevanceToolCall>(
  turns: ReadonlyArray<HistoryTurn<M, TC>>,
  historyBudget: number,
  options: SelectNaiveTurnsOptions
): Array<HistoryTurn<M, TC>>

groupHistoryIntoTurns first collapses a flat message + tool-call history into HistoryTurns (a user message through the next assistant message, inclusive, with attributed tool calls). Then:

import {
  groupHistoryIntoTurns,
  selectRelevantTurns,
} from '@nhtio/adk/batteries/context/thrift'

const turns = groupHistoryIntoTurns(myMessages, myToolCalls)

const kept = selectRelevantTurns(turns, currentUserQuery, {
  estimateTokens,
  keepRecent: 2, // always-kept newest turns, for coreference ("it", "that file")
  historyBudget: myOlderHistoryTokenBudget,
})

selectNaiveTurns is the FIFO comparison baseline — the arm selectRelevantTurns was evaluated against, not a recommended policy. It's exported as an honest, drop-in alternative for a caller who wants plain recency, or who is reproducing the evaluation.

The calibration story

selectRelevantTurns scores a turn's relevance as the fraction of the query's content words it shares (see relevanceToQuery/contentTokens in the source — a deliberately crude, zero-model lexical-overlap signal: no stemming, no semantic embedding, just lowercased alphanumeric runs of length ≥4). The floor that score has to clear scales with how full the older-history budget already is:

const RELEVANCE_FLOOR_MIN = 0.125 // empty window: permissive
const RELEVANCE_FLOOR_MAX = 0.43 // full window: strict
const RELEVANCE_FLOOR_CURVE = 2 // convex

function scaledRelevanceFloor(utilization: number): number

These three constants were calibrated, not guessed — against a 94-turn stress corpus, scored independently by three oracles: gpt-5.5, gemini-3.5-flash, and claude-haiku-4.5 acting as ground-truth judges (the same role a judge model plays in the evaluation matrix on the domain hub).

• RELEVANCE_FLOOR_MIN = 0.125 is the F1-optimal floor (best precision/recall balance when the window has room to spare) — gpt-5.5 and claude-haiku-4.5 landed on this exact value, gemini-3.5-flash landed at 0.111. 0.125 was chosen as the shared default, a hair stricter than gemini's number — the calibration run characterized this as "slightly permissive" rather than under-inclusive, the safer direction to err in when the window has headroom.
• RELEVANCE_FLOOR_MAX = 0.43 is the high-precision floor (precision ≥ 0.9 — "when in doubt, don't miss on the side of dropping something the user is about to ask about"). The underlying calibration run landed on 0.4286, bit-identical across all three judge models — a striking cross-model agreement that grounds it as a real property of the corpus, not judge idiosyncrasy — rounded to the shipped 0.43 constant.
• The curve between them is convex (k = 2, utilization raised to this power) rather than linear — stays permissive across most of the window's life, only tightening sharply once the window is genuinely filling up, which the calibration run validated as the right shape (not an arbitrary choice).

SelectRelevantTurnsOptions exposes floorMin/floorMax/floorCurve overrides — but overriding any of them without re-running the oracle calibration against your own corpus forfeits the calibrated guarantee these numbers carry. Treat the defaults as load-bearing unless you've done the calibration work yourself.

Zero-coupling by construction

Every type in ./contracts is a local, structural (duck-typed) declaration — WorkingMessage, WorkingMemory, WorkingRetrievable, WorkingThought, WorkingTool, WorkingToolRegistry, WorkingToolCall, WorkingImage. A core Message, Memory, Retrievable, Thought, Tool, or ToolRegistry instance satisfies these structurally, but you never have to import them to call this battery, and the battery never imports them to accept your objects. This is "surface, don't impose" in its most literal form: the battery imposes a shape, never a class. The batteries are dependency-free and tree-shakeable: everything above works against plain objects plus an injected estimateTokens, exactly as shown in the usage examples on this page — no @nhtio/adk class is ever required to call subtractToFit.

Where to go next

• Context Batteries hub — the head-to-head matrix, and when to reach for thrift vs. compact.
• Compact battery reference — the paired summarizing strategy, and the isSummaryMessage contract that lets the two compose.
• Token Thrift (The Loop) — the concept, the research, the boundaries.
• Punching Above Its Weights showcase — live demo on a real on-device model.