Type Alias: EstimateTokensFn

type EstimateTokensFn = (
  value: string,
  encoding: string,
  ctx?: unknown,
) => number;

Defined in: src/batteries/context/thrift/contracts.ts:60

Estimate the token cost of a rendered string under a named encoding, optionally resolved against a live dispatch context.

Parameters

Parameter	Type	Description
value	string	The exact string whose token cost should be measured (already rendered — this function never renders anything itself).
encoding	string	The encoding identifier to measure under (e.g. 'cl100k_base', 'gemma') — an opaque string as far as this battery is concerned; the injected function interprets it.
ctx?	unknown	The live dispatch context, when measuring a value whose rendered size can depend on it (an evaluatable/dynamic value). Static content ignores this parameter.

Returns

number

The estimated token count, as a plain synchronous number.

Remarks

This is the ONE capability the subtractive pass cannot perform itself — it has no bundled tokenizer, by design (a context-management algorithm should not force a specific encoding on every caller). Every token-accounting entry point in this battery therefore takes an estimateTokens option of this shape and REQUIRES it (see @nhtio/adk/batteries/context/thrift/subtractive_pass!subtractToFit).

The signature mirrors core's own Tokenizable.estimateTokens(encoding, ctx?) method deliberately — a caller already holding Tokenizable can inject (value, encoding, ctx) => new Tokenizable(value).estimateTokens(encoding, ctx) (or, more cheaply, delegate to Tokenizable.estimateTokens as a static call) and get byte-for-byte parity with what core's own overflow guard counts. The ctx parameter exists for exactly that reason: a dynamic/evaluatable value's rendered size can depend on the live dispatch context (an interpolated system prompt is the canonical example), and a battery that measured only the static form would under-count relative to what the caller's guard actually ships — a real production bug this contract exists to prevent (see the ctx-resolved measurement note on @nhtio/adk/batteries/context/thrift/subtractive_pass!subtractToFit).

A caller measuring under a CUSTOM (non-built-in) encoding registers it once via @nhtio/adk!registerTokenEstimator and otherwise injects Tokenizable.estimateTokens exactly as shown above — the registration makes the custom encoding a drop-in peer of the built-ins, so this battery never needs to know the encoding is non-standard.