Skip to content
1 min read · 141 words

Variable: DEFAULT_ENCODING

ts
const DEFAULT_ENCODING: "cl100k_base" = "cl100k_base";

Defined in: src/batteries/context/thrift/subtractive_pass.ts:74

The token encoding this battery measures under, by default.

Remarks

'cl100k_base' — a widely-available, model-agnostic tiktoken encoding — is the default because this battery is core-agnostic: unlike the flagship reference agent it was extracted from (which hard-codes the gemma encoding because its engine IS Gemma via LiteRT-LM), this battery has no opinion on which model you run. cl100k_base is a reasonable, broadly-supported baseline for a caller who hasn't thought about encodings yet; a caller running Gemma, Claude, or any other model with a distinct tokenizer should pass their own encoding (and a matching EstimateTokensFn) so the budget math agrees with what their model/battery actually counts. The encoding identifier is an opaque string as far as this module is concerned — it is never validated or interpreted here, only threaded through to the injected EstimateTokensFn.