@nhtio/adk/batteries/llm/litert_lm/adapter

Browser/WebGPU executor adapter for Google's LiteRT-LM (@litert-lm/core).

Remarks

On-device LLM inference via WebGPU + a bundled wasm runtime, .litertlm models. Unlike the WebLLM battery (a thin extension of the OpenAI Chat Completions wire shape), LiteRT-LM has its own API — Engine.create() → engine.createConversation({ preface }) → conversation.sendMessageStreaming(): ReadableStream<Message> — with native Message/Tool/tool_calls/tool_response shapes (tool-call arguments arrive as a parsed object, not a JSON string). So this is a standalone adapter that reuses the ADK's format-agnostic render helpers but maps history/tools/results to LiteRT's shapes.

Three pluggable layers mirror the other LLM batteries: swappable translation helpers, three-layer options merging (constructor → executor() overrides → ctx.stash.liteRtLm), and an injectable/lazy engine (engine or createEngine, defaulting to a dynamic @litert-lm/core import).

The published @litert-lm/core docs lag the library — every wire field here is mapped against the installed package's type declarations, the source of truth. The dependency is young (pinned exact); re-verify on upgrade.

Classes

Class	Description
LiteRtLmAdapter	Cross-environment executor adapter for LiteRT-LM.

Functions

Function

Description

isEngineContextOverflowMessage

Does this raw engine message report an INPUT context-cap overflow — the prompt's token ids exceed the engine's fixed maxNumTokens? The LiteRT-web runtime throws e.g. Input token ids are too long. Exceeding the maximum number of tokens allowed: 12596 >= 12288. Matched so the raw throw can be translated into the typed E_LITERT_LM_CONTEXT_OVERFLOW instead of the generic stream error — this is the ENGINE BACKSTOP that fires when the optional pre-dispatch guard is unarmed or undercounts. Exported so a host can classify a thrown/caught error the same way.