LiteRT-LM

LiteRT-LM runs Google's .litertlm models on-device via WebGPU and a bundled wasm runtime (@litert-lm/core). Unlike WebLLM — which is a thin skin over the OpenAI wire — this is a standalone battery driving LiteRT's own native API: Engine.create(), then createConversation({ preface }), then sendMessageStreaming() returning a ReadableStream<Message>. It reuses the ADK's render helpers and the shared text-parser layer, but the engine underneath is entirely its own.

import { LiteRtLmAdapter } from '@nhtio/adk/batteries/llm/litert_lm'

const executor = new LiteRtLmAdapter({
  model:
    'https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it-web.litertlm',
})

@litert-lm/core is an optional, preview-stage peer (pinned exact). Install it yourself; expect it to move.

Why the experimental library, and not the one that already worked

Fair question. @litert-lm/core is preview-stage and we just told you to expect churn. The mature, battle-tested option was Google's MediaPipe LLM Inference API — the thing that ran Gemma on-device before LiteRT-LM existed. So why build a first-class battery on the new, volatile runtime instead of the proven one?

Because the proven one is a dead end Google already walked away from. The MediaPipe LLM Inference API is in maintenance-only mode — no new features, no new optimizations — and Google's own docs say so on every platform page, each pointing you at LiteRT-LM as the migration target. LiteRT-LM is where the KV-cache work, the multimodal input, and every future model land. "Battle-tested" and "end-of-life" are, infuriatingly, the same sentence here.

So the real choice was never "stable vs experimental." It was "experimental, but where Google is actually spending its money" versus "stable, but abandoned by the only people who can maintain it." We took the one with a pulse and pinned it exact so its churn can't blindside you. Building the battery on the deprecated runtime would be shipping you onto a foundation its own author has already stopped pouring — which is a far worse bet than a peer that moves.

The headline: in the browser, it's Gemma or nothing

This isn't a limitation we chose — it's one we hit, then tried to get around, then proved we couldn't.

The @litert-lm/core web runtime will run exactly two models: gemma-4-E2B-it-web and gemma-4-E4B-it-web. Not "Gemma is best-supported." Two files. The evidence:

• Community non-Gemma .litertlm builds load, then die at generation. We added a real SmolLM2-360M community .litertlm and ran it on a real GPU: it loaded — read its template, built the engine, reached generation — then failed on both sendMessage and sendMessageStreaming with Streaming kTfLitePrefillDecode models is not supported yet. Loadability was never the wall; the prefill/decode kernel is.
• The file format is the discriminator. litert-lm-peek on the working gemma-4-E2B-it-web build shows model_type: tf_lite_artisan_text_decoder. The community builds — and anything the public litert-torch export_hf CLI produces — are tf_lite_prefill_decode, the type the web runtime rejects.
• You cannot convert your way out. The public CLI has no flag that emits the Artisan decoder; it only produces prefill_decode. "Artisan" is an internal Google export path. We installed the toolchain, converted SmolLM2 successfully, and the output was the wrong model_type — confirmed by peeking the file.
• Google's own JS doc agrees. The LiteRT-LM Web API doc explicitly supports "only" those two Gemma -web files, "working toward" general .litertlm support.

So: for the browser, LiteRT-LM is a Gemma runtime. If you need a non-Gemma model on-device in a tab, use transformers.js (ONNX, env-neutral, broad family support). The full investigation — including the conversion attempt — is in the on-device build showcase.

Tested entries

Entry	Result
gemma-4-E2B-it-web.litertlm	✅ Real WebGPU tool-calling, end-to-end. The one that works.
SmolLM2-360M (community, non-web)	⛔ Loads, then kTfLitePrefillDecode at generation — the failure behind the Gemma-only wall above. Recorded as a probe finding.
Gemma3-1B-IT (non-web .litertlm)	⛔ Load probe: a non--web build; whether the WebGPU runtime loads it is recorded as data, not treated as a hard failure.
gemma-4-E2B-it-web (image gate)	◍ Probe: does 0.13.1 WASM honor image content items + visionModalityEnabled? Outcome recorded as data.

Text-out, and the runtime that under-delivers its own types

LiteRT-LM is text-out. The runtime returns Message.content as a structured MessageContentItem[] — but on the gemma-4-web model every item is {type:'text'}. The .d.ts types tool_calls and channels on the output message; the runtime never populates them. Trust the types for inputs; trust a real run for what the runtime actually emits — the published docs lag the library, and the types over-promise on output. So tool calls and reasoning come back as text, extracted by the shared parser layer (toolCallParser / reasoningParser, default 'auto').

Two specifics that follow from that:

• toolDelivery defaults to 'prompt'. Passing tools through the native preface.tools channel makes the gemma-4 .litertlm's own bundled chat template throw Failed to apply template: undefined value (template:80) — a known Gemma-4 template bug that bites multiple runtimes, not just this one. So the battery renders tool definitions as system-prompt text instead, and the gemma parser extracts the decoder-stripped call:NAME{…} form the model emits at runtime. Set toolDelivery: 'native' only if your .litertlm has a template that survives preface.tools.
• The sampler is an enum. samplerParams.type is the numeric SamplerType (1 TOP_K / 2 TOP_P / 3 GREEDY). GREEDY requires k ≤ 1; the validation enforces it.

Media + errors

Media INPUT is gated by visionModalityEnabled / audioModalityEnabled. Media OUTPUT rides the opt-in extractMediaOutputs hook (default absent → text-out, unchanged) — see the Transformers.js page for the shared seam's mechanics. Errors are the typed E_LITERT_LM_* family (context overflow, stream error, invalid tool-call args) plus E_INVALID_LITERT_LM_OPTIONS and E_UNSUPPORTED_MEDIA_MODALITY. Full option + exception detail in Assembly → LLM batteries.