---
url: 'https://adk.nht.io/batteries/llm/litert.md'
description: >-
  On-device LLM execution via Google's LiteRT-LM and .litertlm models on WebGPU.
  A standalone native-API battery, not an OpenAI subclass. And the headline you
  need before you build: in the browser, it runs Gemma and only Gemma — proven
  at the file-format level, not assumed.
---

# LiteRT-LM

## LLM summary — LiteRT-LM battery

* [`LiteRtLmAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/litert_lm/adapter/classes/LiteRtLmAdapter) from `@nhtio/adk/batteries/llm/litert_lm`. On-device LLM via Google's `.litertlm` models + `@litert-lm/core` (an OPTIONAL peer; preview/volatile, pinned exact) on WebGPU. A STANDALONE native-API battery — NOT an OpenAI-wire subclass: it drives `Engine.create() → engine.createConversation({ preface }) → conversation.sendMessageStreaming(): ReadableStream<Message>`.
* WHY this preview-stage runtime over the mature MediaPipe LLM Inference API: MediaPipe LLM is in maintenance-only mode (Google's own docs route every platform to LiteRT-LM as the migration target). The battle-tested option is EoL; LiteRT-LM is where Google now spends — so we follow, pinned exact to contain the churn.
* **Browser support is Gemma-ONLY, proven at the file-format level.** The `@litert-lm/core` web runtime runs `gemma-4-E2B-it-web` / `gemma-4-E4B-it-web` and nothing else: community non-Gemma `.litertlm` builds LOAD but fail generation with `kTfLitePrefillDecode … not supported`. The public `litert-torch export_hf` CLI emits `model_type: tf_lite_prefill_decode`; the working Gemma builds are `tf_lite_artisan_text_decoder` (a non-public Google export path); the JS API doc itself allow-lists only the two Gemma `-web` files. Converting your own non-Gemma model is a dead end today.
* TEXT-out: tool calls + reasoning arrive as TEXT (the runtime emits structured `content` items but they're all `{type:'text'}`; `tool_calls`/`channels` typed in the `.d.ts` but never populated by the runtime). Parsed by `toolCallParser`/`reasoningParser` (default `'auto'`). `toolDelivery` defaults to `'prompt'` because native `preface.tools` throws `template:80` on the gemma-4 `.litertlm`. Sampler is an enum (GREEDY needs k≤1).
* Media INPUT supported per modality flags; media OUTPUT via the opt-in `extractMediaOutputs` hook (default off). `STASH_KEY` `liteRtLm`. Exceptions: `E_INVALID_LITERT_LM_OPTIONS`, `E_LITERT_LM_{CONTEXT_OVERFLOW,STREAM_ERROR,INVALID_TOOL_CALL_ARGS}`, `E_UNSUPPORTED_MEDIA_MODALITY`.

LiteRT-LM runs Google's `.litertlm` models on-device via WebGPU and a bundled wasm runtime
(`@litert-lm/core`). Unlike WebLLM — which is a thin skin over the OpenAI wire — this is a **standalone**
battery driving LiteRT's own native API: `Engine.create()`, then `createConversation({ preface })`, then
`sendMessageStreaming()` returning a `ReadableStream<Message>`. It reuses the ADK's render helpers and the
shared text-parser layer, but the engine underneath is entirely its own.

```ts
import { LiteRtLmAdapter } from '@nhtio/adk/batteries/llm/litert_lm'

const executor = new LiteRtLmAdapter({
  model:
    'https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it-web.litertlm',
})
```

`@litert-lm/core` is an optional, preview-stage peer (pinned exact). Install it yourself; expect it to move.

## Why the experimental library, and not the one that already worked

Fair question. `@litert-lm/core` is preview-stage and we just told you to expect churn. The mature,
battle-tested option was Google's MediaPipe LLM Inference API — the thing that ran Gemma on-device before
LiteRT-LM existed. So why build a first-class battery on the new, volatile runtime instead of the proven one?

Because the proven one is a dead end Google already walked away from. The MediaPipe LLM Inference API is in
**maintenance-only mode** — no new features, no new optimizations — and Google's own docs say so on every
platform page, each pointing you at LiteRT-LM as the migration target. LiteRT-LM is where the KV-cache work,
the multimodal input, and every future model land. "Battle-tested" and "end-of-life" are, infuriatingly, the
same sentence here.

So the real choice was never "stable vs experimental." It was "experimental, but where Google is actually
spending its money" versus "stable, but abandoned by the only people who can maintain it." We took the one
with a pulse and pinned it exact so its churn can't blindside you. Building the battery on the deprecated
runtime would be shipping you onto a foundation its own author has already stopped pouring — which is a far
worse bet than a peer that moves.

## The headline: in the browser, it's Gemma or nothing

This isn't a limitation we chose — it's one we hit, then tried to get around, then proved we couldn't.

The `@litert-lm/core` web runtime will run exactly two models: `gemma-4-E2B-it-web` and `gemma-4-E4B-it-web`.
Not "Gemma is best-supported." Two files. The evidence:

* **Community non-Gemma `.litertlm` builds load, then die at generation.** We added a real `SmolLM2-360M`
  community `.litertlm` and ran it on a real GPU: it loaded — read its template, built the engine, reached
  generation — then failed on both `sendMessage` and `sendMessageStreaming` with
  `Streaming kTfLitePrefillDecode models is not supported yet`. Loadability was never the wall; the
  prefill/decode kernel is.
* **The file format is the discriminator.** `litert-lm-peek` on the working `gemma-4-E2B-it-web` build shows
  `model_type: tf_lite_artisan_text_decoder`. The community builds — and anything the public
  `litert-torch export_hf` CLI produces — are `tf_lite_prefill_decode`, the type the web runtime rejects.
* **You cannot convert your way out.** The public CLI has no flag that emits the Artisan decoder; it only
  produces `prefill_decode`. "Artisan" is an internal Google export path. We installed the toolchain,
  converted SmolLM2 successfully, and the output was the wrong `model_type` — confirmed by peeking the file.
* **Google's own JS doc agrees.** The LiteRT-LM Web API doc explicitly supports "only" those two Gemma `-web`
  files, "working toward" general `.litertlm` support.

So: **for the browser, LiteRT-LM is a Gemma runtime.** If you need a non-Gemma model on-device in a tab, use
[transformers.js](./transformers-js) (ONNX, env-neutral, broad family support). The full investigation —
including the conversion attempt — is in the [on-device build showcase](/showcase/building-on-device-batteries).

## Tested entries

| Entry | Result |
| --- | --- |
| `gemma-4-E2B-it-web.litertlm` | ✅ Real WebGPU tool-calling, end-to-end. The one that works. |
| `SmolLM2-360M` (community, non-web) | ⛔ Loads, then `kTfLitePrefillDecode` at generation — the failure behind the Gemma-only wall above. Recorded as a probe finding. |
| `Gemma3-1B-IT` (non-web `.litertlm`) | ⛔ Load probe: a non-`-web` build; whether the WebGPU runtime loads it is recorded as data, not treated as a hard failure. |
| `gemma-4-E2B-it-web` (image gate) | ◍ Probe: does 0.13.1 WASM honor image content items + `visionModalityEnabled`? Outcome recorded as data. |

## Text-out, and the runtime that under-delivers its own types

LiteRT-LM is text-out. The runtime returns `Message.content` as a structured `MessageContentItem[]` — but on
the gemma-4-web model every item is `{type:'text'}`. The `.d.ts` types `tool_calls` and `channels` on the
output message; the runtime never populates them. **Trust the types for inputs; trust a real run for what the
runtime actually emits** — the published docs lag the library, and the types over-promise on output. So tool
calls and reasoning come back as text, extracted by the shared parser layer (`toolCallParser` /
`reasoningParser`, default `'auto'`).

Two specifics that follow from that:

* **`toolDelivery` defaults to `'prompt'`.** Passing tools through the native `preface.tools` channel makes
  the gemma-4 `.litertlm`'s own bundled chat template throw `Failed to apply template: undefined value`
  (`template:80`) — a known Gemma-4 template bug that bites multiple runtimes, not just this one. So the battery
  renders tool definitions as system-prompt text instead, and the `gemma` parser extracts the
  decoder-stripped `call:NAME{…}` form the model emits at runtime. Set `toolDelivery: 'native'` only if your
  `.litertlm` has a template that survives `preface.tools`.
* **The sampler is an enum.** `samplerParams.type` is the numeric `SamplerType` (`1` TOP\_K / `2` TOP\_P /
  `3` GREEDY). GREEDY requires `k ≤ 1`; the validation enforces it.

## Media + errors

Media INPUT is gated by `visionModalityEnabled` / `audioModalityEnabled`. Media OUTPUT rides the opt-in
`extractMediaOutputs` hook (default absent → text-out, unchanged) — see the
[Transformers.js page](./transformers-js#media-output) for the shared seam's mechanics. Errors are the typed
`E_LITERT_LM_*` family (context overflow, stream error, invalid tool-call args) plus
`E_INVALID_LITERT_LM_OPTIONS` and `E_UNSUPPORTED_MEDIA_MODALITY`. Full option + exception detail in
[Assembly → LLM batteries](/assembly/batteries-llm).