---
url: 'https://adk.nht.io/batteries/llm/webllm.md'
description: >-
  Browser-native LLM execution via MLC + WebGPU. A wire-format battery whose
  model runs in the tab — no server, no API key. The catch isn't the battery,
  it's the model: a 3B in a 4096-token window, and a hard rule about never
  mixing embedding runtimes.
---

# WebLLM

## LLM summary — WebLLM battery

* [`WebLLMChatCompletionsAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/webllm_chat_completions/adapter/classes/WebLLMChatCompletionsAdapter) from `@nhtio/adk/batteries/llm/webllm_chat_completions`. Browser-native model execution via MLC's WebLLM (WebGPU). A complete [`DispatchExecutorFn`](https://adk.nht.io/api/@nhtio/adk/dispatch_runner/type-aliases/DispatchExecutorFn); a thin extension of the OpenAI Chat Completions wire shape, so tool calls come back STRUCTURED (no text parsing). `STASH_KEY` `webLLMChatCompletions`.
* Runs the model IN THE TAB: no server, no API key, no network at inference time. Requires WebGPU.
* VERIFIED (in production / the Ask ADK eval — NOT the real-model matrix, which has no WebLLM entries): **Llama-3.2-3B-Instruct-q4f32\_1-MLC** (the model behind the [Ask ADK](/showcase/ask-adk) docs agent — hard 4096-token context, forcing higher silently corrupts output) and **Qwen3.5-2B** (Ask ADK bake-off tie with Llama-3.2-3B). WebLLM's prebuilt catalog lists ~100 MLC models; those two are what we ran — the rest are available-but-unconfirmed.
* Recommendation, not a rule: don't mix transformers.js and WebLLM/MLC embedding vectors in one corpus unless you've measured their cross-runtime compatibility — we saw the same arctic-embed model differ by ~0.77 cosine across the two runtimes.
* Media: image/audio/document via Chat Completions content blocks; video throws `E_UNSUPPORTED_MEDIA_MODALITY`. Exceptions: `E_INVALID_WEBLLM_CHAT_COMPLETIONS_OPTIONS`, `E_WEBLLM_CHAT_COMPLETIONS_{CONTEXT_OVERFLOW,STREAM_ERROR,INVALID_TOOL_CALL_ARGS}`, `E_UNSUPPORTED_MEDIA_MODALITY`.

WebLLM runs a real language model inside the browser tab, on the user's GPU, via MLC's WebGPU runtime. No
server, no API key, no token leaving the machine at inference time. As a battery it is a thin extension of the
OpenAI Chat Completions wire shape, which means — unlike the other two on-device batteries — it hands back
**structured** tool calls. You don't parse text here; MLC's compiled grammar already did. That makes it the
easiest on-device battery to reason about, and the [Ask ADK agent](/showcase/ask-adk) in this site's own
header is the proof it holds up in production: a 3B model, a browser tab, grounded answers with citations.

If you run a *small* model whose compiled grammar occasionally lets a call slip into `content` instead of
`tool_calls`, the opt-in `localToolCallParser` recovers it — **only when the engine returned no structured
call** (the engine always wins), the same fallback the other native-API batteries expose. See
[the shared contract](/batteries/llm/shared-contract#the-same-leak-on-the-native-api-batteries-localtoolcallparser).

```ts
import { WebLLMChatCompletionsAdapter } from '@nhtio/adk/batteries/llm/webllm_chat_completions'

const executor = new WebLLMChatCompletionsAdapter({
  model: 'Llama-3.2-3B-Instruct-q4f32_1-MLC',
})
```

## Verified models

WebLLM's prebuilt config advertises roughly a hundred MLC-compiled models. That list is a **catalog of what's
available**, not a claim about what works in an agent loop — and we are not going to launder one into the
other. Unlike the [transformers.js](./transformers-js) and [LiteRT-LM](./litert) pages, these rows are *not*
from the gated real-model matrix (it has no WebLLM entries) — they're from Ask ADK production + its eval:

| Model | Source | What it's for / what bit |
| --- | --- | --- |
| `Llama-3.2-3B-Instruct-q4f32_1-MLC` | Ask ADK production | The [Ask ADK](/showcase/ask-adk) docs agent. **Hard 4096-token context window** — it's the model's compiled limit, and forcing a higher `contextWindow` silently corrupts output rather than erroring. Budget against 4096. |
| `Qwen3.5-2B` | Ask ADK eval | Bake-off tie with Llama-3.2-3B (9/12 on the same eval, with *identical* failures — pipeline-bound, not model-bound). A viable swap. |
| *(~100 others in the MLC catalog)* | catalog only | Available to load; we didn't run them. "It's in the catalog" is not "it works." |

The 4096 cap is the single most important fact on this page if you're building with the 3B. It is not a
`contextWindow` you can raise — it's the window the model was compiled with, so the ADK's token budgeting has
to treat it as a hard ceiling. The Ask ADK showcase walks through exactly how RAG-first budgeting sheds
context to fit it.

## The cross-runtime embedding trap

::: warning Our recommendation: don't mix embedding runtimes in one corpus
If you also use embeddings, think twice before indexing some vectors with a transformers.js embedder and
others with the WebLLM/MLC build of "the same" model. We measured it: the same arctic-embed model produced
vectors only **~0.77 cosine** similar across the two runtimes — different enough that nearest-neighbor search
returns garbage when they're mixed. Nothing in the kit stops you; it's your corpus. But it cost us a real
migration, so unless you've measured cross-runtime compatibility for your model and retrieval threshold, pick
one embedding runtime per corpus and stay there.
:::

## Media + errors

Image/audio/document map to Chat Completions content blocks (it's a wire-shape battery); video throws
`E_UNSUPPORTED_MEDIA_MODALITY` unless `unsupportedMediaPolicy` degrades it. Errors are the typed
`E_WEBLLM_CHAT_COMPLETIONS_*` family (context overflow, stream error, invalid tool-call args) plus
`E_INVALID_WEBLLM_CHAT_COMPLETIONS_OPTIONS`. Full surface in [Assembly → LLM batteries](/assembly/batteries-llm).