WebLLM

WebLLM runs a real language model inside the browser tab, on the user's GPU, via MLC's WebGPU runtime. No server, no API key, no token leaving the machine at inference time. As a battery it is a thin extension of the OpenAI Chat Completions wire shape, which means — unlike the other two on-device batteries — it hands back structured tool calls. You don't parse text here; MLC's compiled grammar already did. That makes it the easiest on-device battery to reason about, and the Ask ADK agent in this site's own header is the proof it holds up in production: a 3B model, a browser tab, grounded answers with citations.

If you run a small model whose compiled grammar occasionally lets a call slip into content instead of tool_calls, the opt-in localToolCallParser recovers it — only when the engine returned no structured call (the engine always wins), the same fallback the other native-API batteries expose. See the shared contract.

import { WebLLMChatCompletionsAdapter } from '@nhtio/adk/batteries/llm/webllm_chat_completions'

const executor = new WebLLMChatCompletionsAdapter({
  model: 'Llama-3.2-3B-Instruct-q4f32_1-MLC',
})

Verified models

WebLLM's prebuilt config advertises roughly a hundred MLC-compiled models. That list is a catalog of what's available, not a claim about what works in an agent loop — and we are not going to launder one into the other. Unlike the transformers.js and LiteRT-LM pages, these rows are not from the gated real-model matrix (it has no WebLLM entries) — they're from Ask ADK production + its eval:

Model	Source	What it's for / what bit
Llama-3.2-3B-Instruct-q4f32_1-MLC	Ask ADK production	The Ask ADK docs agent. Hard 4096-token context window — it's the model's compiled limit, and forcing a higher contextWindow silently corrupts output rather than erroring. Budget against 4096.
Qwen3.5-2B	Ask ADK eval	Bake-off tie with Llama-3.2-3B (9/12 on the same eval, with identical failures — pipeline-bound, not model-bound). A viable swap.
(~100 others in the MLC catalog)	catalog only	Available to load; we didn't run them. "It's in the catalog" is not "it works."

The 4096 cap is the single most important fact on this page if you're building with the 3B. It is not a contextWindow you can raise — it's the window the model was compiled with, so the ADK's token budgeting has to treat it as a hard ceiling. The Ask ADK showcase walks through exactly how RAG-first budgeting sheds context to fit it.

The cross-runtime embedding trap

Our recommendation: don't mix embedding runtimes in one corpus

If you also use embeddings, think twice before indexing some vectors with a transformers.js embedder and others with the WebLLM/MLC build of "the same" model. We measured it: the same arctic-embed model produced vectors only ~0.77 cosine similar across the two runtimes — different enough that nearest-neighbor search returns garbage when they're mixed. Nothing in the kit stops you; it's your corpus. But it cost us a real migration, so unless you've measured cross-runtime compatibility for your model and retrieval threshold, pick one embedding runtime per corpus and stay there.

Media + errors

Image/audio/document map to Chat Completions content blocks (it's a wire-shape battery); video throws E_UNSUPPORTED_MEDIA_MODALITY unless unsupportedMediaPolicy degrades it. Errors are the typed E_WEBLLM_CHAT_COMPLETIONS_* family (context overflow, stream error, invalid tool-call args) plus E_INVALID_WEBLLM_CHAT_COMPLETIONS_OPTIONS. Full surface in Assembly → LLM batteries.