---
url: 'https://adk.nht.io/batteries/llm/transformers-js.md'
description: >-
  On-device ONNX inference that runs the SAME battery in Node and the browser —
  onnxruntime-node natively, onnxruntime-web on WebGPU, no navigator.gpu gate.
  Text-out with a real parser layer, genuine multimodal input, a wired
  media-output seam, and the broadest tested-model table in this section.
---

# Transformers.js

## LLM summary — Transformers.js battery

* [`TransformersJsAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/transformers_js/adapter/classes/TransformersJsAdapter) from `@nhtio/adk/batteries/llm/transformers_js`. On-device ONNX text generation via `@huggingface/transformers` (OPTIONAL peer). ENVIRONMENT-NEUTRAL: runs in Node (`onnxruntime-node`, native, no GPU) AND the browser (`onnxruntime-web`, WASM/WebGPU); NOT gated on `navigator.gpu`. `isAvailable()` is `true` whenever the peer imports. `STASH_KEY` `transformersJs`.
* TEXT-out: tool calls + reasoning parsed from the model's TEXT via `toolCallParser` / `reasoningParser` (both default `'auto'`). Tool-call `arguments` are a plain object (no JSON.parse). Streaming v1: reports prose deltas live but stops prose once a tool-call/reasoning start-marker appears; the authoritative clean message is persisted after generation.
* Multimodal INPUT (image/audio) via the model's processor: positional `_call(text, images, audio)`. PCM-WAV audio is decoded env-neutrally (no `AudioContext` / no peer for the WAV fast path). Media OUTPUT via opt-in `extractMediaOutputs` hook (default off → text-out unchanged) → assistant `Message.attachments`.
* [`TransformersJsAdapter.dispose`](https://adk.nht.io/api/@nhtio/adk/batteries/llm/transformers_js/adapter/classes/TransformersJsAdapter#dispose) frees the ONNX sessions (`reset()` only nulls the JS refs). Call it when loading many models in one long-lived browser session, or stale sessions accumulate until WebGPU/wasm memory fails.
* Exceptions: `E_INVALID_TRANSFORMERS_JS_OPTIONS`, `E_TRANSFORMERS_JS_{CONTEXT_OVERFLOW,STREAM_ERROR,INVALID_TOOL_CALL_ARGS,TOOL_PARSE_FAILED}`, `E_UNSUPPORTED_MEDIA_MODALITY`.

This is the only battery in the section that runs the **same code in Node and the browser**. ONNX Runtime
auto-selects its backend — `onnxruntime-node` (native, CPU, no GPU required) in Node, `onnxruntime-web`
(WASM + WebGPU) in the browser — and the battery does not gate on `navigator.gpu`. `isAvailable()` returns
`true` whenever the `@huggingface/transformers` peer imports, full stop. That env-neutrality is the headline:
the agent you test in Node CI is the agent that runs in the user's tab.

```ts
import { TransformersJsAdapter } from '@nhtio/adk/batteries/llm/transformers_js'

const executor = new TransformersJsAdapter({
  model: 'onnx-community/gemma-4-E2B-it-ONNX',
  dtype: 'q4',
})
```

It is text-out, like [LiteRT-LM](./litert): the model emits a string, and tool calls + reasoning are parsed
back out by the shared `chat_common` layer (`toolCallParser` / `reasoningParser`, both default `'auto'` —
try-all, first confident match wins). Streaming v1 reports prose deltas live, then stops streaming prose the
moment a tool-call or reasoning start-marker appears; the authoritative clean message is persisted after
generation completes (so the persisted message is always correct — only live prose preceding a tool call is
truncated).

## Tested models

The broadest table in this section, because env-neutral Node execution makes it cheap to run real weights in
CI. Every row is a real matrix entry; every quirk traces to a real generation, not a guess.

| Model | Parser | Notes (the real quirk) |
| --- | --- | --- |
| Gemma 4 E2B (`onnx-community/gemma-4-E2B-it-ONNX`, q4) | `gemma` | Text + image + audio + the combined image-and-audio turn. Emits decoder-stripped `call:NAME{k:v}` at runtime, not the template's `<\|tool_call>`. |
| Qwen3 0.6B (q4f16) | `hermes` | Reasons *before* the call; at a tight budget the `<think>` eats the tokens and the call truncates mid-JSON — needs ~320-token budget so the hermes JSON completes. |
| DeepSeek-R1-Distill-Qwen 1.5B (q4) | think-tag | Reasoning-only; **q4f16 crashes** this graph (ONNX Cast-node error) → use q4. May spend its whole budget reasoning without concluding. |
| Llama 3.2 1B (q4f16) | `auto`→`llama3_json` | Emits bare `{"name","parameters"}` JSON (not pythonic) — the `auto` driver claims it. |
| SmolLM2 360M (q4) | — | Text/streaming baseline; no tool/reasoning assumptions. |
| Phi-4-mini (q4) | `phi` | The q4 quant **declines tool calls** — text baseline only here; the `phi` parser (`functools[...]`) is proven via the capture tier. |
| Granite 4.0 350M | `hermes` | Emits Hermes-style `<tool_call>{json}</tool_call>` — **no Granite-specific parser needed**. |
| Qwen2.5-Coder 0.5B (q4) | `auto` → `llama3_json` | Emits fenced ` ```json {name,arguments} ``` ` (not `<tool_call>`); the `llama3_json` parser un-fences and claims it. q4f16 crashes → q4. |
| Nemotron 3 Nano 4B | `qwen3_coder` | Mamba-hybrid LOAD-PROBE; emits qwen3\_coder XML + a stray `</think>` (orphan-recovered). |
| Qwen2.5-VL 3B (image, q4f16) | — | PROBE: q4f16 VL preprocessing throws a `Tensor size != data-length` error **upstream in transformers.js** — a run failure here is DATA, not our bug. |

Embeddings (separate `transformers_js_embed` battery, same runtime): `all-MiniLM-L6-v2`, `BGE small en v1.5`,
`Snowflake Arctic embed S` (asymmetric query/document prefixes), and `EmbeddingGemma 300M` — all verified
unit-norm + deterministic.

## Multimodal input

"Text-out" is about output structure, not input. transformers.js genuinely **perceives** image and audio: a
multimodal model's processor is called positionally as `_call(text, images, audio)`, and the model consumes
them. Gemma-4-E2B transcribes speech, names an image's dominant colour, and does both in a single turn.

One env-neutrality detail worth knowing: transformers.js's own `read_audio` decodes via the Web Audio API
(`AudioContext`), which **does not exist in Node**. So for the common case — uncompressed PCM WAV at 16 kHz,
exactly what speech models want — the battery decodes the RIFF itself with a `DataView`, dependency-free, in
Node and the browser alike, *before* it ever imports the heavy peer. Compressed containers (mp3/flac/ogg) fall
back to `read_audio` (browser-only, by that path's nature).

## Media output {#media-output}

Media output is wired but **opt-in**. By default the battery is text-out (the tested chat checkpoints emit
text), so it attaches nothing — byte-for-byte unchanged. Supply an `extractMediaOutputs` hook and a wrapped
media-emitting model's generated audio/image is surfaced as an assistant `Message.attachments` entry:

```ts
const executor = new TransformersJsAdapter({
  model: 'onnx-community/Llama-3.2-1B-Instruct-q4f16',
  // Turn the model's text answer into spoken audio and attach it to the turn.
  extractMediaOutputs: async (rawResult) => {
    const wavBytes = await synthesizeSpeechFromResult(rawResult) // your TTS
    return [{ kind: 'audio', mimeType: 'audio/wav', bytes: wavBytes }]
  },
})
```

The adapter persists each output via `ctx.storeMediaBytes`, builds a first-party `Media.toolGenerated(...)`,
and attaches it to the assistant message (a media-only turn — empty text + attachment — is legitimate). The
hook receives the raw generation result; what it does with it is yours. This is the same seam LiteRT-LM
exposes. (A dedicated text-to-audio/image *generation battery* — emitting media as the model's own task — is a
separate, future thing; this hook is for an LLM turn that produces media alongside or instead of text.)

::: warning Call `dispose()` in long-lived browser sessions
`reset()` only nulls the cached pipeline reference — it does NOT free the underlying ONNX Runtime sessions or
the WebGPU/wasm device memory they hold. Loading many models back-to-back in one browser session (a model
picker, a test matrix) without disposing accumulates those sessions until the heap is exhausted, surfacing as
`Can't create a session … memory copy`. `await adapter.dispose()` releases the model's ONNX sessions; the next
dispatch re-resolves a fresh pipeline. This is exactly the bug the [build showcase](/showcase/building-on-device-batteries)
recounts hitting in the WebGPU matrix.
:::

## Errors

Typed `E_TRANSFORMERS_JS_*` family — context overflow, stream error, invalid tool-call args, tool-parse failed
— plus `E_INVALID_TRANSFORMERS_JS_OPTIONS` and `E_UNSUPPORTED_MEDIA_MODALITY`. Full option + exception detail
in [Assembly → LLM batteries](/assembly/batteries-llm).