---
url: 'https://adk.nht.io/batteries/specialists/stt.md'
description: >-
  TransformersJsSttAdapter — on-device Whisper transcription, Node and browser
  alike. 16 kHz mono PCM in, transcript (and optional timestamped segments) out.
  No cloud endpoint, no API key.
---

# Speech-to-Text (STT)

## LLM summary — STT specialist battery

* [`TransformersJsSttAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/classes/TransformersJsSttAdapter) from `@nhtio/adk/batteries/specialists/stt/transformers_js`. Whisper-family automatic-speech-recognition via `@huggingface/transformers` (OPTIONAL peer). ENVIRONMENT-NEUTRAL — Node (`onnxruntime-node`) and browser (`onnxruntime-web`/WebGPU); `isAvailable()` is always `true`, no WebGPU requirement.
* Method: `async transcribe(input: SpecialistAudioInput, opts?: TranscribeOptions): Promise<TranscribeResult>`. `TranscribeResult = { text: string, segments?: SttSegment[] }`. `SttSegment = { start: number, end: number | null, text: string }`.
* `TranscribeOptions`: `language?: string` (e.g. `'english'`, improves accuracy, default auto-detect), `translate?: boolean` (translate to English rather than transcribe verbatim, default `false`), `timestamps?: boolean` (populate `segments`, default `false`).
* `SpecialistAudioInput` = pre-decoded `{ pcm: Float32Array, sampleRate: number }` OR any encoded-container form (`Uint8Array`, `{ bytes, mimeType? }`, or a duck-typed `{ mimeType, asBytes() }` — a real `Media` satisfies this with zero import). Encoded containers decode via an injectable `DecodeAudioFn` (default: lazy `audio-decode` peer + downmix-to-mono); either path resamples to 16 kHz mono PCM before the pipeline call.
* `TransformersJsSttAdapterOptions`: `model: string` (REQUIRED, no default — e.g. `onnx-community/whisper-base`), `pipeline?`, `createPipeline?`, `device?`, `dtype?`, `chunkLengthS?` (default `30`), `decodeAudio?: DecodeAudioFn`, `modelSource?`, `onInitProgress?`, `isAvailable?`, plus the shared `BatteryLifecycleHooks` (`onLifecycle`/`onLoading`/`onCompiling`/`onReady`/`onGenerating`/`onComplete`/`onError`).
* Lifecycle: `preload()`, `reset()` (drops the JS pipeline reference only), `dispose()` (awaits the pipeline's own `dispose()` to free ONNX sessions/GPU-wasm memory, then `reset()`s — call this in long-lived sessions loading many models).
* Browser note: the auto-selected quantization can fail ONNX session creation for whisper-tiny/base ("Missing required scale ... DequantizeLinear"); pin `dtype: 'fp32'` in the browser.
* Exceptions: `E_INVALID_TRANSFORMERS_JS_STT_OPTIONS` (529, fatal — bad options), `E_TRANSFORMERS_JS_STT_ENGINE_ERROR` (502, non-fatal — pipeline/runtime failure).

[`TransformersJsSttAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/classes/TransformersJsSttAdapter) turns spoken audio into text with a Whisper-family model, running on-device
through `@huggingface/transformers`. Construct it once, call [`TransformersJsSttAdapter.transcribe`](https://adk.nht.io/api/@nhtio/adk/batteries/classes/TransformersJsSttAdapter#transcribe) as
many times as you have audio clips. There is no network hop and no API key — the whole job runs against local
compute, in Node (`onnxruntime-node`) or the browser (`onnxruntime-web`, WebGPU when available), auto-selected
the same way the [Transformers.js LLM battery](/batteries/llm/transformers-js) selects its backend.

```ts
import { TransformersJsSttAdapter } from '@nhtio/adk/batteries/specialists/stt/transformers_js'

const stt = new TransformersJsSttAdapter({ model: 'onnx-community/whisper-base' })

const { text } = await stt.transcribe({ bytes: wavBytes, mimeType: 'audio/wav' })
console.log(text) // "The quick brown fox jumps over the lazy dog."

await stt.dispose() // release the ONNX session when you're done with this adapter
```

## Input forms

`transcribe` accepts audio in whichever of these shapes is convenient:

* **Already-decoded PCM** — `{ pcm: Float32Array, sampleRate: number }`. Skips container decoding entirely;
  useful if you're capturing raw samples from a microphone stream.
* **Raw bytes** — a bare `Uint8Array` (MIME type unknown).
* **Bytes + MIME** — `{ bytes: Uint8Array, mimeType?: string }`.
* **A `Media`-like value** — anything with `{ mimeType: string, asBytes(): Promise<Uint8Array> }`. A real
  `@nhtio/adk` `Media` instance satisfies this structurally, so you can hand one straight through without this
  battery ever importing the class.

Whichever form you pass, the adapter normalizes it to 16 kHz mono PCM — the shape Whisper-family models
expect — before the pipeline call.

::: tip Why 16 kHz mono, specifically
Whisper's own preprocessing was trained against 16 kHz mono. Passing pre-decoded PCM at a different sample rate
(say, 44.1 kHz from a microphone capture) is fine — [`resampleTo`](https://adk.nht.io/api/@nhtio/adk/lib/utils/audio/functions/resampleTo) (a plain linear-interpolation resample,
no peer dependency) brings it to 16 kHz, and is a no-op when the input is already at that rate. Encoded
containers (WAV/MP3/FLAC/…) go through an injectable `DecodeAudioFn` first — default: a lazy `audio-decode`
peer import plus a downmix to mono — then the same resample.
:::

## Method and options

```ts
async transcribe(input: SpecialistAudioInput, opts?: TranscribeOptions): Promise<TranscribeResult>
```

| Option | Type | Default | Effect |
| --- | --- | --- | --- |
| `language` | `string` | auto-detect | Named source language (e.g. `'english'`) improves accuracy when known. |
| `translate` | `boolean` | `false` | Translate the recognized speech to English instead of transcribing verbatim. |
| `timestamps` | `boolean` | `false` | Populate `TranscribeResult.segments` with per-chunk start/end times. |

`TranscribeResult` is `{ text: string, segments?: SttSegment[] }`. Each `SttSegment` is
`{ start: number, end: number | null, text: string }` — `end` is `null` when the engine can't bound the chunk.

```ts
const { text, segments } = await stt.transcribe(
  { bytes: wavBytes, mimeType: 'audio/wav' },
  { timestamps: true }
)
for (const seg of segments ?? []) {
  console.log(`[${seg.start.toFixed(1)}s] ${seg.text}`)
}
```

Constructor options (`TransformersJsSttAdapterOptions`):

| Option | Type | Default | Notes |
| --- | --- | --- | --- |
| `model` | `string` | — (required) | The Whisper (or compatible) ASR model id, e.g. `onnx-community/whisper-base`. No default — Whisper-family models are multi-hundred-megabyte downloads and this battery never triggers one silently. |
| `pipeline` | `TransformersJsSttPipeline` | — | A pre-built pipeline; when set, skips lazy creation entirely. |
| `createPipeline` | `CreateTransformersJsSttPipeline` | dynamic import + `pipeline('automatic-speech-recognition', …)` | Override the pipeline factory (test doubles, custom loading). |
| `device` | `'auto' \| 'webgpu' \| 'wasm' \| 'cpu' \| 'gpu' \| …` | environment default | Forwarded to `pipeline()`. |
| `dtype` | `'auto' \| 'fp32' \| 'fp16' \| 'q8' \| 'q4' \| …` | environment default | Quantization/precision. See the browser note below. |
| `chunkLengthS` | `number` | `30` | Length of audio chunks the pipeline processes at once, in seconds. |
| `decodeAudio` | `DecodeAudioFn` | [`defaultDecodeAudio`](https://adk.nht.io/api/@nhtio/adk/batteries/specialists/_shared/variables/defaultDecodeAudio) | Override the container-decode step. |
| `modelSource` | `TransformersJsSttModelSource` | HuggingFace Hub | Serve model files from OPFS, a bundled source, or elsewhere. |
| `onInitProgress` | `ProgressCallback` | — | Raw model-load progress reports. |
| `isAvailable` | `() => boolean` | `true` | Override the availability probe. |
| lifecycle hooks | `BatteryLifecycleHooks` | — | `onLifecycle`/`onLoading`/`onCompiling`/`onReady`/`onGenerating`/`onComplete`/`onError` — the same normalized phase machine (`loading → compiling → ready → generating → complete`/`error`) documented on the [shared contract](/batteries/llm/shared-contract) page. |

## Lifecycle

`preload()` eagerly loads and caches the pipeline (idempotent, single-flight — concurrent calls share one
load). `reset()` drops the cached JS reference only. `dispose()` awaits the pipeline's own `dispose()` — this
is what actually frees the ONNX Runtime sessions and any WebGPU/wasm device memory — then calls `reset()`.

::: warning Call `dispose()` in long-lived browser sessions
Same trap as the [Transformers.js LLM battery](/batteries/llm/transformers-js#media-output): `reset()` alone
leaves the underlying ONNX session (and any GPU/wasm buffers) alive. Loading several Whisper models
back-to-back in one browser tab without disposing accumulates sessions until memory runs out.
:::

::: tip Browser: pin `dtype: 'fp32'`
The browser's auto-selected quantization fails ONNX session creation for whisper-tiny/-base with an error like
`Missing required scale ... DequantizeLinear` (from the model's `MatMulNBits` weights). `fp32` is the known-good
cross-execution-provider combination for these small models — tiny/base sit around 150-500MB at fp32, a
reasonable one-time download for a browser tab:

```ts
const stt = new TransformersJsSttAdapter({
  model: 'onnx-community/whisper-tiny.en',
  dtype: 'fp32' as never, // browser: auto-quant fails session creation for whisper
})
```

:::

## Errors

`E_INVALID_TRANSFORMERS_JS_STT_OPTIONS` (fatal, bad constructor options) and
`E_TRANSFORMERS_JS_STT_ENGINE_ERROR` (non-fatal, pipeline load/runtime failure — including a malformed
pipeline result). Both follow the same two-category exception scheme documented across every bundled battery.

## A complete example

```ts
import { readFile } from 'node:fs/promises'
import { TransformersJsSttAdapter } from '@nhtio/adk/batteries/specialists/stt/transformers_js'

const stt = new TransformersJsSttAdapter({ model: 'onnx-community/whisper-base' })

try {
  const bytes = new Uint8Array(await readFile('meeting-clip.wav'))
  const { text } = await stt.transcribe({ bytes, mimeType: 'audio/wav' })
  console.log(text)
} finally {
  await stt.dispose()
}
```

## Where to go next

* [Specialist batteries — overview](/batteries/specialists/) — the thesis, the three adapters at a glance,
  and how to wire one into an agent as a tool.
* [OCR](/batteries/specialists/ocr) and [Caption](/batteries/specialists/caption) — the other two specialists.
* [Bring your own tools](/assembly/byo-tools) — the pattern for wrapping this adapter in a `transcribe_audio`
  Tool.
* [Assembly → Specialist batteries](/assembly/batteries-specialists) — the full option/exception surface.