---
url: 'https://adk.nht.io/assembly/batteries-specialists.md'
description: >-
  On-device STT, OCR, and image-captioning adapters — the full option/exception
  surface for wiring one in yourself.
---

# Specialist batteries

## LLM summary — Specialist batteries

* `@nhtio/adk/batteries/specialists/stt/transformers_js` ships `TransformersJsSttAdapter` — Whisper-family ASR via `@huggingface/transformers` (OPTIONAL peer). `async transcribe(input: SpecialistAudioInput, opts?: TranscribeOptions): Promise<TranscribeResult>`. `TranscribeResult = { text: string, segments?: SttSegment[] }`, `SttSegment = { start: number, end: number | null, text: string }`. `TranscribeOptions = { language?: string, translate?: boolean, timestamps?: boolean }`.
* `@nhtio/adk/batteries/specialists/ocr/tesseract_js` ships `TesseractJsOcrAdapter` — OCR via `tesseract.js` (OPTIONAL peer, pure WASM, no native binary). `async recognize(input: SpecialistImageInput, opts?: RecognizeOptions): Promise<RecognizeResult>`. `RecognizeResult = { text: string, confidence?: number }`. `RecognizeOptions = { languages?: readonly string[] }` — a per-call value must equal the constructor's `languages` set (order-insensitive) or the call throws.
* `@nhtio/adk/batteries/specialists/caption/transformers_js` ships `TransformersJsCaptionAdapter` — image-to-text captioning via `@huggingface/transformers`'s `image-to-text` pipeline (OPTIONAL peer). `async describe(input: SpecialistImageInput, opts?: DescribeOptions): Promise<DescribeResult>`. `DescribeResult = { text: string }`. `DescribeOptions = { maxNewTokens?: number }`.
* All three are ENVIRONMENT-NEUTRAL (Node + browser; `isAvailable()` always `true`, no WebGPU/platform gate) and surfaced from the top-level `@nhtio/adk/batteries/specialists` aggregate barrel — none is excluded the way `WebLLMEmbeddingsAdapter` is excluded from the embeddings barrel.
* Input shapes are duck-typed, zero-core-import: `SpecialistAudioInput` = `{ pcm: Float32Array, sampleRate: number }` (pre-decoded) or an encoded-container form (`Uint8Array` / `{ bytes, mimeType? }` / `{ mimeType, asBytes() }`); `SpecialistImageInput` = `Uint8Array` / `{ bytes, mimeType? }` / `{ mimeType, asBytes() }`. A real `@nhtio/adk` `Media` satisfies both structurally — no `Media` import in any specialist adapter.
* Shared construction posture: eager `.unknown(false)` options validation, `preload()` / `reset()` / `dispose()` lifecycle, static + instance `isAvailable()`, the shared `BatteryLifecycleHooks` (`onLifecycle?`/`onLoading?`/`onCompiling?`/`onReady?`/`onGenerating?`/`onComplete?`/`onError?`).
* STT-only fields: `model: string` (required, no default), `pipeline?`, `createPipeline?`, `device?`, `dtype?`, `chunkLengthS?` (default `30`), `decodeAudio?: DecodeAudioFn`, `modelSource?`, `onInitProgress?`.
* Caption-only fields: `model: string` (required, no default), `pipeline?`, `createPipeline?`, `device?`, `dtype?`, `modelSource?`, `onInitProgress?`.
* OCR-only fields: `languages: readonly string[]` (required, non-empty, no default), `langPath?`, `cachePath?`, `workerOptions?` (spread after `langPath`/`cachePath`), `tesseract?`, `createWorker?`. OCR holds ONE cached worker across `recognize()` calls (construct-once) rather than one worker per call (the posture of the media battery's own `tesseract_js` engine); `reset()` and `dispose()` are ALIASES for it (both terminate the worker).
* Two-category exceptions per adapter: `E_INVALID_TRANSFORMERS_JS_STT_OPTIONS` / `E_TRANSFORMERS_JS_STT_ENGINE_ERROR`; `E_INVALID_TESSERACT_JS_OCR_OPTIONS` / `E_TESSERACT_JS_OCR_ENGINE_ERROR`; `E_INVALID_TRANSFORMERS_JS_CAPTION_OPTIONS` / `E_TRANSFORMERS_JS_CAPTION_ENGINE_ERROR`. The `_INVALID_..._OPTIONS` member is fatal (bad constructor input); the `_ENGINE_ERROR` member is non-fatal (pipeline/worker load or runtime failure).
* No agent integration ships: no `Tool` class, no forged tool, no `TurnRunnerConfig` wiring. See [Specialist batteries](/batteries/specialists/) for the recipes (BYO tool, direct call, `Media.stash` courtesy write) that wire one in.
* `@huggingface/transformers` and `tesseract.js` are optional peer dependencies, imported lazily — a consumer using only one specialist (or none) pays nothing for the others.

ADK has no perception primitive, the same way it has no embeddings primitive: the harness only ever sees a
{@link @nhtio/adk!Message} and its text content, never a waveform or a pixel buffer. So these three adapters are
not executors and do not plug into a callback slot. They are specialists: narrow, single-purpose models that
turn one modality into text you hand to a text-only LLM yourself. See
[Specialist batteries](/batteries/specialists/) for the thesis and the wiring recipes; this page is the
API-surface reference.

Unlike the embeddings batteries — which are the *same* battery three times over, differing only in engine — the
three specialists are three genuinely different jobs with three different method names and option surfaces.
What they share is a construction posture, not a call shape:

```typescript
import { TransformersJsSttAdapter } from '@nhtio/adk/batteries/specialists/stt/transformers_js'
import { TesseractJsOcrAdapter } from '@nhtio/adk/batteries/specialists/ocr/tesseract_js'
import { TransformersJsCaptionAdapter } from '@nhtio/adk/batteries/specialists/caption/transformers_js'

const stt = new TransformersJsSttAdapter({ model: 'onnx-community/whisper-base' })
const ocr = new TesseractJsOcrAdapter({ languages: ['eng'] })
const caption = new TransformersJsCaptionAdapter({ model: 'Xenova/vit-gpt2-image-captioning' })

const { text: transcript } = await stt.transcribe({ bytes: wavBytes, mimeType: 'audio/wav' })
const { text: recognized } = await ocr.recognize({ bytes: pngBytes, mimeType: 'image/png' })
const { text: described } = await caption.describe({ bytes: pngBytes, mimeType: 'image/png' })

await Promise.all([stt.dispose(), ocr.dispose(), caption.dispose()])
```

## Method surfaces, side by side

| | STT | OCR | Caption |
| --- | --- | --- | --- |
| Class | [`TransformersJsSttAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/classes/TransformersJsSttAdapter) | [`TesseractJsOcrAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/specialists/ocr/tesseract_js/adapter/classes/TesseractJsOcrAdapter) | [`TransformersJsCaptionAdapter`](https://adk.nht.io/api/@nhtio/adk/batteries/specialists/caption/transformers_js/adapter/classes/TransformersJsCaptionAdapter) |
| Subpath | `@nhtio/adk/batteries/specialists/stt/transformers_js` | `@nhtio/adk/batteries/specialists/ocr/tesseract_js` | `@nhtio/adk/batteries/specialists/caption/transformers_js` |
| Method | `transcribe(input, opts?)` | `recognize(input, opts?)` | `describe(input, opts?)` |
| Return | `{ text, segments? }` | `{ text, confidence? }` | `{ text }` |
| Required constructor field | `model: string` | `languages: readonly string[]` | `model: string` |
| Engine | `@huggingface/transformers` `automatic-speech-recognition` | `tesseract.js` (WASM) | `@huggingface/transformers` `image-to-text` |

All three: eager `.unknown(false)` option validation at construction, `preload()` / `reset()` / `dispose()`
lifecycle, static + instance `isAvailable()` (always `true` — none of these three has a WebGPU or platform
requirement), and the shared `BatteryLifecycleHooks` boot-progress contract documented on the
[shared LLM contract](/batteries/llm/shared-contract) page.

## Input shapes: zero core imports

None of the three adapters imports `Media` from `@nhtio/adk`. Instead each accepts a locally-declared duck
type:

* `SpecialistAudioInput` (STT only) — pre-decoded `{ pcm: Float32Array, sampleRate: number }`, or any of the
  encoded-container forms below.
* `SpecialistImageInput` (OCR, Caption) — a bare `Uint8Array`, `{ bytes: Uint8Array, mimeType?: string }`, or
  `{ mimeType: string, asBytes(): Promise<Uint8Array> }`.

A real `@nhtio/adk` `Media` instance satisfies both structurally — you can pass one straight through without
either side importing the other. This is the same posture the [media battery](/batteries/media/)'s own engines
follow: duck-typed structural contracts instead of a concrete core-class coupling between a battery and ADK's
core classes.

## Constructor options, per adapter

### STT — `TransformersJsSttAdapterOptions`

| Option | Type | Default |
| --- | --- | --- |
| `model` | `string` | — (required) |
| `pipeline` | `TransformersJsSttPipeline` | — |
| `createPipeline` | `CreateTransformersJsSttPipeline` | dynamic import + `pipeline('automatic-speech-recognition', …)` |
| `device` | `'auto' \| 'webgpu' \| 'wasm' \| 'cpu' \| 'gpu' \| …` | environment default |
| `dtype` | `'auto' \| 'fp32' \| 'fp16' \| 'q8' \| 'q4' \| …` | environment default |
| `chunkLengthS` | `number` | `30` |
| `decodeAudio` | `DecodeAudioFn` | [`defaultDecodeAudio`](https://adk.nht.io/api/@nhtio/adk/batteries/specialists/_shared/variables/defaultDecodeAudio) |
| `modelSource` | `TransformersJsSttModelSource` | HuggingFace Hub |
| `onInitProgress` | `ProgressCallback` | — |
| `isAvailable` | `() => boolean` | `true` |
| lifecycle hooks | `BatteryLifecycleHooks` | — |

### OCR — `TesseractJsOcrAdapterOptions`

| Option | Type | Default |
| --- | --- | --- |
| `languages` | `readonly string[]` | — (required, non-empty) |
| `langPath` | `string` | tesseract.js default |
| `cachePath` | `string` | tesseract.js default |
| `workerOptions` | `Record<string, unknown>` | — (spread after `langPath`/`cachePath`) |
| `tesseract` | `CreateTesseractJsModule` | `import('tesseract.js')` |
| `createWorker` | `CreateTesseractJsWorker` | `mod.createWorker(languages, undefined, { langPath, cachePath, ...workerOptions })` |
| `isAvailable` | `() => boolean` | `true` |
| lifecycle hooks | `BatteryLifecycleHooks` | — |

### Caption — `TransformersJsCaptionAdapterOptions`

| Option | Type | Default |
| --- | --- | --- |
| `model` | `string` | — (required) |
| `pipeline` | `TransformersJsCaptionPipeline` | — |
| `createPipeline` | `CreateTransformersJsCaptionPipeline` | dynamic import + `pipeline('image-to-text', …)` |
| `device` | `'auto' \| 'webgpu' \| 'wasm' \| 'cpu' \| 'gpu' \| …` | environment default |
| `dtype` | `'auto' \| 'fp32' \| 'fp16' \| 'q8' \| 'q4' \| …` | environment default |
| `modelSource` | `TransformersJsCaptionModelSource` | HuggingFace Hub |
| `onInitProgress` | `ProgressCallback` | — |
| `isAvailable` | `() => boolean` | `true` |
| lifecycle hooks | `BatteryLifecycleHooks` | — |

None of the three models defaults — a Whisper checkpoint, a captioning checkpoint, and a language-pack list are
all deployment decisions with real download-size and accuracy trade-offs, so naming one is the caller's
responsibility, the same posture the embeddings batteries take on `model`.

## Lifecycle: one divergence worth knowing

STT and Caption follow the embeddings adapters' split: `reset()` drops the cached JS pipeline reference,
`dispose()` additionally awaits the pipeline's own `dispose()` to free ONNX sessions and GPU/wasm buffers.

OCR does not have that split. A live `tesseract.js` worker is one heavyweight WASM resource with no lighter
"drop the reference, keep something warm" tier, so `reset()` and `dispose()` are **the same call** — both
terminate the cached worker. This is also why OCR's own posture (construct-once, one worker across every
`recognize()` call) diverges from the [media battery's `tesseract_js` engine](/batteries/media/fleet), which
boots and tears down a fresh worker per conversion call instead — the right call for a stateless, arbitrarily
concurrent pipeline, but a needless ~1-2s WASM boot if you already know you're calling `recognize()` repeatedly
against one adapter instance.

## Exceptions

Each adapter follows the same two-category scheme as every other bundled battery: a fatal, options-validation
exception (analogous to an HTTP 529) raised synchronously at construction, and a non-fatal, engine-level
exception (analogous to an HTTP 502) raised from the async method call.

| Adapter | Fatal (options) | Non-fatal (engine) |
| --- | --- | --- |
| STT | `E_INVALID_TRANSFORMERS_JS_STT_OPTIONS` | `E_TRANSFORMERS_JS_STT_ENGINE_ERROR` |
| OCR | `E_INVALID_TESSERACT_JS_OCR_OPTIONS` | `E_TESSERACT_JS_OCR_ENGINE_ERROR` |
| Caption | `E_INVALID_TRANSFORMERS_JS_CAPTION_OPTIONS` | `E_TRANSFORMERS_JS_CAPTION_ENGINE_ERROR` |

OCR's engine exception also covers a per-call `languages` mismatch against the cached worker (see the
[OCR page](/batteries/specialists/ocr#method-and-options)) — tesseract.js v7 has no safe way to re-language an
already-booted worker, so a genuinely different set throws rather than silently switching languages mid-stream.

## Wiring a specialist into an agent

These adapters have no agent-facing wiring of their own — no forged tool, no `TurnRunnerConfig` option. You
decide how a specialist's output reaches your text-only LLM: as a BYO [`Tool`](https://adk.nht.io/api/@nhtio/adk/forge/classes/Tool) the model calls itself, a
direct call in your own pipeline before the turn starts, or a courtesy `Media.stash` entry an LLM battery's
`fallback-stash` `UnsupportedMediaPolicy` reads automatically. All three recipes, with full code, are on the
[Specialist batteries](/batteries/specialists/#agent-integration-recipes) hub page.

## Optional peer dependencies

`@huggingface/transformers` (STT, Caption) and `tesseract.js` (OCR) are both **optional** peer dependencies,
imported lazily. A consumer using only one specialist — or none — installs nothing extra for the others and
pays nothing in their bundle:

```bash
pnpm add @huggingface/transformers   # for TransformersJsSttAdapter / TransformersJsCaptionAdapter
pnpm add tesseract.js                # for TesseractJsOcrAdapter
```

Note the transformers.js Node backend (`onnxruntime-node`) is a compiled native addon — the same platform
consideration as the [embeddings batteries](/assembly/batteries-embeddings#optional-peer-dependencies) for
Lambda / Alpine / ARM deployments.