Specialist batteries

ADK has no perception primitive, the same way it has no embeddings primitive: the harness only ever sees a {@link @nhtio/adk!Message} and its text content, never a waveform or a pixel buffer. So these three adapters are not executors and do not plug into a callback slot. They are specialists: narrow, single-purpose models that turn one modality into text you hand to a text-only LLM yourself. See Specialist batteries for the thesis and the wiring recipes; this page is the API-surface reference.

Unlike the embeddings batteries — which are the same battery three times over, differing only in engine — the three specialists are three genuinely different jobs with three different method names and option surfaces. What they share is a construction posture, not a call shape:

import { TransformersJsSttAdapter } from '@nhtio/adk/batteries/specialists/stt/transformers_js'
import { TesseractJsOcrAdapter } from '@nhtio/adk/batteries/specialists/ocr/tesseract_js'
import { TransformersJsCaptionAdapter } from '@nhtio/adk/batteries/specialists/caption/transformers_js'

const stt = new TransformersJsSttAdapter({ model: 'onnx-community/whisper-base' })
const ocr = new TesseractJsOcrAdapter({ languages: ['eng'] })
const caption = new TransformersJsCaptionAdapter({ model: 'Xenova/vit-gpt2-image-captioning' })

const { text: transcript } = await stt.transcribe({ bytes: wavBytes, mimeType: 'audio/wav' })
const { text: recognized } = await ocr.recognize({ bytes: pngBytes, mimeType: 'image/png' })
const { text: described } = await caption.describe({ bytes: pngBytes, mimeType: 'image/png' })

await Promise.all([stt.dispose(), ocr.dispose(), caption.dispose()])

Method surfaces, side by side

	STT	OCR	Caption
Class	TransformersJsSttAdapter	TesseractJsOcrAdapter	TransformersJsCaptionAdapter
Subpath	@nhtio/adk/batteries/specialists/stt/transformers_js	@nhtio/adk/batteries/specialists/ocr/tesseract_js	@nhtio/adk/batteries/specialists/caption/transformers_js
Method	transcribe(input, opts?)	recognize(input, opts?)	describe(input, opts?)
Return	{ text, segments? }	{ text, confidence? }	{ text }
Required constructor field	model: string	languages: readonly string[]	model: string
Engine	@huggingface/transformers automatic-speech-recognition	tesseract.js (WASM)	@huggingface/transformers image-to-text

All three: eager .unknown(false) option validation at construction, preload() / reset() / dispose() lifecycle, static + instance isAvailable() (always true — none of these three has a WebGPU or platform requirement), and the shared BatteryLifecycleHooks boot-progress contract documented on the shared LLM contract page.

Input shapes: zero core imports

None of the three adapters imports Media from @nhtio/adk. Instead each accepts a locally-declared duck type:

• SpecialistAudioInput (STT only) — pre-decoded { pcm: Float32Array, sampleRate: number }, or any of the encoded-container forms below.
• SpecialistImageInput (OCR, Caption) — a bare Uint8Array, { bytes: Uint8Array, mimeType?: string }, or { mimeType: string, asBytes(): Promise<Uint8Array> }.

A real @nhtio/adk Media instance satisfies both structurally — you can pass one straight through without either side importing the other. This is the same posture the media battery's own engines follow: duck-typed structural contracts instead of a concrete core-class coupling between a battery and ADK's core classes.

Constructor options, per adapter

STT — TransformersJsSttAdapterOptions

Option	Type	Default
model	string	— (required)
pipeline	TransformersJsSttPipeline	—
createPipeline	CreateTransformersJsSttPipeline	dynamic import + pipeline('automatic-speech-recognition', …)
device	'auto' | 'webgpu' | 'wasm' | 'cpu' | 'gpu' | …	environment default
dtype	'auto' | 'fp32' | 'fp16' | 'q8' | 'q4' | …	environment default
chunkLengthS	number	30
decodeAudio	DecodeAudioFn	defaultDecodeAudio
modelSource	TransformersJsSttModelSource	HuggingFace Hub
onInitProgress	ProgressCallback	—
isAvailable	() => boolean	true
lifecycle hooks	BatteryLifecycleHooks	—

OCR — TesseractJsOcrAdapterOptions

Option	Type	Default
languages	readonly string[]	— (required, non-empty)
langPath	string	tesseract.js default
cachePath	string	tesseract.js default
workerOptions	Record<string, unknown>	— (spread after langPath/cachePath)
tesseract	CreateTesseractJsModule	import('tesseract.js')
createWorker	CreateTesseractJsWorker	mod.createWorker(languages, undefined, { langPath, cachePath, ...workerOptions })
isAvailable	() => boolean	true
lifecycle hooks	BatteryLifecycleHooks	—

Caption — TransformersJsCaptionAdapterOptions

Option	Type	Default
model	string	— (required)
pipeline	TransformersJsCaptionPipeline	—
createPipeline	CreateTransformersJsCaptionPipeline	dynamic import + pipeline('image-to-text', …)
device	'auto' | 'webgpu' | 'wasm' | 'cpu' | 'gpu' | …	environment default
dtype	'auto' | 'fp32' | 'fp16' | 'q8' | 'q4' | …	environment default
modelSource	TransformersJsCaptionModelSource	HuggingFace Hub
onInitProgress	ProgressCallback	—
isAvailable	() => boolean	true
lifecycle hooks	BatteryLifecycleHooks	—

None of the three models defaults — a Whisper checkpoint, a captioning checkpoint, and a language-pack list are all deployment decisions with real download-size and accuracy trade-offs, so naming one is the caller's responsibility, the same posture the embeddings batteries take on model.

Lifecycle: one divergence worth knowing

STT and Caption follow the embeddings adapters' split: reset() drops the cached JS pipeline reference, dispose() additionally awaits the pipeline's own dispose() to free ONNX sessions and GPU/wasm buffers.

OCR does not have that split. A live tesseract.js worker is one heavyweight WASM resource with no lighter "drop the reference, keep something warm" tier, so reset() and dispose() are the same call — both terminate the cached worker. This is also why OCR's own posture (construct-once, one worker across every recognize() call) diverges from the media battery's tesseract_js engine, which boots and tears down a fresh worker per conversion call instead — the right call for a stateless, arbitrarily concurrent pipeline, but a needless ~1-2s WASM boot if you already know you're calling recognize() repeatedly against one adapter instance.

Exceptions

Each adapter follows the same two-category scheme as every other bundled battery: a fatal, options-validation exception (analogous to an HTTP 529) raised synchronously at construction, and a non-fatal, engine-level exception (analogous to an HTTP 502) raised from the async method call.

Adapter	Fatal (options)	Non-fatal (engine)
STT	E_INVALID_TRANSFORMERS_JS_STT_OPTIONS	E_TRANSFORMERS_JS_STT_ENGINE_ERROR
OCR	E_INVALID_TESSERACT_JS_OCR_OPTIONS	E_TESSERACT_JS_OCR_ENGINE_ERROR
Caption	E_INVALID_TRANSFORMERS_JS_CAPTION_OPTIONS	E_TRANSFORMERS_JS_CAPTION_ENGINE_ERROR

OCR's engine exception also covers a per-call languages mismatch against the cached worker (see the OCR page) — tesseract.js v7 has no safe way to re-language an already-booted worker, so a genuinely different set throws rather than silently switching languages mid-stream.

Wiring a specialist into an agent

These adapters have no agent-facing wiring of their own — no forged tool, no TurnRunnerConfig option. You decide how a specialist's output reaches your text-only LLM: as a BYO Tool the model calls itself, a direct call in your own pipeline before the turn starts, or a courtesy Media.stash entry an LLM battery's fallback-stash UnsupportedMediaPolicy reads automatically. All three recipes, with full code, are on the Specialist batteries hub page.

Optional peer dependencies

@huggingface/transformers (STT, Caption) and tesseract.js (OCR) are both optional peer dependencies, imported lazily. A consumer using only one specialist — or none — installs nothing extra for the others and pays nothing in their bundle:

pnpm add @huggingface/transformers   # for TransformersJsSttAdapter / TransformersJsCaptionAdapter
pnpm add tesseract.js                # for TesseractJsOcrAdapter

Note the transformers.js Node backend (onnxruntime-node) is a compiled native addon — the same platform consideration as the embeddings batteries for Lambda / Alpine / ARM deployments.