Generation — Transformers.js

EXPERIMENTAL

This engine is newer and less battle-tested than the OpenAI/Gemini engines, downloads a ~2GB model on first use, and takes on the order of minutes per image on CPU/WASM. Treat it as a viable path for sovereignty-sensitive or offline deployments, not yet as a drop-in production default.

The transformers.js engine is the only one in this domain that runs entirely on-device — no API key, no network call per image, at the cost of a large model download and materially slower per-image latency. It drives DeepSeek's Janus model family through @huggingface/transformers' MultiModalityCausalLM.

import { TransformersJsGenerationAdapter } from '@nhtio/adk/batteries/generation/transformers_js'

const generator = new TransformersJsGenerationAdapter({
  model: 'onnx-community/Janus-Pro-1B-ONNX',
})

const [image] = await generator.generate('A watercolor illustration of a lighthouse at dawn', {
  doSample: true,
})

Janus-only, no pipeline task

@huggingface/transformers does not expose a pipeline('text-to-image') task — text-to-image generation only exists through Janus's own MultiModalityCausalLM.generate_images() method, confirmed against the installed package's own source. That means this adapter is necessarily model-family-specific: it targets Janus checkpoints (e.g. onnx-community/Janus-Pro-1B-ONNX), not an arbitrary diffusion model, and does not generalize to other transformers.js model families the way the embeddings/caption batteries' pipeline() calls do.

Options

Option	Type	Default	Notes
model	string	(required)	e.g. 'onnx-community/Janus-Pro-1B-ONNX'. No default anywhere in this domain.
janusModel	TransformersJsGenerationModel	—	Pre-built model instance; skips lazy creation entirely.
processor	TransformersJsGenerationProcessor	—	Pre-built processor instance; skips lazy creation entirely.
createModel	CreateTransformersJsGenerationModel	dynamic-import MultiModalityCausalLM.from_pretrained	Override the model factory.
createProcessor	CreateTransformersJsGenerationProcessor	dynamic-import AutoProcessor.from_pretrained	Override the processor factory.
device	string	environment default	Forwarded to from_pretrained().
dtype	string	environment default	Quantization/precision, forwarded to from_pretrained().
modelSource	(req: {repo, filename}) => bytes|string|Response|undefined	—	Custom OPFS/other byte resolver; falls through to HF when it returns undefined.
onInitProgress	(info: unknown) => void	—	Model-load progress callback.
isAvailable	() => boolean	always returns true (see below)	Override the availability probe.
encodeImage	(image) => Promise<Uint8Array>	env-branched toBlob/toSharp	Bypasses the PNG-encoding env-branch entirely — the primary hermetic-test seam.
doSample	boolean	true	do_sample. Matches the model card's own image-generation example.
temperature	number	—	temperature, read only when doSample is truthy.
topK	number	—	top_k, read directly in MultinomialSampler.sample().
guidanceScale	number	—	guidance_scale; classifier-free guidance activates when > 1.
repetitionPenalty	number	—	repetition_penalty.
minNewTokens	number	processor.num_image_tokens	min_new_tokens — the fixed per-image token budget when omitted.
maxNewTokens	number	processor.num_image_tokens	max_new_tokens — the fixed per-image token budget when omitted.
chatTemplate	string	'text_to_image'	Forwarded to the processor call.
role	string	'<|User|>'	Conversation role for the single-turn prompt (Janus convention).

The nine sampling/template knobs — doSample, temperature, topK, guidanceScale, repetitionPenalty, minNewTokens, maxNewTokens, chatTemplate, role — can also be passed per-call to generate(), overriding the adapter-level default of the same name. model and every construction seam (janusModel/processor/createModel/createProcessor/device/dtype/ modelSource/onInitProgress/isAvailable/encodeImage) are constructor-only.

isAvailable() always returns true

Unlike a WebGPU-gated battery, transformers.js is environment-neutral, so the default availability probe unconditionally returns true — it does not test whether @huggingface/transformers is actually installed. A missing optional peer surfaces later, as an E_TRANSFORMERS_JS_GENERATION_ENGINE_ERROR thrown from preload()/generate() when the lazy import fails. If you need a real up-front capability check, supply your own isAvailable override, or handle the lazy-load failure at first use.

top_p is intentionally absent

Unlike topK, there is no topP knob. The installed @huggingface/transformers build's TopPLogitsWarper branch is dead/commented-out code — MultinomialSampler.sample() (the sampler generate_images actually drives) never reads top_p. Exposing it would be a documented option that silently does nothing, so it is omitted rather than shipped as a no-op.

generate()

const outputs = await generator.generate('A neon-lit alley in the rain', {
  doSample: true,
  temperature: 1.0,
  topK: 100,
  guidanceScale: 5,
})

Internally: the processor builds the conversation tensors (processor([{ role, content: prompt }], { chat_template: chatTemplate })), the per-call and adapter-level knobs are merged into a single snake_case options bag (do_sample, temperature, top_k, guidance_scale, repetition_penalty, min_new_tokens, max_new_tokens — the last two default to the processor's own num_image_tokens, mirroring the official model-card example verbatim), and model.generate_images(options) runs. An empty result array throws E_TRANSFORMERS_JS_GENERATION_ENGINE_ERROR — this engine treats "no images" as a failure, never a valid empty success. Each RawImage-like result is encoded to PNG bytes via rawImageToEncodedBytes (see below); every output's mimeType is 'image/png'.

edit() is unsupported

await generator.edit([{ bytes, mimeType: 'image/png' }], 'Add a red bicycle')
// throws E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION(['edit', '...'])

Janus's MultiModalityCausalLM.generate_images() is text-conditioned image generation only — there is no image-conditioned edit/inpaint entry point anywhere in the installed @huggingface/transformers API. This call always throws; it is not conditionally supported based on model or options. If you need edits, use the OpenAI or Gemini engine.

Perf reality

• Download: ~2GB of model weights on first use (cached thereafter via the standard transformers.js cache, or your own modelSource resolver).
• Latency: minutes per image on CPU/WASM. There is no WebGPU requirement — ONNX Runtime auto-selects an execution provider — but this engine is materially slower than the OpenAI/Gemini engines regardless of accelerator.
• Isolation: because of the download size and per-call latency, running this engine behind forkIsolated (construct the real adapter inside the guest process, expose a thin generate/edit facade host-side) keeps that weight off your main process — see Recipes for the exact wiring.

Exceptions

Exception	Status	Fatal?	Thrown when
E_INVALID_TRANSFORMERS_JS_GENERATION_OPTIONS	529	Yes	Adapter options fail schema validation (missing/empty model, unknown key).
E_TRANSFORMERS_JS_GENERATION_ENGINE_ERROR	502	No	Model/processor fail to load, generate_images throws, or the result is empty.
E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION	501	Yes	edit() is called — always, unconditionally.

Where to go next

• Recipes — running this engine behind forkIsolated, unchanged.
• Isolation battery — the wrapper this engine is designed to run behind.
• OpenAI / Gemini — the engines to reach for when you need edit().
• Assembly → Generation batteries — full option/exception tables, all three engines.