@nhtio/adk/batteries/generation/transformers_js/adapter

transformers.js (on-device, EXPERIMENTAL) Generation adapter battery — DeepSeek Janus text→image.

Remarks

Media-generation battery backed by transformers.js's Janus (MultiModalityCausalLM) model family (documented reference model: onnx-community/Janus-Pro-1B-ONNX). Environment-neutral — runs in Node (via onnxruntime-node) and the browser (via onnxruntime-web / WebGPU), auto-selected by the package; there is no WebGPU requirement.

EXPERIMENTAL. Janus is a ~2GB multimodal model; on WASM/CPU a single image can take minutes to generate. This battery exists for completeness of the on-device story, not as a fast path — prefer the OpenAI/Gemini engines unless running fully offline is the point.

There is no pipeline('text-to-image') task in transformers.js (verified against the installed @huggingface/transformers source — no such task is registered in src/pipelines.js). The only image-generation surface is MultiModalityCausalLM.generate_images(options) (src/models/multi_modality/modeling_multi_modality.js), fed by VLChatProcessor (src/models/janus/processing_janus.js) — both exported from the package root, per the official onnx-community/Janus-Pro-1B-ONNX model card:

import {
  AutoProcessor,
  MultiModalityCausalLM,
} from "@huggingface/transformers";
const processor = await AutoProcessor.from_pretrained(model_id);
const model = await MultiModalityCausalLM.from_pretrained(model_id);
const inputs = await processor([{ role: "<|User|>", content: prompt }], {
  chat_template: "text_to_image",
});
const outputs = await model.generate_images({
  ...inputs,
  min_new_tokens: processor.num_image_tokens,
  max_new_tokens: processor.num_image_tokens,
  do_sample: true,
});

This adapter mirrors that shape exactly, with every knob overridable per-call (see ./types for the full source-verified knob provenance) and both the model and processor typed structurally (duck-typed local interfaces, never a hard import of the peer's classes) so fake-model/fake-processor unit tests never load the real peer.

generate_images returns RawImage[] — raw decoded pixel data, not encoded bytes. Encoding to PNG is env-branched (browser toBlob('image/png') / Node toSharp().png().toBuffer()), handled by @nhtio/adk/batteries/generation/transformers_js/helpers!rawImageToEncodedBytes, which also accepts an injectable encodeImage override — the primary hermetic-test seam.

edit() is unsupported. Janus's only generation entry point (generate_images) is text-conditioned; there is no image-conditioned edit/inpaint surface in the installed @huggingface/transformers API. Calling edit() throws @nhtio/adk/batteries/generation/transformers_js/exceptions!E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION.

@huggingface/transformers is an optional peer dependency, imported lazily (only inside the default model/processor factories, i.e. only when no janusModel/processor/createModel/createProcessor override is supplied).

Classes

Class	Description
TransformersJsGenerationAdapter	Media generation adapter for transformers.js's Janus (MultiModalityCausalLM) text→image model family.