Generation batteries

The Generation battery hub page covers the thesis and a quick start; each engine's own page (OpenAI, Gemini, Transformers.js, Local Diffusion) covers its wire behavior in narrative form. This page is the side-by-side option/exception reference.

Barrels, side by side

	OpenAI	Gemini	Transformers.js	Local Diffusion
Subpath	@nhtio/adk/batteries/generation/openai	@nhtio/adk/batteries/generation/gemini	@nhtio/adk/batteries/generation/transformers_js	@nhtio/adk/batteries/generation/local_diffusion
Adapter	OpenAIGenerationAdapter	GeminiGenerationAdapter	TransformersJsGenerationAdapter	LocalDiffusionGenerationAdapter
Options type	OpenAIGenerationAdapterOptions	GeminiGenerationAdapterOptions	TransformersJsGenerationAdapterOptions	LocalDiffusionGenerationAdapterOptions
Validator	validateOpenAIGenerationOptions	validateGeminiGenerationOptions	validateTransformersJsGenerationOptions	validateOptions (subpath-only)
Transport	raw fetch, OpenAI /v1/images/* shape	raw fetch, native generativelanguage REST	on-device, optional @huggingface/transformers peer	on-device, stdio subprocess
edit()	Yes (multipart)	Yes	Always throws	Yes (protocol im2im)

The aggregate barrel @nhtio/adk/batteries/generation re-exports everything from the three environment-neutral subpaths (OpenAI, Gemini, transformers.js) — adapters, option/schema types, exceptions, and toBytes (renamed toGenerationBytes to avoid colliding with the specialists battery's own toBytes export when both aggregates are imported side by side). The Node-only local-diffusion engine stays out of the aggregate and is imported directly from @nhtio/adk/batteries/generation/local_diffusion.

The shared contract

Every engine implements the same method surface:

Method	Signature	Notes
generate	(prompt: string, opts?: GenerateOptions) => Promise<GeneratedMediaOutput[]>
edit	(inputs: GenerationImageInput | GenerationImageInput[], prompt: string, opts?: EditOptions) => Promise<GeneratedMediaOutput[]>	transformers.js always throws E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION.
isAvailable	() => boolean	OpenAI/Gemini: true whenever fetch exists. transformers.js: always true (it does NOT probe the peer — a missing @huggingface/transformers surfaces as an engine error at first preload()/generate()), unless an isAvailable override is supplied.
preload	() => Promise<void>	Warms the model/processor (transformers.js) or is a no-op (OpenAI/Gemini — no local state to warm).
reset	() => void	Clears any lazily-created model/processor instance.

GeneratedMediaOutput:

interface GeneratedMediaOutput {
  kind: MediaKind    // 'image' for every engine today
  mimeType: string
  bytes: Uint8Array
  filename?: string
}

Re-exported type-only from the LLM Chat Completions battery's own shared types (chat_common/types.ts), so generated media and LLM-battery media-output describe the same shape.

GenerationImageInput — the edit() input duck

type GenerationImageInput = Uint8Array | GenerationBytesInput | GenerationMediaLike

interface GenerationBytesInput {
  bytes: Uint8Array
  mimeType?: string
}

interface GenerationMediaLike {
  mimeType: string
  asBytes(): Promise<Uint8Array>
}

A real @nhtio/adk Media instance satisfies GenerationMediaLike structurally — no import from this battery's _shared layer to Media's module, and no import the other way. This is CONTRIBUTING.md Design Decision #13's tier-2 rule in practice: battery _shared layers describe what they need by shape, never by importing a concrete core class. toGenerationBytes(input) (the aggregate's renamed export of _shared's toBytes) normalizes any of the three forms to { bytes, mimeType }.

BaseGenerationAdapterOptions — the one required field

interface BaseGenerationAdapterOptions {
  model: string // required, no default, on every engine
}

Owned by the OpenAI battery (openai/types.ts), extended by Gemini and transformers.js in turn. No engine in this domain defaults model — naming it is always the caller's job, exactly as with the embeddings batteries.

OpenAI — OpenAIGenerationAdapterOptions

Option	Type	Default
model	string	(required)
apiKey	string	—
baseURL	string	https://api.openai.com/v1
headers	Record<string, string>	—
fetch	typeof fetch	global fetch
responseFormatMode	'auto' | 'send' | 'omit'	'auto'
size	string	—
quality	'low' | 'medium' | 'high' | 'auto'	—
background	'transparent' | 'opaque' | 'auto'	—
outputFormat	'png' | 'jpeg' | 'webp'	'png' (adapter runtime default; not schema-filled)
requestTimeoutMs	number	0 (disabled)
retry.maxAttempts	number	1
retry.baseDelayMs	number	500
retry.maxDelayMs	number	30000
retry.retriableStatuses	number[]	[429, 500, 502, 503, 504]
retry.honorRetryAfter	boolean	true

OpenAIGenerateOptions (per-call, generate()): n?: number (sent when defined), size?, quality? (same enum as above), outputFormat?, background?. OpenAIEditOptions (per-call, edit() only): n?: number, mask?: GenerationImageInput, size?: string, quality? (same enum). Multipart field name for each edit image: EDIT_IMAGE_FIELD_NAME = 'image[]' (probe-confirmed, exported so a custom transport/mock can match it exactly).

Gemini — GeminiGenerationAdapterOptions

Extends the OpenAI shapes (GenerationRetryConfig, GenerateOptions, EditOptions, BaseGenerationAdapterOptions) via re-export-then-extend:

Option	Type	Default
model	string	(required)
apiKey	string	— (sent as x-goog-api-key)
baseURL	string	https://generativelanguage.googleapis.com/v1beta
headers	Record<string, string>	—
fetch	typeof fetch	global fetch
responseModalities	('TEXT' | 'IMAGE')[]	['TEXT', 'IMAGE']
aspectRatio	string	— (only sent when set)
requestTimeoutMs	number	0 (disabled)
retry.*	—	identical defaults to OpenAI's retry shape

GeminiRequestPart/GeminiContent/GeminiGenerationConfig mirror the native generateContent request body; GeminiResponsePart tolerates both camelCase inlineData and snake_case inline_data on responses (requests always send camelCase). candidateCount is only sent when the call's n > 1.

Transformers.js — TransformersJsGenerationAdapterOptions

Extends BaseGenerationAdapterOptions and BatteryLifecycleHooks:

Option	Type	Default
model	string	(required)
janusModel	TransformersJsGenerationModel	—
processor	TransformersJsGenerationProcessor	—
createModel	CreateTransformersJsGenerationModel	dynamic-import MultiModalityCausalLM.from_pretrained
createProcessor	CreateTransformersJsGenerationProcessor	dynamic-import AutoProcessor.from_pretrained
device	string	environment default
dtype	string	environment default
modelSource	TransformersJsGenerationModelSource	— (falls through to HF when undefined)
onInitProgress	TransformersJsGenerationProgressCallback	—
isAvailable	() => boolean	always true (does NOT probe the peer)
encodeImage	EncodeRawImageFn	env-branched toBlob/toSharp
doSample	boolean	true
temperature	number	—
topK	number	—
guidanceScale	number	—
repetitionPenalty	number	—
minNewTokens	number	processor.num_image_tokens
maxNewTokens	number	processor.num_image_tokens
chatTemplate	string	'text_to_image'
role	string	'<|User|>'

Every sampling knob (doSample…role) is also accepted per-call on generate(), overriding the adapter-level default. There is deliberately no topP option — the installed @huggingface/transformers build's TopPLogitsWarper branch is dead code; MultinomialSampler.sample() never reads top_p.

Local Diffusion — LocalDiffusionGenerationAdapterOptions

This engine runs generation in a BYO (Bring Your Own) local inference subprocess, coordinating over a standard stdin/stdout line protocol. The ADK ships the protocol and the process driver; the consumer supplies a protocol-speaking backend. The library does not bundle Python, torch, or a specific engine runtime.

This engine is Node-only and reachable only through its own subpath (@nhtio/adk/batteries/generation/local_diffusion) — it is deliberately excluded from the environment-neutral @nhtio/adk/batteries/generation aggregate barrel, since it spawns a subprocess.

Extends BaseGenerationAdapterOptions and BatteryLifecycleHooks:

Option	Type	Default	Meaning
model	string	(required)	Path/id of the checkpoint the backend loads. Passed to the spawner in its { command, args, model } context; it is not included in the t2im/im2im command JSON.
command	string	(required)	The backend executable the default spawner launches.
args	string[]	[]	Arguments passed to command by the default spawner.
spawn	DiffusionBackendSpawner	(default: lazy node:child_process.spawn)	Override the process factory. Receives { command, args, model }; returns a DiffusionBackendProcess.
fs	DiffusionFsLike	(default: lazy node:fs/promises)	{ readFile, unlink } used only for backend-written path results.
outputDir	string	—	The only directory under which the adapter will delete backend-written result files (containment-checked).
maxDecodedBytes	number	52428800 (50 MiB)	Max decoded size of an inline-base64 image.
maxLineBytes	number	1048576 (1 MiB)	Cap on a single protocol line (framer buffer).
commandPrefix	string	'b2py'	Override host→backend command line prefix.
eventPrefix	string	'sdbk'	Override backend→host event line prefix.
protocol	Partial<ProtocolConfig>	—	Bulk protocol-tag override (merged under the granular fields below).
ops	Partial<ProtocolConfig['ops']>	{ generate: 't2im', edit: 'im2im' }	Override operation sub-tags.
events	Partial<ProtocolConfig['events']>	DiffusionBee compatible	Override event sub-tags (mdld, rdy, dnpr, nwim, done, err).
control	Partial<ProtocolConfig['control']>	DiffusionBee compatible	Override control sub-tags (__stop__, __shutdown__).
startupTimeoutMs	number	30000	Deadline waiting for the rdy event.
requestTimeoutMs	number	0 (disabled)	Timeout for an individual generate or edit request.
abortGraceMs	number	5000	Time to hold the slot after __stop__ before kill + respawn.
disposeGraceMs	number	5000	Time to wait after __shutdown__ before force-killing.
isAvailable	() => boolean	(default: Node-host capability check)	Override the availability probe.
(SD knobs)	negativePrompt string, steps number, cfgScale number, sampler string, seed number, width number, height number	—	Defaults folded into each command's JSON; each is also overridable per call.
(lifecycle hooks)	onLifecycle, onLoading, onCompiling, onReady, onGenerating, onComplete, onError — each (report) => void	—	Inherited from BatteryLifecycleHooks (shared across the on-device batteries). onGenerating fires per dnpr frame with a progress in 0..1; the firehose onLifecycle sees every phase.

The Wire Protocol

The local diffusion battery orchestrates the backend using a strict byte-oriented line protocol over stdin/stdout.

• Host→backend commands (stdin):
  • Generate: b2py t2im <rid> <json-args> — json-args = { "prompt": string, negativePrompt?, steps?, cfgScale?, sampler?, seed?, width?, height? } (SD knobs included only when set).
  • Edit: b2py im2im <rid> <json-args> — same knobs plus "images": [{ "b64": <base64 image bytes>, "mimeType"?: string }] carrying the input image(s) to edit.
  • Stop (advisory cancel): b2py __stop__ <rid>
  • Shutdown: b2py __shutdown__
• Backend→host events (stdout):
  • Model load: sdbk mdld <0..1> (no rid)
  • Ready: sdbk rdy (no payload)
  • Step progress: sdbk dnpr <rid> <0..1>
  • Image generated: sdbk nwim <rid> {"mimeType": "image/png", "b64": "..."} or {"mimeType": "image/png", "path": "/safe/..."} (Exactly one of b64 or path present).
  • Terminal OK: sdbk done <rid>
  • Terminal Error: sdbk err <rid> {"message": "..."}

All prefixes and tags are configurable. rid is a non-negative safe integer echoed on every request-scoped frame. The adapter handles graceful termination (best-effort __stop__, hard stop fallback). Any path output returned by the backend is safely managed and only deleted if it falls under the configured outputDir.

Reference Backend

A Python reference implementation using diffusers ships in the repository at docs/assembly/examples/local-diffusion-backend.py. This is purely a reference/example, not a shipped dependency.

Node-only callout

The LocalDiffusionGenerationAdapter orchestrates local system processes and is restricted to Node.js environments. Use lazy loading when constructing your adapter if operating in a dual-target codebase.

import { LocalDiffusionGenerationAdapter } from '@nhtio/adk/batteries/generation/local_diffusion'

const checkpoint = '/models/v1-5-pruned-emaonly.safetensors'

const adapter = new LocalDiffusionGenerationAdapter({
  // `command` is REQUIRED — the default spawner runs `spawn(command, args)` via node:child_process.
  command: 'python',
  // The default spawner does NOT forward `model` into the process args, so pass the checkpoint to your
  // backend explicitly here (the reference backend takes `--model`).
  args: ['local-diffusion-backend.py', '--model', checkpoint],
  // `model` is REQUIRED — it is handed to the spawner in its { command, args, model } context (use a
  // custom `spawn` if you want to derive args from it); it is NOT sent in the per-request command JSON.
  model: checkpoint,
  // Optional: only needed if your backend returns image results as a file `path` (not inline `b64`).
  // outputDir bounds where the adapter may delete those result files.
  // outputDir: '/tmp/diffusion-out',
})

const [image] = await adapter.generate('a red bicycle', { steps: 20, cfgScale: 7 })
// image: { kind: 'image', mimeType: 'image/png', bytes: Uint8Array }

A spawn override is only needed to customize how the process is launched (a wrapper script, env vars, a non-child_process transport); the default spawner covers the common command + args case.

Exceptions

Exception	Engine	Status	Fatal?	Thrown when
E_INVALID_OPENAI_GENERATION_OPTIONS	OpenAI	529	Yes	Adapter options fail schema validation.
E_OPENAI_GENERATION_HTTP_ERROR	OpenAI	502	No	Upstream HTTP call fails, retries exhausted. [status, detail].
E_OPENAI_GENERATION_REQUEST_TIMEOUT	OpenAI	504	No	requestTimeoutMs elapses. [requestTimeoutMs].
E_OPENAI_GENERATION_MALFORMED_RESPONSE	OpenAI	502	No	2xx response has no usable data[].b64_json. [detail].
E_INVALID_GEMINI_GENERATION_OPTIONS	Gemini	529	Yes	Adapter options fail schema validation.
E_GEMINI_GENERATION_HTTP_ERROR	Gemini	502	No	Upstream HTTP call fails, retries exhausted. [status, detail].
E_GEMINI_GENERATION_REQUEST_TIMEOUT	Gemini	504	No	requestTimeoutMs elapses. [requestTimeoutMs].
E_GEMINI_GENERATION_MALFORMED_RESPONSE	Gemini	502	No	No candidates, or zero image parts (refusal-shaped response); detail embeds any text parts.
E_INVALID_TRANSFORMERS_JS_GENERATION_OPTIONS	Transformers.js	529	Yes	Adapter options fail schema validation.
E_TRANSFORMERS_JS_GENERATION_ENGINE_ERROR	Transformers.js	502	No	Model/processor load fails, generate_images throws, or the result is empty.
E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION	Transformers.js	501	Yes	edit() is called — always, unconditionally. [operation, reason].
E_INVALID_LOCAL_DIFFUSION_OPTIONS	Local Diffusion	529	Yes	Adapter options fail schema validation. [detail].
E_LOCAL_DIFFUSION_BACKEND_ERROR	Local Diffusion	502	No	Spawn failure, an err frame, or the child failing (error/exit/close/stdout-end) after startup completed. [detail, exitCode?, signal?].
E_LOCAL_DIFFUSION_STARTUP_TIMEOUT	Local Diffusion	504	No	Startup failed: no rdy before startupTimeoutMs, or the child errored/exited/closed (or its stdout ended) while still starting. [ms].
E_LOCAL_DIFFUSION_REQUEST_TIMEOUT	Local Diffusion	504	No	A request exceeds requestTimeoutMs. [ms].
E_LOCAL_DIFFUSION_ABORTED	Local Diffusion	499	No	The caller's AbortSignal fired.
E_LOCAL_DIFFUSION_MALFORMED_FRAME	Local Diffusion	502	No	A malformed/oversized frame or an invalid/oversized image payload. [detail].
E_LOCAL_DIFFUSION_BUSY	Local Diffusion	409	No	A second call arrives while one is in flight (single-flight).
E_LOCAL_DIFFUSION_DISPOSED	Local Diffusion	409	No	A call after dispose(), or the rejection raised on an interrupted preload/request.

19 exceptions total across the four engines. The 529/501-fatal members are config/call-site bugs raised before any network/inference call happens; the rest are non-fatal, raised from the async generate/edit path (or process lifecycle) itself.

Decision tree

• Need cloud-quality generation and edit(), no local compute budget → OpenAI or Gemini.
• Need edit() specifically through a gateway/LB that 404s OpenAI-shaped /v1/images/edits → Gemini — probe-confirmed working through the same gateway topology.
• Need on-device/offline generation, sovereignty over model weights matters more than latency, generate()- only is acceptable, and a ~2GB download + minutes-per-image is tolerable → Transformers.js — run it behind forkIsolated for anything beyond a throwaway script.
• Have a specific local model like Stable Diffusion, require generate()/edit() and cancellation support, and can supply a Python inference subprocess → Local Diffusion.
• None of the above → this domain is opt-in like every other battery; skip it entirely if your agent never produces media.

Where to go next

• Generation battery — the thesis and quick start.
• OpenAI / Gemini / Transformers.js / Local Diffusion — narrative per-engine reference.
• Recipes — BYO Tool wiring, edit-tool, Media.stash, isolation, live-testing.
• Embeddings batteries — the same one-contract/four-constructors shape this domain mirrors.