Generation — Local Diffusion

Unlike the HTTP-backed OpenAI and Gemini engines, and the in-process on-device Transformers.js Janus engine, the local-diffusion engine runs generation in a Bring-Your-Own inference subprocess and coordinates with it over a plain stdin/stdout line protocol (modeled on DiffusionBee). The ADK ships the protocol and drives the process; you supply a protocol-speaking backend — for example a small Python diffusers script that loads one of your local .safetensors checkpoints. Reach for it when you want a self-hosted local Stable-Diffusion pipeline with streamed per-step progress and best-effort cancellation, and neither the cloud engines nor the ONNX-only Janus engine fits.

Node-only, subpath-only

This engine spawns a child process, so it imports node:* builtins (lazily, inside methods) and is reachable only from its own subpath @nhtio/adk/batteries/generation/local_diffusion. It is deliberately excluded from the environment-neutral @nhtio/adk/batteries/generation aggregate barrel — importing it will not load in a browser or Worker bundle. The library ships the protocol, not a Python/torch runtime; you provide the inference process.

import { LocalDiffusionGenerationAdapter } from '@nhtio/adk/batteries/generation/local_diffusion'

const checkpoint = '/models/v1-5-pruned-emaonly.safetensors'

const generator = new LocalDiffusionGenerationAdapter({
  command: 'python', // REQUIRED — the default spawner runs `spawn(command, args)`.
  args: ['local-diffusion-backend.py', '--model', checkpoint], // pass the model to YOUR backend here.
  model: checkpoint, // REQUIRED — handed to the spawner context; NOT sent in the request JSON.
})

const [image] = await generator.generate('a watercolor lighthouse at dawn', { steps: 20, cfgScale: 7 })
// image: { kind: 'image', mimeType: 'image/png', bytes: Uint8Array }

Options

Extends BaseGenerationAdapterOptions (model) and BatteryLifecycleHooks.

Option	Type	Default	Notes
model	string	(required)	Checkpoint id/path the backend loads. Handed to the spawner's { command, args, model } context; NOT sent in the t2im/im2im command JSON.
command	string	(required)	Backend executable the default spawner launches.
args	string[]	[]	Arguments passed to command by the default spawner.
spawn	DiffusionBackendSpawner	(lazy node:child_process.spawn)	Override the process factory. Receives { command, args, model }; returns a DiffusionBackendProcess.
fs	DiffusionFsLike	(lazy node:fs/promises)	{ readFile, unlink } used only for backend-written path results.
outputDir	string	—	The ONLY directory under which the adapter may delete backend-written result files (lexical containment).
maxDecodedBytes	number	52428800	Max decoded size of an inline-base64 image (bounded pre-decode).
maxLineBytes	number	1048576	Cap on a single protocol line (framer buffer).
commandPrefix	string	'b2py'	Host→backend command line prefix.
eventPrefix	string	'sdbk'	Backend→host event line prefix.
protocol	Partial<ProtocolConfig>	—	Bulk protocol-tag override (merged under the granular fields).
ops	Partial<ProtocolConfig['ops']>	{ generate: 't2im', edit: 'im2im' }	Operation sub-tags.
events	Partial<ProtocolConfig['events']>	DiffusionBee	Event sub-tags (mdld/rdy/dnpr/nwim/done/err).
control	Partial<ProtocolConfig['control']>	DiffusionBee	Control sub-tags (__stop__/__shutdown__).
startupTimeoutMs	number	30000	Deadline waiting for the rdy event.
requestTimeoutMs	number	0 (disabled)	Per-request timeout.
abortGraceMs	number	5000	Time to hold the slot after __stop__ before kill + respawn.
disposeGraceMs	number	5000	Time to wait after __shutdown__ before force-killing.
isAvailable	() => boolean	(Node-host check)	Override the availability probe.
negativePrompt / steps / cfgScale / sampler / seed / width / height	SD knobs	—	Defaults folded into each command's JSON; each is also overridable per call.
onLifecycle / onLoading / onCompiling / onReady / onGenerating / onComplete / onError	(report) => void	—	Inherited BatteryLifecycleHooks. onGenerating fires per dnpr frame with a progress in 0..1.

The wire protocol

The adapter drives the backend over a byte-safe, newline-delimited line protocol. All prefixes and tags are configurable; the DiffusionBee-compatible defaults are shown.

Host → backend (stdin):

Command	Meaning
b2py t2im <rid> <json>	Generate. json = { "prompt": string, negativePrompt?, steps?, cfgScale?, sampler?, seed?, width?, height? } (knobs included only when set).
b2py im2im <rid> <json>	Edit. Same knobs plus "images": [{ "b64": <base64>, "mimeType"?: string }].
b2py __stop__ <rid>	Advisory cancel of the current request.
b2py __shutdown__	Graceful shutdown request.

Backend → host (stdout):

Event	Meaning
sdbk mdld <0..1>	Startup model-load progress (no rid) → onLoading.
sdbk rdy	Backend ready (no payload) → resolves preload().
sdbk dnpr <rid> <0..1>	Per-step generation progress → onGenerating { progress }.
sdbk nwim <rid> {…}	One finished image: { "mimeType": string } plus EXACTLY one of "b64" (inline) or "path" (a backend-written file).
sdbk done <rid>	Request completed successfully.
sdbk err <rid> {"message": string}	Request failed → E_LOCAL_DIFFUSION_BACKEND_ERROR.

rid is a non-negative safe integer the host assigns per request and the backend MUST echo on every request-scoped frame. Frames whose rid does not match the active request are discarded (fenced), so a cancelled call's late frames cannot bleed into the next one.

Lifecycle: preload / reset / dispose

• preload() spawns the backend, attaches failure listeners, and resolves on rdy (bounded by startupTimeoutMs). It is single-flight and cached; a spawn/startup failure rejects and clears the cache so a later call re-spawns.
• reset() is a synchronous hard stop: it invalidates the current process (epoch bump), rejects any in-flight preload/request with a typed error, kills the child, and returns to idle. It is non-terminal — the adapter can be preload()ed again.
• dispose() / [Symbol.asyncDispose] is the terminal two-phase shutdown: it rejects in-flight work with E_LOCAL_DIFFUSION_DISPOSED, writes __shutdown__, waits up to disposeGraceMs for the child to close, then force-kills. It is idempotent (a cached promise); any call after dispose() rejects with E_LOCAL_DIFFUSION_DISPOSED.

Cancellation is best-effort

Pass a per-call AbortSignal, or set requestTimeoutMs. Either writes an advisory __stop__ frame and rejects the caller immediately (E_LOCAL_DIFFUSION_ABORTED / E_LOCAL_DIFFUSION_REQUEST_TIMEOUT) — but the single-flight slot stays held (a next call gets E_LOCAL_DIFFUSION_BUSY) until either the cancelled rid's terminal frame confirms the backend stopped OR abortGraceMs elapses, at which point the adapter kills and respawns the backend before admitting the next call. __stop__ is only advisory; the guaranteed stop is reset() / dispose().

const controller = new AbortController()
const p = generator.generate('a long, slow render', { signal: controller.signal })
setTimeout(() => controller.abort(), 1_000)
// p rejects with E_LOCAL_DIFFUSION_ABORTED; the slot frees once the backend acknowledges or the grace elapses.

Result files and outputDir

A nwim frame may return an image inline (b64) or as a backend-written file path. For a path, the adapter reads it through the injectable fs seam, then deletes it — but ONLY when the path lexically resolves inside (or is equal to) the configured outputDir (it resolves .. and rejects sibling-prefix escapes such as /output2). A path outside outputDir is read but never unlinked. Cleanup failure is diagnostic-only and never fails the generation. Containment is lexical, not realpath-based: this cleanup trusts that your outputDir does not contain adversarial symlinks.

Reference backend

A runnable Python reference backend (using diffusers from_single_file) ships in the repository at docs/assembly/examples/local-diffusion-backend.py. It reads the command frames on stdin, emits mdld/rdy on startup, streams dnpr per denoise step, writes the PNG and emits nwim + done, and honors __stop__/__shutdown__. It is an example, not a shipped dependency — copy and adapt it to your own inference stack. Any process that speaks the protocol above works.

Exceptions

Exception	Status	Fatal?	Thrown when
E_INVALID_LOCAL_DIFFUSION_OPTIONS	529	Yes	Adapter options fail validation (missing command/model, unknown key). [detail].
E_LOCAL_DIFFUSION_BACKEND_ERROR	502	No	Spawn failure, an err frame, or the child failing (error/exit/close/stdout-end) after startup. [detail, exitCode?, signal?].
E_LOCAL_DIFFUSION_STARTUP_TIMEOUT	504	No	No rdy before startupTimeoutMs, or the child failed while still starting. [ms].
E_LOCAL_DIFFUSION_REQUEST_TIMEOUT	504	No	A request exceeds requestTimeoutMs. [ms].
E_LOCAL_DIFFUSION_ABORTED	499	No	The caller's AbortSignal fired.
E_LOCAL_DIFFUSION_MALFORMED_FRAME	502	No	A malformed/oversized frame or an invalid/oversized image payload. [detail].
E_LOCAL_DIFFUSION_BUSY	409	No	A second call arrives while one is in flight (single-flight).
E_LOCAL_DIFFUSION_DISPOSED	409	No	A call after dispose(), or the rejection raised on an interrupted preload/request.

Where to go next

• OpenAI / Gemini — cloud-quality generation and editing over HTTP.
• Transformers.js — in-process on-device generation (ONNX Janus), no subprocess.
• Recipes — wiring generate()/edit() into a real Tool.
• Assembly → Generation batteries — all four engines' option and exception tables side by side.