Generation batteries
The Generation battery hub page covers the thesis and a quick start; each engine's own page (OpenAI, Gemini, Transformers.js) covers its wire behavior in narrative form. This page is the side-by-side option/exception reference.
Barrels, side by side
| OpenAI | Gemini | Transformers.js | |
|---|---|---|---|
| Subpath | @nhtio/adk/batteries/generation/openai | @nhtio/adk/batteries/generation/gemini | @nhtio/adk/batteries/generation/transformers_js |
| Adapter | OpenAIGenerationAdapter | GeminiGenerationAdapter | TransformersJsGenerationAdapter |
| Options type | OpenAIGenerationAdapterOptions | GeminiGenerationAdapterOptions | TransformersJsGenerationAdapterOptions |
| Validator | validateOpenAIGenerationOptions | validateGeminiGenerationOptions | validateTransformersJsGenerationOptions |
| Transport | raw fetch, OpenAI /v1/images/* shape | raw fetch, native generativelanguage REST | on-device, optional @huggingface/transformers peer |
edit() | Yes (multipart) | Yes | Always throws |
The aggregate barrel @nhtio/adk/batteries/generation re-exports everything from all three subpaths — adapters, option/schema types, exceptions, and toBytes (renamed toGenerationBytes to avoid colliding with the specialists battery's own toBytes export when both aggregates are imported side by side).
The shared contract
Every engine implements the same method surface:
| Method | Signature | Notes |
|---|---|---|
generate | (prompt: string, opts?: GenerateOptions) => Promise<GeneratedMediaOutput[]> | |
edit | (inputs: GenerationImageInput | GenerationImageInput[], prompt: string, opts?: EditOptions) => Promise<GeneratedMediaOutput[]> | transformers.js always throws E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION. |
isAvailable | () => boolean | OpenAI/Gemini: true whenever fetch exists. transformers.js: always true (it does NOT probe the peer — a missing @huggingface/transformers surfaces as an engine error at first preload()/generate()), unless an isAvailable override is supplied. |
preload | () => Promise<void> | Warms the model/processor (transformers.js) or is a no-op (OpenAI/Gemini — no local state to warm). |
reset | () => void | Clears any lazily-created model/processor instance. |
GeneratedMediaOutput:
interface GeneratedMediaOutput {
kind: MediaKind // 'image' for every engine today
mimeType: string
bytes: Uint8Array
filename?: string
}Re-exported type-only from the LLM Chat Completions battery's own shared types (chat_common/types.ts), so generated media and LLM-battery media-output describe the same shape.
GenerationImageInput — the edit() input duck
type GenerationImageInput = Uint8Array | GenerationBytesInput | GenerationMediaLike
interface GenerationBytesInput {
bytes: Uint8Array
mimeType?: string
}
interface GenerationMediaLike {
mimeType: string
asBytes(): Uint8Array | Promise<Uint8Array>
}A real @nhtio/adk Media instance satisfies GenerationMediaLike structurally — no import from this battery's _shared layer to Media's module, and no import the other way. This is CONTRIBUTING.md Design Decision #13's tier-2 rule in practice: battery _shared layers describe what they need by shape, never by importing a concrete core class. toGenerationBytes(input) (the aggregate's renamed export of _shared's toBytes) normalizes any of the three forms to { bytes, mimeType }.
BaseGenerationAdapterOptions — the one required field
interface BaseGenerationAdapterOptions {
model: string // required, no default, on every engine
}Owned by the OpenAI battery (openai/types.ts), extended by Gemini and transformers.js in turn. No engine in this domain defaults model — naming it is always the caller's job, exactly as with the embeddings batteries.
OpenAI — OpenAIGenerationAdapterOptions
| Option | Type | Default |
|---|---|---|
model | string | (required) |
apiKey | string | — |
baseURL | string | https://api.openai.com/v1 |
headers | Record<string, string> | — |
fetch | typeof fetch | global fetch |
responseFormatMode | 'auto' | 'send' | 'omit' | 'auto' |
size | string | — |
quality | 'low' | 'medium' | 'high' | 'auto' | — |
background | 'transparent' | 'opaque' | 'auto' | — |
outputFormat | 'png' | 'jpeg' | 'webp' | 'png' (adapter runtime default; not schema-filled) |
requestTimeoutMs | number | 0 (disabled) |
retry.maxAttempts | number | 1 |
retry.baseDelayMs | number | 500 |
retry.maxDelayMs | number | 30000 |
retry.retriableStatuses | number[] | [429, 500, 502, 503, 504] |
retry.honorRetryAfter | boolean | true |
OpenAIGenerateOptions (per-call, generate()): n?: number (sent when defined), size?, quality? (same enum as above), outputFormat?, background?. OpenAIEditOptions (per-call, edit() only): n?: number, mask?: GenerationImageInput, size?: string, quality? (same enum). Multipart field name for each edit image: EDIT_IMAGE_FIELD_NAME = 'image[]' (probe-confirmed, exported so a custom transport/mock can match it exactly).
Gemini — GeminiGenerationAdapterOptions
Extends the OpenAI shapes (GenerationRetryConfig, GenerateOptions, EditOptions, BaseGenerationAdapterOptions) via re-export-then-extend:
| Option | Type | Default |
|---|---|---|
model | string | (required) |
apiKey | string | — (sent as x-goog-api-key) |
baseURL | string | https://generativelanguage.googleapis.com/v1beta |
headers | Record<string, string> | — |
fetch | typeof fetch | global fetch |
responseModalities | ('TEXT' | 'IMAGE')[] | ['TEXT', 'IMAGE'] |
aspectRatio | string | — (only sent when set) |
requestTimeoutMs | number | 0 (disabled) |
retry.* | — | identical defaults to OpenAI's retry shape |
GeminiRequestPart/GeminiContent/GeminiGenerationConfig mirror the native generateContent request body; GeminiResponsePart tolerates both camelCase inlineData and snake_case inline_data on responses (requests always send camelCase). candidateCount is only sent when the call's n > 1.
Transformers.js — TransformersJsGenerationAdapterOptions
Extends BaseGenerationAdapterOptions and BatteryLifecycleHooks:
| Option | Type | Default |
|---|---|---|
model | string | (required) |
janusModel | TransformersJsGenerationModel | — |
processor | TransformersJsGenerationProcessor | — |
createModel | CreateTransformersJsGenerationModel | dynamic-import MultiModalityCausalLM.from_pretrained |
createProcessor | CreateTransformersJsGenerationProcessor | dynamic-import AutoProcessor.from_pretrained |
device | string | environment default |
dtype | string | environment default |
modelSource | TransformersJsGenerationModelSource | — (falls through to HF when undefined) |
onInitProgress | TransformersJsGenerationProgressCallback | — |
isAvailable | () => boolean | always true (does NOT probe the peer) |
encodeImage | EncodeRawImageFn | env-branched toBlob/toSharp |
doSample | boolean | true |
temperature | number | — |
topK | number | — |
guidanceScale | number | — |
repetitionPenalty | number | — |
minNewTokens | number | processor.num_image_tokens |
maxNewTokens | number | processor.num_image_tokens |
chatTemplate | string | 'text_to_image' |
role | string | '<|User|>' |
Every sampling knob (doSample…role) is also accepted per-call on generate(), overriding the adapter-level default. There is deliberately no topP option — the installed @huggingface/transformers build's TopPLogitsWarper branch is dead code; MultinomialSampler.sample() never reads top_p.
Exceptions
| Exception | Engine | Status | Fatal? | Thrown when |
|---|---|---|---|---|
E_INVALID_OPENAI_GENERATION_OPTIONS | OpenAI | 529 | Yes | Adapter options fail schema validation. |
E_OPENAI_GENERATION_HTTP_ERROR | OpenAI | 502 | No | Upstream HTTP call fails, retries exhausted. [status, detail]. |
E_OPENAI_GENERATION_REQUEST_TIMEOUT | OpenAI | 504 | No | requestTimeoutMs elapses. [requestTimeoutMs]. |
E_OPENAI_GENERATION_MALFORMED_RESPONSE | OpenAI | 502 | No | 2xx response has no usable data[].b64_json. [detail]. |
E_INVALID_GEMINI_GENERATION_OPTIONS | Gemini | 529 | Yes | Adapter options fail schema validation. |
E_GEMINI_GENERATION_HTTP_ERROR | Gemini | 502 | No | Upstream HTTP call fails, retries exhausted. [status, detail]. |
E_GEMINI_GENERATION_REQUEST_TIMEOUT | Gemini | 504 | No | requestTimeoutMs elapses. [requestTimeoutMs]. |
E_GEMINI_GENERATION_MALFORMED_RESPONSE | Gemini | 502 | No | No candidates, or zero image parts (refusal-shaped response); detail embeds any text parts. |
E_INVALID_TRANSFORMERS_JS_GENERATION_OPTIONS | Transformers.js | 529 | Yes | Adapter options fail schema validation. |
E_TRANSFORMERS_JS_GENERATION_ENGINE_ERROR | Transformers.js | 502 | No | Model/processor load fails, generate_images throws, or the result is empty. |
E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION | Transformers.js | 501 | Yes | edit() is called — always, unconditionally. [operation, reason]. |
11 exceptions total across the three engines. The 529/501-fatal members are config/call-site bugs raised before any network/inference call happens; the rest are non-fatal, raised from the async generate/edit path itself.
Decision tree
- Need cloud-quality generation and
edit(), no local compute budget → OpenAI or Gemini. - Need
edit()specifically through a gateway/LB that 404s OpenAI-shaped/v1/images/edits→ Gemini — probe-confirmed working through the same gateway topology. - Need on-device/offline generation, sovereignty over model weights matters more than latency,
generate()- only is acceptable, and a ~2GB download + minutes-per-image is tolerable → Transformers.js — run it behindforkIsolatedfor anything beyond a throwaway script. - None of the above → this domain is opt-in like every other battery; skip it entirely if your agent never produces media.
Where to go next
- Generation battery — the thesis and quick start.
- OpenAI / Gemini / Transformers.js — narrative per-engine reference.
- Recipes — BYO Tool wiring, edit-tool,
Media.stash, isolation, live-testing. - Embeddings batteries — the same one-contract/three-constructors shape this domain mirrors.