Skip to content
7 min read · 1,392 words

Generation batteries

The Generation battery hub page covers the thesis and a quick start; each engine's own page (OpenAI, Gemini, Transformers.js) covers its wire behavior in narrative form. This page is the side-by-side option/exception reference.

Barrels, side by side

OpenAIGeminiTransformers.js
Subpath@nhtio/adk/batteries/generation/openai@nhtio/adk/batteries/generation/gemini@nhtio/adk/batteries/generation/transformers_js
AdapterOpenAIGenerationAdapterGeminiGenerationAdapterTransformersJsGenerationAdapter
Options typeOpenAIGenerationAdapterOptionsGeminiGenerationAdapterOptionsTransformersJsGenerationAdapterOptions
ValidatorvalidateOpenAIGenerationOptionsvalidateGeminiGenerationOptionsvalidateTransformersJsGenerationOptions
Transportraw fetch, OpenAI /v1/images/* shaperaw fetch, native generativelanguage RESTon-device, optional @huggingface/transformers peer
edit()Yes (multipart)YesAlways throws

The aggregate barrel @nhtio/adk/batteries/generation re-exports everything from all three subpaths — adapters, option/schema types, exceptions, and toBytes (renamed toGenerationBytes to avoid colliding with the specialists battery's own toBytes export when both aggregates are imported side by side).

The shared contract

Every engine implements the same method surface:

MethodSignatureNotes
generate(prompt: string, opts?: GenerateOptions) => Promise<GeneratedMediaOutput[]>
edit(inputs: GenerationImageInput | GenerationImageInput[], prompt: string, opts?: EditOptions) => Promise<GeneratedMediaOutput[]>transformers.js always throws E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION.
isAvailable() => booleanOpenAI/Gemini: true whenever fetch exists. transformers.js: always true (it does NOT probe the peer — a missing @huggingface/transformers surfaces as an engine error at first preload()/generate()), unless an isAvailable override is supplied.
preload() => Promise<void>Warms the model/processor (transformers.js) or is a no-op (OpenAI/Gemini — no local state to warm).
reset() => voidClears any lazily-created model/processor instance.

GeneratedMediaOutput:

ts
interface GeneratedMediaOutput {
  kind: MediaKind    // 'image' for every engine today
  mimeType: string
  bytes: Uint8Array
  filename?: string
}

Re-exported type-only from the LLM Chat Completions battery's own shared types (chat_common/types.ts), so generated media and LLM-battery media-output describe the same shape.

GenerationImageInput — the edit() input duck

ts
type GenerationImageInput = Uint8Array | GenerationBytesInput | GenerationMediaLike

interface GenerationBytesInput {
  bytes: Uint8Array
  mimeType?: string
}

interface GenerationMediaLike {
  mimeType: string
  asBytes(): Uint8Array | Promise<Uint8Array>
}

A real @nhtio/adk Media instance satisfies GenerationMediaLike structurally — no import from this battery's _shared layer to Media's module, and no import the other way. This is CONTRIBUTING.md Design Decision #13's tier-2 rule in practice: battery _shared layers describe what they need by shape, never by importing a concrete core class. toGenerationBytes(input) (the aggregate's renamed export of _shared's toBytes) normalizes any of the three forms to { bytes, mimeType }.

BaseGenerationAdapterOptions — the one required field

ts
interface BaseGenerationAdapterOptions {
  model: string // required, no default, on every engine
}

Owned by the OpenAI battery (openai/types.ts), extended by Gemini and transformers.js in turn. No engine in this domain defaults model — naming it is always the caller's job, exactly as with the embeddings batteries.

OpenAI — OpenAIGenerationAdapterOptions

OptionTypeDefault
modelstring(required)
apiKeystring
baseURLstringhttps://api.openai.com/v1
headersRecord<string, string>
fetchtypeof fetchglobal fetch
responseFormatMode'auto' | 'send' | 'omit''auto'
sizestring
quality'low' | 'medium' | 'high' | 'auto'
background'transparent' | 'opaque' | 'auto'
outputFormat'png' | 'jpeg' | 'webp''png' (adapter runtime default; not schema-filled)
requestTimeoutMsnumber0 (disabled)
retry.maxAttemptsnumber1
retry.baseDelayMsnumber500
retry.maxDelayMsnumber30000
retry.retriableStatusesnumber[][429, 500, 502, 503, 504]
retry.honorRetryAfterbooleantrue

OpenAIGenerateOptions (per-call, generate()): n?: number (sent when defined), size?, quality? (same enum as above), outputFormat?, background?. OpenAIEditOptions (per-call, edit() only): n?: number, mask?: GenerationImageInput, size?: string, quality? (same enum). Multipart field name for each edit image: EDIT_IMAGE_FIELD_NAME = 'image[]' (probe-confirmed, exported so a custom transport/mock can match it exactly).

Gemini — GeminiGenerationAdapterOptions

Extends the OpenAI shapes (GenerationRetryConfig, GenerateOptions, EditOptions, BaseGenerationAdapterOptions) via re-export-then-extend:

OptionTypeDefault
modelstring(required)
apiKeystring— (sent as x-goog-api-key)
baseURLstringhttps://generativelanguage.googleapis.com/v1beta
headersRecord<string, string>
fetchtypeof fetchglobal fetch
responseModalities('TEXT' | 'IMAGE')[]['TEXT', 'IMAGE']
aspectRatiostring— (only sent when set)
requestTimeoutMsnumber0 (disabled)
retry.*identical defaults to OpenAI's retry shape

GeminiRequestPart/GeminiContent/GeminiGenerationConfig mirror the native generateContent request body; GeminiResponsePart tolerates both camelCase inlineData and snake_case inline_data on responses (requests always send camelCase). candidateCount is only sent when the call's n > 1.

Transformers.js — TransformersJsGenerationAdapterOptions

Extends BaseGenerationAdapterOptions and BatteryLifecycleHooks:

OptionTypeDefault
modelstring(required)
janusModelTransformersJsGenerationModel
processorTransformersJsGenerationProcessor
createModelCreateTransformersJsGenerationModeldynamic-import MultiModalityCausalLM.from_pretrained
createProcessorCreateTransformersJsGenerationProcessordynamic-import AutoProcessor.from_pretrained
devicestringenvironment default
dtypestringenvironment default
modelSourceTransformersJsGenerationModelSource— (falls through to HF when undefined)
onInitProgressTransformersJsGenerationProgressCallback
isAvailable() => booleanalways true (does NOT probe the peer)
encodeImageEncodeRawImageFnenv-branched toBlob/toSharp
doSamplebooleantrue
temperaturenumber
topKnumber
guidanceScalenumber
repetitionPenaltynumber
minNewTokensnumberprocessor.num_image_tokens
maxNewTokensnumberprocessor.num_image_tokens
chatTemplatestring'text_to_image'
rolestring'<|User|>'

Every sampling knob (doSamplerole) is also accepted per-call on generate(), overriding the adapter-level default. There is deliberately no topP option — the installed @huggingface/transformers build's TopPLogitsWarper branch is dead code; MultinomialSampler.sample() never reads top_p.

Exceptions

ExceptionEngineStatusFatal?Thrown when
E_INVALID_OPENAI_GENERATION_OPTIONSOpenAI529YesAdapter options fail schema validation.
E_OPENAI_GENERATION_HTTP_ERROROpenAI502NoUpstream HTTP call fails, retries exhausted. [status, detail].
E_OPENAI_GENERATION_REQUEST_TIMEOUTOpenAI504NorequestTimeoutMs elapses. [requestTimeoutMs].
E_OPENAI_GENERATION_MALFORMED_RESPONSEOpenAI502No2xx response has no usable data[].b64_json. [detail].
E_INVALID_GEMINI_GENERATION_OPTIONSGemini529YesAdapter options fail schema validation.
E_GEMINI_GENERATION_HTTP_ERRORGemini502NoUpstream HTTP call fails, retries exhausted. [status, detail].
E_GEMINI_GENERATION_REQUEST_TIMEOUTGemini504NorequestTimeoutMs elapses. [requestTimeoutMs].
E_GEMINI_GENERATION_MALFORMED_RESPONSEGemini502NoNo candidates, or zero image parts (refusal-shaped response); detail embeds any text parts.
E_INVALID_TRANSFORMERS_JS_GENERATION_OPTIONSTransformers.js529YesAdapter options fail schema validation.
E_TRANSFORMERS_JS_GENERATION_ENGINE_ERRORTransformers.js502NoModel/processor load fails, generate_images throws, or the result is empty.
E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATIONTransformers.js501Yesedit() is called — always, unconditionally. [operation, reason].

11 exceptions total across the three engines. The 529/501-fatal members are config/call-site bugs raised before any network/inference call happens; the rest are non-fatal, raised from the async generate/edit path itself.

Decision tree

  • Need cloud-quality generation and edit(), no local compute budget → OpenAI or Gemini.
  • Need edit() specifically through a gateway/LB that 404s OpenAI-shaped /v1/images/editsGemini — probe-confirmed working through the same gateway topology.
  • Need on-device/offline generation, sovereignty over model weights matters more than latency, generate()- only is acceptable, and a ~2GB download + minutes-per-image is tolerable → Transformers.js — run it behind forkIsolated for anything beyond a throwaway script.
  • None of the above → this domain is opt-in like every other battery; skip it entirely if your agent never produces media.

Where to go next