Skip to content
3 min read · 656 words

Generation Battery

Every other battery domain in this repo helps an agent consume media — transcribe audio, OCR a scan, embed a document. This one is the other direction: agents that produce media, not just consume it. A tool that renders a chart, illustrates a scene, or recolors a product photo needs a text→image (or image→image) engine behind it, and this domain is that engine — three of them, in fact, sharing one contract.

The thesis

1. One shared contract, three engines. Every engine in this domain exposes the same two calls — generate(prompt, opts?) and edit(inputs, prompt, opts?) — returning the same Promise<GeneratedMediaOutput[]> shape, regardless of whether the bytes came from an OpenAI-compatible JSON response, a native Gemini generateContent call, or an on-device Janus model. Swap OpenAIGenerationAdapter for GeminiGenerationAdapter and nothing about your calling code changes except the constructor — exactly the pattern the embeddings batteries already established for this codebase.

2. Three constructors, one shape. BaseGenerationAdapterOptions ({ model: string }, required, no default) is owned by the OpenAI battery and extended by Gemini and transformers.js in turn — the same extension chain the transformers.js Embeddings battery uses over the OpenAI Embeddings option type. Each engine only adds what its own transport actually needs: HTTP auth/retry/timeout fields for OpenAI and Gemini, model/processor injection seams for transformers.js.

OpenAIGenerationAdapterGeminiGenerationAdapterTransformersJsGenerationAdapter
Subpath@nhtio/adk/batteries/generation/openai@nhtio/adk/batteries/generation/gemini@nhtio/adk/batteries/generation/transformers_js
Transportraw fetch, OpenAI /v1/images/* shaperaw fetch, native generativelanguage RESTon-device, @huggingface/transformers (optional peer)
AuthapiKeyAuthorization: BearerapiKeyx-goog-api-key, or gateway headersnone
generate()YesYesYes (EXPERIMENTAL)
edit()Yes (multipart)YesThrows E_TRANSFORMERS_JS_GENERATION_UNSUPPORTED_OPERATION
Cost profileper-image API costper-image API costone-time ~2GB download, CPU/WASM compute (minutes/image)
Environmentanywhere (Node/browser/edge)anywhere (Node/browser/edge)environment-neutral, no WebGPU requirement

Each engine's own page (OpenAI, Gemini, transformers.js) covers its full option surface and wire-level behavior; Assembly → Generation batteries is the side-by-side reference across all three.

Quick start

ts
import { OpenAIGenerationAdapter } from '@nhtio/adk/batteries/generation/openai'

const generator = new OpenAIGenerationAdapter({
  model: 'gpt-image-1',
  apiKey: process.env.OPENAI_API_KEY,
})

const [output] = await generator.generate('A watercolor illustration of a lighthouse at dawn')
// output: { kind: 'image', mimeType: 'image/png', bytes: Uint8Array, filename: 'generated-1.png' }

Swap the constructor for the Gemini engine and the call shape is identical:

ts
import { GeminiGenerationAdapter } from '@nhtio/adk/batteries/generation/gemini'

const generator = new GeminiGenerationAdapter({
  model: 'gemini-2.5-flash-image',
  apiKey: process.env.GEMINI_API_KEY,
})

const [output] = await generator.generate('A watercolor illustration of a lighthouse at dawn')

GeneratedMediaOutput — bytes out, Media handoff is yours

generate/edit resolve GeneratedMediaOutput[]:

ts
interface GeneratedMediaOutput {
  kind: MediaKind        // 'image' for every engine in this domain today
  mimeType: string       // e.g. 'image/png'
  bytes: Uint8Array      // the decoded image bytes
  filename?: string      // e.g. 'generated-1.png'
}

That's plain bytes, deliberately — not a Media instance, not a ToolCall result. Turning GeneratedMediaOutput.bytes into a first-party Media that lands on a tool result (persisted via ctx.storeMediaBytes, trust-tiered via Media.toolGenerated) is consumer-land: your Tool handler's job, not the adapter's. See Recipes for the exact wiring.

What this is not

No agent primitive

ADK has no GenerationTool class, no forged tool, no TurnRunnerConfig wiring for image generation. A generate/edit call never touches ctx.tools, ctx.turnMessages, or ctx.turnToolCalls on its own — these adapters don't know a DispatchContext exists. Exactly the same posture as the embeddings and specialists domains: the adapter is the whole product.

You wire it, on purpose

This is deliberate, not an oversight — mirroring the specialists battery's own framing: whether a generation call becomes a Tool the model invokes, a step in your own pipeline the model never sees, or a courtesy Media.stash write, is a decision only your deployment can make. The Recipes page covers all of these.

Where to go next

  • OpenAI/v1/images/generations + /v1/images/edits, responseFormatMode, multipart edit details, gateway routing.
  • Gemini — native generateContent REST, response-modality defaults, image-part ordering, refusal surfacing.
  • Transformers.js — the EXPERIMENTAL on-device Janus engine, real knobs, perf reality.
  • Recipes — BYO Tool wiring, edit-tool consuming inbound Media, Media.stash captions, running the on-device engine behind isolation, live-testing.
  • Assembly → Generation batteries — the full option/exception surface, side by side.
  • Bring your own tools — the Tool construction pattern used throughout the recipes.