---
url: 'https://adk.nht.io/batteries/media/engines.md'
description: >-
  Self-declaring capability engines: the convert/mutate/edit taxonomy, why
  mutate and edit are different capabilities, and declarations as plain data.
---

# The Capability Model

## LLM summary — Media engine capability model

* An engine is a self-declaring capability provider (`MediaEngine` from `@nhtio/adk/batteries/media/contracts`): `{ id, converts?: ConvertCapability[], mutates?: MutateCapability[], edits?: EditCapability[] }`. Three capability shapes only: ConvertCapability `{ from: MimePattern[], to: string[], convert(request) → { outputs: [{bytes, mimeType, meta?}] } }` (format change — OCR is image/*→txt, transcription is pcm→txt/srt/vtt/json, audio decode is audio/*→pcm, PDF embedded-image extraction is pdf→images multi-output, generation is EMPTY\_MIME→anything), MutateCapability `{ over, ops, encodes, mutate(request) → {bytes, mimeType} }` (same-format FUSED transform: resize/rotate/flip/strip\_metadata fold into ONE decode/encode, optional re-encode via request.format), and EditCapability `{ over, ops, edit(request) → {bytes, mimeType, summary?} }` (structural document ops, ONE named op per request — `sheet.update_cells` etc.).
* Mutate vs edit split rationale: mutate exists for FUSION (adjacent image transforms share one decode/encode — a fused request shape with fixed op fields), edit exists for FIDELITY ARBITRATION (one op per request with open args; same ops declarable by engines that differ in what they preserve — exceljs keeps styling, sheetjs CE strips it; supply order picks). Do not contort one into the other.
* Declarations are plain data — the registry never calls engine code to know the shape of the world. Duck-typed guards validate every engine + capability entry at construction (`E_INVALID_MEDIA_PIPELINE_CONFIG` naming the index).
* How supply/dispatch/selection middleware work: see Engine Dispatch. Bundled implementations: see The Bundled Fleet. Generation (`EMPTY_MIME`): see Generating Media. Custom engines + runtime contracts: see BYO Runtime.

Every capability with more than one viable implementation is a contract, not a dependency. You compose the implementation when you build the pipeline, the choice is visible at the call site, and swapping it later touches one line. Nothing is ever auto-selected — that's the ["bring your own everything"](../../what-adk-is) rule applied to media processing, and it is not negotiable here either. A library that picks your OCR engine for you has also picked your WASM payload, your model downloads, your licensing exposure, and your failure modes. We'd rather hand you a table of trade-offs and make you read it — that table lives on [The Bundled Fleet](./fleet).

## Three capabilities. That's the whole taxonomy.

A media engine does exactly three things, and the contract refuses to pretend otherwise:

* **Convert** — change the format. `docx → pdf`. And once you see it that way, a lot of things that usually get bespoke interfaces turn out to be conversions wearing costumes: OCR is `image/* → txt`. Transcription is `pcm → txt`. Decoding audio is `audio/* → pcm`. Extracting a PDF's embedded images is `pdf → images` (multi-output). And generating a new file is a conversion whose source format happens to be nothing — `EMPTY_MIME → xlsx` (see [Generating Media](./generating)). One declaration shape covers all of them.
* **Mutate** — change the content, keep the family. Resize, rotate, flip, strip metadata — with an optional re-encode riding the same call, because making a fused resize-and-convert cost two decode/encode cycles to satisfy a taxonomy would be vandalism.
* **Edit** — restructure the document. Insert a worksheet row, rename a sheet, splice columns. One named op per request.

::: info Why mutate and edit are different capabilities (and not one)
The obvious "simplification" — one mutation capability for both pixels and worksheets — was considered and rejected, because the two shapes exist for opposite reasons:

**Mutate exists for fusion.** Adjacent image transforms (`image resize | image rotate | image format`) fold into ONE engine request with fixed fields (`resize`, `rotate`, `flip`, `format`), because every image op pays the same decode/encode toll and paying it once instead of three times is the whole point. The request shape is closed and image-shaped on purpose — fusion only works when the engine sees all the folded ops at once.

**Edit exists for fidelity arbitration.** A worksheet splice is not a pixel pass: there's no decode/encode to amortize, so edits run one named op per request with open, verb-shaped args (`{ op: 'sheet.update_cells', args }`). What edits need instead is something mutate never needed — **two engines truthfully declaring the same ops while differing in what they preserve.** ExcelJS keeps fonts, fills, and comments through an edit; SheetJS CE strips them. Same declared capability, different fidelity, and the registry doesn't rank fidelity — your engines-array order does. That arbitration is the [fleet page's two-spreadsheet-engines story](./fleet#two-spreadsheet-engines-the-consumer-picks-the-trade-off).

Contorting worksheet ops into the fused image request (or un-fusing image ops into named requests) would have cost one of the two properties. Two shapes, each earning its keep.
:::

## Declarations are plain data

An engine declares what it can do as **plain data** — the registry never has to call engine code to know the shape of the world:

```typescript
import type { MediaEngine } from '@nhtio/adk/batteries/media/contracts'

const myEngine: MediaEngine = {
  id: 'my-engine', // shows up in error messages, so name it honestly
  converts: [
    { from: ['application/pdf'], to: ['txt', 'html'], convert: async (request) => ({ outputs: [/* … */] }) },
  ],
  mutates: [
    { over: ['image/*'], ops: ['resize', 'rotate'], encodes: ['png', 'jpg'], mutate: async (request) => ({ /* … */ }) },
  ],
  edits: [
    { over: ['application/vnd…spreadsheetml.sheet'], ops: ['sheet.update_cells'], edit: async (request) => ({ /* … */ }) },
  ],
}
```

A new capability is a new edge in the data — never a new contract, never a new slot, never a registry PR. Plain data is also the only thing the registry trusts: it reads declarations to decide what a deployment can do, and it never calls into an engine to find out. An engine that lies in its declarations gets caught the moment it's asked to deliver, not before — which is exactly why the declarations are validated and the outputs are validated and the types are treated as the least of the three.

The guard is the contract: TypeScript types are a courtesy; construction duck-checks every engine and every declared capability entry, and a malformed one fails loud with the index named — `engines[1] ("sharp") does not implement the MediaEngine contract` — never silently mid-chain. A misbehaving engine's *output* is also validated at the step boundary, so a BYO engine that returns garbage produces a clear error pointing at the engine — not corrupt media three steps later with your name on it. Loud at construction, loud at the boundary, never quietly wrong in between: the failure you can't see is the only kind this contract is afraid of.

## Read next

* [Engine Dispatch](./dispatch) — ordered supply, the one selection rule, multi-hop pathfinding, selection middleware.
* [The Bundled Fleet](./fleet) — every bundled engine, its trade-offs, and the two-spreadsheet-engines story.
* [BYO Runtime & Custom Engines](./runtime) — implementing the contract yourself.