---
url: 'https://adk.nht.io/batteries/media/forge.md'
description: >-
  forgeMediaTools: the composite media_query surface, the granular per-verb
  surface, media referencing by id, the gate seam, and the failure-string
  contract.
---

# Agent Tools

## LLM summary — Media agent tools (forge)

* `forgeMediaTools(mp, { surface, resolveMedia?, gate?, overrides? })` from `@nhtio/adk/batteries/media/forge` returns `Record<string, Tool>` keyed by tool name; register selectively or `Object.values(tools)` on `TurnRunnerConfig.tools`.
* `surface: 'composite'` → `list_media` + ONE `media_query` tool taking `{ media_id, q?: string, ops?: MediaOp[] }` (exactly one of q/ops). Its description embeds the engine-narrowed verb grammar + toPipe-generated examples (regexes shown JSON-escaped).
* `surface: 'granular'` → `list_media` + one tool per available verb (`select`, `chunk`, `sheet_update_cells`, …), schemas generated from the verb table; media-ref args accept plain id strings.
* Media referencing: LLM batteries render `[media id: <id> | <filename>]` markers beside every attachment/media result; `list_media` enumerates `{id, filename, kind, mimeType, source, origin}` for everything visible in the turn. `resolveMedia` (default: scan `ctx.turnMessages[].attachments` + `ctx.turnToolCalls[].results`) resolves ids; `@id` refs inside statements resolve against the same turn state.
* Generation sentinel: `media_id: "empty:<format>"` creates NEW media instead of resolving — full semantics on the Generating Media page. Advertisement (description section, schema hint, MEDIA\_NOT\_FOUND exemplar) appears only when `convertTargets(EMPTY_MIME)` is non-empty.
* Results: mutation chains → `Media.toolGenerated` persisted via `ctx.storeMediaBytes` (first-party, source-tagged `tool:<name>`); media lists → `Media[]`; text/json → strings.
* Failures are readable strings the model can repair from: `Error (MEDIA_NOT_FOUND): … Visible media ids: …` (+ the empty: exemplar when creatable), `Error (UNKNOWN_VERB): … did you mean …`, `Error (ENGINE_REQUIRED): … Do not retry …`, `Error (STEP_FAILED): …`.
* `gate?: ToolGateFn = (ctx, { tool, args }) => void|Promise<void>` runs before ANY execution; a throw aborts through the standard tool-error path. Canonical implementation awaits `ctx.waitFor({ reason: 'tool_approval', payload: call })`. The same optional gate exists on the scrapper and searxng factories.

The pipeline core never imports anything agent-shaped — you can use it in a build script, a worker, a CLI. The forge is the layer that knows the ADK loop: it mints [`Tool`](https://adk.nht.io/api/@nhtio/adk/forge/classes/Tool) instances over a configured pipeline, wires media in by reference and out as first-class [`Media`](https://adk.nht.io/api/@nhtio/adk/common/classes/Media), and renders every failure as a string a model can act on.

## Two surfaces, your choice

```typescript
import { createMediaPipeline } from '@nhtio/adk/batteries/media'
import { forgeMediaTools } from '@nhtio/adk/batteries/media/forge'

const mp = await createMediaPipeline({ engines: [/* … */] })

const tools = forgeMediaTools(mp, { surface: 'composite' })
// → { list_media, media_query }

const runner = new TurnRunner({
  ...storageCallbacks,
  executorCallback,
  tools: Object.values(tools),
})
```

**`composite`** is one `media_query` tool whose args are `{ media_id, q }` — the model writes pipe statements. Multi-step work costs one round-trip, the tool list stays small, and the description embeds the deployment's actual grammar: only the verbs your engines support, with few-shot examples generated from real plans via `toPipe()` (so they can never drift from the parser — including showing regexes in their JSON-escaped form, which is how the model must write them).

::: info Field note: the first live model, cold
The first time a real model touched this surface — live API, no fine-tuning, nothing but the grammar in the tool description — it read the inline `[media id: …]` marker on the attachment and wrote `redact match=/\d{3}-\d{2}-\d{4}/ replace="[SSN]"` on the first attempt: regex literal, named args, quoted value, correct verb. Asked to find a document with no id in its prompt, it called `list_media`, read the JSON, and drove `media_query` with the discovered id. Asked to convert in a deployment with zero engines, it attempted once, read the do-not-retry directive, and fell back gracefully. One model through one gateway is a data point, not a proof — but the whole battery is a bet that pipes-plus-`key=value` is the one idiom every model already knows, and the first time the bet was actually placed, it paid. The gated live suite (`forge_real_llm.node.spec.ts`, `TEST_OPENAI_*` env) re-places it on demand.
:::

**`granular`** mints one narrow tool per available verb — `select`, `chunk`, `sheet_update_cells`, and so on, with schemas generated from the same verb table. Better for small models that handle flat tools more reliably than statement composition; costs a bigger tool list and one round-trip per step. We recommend `composite` and we built the whole grammar so we could — but "small model, flat tools" is a real constraint, not a failure of taste, so the granular surface is a first-class citizen rather than a grudging one.

Either way, **tools follow engine supply**. No convert engine ⇒ no `convert` tool, no `convert` verb in the grammar text. A tool list that advertises capabilities the deployment can't deliver is lying to the model, and a model that's been lied to retries forever. The returned record is keyed by tool name, so what you got is inspectable at the call site.

## How the model references media

Models can't pass bytes; they pass ids. This sounds obvious until you notice most stacks never actually *show* the model an id — the file goes into the context as an opaque content block and the model is left gesturing at "the PDF." Two mechanisms keep ids discoverable here:

1. **Inline markers.** Every LLM battery renders `[media id: 01906c2e… | report.pdf]` immediately before each media content block — attachments and tool results alike. The marker is harness-authored from the harness-controlled `Media.id`, renders outside the untrusted envelope, and carries no authority; it's a structural reference, nothing more.
2. **`list_media`.** The guaranteed fallback: enumerates every media visible in the turn (user attachments and prior tool results) with id, filename, kind, MIME, and origin.

`resolveMedia` turns those ids back into bytes — the default scans the turn's messages and tool calls, and you can supply your own for external media stores. The same resolution serves `@id` refs *inside* statements (`merge with=@…`, `diff with=@…`), so multi-file operations work with ids the model already has. A miss doesn't just say no — it says no *and* enumerates what the model could have asked for, because a bare `MEDIA_NOT_FOUND` invites the model to hallucinate another id and try again:

```text
Error (MEDIA_NOT_FOUND): no media with id "doc-7".
Visible media ids: 01906c2e… (report.pdf), 01906c2f… (data.xlsx)
To create NEW media instead, pass media_id "empty:<format>" (available: xlsx, csv, png, …).
```

That last line only appears when creation is actually possible: `media_id` has one reserved namespace, `empty:<format>`, which creates a brand-new blank file and runs the statement against it — creation and population in one round-trip. The sentinel, the reachability rule behind what's creatable, and the defaults the model is told all live on [Generating Media](./generating); from this page's perspective it's one more way a `media_id` resolves, and every bit of its advertisement (the description section, the schema hint, the `MEDIA_NOT_FOUND` exemplar above) disappears when the deployment can't create anything.

## What comes back

Mutation chains return first-party `Media` — persisted through `ctx.storeMediaBytes`, trust-tiered `first-party`, source-tagged `tool:media_query` — which lands on `ToolCall.results` and renders natively (with its own id marker, so the model can chain operations on the result). `split`/`extract assets` return `Media[]`. Reads (`extract text`, `chunk`, `diff`, `transcribe`) return strings.

Every processing failure — DSL errors included — comes back as a readable `Error (CODE): …` string rather than a thrown tool error, because the model is expected to *repair and retry*: fix the verb it typo'd, quote the sheet name, pick a supported format, or stop calling a verb the deployment doesn't run (`Do not retry this verb here`).

## The gate seam

File mutation is a side effect, and "the model redacted the wrong contract" is not an error message you want to compose for the first time in an incident channel. The [gates page](../../the-loop/gates) is blunt about whose job safety is — yours — and `forgeMediaTools` wires the seam in:

```typescript
const tools = forgeMediaTools(mp, {
  surface: 'composite',
  gate: async (ctx, call) => {
    const verdict = await ctx.waitFor<{ approved: boolean }>({
      reason: 'tool_approval',
      payload: call, // { tool: 'media_query', args: { media_id, q } }
    })
    if (!verdict.approved) throw new Error('operator denied the operation')
  },
})
```

The gate runs before any bytes move; a throw aborts through the standard tool-error path (`E_TOOL_DOWNSTREAM_ERROR` with your denial as `cause`). No gate configured means zero overhead and zero behavior change — which also means that if you skip it, nothing will remind you that you skipped it. The same optional `gate` exists on the [SearXNG](../tools/searxng) and [Scrapper](../tools/scrapper) factories — network-side-effecting tools deserve the same seam.

## Overrides

`overrides` renames or re-describes minted tools, keyed by default name:

```typescript
forgeMediaTools(mp, {
  surface: 'composite',
  overrides: { media_query: { name: 'process_file' } },
})
// → { list_media, process_file }
```