@nhtio/adk/batteries/media

A knex-inspired local media pipeline: one declarative plan, three front-ends (chainable builder, pipe string, JSON ops), engines as composable seams.

Remarks

Most implementations give you two ways to process media — ship it to an external API, or flood the model's context window with the bytes. This battery is the third option: full control of media — local processing, no external APIs by default, cross-environment where possible. Your data stays in your application and infrastructure unless you deliberately wire an engine that sends it elsewhere.

The entry point is createMediaPipeline:

import { createMediaPipeline } from "@nhtio/adk/batteries/media";

const mp = await createMediaPipeline({
  engines: [
    // an ordered array of self-declaring engines; resolvers run eagerly at construction,
    // but each engine's heavy peer dependency loads lazily on first actual use
    () =>
      import("@nhtio/adk/batteries/media/engines/jimp").then((m) =>
        m.jimpEngine(),
      ),
  ],
});

// chainable builder (implementor DX)
const redacted = await mp(payload)
  .select({ pages: [2, 3] })
  .redact({ match: [/\d{3}-\d{2}-\d{4}/] });

// pipe string (the LLM-facing DSL — same plan underneath)
const result = await mp.query(
  payload,
  "select pages=2-3 | redact match=/\\d{3}-\\d{2}-\\d{4}/",
);

Media pipelines have no standard — every tool invented its own grammar. LLMs don't share one mental model of that chaos, so the pipe DSL projects every operation onto the one transformation idiom every model already groks: shell pipes with key=value args. The verbs a deployment advertises narrow to the engines it configured; everything else fails loud with a model-actionable message.

Classes

Class	Description
MediaChain	The chainable builder. Immutable — every verb returns a new instance sharing the executor. Thenable — await runs the chain and resolves to the terminal step's natural type.

Interfaces

Interface	Description
AudioNamespace	audio.* verbs.
CapabilityProbe	The minimal synchronous capability probe the validator narrows against. The pipeline's engine registry implements it; tests stub it.
CellUpdate	A cell update for sheet.updateCells.
EngineRegistry	Ordered capability-filtered dispatch over the pipeline's engines.
EngineSelectionContext	The request summary a selection stage sees: enough to implement content- and format-dependent rules without exposing dispatch internals.
ImageNamespace	image.* verbs (the runtime fuses adjacent image steps into one engine pass).
MediaOp	The JSON ops front-end's step shape — identical to MediaStep minus the span.
MediaPipeline	The callable pipeline returned by createMediaPipeline: mp(payload) opens a chainable builder; mp.query(payload, q) runs a pipe-string statement; mp.ops(payload, ops) runs a JSON ops array. All three compile to the same plan and execute on the same runtime.
MediaPipelineConfig	Configuration for createMediaPipeline.
MediaPlan	The neutral plan: an ordered, linear list of steps. No branching.
MediaStep	One transform step: a canonical verb id plus its validated content args.
RawArgValue	A parsed-but-unvalidated arg value, before the verb's arg schema refines it.
RawSegment	One parsed segment: verb words plus named args, all position-bearing.
RegExpRef	A serializable regex reference. Never a live RegExp in the IR.
ResizeOptions	Options accepted by image resize.
RunOptions	Per-run options for query/ops.
SheetNamespace	sheet.* verbs, mirrored from the verb table.
SlidesNamespace	slides.* verbs.
SourceSpan	Source span carried by steps parsed from a pipe string, for error mapping.
StepContext	The mutable execution context threaded through interceptors and step impls.
StepPayload	A single in-flight value travelling between steps.
ValidateOptions	Options for validateSegments / validateOps.
VerbArgSpec	Declaration of one named arg on a verb.
VerbSpec	One verb table entry.

Type Aliases

Type Alias	Description
ChainExecutor	The function a builder calls to execute its accumulated ops. Bound by the pipeline.
ChainInput	The input shapes mp(...) accepts.
EngineSelectionMiddlewareFn	A selection-middleware stage — the seam for quality heuristics and implementor overrides when several engines can perform the same transform (e.g. route complex workbooks past a pure-JS converter to LibreOffice). Same onion shape as the battery's use interceptors; a fresh runner is minted per dispatch. Keep stages cheap or memoized: they run with bytes in hand on every dispatch.
FormatFamily	Broad input families used for verb-applicability checks.
MediaArgJson	A structured (JSON-shaped) arg value. In the pipe surface these are written as quoted JSON (updates='[{"address":"B2","value":3}]'); in ops they are plain JSON. null is legal only inside structured values (cell values), never as a top-level scalar.
MediaArgScalar	Scalar arg values the flat pipe grammar can express directly.
MediaArgValue	The value space of a single verb arg in the IR.
MediaChainRef	How other media are referenced from builder verbs: an id string or { mediaId }.
MediaRef	A reference to another media participating in a multi-input verb (merge, diff, apply_patch, slides.update_image).
MediaRefResolver	Resolves a media-ref id to its payload — supplied by the consumer (or the agent forge).
MediaStepMiddlewareFn	Consumer step interceptor — the use seam. Same onion shape as the tool batteries.
PlanResult	The settled result of executing a whole plan.
StepImpl	One verb's implementation. Registered into the runtime's step registry.
StepResult	What a step produces. media continues the chain (or terminates as bytes); media-list and data are terminal-only shapes enforced by the plan compiler.
VerbArgType	The value type of a single declared arg.
VerbOutput	What awaiting a chain ending in this verb resolves to.
VerbRequirement	A verb's capability requirement against the deployment's engine registry. Verbs with no requirement are always advertised; input-conditional engine needs (e.g. extract.text needing OCR only for images) stay runtime checks inside the step implementation.

Variables

Variable	Description
E_INVALID_MEDIA_PIPELINE_CONFIG	Thrown when createMediaPipeline receives an invalid configuration — a malformed engine, a resolver that resolved to a value failing its contract guard, or an invalid option shape.
E_MEDIA_BAD_ARG	Thrown when an arg value fails its declared type/enum/constraint — including descending ranges, invalid regex sources, malformed embedded JSON, and out-of-enum values.
E_MEDIA_ENGINE_REQUIRED	Thrown at plan validation when a verb (or the specific input it is applied to) requires an engine that is not configured on this pipeline. The message states which engine slot is missing and — when the failure is input-specific — that the verb should not be retried on this media in this deployment.
E_MEDIA_MISSING_ARG	Thrown when a verb's required arg is absent from the statement.
E_MEDIA_NOT_PIPE_EXPRESSIBLE	Thrown by toPipe() when the plan contains constructs the flat pipe grammar cannot express — currently only nested-builder media refs. Quoted-JSON structured args DO render; use toOps() for a total serialization.
E_MEDIA_PIPE_SYNTAX	Thrown by parsePipe when the pipe expression fails to tokenize or parse. The message carries line/col position and a corrective exemplar.
E_MEDIA_STEP_FAILED	Thrown by the step runtime when a step implementation fails mid-execution (corrupt input, engine error). Carries the underlying error as cause.
E_MEDIA_STEP_UNAVAILABLE	Thrown by the step runtime when a plan step has no registered implementation. Distinct from E_MEDIA_ENGINE_REQUIRED: this indicates the battery itself does not (yet) implement the verb, not that the consumer omitted an engine.
E_MEDIA_UNKNOWN_ARG	Thrown when a statement passes an arg a verb does not declare. The message includes a did-you-mean suggestion and the verb's declared args.
E_MEDIA_UNKNOWN_VERB	Thrown when a statement names a verb that does not exist in the verb table. The message includes a did-you-mean suggestion (Levenshtein over folded verb forms, including suffix-word matches) and the list of verbs available in this deployment.
E_MEDIA_UNSUPPORTED_OP	Thrown at plan validation when a verb cannot apply to the input's format family (e.g. a sheet.* verb on a PDF), or a namespace verb is used on the wrong media kind.
EXT_TO_MIME	A small extension → MIME lookup for filenames produced by transforms.
FOLDED_VERBS	All folded verb word-sequences, for the parser's longest-match verb recognition and for generated grammar text.
MIME	Well-known MIME constants used across the battery.
VERB_INDEX	Map of canonical verb id → spec, for direct lookup.
VERBS	The canonical verb table. Order is presentation order in generated grammar text.

Functions

Function	Description
availableVerbs	The folded verb forms available under a given capability configuration.
buildEngineRegistry	Build the registry over a resolved, validated engine array.
createMediaPipeline	Create a media pipeline.
familyOf	Classify a MIME type into the verb table's broad format family.
fromOps	Build a MediaPlan from a JSON ops array. The inverse of toOps.
isMediaRef	true when value is a MediaRef.
isRegExpRef	true when value is a RegExpRef.
lowerSegments	Lower raw segments to an (unvalidated) MediaPlan. Quoted-JSON arg parsing and type/enum checks happen in the validator, which consumes the raw segments — this lowering exists for tooling that wants the structural plan without validation.
parsePipeRaw	Parse a pipe expression into raw, position-bearing segments.
replaceExtension	Replace (or add) a filename's extension.
suggestVerbs	Suggest the nearest verbs to an unknown input, for did-you-mean errors. Matches whole folded forms AND suffix words (resize suggests image resize), per the frozen error model.
toOps	Render a MediaPlan to its JSON ops array. Total — every plan has an ops form.
toPipe	Render a MediaPlan to its canonical pipe string.
unsupportedForMutationReason	Reason a mutation verb cannot run against this MIME type, or undefined when mutation is allowed. Returns a reason string (not a throw) so callers compose model-actionable failures.
validateOps	Validate a JSON ops array into a MediaPlan. The same checks as the pipe path — verbs fold the same way, args validate against the same specs — so the two front-ends produce identical plans for equivalent statements.
validateSegments	Validate raw pipe segments into a MediaPlan.

References

AsrConvertOptions

Re-exports AsrConvertOptions

---

bytesToPcm

Re-exports bytesToPcm

---

ConvertCapability

Re-exports ConvertCapability

---

ConvertOptions

Re-exports ConvertOptions

---

ConvertOutput

Re-exports ConvertOutput

---

ConvertRequest

Re-exports ConvertRequest

---

ConvertResult

Re-exports ConvertResult

---

EditCapability

Re-exports EditCapability

---

EditRequest

Re-exports EditRequest

---

EditResult

Re-exports EditResult

---

EditSummary

Re-exports EditSummary

---

EMPTY_MIME

Re-exports EMPTY_MIME

---

EngineResolver

Re-exports EngineResolver

---

foldVerb

Re-exports foldVerb

---

ImagesConvertOptions

Re-exports ImagesConvertOptions

---

implementsMediaEngine

Re-exports implementsMediaEngine

---

MediaEngine

Re-exports MediaEngine

---

MimePattern

Re-exports MimePattern

---

MutateCapability

Re-exports MutateCapability

---

MutateRequest

Re-exports MutateRequest

---

OcrConvertOptions

Re-exports OcrConvertOptions

---

PCM_MIME

Re-exports PCM_MIME

---

pcmToBytes

Re-exports pcmToBytes