--- url: 'https://adk.nht.io/assembly/byo-storage.md' description: >- Wire the 27 required storage/context callbacks — the complete persistence contract for ADK. --- # Bring your own storage ## LLM summary — Bring your own storage * ADK does not persist anything. No default in-memory store, no hidden database. * 27 storage/context callbacks plus the required `executorCallback` are strictly required at [`TurnRunner`](https://adk.nht.io/api/@nhtio/adk/turn_runner/classes/TurnRunner) construction. Missing even one throws a validation error immediately. * The runtime validator enforces parameter count: fetch callbacks require `.arity(1)` (exactly 1 parameter: `ctx`), while store, mutate, and delete callbacks require `.arity(2)` (exactly 2 parameters: `ctx, value` or `ctx, id`). Failing to declare these parameters in your lambdas will crash your setup. * The 6 persisted primitives are: [`Message`](https://adk.nht.io/api/@nhtio/adk/common/classes/Message), [`Memory`](https://adk.nht.io/api/@nhtio/adk/common/classes/Memory), [`Thought`](https://adk.nht.io/api/@nhtio/adk/common/classes/Thought), [`ToolCall`](https://adk.nht.io/api/@nhtio/adk/forge/classes/ToolCall), [`Retrievable`](https://adk.nht.io/api/@nhtio/adk/common/classes/Retrievable), and standing instructions (`string | Tokenizable`). There is no `StandingInstruction` class. * Every callback can be a legal no-op, but they must still satisfy the arity constraint. * Which callbacks are safe to noop: [`Memory`](https://adk.nht.io/api/@nhtio/adk/common/classes/Memory), [`Thought`](https://adk.nht.io/api/@nhtio/adk/common/classes/Thought), [`Retrievable`](https://adk.nht.io/api/@nhtio/adk/common/classes/Retrievable), and standing instructions can be nooped without breaking core execution loops. * Which callbacks are NOT safe to noop: [`Message`](https://adk.nht.io/api/@nhtio/adk/common/classes/Message) (noop equals agent amnesia) and [`ToolCall`](https://adk.nht.io/api/@nhtio/adk/forge/classes/ToolCall) (noop breaks tool-using execution chains). * `fetchToolsCallback` is a required [`TurnRunnerConfig`](https://adk.nht.io/api/@nhtio/adk/turn_runner/interfaces/TurnRunnerConfig) context callback, not a storage callback. It returns `Tool[] | Promise`. Like all fetch callbacks, ADK does not call this automatically; your input pipeline middleware, dispatch middleware, or executor must invoke it. * Store/mutate callbacks fire when your executor explicitly calls `ctx.storeMessage()` or `ctx.mutateToolCall()`. ADK does not auto-call mutate on tool completion; your executor (or your LLM battery) must handle this. * There is NO 27-callback storage battery in `@nhtio/adk/batteries/storage/in_memory`. That battery exports only `InMemorySpoolReader` and `InMemorySpoolStore`, which are exclusively for `SpooledArtifact` byte-spooling, not the 27-callback core contract. ADK does not persist anything. No default in-memory store, no hidden database, no polite little cache waiting behind the curtain. You provide the storage layer, or there is no storage layer. ADK validates, routes, and orchestrates. It does not touch a database, write to disk, or cache in memory. When the executor writes a message via `ctx.storeMessage()`, ADK calls the callback you wired. Your callback does the write, and your storage system holds the state. If you fail to wire all 27 required storage/context callbacks plus the required `executorCallback`, the runner throws an error during construction. The assembly is incomplete, and execution does not start. ## The 27 Callbacks All 27 storage/context callbacks are properties of the [`TurnRunnerConfig`](https://adk.nht.io/api/@nhtio/adk/turn_runner/interfaces/TurnRunnerConfig) object. The runtime schema validates that they exist and enforces strict arity requirements: * **Retrieval callbacks** require exactly **1 parameter** (`ctx`). * **Persistence callbacks (store/mutate/delete)** require exactly **2 parameters** (`ctx, target`). * **Byte-persistence conduits** require exactly **3 parameters** (`ctx, id, bytes`). If you pass an arity-0 arrow function (like `async () => []`), the schema validator will reject your configuration and throw. The runtime validator counts your callback's `function.length`. Yes, that's how strict it is. No, you can't use a wrapper that hides the parameters. ### Retrieval Callbacks (7) These functions are exposed on the turn context. ADK does **not** call them automatically. When a turn starts, `ctx.turnMemories`, `ctx.turnRetrievables`, `ctx.turnMessages`, `ctx.turnThoughts`, and `ctx.turnToolCalls` start empty; `ctx.standingInstructions` is seeded from [`RawTurnContext`](https://adk.nht.io/api/@nhtio/adk/types/interfaces/RawTurnContext), and `ctx.tools` is a `ToolRegistry` seeded from [`TurnRunnerConfig`](https://adk.nht.io/api/@nhtio/adk/turn_runner/interfaces/TurnRunnerConfig). Your `turnInputPipeline` middleware should, if you want those records in the turn context before execution, call these retrieval methods and manually `.add()` each returned item into its corresponding context Set. Executors or dispatch middleware may also call them. If your pipeline omits this, the context remains empty, and the executor runs without history. | Callback | Signature | Available on context as | Returns | | :--- | :--- | :--- | :--- | | `fetchMemoriesCallback` | `(ctx) => ...` | `ctx.fetchMemories()` | `Memory[]` | | `fetchMessagesCallback` | `(ctx) => ...` | `ctx.fetchMessages()` | `Message[]` | | `fetchThoughtsCallback` | `(ctx) => ...` | `ctx.fetchThoughts()` | `Thought[]` | | `fetchToolCallsCallback` | `(ctx) => ...` | `ctx.fetchToolCalls()` | `ToolCall[]` | | `fetchToolsCallback` | `(ctx) => ...` | `ctx.fetchTools()` | `Tool[]` | | `fetchRetrievablesCallback` | `(ctx) => ...` | `ctx.fetchRetrievables()` | `Retrievable[]` | | `refreshStandingInstructionsCallback` | `(ctx) => ...` | `ctx.refreshStandingInstructions()` | `(string \| Tokenizable)[] \| Promise<(string \| Tokenizable)[]>` | `fetchToolsCallback` is required by [`TurnRunnerConfig`](https://adk.nht.io/api/@nhtio/adk/turn_runner/interfaces/TurnRunnerConfig) and maps to `ctx.fetchTools()`, but it is not a storage callback and does not correspond to one of the six persisted primitives. ### Persistence Callbacks (18) Three operations for each of the six primitives. All of these require exactly 2 parameters. | Primitive | Store `(ctx, val)` | Mutate `(ctx, updated)` | Delete `(ctx, key)` | | :--- | :--- | :--- | :--- | | Messages | `storeMessageCallback` | `mutateMessageCallback` | `deleteMessageCallback` | | Memories | `storeMemoryCallback` | `mutateMemoryCallback` | `deleteMemoryCallback` | | Thoughts | `storeThoughtCallback` | `mutateThoughtCallback` | `deleteThoughtCallback` | | ToolCalls | `storeToolCallCallback` | `mutateToolCallCallback` | `deleteToolCallCallback` | | Retrievables | `storeRetrievableCallback` | `mutateRetrievableCallback` | `deleteRetrievableCallback` | | Standing Instructions | `storeStandingInstructionCallback` | `mutateStandingInstructionCallback` | `deleteStandingInstructionCallback` | ::: danger Complete your delete signatures Even if your application never deletes records, your delete callbacks must declare both parameters to pass validation: ```typescript deleteMemoryCallback: async (_ctx, _id) => {} ``` ::: ### Byte-Persistence Conduits (2) Two conduits route arbitrary bytes into your storage and hand back a reader. Unlike the persistence callbacks above, these **return a value** (a reader) and are **not** mutations — they do not add to the turn Sets or fire `stored*` hooks. They require exactly 3 parameters (`ctx, id, bytes`), where `bytes` is `string | Uint8Array | ReadableStream`. | Callback | Signature | Available on context as | Returns | | :--- | :--- | :--- | :--- | | `storeMediaBytesCallback` | `(ctx, id, bytes) => ...` | `ctx.storeMediaBytes(id, bytes)` | `MediaReader` | | `storeRetrievableBytesCallback` | `(ctx, id, bytes) => ...` | `ctx.storeRetrievableBytes(id, bytes)` | `SpoolReader` | A tool handler that produces media persists the bytes through `ctx.storeMediaBytes(...)`, then builds a [`Media`](https://adk.nht.io/api/@nhtio/adk/common/classes/Media) from the returned reader (`Media.toolGenerated({ reader })`) and stores the owning primitive ([`Message`](https://adk.nht.io/api/@nhtio/adk/common/classes/Message) attachment or [`ToolCall`](https://adk.nht.io/api/@nhtio/adk/forge/classes/ToolCall) result) via the normal `store*` path. `ctx.storeRetrievableBytes(...)` is the same pattern for large extracted RAG text: persist the bytes, wrap the returned [`SpoolReader`](https://adk.nht.io/api/@nhtio/adk/spooled_artifact/interfaces/SpoolReader) in a [`SpooledArtifact`](https://adk.nht.io/api/@nhtio/adk/spooled_artifact/classes/SpooledArtifact), and pass it as `Retrievable.content`. **Persisting bytes alone is not enough** — the framework never sees the media or retrievable until you store the owning primitive. A `ByteStore` battery (`InMemorySpoolStore`, `OpfsSpoolStore`, a Flydrive-backed store) is the natural backing for both conduits: `storeRetrievableBytes` returns its [`SpoolReader`](https://adk.nht.io/api/@nhtio/adk/spooled_artifact/interfaces/SpoolReader) directly; `storeMediaBytes` can wrap stored bytes in an in-memory or store-backed [`MediaReader`](https://adk.nht.io/api/@nhtio/adk/common/interfaces/MediaReader). ## The Six Primitives ### Messages User and assistant conversation entries are [`Message`](https://adk.nht.io/api/@nhtio/adk/common/classes/Message); tool results live on [`ToolCall`](https://adk.nht.io/api/@nhtio/adk/forge/classes/ToolCall).`results`. If you noop these, your agent has total amnesia. It cannot continue conversations or accumulate context across iterations. ### Memories Durable facts accumulated across turns (e.g., user preferences or long-term constraints). Noop [`Memory`](https://adk.nht.io/api/@nhtio/adk/common/classes/Memory) callbacks if your agent does not need cross-turn memory. If you implement them, your executor or your output middleware controls when they get written via `ctx.storeMemory()`. ### Thoughts Reasoning traces produced by extended-thinking models. Stored for debugging and auditing. You can safely noop [`Thought`](https://adk.nht.io/api/@nhtio/adk/common/classes/Thought) callbacks if you do not expose or audit the agent's internal reasoning. ### ToolCalls Records of tool invocations: name, arguments, and execution results. If your agent uses tools, implement [`ToolCall`](https://adk.nht.io/api/@nhtio/adk/forge/classes/ToolCall) persistence. Nooping these breaks tool execution tracking; the model will struggle to determine what ran or what failed. ADK does not call mutate automatically. If your executor registers a pending tool call and then updates it with execution output, the executor (or the LLM adapter battery) must explicitly call `ctx.mutateToolCall()`. ### Retrievables Knowledge chunks injected via RAG. Safe to noop unless you are implementing retrieval pipelines that persist or retrieve context blocks. See [Bring your own retrieval](./byo-retrieval). ### Standing Instructions Operator instructions or tenant-level system prompts. These are raw `string` or `Tokenizable` instances—there is no `StandingInstruction` class. The delete signature is unique because it accepts the instruction value itself rather than an ID: `(ctx, value: string | Tokenizable)`. ## Store, Mutate, Delete Semantics These three operations have distinct execution boundaries: * **Store** creates a new record. On a [`TurnContext`](https://adk.nht.io/api/@nhtio/adk/types/interfaces/TurnContext), calling `ctx.storeMessage(message)` or another `ctx.store*` method delegates directly to the callback you wired. On a dispatch context, `ctx.store*` updates the dispatch-local Sets immediately, then the callback work is queued and flushed to the parent turn through the dispatch runner only after a successful iteration/ack. * **Mutate** updates state. When your executor or adapter explicitly calls `ctx.mutateToolCall(updatedToolCall)` or another `ctx.mutate*` method, ADK passes exactly that updated primitive or standing-instruction value to your callback. ADK does not enforce a patch format or diffing strategy; any patching or merging is adapter-local before calling the context method. * **Delete** removes records. The delete callback receives the identifier (or the value itself, in the case of standing instructions) and must invalidate the record. ## The No-Op Contract Every callback can be a legal no-op as long as it respects the arity constraints: * Retrieval callbacks: `async (_ctx) => []` * Store/Mutate/Delete callbacks: `async (_ctx, _valOrId) => {}` ::: danger No-ops must be explicit You cannot omit a callback from the configuration object. The runtime validator checks for the existence of all 27 keys and verifies their parameter lengths. If any key is missing or has incorrect arity, initialization fails. ::: ::: tip Minimum Viable Setup for Prototyping For your first working agent, implement all [`Message`](https://adk.nht.io/api/@nhtio/adk/common/classes/Message) and [`ToolCall`](https://adk.nht.io/api/@nhtio/adk/forge/classes/ToolCall) callbacks (fetch/store/mutate/delete). Noop the other 17 callbacks using the pattern below. This gives you an agent that tracks conversations and tools. ::: ## The Storage Adapter Pattern While inline lambdas are legal, they quickly become unreadable. Group all 27 callbacks into a single object literal—your storage adapter—and spread it into your configuration. We recommend exporting a base noop adapter and overriding only what your application requires: ```typescript import { TurnRunner } from '@nhtio/adk' import { noopStorageAdapter } from './noop-storage' import { myMessagesAdapter } from './messages-adapter' const runner = new TurnRunner({ executorCallback: myExecutor, // 1. Establish the 27-callback baseline ...noopStorageAdapter, // 2. Override specific components fetchMessagesCallback: myMessagesAdapter.fetch, storeMessageCallback: myMessagesAdapter.store, mutateMessageCallback: myMessagesAdapter.mutate, deleteMessageCallback: myMessagesAdapter.delete, }) ``` ## The Noop Reference Adapter This is the canonical baseline. Maintain this file in your codebase to satisfy testing and partial implementation requirements. ```ts // The 27-callback no-op storage adapter. // // Spread this into TurnRunnerConfig as a baseline, then override only the // callbacks you actually want to do work. The runtime validator requires the // declared arity (1 for fetch, 2 for store/mutate/delete, 3 for the byte // conduits); a wrong-arity callback throws E_INVALID_TURN_RUNNER_CONFIG at // construction. import { isInstanceOf, inMemoryMediaReader } from '@nhtio/adk' import { InMemorySpoolStore } from '@nhtio/adk/batteries/storage/in_memory' import type { MemoryRetrievalFn, MessageRetrievalFn, ThoughtRetrievalFn, ToolCallRetrievalFn, ToolsRetrievalFn, RetrievableRetrievalFn, StandingInstructionsRefreshFn, MemoryStoreFn, MemoryMutateFn, MemoryDeleteFn, MessageStoreFn, MessageMutateFn, MessageDeleteFn, ThoughtStoreFn, ThoughtMutateFn, ThoughtDeleteFn, ToolCallStoreFn, ToolCallMutateFn, ToolCallDeleteFn, RetrievableStoreFn, RetrievableMutateFn, RetrievableDeleteFn, StandingInstructionStoreFn, StandingInstructionMutateFn, StandingInstructionDeleteFn, MediaBytesStoreFn, RetrievableBytesStoreFn, ConduitBytes, } from '@nhtio/adk' // The byte-persistence conduits RETURN a reader — they are not void no-ops. // The minimal honest implementation drains the bytes into an in-memory // store/reader; swap these for your own ByteStore in production. // // ONE store instance, created once and reused across every call. Constructing a // `new InMemorySpoolStore()` *inside* the callback would give each write its own // throwaway store: the returned reader works for that call, but the bytes are not // retained under any coherent store identity, so nothing written under one id is // retrievable later. That is fake persistence that still passes validation — // exactly the anti-pattern the byte-conduit contract exists to prevent. A real // adapter swaps this for a durable `ByteStore` (e.g. `OpfsSpoolStore`), but it // must still be a single, stable backing store, not one per call. const retrievableSpool = new InMemorySpoolStore() const conduitToBytes = async (bytes: ConduitBytes): Promise => { if (typeof bytes === 'string') return new TextEncoder().encode(bytes) if (!isInstanceOf(bytes, 'ReadableStream', ReadableStream)) return bytes const reader = bytes.getReader() const chunks: Uint8Array[] = [] for (;;) { const { done, value } = await reader.read() if (done) break if (value) chunks.push(value) } const total = chunks.reduce((n, c) => n + c.byteLength, 0) const out = new Uint8Array(total) let offset = 0 for (const c of chunks) { out.set(c, offset) offset += c.byteLength } return out } export const noopStorageAdapter = { // Memories fetchMemoriesCallback: (async (_ctx) => []) as MemoryRetrievalFn, storeMemoryCallback: (async (_ctx, _m) => {}) as MemoryStoreFn, mutateMemoryCallback: (async (_ctx, _m) => {}) as MemoryMutateFn, deleteMemoryCallback: (async (_ctx, _id) => {}) as MemoryDeleteFn, // Messages fetchMessagesCallback: (async (_ctx) => []) as MessageRetrievalFn, storeMessageCallback: (async (_ctx, _m) => {}) as MessageStoreFn, mutateMessageCallback: (async (_ctx, _m) => {}) as MessageMutateFn, deleteMessageCallback: (async (_ctx, _id) => {}) as MessageDeleteFn, // Thoughts fetchThoughtsCallback: (async (_ctx) => []) as ThoughtRetrievalFn, storeThoughtCallback: (async (_ctx, _t) => {}) as ThoughtStoreFn, mutateThoughtCallback: (async (_ctx, _t) => {}) as ThoughtMutateFn, deleteThoughtCallback: (async (_ctx, _id) => {}) as ThoughtDeleteFn, // ToolCalls fetchToolCallsCallback: (async (_ctx) => []) as ToolCallRetrievalFn, storeToolCallCallback: (async (_ctx, _tc) => {}) as ToolCallStoreFn, mutateToolCallCallback: (async (_ctx, _tc) => {}) as ToolCallMutateFn, deleteToolCallCallback: (async (_ctx, _id) => {}) as ToolCallDeleteFn, // Tools (supplementary tools the model can see, fetched per turn) fetchToolsCallback: (async (_ctx) => []) as ToolsRetrievalFn, // Retrievables fetchRetrievablesCallback: (async (_ctx) => []) as RetrievableRetrievalFn, storeRetrievableCallback: (async (_ctx, _r) => {}) as RetrievableStoreFn, mutateRetrievableCallback: (async (_ctx, _r) => {}) as RetrievableMutateFn, deleteRetrievableCallback: (async (_ctx, _id) => {}) as RetrievableDeleteFn, // Standing instructions (string | Tokenizable — no class primitive) refreshStandingInstructionsCallback: (async (_ctx) => []) as StandingInstructionsRefreshFn, storeStandingInstructionCallback: (async (_ctx, _v) => {}) as StandingInstructionStoreFn, mutateStandingInstructionCallback: (async (_ctx, _v) => {}) as StandingInstructionMutateFn, deleteStandingInstructionCallback: (async (_ctx, _v) => {}) as StandingInstructionDeleteFn, // Byte-persistence conduits (arity 3 — return a reader) storeMediaBytesCallback: (async (_ctx, _id, bytes) => inMemoryMediaReader(await conduitToBytes(bytes))) as MediaBytesStoreFn, storeRetrievableBytesCallback: ((_ctx, id, bytes) => retrievableSpool.write(id, bytes)) as RetrievableBytesStoreFn, } ``` ## The Batteries Alternative The `@nhtio/adk/batteries/storage` package does **not** provide a 27-callback storage adapter. The storage batteries, including `InMemorySpoolStore` and deep-imported `OpfsSpoolStore`, are exclusively for persisting artifact bytes (`SpooledArtifact`), not the structured core primitives. For structured database storage, write the adapter yourself using the 27-callback contract. ## Callback Timing Reference | Operation Type | Trigger | | :--- | :--- | | `fetch*` Callbacks | **Never called automatically by ADK.** Your `turnInputPipeline` middleware should call them (e.g., `ctx.fetchMessages()`) and manually insert the returned items into the context sets if you want those records available before execution. | | `store*` Callbacks | Fired instantly when the executor calls `ctx.storeMessage(message)` or equivalent methods on a turn context; dispatch-context store calls update local Sets immediately and flush through the dispatch runner only after successful iteration/ack. | | `mutate*` Callbacks | Fired when the executor or adapter explicitly calls `ctx.mutateMessage(updatedMessage)`, `ctx.mutateToolCall(updatedToolCall)`, or equivalent methods. | | `delete*` Callbacks | Fired on explicit deletion triggers from your execution block. | ## Persisting Media [`Media`](https://adk.nht.io/api/@nhtio/adk/common/classes/Media) is a pointer, not a byte array. It wraps a streaming source, which does not survive JSON serialization. When persisting [`Message`](https://adk.nht.io/api/@nhtio/adk/common/classes/Message) attachments or [`ToolCall`](https://adk.nht.io/api/@nhtio/adk/forge/classes/ToolCall) results: 1. **Serialize Metadata Only:** `Media.toJSON()` serializes metadata such as id, kind, MIME type, filename, source, trust tier, modality hazard, and stash. Store any hashes yourself. When restoring, your fetch callbacks must reconstruct the `Media` instance and its reader. 2. **Handle Byte Durability:** If the media source is transient, you must drain `media.stream()` during the store callback and write the bytes to your own asset store. If the source is a permanent URI, you can safely store the URI string and re-instantiate the pointer during fetch. ## Testing Your Storage Implementation Verify your callbacks individually before mounting them to a runner: 1. Call your store callback with a mocked primitive (e.g., `new Message(...)`). Confirm it inserts cleanly into your database. 2. Call your mutate callback with the actual updated primitive or standing-instruction value. Confirm the target record updates without dropping adjacent fields. 3. Call your fetch callback. Assert that the returned array matches the schema expectations and contains the records you inserted.