Read the Wire

Your agent just did something inexplicable — answered a question it clearly had the data for, or looped on a tool it already called, or apologized for a mistake it didn't make. You're about to guess why. Our discipline on this build: don't, not yet. The easy move is to theorize: "the model must not understand X," "it's probably a prompt-wording issue," "small models just aren't reliable at this." All three might be true. None of them earned acting on until we'd looked at the wire — the assembled prompt that actually went out, and the raw generation that actually came back — because in an agent built from inspectable pipelines, the model is the only piece you can't step through. Everything feeding it is your code. Read that first.

The doctrine

The model is the one non-deterministic component in this stack, by construction — that's what sampling from a distribution means, and no amount of temperature-lowering changes what it fundamentally is. Everything around the model is not: the subtractive pass that built this dispatch's context (one model call and its response — the unit a turn is built out of), the parser that read its tool call, the gate cascade that decided whether to ack — all of it is ordinary deterministic code with ordinary bugs, and all of it is inspectable. When a turn goes wrong, the prior for "it's a bug in the deterministic code around the model" is much higher than the prior for "the model is fundamentally incapable," and the deterministic code is the part you can actually fix. So inspect the deterministic parts first. Reading the wire is how you find out which one you're looking at, instead of theorizing about the one part that's rolling dice.

In practice, most "the model is dumb" symptoms turn out to be about the context the model was handed, not about the model:

• The result was in the prompt the whole time. The model produced a correct tool call, got a correct result back, and then failed to use it — not because it can't reason from a tool result, but because the subtractive pass or a prior gate shed the exact turn that made the result legible before the next generation ran.
• The pass shed the thing the model just asked for. A model calls a tool, the result comes back, and the next iteration's context-shedding pass — running its own budget logic with no visibility into what the model just requested — evicts that exact result to make room. The model isn't confused; its own answer was taken away from it between dispatches.
• The contradictory directive that looked like a slow model. A turn that should take one planning dispatch and a handful of tool calls instead grinds through nineteen dispatches. Nothing about the model changed between the 19-dispatch run and the 7-dispatch run that followed the fix — one contradictory instruction was removed from the assembled context. "The model is slow at this task" and "the model is being asked to do two incompatible things" produce the identical external symptom. Only the wire tells them apart.

Re-run triage: fixable vs. irreducible

The fastest way to tell a real bug from ordinary sampling variance is the cheapest experiment available: re-run the identical failing dispatch, same context, several times, and watch what comes back. In plain terms, there are only two outcomes: either the same context produces the same failure every time (a real bug, worth chasing), or it produces a mix of failures and successes (bad luck on the sample, not worth chasing).

• Reproduces identically, every time. The failure isn't the model rolling badly — the context in front of it forces the bad output deterministically. Go find the contradiction, the missing result, the malformed instruction. This is a real, fixable bug, and it will still be there on run six because it isn't luck. A self-handle loop case reproduced identically eight times out of eight — a genuine contradiction was sitting in the context, findable and fixable.
• Scatters — different failures or different successes across runs. This is sampling variance, not a static defect. A leaked-envelope case re-run five times produced one leak and four clean (differently phrased) runs. There is no single line of context to "fix" here; the correct response is to make sure the recovery path handles the bad roll gracefully — which the bounded-gate design in Behavioral Rails already does — and move on. Chasing a deterministic root cause for a genuinely stochastic outcome burns time for nothing.

The rule of thumb: if you can't reproduce it on demand, you can't fix it in the deterministic code, and trying to will just churn the prompt until the symptom happens to go away in whatever sample you're staring at.

Don't cross-reference different runs to build one theory

A theory has to be built from one run's actual inputs and outputs, not from stitching together the interesting parts of several unrelated runs. If run 3 showed the leak and run 5 showed the recovery, that's two separate data points about the distribution of outcomes — not evidence about what happened in run 3. Diagnose one run at a time.

Bisect on the real model, not in your head

Re-run triage tells you whether something is deterministic. Bisection tells you what, when the "what" isn't obvious from staring at one transcript. The move is the same one you'd use on any flaky test: hold everything constant except one variable — an ablation, in the sense of removing exactly one thing and observing the effect — and check whether the failure follows the variable.

The self-apology loop is the clean example. A model that apologizes for a formatting mistake, sees its own apology land in context, and on the next turn apologizes for that, spiraling into an apology about an apology. The suspect variable was "how many prior apologies are sitting in context." Bisected directly on the real model at temperature 1.0, not reasoned about in the abstract: with three accumulated apologies left in context, 3 of 6 runs produced a fourth apology. With the same starting point but the accumulated apologies stripped down to just the newest one, 0 of 6 reproduced — and a follow-up batch of 8 came back clean 8 of 8. That's not a plausible theory about apology accumulation. That's a measured, ablated result, and the fix (strip stale corrective apologies before they can compound) follows directly from it instead of from a guess about what "should" help.

The ordering matters: ablate first, theorize second. A theory formed before the ablation is just a hypothesis wearing the costume of a finding.

Verify the instrumentation before trusting the finding

A subtler trap: the tooling you're using to observe the agent can itself be the thing lying to you, and the false signal it produces looks exactly like a real one.

The concrete case: a dispatch-count metric was backed by a ring buffer capped at the last twelve entries. Every turn, no matter how long it actually ran, reported as exactly twelve iterations — because the buffer silently dropped anything older than the cap, not because every turn coincidentally took twelve dispatches. That number was on the verge of being written up as "systematic over-iteration: every turn burns exactly 12 dispatches," which would have sent the investigation looking for a bug that produces suspiciously uniform behavior — a much harder (and wrong) thing to find, since no such bug existed. Reading the uncapped wire instead showed a healthy, unremarkable spread — 7, 11, 5, 17, 10 — turn to turn. The uniformity was the instrument, not the agent.

If a measurement is suspiciously clean, check whether it's capped, truncated, or sampled before you trust what it seems to be telling you. A cap doesn't announce itself; it just makes the truth invisible past the edge and substitutes something that looks like a stable, wrong, number.

The shipped seams

None of the above requires hand-rolled logging bolted onto a fork of a battery. The LLM batteries ship two observer hooks — callbacks the harness itself calls at a fixed point, you never call them — for exactly this purpose:

• onPromptAssembled, a PromptAssembledObserverFn — fires once per terminal generation, the instant assembly completes and before the engine dispatch, with an observation object: the rendered preface, the per-turn messages, and the rendered tools block (API batteries get a requestBody instead). This is the wire to the model: what the subtractive pass actually kept, in the actual shape the model will see it, not an approximation reconstructed after the fact.
• onRawGeneration, a RawGenerationObserverFn — fires once per terminal generation, after envelope-stripping and reasoning/tool-call parsing but before the result is persisted, with an observation object: the complete rawText, the residual cleanedText, the extracted reasoning, and the extracted toolCalls. This is the wire from the model — the same text the parsers actually saw, plus what they made of it, before gate classification touches any of it.

Both hooks hand you a structured observation, not a bare string, so wiring either to a JSONL dump means picking the field you want off the object. Every LLM battery accepts both hooks as constructor options — here wired on LiteRtLmAdapter:

// A trivial one-line-per-call appender; swap in whatever sink you actually use.
declare function appendJsonl(file: string, entry: unknown): void

const adapter = new LiteRtLmAdapter({
  // ...
  onPromptAssembled: (observation) =>
    appendJsonl('dispatch.jsonl', { at: 'prompt', ...observation }),
  onRawGeneration: (observation) =>
    appendJsonl('dispatch.jsonl', { at: 'generation', ...observation }),
})

That JSONL file is the wire. Re-run triage is grepping it for one dispatch and replaying that exact prompt N times. Bisection is diffing two dispatches and changing one field. Instrumentation-cap checking is asking whether anything upstream of the dump truncated before you saw it. All three techniques on this page operate directly on what these two hooks give you — read it before you theorize.

Where this leads

The findings this doctrine produces don't stay findings — they become the bounded gates and own-voice nudges covered in Behavioral Rails: the false-abstention gate, the fake-citation detector, the parroting self-echo exemption, and the mutual-exclusion rule all trace back to a wire read first and a theory formed second. For the live build these receipts came from, see Punching Above Its Weights — in particular the gate cascade and the shape of the turn, which show the pipeline these hooks tap into.