Execution State
The executor walks a work queue of frames, breadth-first — every step at the current depth runs before anything deeper, so parallel branches advance together instead of one racing ahead while the other starves. A frame is a node paired with the path that reached it, and it carries its own branch-local value and artifact tables, cloned on fan-out. Two branches cannot read each other's writes. That is not an optimisation; it is the difference between a plan whose result depends on the graph and a plan whose result depends on scheduling luck.
The commit protocol is a contract
Ordering here is not an implementation detail you may reasonably tune. It is the whole reason the fold can tell you anything true about a run that is no longer in memory.
At the run level, everything that can refuse happens before anything irreversible. The plan is read and folded, the entry node located, and RunOptions.input validated against its declared fields — and only then is claimRun called. That order is not cosmetic: a plan admits one run EVER and the store exposes no release, so a check running after the claim would burn the plan's only run on a request that never invoked a single tool. If executePlan throws on invalid input, the plan is still runnable — fix the input and call again.
At the node level, then:
- Before invoking: append
node_enteredand await the durable write. Invocation does not begin until it has committed to storage. - After settling: append
node_settled+ everyedge_taken+ the newfrontier_snapshotas one atomic batch.
Given that order, the fold is unambiguous:
| Log | Meaning |
|---|---|
| entered and settled | Completed. Never re-invoked. |
| entered, not settled | In flight. |
no node_entered | Never started. |
A store that does not commit the batch atomically fails runPlanStoreConformance.
The two honest retractions
We are about to state an opinion
These are places where an earlier draft of this design claimed more than it could deliver. Both claims were withdrawn, and the withdrawal is documented rather than quietly dropped.
foldRun does not detect process death. It cannot. A crashed executor's log is run_started, node_entered — byte-identical to a healthy executor currently inside that call. No fold over events can tell them apart, because the difference is liveness, not history. That log folds to outcome: 'running' with no interruption, and the fold does not guess. Process death reaches the history only when whoever resumes appends run_interrupted{kind:'process_death'}.
The lock is coordination, not mutual exclusion. PlanLockFactory is a TTL lease with no fencing token, so it cannot stop a partitioned or GC-stalled holder continuing past expiry. Supply one and concurrent execution becomes unlikely rather than routine. The residual — a double-invoked node — is handled by per-node onIndeterminate and replaySafe.
Only a call is indeterminate
When a run is interrupted, the immediate question is which work might have half-happened. In this plan graph, the answer is narrow:
branch/selectare pure reads over the persisted table — same inputs, same verdict, which is what the cells' no-clock rule buys.transformis a pure read.joinrestores from the frontier.reasoncosts tokens to repeat but performs no external effect.
So RunProjection.indeterminate contains only call frames, and every other kind is re-entered unconditionally. There is no node kind whose recovery is left unspecified.
Each call declares its own policy, and a resume honours them per node:
{
replaySafe: true, // a FACT about the tool
onIndeterminate: 'retry', // a DECISION about this call: 'retry' | 'halt' | 'skip'
}A join arrival that parks at an open barrier is entered and never settled by design — it contributed its tables and waited. It looks identical to an in-flight frame in the log, and it is deliberately not reported as indeterminate: nothing was half-done.
The first arrival produces no node_settled at all. Only the arrival that closes the barrier settles the join, which is why after runs exactly once no matter how many routes converged — and why a parked arrival must not be counted as work left half-done.
Interruption is classified, not flattened
| Cause | Class | Resume |
|---|---|---|
turn_abort | resumable | Frontier intact, same digest. |
process_death | resumable | From the last committed batch. Not self-reported. |
node_failed{handled: false} | halting | Recovery is clonePlan. |
budget_exhausted | halting | The run hit the plan's own maxSteps. A resume re-reports it — the bound has not changed, so neither has the answer. Recovery is raising maxSteps and cloning. |
join_unsatisfiable | halting | A branch left a route unfired; not a hang. |
node threw with an error edge | not an interruption | The run traverses the edge and may still complete. |
That last row matters. A node that throws where an error edge exists is a handled failure: the failure is recorded, the edge is traversed, and the run's final outcome may be completed with no interruption at all.
Resuming
const interrupted = await orchestration.executePlan('my-plan', { input, signal, invokeCall })
// outcome: 'aborted', interruption: { kind: 'turn_abort' }
// frontier.frames names exactly where it stopped
const finished = await orchestration.executePlan('my-plan', {
input,
resumeRunId: interrupted.runId,
invokeCall,
})
// Only the remaining work runs. Nothing already settled is re-invoked.The frontier is rebuilt from the last frontier_snapshot, then advanced: each later node_settled removes its frame, and each later edge_taken adds its to frame with the outputs and artifacts that event carries. That per-event payload is why a resumed transform can still read an artifact the pre-interruption run produced.
We are about to state an opinion
Registering a durable store's reader resolver is load-bearing for resume, not optional hygiene. Artifacts persist in the log as {tag, locator} handles — pointers, never bytes — and rebinding one needs a live resolver for its tag. In-memory and fetch resolvers auto-register; a durable one is yours to register, because only you hold the binding a serialised locator cannot carry. Without it the fold throws E_NO_READER_RESOLVER naming the tag, rather than quietly returning a half-built projection.