Skip to content
5 min read · 918 words

Text, Data & Patches

Lossy formats are still media. A txt file can be created, appended to, patched, and diffed; a JSON document can have a property set three levels deep; a YAML config can be merged into. None of that needs an engine, none of it leaves memory, and all of it composes with the generation sentinelempty:json | data set path=… value=… is a complete create-then-populate chain with zero dependencies.

append

text
append text="line two"
append text="b,2" newline=true

Appends to text-family media (txt/md/csv/yaml). newline defaults to true: the appended text is separated from existing content and terminated, so repeated appends build clean lines — empty:csv | append text="a,1" | append text="b,2" yields a two-row CSV.

The data verbs: structural ops on JSON and YAML

text
data set path=config.retries value='3'
data set path=servers[0].host value='"db-1"'
data merge fragment='{"a":{"y":2}}'
data delete path=legacy.flag

data set walks a dot/bracket path, creating missing containers along the way (objects for name segments, arrays for index segments). value is JSON-encoded — '3' is a number, '"three"' is a string, '{"k":1}' is an object; a bare unquoted string is accepted as a convenience because models reach for it constantly. data merge deep-merges an object fragment at the root (strategy=shallow for spread semantics). data delete removes a key or array element and fails readably when the path doesn't exist.

The rule that makes these trustworthy: output format follows input format. data set on a YAML file parses YAML, mutates the value tree, and re-serializes YAML — it does not silently convert your config to JSON because that was easier. The format is the user's; the verb only touches the value.

apply_patch: two dialects, one verb

apply_patch dispatches on the patch's first line. A unified diff — exactly what the diff verb emits — applies to the primary media:

text
diff with=@other          → returns a patch (and structured changes)
apply_patch patch='…'     → applies it

That pairing is a tested contract: diff A with=@B piped back through apply_patch on A reproduces B byte-exact. The model gets a working compare→patch→apply loop with no new schema to learn.

For multi-file work, the structured envelope:

text
*** Begin Patch
*** Add File: docs/new.md
+# Title
*** Update File: src/index.ts
*** Move to: src/main.ts
@@
 context line
-removed line
+added line
*** End Patch

The primary media plus any with=@refs form the virtual workspace (keyed by filename); *** Add File: can grow it, so the result may be multiple media. Context matching is exact and refuses ambiguity — a hunk whose context matches two locations fails rather than guessing — and workspace paths are traversal-guarded (no absolute paths, no ..).

Field note: the dialect the models already speak

The structured envelope isn't homegrown — it derives from the GitHub Copilot apply_patch format, one of the most heavily exercised patch dialects in existence. The whole battery bets on idioms models already know (pipes, key=value); a patch grammar that millions of Copilot sessions have already taught the models is the same bet, placed again. The parser preserves the dialect exactly — inventing a "better" variation would forfeit the pretraining.

That bet was placed live against a 20B model during development. The model knew the envelope structure cold — *** Begin Patch, *** Add File:, *** End Patch — from pretraining alone. What it missed was the + prefix on add-file content lines (the unified-diff convention the parser requires). One sentence in the verb description fixed it. The lesson: the dialect IS in the weights, but the + convention is the one detail that doesn't always come with it — so the verb description now says it explicitly.

redact and update_text, per format

redact replaces matching text (literal strings preferred; /regex/ supported), and what "in place" means depends on the container:

InputWhat happens
text-family (txt/md/csv/json)direct string replacement
DOCX / PPTXin-place XML text-node replacement, run-aggregated — Word splits text across formatting runs, and the matcher aggregates per paragraph so a pattern spanning a bold/regular boundary still matches
ODT / ODS / ODPthe same, over the ODF container's content.xml
PDFvisual redaction only — pdf-lib draws over matching pages and strips document metadata. Content streams keep the original text.

PDF redaction is visual, not content-level

redact on a PDF draws an opaque box over matching pages and strips metadata. The original text is still in the content stream. Anyone who copies text out of the "redacted" PDF, runs pdftotext, or opens it in a parser gets the SSN you thought you removed. This is not a limitation we can engineer away — PDF text is positioned glyphs, not editable paragraphs, and nothing short of re-rendering the document truly removes it.

If you are redacting a PDF for any reason where the hidden text being recoverable is a problem — legal disclosure, PII, anything you'd be embarrassed to leak — do not ship the visually-redacted PDF. Extract the text first (extract text | redact …) and work with the text artifact, or rasterize the pages. The verb description and the tool result both carry this warning so the model sees it too, but the call to treat a visual redaction as "safe to release" is yours, and it's the wrong call.

update_text anchor=… replace=… is the surgical sibling — replace the first occurrence of an anchor string, format-preserving, across text/DOCX/PPTX/ODF. A missing anchor is a readable failure naming the anchor, not a silent no-op. PDF is excluded for the same glyphs-not-paragraphs reason, and the error says so.