Don't Trust, Restore

Wrapping the LLM in deterministic code

Amplifier · document-generation recipe

The Frame

Preservation is a code guarantee, not a prompt request

When Amplifier's document-generation recipe regenerates a doc, you want the sections that already work left alone. But the model won't reliably leave them alone — so the recipe enforces preservation in Python instead of asking for it.

First, the payoff: does the guarantee actually hold?

AND — keep the working sections untouched.
BUT — the LLM paraphrases and rewrites them anyway.
THEREFORE — restore each one byte-for-byte with code.

The Proof

100% preservation, regardless of LLM behavior

The recipe's v7.4.0 changelog states it "ensures 100% preservation of unchanged sections regardless of LLM behavior." An independent rerun of its restore logic returned a byte-identical section even after the LLM had paraphrased it.

So what exactly is being preserved?

100%

preservation of unchanged sections, per v7.4.0 changelog

85 B

original section, restored byte-identical after paraphrase (independent rerun)

The Ask

Regenerate from an outline, keep the "none" sections exactly

The recipe takes an existing document (via existing_document_path, added in v7.1.0) and classifies each section: none, add, remove, or update. Sections marked action_needed=="none" must come out exactly as they went in.

The obvious way to enforce that? Just ask the model.

none — keep exactly as-is
add — new section
remove — drop it
update — rewrite it

The Tension

"Copy VERBATIM, do not touch them" — and it wasn't enough

The prompt-only fix was actually tried: a sibling recipe, book2/machine/recipes/draft-chapter-v2.yaml, simply tells the model to copy the "keep" sections verbatim. That's exactly the instruction the document-generation recipe stopped trusting.

Why stop trusting it? The recipe says so itself.

"ONLY rewrite the sections marked 'revise'. Copy sections marked 'keep' VERBATIM — do not touch them at all." draft-chapter-v2.yaml, lines 197–198

The Turn

"LLM can't be trusted to skip preserved sections"

The recipe names the root cause in its own v7.4.0 changelog — then pivots. Instead of tuning the prompt harder, it introduces a deterministic Python restore step as the fix. Stop asking the model; start enforcing with code.

Here's the code that does the enforcing.

"DETERMINISTIC SECTION RESTORE: LLM can't be trusted to skip preserved sections." document-generation.yaml — v7.4.0 changelog (2026-01-27)

The Mechanism

After each pass, Python overwrites the section with the original lines

The restore finds each preserved section by its heading and replaces it with the original lines — a verbatim list-slice swap, not a re-render or a summary. The original text wins every time.

And it runs not once, but twice.

# restore preserved section verbatim
lines = lines[:section['start']] \
      + original_lines \
      + lines[section['end']:]

The Payoff

The deterministic sandwich: let the LLM work, then restore ground truth

Two restore steps run after the LLM's passes — restore-preserved-after-content-fix and restore-preserved-after-quality-fix. The model is free to work; then code overwrites every preserved section, so preservation survives whatever it did.

One pattern, generalized.

1LLM content-fix passthen restore-preserved-after-content-fix (line 1879)
2LLM quality-fix passthen restore-preserved-after-quality-fix (line 2286)

The Takeaway

When determinism matters, use code, not LLMs

Wrap the model, let it do what it's good at — then restore the ground truth deterministically. An unreliable model can still produce a reliable result.

Sources

Research Methodology

Active microsoft/amplifier · recipes/document-generation.yaml (branch main) · file version v8.1.0 (2026-01-30)

Data derived from: local checkout of microsoft/amplifier at /home/ramparte/dev/ANext/Inactive/amplifier (git remote confirmed via git remote -v / git branch --show-current).

Commands run:

grep -n 'in_code_block' recipes/document-generation.yaml ; sed -n '100,145p' recipes/document-generation.yaml — v7.4.0 / v7.5.0 changelog
grep -c 'id: "restore-preserved' recipes/document-generation.yaml — 2 restore steps (lines 1879, 2286)
sed -n '1975,2011p' recipes/document-generation.yaml — lines = lines[:start] + original_lines + lines[end:] (line 2000)
python3 /tmp/test_restore.py — independent rerun: RESTORED == original True, byte-identical True (85-byte section)
sed -n '192,206p' book2/machine/recipes/draft-chapter-v2.yaml — prompt-only "copy VERBATIM" (lines 197–198)
grep -n '^# v7\.' recipes/document-generation.yaml — v7.1.0–v7.5.0 all dated 2026-01-27
git show 0cf1d50 --stat — v8.0.0 merge, PR #197, +1309 / -1257, author Mollie Munoz

Gaps: "11 preserved sections / 0 bytes modified" and "100% fidelity" as a captured run result are NOT in the repo — treated as illustrative. What is verified is the recipe's design guarantee (line 114) plus the independent byte-identical reproduction. Per-version v7.x authorship/PR numbers exist only as changelog text; the topic's named repo colombod/code-story was not found.

Primary contributor: Mollie Munoz (mollie.munoz@microsoft.com) — author of the v8.0.0 merge commit (0cf1d50 / PR #197); co-authored by the Amplifier bot.