Context That Never Forgets

Graceful degradation for long AI sessions

The Problem

When a long session fills the window, the model silently goes amnesiac

Mechanical truncation chops the oldest messages, so the model loses files, decisions, and tool outputs — then hallucinates to fill the gaps.

amplifier-bundle-context-managed is a drop-in replacement for context-simple that fixes exactly this.

Naive fix: mechanically chop old messages when the context window fills
The model silently loses files, decisions, and tool outputs
Then it hallucinates to fill the gaps it can't see

Proof — It's Real

A shipped, public Microsoft bundle — not a sketch

microsoft/amplifier-bundle-context-managed ships as bundle v0.1.0: two modules, an MIT license, and OSS compliance files, all backed by tests.

So when it claims to replace truncation, that claim has working, tested code behind it.

v0.1.0

Bundle & both modules

2,140

LOC across two modules (1,721 + 419)

364

Test functions

63

Commits, mostly by bkrabach

Why Truncation Fails

Chopping the oldest messages discards exactly what's still needed

The window fills, so the naive approach deletes the earliest turns — but those turns hold files, decisions, and tool results the model still depends on.

The fix isn't to delete less. It's to stop deleting and start compressing.

Files the model opened earlier — gone
Decisions already made — gone
Tool outputs it acted on — gone

Mechanism — The Gradient

The LLM compresses the old and keeps the recent verbatim

Instead of chopping, the bundle uses the LLM itself to build rolling tiered summaries of earlier conversation while a recent window stays full-detail — a deliberate compressed-to-verbatim gradient.

The gradient is intentional by design — but it has to stay inside a budget.

40K

Verbatim recent-window tokens

3

Max summary tiers for older context

1,500

Target tokens per summary

200K

Default max token budget

Mechanism — Budget Cascade

A staged cascade means the message list never exceeds budget

Budget-aware thresholds fire in order as usage climbs, ending in a hard inline-compact cap so the returned messages are always within budget.

Safe under pressure — and, crucially, none of the compressed detail is actually thrown away.

60%

Async summarization beginssummarize_trigger = 0.60

70%

Budget-pressure warningpressure_warning = 0.70

92%

Emergency fallbackemergency_fallback = 0.92

100%

Hard inline-compact capnever returns more tokens than budget

The Turn — Nothing Is Deleted

Everything is persisted to a versioned transcript on disk

The full conversation writes to transcript.jsonl (format version 1.0.0), which reloads on mount so long-running sessions survive process restarts. Large tool results over 50,000 chars are pointer-filed to disk.

So the detail that was summarized away isn't lost — it's still sitting on disk, waiting.

transcript.jsonl — versioned header 1.0.0
Reloads on mount — sessions survive restarts
Tool results over 50,000 chars pointer-filed to disk

Payoff — Recover On Demand

The model pulls the exact verbatim turn back when it needs it

When a detail was summarized away, the bundled read_transcript tool retrieves verbatim turn ranges or searches history — so summaries are hints to verify, not ground truth, and nothing is ever truly gone.

Graceful degradation delivered: the model recovers ground truth instead of hallucinating it.

read_transcript — verbatim turn ranges by start_turn / end_turn
search across full history, with tool inputs and results
Rate-limited to 3 calls per turn

The Pattern To Keep

Treat context as a recoverable gradient, not a fixed window

Compress the old, keep the recent verbatim, persist everything to disk, and recover exact detail on demand. That's how long sessions degrade gracefully instead of going amnesiac.

Sources

Research Methodology

Repository: microsoft/amplifier-bundle-context-managed — created 2026-04-20, most recent commit 2026-05-06. Shipped · v0.1.0

Data as of: repository pushedAt 2026-07-10.

Research performed (git / gh / grep):

Repo identity: gh repo view microsoft/amplifier-bundle-context-managed --json name,description,createdAt,pushedAt,url
Commit history: git log --oneline | wc -l (63 commits) · git log --format='%an' | sort | uniq -c | sort -rn
Thresholds & config defaults: sed -n '190,230p' modules/context-managed/amplifier_module_context_managed/__init__.py
Transcript & large results: grep -n 'TRANSCRIPT_FORMAT_VERSION|transcript.jsonl|_handle_large_result|large_result_threshold' modules/context-managed/.../__init__.py
read_transcript tool: grep -n 'rate_limit_per_turn|read_transcript|start_turn|end_turn|search' modules/tool-transcript/.../__init__.py
Line counts & versions: wc -l modules/*/.../__init__.py (1,721 + 419 = 2,140) · grep version modules/*/pyproject.toml bundle.md
Test suite size: grep -rh 'def test_' tests/ modules/*/tests/ | wc -l (364)

Gaps: The ~50 s summarization window in a code comment is descriptive, not a measured benchmark. Docs (docs/CONFIGURATION.md, bundle.md prose) list a stale 0.80 trigger; the verified code default is 0.60.

Primary contributors: bkrabach (62 of 63 commits), David Koleczek (1).