Performance Fix · Context Compaction

You're about to become
the bottleneck again.

Good news: this time, it's not the agent's fault.

A detective story about a silent stall that grew to multiple minutes deep in our longest sessions — and how we killed it.

Wait, What?

Here's the setup — and the payoff

Two words we'll use precisely, because it matters: a turn is one full ask-to-response interaction with you. A step is a single LLM request/response inside that turn — a turn with 10 tool-call round-trips contains 10 steps.

Before

Every single step inside a turn silently stalled before the model ever saw its prompt — seconds early on, climbing past six minutes per step only once a session's accumulated history grew large (roughly 150–200+ steps in). A turn with 10 steps paid that cost 10 separate times. The agent was the slow part of the loop. You waited on it.

After

Each step now finishes in under a second — even the steps that used to stall for minutes deep in a long session. Turns complete measurably faster end-to-end — which shifts the pace of the conversation back toward how fast you can read and respond.

The constraint just flipped. Welcome back to the critical path.

01 · The Mystery
🔍

A session that felt slow, step by step

A long-running Amplifier session had big, unexplained gaps of wall-clock time between LLM calls — each gap sitting inside a single step, not the turn as a whole. And during those gaps, zero events were logged. Not even streaming telemetry. We pointed context-intelligence (graph-based session analysis) at it and dumped every event in a clean gap.

03:04:01.069 tool:pre (todo) 03:04:01.072 tool:post (todo finished — 3ms) 03:04:01.432 provider:request (redaction applied) ... 5 minutes 59 seconds of ABSOLUTE SILENCE — 0 events ... 03:10:00.564 context:compaction 03:10:00.575 llm:request

This particular gap — nearly six minutes — was pulled from deep in an unusually long-running session, hundreds of steps in. Early on, the same stall was only tens of seconds; more on how it grew, next slide.

Worth sitting with: at that ~6-minute-per-step rate, a single turn needing ~10 tool-call steps would burn roughly an hour of pure stall (10 × ~6 min) — on top of the actual work. Illustrative math, not a directly observed single-turn total: the per-step figure is measured; the 10-step multiplication just shows how it compounds within one turn.

02 · The False Lead

Not flat. Growing.

First guess: a flat ~6-minute stall on every step — maybe rate-limiting or provider throttling. So we mapped the whole session: 165 turns, 1,095 steps (~6.6 steps per turn on average, no chattier-over-time trend). That confirms the bug hit every single step — every raw LLM request — not once per turn. A turn with 10 tool-call round-trips paid this cost 10 separate times.

Turn (~)Steps so far (~)Stall observedSeverity
1–2~1271–73 secSignificant
17–20~8794–104 secSignificant
43–50~176120–126 secSevere
100~616224–239 sec (~4 min)Severe
160–164~950+358–389 sec (~6–6.5 min)Severe
164 (post-fix)~1,009+0.60–0.95 secFixed

Honest caveat: this session had already been resumed 18 times over ~3 weeks and carried 2,285+ messages of accumulated history even at turn 1 — that's why turn 1–2 already shows a real 71–73s stall, not a negligible one. The proven pattern is that impact scales with accumulated context size (message/token volume), not simply turn count starting from zero. A session that stays short in a single sitting carries far less history and sits much further down this curve — the multi-minute stalls above are what a long-lived, frequently-resumed, heavily-used session produces. That's a real and common pattern for serious work, but not every session. Place your own sessions on this curve by message/token count, not turn count.

71s → 368s+. Growth tracks conversation history the entire way — not the clock.

Good News, With Numbers

Your delegated agents were mostly untouched

The analyzed session spawned 267 subsessions — delegates, spawned agents, recipe steps, attractor runs, forked skills: anything that creates a child session. Broken out by context configuration, the numbers reconcile exactly:

264

98.88% — Bounded. depth=none + scope=conversation (179) or depth=recent + scope=conversation (85). Not meaningfully exposed to this cost.

2

0.75% — Partial. depth=all + scope=agents: full turn depth inherited, but the heavy tool-result payloads were excluded — meaningfully less exposed than full risk.

1

0.37% — Full-history-exposed. depth=all + scope=full: the only config that replicates both the parent's full growing history and its tool-result payloads.

264 + 2 + 1 = 267, exactly. This was fundamentally a top-level, long-session problem — not something silently degrading your whole fleet of delegated agents too.

03 · The Real Clue

It tracks history size, not the clock

r = 0.994

correlation with message count

r = 0.993

correlation with token count

~2.4

growth exponent — superlinear, not linear

Ruled out: samples at 05:00, 09:00, 19:00, and 23:00 all sit on the same rising curve — step index predicts the stall, not clock hour. Rate-limit headroom was essentially untouched. No queueing (waiting_requests=0) the entire time.

04 · The Root Cause

Found it — in the real code

Traced to SimpleContextManager, the compaction logic inside the context-simple module. Two separate O(n²) hot loops, both re-scanning the entire message history on every single removal/truncation candidate, instead of once:

# Called once per candidate, on every compaction — i.e. every single STEP def _estimate_tokens(self, messages): return sum(len(str(msg)) // 4 for msg in messages)

This ran on every single step (each raw LLM request) inside a turn — not once per turn. A turn with 10 tool-call round-trips paid this cost 10 separate times, and as the conversation grew, each of those payments got quadratically worse.

05 · Proof, Not Assertion

A synthetic benchmark, built to check our own math

Predicted

If this is really quadratic, 1.5× more messages should mean 1.5² = 2.25× more time.

Measured

1.5× more messages → 2.26–2.28× more time. Timed out at 30s for just 1,200 messages.

Independent reproduction, matching production telemetry almost exactly.

06 · The Second Opinion

Six lenses, before touching any code

Goal Clarity

Simplicity

Engineering Risk

Momentum & Proof

End-User Impact

Adversarial Testing

The council found a second hidden O(n²) path we'd missed — and that the existing test suite was vacuous: a misconfiguration meant it silently never exercised real compaction at all. The safety net had a hole in it.

Scope: restructure both loops to compute things once per pass. No caching layer. No rewrite. Fix the vacuous tests first, as a blocking prerequisite.

Credit Where It's Due

Who actually solved this

context-intelligence

This bug produced zero error events in raw telemetry — pure silent wall-clock stalling. It would have been invisible without graph-based session correlation. Three investigative dispatches ran 51 graph queries to nail the pattern, then independently re-measured the live production impact after the fix shipped (368s → 0.6–0.95s at matched scale) as real proof, not an assumption.

The council (six-lens review)

Caught a second hidden O(n²) path the initial diagnosis missed, and discovered the existing test suite was silently vacuous — a config mistake meant it never actually exercised real compaction. A hole in the safety net nobody had noticed.

foundation:bug-hunter

TDD-disciplined implementation — red/green regression proof, not just a patch.

foundation:git-ops

Safe, correctly-sequenced shipping: branch, commit, push, PR, merge.

The broader Amplifier ecosystem — evals, the Digital Twin Universe, and tester agents (browser/terminal/reality-check) — exists precisely to prove fixes are real in realistic environments rather than asserted. This particular fix leaned on pytest, a synthetic benchmark, and live production re-measurement instead, in that same "prove it, don't assert it" spirit.

The human's actual role: 10 short steering prompts drove 213 autonomous tool/agent actions — roughly 21 autonomous actions per prompt. The human noticed something felt slow and made a handful of go/verify/ship/share calls — didn't write code, run a test, or craft a single query directly. One nuance worth keeping: the root-cause tracing and fix-design scoping were done by the root agent itself, using systematic-debugging and council skills as its own reasoning scaffold — implementation and shipping were what got delegated to specialist agents (bug-hunter, git-ops).

07 · Fixed Under TDD

Reverted the fix. Watched the bug come back. On purpose.

600 msgs2,400 msgsGrowth
Before (bug)0.41s6.64s16.2× — O(n²)
After (fix)0.007s0.029s4.2× — O(n)

Red/green regression cycle: reverted just the fix, reproduced the original bug live (15.7× slowdown for 4× more messages — the quadratic signature), then restored it and watched it drop back to linear. 37/37 tests passing, clean lint and types.

08 · Shipped

Committed → Pushed → PR'd → Merged

PR: github.com/microsoft/amplifier-module-context-simple/pull/15 Commit: aa8464c Repo: github.com/microsoft/amplifier-module-context-simple

Merged to main, after the council review and independent verification.

09 · The Payoff

Measured live. In the exact session that surfaced the bug.

MeasurementMessagesTokensStall
Pre-fix baseline4,5268.00M368.0s
Post-pull, pre-restart4,5367.95M352.3s (module not hot-reloaded)
Post-restart (A)4,5387.96M0.95s
Post-restart (B)4,5407.96M0.60s

Matched scale — not faster because there was less to do. ~387–615× measured speedup, ~500× average.

Full Circle

So… about that bottleneck.

The agent used to make you wait minutes per turn — and it got worse the longer a session ran. Now turns complete measurably faster, most noticeably in the long-running sessions where that slowdown used to compound the longer you stayed.

The constraint didn't disappear — it moved. It's you now. That's the whole point.

Sources

Research Methodology

Data as of: July 9, 2026

Feature status: Active — merged to main, verified live in production

Research performed:

Gaps: None — every figure in this deck traces to a measurement described above; no estimates or projections were used.

Try It

Pull it. Restart it. Feel it.

A git pull alone won't hot-reload an already-imported module — restart the process to pick up the fix. The next compaction on a large session drops from minutes to under a second.

PR: github.com/microsoft/amplifier-module-context-simple/pull/15 Commit: aa8464c Repo: github.com/microsoft/amplifier-module-context-simple