Performance Fix · Context Compaction
You're about to become
the bottleneck again.
Good news: this time, it's not the agent's fault.
A detective story about a silent stall that grew to multiple minutes deep in our longest sessions — and how we killed it.
Wait, What?
Here's the setup — and the payoff
Two words we'll use precisely, because it matters: a turn is one full ask-to-response interaction with you. A step is a single LLM request/response inside that turn — a turn with 10 tool-call round-trips contains 10 steps.
Before
Every single step inside a turn silently stalled before the model ever saw its prompt — seconds early on, climbing past six minutes per step only once a session's accumulated history grew large (roughly 150–200+ steps in). A turn with 10 steps paid that cost 10 separate times. The agent was the slow part of the loop. You waited on it.
After
Each step now finishes in under a second — even the steps that used to stall for minutes deep in a long session. Turns complete measurably faster end-to-end — which shifts the pace of the conversation back toward how fast you can read and respond.
The constraint just flipped. Welcome back to the critical path.
01 · The Mystery
🔍
A session that felt slow, step by step
A long-running Amplifier session had big, unexplained gaps of wall-clock time between LLM calls — each gap sitting inside a single step, not the turn as a whole. And during those gaps, zero events were logged. Not even streaming telemetry. We pointed context-intelligence (graph-based session analysis) at it and dumped every event in a clean gap.
03:04:01.069 tool:pre (todo)
03:04:01.072 tool:post (todo finished — 3ms)
03:04:01.432 provider:request (redaction applied)
03:10:00.564 context:compaction
03:10:00.575 llm:request
This particular gap — nearly six minutes — was pulled from deep in an unusually long-running session, hundreds of steps in. Early on, the same stall was only tens of seconds; more on how it grew, next slide.
Worth sitting with: at that ~6-minute-per-step rate, a single turn needing ~10 tool-call steps would burn roughly an hour of pure stall (10 × ~6 min) — on top of the actual work. Illustrative math, not a directly observed single-turn total: the per-step figure is measured; the 10-step multiplication just shows how it compounds within one turn.
02 · The False Lead
Not flat. Growing.
First guess: a flat ~6-minute stall on every step — maybe rate-limiting or provider throttling. So we mapped the whole session: 165 turns, 1,095 steps (~6.6 steps per turn on average, no chattier-over-time trend). That confirms the bug hit every single step — every raw LLM request — not once per turn. A turn with 10 tool-call round-trips paid this cost 10 separate times.
| Turn (~) | Steps so far (~) | Stall observed | Severity |
| 1–2 | ~12 | 71–73 sec | Significant |
| 17–20 | ~87 | 94–104 sec | Significant |
| 43–50 | ~176 | 120–126 sec | Severe |
| 100 | ~616 | 224–239 sec (~4 min) | Severe |
| 160–164 | ~950+ | 358–389 sec (~6–6.5 min) | Severe |
| 164 (post-fix) | ~1,009+ | 0.60–0.95 sec | Fixed |
Honest caveat: this session had already been resumed 18 times over ~3 weeks and carried 2,285+ messages of accumulated history even at turn 1 — that's why turn 1–2 already shows a real 71–73s stall, not a negligible one. The proven pattern is that impact scales with accumulated context size (message/token volume), not simply turn count starting from zero. A session that stays short in a single sitting carries far less history and sits much further down this curve — the multi-minute stalls above are what a long-lived, frequently-resumed, heavily-used session produces. That's a real and common pattern for serious work, but not every session. Place your own sessions on this curve by message/token count, not turn count.
71s → 368s+. Growth tracks conversation history the entire way — not the clock.
Good News, With Numbers
Your delegated agents were mostly untouched
The analyzed session spawned 267 subsessions — delegates, spawned agents, recipe steps, attractor runs, forked skills: anything that creates a child session. Broken out by context configuration, the numbers reconcile exactly:
264
98.88% — Bounded. depth=none + scope=conversation (179) or depth=recent + scope=conversation (85). Not meaningfully exposed to this cost.
2
0.75% — Partial. depth=all + scope=agents: full turn depth inherited, but the heavy tool-result payloads were excluded — meaningfully less exposed than full risk.
1
0.37% — Full-history-exposed. depth=all + scope=full: the only config that replicates both the parent's full growing history and its tool-result payloads.
264 + 2 + 1 = 267, exactly. This was fundamentally a top-level, long-session problem — not something silently degrading your whole fleet of delegated agents too.
03 · The Real Clue
It tracks history size, not the clock
r = 0.994
correlation with message count
r = 0.993
correlation with token count
~2.4
growth exponent — superlinear, not linear
Ruled out: samples at 05:00, 09:00, 19:00, and 23:00 all sit on the same rising curve — step index predicts the stall, not clock hour. Rate-limit headroom was essentially untouched. No queueing (waiting_requests=0) the entire time.
04 · The Root Cause
Found it — in the real code
Traced to SimpleContextManager, the compaction logic inside the context-simple module. Two separate O(n²) hot loops, both re-scanning the entire message history on every single removal/truncation candidate, instead of once:
def _estimate_tokens(self, messages):
return sum(len(str(msg)) // 4 for msg in messages)
This ran on every single step (each raw LLM request) inside a turn — not once per turn. A turn with 10 tool-call round-trips paid this cost 10 separate times, and as the conversation grew, each of those payments got quadratically worse.
05 · Proof, Not Assertion
A synthetic benchmark, built to check our own math
Predicted
If this is really quadratic, 1.5× more messages should mean 1.5² = 2.25× more time.
Measured
1.5× more messages → 2.26–2.28× more time. Timed out at 30s for just 1,200 messages.
Independent reproduction, matching production telemetry almost exactly.
06 · The Second Opinion
Six lenses, before touching any code
Goal Clarity
Simplicity
Engineering Risk
Momentum & Proof
End-User Impact
Adversarial Testing
The council found a second hidden O(n²) path we'd missed — and that the existing test suite was vacuous: a misconfiguration meant it silently never exercised real compaction at all. The safety net had a hole in it.
Scope: restructure both loops to compute things once per pass. No caching layer. No rewrite. Fix the vacuous tests first, as a blocking prerequisite.
Credit Where It's Due
Who actually solved this
context-intelligence
This bug produced zero error events in raw telemetry — pure silent wall-clock stalling. It would have been invisible without graph-based session correlation. Three investigative dispatches ran 51 graph queries to nail the pattern, then independently re-measured the live production impact after the fix shipped (368s → 0.6–0.95s at matched scale) as real proof, not an assumption.
The council (six-lens review)
Caught a second hidden O(n²) path the initial diagnosis missed, and discovered the existing test suite was silently vacuous — a config mistake meant it never actually exercised real compaction. A hole in the safety net nobody had noticed.
foundation:bug-hunter
TDD-disciplined implementation — red/green regression proof, not just a patch.
foundation:git-ops
Safe, correctly-sequenced shipping: branch, commit, push, PR, merge.
The broader Amplifier ecosystem — evals, the Digital Twin Universe, and tester agents (browser/terminal/reality-check) — exists precisely to prove fixes are real in realistic environments rather than asserted. This particular fix leaned on pytest, a synthetic benchmark, and live production re-measurement instead, in that same "prove it, don't assert it" spirit.
The human's actual role: 10 short steering prompts drove 213 autonomous tool/agent actions — roughly 21 autonomous actions per prompt. The human noticed something felt slow and made a handful of go/verify/ship/share calls — didn't write code, run a test, or craft a single query directly. One nuance worth keeping: the root-cause tracing and fix-design scoping were done by the root agent itself, using systematic-debugging and council skills as its own reasoning scaffold — implementation and shipping were what got delegated to specialist agents (bug-hunter, git-ops).
07 · Fixed Under TDD
Reverted the fix. Watched the bug come back. On purpose.
| 600 msgs | 2,400 msgs | Growth |
| Before (bug) | 0.41s | 6.64s | 16.2× — O(n²) |
| After (fix) | 0.007s | 0.029s | 4.2× — O(n) |
Red/green regression cycle: reverted just the fix, reproduced the original bug live (15.7× slowdown for 4× more messages — the quadratic signature), then restored it and watched it drop back to linear. 37/37 tests passing, clean lint and types.
08 · Shipped
Committed → Pushed → PR'd → Merged
PR: github.com/microsoft/amplifier-module-context-simple/pull/15
Commit: aa8464c
Repo: github.com/microsoft/amplifier-module-context-simple
Merged to main, after the council review and independent verification.
09 · The Payoff
Measured live. In the exact session that surfaced the bug.
| Measurement | Messages | Tokens | Stall |
| Pre-fix baseline | 4,526 | 8.00M | 368.0s |
| Post-pull, pre-restart | 4,536 | 7.95M | 352.3s (module not hot-reloaded) |
| Post-restart (A) | 4,538 | 7.96M | 0.95s |
| Post-restart (B) | 4,540 | 7.96M | 0.60s |
Matched scale — not faster because there was less to do. ~387–615× measured speedup, ~500× average.
Full Circle
So… about that bottleneck.
The agent used to make you wait minutes per turn — and it got worse the longer a session ran. Now turns complete measurably faster, most noticeably in the long-running sessions where that slowdown used to compound the longer you stayed.
The constraint didn't disappear — it moved. It's you now. That's the whole point.
Sources
Research Methodology
Data as of: July 9, 2026
Feature status: Active — merged to main, verified live in production
Research performed:
- context-intelligence graph analysis of the session — event-level gap accounting at the STEP level (one LLM request/response pair) across full turns, using 51 graph queries across three investigative dispatches
- Full-session trend sample across all 1,095 LLM steps spanning 165 turns (~6.6 steps/turn average, no chattier-over-time trend), joined to
context:compaction event payloads (before_messages, before_tokens)
- Source code inspection of
SimpleContextManager in amplifier_module_context_simple
- Synthetic benchmark reproducing the exact hot-loop shape with realistic message sizes
- Six-lens council design review prior to implementation
- TDD implementation plus independent red/green regression verification (fix reverted and restored)
- Post-ship production remeasurement of the same session, same metric, matched conversation scale
- Subsession fan-out analysis across all 267 child sessions spawned in the analyzed session, by context configuration
- Session-level count of human steering prompts vs. autonomous tool/agent actions across the investigation-to-ship arc (10 prompts, 213 actions)
Gaps: None — every figure in this deck traces to a measurement described above; no estimates or projections were used.
Try It
Pull it. Restart it. Feel it.
A git pull alone won't hot-reload an already-imported module — restart the process to pick up the fix. The next compaction on a large session drops from minutes to under a second.
PR: github.com/microsoft/amplifier-module-context-simple/pull/15
Commit: aa8464c
Repo: github.com/microsoft/amplifier-module-context-simple