Amplifier Stories

Recipes That Survived Scale

Patterns from a switched-off tracking bundle

The Setup

A recipe system to read a team's whole history — and it broke at real scale

In January 2026, Mark Licata built the amplifier-bundle-team-tracking bundle to give an Amplifier team shared visibility — syncing sessions to a GitHub repo and running recipes to bulk-analyze them. The first naive recipes broke the moment they hit real scale.

First, the evidence that it really shipped.

The Proof

The bundle really shipped, and worked at bulk scale

Twelve recipe YAML files, led by an 813-line bulk-analyze recipe. Version 2.0 analyzes 100% of sessions while uploading just 1 in 25 — smart sampling over the full set.

A real system at scale — which is exactly where it started to break.

12

recipe YAML files

813

lines in bulk-analyze recipe

100%

of sessions analyzed

1 in 25

sessions uploaded (4%)

The Complication

The naive recipes broke three ways under scale

The same recipes that worked on a few sessions fell apart once they ran across the full session set at bulk scale. Three distinct failure modes surfaced — each one the rest of this deck resolves.

Fix one: stop crashing on files that aren't there.

Crashed on missing files

Absent transcripts and GitHub 404s aborted the run.

Drowned in API calls

Per-session commits triggered GitHub rate limiting.

Choked on large transcripts

Full ~1.2MB transcripts overwhelmed the analysis.

Fix One · Graceful Degradation

Missing files now skip with a warning instead of aborting

The recipe detects absent transcripts, handles GitHub 404s gracefully, and falls back from compressed to uncompressed — emitting "Transcript not found" and continuing rather than crashing the whole run.

Next: stop drowning in GitHub API calls.

1

Detect the absence

if [ ! -f transcript.jsonl ] → warn and continue.

2

Handle the 404

Processing-state fetch treats 404 and other errors gracefully.

3

Fall back

Compressed → uncompressed; skip, don't crash.

Fix Two · Batch Commits

Batch commits cut GitHub API calls by ~85%

Instead of a commit per session, the sync creates all blobs first, then a single tree and commit — dropping calls from ~600 to ~105 per 25 sessions and clearing the rate-limiting that stalled bulk syncs.

Last failure: the transcripts themselves were too big.

~600

API calls before (per 25 sessions)

~105

API calls after

~85%

fewer GitHub API calls

Fix Three · Size Guardrails

Large transcripts were tamed with rate limits and size guards

Full transcripts run ~1.2MB / ~75k tokens. The recipe capped parallel LLM calls at 2 with backoff, reduced batch size from 40 to 25 for safety, and skips any session over 1M tokens.

Three fixes — and the patterns turned out to outlive the tool.

~1.2MB

full transcript (~75k tokens)

40 → 25

batch size, reduced for safety

>1M

token sessions skipped

2

max concurrent LLM calls

The Payoff

The bundle is switched off — but its patterns live on

Active development stopped on 2026-02-02, ending in an Azure migration design proposal. The successor, microsoft/amplifier-session-sync, rebuilt tracking on Cosmos DB with an Opus worker — stating plainly: "We take the ideas, not the implementation."

So what actually survived was never the tool.

Switched off

Last push 2026-02-02; final work is an Azure migration proposal.

Rebuilt elsewhere

Successor PR #17 (merged 2026-03-03): Cosmos DB + Claude Opus 4.6 worker.

Ideas, not implementation

Successor reuses the pre-computed-index idea, not the bash/GitHub code.

The Takeaway

Scale pressure is a teacher

Graceful missing-file handling, batch commits to cut API calls, and transcript-size guardrails were all forged under scale pressure. The tracking system that spawned them is gone — the recipe-engineering patterns are the reusable artifact worth keeping.

The tool was disposable. The patterns weren't.

Sources

Sources & Research Methodology

Data as of: July 21, 2026 (independently re-verified in a fresh session).

Feature status: Superseded / switched off. GitHub-based bundle last push 2026-02-02; successor microsoft/amplifier-session-sync PR #17 merged 2026-03-03.

Primary source: private repo marklicata/amplifier-bundle-team-tracking (created 2026-01-21), re-cloned to /tmp/tt-clone for inspection.

Research performed:

Repo metadata: gh api repos/marklicata/amplifier-bundle-team-tracking
Recipe inventory: ls recipes/*.yaml | wc -l; wc -l recipes/bulk-analyze-sessions.yaml (12 recipes, 813 lines)
Missing-file & size logic: grep -n -i 'missing|not.found|graceful' recipes/bulk-analyze-sessions.yaml
Batch-commit fix: git show eb6fcaa (~85%, ~600→~105 per 25 sessions)
Dashboard speedup & oversized handling: git show f13a8b0; git show c590be5
Successor: git -C amplifier-session-sync show --stat 1cb033a; gh pr view 17 --repo microsoft/amplifier-session-sync
Contributors: git shortlog -sn --all (Mark Licata, 53 commits)

Gaps / estimates: "Hundreds of sessions" is inferred from batch_size 25/run over weekly runs and "100% analyzed," not a verbatim figure. The exact "switched off" date is inferred from the last push (2026-02-02) plus the Azure migration proposal and the successor's "take the ideas, not the implementation" statement.

Primary contributor: Mark Licata (53 of the bundle's commits; also author of successor PR #17).