Give the AI Eyes

Visual testing for terminal UIs

The Problem

An AI that builds a terminal UI is effectively blind

It can spawn the program and prove its logic with unit tests — but it never sees the rendered screen. The pixels are invisible to it.

So what does "shipping" even look like when you can't watch the thing run?

It Ships Today

terminal-tester is a real bundle, not a sketch

One tui_terminal tool and one tui-test-analyst agent, backed by a passing test suite. It exists and it runs.

Shipped and verified — now here is the gap it was built to close.

1

tool — tui_terminal

1

agent — tui-test-analyst

167

unit tests passing

Why Tests Aren't Enough

Unit tests pass while the pixels lie

Logic assertions confirm the code is correct. They cannot confirm the screen is right — a whole class of visual bugs never shows up in an assertion.

To catch these you need a real screen to break — so here's one.

Truncation — text cut off past the edge
Misalignment — columns and boxes out of place
Resize breakage — layouts that shatter when the terminal changes size

The Proving Ground

The target is amplifier-tui, a real screen to break

amplifier-tui is a Textual-based TUI proof-of-concept — an actual terminal interface with real layout, launched with uv run amplifier-tui run. Concrete enough to have visual bugs worth catching.

Now, how does terminal-tester earn its eyes against a real program?

Mechanism · Drive It

It runs the real TUI in a genuine pseudo-terminal

tui_terminal spawns the actual program in a PTY (pty.fork + /bin/sh -c) and drives it with true keystrokes — not a mock, the real thing responding.

Driving the program is half the trick — the other half is seeing it.

Arrows — UP mapped to real byte sequences
Tab & Ctrl combos — e.g. CTRL+C
F1–F12 — full function-key range
Live resize — TIOCSWINSZ + SIGWINCH re-render

Mechanism · See It

Every frame becomes a rendered PNG screenshot

A capture returns the frame three ways — text, ANSI, and an image_path to a PNG rendered from the terminal buffer. The invisible screen turns into something an AI can look at.

Now the agent has an image — so who does the looking?

pyte — terminal emulation of the buffer
Pillow — renders the buffer to a PNG
Full color — xterm-256, true-color hex RGB, bold

The Payoff

The agent closes its own fix-verify loop

The tui-test-analyst agent, backed by a design-intelligence vision dependency, reads the screenshots against a visual checklist. The loop closes without a human QA in the seat.

Eyes plus judgment — that's the whole idea, and it generalizes.

1

Capture before — screenshot the starting state

2

Act — drive it with real keystrokes

3

Capture after — screenshot the result

4

Compare — vision agent judges the frames

What To Keep

It's Alpha — but the pattern generalizes

Built in ~24 days over 6 commits by Diego Colombo. Honestly early — yet the principle holds: any agent building a visual surface can be handed eyes.

Capture, act, capture, compare — give the agent a way to see, and it can verify itself.

~24

days of development

6

commits, single author

Alpha

maturity · MIT license

Sources

Research Methodology

Primary source: colombod/amplifier-bundle-tui-tester (default branch main), cloned to /tmp/tui-tester-src.

Timeline: first commit 2026-01-30 (bc903d3), last commit 2026-02-23 (4853402) — ~24 days.

Feature status: Alpha (pyproject classifier "Development Status :: 3 - Alpha"); MIT license.

Commands run:

Repo confirmed: gh repo view colombod/amplifier-bundle-tui-tester; git -C /tmp/tui-tester-src remote -v
Tool & agent inventory: cat /tmp/tui-tester-src/README.md ; cat /tmp/tui-tester-src/bundle.md
Tests re-run: cd modules/tool-tui-tester && uv run --with pytest --with pytest-asyncio --with pyte --with Pillow python -m pytest tests/ -q → 167 passed in 42.87s
Operations & capture: cat amplifier_module_tool_tui_tester/__init__.py; cat session_manager.py
PTY & resize: grep -n 'pty.fork\|execvpe\|TIOCSWINSZ\|SIGWINCH' session_manager.py
Keys: cat keys.py (SPECIAL_KEYS table)
Render deps: cat pyproject.toml (pyte>=0.8.0, Pillow>=10.0.0)
Example target: sed -n '1,25p' examples/test-textual-app.md; amplifier-tui pyproject (textual>=0.47.0)
History: git -C /tmp/tui-tester-src rev-list --count HEAD (6); git shortlog -sne --all (Diego Colombo)

Gaps: No release tags inspected; "catches bugs no unit test can" stated as design intent, not a measured end-to-end case. The "835 tests" and Rust/Ratatui backstory were not verifiable and are excluded.

Primary contributor: Diego Colombo — 6 of 6 commits (the largest commit is co-authored "Generated with Amplifier").