# Michael Rothrock

> Research on AI agent reliability, built around the verification surface: the primitive that reliability is a property of what a pipeline stage exposes to be checked, not of the model. Empirical data from 543 hours of autonomous AI-assisted software development, with cross-domain validation in medical imaging and language-model pipelines.

## Research

- [543 Hours of Autonomous Work](https://michael.roth.rocks/research/543-hours/): 97 days of production data from autonomous Claude Code agents. 14,926 prompts, 650 work arcs, 165 shipped releases. Methodology, failure modes, and reproducible patterns.
- [Gate Analysis](https://michael.roth.rocks/research/gate-analysis/): 5,109 cross-model review gate checks testing whether AI agents produce incoherent output. 87% of errors are predictable (49% omission, 38% systematic, 13% incoherent). Decomposition converts incoherence to omission.
- [Trust Topology](https://michael.roth.rocks/research/trust-topology/): A framework for engineering reliability from unreliable AI agents. Reliability is a property of the verification surface (the artifact a stage exposes plus the checks writable over it), not the individual model. A trust topology is the arrangement of those surfaces; four diagnostic properties measure whether the arrangement works. Formal paper: Trust Topology: Verification Surfaces as the Unit of Reliability (doi:10.5281/zenodo.20292194), with medical and language companions.
- [The Overlap Ratio](https://michael.roth.rocks/research/overlap-ratio/): A metric (omega) measuring whether verification gates do independent work or repeat the same check. Software pipeline omega = 0.118, cross-domain validation with medical imaging (0.125 for strong model, 0.767 for weak model). Includes per-gate rejection rates, pairwise Jaccard overlap, and two pipeline architectures (sequential with revision, sequential without revision).

## Blog Posts

- [The Blind Spot Map](https://michael.roth.rocks/blog/the-blind-spot-map/): Low overlap doesn't mean full coverage. Map error types to gates — empty cells are blind spots. File-scoped review cannot see cross-file contradictions.
- [Stage Coverage Beats Gate Density](https://michael.roth.rocks/blog/stage-coverage-beats-gate-density/): One check at each stage catches more than ten checks at a single stage. 96% of stage 4 rejections invisible to stage 2. Cross-stage checks multiply; within-stage checks overlap.
- [Errors Compound Forward](https://michael.roth.rocks/blog/errors-compound-forward/): Error taxonomy from 5,109 gate checks. 49% omission, 38% systematic, 13% incoherent. Errors at one stage get magnified by every downstream stage.
- [The Terraform Destroy](https://michael.roth.rocks/blog/the-terraform-destroy/): A coding agent issued terraform destroy. The fix: deterministic gates on parseable output, not more human reviewers. Verification fatigue makes human gates unreliable at scale.
- [The Missing Gate](https://michael.roth.rocks/blog/the-missing-gate/): A content ingestion pipeline hallucinated a company. The missing gate: a deterministic check between classification and entity extraction.
- [Three Robot Bakers](https://michael.roth.rocks/blog/three-robot-bakers/): Analogy for error propagation in AI pipelines. Robot 1 misreads sugar as salt, robots 2 and 3 faithfully build on the mistake. Checkpoints between stages prevent cascade.

## Key Concepts

- **Gate**: A verification checkpoint between pipeline stages. Pass, fail, or escalate to human.
- **Overlap ratio (omega)**: Fraction of rejected cases caught by more than one gate. Low omega = complementary gates. High omega = redundant gates.
- **Typed coverage (blind spot map)**: Matrix of error types vs gates. Zero-coverage columns are blind spots no gate can detect.
- **Verification surface**: The artifact a pipeline stage exposes plus the deterministic checks writable over it. The unit of reliability: a gate can only catch failures its artifact exposes, so the representation, not the gate, is the primary engineering lever.
- **Capacity (eta)**: The ceiling on what any zero-false-positive check over an artifact can catch, fixed by the representation. How checkable an artifact is, determined by what its structure exposes. Code is more checkable than plans; test results are more checkable than code.
- **Tri-state gate**: Pass/fail/human. Deterministic checks provide hard guarantees. Stochastic (LLM) checks handle judgment calls. Humans see only what needs unique expertise.
- **Stage coverage > gate density**: Checks at different stages catch fundamentally different errors. Cross-stage composition is multiplicative; within-stage composition is additive and overlap-limited.

## Open Source

- [Claude Code Log Analyzer](https://github.com/mrothroc/claude-code-log-analyzer): Compute the overlap ratio on your own Claude Code session logs. Discovers review gates, classifies outcomes and error types, calculates omega.

## Contact

- Site: https://michael.roth.rocks
- LinkedIn: Michael Rothrock
- GitHub: mrothroc