Thinking out loud about AI agent reliability, verification design, and what the data actually says. A new piece most weeks.
Latest · a new one most weeks
The Framework
What a verification surface is, and how stages and gates actually catch failure.
Jul 7, 2026
I stopped asking which model is best and started asking whether I can verify what it produces. That reframing, the verification surface, held across coding agents, medical imaging, and language models.
Jul 1, 2026 Start here
An AI agent can write code that compiles, passes every test, and still ships a privilege escalation. The bug isn't in the code, it's in what you checked against. Same code, different surfaces.
Jun 24, 2026
AI agents rarely write bugs anymore; the risk is the slow rot as they pick inconsistent patterns over time. The fix isn't more checks on the code, it's gating the plan and design before the code is ever written.
Jun 17, 2026
A coding agent rebuilds its understanding from the repo every run, so the repo is its memory. But that memory splits in two: soft living context and hard workflow state. Each needs the opposite rule.
Jun 3, 2026
"Prompt engineering" is being used for two completely different things. One is writing and rhetoric. The other is distributed systems design. They aren't the same kind of hard.
May 5, 2026
Is model choice the most important thing in getting good results from agents? For me, no. The harness around the model is doing the work. Each stage produces a verification surface.
Apr 28, 2026
Does the verification topology generalize? I ran the same 11 gates unchanged across three medical imaging models. Rejection rates scaled cleanly with model weakness: 6.3%, 11%, 93%.
Apr 14, 2026
Tasks that fail early and get revised have half the downstream failure rate. The most expensive thing a pipeline can do is let bad work through early.
Mar 30, 2026
Low overlap doesn't mean you're covered. Map your error types to your gates — the empty cells are where your next investment should go.
Mar 19, 2026
Adding more reviewers doesn't help if they're all looking at the same thing. Checks at different stages catch fundamentally different errors.
Mar 17, 2026
Coding agents don't make random mistakes. 91% of failures are predictable — systematic errors and omissions that compound through every stage of the pipeline.
Mar 5, 2026 Start here
Why AI failures propagate — and why the fix is checkpoints, not better models
In Practice
The day-to-day habits, not the theory.
May 27, 2026
Teams keep asking how to catch weird behavior from production AI agents. The better question: do you have a process? Map it once with a big agent, then run targeted agents against the steps you already run.
May 6, 2026
Fred Brooks said it in 1975, and it's more true now than ever. AI agents drive the cost of code to zero, which finally makes the first version disposable. The code is disposable. The understanding is not.
Apr 16, 2026
Good managers delegate tasks. Great managers delegate outcomes. The same is true for managing AI agents. Define the gates that do the managing, while you do the leading.
Apr 9, 2026
I build by asking questions, not by issuing commands. Four questions from my Claude Code logs that make the biggest difference.
Field Notes
A real incident, and why the fix was almost never a better model.
Jul 15, 2026
My first hackathon. I built a tool that checks AI-written biology claims against the data, then a final test run caught a gap in my own pipeline. Turns out the model can only verify what it was actually shown.
Mar 12, 2026
A coding agent issued a terraform destroy in dev. The fix wasn't better reviewers — it was a deterministic gate that routes only what matters to humans.
Mar 10, 2026
A hallucinated company name in a marketing report. The root cause wasn't the model — it was a missing gate early in the pipeline.
Projects & Commentary
Open source, experiments, and takes on where the field is going.
Jun 10, 2026
Everyone is arguing about one number from OpenAI's harness-engineering post: 3.5 PRs per engineer per day. The argument is in the wrong domain. It's a unit we borrowed from the world that agents are replacing.
Apr 30, 2026
I open-sourced a small ML utility 11 days ago. 3,000 people are using it. ~100 pulls are coming from teams extending it for NVIDIA Blackwell silicon. If you're sitting on something useful, share it.
Apr 17, 2026
Define a model in JSON, train on your Mac, ship to a cloud GPU. No code changes. Open source: mixlab.
Apr 7, 2026
As agents write more code, we're trading tech debt for cognitive debt. Two strategies to stay connected to code you didn't write.