Thinking out loud about AI agent reliability, verification design, and what the data actually says. A new piece most weeks.
Latest · a new one most weeks
The Framework
What a verification surface is, and how stages and gates actually catch failure.
Jul 21, 2026
What you can verify at each pipeline stage isn't arbitrary. It's set by how constrained the artifact is: language becomes plans, code, and tests, and the deterministic ceiling rises.
Jul 7, 2026
I stopped asking which model is best and started asking whether I can verify what it produces. That reframing, the verification surface, held across coding agents, medical imaging, and language models.
Jul 1, 2026 Start here
An AI agent can write code that compiles, passes every test, and still ships a privilege escalation. The bug isn't in the code, it's in what you checked against. Same code, different surfaces.
Jun 24, 2026
AI agents rarely write bugs anymore; the risk is the slow rot as they pick inconsistent patterns over time. The fix isn't more checks on the code, it's gating the plan and design before the code is ever written.
Jun 17, 2026
A coding agent rebuilds its understanding from the repo every run, so the repo is its memory. But that memory splits in two: soft living context and hard workflow state. Each needs the opposite rule.
Jun 3, 2026
"Prompt engineering" is being used for two completely different things. One is writing and rhetoric. The other is distributed systems design. They aren't the same kind of hard.
May 5, 2026
Is model choice the most important thing in getting good results from agents? For me, no. The harness around the model is doing the work. Each stage produces a verification surface.
Apr 28, 2026
Does the verification topology generalize? I ran the same 11 gates unchanged across three medical imaging models. Rejection rates scaled cleanly with model weakness: 6.3%, 11%, 93%.
Apr 14, 2026
Tasks that fail early and get revised have half the downstream failure rate. The most expensive thing a pipeline can do is let bad work through early.
Mar 30, 2026
Low overlap doesn't mean you're covered. Map your error types to your gates — the empty cells are where your next investment should go.
Mar 19, 2026
Adding more reviewers doesn't help if they're all looking at the same thing. Checks at different stages catch fundamentally different errors.
Mar 17, 2026
Coding agents don't make random mistakes. 91% of failures are predictable — systematic errors and omissions that compound through every stage of the pipeline.
Mar 5, 2026 Start here
Why AI failures propagate — and why the fix is checkpoints, not better models
In Practice
The day-to-day habits, not the theory.
Oct 16, 2026
When my agents update the docs, every line is one of three kinds: facts the code can prove, facts nobody acts on, and the reasoning behind decisions. The reasoning is the part worth writing.
Oct 9, 2026
Coding agents are silo engineers, turbocharged. One proposed a fix that made sense from inside its task and would have broken my architecture. Here is how I put the coordination back.
Sep 28, 2026
My EMT instructor said there are three kinds of patients, and only one where your work makes the difference. It is how I decide which agent-written code gets my attention.
Sep 16, 2026
Anthropic's CI stopgaps decayed fast: a bigger machine held 70 days, sharding 29, daily restarts less than one. That pattern is what a wall looks like. There is another place to optimize, and it has a number.
Sep 3, 2026
The old rule for sizing a service was that it should fit in one engineer's head. That turns out to be a useful shape for coding agents too. But the two bounds stopped being the same size, and only one of them matters at 3am.
May 27, 2026
Teams keep asking how to catch weird behavior from production AI agents. The better question: do you have a process? Map it once with a big agent, then run targeted agents against the steps you already run.
May 6, 2026
Fred Brooks said it in 1975, and it's more true now than ever. AI agents drive the cost of code to zero, which finally makes the first version disposable. The code is disposable. The understanding is not.
Apr 16, 2026
Good managers delegate tasks. Great managers delegate outcomes. The same is true for managing AI agents. Define the gates that do the managing, while you do the leading.
Apr 9, 2026
I build by asking questions, not by issuing commands. Four questions from my Claude Code logs that make the biggest difference.
Field Notes
A real incident, and why the fix was almost never a better model.
Jul 15, 2026
My first hackathon. I built a tool that checks AI-written biology claims against the data, then a final test run caught a gap in my own pipeline. Turns out the model can only verify what it was actually shown.
Mar 12, 2026
A coding agent issued a terraform destroy in dev. The fix wasn't better reviewers — it was a deterministic gate that routes only what matters to humans.
Mar 10, 2026
A hallucinated company name in a marketing report. The root cause wasn't the model — it was a missing gate early in the pipeline.
Projects & Commentary
Open source, experiments, and takes on where the field is going.
Sep 22, 2026
I have a day job and a kid. What coding agents gave back wasn't revenue, it was the hours to build things I only wanted to understand. An echo of when science happened in parlors.
Jul 23, 2026
A model in mixlab is a JSON config, so trying a hunch is editing a file. A faithful MolGPT reproduction, then a better molecule generator from one field change: validity 0.988 → 1.000.
Jun 10, 2026
Everyone is arguing about one number from OpenAI's harness-engineering post: 3.5 PRs per engineer per day. The argument is in the wrong domain. It's a unit we borrowed from the world that agents are replacing.
Apr 30, 2026
I open-sourced a small ML utility 11 days ago. 3,000 people are using it. ~100 pulls are coming from teams extending it for NVIDIA Blackwell silicon. If you're sitting on something useful, share it.
Apr 17, 2026
Define a model in JSON, train on your Mac, ship to a cloud GPU. No code changes. Open source: mixlab.
Apr 7, 2026
As agents write more code, we're trading tech debt for cognitive debt. Two strategies to stay connected to code you didn't write.