Your Docs Have Three Kinds of Facts

I've found a successful pattern of regularly having my agents update the docs in the repo. I do it explicitly at the end of long sessions, where the context is full of decisions that need to be recorded. (I wrote more about that in an earlier blog post.)

As you'd expect, the docs grow quite lengthy over time, so updating has now grown to include both adding and removing lines. When I looked closely, I found the docs contain three kinds of facts: facts about the code, which can be generated and checked by CI; facts nobody acts on, which can be deleted; and the reasoning behind the decisions.

The reasoning, or the "why", is critical.

To illustrate, I recently had an issue where a coding agent proposed adding a transaction around an entity write. While reasonable, this is wrong, because adding a transaction would hide the actual bug: multiple services writing to the same entity. There is now a rule in the docs: every entity is owned by only one service, because this prevents duplicate logic and data races. The hard rule is enforced by CI, and now future agents understand the pattern to match.

It looks like we're all hitting similar problems and cultivating our own better practices. Anthropic recently shared a "Redesigning source control" post. They note their engineers spend their time telling Claude "no", because agents "violate core system constraints to satisfy a pattern that should never have been allowed."

That's the same root cause as my entity ownership issue, and it has the same fix: write down the "no" once, with the reason, and future agents (and the engineers driving them) can see it and match the pattern. But this is also backed up by a deterministic check on the core violation, so even if my sage advice is ignored, it still blocks the heathens.

In the spirit of openness, I should also share two attempted approaches that failed. First, I tried "assert this value" annotations in the docs. Turns out they drift just like the prose they guard, and a stale assertion that passes is worse than none because it gives a false assurance. Second, I tried using tools to extract deterministic checks from the prose, but they gave false alarms because the docs often point at things outside the repo.

Agents are going to keep proposing fixes that are reasonable in their component but wrong for the system. The code can show them how it's done. The docs can show them why, and the why is the pattern they follow.

What's the "no" you keep saying to your agents? That's probably the first rule worth writing down.