Quality at machine speed: ~70,000 tests, static analysis, and the review bias nobody warns you about

Part 5 of "From Vision to Production." Publishes Tue Jul 21, 2026.

Here's the uncomfortable arithmetic of building with AI: the code arrives faster than any human can read it. I can generate a day's worth of changes in an afternoon. I cannot review a day's worth of changes in an afternoon — not properly. So the question that decides whether this is a toy or a real practice is simple. When the machine writes the code, who checks it?

For me the answer is: not a person, staring harder. A system of gates.

Gates, not vigilance

The instinct when quality slips is to review more carefully. That doesn't scale, and it puts the weakest link — my tired attention at 6pm — in charge of what ships. Instead, every change moves through a fixed sequence of gates, and nothing merges until all of them pass.

At commit, the cheap, fast checks run: formatting, linting, dead-code detection. At the pull request, the real weight lands — the test suite and static analysis. CI runs the full suite again, clean, plus a duplication scan. And at merge, the last gate is a review by a different model than the one that wrote the code, for reasons I'll get to. The point is that "careful" is no longer a mood I have to summon. It's built into the pipe.

~70,000 tests, and a repo that can't cheat

Underneath the gates is the thing that makes them meaningful: roughly seven thousand automated tests.

The shape is a normal pyramid — a wide base of fast unit tests, a middle band of integration tests, a thin top of end-to-end tests that exercise the app the way a user would. Volume alone isn't the interesting part, though. The interesting part is the dual-repo structure: the product code lives in one repository and a share of the verification — fixtures, evidence, cross-checks — lives in another. Keeping them separate means the tests can't quietly be reshaped to flatter the code they're supposed to challenge. It's a small structural decision that removes a whole category of self-deception.

Static analysis catches what tests don't

Tests tell you the code does what you asked. They don't tell you the code is clean. Two tools sit in the loop for that. Periphery hunts down dead code — the functions and properties that survive because nothing ever forced them to justify their existence. PMD CPD catches copy-paste duplication, the kind that multiplies quietly when code is generated fast. Neither is glamorous. Both remove exactly the sort of debt that AI-speed development accumulates without anyone deciding to.

The review bias nobody warns you about

Now the part that surprised me most, and that I think matters beyond my project.

When I first set up automated review, I had the model that wrote the code review the code. It rubber-stamped itself. Not out of laziness — out of a shared blind spot. A model reviewing its own output brings the same assumptions, the same gaps, the same confident wrong turns to the review that it brought to the writing. It literally cannot see what it cannot see.

The fix is to make the reviewer independent. A different model reviews the merge gate, and it flags things the author model was structurally incapable of noticing. Combine that with the static-analysis tools — which don't have opinions or egos — and you get review that catches what any single perspective, human or machine, would wave through.

Going deeper

There's more here than one post can hold — the gate design, the evidence formats, the failure cases that taught me where a gate needs to sit. I go deeper on all of it in a forthcoming book chapter, "Quality at Machine Speed." I'll link it directly once there's a publish date.

And if you'd rather see the whole quality system running end to end, that's exactly what I'm walking through on stage at CAST 2026 (CAST26), August 3–5, in Cocoa Beach, FL.

Next week, the honest finale: what failed, what it actually cost, and what this means if you're a domain expert thinking about building alone.