The Chapter, and the Company It Keeps

Four weeks ago I put a claim on this blog: trust in software is produced, not felt. Last week I named the four ways AI-assisted delivery quietly stops producing it, and I promised that this post would do something different — walk through what is actually inside the chapter, and point you at the people whose work sits next to mine in the book.

That is what this is. It closes the series.

Let me say what this post is not. It is not a summary that saves you the price of the book. A chapter that can be compressed into a thousand words was not worth writing in the first place. What follows is a map — the five moves the argument makes, in order — so you can decide for yourself whether the territory is worth your time.

Move 1: speed outran justification

The chapter starts from one asymmetry.

Generation capacity went up by an order of magnitude. Verification capacity went up by roughly nothing.

Every team I have worked with in the last two years has felt the first half of that sentence. Very few have done anything deliberate about the second. What opens up between them is what I have been calling the speed trap: the distance between what a team ships and what a team can actually defend.

It is worth being precise about the failure this produces, because it is not the one people expect. Bad code was never the hard problem. Bad code announces itself — it crashes, it fails the build, somebody notices. The hard problem is plausible code: it compiles, it passes the tests, it reads professionally, and it fails in production three weeks later for a reason nobody wrote down.

That distinction is the hinge the rest of the chapter turns on. If your quality system is tuned to catch things that look wrong, it will not catch the thing that is actually going wrong now.

Move 2: most quality gates are theatre

The instinct, once you accept the first move, is to add gates. More checks, more approvals, more green ticks before the release button unlocks.

This mostly does not work, and the chapter is fairly blunt about why. A gate that records that a step ran is not a gate that shows a claim holds. "The test suite executed" is a fact about your pipeline. "This release does not break the ordering that the billing service depends on" is a claim about your software. Most organisations have built an elaborate apparatus for producing the first kind of fact and then read it as though it were the second.

The chapter's answer is RPIQ — Research, Plan, Implement, Quality. The first three letters will be familiar to anyone who has worked with coding agents: understand the problem, decide the approach, write the code. The argument is about the fourth. Quality comes out of implementation, where it lives as an afterthought somebody gets to if there is time, and becomes a phase in its own right — with entry conditions that say what must already be true before it may start, defined outputs that are evidence artefacts rather than a green tick, and a named owner who can be asked "why is this safe?" and answer without hand-waving.

That last part is the whole test, and it is deliberately uncomfortable. If nobody can answer the question, you did not have a gate. You had a ritual.

Move 3: four failure modes traditional QA does not catch

This is the part of the argument that people push back on hardest, usually right before they recognise it.

Plausibility drift. A model's output is optimised to look like a correct answer. A helper that handles the common case and quietly mishandles the empty one is not a bug in the ordinary sense — it is a thing shaped exactly like working code.

Surface inconsistency. Individually correct components can be collectively incoherent. Three services disagree about whether a timestamp is UTC. Each one passes its own tests. The system is still wrong.

Silent regressions. A behaviour change no test noticed, because no test ever encoded that behaviour. The ordering something downstream depends on. The empty-list response a client special-cases. Nobody wrote it down, so nobody defended it.

Documentation rot. This one changed character entirely. A stale document used to be a mild inconvenience for the next person who read it. Now it is an input to systems that generate code at volume — a supply-chain problem that confidently produces wrong output at scale.

What these four have in common is that none of them look like failure while they are happening. Your dashboards stay green. That is precisely the problem.

Move 4: orchestration changes the job

If verification capacity is the thing that did not grow, the obvious response is to try to grow it — and the obvious way to do that is to generate more tests, faster, with the same machines that are generating the code.

The chapter argues this is a trap of its own, and that rebuilding verification capacity is a design problem rather than a tooling problem. It comes down to four decision points: work routing (which unit of work goes to which worker), context engineering (what information the model actually receives), evidence production (the artefacts that show the thing is correct), and gate design (who decides what advances, and on what basis).

Underneath all four sits one principle: automate the evidence, keep the judgment. Machines are genuinely excellent at assembling everything a decision needs. They should not be the thing that makes the decision. The moment the judgment moves into the machine as well, you have automated away the only part of the loop that was producing trust.

This is the part I get asked about most and the part the blog covered least, largely because it is the part that needs more room than a blog post gives it.

Move 5: trust is an output

The chapter ends where the series ended. Trust is not a feeling your stakeholders have about your team. It is an artefact your process either produces or does not.

The working test is simple enough to apply this afternoon: can you explain, to somebody who did not build the thing, why this release is safe — without appealing to the fact that the pipeline was green? If the honest answer is no, the gap between what you ship and what you can defend is wider than your dashboard suggests.

What the chapter has that the blog does not

The five posts gave you these arguments one at a time. The chapter is the connective tissue: the failure modes mapped onto the gates that are supposed to catch them, the orchestration model that holds the gates up, and the through-line from "generation outran verification" to "here is what you change on Monday."

That is the honest reason to buy it rather than re-read the archive. The posts are the claims. The chapter is the method.

Who else is in the book, and why that matters

Erik Seversen coordinated this one, and coordinating a multi-author book on AI is harder than it sounds. The failure mode is obvious in advance: thirty people writing thirty variations of "AI changes everything." What Erik managed instead was range — practitioners writing from inside their own domain, about problems they have personally had to solve.

That is the argument for a book like this over thirty blog posts. My chapter will tell you how to keep software trustworthy at machine speed. It will tell you nothing about what any of this does to hiring, to regulated industries, or to the way an organisation learns. The chapters around mine will.

The series, in order

If you have arrived at the end first, this is the whole argument in five pieces:

1. I Wrote a Chapter About Trust. A Machine Helped Me Test the Argument. — the claim, and what happened when I stress-tested it.
2. The Speed Trap: AI Made Us Faster Than We Can Justify — why velocity stopped being the interesting number.
3. A Gate That Works, and a Gate That Only Looks Like One — RPIQ, and what a gate has to establish to earn the name.
4. Trust Is Produced, Not Felt — the four failure modes, and the honest question to ask about your own pipeline.
5. This one.

Get the book

"Quality at Machine Speed: AI Orchestration, and the Future of Trust in Software Quality" is my chapter in The AI Transformation: Thriving Within Civilization's Next Big Disruption, coordinated by Erik Seversen.

Get it on Amazon

Thank you for reading the series. If you build or ship software with AI somewhere in the loop, I would rather hear that the method broke in your context than that it sounded good in mine. Tell me either way.