The Speed Trap: AI Made Us Faster Than We Can Justify
Last Saturday I promised a deeper look at what actually changes when AI enters the delivery lifecycle. This is it. The short version: the hard part is no longer producing software. It is being able to say, with evidence, why the software you just produced is safe to ship.
Last Saturday I promised a deeper look at what actually changes when AI enters the delivery lifecycle. This is it.
The short version: the hard part is no longer producing software. It is being able to say, with evidence, why the software you just produced is safe to ship.
The lifecycle got faster in one place only
Look at how a change moves through a team that has adopted AI seriously. An idea becomes a specification in minutes. The specification becomes a working branch in minutes. Tests appear alongside the code, already passing. A pull request lands before the person who asked for it has finished their coffee.
Now look at what did not move. Someone still has to read that pull request. Someone still has to decide whether the tests that pass are the tests that matter. Someone still has to hold the change in their head next to the rest of the system and judge whether the two fit.
Generation capacity went up by an order of magnitude. Verification capacity went up by roughly nothing. That gap is the speed trap, and every team I have watched adopt AI-accelerated delivery falls into it at the same point: not when the AI writes bad code, but when it writes so much plausible code that nobody can keep up with checking it.
The trust gap is a throughput problem, not a quality problem
This is the part people get wrong. The failure mode of AI-assisted delivery is not a flood of obviously broken output. Obviously broken output is easy — it fails a test, it fails a build, it fails a glance.
The failure mode is output that is plausible. It compiles. It passes. It reads like something a competent engineer wrote, because in a statistical sense it is. And it is wrong in a way that only shows up three sprints later, in production, in a corner of the system nobody connected to that change.
When a team produces more change than it can justify, the review process does not stop. It degrades. Approvals get faster and thinner. "Looks good to me" stops meaning "I checked" and starts meaning "I did not see anything alarming in ninety seconds." The gate is still there on the org chart. It is no longer there in practice.
That is the trust gap: the distance between what a team ships and what a team can defend.
Orchestration is the discipline that closes it
Orchestration is not prompting better. It is not running more agents in parallel. Both of those raise generation capacity, which is the side of the equation that was never the problem.
Orchestration is the design of the system in which models do work. Concretely, it answers four questions, every time:
- What work goes to which worker?
- What does that worker know when it starts?
- What must it hand back besides the result?
- Who decides whether the result advances, and on what basis?
If those four questions have deliberate answers, you have an orchestrated pipeline. If they have accidental answers, you have a very fast way to generate things nobody has checked.
Those four questions are the four pillars.
Pillar 1 — Work routing
Decompose work into units a model can complete and a human can verify. Those two constraints pull in the same direction, and both of them pull toward small.
The practical test is not "can the model do this?" — the model will attempt anything. The test is: if this unit comes back wrong, how expensive is it to notice? A unit whose failure is invisible until integration is too big, no matter how confidently it was completed.
Routing also means matching the unit to the worker. Not every task needs the most capable model available, and not every task survives the cheapest one. Treating model selection as a per-unit decision rather than a global setting is one of the cheapest quality improvements available.
Pillar 2 — Context engineering
The context window is the new test environment. What the model can see determines what it can be correct about, in the same way that what a test harness sets up determines what the test can actually prove.
Almost every plausible-but-wrong output I have traced back has the same root cause: the model was not lied to, it was simply not told. It did not have the interface contract. It did not have the constraint that was agreed three months ago in a decision record nobody linked. It did not have the two adjacent modules that make the naive implementation unsafe.
So context stops being something you type and becomes something you maintain: curated, versioned, reviewed, and assembled per unit of work. If your specifications, interface contracts and decision records are not in a state where a machine can consume them, they were probably not in a state where a new hire could either.
Pillar 3 — Evidence production
Change the definition of "done." The deliverable is not the artifact. The deliverable is the artifact plus the evidence that it works.
An agent that reports "implemented and tested" has told you nothing you can act on. An agent that hands back the diff, the tests it added, the output of the run, the behaviour it verified and the behaviour it explicitly did not, has produced something a reviewer can examine in a minute instead of an hour.
This is the pillar that actually buys back verification capacity, and it is the one most teams skip. Evidence is cheap for a machine to produce and expensive for a human to reconstruct. Getting that asymmetry the right way round is most of the win.
Pillar 4 — Gate design
A gate is a decision, not a checkbox. Every gate worth having has four properties: a named owner, a defined input, an explicit pass condition, and a real consequence when it fails.
Most pipelines fail the last one. A gate that everybody knows can be waived under deadline pressure is documentation, not control.
The design rule I keep coming back to: automate the evidence, keep the judgment. Machines are excellent at assembling everything a decision needs. They should not be the ones making a decision whose consequences land on a person.
Where this leaves you
The teams getting real leverage out of AI are not the ones generating the most code. They are the ones who rebuilt their verification capacity to match their new generation capacity — by routing work deliberately, engineering context as a first-class artifact, demanding evidence rather than claims, and designing gates that actually hold.
That is the argument at the centre of my chapter, “Quality at Machine Speed,” in Erik Seversen’s AI Transformation.
Next Saturday I get specific about the gate side of it: RPIQ, what belongs at each stage of a quality gate pipeline, and how to tell a gate that works from a gate that only looks like one.
This is part 2 of a five-part series on AI orchestration and the future of trust in software quality. Subscribe to get the rest as it lands.
