Skip to content
fradzano

The Price of the Unwritten Standard

When writing code gets cheap, review becomes the bottleneck, one study finds, and, this piece argues, in part the place where an unwritten standard is reconstructed at every change, by people slowly losing the means to do it.

Or: Who Pays for the Specification Nobody Wrote


Eliyahu Goldratt’s novel The Goal turns on one lesson: time saved anywhere but at the bottleneck buys nothing.

A working paper by Fiona Chen and James Stratton at Harvard, covering 655 firms, shows what that lesson looks like in software. Compared with firms that had not yet adopted coding agents, firms that took them up opened more than a third more pull requests per engineer. They shipped about a seventh more software, a gain the authors cannot statistically tell apart from zero. The time from submitting a change to merging it grew by more than half, and that span covers everything between the two points, not review alone. Code review, the authors conclude, became the bottleneck.

    Coding agents, twelve months after a firm adopts them
    ────────────────────────────────────────────────────────────

    pull requests         pull requests         Jira issues
    opened         ──>    merged         ──>    resolved

      +35.7%                +26.0%                +13.7% *

    ├── opening to merge ──┤
        +56.1% longer

        measured separately, for pull requests that drew
        a comment from a person other than the author:
        waiting for the first such comment   +59.9%
        first such comment to merge          +58.5%

    ────────────────────────────────────────────────────────────
    Change against the mean before agent adoption; firms that
    had not yet adopted serve as the comparison. Counts are
    per engineer and month.
    * Not statistically distinguishable from zero; an increase
      above 30.7% is ruled out.
    After Chen & Stratton (2026), data from 655 firms.

One mechanism is capacity. When work arrives faster than the checking stage can clear it, it piles up, and Little’s law ties the length of the pile to the time spent waiting in it. A related study of public GitHub code, by Demirer, Musolff and Yang, found the same gap between writing code and shipping it.

The Phoenix Project gave this shape a face: Brent, the one engineer the urgent work keeps running through, and at whose pace it moves.

Writing got cheap. Being sure did not.


In the same study, reviewers overwhelmingly work in code bases where they have committed code or opened a pull request before, far more often than random assignment across the firm would give. The study reports this as one pooled figure, with familiarity counted per code base, and does not say whether commits an agent wrote count towards a reviewer’s familiarity.

The plain explanations come first. Reviewers may simply be assigned within their own teams. And a review checks that a change fits the code around it, which takes knowledge of that code however good the specification is.

Part of what the familiar reviewer brings may also be the standard itself: the expectation nobody wrote down, reconstructed at every change. That is a reading, not a finding. W. Edwards Deming told managers to build quality into the product instead of inspecting for it in bulk. Code review is, by analogy, where software goes looking for the quality nobody specified. The Capability Debt traced how the discipline of specification was lost. Its absence did not make the work go away.

A review may be, in part, a standard paid for after the fact.


Peter Naur made the underlying point in 1985. The real product of programming, he argued, is the theory its builders hold: how the program relates to the affairs of the world it helps to handle, why each part of it is what it is, and how to answer a demand for change. That theory is “inextricably bound to human beings”. Re-establishing it “merely from the documentation” is, in his words, “strictly impossible”. When the team that holds it disperses, the program dies, even if it keeps running; reviving it, he added, is rarely needed and at best costly.

The next step is mine, not Naur’s. A reviewer holds such a theory because, at some point, they built part of the thing. Call the understanding that review draws on the stock of theory. If code an agent wrote leaves less theory behind, line for line, while the volume grows, that stock sinks against the code it has to cover. Reviewing may rebuild some of it; how much is an open question.

In the study’s model, the closest thing to this stock is the reviewer’s own productivity, and it is held fixed. Its estimate keeps the reviewer’s own productivity constant and books a one-off loss of about seven per cent, not statistically significant, entirely to worse drafts; its projections never let review productivity fall. The loss is recovered from one number, the share of pull requests that draw a comment. The model also equates time in review with the inverse of review productivity, and time in active review rose by more than half, which the authors themselves call indicative of a decline in review productivity; that number is not among those the calibration uses.

The study’s ceiling is a cap on how much of a firm’s labour can go to review, set by how many of its engineers are senior and how much of the review only they may do. On the authors’ own equations, with the workforce and the number of changes per unit of output fixed, a review productivity that declines as a trend keeps lowering output once that cap binds, for as long as the trend lasts; their projection for fixed review productivity shows a plateau, held at its running maximum by construction. The trend is my step; the rest is their arithmetic.

What I can offer is a probe, not a test. On changes no agent touched, as far as that can be seen, time in active review should rise with a reviewer’s own accumulated exposure to agent-written changes in that code base. Worse drafts predict no such effect there. Workload, the routing of hard changes to experienced reviewers, process change and learning by reviewing would all disturb it, the last in the opposite direction, and the probe cannot hold them apart. A reviewer short of theory may also look less deeply instead of longer, which the argument predicts as well and the probe cannot see. Neither a rise nor its absence would settle the question.

On this argument, the stock of theory sinks under the load that raises the pressure on it.


As I use the words, a specification is written by whoever commissions or builds one piece of work, and binds that work; a standard is agreed among several parties and binds them all, its authors included. The standard is what can be written down, the quality scenarios and the criteria every change must meet: in short, the yardstick. The theory is what cannot, at least not fully. Writing the standard stops the theory being spent, change after change, on reconstructing it. Any reviewer, human or machine, can check against a written standard. A gate built on it raises the floor; it cannot supply what lies above.

Across more than a hundred studies, Grove and Meehl found mechanical prediction almost invariably equal or superior to clinical judgment, where good predictors and a measurable outcome exist. Robyn Dawes had named the reason: people “are much better at selecting and coding information than they are at integrating it”. Carried over to the checking of software, which is my step, it suggests that building the yardstick is the open task and using it is closer to classification. A professor sets the exam and sketches the answers; an assistant marks a hundred scripts against the sketch, and takes the answer it did not foresee back to the professor.

A yardstick set after seeing the candidate bends towards it, so it has to be frozen before the work it judges exists. Researchers preregister their analyses for that reason, and switching a trial’s outcomes unannounced counts as a fault; I take both over by analogy. A frozen standard may still change, but I would hold it to five tests. Every requirement in it could be shown false. It was versioned before the work it judges existed. Its amendments are visible, dated and counted, and never made by a party that has just failed against it. No one whose work it checks wrote it alone. And a candidate broken on purpose fails against it. Testing practice keeps author and tester apart, in degrees rather than absolutely; extending that to the yardstick is my step, and I would apply it to an agent that writes both the code and its tests, in either order. Mutation testing judges a test suite by whether it catches programs with faults planted in them; holding a standard to the same measure is again my step. Someone has to enforce all five, or the freeze is a formality.

Those who will carry the damage belong among the standard’s authors, alongside those who build and those who check; I take this by analogy from the study of commons, where those affected by a rule can take part in changing it. And the standard has to be written while those who hold the theory are still there. Their hours are the bottleneck’s: review falls to those who know the code. So, on my argument, writing it pays only where the standard is reused across many changes and holds still, and whatever does not need their theory should be taken off them, except the work through which that theory forms.

Put the intelligence into the yardstick first, not into each inspection.


The Architecture of Antifragility built a barbell: machine-attested protection on one side, disposable experiments on the other. A machine can check against a written standard; someone still has to understand the standard, and the parts of the system it cannot capture.

So, on my argument, there are two kinds of code. Code regenerable from its specification may come from anyone and may be thrown away; The Debt of Decision called this exit-first. I would count code as regenerable only once it has been regenerated and has passed against a frozen standard, not against tests of its own making. Code that cannot be discarded needs people who hold its theory, and that theory forms where people write the code or work through it by hand. If the stock of theory sinks as I have argued, teams that only review what machines produce may form less of it than the growing code requires. One junior engineer in the study described the change: “My work became less about writing code directly and more about reviewing and correcting what Claude produced”, adding that it still sped up their coding substantially. A senior engineer there said: “I do not actually type much code anymore. But I also have over a decade of software design principles, systems knowledge, and intuition.” How the next such decade forms, the study does not ask. Reading the same programming language is not holding the same theory; the study measures familiarity by code base, not by language.

Between the two kinds lies the fragile middle: agent code nobody understands and nobody can throw away, where the comprehension debt of The Debt of Decision collects and review loses its ground. The Capability Debt argued that holding a system in one’s head is not what becomes scarce as code generation is automated. A note added to that essay now concedes the point for the firms the study measured: there, holding the system in one’s head is what ran short. My argument here goes one step further, and it is not the study’s: inside firms, that capability is also what erodes. The Architecture of Antifragility already gives its experiments an expiry date; a core that can be regenerated could carry one too.

What you cannot throw away, someone must understand.

Sources & Further Reading

Flow and inspection

Theory and judgment

Freezing and independence