Your Code Review Is Probably Broken

Most code review catches between 15 and 30 percent of the defects it was supposed to find. That is not a guess. IBM measured it in the 1970s, and studies at AT&T, HP, and Microsoft have confirmed the same range decade after decade.

Informal review is cheap, asynchronous, and socially acceptable. It is also mostly ineffective at finding bugs. Engineers read too fast, skip the error paths, and avoid pointing out real problems because nobody wants to be the person who holds up the merge.

There is an alternative that consistently reports 60 to 90 percent defect removal rates. It was invented at IBM in 1976 by Michael Fagan. It requires no tools, no AI, and no budget. It does require something most engineering teams refuse to give: structure.

What Is a Fagan Inspection?

A Fagan inspection is a formally defined, multi-step review process with specific roles, time limits, entry criteria, and checklists. Unlike a typical pull request review, it is not a conversation between author and reviewer. It is a structured meeting with a moderator, a reader, inspectors, and the author.

The author is mostly silent. The meeting is strictly timeboxed. And the only goal is to find defects.

The process follows six steps:

  1. Planning. The moderator selects the material, verifies entry criteria, assigns roles, and schedules the meeting. Entry criteria exist for a reason. You do not inspect a draft. The document must be complete, compilable, and tested before it earns the right to consume four people’s time.

  2. Overview. Optional. The author explains context if the inspectors are unfamiliar with the domain.

  3. Preparation. Each inspector reviews the material alone before the meeting. This is non-negotiable. You do not show up cold. Inspectors use checklists tailored to common defect types and annotate issues privately.

  4. Inspection. The meeting itself. The reader, who did not write the code, walks through it line by line and paraphrases aloud. Inspectors raise issues when they spot discrepancies. The author listens. Nobody proposes fixes. The moderator enforces time limits and keeps the meeting focused on defect identification only.

  5. Rework. The author fixes the defects.

  6. Follow-up. The moderator verifies every defect was addressed. If too many defects were found, a reinspection occurs.

Four roles keep the process honest. The moderator plans and controls. The author created the work and answers questions only when asked. The reader paraphrases the code during the meeting, forcing slower, more careful comprehension. The inspectors, usually two to four people, find the defects.

Why Informal Review Fails Where Fagan Inspections Succeed

The difference is not talent. It is process design.

In a typical pull request review, the reviewer reads the diff in a browser, skims the happy path, leaves a few comments, and approves. There is no preparation time. There is no checklist. There is no mechanism that forces the reviewer to examine the error handling or boundary conditions. The social dynamics reward speed and politeness, not thoroughness.

Fagan inspections invert those incentives.

Individual preparation means every inspector has actually read the code before the meeting starts. The reader’s paraphrase forces the group to process the code at comprehension speed instead of skimming speed. Checklists direct attention toward known defect categories instead of whatever catches the eye. Time pressure prevents the meeting from drifting into design debates. And separating defect finding from defect fixing prevents the group from anchoring on the first solution someone suggests.

The result is that Fagan inspections catch most defects before they reach testing or production.

The cost is front-loaded in person-hours.

The Real Trade-Off: Person-Hours vs. Defect Escape

Here is why most teams do not use Fagan inspections.

A single inspection requires four to six people in a room for up to two hours to review roughly 250 lines of code. That is 8 to 12 person-hours for a small change. In a modern CI/CD workflow where teams ship multiple times per day, this looks absurd.

The process also feels bureaucratic. Entry criteria, formal roles, printed checklists, follow-up verification. Most engineers will hate it on principle. And it does not scale to large diffs. A two-thousand-line refactor would require eight separate inspection meetings.

But the calculation changes when you look at total cost instead of meeting cost.

IBM’s original data showed that finding and fixing a defect during inspection cost about one-tenth what it cost to find and fix the same defect during testing. When it escaped to production, the ratio grew to twenty or thirty to one.

So yes, the inspection is expensive. It is still cheaper than debugging production incidents, burning sprint capacity on reactive fixes, and losing customer trust.

The problem is that the savings are invisible. You cannot measure the bug you prevented. The cost of the meeting is immediate and obvious. That is why informal review wins in most organizations. It optimizes for visible speed over invisible quality.

Running a Lightweight Fagan Inspection in 2026

You do not need to adopt the full ceremony. Most teams can get seventy percent of the benefit with twenty percent of the overhead by keeping the core mechanics and dropping the paperwork.

Here is a practical sequence:

Require individual preparation before any synchronous review. If you have not read the code, you do not attend.

Assign a reader who did not write the code to walk through the logic aloud. Do not let the author drive. Paraphrasing forces the group to process every branch.

Use a checklist tailored to your team’s most common defect types. Start with the list below and add items as you learn from escaped defects.

Timebox to ninety minutes. End on time, even if you are not done. Schedule a second session rather than let fatigue destroy quality.

Keep the author passive. They answer clarifying questions only. No defending design choices.

Record defects, not solutions. Fix the defects after the meeting.

To make this concrete, here is a small Python script that plans an inspection, estimates time, and prints role-specific checklists:

#!/usr/bin/env python3
"""
Plan a Fagan-style inspection: estimate time and emit checklists.
"""

import argparse


def count_lines(filepath):
    """Count non-blank lines."""
    try:
        with open(filepath, "r") as f:
            return sum(1 for line in f if line.strip())
    except Exception:
        return 0


def plan_inspection(files, rate=125):
    """
    rate: reviewable lines per hour. Fagan recommended 100-125 for code.
    """
    total = sum(count_lines(f) for f in files)
    hours = total / rate
    minutes = hours * 60

    print(f"Total reviewable lines: {total}")
    print(f"At {rate} lines/hour, schedule: {hours:.1f} hours ({minutes:.0f} min)")

    if hours > 2:
        sessions = int(hours / 2) + 1
        print(f"WARNING: exceeds 2-hour limit. Split into {sessions} sessions.")

    print("\n--- ROLE: Reader ---")
    print("Paraphrase each block aloud. Do not just read variable names.")
    print("Pause at every conditional, loop boundary, and API call.")

    print("\n--- ROLE: Inspector ---")
    checklist = [
        "Off-by-one in loops and array or slice access",
        "Null, None, or undefined dereferences",
        "Resource leaks: files, connections, locks, contexts",
        "Error paths: are they handled, returned, and tested?",
        "Return values: are they checked before use?",
        "Shared mutable state and thread safety",
        "Boundary conditions and input validation",
        "Invariant violations: what should stay true, and does it?",
    ]
    for item in checklist:
        print(f"  [ ] {item}")

    print("\n--- ROLE: Moderator ---")
    print("Start on time. End on time.")
    print("No design debates. No solution proposals.")
    print("Record each defect with: file, line, severity, type.")


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Plan a Fagan inspection")
    parser.add_argument("files", nargs="+", help="Source files to inspect")
    parser.add_argument("--rate", type=int, default=125, help="Lines per hour")
    args = parser.parse_args()

    plan_inspection(args.files, args.rate)

Save it as inspect.py, run python inspect.py src/auth.py src/orders.py, and you have a time estimate and a checklist.

Frequently Asked Questions

What is a Fagan inspection?

A Fagan inspection is a structured, six-step review process with defined roles, entry criteria, and checklists. It was developed by Michael Fagan at IBM in 1976 to find defects in software work products before testing.

How is a Fagan inspection different from a pull request review?

A pull request review is typically asynchronous, informal, and driven by the author. A Fagan inspection is a synchronous meeting with assigned roles, mandatory individual preparation, strict time limits, and a rule that the author remains silent while others find defects.

Why are Fagan inspections not more common?

They are expensive in person-hours, feel bureaucratic to modern teams, and do not scale well to large, frequent changes. The cost is visible and immediate. The prevented defects are invisible.

Can Fagan inspections work in an agile or CI/CD environment?

Yes, but with modifications. Most teams use lightweight versions: mandatory individual preparation, a reader who paraphrases, a checklist, and a strict timebox. The full formal process is usually reserved for critical or high-risk modules.

Try It on One Module

You do not need to rewrite your process. Pick one module that has escaped defects in the last month. Gather three engineers who did not write it. Give them the code and a checklist twenty-four hours in advance. Schedule ninety minutes. Assign a reader. Make the author listen.

Measure what you find. Then decide whether the meeting cost more than the bugs would have.

If you want the original data, Fagan’s 1976 paper “Design and Code Inspections to Reduce Errors in Program Development” is still the best reference. It is fifty years old, and most teams have still not caught up.