The Most Effective Quality Process Nobody Uses

In 1976, Michael Fagan published a paper in the IBM Systems Journal that described a review process so effective it became the gold standard for software quality. Fagan inspections caught 60 to 90 percent of all defects before a single test ran. A 2002 NASA study found that every hour spent in inspection prevented an average of 33 hours of maintenance work later.

By any measurable standard, this was the best review process software engineering has ever produced.

Almost nobody uses it today.

The question is not whether Fagan inspections worked. They worked almost too well. The question is why a process with that track record vanished from mainstream development, and whether we lost something important when we replaced it.

What Fagan Inspections Actually Looked Like

Fagan did not invent code review. He invented a specific, highly structured ritual for finding defects.

The process had six rigid phases:

  1. Planning: A moderator selected participants and verified the material met entry criteria.
  2. Overview: The author explained the background. This was context-setting, not review.
  3. Preparation: Every participant reviewed the material independently, at roughly 150 lines per hour, with a personal list of suspected defects.
  4. Inspection Meeting: The team convened for no more than two hours. A reader narrated the logic aloud. A recorder logged defects. The moderator kept the meeting focused on finding defects, not solving them.
  5. Rework: The author fixed every logged defect.
  6. Follow-up: The moderator verified corrections were made and no new defects were introduced.

The roles were specific and non-overlapping. The moderator ran the process but was not the technical authority. The reader narrated the code but did not defend it. The author was present but prohibited from explaining intent during the meeting. The purpose was not collaboration. It was cold, systematic defect detection.

This is the part that made it work. The social dynamics of normal engineering meetings were engineered out.

Why the Numbers Were So Good

The defect detection rate was not an accident. It came from a few deliberate design choices.

Independent preparation meant that six people examined the same code in isolation before any group discussion could bias them. The overlap between their lists told you how obvious a defect was. The items that only one person found were often the most valuable.

The two-hour hard cap prevented fatigue from destroying judgment. Fagan knew that inspection effectiveness drops off a cliff after about two hours. The 150-lines-per-hour rate was similarly deliberate. Go faster and you start seeing what you expect to see instead of what is actually there.

The no-solutions rule kept meetings focused. Nothing destroys an inspection faster than a room full of engineers designing a fix for a defect they have not fully characterized yet.

These constraints were not bureaucratic overhead. They were the mechanism. Remove them and you get something friendlier but less effective.

What Killed Fagan Inspections

If the process was this effective, why did it disappear?

The short answer is that it was expensive in exactly the way modern software development refuses to tolerate.

A single Fagan inspection consumed 15 to 20 percent of the effort spent writing the code being reviewed. In one documented case, 348 lines required 27.3 person-hours of inspection. That ratio is unfathomable when teams ship multiple times per day.

The scheduling alone was a full-time job. You needed five or six people in a room for two hours, plus preparation, plus follow-up. In a large organization, finding a two-hour slot where a moderator, reader, two reviewers, recorder, and author were all available could take days.

The rigid role structure also did not scale. Fagan inspections assumed a stable team with enough people to fill all roles. In a startup, a five-person team might not have anyone who can serve as a dedicated moderator without destroying velocity.

The biggest factor was cultural. Fagan inspections were deliberately uncomfortable. The author sat silently while colleagues narrated their code and logged its defects. There was no room for “this is just a first pass.” The process assumed defects are expensive and social friction is cheap. Modern engineering runs on the opposite assumption.

What We Replaced It With

The industry did not abandon structured review. It replaced it with pull requests.

Pull request review is asynchronous, low-ceremony, and embedded directly in the development workflow. A reviewer can look at a diff between meetings, on their phone, or while waiting for CI to finish. There are no assigned roles. The author and the reviewer are often the same person who just needs one more approval to merge.

This is a massive improvement in accessibility, speed, and developer experience. It is also a massive regression in defect detection.

Multiple studies found that informal review catches roughly half the defects that structured inspection catches. A 2009 experiment by Basili and others compared Fagan-style inspection with lightweight review and found that the lighter process caught significantly fewer defects in the same material.

The problem is not that reviewers are lazy. The process is not designed to find defects. It is designed to let people ship code with a second pair of eyes on it, which is not the same thing.

What We Actually Lost

Pull request review optimizes for throughput. Fagan inspection optimized for thoroughness. These are genuinely different goals, and neither is wrong. The mistake is assuming that the lighter process is a strict superset of the heavier one.

Here is what disappeared:

Independent preparation. In a pull request, the reviewer sees the diff cold. They have not spent an hour reading surrounding context, tracing data flow, and building a mental model. They are reacting to a notification. The depth of examination is not comparable.

The reader role. Having someone narrate code aloud forces the group to proceed at a pace the slowest person can follow. It surfaces assumptions that silent reading hides. A diff on a screen lets your eyes skip the boring parts. A reader does not skip.

Defect-only focus. Pull request comments drift into style and architecture opinions. These are valuable, but they are not defect detection. Every minute spent debating indentation is a minute not spent finding a null dereference.

Measurable process data. Fagan inspections produced hard numbers: defects per hour, preparation time, rework time, defect density by module. Modern review tools count comments and approvals, which tell you almost nothing about review quality.

A Practical Middle Ground

You are not going to run full Fagan inspections in a modern continuous-deployment environment. But you can borrow the parts that matter.

The most important transferable idea is structured independent preparation. Before a deep async review, require reviewers to spend time with the material alone. Not a quick skim. Actual preparation.

Instead of a generic “LGTM,” enforce a lightweight checklist that mimics the discipline Fagan built into roles and rules:

from dataclasses import dataclass, field
from typing import List, Optional
from enum import Enum

class DefectSeverity(Enum):
    MINOR = "minor"
    MAJOR = "major"
    CRITICAL = "critical"

@dataclass
class ReviewEntry:
    line_number: Optional[int]
    category: str
    severity: DefectSeverity
    description: str

@dataclass
class InspectionReport:
    reviewer: str
    prep_time_minutes: int
    entries: List[ReviewEntry] = field(default_factory=list)

    def defect_count(self) -> int:
        return len(self.entries)

def run_inspection_checklist(
    code: str,
    reviewer: str,
    prep_time_minutes: int
) -> InspectionReport:
    """Structured prep produces structured output.

    Mimics the Fagan prep phase: reviewer spends focused
    time with the material, then logs findings against a
    consistent taxonomy instead of ad hoc comments.
    """
    report = InspectionReport(
        reviewer=reviewer,
        prep_time_minutes=prep_time_minutes
    )

    # Example: check for missing null handling
    if "->" in code and "null" not in code.lower():
        report.entries.append(ReviewEntry(
            line_number=None,
            category="null-safety",
            severity=DefectSeverity.MAJOR,
            description="No explicit null handling in pointer function"
        ))

    return report

This is not a Fagan inspection. It is a way to recover one of its most important properties: review output should be structured, measurable, and focused on defect categories rather than opinions.

Another transferable idea is the time-boxed deep review. Pick one critical module per sprint. Schedule a 90-minute focused review with independent preparation. Log defects only. No solutions, no style debates, no design arguments.

The cost is real. But if the NASA ratio holds even roughly, one focused hour now saves dozens of hours of debugging later.

The Uncomfortable Truth

Fagan inspections did not fail. They were rejected because the rigor they required was incompatible with the speed the industry prioritized.

That tradeoff made sense for a lot of software. A typo in a landing page button does not need a five-person formal inspection. But the culture that replaced Fagan inspections treats all code the same way, and that is where the cost hides.

The most expensive defects are in the code that looks correct, passes tests, and fails in production in ways that cost real money. That is exactly the code that benefits most from a process designed to find defects instead of a process designed to approve diffs.

Pull request review is here to stay, and that is fine. But pretending it is a replacement for structured inspection is not fine. It is a different tool for a different job, and teams that only own one of them are going to keep finding expensive bugs that a better process would have caught before shipping.

FAQ

What is a Fagan inspection?

A structured, multi-phase review process for finding defects in software artifacts, developed by Michael Fagan at IBM in the 1970s. It involves six phases (planning, overview, preparation, inspection meeting, rework, follow-up) with specific roles for participants. The meeting focuses exclusively on logging defects, not solving them.

How effective were Fagan inspections?

IBM reported defect removal rates over 90 percent. A 2002 NASA study found each hour of inspection prevented an average of 33 hours of maintenance. Independent studies consistently found structured inspection catches roughly twice as many defects as informal review.

Why did teams stop using Fagan inspections?

The process consumed 15 to 20 percent of total project effort, required difficult scheduling of multiple participants, and was culturally rigid. As software teams moved toward faster release cycles, the overhead became unsustainable. Pull request review replaced it as the default because it is faster and easier to integrate into normal workflow, even though it catches fewer defects.

Can modern teams still benefit from Fagan inspections?

Not in the original form. The full six-phase process with assigned roles does not fit continuous deployment. But the core ideas, independent preparation, time-boxed focused review, structured defect logging, and separating defect finding from solution design, can be adapted. Teams that apply these selectively to critical code get much of the benefit without the overhead.