An LLM Can Pre-Inspect Your Code. It Cannot Run the Meeting.

The Short Answer Is No, but the Long Answer Saves You Hours

A full Fagan inspection needs a moderator, a reader, two to four inspectors, and the author. The team spends two hours reviewing roughly 250 lines of code at 125 lines per hour. That is eight to twelve person-hours for a small change.

An LLM can read 250 lines in under a second. It can run a checklist, flag suspicious patterns, and generate a defect log before any human opens the file.

That does not mean it can replace the team. It means the team should not start the meeting until the LLM has finished its homework.

What an LLM Can Actually Do in a Fagan Inspection

Fagan inspections have four distinct roles. Each role has a job that maps differently to what an LLM can and cannot do.

The moderator plans the inspection, enforces entry criteria, and keeps the meeting on track. An LLM can generate a checklist, verify that tests pass before inspection, and estimate time based on line count. It cannot tell whether a room full of engineers is drifting into a design debate. The moderator role is social, and LLMs are not social.

The reader paraphrases the code aloud during the meeting to force the group to process logic at comprehension speed instead of skimming speed. An LLM can summarize code, but it cannot make a room of humans slow down. The reader’s real value is social enforcement. The LLM has no leverage.

The inspectors find defects by examining the code against checklists and domain knowledge. This is where the LLM is most useful. It can check for null dereferences, resource leaks, unhandled error paths, and off-by-one errors faster than any human. It does not get tired. It does not skip the boring utility functions because they look safe.

The author answers questions and fixes defects. The LLM cannot replace the author because it did not write the code and it cannot verify intent.

So the realistic answer is not replacement. It is role substitution. The LLM can serve as a tireless pre-inspector that handles the mechanical parts of the inspector role before the humans enter the room.

Where LLMs Actually Help: The Preparation Phase

The most important phase of a Fagan inspection is preparation. Every inspector reviews the material alone before the meeting. Studies show that preparation finds roughly half the defects discovered in the full process. If the LLM can supercharge that phase, the human meeting becomes shorter and sharper.

Here is a practical approach. Feed the LLM the code, a checklist tailored to your language, and a prompt that forces structured output. Use the LLM output as the starting point for human inspectors, not as a replacement.

The prompt matters more than the model. A vague request like “review this code” will give you generic platitudes. A structured prompt with a checklist and output format will give you actionable defects.

Below is a Python script that runs an LLM-based pre-inspection using an OpenAI-compatible API. It reads a source file, applies a Fagan-style checklist, and outputs a structured defect log.

#!/usr/bin/env python3
"""
LLM pre-inspection for Fagan-style review.
Produces structured defect log from a checklist.
"""

import argparse
import os
from pathlib import Path


def build_prompt(code: str, filepath: str, language: str) -> str:
    checklist = {
        "python": [
            "Missing null/None checks before dereferencing",
            "Unclosed file handles or missing context managers",
            "Bare except clauses that swallow exceptions",
            "Mutable default arguments in function signatures",
            "Race conditions in shared mutable state",
            "Missing input validation on public functions",
        ],
        "typescript": [
            "Non-null assertions (e.g., !) without justification",
            "Missing await on async calls",
            "Any types that should be narrowed",
            "Unhandled promise rejections",
            "Missing input validation on public functions",
        ],
        "rust": [
            "Unwrap or expect without comment explaining invariants",
            "Missing error propagation where Result is returned",
            "Unsafe blocks without safety comments",
            "Clone on large data structures in hot paths",
            "Missing input validation on public functions",
        ],
    }

    items = checklist.get(language, checklist["python"])
    checklist_text = "\n".join(f"  - {item}" for item in items)

    return (
        "You are a Fagan inspection assistant. Review the following code "
        "against the checklist. For each defect found, output a JSON object "
        "with keys: line, category, severity (minor/major/critical), description. "
        "If no defect is found for a checklist item, omit it. "
        "Be specific about line numbers and exact variable names. "
        "Do not suggest fixes. Fagan inspections only log defects, they do not solve them.\n\n"
        f"File: {filepath}\n"
        f"Language: {language}\n\n"
        "Checklist:\n"
        f"{checklist_text}\n\n"
        "Code:\n```\n"
        f"{code}\n"
        "```\n\n"
        "Output strict JSON array of defects only. No markdown, no preamble."
    )


def inspect_file(filepath: str, api_key: str, base_url: str) -> list:
    """Run LLM pre-inspection and return parsed defect list."""
    from openai import OpenAI

    path = Path(filepath)
    code = path.read_text()

    # Infer language from extension
    ext_map = {
        ".py": "python",
        ".ts": "typescript",
        ".tsx": "typescript",
        ".rs": "rust",
    }
    language = ext_map.get(path.suffix, "python")

    client = OpenAI(api_key=api_key, base_url=base_url)
    response = client.chat.completions.create(
        model=os.environ.get("INSPECTION_MODEL", "gpt-4.1-mini"),
        messages=[
            {"role": "system", "content": "You are a precise code inspection tool."},
            {"role": "user", "content": build_prompt(code, filepath, language)},
        ],
        temperature=0.2,
        max_tokens=2048,
    )

    raw = response.choices[0].message.content.strip()

    # Some models wrap JSON in markdown fences. Strip them.
    if raw.startswith("```json"):
        raw = raw[7:]
    if raw.startswith("```"):
        raw = raw[3:]
    if raw.endswith("```"):
        raw = raw[:-3]
    raw = raw.strip()

    import json
    try:
        defects = json.loads(raw)
        if not isinstance(defects, list):
            return []
        return defects
    except json.JSONDecodeError:
        print("Warning: LLM returned non-JSON output:")
        print(raw[:500])
        return []


def main():
    parser = argparse.ArgumentParser(description="LLM pre-inspection for Fagan review")
    parser.add_argument("files", nargs="+", help="Source files to pre-inspect")
    parser.add_argument("--api-key", default=os.environ.get("OPENAI_API_KEY"))
    parser.add_argument(
        "--base-url",
        default=os.environ.get("OPENAI_BASE_URL", "https://api.openai.com/v1"),
    )
    args = parser.parse_args()

    if not args.api_key:
        raise SystemExit("Error: set OPENAI_API_KEY or pass --api-key")

    all_defects = []
    for f in args.files:
        defects = inspect_file(f, args.api_key, args.base_url)
        all_defects.extend({"file": f, **d} for d in defects)

    if not all_defects:
        print("No defects flagged by LLM pre-inspection.")
        return

    print(f"LLM pre-inspection found {len(all_defects)} defect(s):\n")
    for d in all_defects:
        print(
            f"[{d['severity'].upper()}] {d['file']}:{d.get('line', '?')} "
            f"({d['category']}) {d['description']}"
        )


if __name__ == "__main__":
    main()

Install the OpenAI SDK with pip install openai, set your API key, and run python inspect.py src/auth.py. The script outputs a structured defect log that human inspectors can use as their starting checklist.

This is not the inspection. It is the preparation. The defects still need human judgment.

What LLMs Miss: The Defects That Cost Real Money

The most expensive defects are not syntax errors. They are wrong assumptions.

An LLM will catch the missing null check. It will not catch that the null case should never happen because the upstream service guarantees the field, and the real bug is that your team changed the contract three weeks ago and forgot to update the downstream code.

An LLM will flag an unhandled exception. It will not notice that the exception handler is missing because the author assumed the database transaction would roll back automatically, and that assumption is false for the specific isolation level your team uses.

These are intent-level defects. They require domain knowledge, historical context, and the ability to ask “what did the author assume that is not true?” An LLM has no access to your team’s undocumented assumptions. It has no memory of the architecture decision from last quarter. It cannot look at a requirement document and notice the code implements the wrong requirement.

Fagan inspections were designed to catch exactly these defects. The human inspectors bring different mental models and different contexts. When they disagree about what the code should do, the gap between their assumptions is where the real bugs live.

An LLM has one mental model, and it is trained on public code, not your codebase.

The Real Trade-Off: Coverage vs. Comprehension

LLM pre-inspection gives you coverage. It reads every line, checks every function, and never gets distracted by a Slack notification. It is the perfect narrow inspector.

Human inspection gives you comprehension. It connects the code to the spec, the spec to the database schema, and the schema to the business rule that changed in the last sprint planning meeting. It is the only way to find defects that live outside the code.

The trade-off is not human versus machine. It is deep versus wide.

If you replace your inspection team with an LLM, you will catch more syntax-level defects and miss more intent-level defects. The net result depends on what kind of bugs are killing you. If your production incidents are mostly null dereferences and resource leaks, an LLM will help. If your incidents are mostly wrong business logic and missed edge cases, an LLM will give you false confidence.

How to Run a Hybrid Inspection in Practice

Here is a practical workflow that uses the LLM for what it is good at and preserves the human team for what only humans can do.

Before the meeting, run the LLM pre-inspection script on the code. Give every human inspector the output plus the original code. Require them to verify or dismiss each LLM finding during their individual preparation. This forces them to examine lines they might have skimmed.

During the meeting, the reader still paraphrases the code aloud. The inspectors still raise issues. But now they start with a curated list of mechanical defects already triaged. The meeting time can shrink from two hours to ninety minutes, or the team can review more code in the same window.

The moderator adds one new responsibility: track LLM accuracy. Log how many LLM-flagged defects were real, how many were false positives, and how many human-only defects the LLM missed. After three or four inspections, you will know whether your checklist prompt needs tuning.

This is the same process NASA used when they experimented with tool-assisted inspection in the 1990s. Automated checkers found the easy defects and left the hard ones for humans. The humans performed better because they were not mentally exhausted from finding the easy ones.

The Bottom Line

An LLM cannot replace a full inspection team because inspection is not just reading code. It is a structured social process designed to surface gaps between what the author assumed and what the code actually does.

What an LLM can do is make the human team more effective by removing the mechanical load. It can play a fifth role: the tireless pre-inspector that prepares the battlefield before the humans arrive.

If you are skipping inspections because they are too expensive, an LLM does not make them free. It makes them shorter. That is often enough.

FAQ

What is a Fagan inspection?

A structured, six-phase review process invented by Michael Fagan at IBM in 1976. It uses defined roles (moderator, reader, inspectors, author), mandatory individual preparation, and timeboxed meetings to find defects before testing. It consistently reports 60 to 90 percent defect removal rates.

Can an LLM replace a human code reviewer?

Not for the defects that matter most. An LLM catches mechanical issues like missing null checks and resource leaks. It misses intent-level defects like wrong assumptions about upstream contracts, missing business logic, and mismatches between code and requirements.

How do you integrate an LLM into a Fagan inspection?

Use the LLM during the preparation phase. Run it against a checklist, feed its output to human inspectors before the meeting, and require them to verify each finding. This shortens the meeting and improves human focus without removing human judgment.

What makes an LLM checklist prompt effective?

Specificity. Generic prompts produce generic output. Effective prompts name exact defect categories, require structured JSON output, forbid solution proposals, and ask for exact line numbers and variable names. The checklist should match your language and your team’s most common escaped defects.

Try It on One File

Pick a file that caused a production incident in the last month. Write a checklist of five specific defect types your team cares about. Run the script above with that checklist. Give the output to two engineers who did not write the file.

Compare the LLM’s findings with the human findings. Count the overlap. Count what each side missed. That gap is your answer.