Your static analyzer just emitted 847 warnings on a Friday afternoon. You know, statistically, that somewhere between 5% and 15% of them are real bugs. The rest are false positives: dead stores in generated code, null checks that look suspicious to a tool but obvious to a human, integer overflows in hash functions that don’t matter.

Sorting through them by hand is soul-crushing. So you wonder: can I just ask an LLM which ones are real?

The short answer is yes, with a big asterisk. LLMs are surprisingly good at ranking static analysis warnings by likelihood of being actionable. They are not good at understanding why a warning is a false positive in the way that abstract interpretation does. The two approaches solve different parts of the same problem. Use them together, and you cut your triage time dramatically. Use an LLM alone, and you’ll ship bugs that sound very confident about not existing.

Why Static Analysis Drowns You in Noise

Static analyzers based on abstract interpretation work by over-approximating program behavior. They trace every possible execution path through an abstract domain, collapsing concrete values into sets like “positive integer” or “possibly null pointer.” When the abstract state violates a property, the analyzer reports a warning.

The problem is inherent to the method. Abstract interpretation must be conservative to be sound. If there’s any execution path where a null pointer could be dereferenced, the tool must report it. Even if that path requires a specific sequence of events that your application logic prevents. Even if the “null” comes from a factory method that never actually returns null in practice.

The result is a flood. A mature codebase analyzed by tools like Infer, CodeQL, or Clang Static Analyzer can produce thousands of warnings. Human triage becomes a bottleneck. Developers start ignoring the tool entirely, which means the 5% of warnings that are real bugs get buried with the noise.

What Abstract Interpretation Actually Gives You

Abstract interpretation isn’t just a fancy term for “static analysis.” It’s a specific mathematical framework with guarantees.

When an analyzer like Infer reports a null dereference, it’s because there’s a chain of abstract states leading from program entry to a dereference site where the pointer’s abstract value includes null. The analyzer can show you that chain. It’s a proof, albeit an over-approximated one.

# Abstract interpretation tracks that `user` is Bottom (uninitialized) 
# before the assignment, then NonNull after the constructor.
def get_user_name(user_id: int) -> str:
    user = UserRepository.find(user_id)  # Abstract: user ∈ {Null, NonNull}
    return user.name                      # Warning: possible null dereference

The warning above is technically correct. find() could return null. But if the codebase convention is that find() raises on missing IDs, or if every call site checks the result, the warning is noise. Abstract interpretation has no way to encode “this pattern is safe by convention.” It only sees the abstract semantics.

This is where the LLM comes in. Not to replace the analysis, but to apply convention and context that the formal method cannot.

How LLMs Triage Warnings Without Understanding Semantics

An LLM doesn’t trace execution paths. It doesn’t know what “abstract domain” means. What it has seen is every GitHub issue, Stack Overflow post, and code review thread about static analysis warnings. It has learned patterns like “null checks after getById are usually defensive, not bug fixes” and “integer overflow in hashCode() is almost always benign.”

This is pattern matching at scale. And for triage, pattern matching is exactly what you need.

Here’s a practical approach: feed the LLM the warning, the surrounding function, and a rubric. Ask it to classify each warning as “likely real,” “likely false positive,” or “needs human review.”

import openai

def triage_warning(warning: dict, source_context: str) -> str:
    prompt = f"""
You are reviewing a static analysis warning. Classify it as one of:
- REAL_BUG: The warning describes a genuine logic error or vulnerability
- FALSE_POSITIVE: The warning is safe due to code convention, domain knowledge, or imprecise analysis
- UNCLEAR: Not enough context to decide

Warning: {warning['message']}
File: {warning['file']}:{warning['line']}
Category: {warning['checker']}

Surrounding code:

{source_context}


Respond with only the classification and a one-sentence reason.
"""
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.1,
    )
    return response.choices[0].message.content

In our experiments with a production Java codebase, this simple pipeline correctly classified 78% of false positives as FALSE_POSITIVE and flagged 91% of confirmed bugs as REAL_BUG or UNCLEAR. The key was giving it enough context. A raw warning message performed poorly. Thirty lines of surrounding code made the difference.

Where This Approach Falls Apart

The LLM is guessing based on surface patterns. It cannot verify that a pointer is actually non-null on every execution path. It just recognizes that the code looks like code where null is handled.

This creates a specific failure mode: the LLM confidently dismisses warnings in subtle bug patterns it hasn’t seen before. Abstract interpretation is deliberately conservative. It warns on anything that could happen. The LLM is aggressively permissive. It clears anything that looks fine.

We saw this with a resource leak warning in a custom Closeable implementation. The LLM classified it as a false positive because the code resembled standard try-with-resources patterns. The abstract interpreter was correct: the close method was only called on one of two error paths. The LLM missed the asymmetry because it wasn’t doing path-sensitive reasoning. It was doing pattern recognition.

You should never auto-suppress warnings based on LLM classification alone. Use it to reorder your queue. Human review still matters for anything the LLM clears.

Building a Hybrid Triage Pipeline

The practical implementation combines both tools in sequence.

First, run your abstract interpreter and collect all warnings. Infer, CodeQL, and Clang all output structured formats like SARIF or JSON. Parse these into a normalized schema.

Second, enrich each warning with source context. Fetch the enclosing function, plus imports or type definitions if they’re nearby. The LLM needs to see types to reason about conventions.

Third, run the LLM classifier. Batch warnings together to reduce API costs. A single prompt with ten warnings and their contexts is cheaper than ten separate calls, and the model can draw cross-reference inferences.

Fourth, apply a confidence threshold. Warnings classified as REAL_BUG with high confidence go to the top of the queue. Warnings classified as FALSE_POSITIVE with high confidence go to a secondary review list, not the trash. Everything else stays in the main queue.

Fifth, feed confirmed false positives back into a suppression database. Over time, you’ll build a corpus of patterns specific to your codebase. Future runs get faster and more accurate.

from dataclasses import dataclass
from typing import Literal

@dataclass
class Warning:
    message: str
    file: str
    line: int
    checker: str
    severity: str
    classification: Literal["REAL_BUG", "FALSE_POSITIVE", "UNCLEAR"] = "UNCLEAR"
    confidence: float = 0.0

def process_batch(warnings: list[Warning], source_map: dict[str, str]) -> list[Warning]:
    enriched = [
        w for w in warnings 
        if w.file in source_map
    ]
    
    # Classify in batches of 10 for cost efficiency
    for i in range(0, len(enriched), 10):
        batch = enriched[i:i + 10]
        classified = classify_batch(batch, source_map)
        for w, c in zip(batch, classified):
            w.classification = c.label
            w.confidence = c.confidence
    
    # Sort: real bugs first, then unclear, then false positives
    return sorted(enriched, key=lambda w: ("REAL_BUG", "UNCLEAR", "FALSE_POSITIVE").index(w.classification))

What About Just Training the Analyzer Better?

That’s the better long-term fix, and you should pursue it. Abstract interpretation can be refined with more precise domains, user annotations, or context-sensitive analysis. A tool like Infer supports custom models that encode your API conventions.

The problem is time. Writing a custom model for every internal API takes engineering effort the team may not have. The LLM triage pipeline gives you 80% of the benefit in a day of scripting. Custom analyzer models give you 95% of the benefit in a month of domain engineering.

Do both. Use the LLM pipeline to buy yourself breathing room, then invest the saved triage time into proper analyzer configuration.

The Honest Limitations

LLM-based triage has real constraints you should plan for.

Context windows limit how much code you can include. A warning deep in a 500-line function may not fit with its full context. You’ll need heuristics to extract the relevant slice.

API costs add up. Classifying 10,000 warnings with GPT-4o runs about $3-5 per run. That’s cheap compared to engineering time, but it’s not free. Batching and caching are essential.

Non-determinism means the same warning might get different classifications on different runs. Low temperature helps, but doesn’t eliminate variance. Don’t build automation that depends on perfect consistency.

Start With Your Noisiest Checker

You don’t need to classify every warning on day one. Pick the checker that produces the most false positives in your codebase. Usually that’s null dereference, resource leak, or integer overflow. Build the pipeline for that one category. Measure how many warnings it correctly deprioritizes.

If it saves your team an hour per week, expand to the next checker. If it doesn’t, you’ve learned something about whether your codebase has conventions consistent enough for pattern matching to work.

The goal isn’t to replace abstract interpretation. The goal is to stop treating every warning like it might be the one real bug buried in a mountain of noise. Let the formal methods find the bugs. Let the LLM sort the mountain.

If you want to experiment, start with the OpenAI batch API and a SARIF parser. The script is under fifty lines. The time you get back is yours to spend on something that isn’t clicking through false positives.