3 posts
Differential testing lets you find bugs without knowing the correct answer. The problem is that correlated mistakes look like agreement.
You can trust differential testing without a formal proof, but only if you understand exactly where it falls apart. The weakness is called common-mode failure.…
Running multiple LLMs in parallel catches errors that any single model will confidently ship.
You ship a prompt to GPT-4o. It returns a JSON blob with a confidence of 0.97. You ship the same prompt to Claude 3.5 Sonnet. It returns a different JSON blob,…
N-version programming sounds simple: run multiple implementations and pick the best answer. In practice, 'best' is a lot harder to define than 'most common.'
You have five implementations of the same function. Three return the same result. One is slightly different. One throws an exception. Which one is right? Most…