The results
| Reviewer | Overall | Real bugs caught | Comments valid |
|---|---|---|---|
| Drape Deep Review | 70% | 89.1% | 99.1% |
| Drape Review | 61% | 81.8% | 98.9% |
| Cursor Bugbot | 54% | 74.5% | 98.8% |
| Claude Code (Opus 5.5) | 53% | 79.3% | 92.6% |
| Greptile | 39% | 62.8% | 95.6% |
| Cubic | 38% | 62.4% | 90.6% |
| Qodo | 37% | 56.1% | 94.6% |
crbench leaderboard, November 2026. Overall weighs bugs caught by severity and charges for false alarms. This table lists the reviewers you can buy; the full leaderboard also has research runs.
Big pull requests and cross-file bugs separate reviewers
Most AI reviewers look good on small pull requests. On large ones, and on bugs whose cause sits in a file the diff doesn’t show, they miss real bugs. See how Drape reviews a pull request →
How crbench is built
Real pull requests — and a bug only counts when something proves it.
crbench starts from real pull requests, many of them collected by Martian’s Code Review Bench — we’re grateful for the work that went into it. On top of those, crbench sets a higher bar for what counts as a bug.
Every defect needs proof: a fix or revert that landed after the change merged, independent corroboration, or an audit. Any valid finding earns credit, not only the ones on an answer key, so a reviewer that finds a real bug nobody listed isn’t scored as noise.
Every label we dropped is published with its reason, and anyone can submit a reviewer and re-run the scoring.
What doesn’t count as a bug
- A call that compiles. If the merged PR builds, “won’t compile” isn’t a defect.
- A change the language allows. If
tsc --strictpasses, it isn’t an interface break. - An import the same PR defines. The class exists, so the import resolves.
- A null-safe comparison. Comparing a missing value with
!=can’t throw. - A style preference. “Consider extracting a constant” is advice, not a bug.