Technical Performance · Index
Thank you for reading this post, don't forget to subscribe!
Home / Indexes / Technical Performance / Model Benchmarks 2026
AI Model Benchmarks 2026
AI capability is improving faster than benchmarks can keep up — SWE-bench, MMLU, GPQA and the frontier-model race, in primary-sourced numbers. Every figure is linked to its source and dated.
By Report AI · Updated 11 Nov 2026 · 7 min read · 9 sourced figures
| Figure | Was | Now | Change | Why |
|---|---|---|---|---|
| Top SWE-bench Verified score, 2026 (stat tile) | 95%+, credited to Stanford HAI | 95%+ (97.0% top raw score, Vals.ai, 1 Sep 2026; BenchLM 96%, 29 Jul 2026) | CORRECTION source corrected |
Stanford HAI’s 2026 report expresses SWE-bench progress as a share of the human baseline; the raw 95–97% scores come from the Vals.ai and BenchLM leaderboards. MEDIUM |
| SWE-bench, 2023 → 2024 | 4.4% → 71.7% on “SWE-bench” | 4.4% on the original full SWE-bench (2023); 71.7% on SWE-bench Verified (OpenAI o3 preview, Dec 2024) | CORRECTION versions labelled |
The two endpoints are different test versions, and 71.7% was a pre-release o3 score; each is now stated with its version and date, and the +67.3 pp gain Stanford HAI reports is labelled as spanning both. MEDIUM |
Correction, 11 November 2026: the 2026 stat tile credited leaderboard scores to the wrong source, and the 2023–2024 SWE-bench comparison paired two different test versions without saying so; both are corrected in the tiles, takeaways, analysis, tables, FAQ and structured data. We publish the previous value alongside the new one so any figure can be audited over time; this correction is logged in our corrections log.
Key takeaways
- In 2023 the best model solved 4.4% of the original full SWE-bench coding tasks; in December 2024 OpenAI’s o3 preview, not yet released, scored 71.7% on SWE-bench Verified, a 500-task human-validated subset. Stanford HAI reports the pair as a +67.3 pp one-year gain, but the two endpoints are different test versions. (Stanford HAI AI Index 2025)
- MMLU is now saturated above 92%, near its ~95% ceiling — pushing labs to harder tests like HLE and FrontierMath. (Stanford HAI)
- The US-China frontier gap is razor-thin: the top U.S. model leads by just 2.7% as of March 2026. (Stanford HAI)
- The closed-weight lead on Chatbot Arena narrowed from 8.0% (Jan 2024) to 1.7% (Feb 2025).
Capability is outpacing the benchmarks
AI model capability is improving faster than benchmarks can keep up. The best model of 2023 solved 4.4% of the original full SWE-bench coding tasks, while OpenAI’s o3 preview scored 71.7% on SWE-bench Verified, a 500-task human-validated subset released in August 2024, when the score was announced in December 2024 before o3 was generally available (Stanford HAI, AI Index 2025); because the two figures come from different test versions, they are not a like-for-like comparison. MMLU is now effectively saturated above 92%, and the gap between the top U.S. and Chinese models has stayed in the single digits all year (Stanford HAI 2026). In a single year (2023–2024), scores rose 18.8, 48.9, and 67.3 percentage points on MMMU, GPQA, and SWE-bench respectively; the SWE-bench gain compares the full test in 2023 with SWE-bench Verified in 2024.
MMLU launched in 2020 near 32% frontier accuracy; by Q1 2026 every frontier system scores above 92%, near the benchmark’s ~95% ceiling. As classic benchmarks saturate, the open question is no longer whether models can pass the test, but which test is hard enough to still discriminate between them.
Deep dive: why benchmarks keep changing
As classic benchmarks saturate, frontier labs have moved to harder tests. The benchmarks model cards actually report in 2026 include Humanity’s Last Exam (HLE), FrontierMath, ARC-AGI-2, GPQA Diamond, SWE-bench Verified, AIME 2025, and τ-bench. Even these are falling quickly — frontier models gained roughly 30 percentage points on HLE in a single year, a test explicitly designed to be hard for AI (Stanford HAI 2026).
| Benchmark | One-year gain (pp) |
|---|---|
| SWE-bench (full 2023 → Verified 2024) | +67.3 |
| GPQA | +48.9 |
| MMMU | +18.8 |
One-year benchmark gains, 2023→2024 · Source: Stanford HAI · percentage points
The competitive landscape
The frontier is close and contested. In February 2025, China’s DeepSeek-R1 briefly matched the top U.S. model; since then the lead has changed hands repeatedly while staying in the single digits. As of March 2026 the top U.S. model leads by 2.7%. The open-vs-closed gap has also compressed — from 8.0% in January 2024 to 1.7% by February 2025 on the Chatbot Arena Leaderboard (Stanford HAI 2026).
The numbers in full
| Metric | Figure | Source |
|---|---|---|
| SWE-bench (coding): full SWE-bench 2023 → SWE-bench Verified, Dec 2024 (o3 preview) | 4.4% → 71.7% | Stanford HAI |
| GPQA one-year gain | +48.9 pp | Stanford HAI |
| MMMU one-year gain | +18.8 pp | Stanford HAI |
| Top US lead over China (Mar 2026) | 2.7% | Stanford HAI 2026 |
| Closed–open gap (Jan 2024) | 8.0% | Stanford HAI 2026 |
| Closed–open gap (Feb 2025) | 1.7% | Stanford HAI 2026 |
| HLE one-year gain | +30 pp | Stanford HAI 2026 |
| Inference cost decline (late ’22–late ’24) | ~280× | Stanford HAI 2025 |
Figures reflect each Stanford HAI AI Index release; one-year benchmark gains are for 2023–2024.
Both tables on this page as CSV · 30 data cells, one row per cell, with the same sources, dates and confidence ratings shown here. Please cite Report AI and link to this page. How we source and rate figures.
Frequently asked
What is the best AI benchmark in 2026?
There is no single benchmark. As MMLU and similar tests saturate above 92%, frontier labs now report on harder ones — Humanity’s Last Exam (HLE), FrontierMath, ARC-AGI-2, GPQA Diamond, and SWE-bench Verified, among others. (Stanford HAI)
How fast is AI improving on benchmarks?
Very fast. In 2023–2024 alone, coding scores went from 4.4% on the original full SWE-bench (2023) to 71.7% on SWE-bench Verified (OpenAI’s o3 preview, December 2024), a +67.3 pp gain that Stanford HAI reports across two test versions, and frontier models have since gained about 30 points in a year on Humanity’s Last Exam. (Stanford HAI)
Are US or Chinese AI models better?
They are very close. The lead has changed hands repeatedly since early 2025; as of March 2026 the top U.S. model leads China’s best by about 2.7%, per Stanford HAI. (Stanford HAI)
Primary sources
• Stanford HAI — AI Index 2026, Technical Performance (MMLU saturation · US-China gap · HLE) — hai.stanford.edu
• Vals.ai — SWE-bench Verified leaderboard (top raw score 97.0%, Claude Opus 5, 1 Sep 2026) — vals.ai