The frontier claims register

ModelBenchmark and date

Model details

OpenAI · released 2026-09 · Closed · proprietary

ARC Prize and Scale SEAL had not published results for this model as of 30 Sep 2026.

Recorded result

BenchmarkDeepSWE v1.1

Reported result75.2%

Reported date2026-09-29

Self-reported by the lab

Read the primary source ↗

All sourced claims

JavaScript is off, so the complete archived register is shown below.

GPT-6.1 Sol

OpenAI2026-09closed (proprietary)

4 claimed · none independently checked

DeepSWE v1.175.2%Self-reported · no independent checkhigh effort; OpenAI chartsource
OSWorld 2.071.4%Self-reported · no independent checkmax effort; v2026.08.08 offline set, partial rewardsource
AutomationBench36.1%Self-reported · no independent checkmax effort; AutomationBench 1.0.6source
Terminal-Bench-Science 0.157%Self-reported · no independent checkmax effort; $5.47 per tasksource

Claude Sonnet 5.5

Anthropic2026-09closed (proprietary)

9 claimed · none independently checked

SWE-bench Pro81.3%Self-reported · no independent checkadaptive thinking, max effort; average of 5 trials; Anthropic harness, not the Scale SEAL leaderboardsource
SWE-bench Multilingual90.3%Self-reported · no independent checkadaptive thinking, max effort; average of 5 trialssource
SWE-bench Multimodal54.3%Self-reported · no independent checkadaptive thinking, max effort; average of 5 trialssource
DeepSWE v1.171%Self-reported · no independent check113 tasks; average of 5 trialssource
FrontierCode v1.1 Main46.2%Self-reported · no independent checkmean@5source
Terminal-Bench 4.070.6%Self-reported · no independent checksafeguards on; about 1.2% of requests answered by a fallback modelsource
HLE (no tools)56.9%Self-reported · no independent checkthinking auto; Anthropic harness with Opus 4.6 as grader; not the Scale SEAL leaderboardsource
HLE (Search + Code)64.5%Self-reported · no independent checkweb search and fetch, programmatic tool calling, code executionsource
AutomationBench44.7%Self-reported · no independent checkadaptive thinking, max effort; average of 5 trialssource

GPT-6 Sol

OpenAI2026-09closed (proprietary)

7 claimed · 3 independently checked

ARC-AGI-195.5%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-289.6%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-34.62%Checked by ARC Prize Foundationmax reasoning effort; semi-private; Standard harness (23.0 with the Provider Adapter harness)source
DeepSWE v1.168.8%Self-reported · no independent checkmax effortsource
OSWorld 2.064.4%Self-reported · no independent checkmax effort; v2026.08.08 offline set, partial reward; from OpenAI's GPT-6.1 Sol postsource
AutomationBench33.2%Self-reported · no independent checkxhigh effort; $0.27 per tasksource
Agents' Last Exam56.4%Self-reported · no independent checkmax effort; Agents' Last Exam V1source

GPT-6 Luna

OpenAI2026-09closed (proprietary)

4 claimed · 3 independently checked

ARC-AGI-186.7%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-259.3%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-30.2%Checked by ARC Prize Foundationmedium reasoning effort; semi-private; Standard harness (0.6 with the Provider Adapter harness)source
DeepSWE v1.166.6%Self-reported · no independent checkmax effortsource

Claude Opus 5.5

Anthropic2026-09closed (proprietary)

11 claimed · 2 independently checked

ARC-AGI-198.5%Checked by ARC Prize Foundationhigh reasoning effort; semi-privatesource
ARC-AGI-293.3%Checked by ARC Prize Foundationhigh reasoning effort; semi-private (91.7 at max)source
SWE-bench Pro89.9%Self-reported · no independent checkadaptive thinking, max effort; average of 5 trials; Anthropic harness, not the Scale SEAL leaderboardsource
SWE-bench Multilingual93.9%Self-reported · no independent check300 problems, 9 languages; adaptive thinking max; average of 5 trialssource
SWE-bench Multimodal61.4%Self-reported · no independent checkadaptive thinking max; average of 5 trialssource
DeepSWE v1.174.2%Self-reported · no independent check113 tasks; adaptive thinking max; average of 5 trialssource
FrontierCode v1.1 Main54.4%Self-reported · no independent checkadaptive thinking max; mean@5source
Terminal-Bench 4.066.4%Self-reported · no independent checkxhigh effortsource
HLE (no tools)64.4%Self-reported · no independent checkthinking auto; Anthropic harness with Opus 4.6 as grader; not the Scale SEAL leaderboardsource
HLE (Search + Code)67.7%Self-reported · no independent checkweb search and fetch, programmatic tool calling, code execution; confirmed cheating re-graded as wrongsource
OSWorld 2.081.8 % (partial credit)Self-reported · no independent checkpartial credit; 48.7 with strict scoringsource

GPT-6 Astra

OpenAI2026-09closed (proprietary)

15 claimed · 5 independently checked

ARC-AGI-198.5%Checked by ARC Prize Foundationhigh reasoning effort; semi-privatesource
ARC-AGI-295%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-362.71%Checked by ARC Prize Foundationmax reasoning effort; semi-private; Standard harnesssource
ARC-AGI-399.95%Checked by ARC Prize Foundationhigh reasoning effort; semi-private; Provider Adapter harness, which keeps the model's reasoning state; not comparable with Standard-harness resultssource
HLE54.8%Checked by Scale SEALScale SEAL leaderboardsource
Terminal-Bench 4.057.9%Self-reported · no independent checkOpenAI launch tablesource
DeepSWE v1.174.1%Self-reported · no independent checkOpenAI launch tablesource
FrontierCode v1.1 Main53.3%Self-reported · no independent checkOpenAI launch tablesource
FrontierCode v1.1 Extended64.5%Self-reported · no independent checkOpenAI launch tablesource
GPQA Diamond96%Self-reported · no independent checkOpenAI launch tablesource
FrontierMath v2 — Tier 497.6%Self-reported · no independent checkOpenAI launch tablesource
OSWorld 2.072.6%Self-reported · no independent checkv2026.08.08 offline set, partial score; OpenAI's later GPT-6.1 Sol post charts 73.5 at max effortsource
AutomationBench41.4%Self-reported · no independent checkOpenAI launch tablesource
Terminal-Bench-Science 0.164.6%Self-reported · no independent checkOpenAI launch chart, max effort; OpenAI's later GPT-6.1 Sol post gives 68.1source
HLE (Search + Code)57.2%Self-reported · no independent checkOpenAI launch table, 'with tools'; not the Scale SEAL settingsource

Claude Fable 5.1

Anthropic2026-09closed (proprietary)

26 claimed · 4 independently checked

SWE-bench Pro81.2 % resolvedSelf-reported · no independent checkadaptive thinking, max effort; average of 5 trialssource
SWE-bench Multilingual89.1 % resolvedSelf-reported · no independent check300 problems across 9 languages; adaptive thinking max; average of 5 trialssource
SWE-bench Multimodal54.7 % resolvedSelf-reported · no independent checkadaptive thinking max; average of 5 trialssource
DeepSWE v1.167.4%Self-reported · no independent check113 tasks; adaptive thinking max; average of 5 trials; hidden tests can reject valid but non-reference solutionssource
FrontierCode v1.1 Main50.9%Self-reported · no independent checkmedium reasoning; Anthropic comparison table; older Fable 5 is 53.5 at xhigh, so this is not an apples-to-apples regressionsource
FrontierCode v1.1 Extended63.6%Self-reported · no independent checkmedium reasoning; older Fable 5 is 64.9 at xhigh; the card says some out-of-scope edits are counted as failuressource
FrontierSWE v20.57 dominance scoreSelf-reported · no independent checkProximal agent harness; max reasoning; 5 trials per tasksource
Terminal-Bench 4.055.8%Self-reported · no independent checkClaude Code with --bare; max thinking; 15 trials per tasksource
Terminal-Bench-Science 0.152.6%Self-reported · no independent checkClaude Code with --bare; max thinking; 10 trials per task, 700 trials totalsource
CursorBench 3.2.073.4%Self-reported · no independent checkmax effort; 5 runs; Cursor independently reports the benchmark, but no steward check was foundsource
OSWorld 2.077.9 % partial passSelf-reported · no independent checkproduction safeguards active, with Opus 4.8 fallback; default 1080p; max 500 actions; Pass@1, average of 5 runssource
OSWorld 2.041.7 % strict passSelf-reported · no independent checksame production-safeguard setup as the partial score; results supersede older OSWorld figures and are not comparable to earlier task releasessource
OfficeQA Pro69%Self-reported · no independent checkMessages API; extracted text sandbox; code execution; output limit 128K; production safeguards and fallbacksource
Legal Agent Benchmark19.09 % all-passSelf-reported · no independent checkpublic Messages API; adaptive max; internal reimplementation; 1,235 problems, 16 defects excluded; n=5source
Legal Agent Benchmark90.81 % criterion passSelf-reported · no independent checksame internal benchmark and production-safeguard setup; criterion pass is not the all-pass metricsource
GDPval-AA v21853 EloSelf-reported · no independent checkArtificial Analysis; max effort; shell and web; blind pairwise comparison over 220 tasks and 44 occupationssource
Toolathlon Verified77.8 % Pass@1Self-reported · no independent check108 tasks; adaptive max; 3 trials each; production safety active with Opus 4.8 fallbacksource
Toolathlon Verified81.5 % Pass@3Self-reported · no independent checksame 108-task internal harness; three attemptssource
AutomationBench31.4%Self-reported · no independent checkmax effort; private held-out leaderboard; production safeguardssource
ARC-AGI-197.5%Self-reported · no independent checkmax effort; semi-private setsource
ARC-AGI-290%Self-reported · no independent checkmax effort; semi-private setsource
ARC-AGI-197.5%Checked by ARC Prize FoundationARC Prize's semi-private evaluation; max effortsource
ARC-AGI-290%Checked by ARC Prize FoundationARC Prize's semi-private evaluation; max effortsource
Maximum output128000 tokensSelf-reported · no independent checkmodel recordsource
FrontierMath v2 — Tiers 1-30.9 scoreChecked by Epoch AIFable 5.1 max run; verification_code scorersource
FrontierMath v2 — Tier 40.88 scoreChecked by Epoch AIFable 5.1 max run; verification_code scorersource

Claude Mythos 5.1

Anthropic2026-09closed (proprietary)

6 claimed · none independently checked

Terminal-Bench 4.060.9%Self-reported · no independent checkClaude Code with --bare; max thinking; 10 trials per task; Mythos safeguards configurationsource
BioMysteryBench Human Solvable90.3%Self-reported · no independent checkbash and file editor; restricted domains; adaptive max; Mythos access configurationsource
BioMysteryBench Human Difficult44.1%Self-reported · no independent checksame BioMysteryBench tool setup; difficult subsetsource
SpatialBench Verified77.6%Self-reported · no independent checklife-science evaluation; tool and grader details as specified in the system cardsource
ProteinGym Hard49.3%Self-reported · no independent checkmaximum adaptive reasoning; no external tools; protein-design subsetsource
Protocol Troubleshooting70.2%Self-reported · no independent checkbash, file editor, and web search for protocols; maximum adaptive reasoningsource

Claude Opus 5

Anthropic2026-07closed (proprietary)

3 claimed · 3 independently checked

ARC-AGI-197.5%Checked by ARC Prize Foundationmax effortsource
ARC-AGI-290.4%Checked by ARC Prize Foundationmax effortsource
ARC-AGI-330.16%Checked by ARC Prize Foundationhigh effortsource

Claude Opus 4.8

Anthropic2026-05closed (proprietary)

0 claimed · none independently checked

Claude Sonnet 5

Anthropic2026-06closed (proprietary)

0 claimed · none independently checked

Gemini 3.8 Flash

Google DeepMind2026-09closed (proprietary)

2 claimed · none independently checked

Humanity's Last Exam, HLE-Verified54.9%Self-reported · no independent checkGoogle DeepMind product page; reasoning, tools, harness, and date-window not statedsource
Maximum output65536 tokensSelf-reported · no independent checkmodel recordsource

Gemini 3.7 Flash

Google DeepMind2026-08closed (proprietary)

9 claimed · none independently checked

Artificial Analysis Intelligence Index56 index pointsSelf-reported · no independent checkGoogle comparison table; Aug 2026 snapshot; composition and settings not statedsource
FrontierCode 1.1 Main43.6%Self-reported · no independent checkGoogle comparison table; production code-quality setting; harness not statedsource
DeepSWE v1.165.3%Self-reported · no independent checkGoogle comparison table; long-horizon engineering; harness not statedsource
Code Arena1588 EloSelf-reported · no independent checkGoogle comparison table; arena protocol not statedsource
Terminal-Bench 2.185.8%Self-reported · no independent checkGoogle comparison table; agentic terminal coding; harness not statedsource
AutomationBench30.4%Self-reported · no independent checkGoogle comparison table; private set; harness not statedsource
GDPval-AA v21525 EloSelf-reported · no independent checkArtificial Analysis comparison row; task and grader details not stated in the cardsource
CharXiv (no tools)84.5%Self-reported · no independent checkno tools, as named in Google's tablesource
CharXiv (with tools)88.7%Self-reported · no independent checktools on, as named in Google's tablesource

Gemini 3.6 Flash

Google DeepMind2026-07closed (proprietary)

6 claimed · none independently checked

SWE-Bench Pro Public58.7%Self-reported · no independent checkGoogle comparison table; public split; harness not statedsource
DeepSWE v1.149%Self-reported · no independent checkGoogle comparison table; harness not statedsource
Terminal-Bench 2.178%Self-reported · no independent checkTerminus-2 harnesssource
OSWorld-Verified83%Self-reported · no independent checkGoogle comparison table; computer-use setting not otherwise statedsource
CharXiv (no tools)85.2%Self-reported · no independent checkno tools, as named in Google's tablesource
CharXiv (with tools)89.4%Self-reported · no independent checktools on, as named in Google's tablesource

Gemini 3.5 Flash

Google DeepMind2026-05closed (proprietary)

5 claimed · none independently checked

Terminal-Bench 2.176.2%Self-reported · no independent checkTerminus-2 harnesssource
SWE-Bench Pro Public55.1%Self-reported · no independent checksingle attempt, public splitsource
MCP Atlas83.6%Self-reported · no independent checkGoogle comparison table; subset/harness not statedsource
Toolathlon56.5%Self-reported · no independent checkGoogle comparison table; attempt count not statedsource
OSWorld-Verified78.4%Self-reported · no independent checkGoogle comparison table; computer-use setting not statedsource

Gemini 3.1 Pro

Google DeepMind2026-02closed (proprietary)

6 claimed · 1 independently checked

HLE (no tools)44.4%Self-reported · no independent checkno toolssource
HLE (Search + Code)51.4%Self-reported · no independent checkSearch blocklist + Codesource
GPQA Diamond (no tools)94.3%Self-reported · no independent checkno toolssource
Terminal-Bench 2.068.5%Self-reported · no independent checkTerminus-2 harnesssource
SWE-Bench Verified80.6%Self-reported · no independent checksingle attemptsource
METR time horizon (TH1.1)384.15 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Gemma 4 31B

Google DeepMind2026-04open weights (Apache License 2.0)

9 claimed · none independently checked

MMLU Pro85.2%Self-reported · no independent checkinstruction-tuned model card result; other harness details not statedsource
AIME 202689.2%Self-reported · no independent checkno toolssource
LiveCodeBench v680%Self-reported · no independent checkinstruction-tuned model card result; harness not statedsource
Codeforces2150 EloSelf-reported · no independent checkmodel card result; contest protocol not statedsource
GPQA Diamond84.3%Self-reported · no independent checkinstruction-tuned model card result; tools not statedsource
τ276.9%Self-reported · no independent checkaverage over 3source
HLE19.5%Self-reported · no independent checkno toolssource
HLE26.5%Self-reported · no independent checkwith searchsource
MRCR v2, 8-needle, 128K66.4 % averageSelf-reported · no independent check128K context; 8 needles; average metricsource

GPT-5.6 Sol

OpenAI2026-07closed (proprietary)

3 claimed · 3 independently checked

ARC-AGI-196.5%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-292.5%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-37.78%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource

GPT-5.6 Terra

OpenAI2026-07closed (proprietary)

3 claimed · 3 independently checked

ARC-AGI-196.5%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-283.9%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-30.8%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource

GPT-5.6 Luna

OpenAI2026-07closed (proprietary)

3 claimed · 3 independently checked

ARC-AGI-188%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-259.5%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource
ARC-AGI-30.18%Checked by ARC Prize Foundationmax reasoning effort; semi-privatesource

GPT-5.6 Cyber

OpenAI2026-08closed (proprietary)

0 claimed · none independently checked

GPT-5.5

OpenAI2026-04closed (proprietary)

0 claimed · none independently checked

GPT-5.5 Pro

OpenAI2026-04closed (proprietary)

0 claimed · none independently checked

GPT-5.4

OpenAI2026-03closed (proprietary)

1 claimed · 1 independently checked

METR time horizon (TH1.1)341.74 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-5.4 Pro

OpenAI2026-03closed (proprietary)

0 claimed · none independently checked

GPT-5.4 mini

OpenAI2026-03closed (proprietary)

0 claimed · none independently checked

GPT-5.3-Codex

OpenAI2026-02closed (proprietary)

1 claimed · 1 independently checked

METR time horizon (TH1.1)349.53 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-5.2-Codex

OpenAI2026-01closed (proprietary)

0 claimed · none independently checked

GPT-5.1-Codex

OpenAI2025-11closed (proprietary)

0 claimed · none independently checked

GPT-5.1-Codex-Max

OpenAI2025-12closed (proprietary)

1 claimed · 1 independently checked

METR time horizon (TH1.1)223.71 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-5.1-Codex Mini

OpenAI2025-11closed (proprietary)

0 claimed · none independently checked

GPT-5-Codex

OpenAI2025-09closed (proprietary)

0 claimed · none independently checked

o3

OpenAI2025-04closed (proprietary)

1 claimed · 1 independently checked

METR time horizon (TH1.1)119.73 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

o4-mini

OpenAI2025-04closed (proprietary)

0 claimed · none independently checked

Qwen3.8-Flash-Next

Qwen2026-08open weights (Qwen Community License 1.0)

7 claimed · none independently checked

DeepSWE v1.158.7%Self-reported · no independent checkhighest across Claude Code and mini-SWE-agent; temperature 1.0, top_p 0.95, 256K contextsource
SWE-bench Pro62.5%Self-reported · no independent checkClaude Code; temperature 1.0, top_p 0.95, 256K context; corrected tasks and re-evaluated baselinessource
SWE-bench Multilingual81%Self-reported · no independent checkmini-SWE-agent; temperature 1.0, top_p 0.95, 256K contextsource
Toolathlon Verified73.5 % Pass@1Self-reported · no independent checkQwen model-card comparison table; default thinking mode; exact harness not statedsource
GPQA Diamond91.7%Self-reported · no independent checkQwen model-card comparison table; default thinking mode; exact shot count not statedsource
HLE35.9%Self-reported · no independent checkQwen model-card comparison table; judged by GPT-4o; exact subset and harness not statedsource
LiveCodeBench v691.9%Self-reported · no independent checkQwen model-card comparison table; exact harness and date window not statedsource

Qwen3.8-27B

Qwen2026-08open weights (Apache License 2.0)

6 claimed · none independently checked

Terminal Bench 2.173%Self-reported · no independent checkTerminus harness; model-card comparison table; other settings not statedsource
SWE-bench Pro61.7%Self-reported · no independent checkClaude Code; temperature 1.0, top_p 0.95, 256K context; corrected tasks and re-evaluated baselinessource
DeepSWE v1.142.2%Self-reported · no independent checkClaude Code; temperature 1.0, top_p 1.0; 256K contextsource
OSWorld-Verified84.3%Self-reported · no independent checkOSWorld scaffold; model-card table; tool and action settings not statedsource
HLE30.8%Self-reported · no independent checkmodel-card comparison table; judged by GPT-4o; exact subset not statedsource
LiveCodeBench v690.3%Self-reported · no independent checkmodel-card comparison table; exact harness and date window not statedsource

DeepSeek-V4-Pro

DeepSeek2026-04open weights (MIT License)

8 claimed · none independently checked

MMLU90.1 % EMSelf-reported · no independent checkbase model; 5-shotsource
MMLU-Pro73.5 % EMSelf-reported · no independent checkbase model; 5-shotsource
LongBench-V251.5 % EMSelf-reported · no independent checkbase model; 1-shotsource
HLE37.7 % Pass@1Self-reported · no independent checkDeepSeek-V4-Pro-Max; max reasoning; comparison table, no tools rowsource
HLE48.2 % Pass@1Self-reported · no independent checkDeepSeek-V4-Pro-Max; max reasoning; with toolssource
Terminal Bench 2.067.9 % accuracySelf-reported · no independent checkDeepSeek-V4-Pro-Max; max reasoning; comparison table, harness not statedsource
SWE Verified80.6 % resolvedSelf-reported · no independent checkDeepSeek-V4-Pro-Max; max reasoning; comparison table, harness not statedsource
Toolathlon51.8 % Pass@1Self-reported · no independent checkDeepSeek-V4-Pro-Max; max reasoning; comparison table, harness not statedsource

DeepSeek-V4-Flash

DeepSeek2026-04open weights (MIT License)

0 claimed · none independently checked

Kimi K3

Moonshot AI2026-07open weights (Kimi K3 License)

9 claimed · none independently checked

GPQA Diamond93.5%Self-reported · no independent checkmax reasoning; temperature 1.0; top_p 0.95 for single-step taskssource
HLE-Full43.5%Self-reported · no independent checkmax reasoning; without tools; temperature 1.0; top_p 0.95 for single-step taskssource
HLE-Full56%Self-reported · no independent checkmax reasoning; with tools; general tools; temperature 1.0; top_p 0.95 single-step and 1.0 agenticsource
DeepSWE v1.167.5%Self-reported · no independent checkKimi Code harness; the card says the 67.3 leaderboard result is mini-SWE-agent, but reports 67.5 for K3's runsource
Terminal-Bench 2.188.3%Self-reported · no independent checkKimi Code harness; max reasoning; agentic top_p 1.0source
FrontierSWE81.2 dominance scoreSelf-reported · no independent checkKimi Code harness; dominance recomputed with official script; current as of 2026-07-16source
SWE-Marathon42%Self-reported · no independent checkClaude Code harness; H20-calibrated branch of tasks as of 2026-07-09, before final v1.1source
GDPval-AA v21686 EloSelf-reported · no independent checkArtificial Analysis scores cited by Kimi; board observed as of 2026-07-23source
Toolathlon-Verified76.5 % Pass@1Self-reported · no independent checkKimi comparison table; grader and trial count not stated in the rowsource

Kimi K2.6

Moonshot AI2026-04open weights (Modified MIT License)

6 claimed · none independently checked

HLE-Full36.4%Self-reported · no independent checkmax generation 98,304; without toolssource
HLE-Full55.5%Self-reported · no independent checkmax generation 98,304; with tools uses search, code interpreter, and web browsing plus context managementsource
Terminal-Bench 2.066.7%Self-reported · no independent checkTerminus-2; preserved thinking; default agent frameworksource
SWE-Bench Pro58.6%Self-reported · no independent checkin-house SWE-agent-derived harness with bash/createfile/insert/view/strreplace/submit toolssource
SWE-Bench Verified80.2%Self-reported · no independent checksame in-house minimal-tool harness; default temperature 1.0, top_p 1.0, 262,144 contextsource
LiveCodeBench v689.6%Self-reported · no independent checkmodel-card comparison table; exact sampling not stated in rowsource

GLM-5.3

Zhipu AI / Z.ai2026-08open weights (glm-5.3)

8 claimed · none independently checked

Terminal Bench 2.188.2%Self-reported · no independent checkClaude Code 2.1.207; temperature 1.0, top_p 1.0, max_new_tokens 65,536; 6-hour timeoutsource
Terminal Bench 3.028.3%Self-reported · no independent checkClaude Code 2.1.207; max reasoning; 400K context; 128K output; avg@3; Tool Search disabled; official task verifiersource
DeepSWE v1.166.9%Self-reported · no independent checkmini-SWE-agent; temperature 0.95, top_p 1.0; 6-hour timeout; 400K contextsource
FrontierSWE78.1 dominance scoreSelf-reported · no independent checkProximal evaluation; max effort; 1M context; 128K output; score current as of 2026-08-14source
SWE-Marathon v1.142.5%Self-reported · no independent checkClaude Code 2.1.207; max reasoning; temperature 1.0, top_p 0.95; 1M context; 128K outputsource
Toolathlon Verified73 % Pass@1Self-reported · no independent checkofficial evaluation service; average over 3 independent runssource
HLE with tools62.5%Self-reported · no independent checktemperature 1.0, top_p 0.95; max generation 163,840; max context 300K with context management; GPT-5.6 Luna medium judgesource
GDPval-AA v21769 EloSelf-reported · no independent checkArtificial Analysis evaluation; exact task/run details not stated in cardsource

MiniMax-M3

MiniMax2026-06open weights (MiniMax Community License)

0 claimed · none independently checked

MiniMax-M2.7

MiniMax2026-04research only (NON-COMMERCIAL LICENSE)

10 claimed · none independently checked

MLE Bench Lite66.6 % medal rateSelf-reported · no independent check22 ML competitions; card describes internal model self-evolution comparisonsource
SWE-Pro56.22%Self-reported · no independent checkMiniMax card; scaffold and run count not stated in the paragraphsource
SWE-bench Multilingual76.5%Self-reported · no independent checkMiniMax card; scaffold and run count not stated in the paragraphsource
Multi SWE Bench52.7%Self-reported · no independent checkMiniMax card; scaffold and run count not stated in the paragraphsource
VIBE-Pro55.6%Self-reported · no independent checkMiniMax card; benchmark harness not stated in the paragraphsource
Terminal Bench 257%Self-reported · no independent checkMiniMax card; benchmark harness not stated in the paragraphsource
NL2Repo39.8%Self-reported · no independent checkMiniMax card; benchmark harness not stated in the paragraphsource
GDPval-AA1495 EloSelf-reported · no independent checkMiniMax card; claimed highest among open-weight models; board date not statedsource
Toolathlon46.3 % accuracySelf-reported · no independent checkMiniMax card; benchmark harness and attempt count not statedsource
MM Claw62.7%Self-reported · no independent checkMiniMax card; end-to-end benchmark; harness and attempt count not statedsource

GPT-2

OpenAI2019-02

1 claimed · 1 independently checked

METR time horizon (TH1.1)0.05 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-3 (davinci-002)

OpenAI2020-05

1 claimed · 1 independently checked

METR time horizon (TH1.1)0.14 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-3.5 Turbo Instruct

OpenAI2022-03

1 claimed · 1 independently checked

METR time horizon (TH1.1)0.6 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-4

OpenAI2023-03

1 claimed · 1 independently checked

METR time horizon (TH1.1)3.99 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-4 (1106)

OpenAI2023-11

1 claimed · 1 independently checked

METR time horizon (TH1.1)4.04 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude 3 Opus

Anthropic2024-03

1 claimed · 1 independently checked

METR time horizon (TH1.1)3.95 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-4 Turbo

OpenAI2024-04

1 claimed · 1 independently checked

METR time horizon (TH1.1)3.73 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-4o

OpenAI2024-05

1 claimed · 1 independently checked

METR time horizon (TH1.1)6.99 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude 3.5 Sonnet (Jun 2024)

Anthropic2024-06

1 claimed · 1 independently checked

METR time horizon (TH1.1)11.4 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

o1-preview

OpenAI2024-09

1 claimed · 1 independently checked

METR time horizon (TH1.1)20.33 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude 3.5 Sonnet (Oct 2024)

Anthropic2024-10

1 claimed · 1 independently checked

METR time horizon (TH1.1)20.52 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

o1

OpenAI2024-12

1 claimed · 1 independently checked

METR time horizon (TH1.1)38.83 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude 3.7 Sonnet

Anthropic2025-02

1 claimed · 1 independently checked

METR time horizon (TH1.1)60.39 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude Opus 4

Anthropic2025-05

1 claimed · 1 independently checked

METR time horizon (TH1.1)100.37 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude Opus 4.1

Anthropic2025-08

1 claimed · 1 independently checked

METR time horizon (TH1.1)100.47 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-5

OpenAI2025-08

1 claimed · 1 independently checked

METR time horizon (TH1.1)203.01 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Gemini 3 Pro

Google DeepMind2025-11

1 claimed · 1 independently checked

METR time horizon (TH1.1)224.33 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude Opus 4.5

Anthropic2025-11

1 claimed · 1 independently checked

METR time horizon (TH1.1)292.99 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

GPT-5.2

OpenAI2025-12

1 claimed · 1 independently checked

METR time horizon (TH1.1)352.25 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude Opus 4.6

Anthropic2026-02

1 claimed · 1 independently checked

METR time horizon (TH1.1)718.81 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Claude Mythos Preview (early)

Anthropic2026-04

1 claimed · 1 independently checked

METR time horizon (TH1.1)1044.78 minutesChecked by METRp50 horizon length; METR-Horizon-v1.1source

Magistral Small 1.2

Mistral2025-09open weights (Apache 2.0)

2 claimed · 1 independently checked

GPQA Diamond70.07%Self-reported · no independent checkMistral-reported resultsource
GPQA Diamond47.6%Checked by Epoch AIEpoch AI independent runsource

Mistral Small 3.1

Mistral2025-03open weights (Apache 2.0)

2 claimed · 1 independently checked

GPQA Diamond45.96%Self-reported · no independent checkMistral-reported resultsource
GPQA Diamond41.9%Checked by Epoch AIEpoch AI independent runsource

Devstral Small 1.1

Mistral2025-07open weights (Apache 2.0)

2 claimed · 2 independently checked

SWE-bench Verified53.6%Checked by SWE-bench maintainersOpenHands harness; checked: truesource
SWE-bench Verified38%Checked by SWE-bench maintainersSWE-agent harness; checked: truesource

Devstral 2 123B

Mistral2025-12

2 claimed · 1 independently checked

SWE-bench Verified72.2%Self-reported · no independent checkMistral-reported resultsource
SWE-bench Verified53.8%Checked by SWE-bench maintainersmini-SWE-agent harness; checked: truesource

Devstral Small 2 24B

Mistral2025-12open weights (Apache 2.0)

2 claimed · 1 independently checked

SWE-bench Verified68%Self-reported · no independent checkMistral-reported resultsource
SWE-bench Verified56.4%Checked by SWE-bench maintainersmini-SWE-agent harness; checked: truesource

Llama 4 Maverick

Meta2025-04

1 claimed · 1 independently checked

SWE-bench Verified21.04%Checked by SWE-bench maintainerschecked: truesource

Llama 4 Scout

Meta2025-04

1 claimed · 1 independently checked

SWE-bench Verified9.06%Checked by SWE-bench maintainerschecked: truesource

Grok 3

xAI2025-02closed

1 claimed · 1 independently checked

ARC-AGI-20%Checked by ARC Prize FoundationARC Prize independent evaluationsource

Grok 4.5

xAI2026-08closed

1 claimed · 1 independently checked

ARC-AGI-252.6%Checked by ARC Prize FoundationARC Prize independent evaluationsource

Grok 4.6

xAI2026-08closed

1 claimed · 1 independently checked

ARC-AGI-267.1%Checked by ARC Prize FoundationARC Prize independent evaluationsource

The record · archived snapshot 2026-09-30

Read the conditions with the score.

225 sourced claims cover 74 models. 66 carry an independent check from ARC Prize Foundation, Epoch AI, METR, SWE-bench maintainers, Scale SEAL; the remainder are the publishing lab's own word. 225 claims record a harness, effort setting or attempt count.

A solid mark means a benchmark steward re-ran or published the result. A hollow mark means the lab reported it. Every score in the inspector links to its primary source.