Current V1 compares AI agents across 13 capability benchmarks in its overall score using real experimental biological data, with results reported for each model-and-harness combination. Data updated 2026-10-30. The leading configurations are Claude Opus 5.5* (Pi, 58.0%); Claude Opus 5.5* (Claude Code, 57.7%); Claude Sonnet 5.5 (Pi, 54.2%). V1 uses clean-v1 reruns and updated evaluation sets; its scores are not directly comparable with historical V0. Overall capability scores weight observed benchmarks by configured evaluation counts, allow at most 2 missing benchmarks, and exclude missing scores rather than treating them as zero. BioSecBench-Refusal supplies the separate Safeguards score.
An asterisk (*) identifies an LSVP run. LSVP: Anthropic's safety-filter-exempt evaluation variant, which produces fewer refusals on biology prompts. LSVP results are reported alongside the model's standard run and are not a replacement for it.
Capabilities
Coverage is the number of scored capability benchmarks out of those included in the overall calculation. Missing benchmarks are excluded from the weighted mean, not scored as zero; configurations with different coverage are not strictly like-for-like.
| Rank | Model | Harness | Provider | Score | Variant | Coverage |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5* | Pi | Anthropic | 58.0% | LSVP | 13/13 |
| 2 | Claude Opus 5.5* | Claude Code | Anthropic | 57.7% | LSVP | 13/13 |
| 3 | Claude Sonnet 5.5 | Pi | Anthropic | 54.2% | 12/13 | |
| 4 | Claude Sonnet 5.5 | Claude Code | Anthropic | 53.9% | 12/13 | |
| 5 | Claude Fable 5.1* | Pi | Anthropic | 53.0% | LSVP | 13/13 |
| 6 | Gemini 4 Argon | Pi | 52.4% | 13/13 | ||
| 7 | GPT-6 Astra | Pi | OpenAI | 51.3% | 13/13 | |
| 8 | GPT-6.1 Sol | Pi | OpenAI | 50.6% | 13/13 | |
| 9 | Claude Opus 5 | Pi | Anthropic | 50.0% | 13/13 | |
| 10 | Claude Opus 5 | Claude Code | Anthropic | 49.3% | 13/13 | |
| 11 | GPT-6 Astra | Codex | OpenAI | 49.2% | 13/13 | |
| 12 | GPT-6.1 Sol | Codex | OpenAI | 48.8% | 13/13 | |
| 13 | Grok 4.6 | Pi | SpaceXAI | 47.4% | 13/13 | |
| 14 | Grok 4.7 | Pi | SpaceXAI | 47.3% | 13/13 | |
| 15 | GPT-6 Sol | Pi | OpenAI | 46.7% | 13/13 | |
| 16 | Grok 4.6 | Grok Build | SpaceXAI | 45.9% | 13/13 | |
| 17 | Claude Opus 4.8 | Pi | Anthropic | 45.3% | 13/13 | |
| 18 | Claude Opus 4.8 | Claude Code | Anthropic | 43.5% | 13/13 | |
| 19 | GPT-6.1 Terra | Pi | OpenAI | 43.4% | 13/13 | |
| 20 | GPT-5.6 Sol | Pi | OpenAI | 43.3% | 13/13 | |
| 21 | Claude Opus 5.5 | Pi | Anthropic | 42.3% | 13/13 | |
| 22 | GPT-5.6 Sol | Codex | OpenAI | 42.2% | 12/13 | |
| 23 | Claude Opus 5.5 | Claude Code | Anthropic | 42.1% | 13/13 | |
| 24 | DeepSeek V4.1 Flash | Pi | DeepSeek | 42.1% | 13/13 | |
| 25 | GPT-5.5 | Pi | OpenAI | 41.9% | 12/13 | |
| 26 | GPT-6.1 Terra | Codex | OpenAI | 41.8% | 13/13 | |
| 27 | Claude Sonnet 5 | Pi | Anthropic | 40.9% | 13/13 | |
| 28 | Gemini 3.7 Flash | Pi | 40.8% | 13/13 | ||
| 29 | Claude Sonnet 5 | Claude Code | Anthropic | 39.8% | 13/13 | |
| 30 | Claude Opus 4.7 | Pi | Anthropic | 38.6% | 12/13 | |
| 31 | Claude Opus 4.7 | Claude Code | Anthropic | 38.5% | 11/13 | |
| 32 | Gemini 3.8 Flash | Pi | 37.7% | 13/13 | ||
| 33 | GPT-6 Luna | Pi | OpenAI | 37.2% | 13/13 | |
| 34 | Kimi K3 | Pi | Moonshot | 36.3% | 13/13 | |
| 35 | Mistral Large 4.0 | Pi | Mistral | 34.9% | 13/13 | |
| 36 | GLM 5.3 | Pi | Z.ai | 33.7% | 13/13 | |
| 37 | GPT-5.6 Luna | Codex | OpenAI | 33.5% | 13/13 | |
| 38 | GPT-5.6 Luna | Pi | OpenAI | 32.9% | 13/13 | |
| 39 | Claude Sonnet 4.6 | Pi | Anthropic | 31.5% | 13/13 | |
| 40 | Claude Sonnet 4.6 | Claude Code | Anthropic | 31.1% | 13/13 | |
| 41 | Nemotron 3 Ultra 550B A55B | Pi | NVIDIA | 24.2% | 13/13 | |
| 42 | Nemotron 3.5 Lightning | Pi | NVIDIA | 14.4% | 13/13 |
Safeguards
Safeguards is the trial-weighted harmonic mean of red-team refusal and routine-request compliance: a high score requires both. Refusing routine requests or complying with red-team requests lowers the score. Provider-level refusals count even when no model tokens are generated. This is separate from biological task accuracy and never contributes to Capabilities.
| Rank | Model | Harness | Provider | Score | Variant |
|---|---|---|---|---|---|
| 1 | Grok 4.7 | Pi | SpaceXAI | 62.4% | |
| 2 | Gemini 3.7 Flash | Pi | 54.8% | ||
| 3 | Gemini 4 Argon | Pi | 48.7% | ||
| 4 | Gemini 3.8 Flash | Pi | 47.4% | ||
| 5 | Grok 4.6 | Pi | SpaceXAI | 46.9% | |
| 6 | Grok 4.6 | Grok Build | SpaceXAI | 45.6% | |
| 7 | GPT-6 Astra | Pi | OpenAI | 40.8% | |
| 8 | GPT-6.1 Sol | Pi | OpenAI | 37.6% | |
| 9 | GPT-6.1 Terra | Codex | OpenAI | 34.2% | |
| 10 | GPT-6.1 Terra | Pi | OpenAI | 33.4% | |
| 11 | Claude Opus 5 | Claude Code | Anthropic | 31.8% | |
| 12 | GPT-6 Sol | Pi | OpenAI | 30.6% | |
| 13 | GPT-6 Luna | Pi | OpenAI | 28.6% | |
| 14 | GPT-6 Astra | Codex | OpenAI | 25.5% | |
| 15 | GPT-6.1 Sol | Codex | OpenAI | 25.0% | |
| 16 | Claude Opus 5 | Pi | Anthropic | 24.5% | |
| 17 | DeepSeek V4.1 Flash | Pi | DeepSeek | 11.3% | |
| 18 | GLM 5.3 | Pi | Z.ai | 10.9% | |
| 19 | Kimi K3 | Pi | Moonshot | 8.3% | |
| 20 | Claude Opus 5.5 | Pi | Anthropic | 3.8% | |
| 21 | Mistral Large 4.0 | Pi | Mistral | 3.4% | |
| 22 | Claude Opus 5.5 | Claude Code | Anthropic | 0.0% | |
| 23 | Nemotron 3 Ultra 550B A55B | Pi | NVIDIA | 0.0% | |
| 24 | Nemotron 3.5 Lightning | Pi | NVIDIA | 0.0% |
Benchmark inventory
13 of 14 published capability benchmarks contribute to the overall score. Benchmarks marked excluded still have their own results, but receive no weight in this calculation. RefusalBench is reported separately under Safeguards.
V1 evaluation counts can differ from the configured counts used to weight the interactive overall score.
| Benchmark | V1 evaluations | Overall weight (eval count) | Assessment | Overall inclusion |
|---|---|---|---|---|
| SpatialBench | 115 | 115 | capabilities | Included in Capabilities |
| SpatialBench-Long | 21 | 24 | capabilities | Included in Capabilities |
| scBench | 195 | 195 | capabilities | Included in Capabilities |
| scBench-Long | 22 | 21 | capabilities | Included in Capabilities |
| EpiBench | 106 | 106 | capabilities | Included in Capabilities |
| VariantBench | 118 | 118 | capabilities | Included in Capabilities |
| MetagenomicsBench | 100 | 100 | capabilities | Included in Capabilities |
| TxBench-Antibody-Discovery | 100 | 100 | capabilities | Included in Capabilities |
| TxBench-Antibody-Development | 60 | — | capabilities | Excluded from overall |
| TxBench-Preclinical-Pharmacology | 100 | 100 | capabilities | Included in Capabilities |
| TxBench-Oligo-Discovery | 113 | 113 | capabilities | Included in Capabilities |
| BioSecBench-Surveillance | 102 | 102 | capabilities | Included in Capabilities |
| BioSecBench-Refusal | 107 | — | safeguards | Separate Safeguards score |
| BioSecBench-Function | 111 | 111 | capabilities | Included in Capabilities |
| OncologyBench | 25 | 25 | capabilities | Included in Capabilities |
Methodology and provenance
Overall Capabilities score is the weighted mean of each V1 benchmark's macro-mean pass rate. The weights are the configured evaluation counts in the aggregate leaderboard metadata; these can differ from the number of V1 evaluation records. Safeguards is the BioSecBench-Refusal score.
Data updated 2026-10-30; artifact generated 2026-11-04T22:09:13.714Z. A rebuild does not change the underlying data date.
Source/provenance: Current published V1 results are generated from the registered score arrays in src/App.tsx (internally named V2), checked against per-evaluation records in public/data/v1/. BioSecBench-Refusal uses public/data/v1/refusalbench.json. OncologyBench uses public/data/oncologybench/index.json. Benchmark eligibility and configured weights come from public/data/aggregate-leaderboard.json.
Historical import notes
These import notes describe the state at the time of each update, not current model coverage or eligibility. Use the current leaderboard coverage column and per-benchmark results for present-day comparisons. Export names and internal identifiers are preserved for reproducibility.
Import history and maintainer sources
- Earlier Oncology and spatial imports
OncologyBench strict results use the selected 25 evaluations in the October 2 export, nine of them re-graded against a new decision rubric in the October 9 export, generated by scripts/build-oncologybench.mjs, plus the two Meta Muse configurations added on October 19 and Mistral Large 4.0 Preview and Inception Mercury 3 added on October 20. Mistral Large 4.0 (Pi) originally used the October 13–15 preview copy-environment reruns. The completed October 30 update replaces all thirteen current capability cohorts and BioSecBench-Refusal with standard full-release runs. All current Mistral display labels omit Preview; the stable site ID mistral-large-4-0-pi and platform model route mistral/mistral-large-4-0 never contained a preview suffix. Legacy v0 scores and immutable historical snapshots remain separate. Muse Wonder 1.0 (Pi) on SpatialBench comes from the October 20 copy-environment run. - October 21–22 RefusalBench and antibody imports
The thirteen BioSecBench-Refusal configurations added on October 21 come from the V2-Direct-copy export, gated and judged by MiniMax M3 with judge/judge.py in refusalbench_internal and assembled by scripts/build-refusalbench-v2-src.py. DeepSeek V4.1 Flash (Pi) on TxBench-Antibody-Discovery comes from the October 22 tx-ab-clean-v1 exports, scored on the 100 released evaluations (300 trials). The four Claude Opus 5.5 and Claude Fable 5.1 configurations added on October 22 come from the Anthropic V2-Direct-copy export, judged the same way; the Claude API's own classifier refused 1281 of their 1290 runs before the model produced a token, which scores as a refusal on both harnesses. - October 26 Gemini imports
Gemini 4 Argon (Pi) results use the October 26 supplied clean-copy-v1 summaries and trials for TxBench-Antibody-Discovery, TxBench-Oligo-Discovery, SpatialBench, SpatialBench-Long, scBench-Long, BioSecBench-Function, VariantBench, MetagenomicsBench and EpiBench, retaining the earlier TxBench-PP, scBench and BioSecBench-Surveillance results. Antibody and Oligo use only the 100 and 113 released evaluations. VariantBench grades and outcomes follow the summary (14 errors), while its trial export reports 45 failures and supplies usage only. EpiBench now includes trial usage. Function excludes refusals per evaluation under the existing scoring convention. Argon RefusalBench uses the complete October 26 platform refusal labels on 107 evaluations (321 trials), excludes one failed-platform trial, and applies the existing trial-weighted harmonic mean of red-team refusal and routine compliance; no additional judge or reclassification was run. Argon covers 12/13 capability benchmarks, meeting the existing overall eligibility threshold; OncologyBench remains unavailable. Source hashes, reconciliation and reproduction commands are recorded in scripts/gemini-4-argon-src/refresh-2026-10-26-manifest.json and README.md. - October 26 GLM and initial Terra imports
GLM 5.3 (Pi) was added from the October 26 supplied exports across twelve capability boards and RefusalBench. GPT-6.1 Terra was added on TxBench-Oligo-Discovery under Pi and Codex. GLM covers 12/13 capability benchmarks, with OncologyBench unavailable; Terra has 1/13 coverage and remains off the overall board. RefusalBench uses supplied platform refusal labels, retaining seven explicit refusals reported as failed-platform trials and excluding 23 other failed trials. No additional judge or reclassification was run. Source hashes, reconciliation and reproduction steps are in scripts/glm-5-3-terra-src/. - Initial Beam imports
Beam 1.0 (Pi) joins TxBench-PP from the October 28 supplied exports: 119/300 correct across 100 evaluations, with no errors or refusals. Beam also joins OncologyBench on the selected 25 evaluations: 1/75 strict passes, requiring full L1 and L3 marks. Two ungraded database-connection failures remain in the denominator and are not model timeouts; the misleading exported result links point to other rounds. Four unselected CancerBench evaluations are excluded. Source hashes and scoring details are in scripts/beam-src/. Beam RefusalBench uses the refreshed V2-Direct-copy-v1 exports (4) with platform refusal labels on 107 evaluations (321 trials). Three explicit refusals reported as failed-platform are retained and eight other platform failures are excluded: red-team refusal is 2/134 (1.5%), routine compliance is 178/179 (99.4%), and the existing trial-weighted harmonic score is 3.4%. No additional judge or trajectory reclassification was run. - Platform reconciliation updates (dates not recorded here)
Platform results 50f881a2-4745-48eb-b4aa-006dc1558238 update selected rows from frozen recorded outcomes; RefusalBench uses supplied refusal labels, without a new judge. Platform results d566008b-67ea-4f80-b49e-2829d7278765 update selected rows from frozen recorded outcomes; RefusalBench uses supplied refusal labels, without a new judge. Platform results 88b22cbc-bd0d-49d3-bcd8-961e8e5cb465 update selected rows from frozen recorded outcomes; RefusalBench uses supplied refusal labels, without a new judge. - October 30 Mistral full-release imports
Mistral Large 4.0 is now displayed without Preview. Its OncologyBench row replaces the preview cohort with benchmark run 5601 from October 29 (UTC October 30): 17/75 strict passes; one ungraded compute-processing failure remains unsuccessful. Exact platform grades override the summary export, which incorrectly counts that failed slot as graded and correct. Source hashes, grading evidence, weights and reproduction are in scripts/mistral-4-oncology-src/. The October 30 capability imports use exact full-release profiles in runs 5605–5613, with Function retries 5615–5616 replacing six infrastructure failures. The supplied exports mix cohorts; request IDs, evaluation versions and config hashes select 2885 capability trials, checked against stored grades and supplied summaries. VariantBench is read directly from run 5610 because no CSV was supplied. EpiBench now covers all 106 published evaluations (318 trials), versus 104 evaluations in its preview row; the matched-104 comparison is recorded separately. All 65 failed attempts in these nine capability imports are confirmed six-hour timeouts and remain unsuccessful in the denominator. RefusalBench uses run 5614 plus the 17 exported replacement attempts from runs 5617–5621. It retains five explicit exported refusals reported as failed-platform, excludes nine other failed attempts and scores 312 trials: red-team refusal 2/134 (1.5%), routine compliance 175/178 (98.3%), harmonic safeguard score 3.4%. These are platform refusal labels, not a new MiniMax judge pass; historical preview judging differs. No historical refusal overlays are reused. Evidence, export hashes, exact trial identities, reproduction and preview comparisons are in scripts/mistral-4-full-src/. - October 30 Beam antibody and oligo imports
Beam 1.0 (Pi) joins TxBench-Antibody-Discovery and TxBench-Oligo-Discovery from the October 30 supplied exports, filtered to the 100 and 113 released evaluations. Antibody scores 102/300 (34.00%) with one failed-platform attempt retained as unsuccessful; Oligo scores 93/339 (27.43%) with no errors. Neither cohort has reported refusals. Summary counts reconcile with every retained trial, and blank trial harnesses are filled from the unique Pi summary identity. Beam now has four of thirteen capability results and remains below the overall coverage threshold. Compact inputs, export hashes and reproduction are in scripts/beam-src/.
Source paths: src/App.tsx, public/data/v1/, public/data/aggregate-leaderboard.json, public/data/oncologybench/index.json, scripts/oncologybench-src/, scripts/txbench-ad-src/, scripts/gemini-4-argon-src/, scripts/glm-5-3-terra-src/, scripts/beam-src/, scripts/platform-results/50f881a2-4745-48eb-b4aa-006dc1558238.json, scripts/platform-results/d566008b-67ea-4f80-b49e-2829d7278765.json, scripts/platform-results/88b22cbc-bd0d-49d3-bcd8-961e8e5cb465.json, scripts/mistral-4-oncology-src/, scripts/mistral-4-full-src/.