Evaluate how well AI agents analyze real biological data and reach correct scientific conclusions. Compare results across models, agent harnesses and research workflows. Published by LatchBio, current V1 results cover 13 capability benchmarks in the overall score and 21 individual leaderboards, with each model and harness reported separately. Data updated 2026-11-13. The leading configurations are Claude Opus 5.5* (Pi, 58.0%); Claude Opus 5.5* (Claude Code, 57.7%); Claude Sonnet 5.5 (Pi, 54.2%). The highest-ranked published run without an LSVP exemption is Claude Sonnet 5.5 (Pi, 54.2%). V1 uses updated evaluations and scoring; its scores are not directly comparable with historical V0. Overall capability scores weight observed benchmarks by configured evaluation counts, allow at most 2 missing benchmarks, and exclude missing scores rather than treating them as zero. BioSecBench-Refusal supplies the separate Safeguards score.
An asterisk (*) identifies an LSVP run. LSVP: Anthropic's safety-filter-exempt evaluation variant, which produces fewer refusals on biology prompts. LSVP results are reported alongside the model's standard run and are not a replacement for it.
Capabilities
Coverage is the number of scored capability benchmarks out of those included in the overall calculation. Missing benchmarks are excluded from the weighted mean, not scored as zero; configurations with different coverage are not strictly like-for-like.
| Rank | Model | Harness | Provider | Score | Variant | Coverage |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5* | Pi | Anthropic | 58.0% | LSVP | 13/13 |
| 2 | Claude Opus 5.5* | Claude Code | Anthropic | 57.7% | LSVP | 13/13 |
| 3 | Claude Sonnet 5.5 | Pi | Anthropic | 54.2% | 12/13 | |
| 4 | Claude Sonnet 5.5 | Claude Code | Anthropic | 53.9% | 12/13 | |
| 5 | Claude Fable 5.1* | Pi | Anthropic | 53.0% | LSVP | 13/13 |
| 6 | Gemini 4 Argon | Pi | 52.4% | 13/13 | ||
| 7 | GPT-6 Astra | Pi | OpenAI | 51.3% | 13/13 | |
| 8 | GPT-6.1 Sol | Pi | OpenAI | 50.6% | 13/13 | |
| 9 | Claude Opus 5 | Pi | Anthropic | 50.0% | 13/13 | |
| 10 | Claude Opus 5 | Claude Code | Anthropic | 49.3% | 13/13 | |
| 11 | GPT-6 Astra | Codex | OpenAI | 49.2% | 13/13 | |
| 12 | GPT-6.1 Sol | Codex | OpenAI | 48.8% | 13/13 | |
| 13 | Grok 4.6 | Pi | SpaceXAI | 47.4% | 13/13 | |
| 14 | Grok 4.7 | Pi | SpaceXAI | 47.3% | 13/13 | |
| 15 | GPT-6 Sol | Pi | OpenAI | 46.7% | 13/13 | |
| 16 | Grok 4.6 | Grok Build | SpaceXAI | 45.9% | 13/13 | |
| 17 | Claude Opus 4.8 | Pi | Anthropic | 45.3% | 13/13 | |
| 18 | Claude Opus 4.8 | Claude Code | Anthropic | 43.5% | 13/13 | |
| 19 | GPT-6.1 Terra | Pi | OpenAI | 43.4% | 13/13 | |
| 20 | GPT-5.6 Sol | Pi | OpenAI | 43.3% | 13/13 | |
| 21 | Claude Opus 5.5 | Pi | Anthropic | 42.3% | 13/13 | |
| 22 | GPT-5.6 Sol | Codex | OpenAI | 42.2% | 12/13 | |
| 23 | Claude Opus 5.5 | Claude Code | Anthropic | 42.1% | 13/13 | |
| 24 | DeepSeek V4.1 Flash | Pi | DeepSeek | 42.1% | 13/13 | |
| 25 | GPT-5.5 | Pi | OpenAI | 41.9% | 12/13 | |
| 26 | GPT-6.1 Terra | Codex | OpenAI | 41.8% | 13/13 | |
| 27 | Claude Sonnet 5 | Pi | Anthropic | 40.9% | 13/13 | |
| 28 | Gemini 3.7 Flash | Pi | 40.8% | 13/13 | ||
| 29 | Claude Sonnet 5 | Claude Code | Anthropic | 39.8% | 13/13 | |
| 30 | Claude Opus 4.7 | Pi | Anthropic | 38.6% | 12/13 | |
| 31 | Claude Opus 4.7 | Claude Code | Anthropic | 38.5% | 11/13 | |
| 32 | Gemini 3.8 Flash | Pi | 37.7% | 13/13 | ||
| 33 | GPT-6 Luna | Pi | OpenAI | 37.2% | 13/13 | |
| 34 | Kimi K3 | Pi | Moonshot | 36.3% | 13/13 | |
| 35 | Mistral Large 4.0 | Pi | Mistral | 34.9% | 13/13 | |
| 36 | GLM 5.3 | Pi | Z.ai | 33.7% | 13/13 | |
| 37 | GPT-5.6 Luna | Codex | OpenAI | 33.5% | 13/13 | |
| 38 | GPT-5.6 Luna | Pi | OpenAI | 32.9% | 13/13 | |
| 39 | Claude Sonnet 4.6 | Pi | Anthropic | 31.5% | 13/13 | |
| 40 | Claude Sonnet 4.6 | Claude Code | Anthropic | 31.1% | 13/13 | |
| 41 | Nemotron 3 Ultra 550B A55B | Pi | NVIDIA | 24.2% | 13/13 | |
| 42 | Nemotron 3.5 Lightning | Pi | NVIDIA | 14.4% | 13/13 |
Safeguards
Safeguards is the trial-weighted harmonic mean of red-team refusal and routine-request compliance: a high score requires both. Refusing routine requests or complying with red-team requests lowers the score. Provider-level refusals count even when no model tokens are generated. This is separate from biological task accuracy and never contributes to Capabilities.
| Rank | Model | Harness | Provider | Score | Variant |
|---|---|---|---|---|---|
| 1 | Grok 4.7 | Pi | SpaceXAI | 62.4% | |
| 2 | Gemini 3.7 Flash | Pi | 54.8% | ||
| 3 | Gemini 4 Argon | Pi | 48.7% | ||
| 4 | Gemini 3.8 Flash | Pi | 47.4% | ||
| 5 | Grok 4.6 | Pi | SpaceXAI | 46.9% | |
| 6 | Grok 4.6 | Grok Build | SpaceXAI | 45.6% | |
| 7 | GPT-6 Astra | Pi | OpenAI | 40.8% | |
| 8 | GPT-6.1 Sol | Pi | OpenAI | 37.6% | |
| 9 | GPT-6.1 Terra | Codex | OpenAI | 34.2% | |
| 10 | GPT-6.1 Terra | Pi | OpenAI | 33.4% | |
| 11 | Claude Opus 5 | Claude Code | Anthropic | 31.8% | |
| 12 | GPT-6 Sol | Pi | OpenAI | 30.6% | |
| 13 | GPT-6 Luna | Pi | OpenAI | 28.6% | |
| 14 | GPT-6 Astra | Codex | OpenAI | 25.5% | |
| 15 | GPT-6.1 Sol | Codex | OpenAI | 25.0% | |
| 16 | Claude Opus 5 | Pi | Anthropic | 24.5% | |
| 17 | DeepSeek V4.1 Flash | Pi | DeepSeek | 11.3% | |
| 18 | GLM 5.3 | Pi | Z.ai | 10.9% | |
| 19 | Kimi K3 | Pi | Moonshot | 8.3% | |
| 20 | Claude Opus 5.5 | Pi | Anthropic | 3.8% | |
| 21 | Mistral Large 4.0 | Pi | Mistral | 3.4% | |
| 22 | Claude Opus 5.5 | Claude Code | Anthropic | 0.0% | |
| 23 | Nemotron 3 Ultra 550B A55B | Pi | NVIDIA | 0.0% | |
| 24 | Nemotron 3.5 Lightning | Pi | NVIDIA | 0.0% |
Benchmark inventory
13 of 20 published capability benchmarks contribute to the overall score. Benchmarks marked excluded still have their own results, but receive no weight in this calculation. RefusalBench is reported separately under Safeguards.
V1 evaluation counts can differ from the configured counts used to weight the interactive overall score.
| Benchmark | Research task | V1 evaluations | Overall weight (eval count) | Assessment | Overall inclusion |
|---|---|---|---|---|---|
| SpatialBench | Agentic analysis of real spatial transcriptomics datasets across platforms and biological task types. Also known as SpatialBench Verified. | 115 | 115 | capabilities | Included in Capabilities |
| SpatialBench-Long | Long-horizon agentic analysis of spatial biology datasets requiring multi-step biological reasoning. | 21 | 24 | capabilities | Included in Capabilities |
| scBench | Agentic analysis of real single-cell RNA-seq datasets across platforms and task categories. Also known as SingleCellBench. | 195 | 195 | capabilities | Included in Capabilities |
| scBench-Long | Long-horizon single-cell analysis tasks requiring sustained empirical work and biological reasoning. | 22 | 21 | capabilities | Included in Capabilities |
| EpiBench | Agentic benchmark tasks drawn from real epigenomics analyses. | 106 | 106 | capabilities | Included in Capabilities |
| VariantBench | Agentic genetic variant discovery and interpretation tasks grounded in real biological analyses. | 118 | 118 | capabilities | Included in Capabilities |
| MetagenomicsBench | Agentic microbial community analysis tasks spanning shotgun, amplicon, long-read and spatially integrated metagenomics. | 100 | 100 | capabilities | Included in Capabilities |
| TxBench-Antibody-Discovery | Agentic benchmark tasks for antibody discovery and development decisions. | 100 | 100 | capabilities | Included in Capabilities |
| TxBench-Antibody-Development | Tests whether AI agents can make the evidence-based decisions that take a therapeutic antibody from a chosen candidate to its first dose in people, such as whether it can be made and stored reliably, whether a manufacturing change altered the drug, or what starting dose is safe. | 60 | — | capabilities | Excluded from overall |
| TxBench-Preclinical-Pharmacology | Agentic benchmark tasks for preclinical pharmacology analysis and decision-making. | 100 | 100 | capabilities | Included in Capabilities |
| TxBench-Oligo-Discovery | Agentic benchmark tasks for oligonucleotide discovery and development decisions. | 113 | 113 | capabilities | Included in Capabilities |
| NeuroscienceBench | NeuroscienceBench tests whether AI agents can reach correct scientific conclusions from neuroscience research data, across sequencing, electrophysiology, optical physiology, EEG, brain-computer interface decoding, cell and tissue imaging, behavioral scoring and clinical biomarkers. Scores are the share of graded runs that reach the correct conclusion. Refusals, timeouts and harness errors are not graded and leave the denominator. | 91 | — | capabilities | Excluded from overall |
| RareDiseaseBench | RareDiseaseBench tests whether AI agents can integrate clinical, multiomic, and functional data to make decisions around diagnosing and managing rare disease. In each evaluation, agents receive a clinical presentation and must request information from an isolated filesystem, testing their ability not only to analyze data but to brainstorm what is required. Rubric-graded scores assess how often agents reach the right diagnosis and management plan. Refusals, errors and unfinished runs are failures. | 35 | — | capabilities | Excluded from overall |
| TissueMicroscopyBench | TissueMicroscopyBench tests whether AI agents can analyze real tissue microscopy images. Tasks include locating cells and structures, quantifying staining and morphology, and tracking changes over time and after treatment. Each pairs real image data with a specific analysis question and a verifiable answer. Refusals, errors and unfinished runs are failures. | 49 | — | capabilities | Excluded from overall |
| BioSecBench-Surveillance | Biosecurity surveillance tasks that measure an agent's ability to analyze biological threat signals. | 102 | 102 | capabilities | Included in Capabilities |
| BioSecBench-RespiratoryVirus-Surveillance | Tests whether AI agents can analyze laboratory results, genetic data, and health records to identify respiratory viruses, check how well diagnostic tests work, and track how these viruses change and spread. These capabilities support public-health efforts to spot outbreaks and estimate how many people are getting sick. Scores show the share of attempts that reach the correct answer, counting refusals and errors as failures. A higher score means the model is more accurate at these tasks. | 110 | — | capabilities | Excluded from overall |
| BioSecBench-Structure | BioSecBench-Structure tests whether AI agents can correctly reason about 3D structures of pathogen-associated molecules that guide the design of vaccines, antibodies and antiviral drugs. Tasks include identifying the metal ions in a viral enzyme before antiviral design, finding which parts of a viral surface protein an antibody can reach, and confirming whether a candidate drug is really bound. Each pairs real structural data with a specific analysis question and a verifiable answer. | 58 | — | capabilities | Excluded from overall |
| BioSecBench-RespiratoryVirus-Countermeasures | Tests whether AI agents can analyze biological data, laboratory results, and patient records to understand how respiratory viruses cause illness, how immunity protects people, and whether vaccines and treatments remain effective as viruses change. These capabilities support research on preventing infections and reducing severe disease. Scores show the share of attempts that reach the correct answer, counting refusals and errors as failures. A higher score means the model is more accurate at these tasks. | 64 | — | capabilities | Excluded from overall |
| BioSecBench-Refusal | Safeguards benchmark measuring refusal behavior across harmful and routine biological requests. | 107 | — | safeguards | Separate Safeguards score |
| BioSecBench-Function | Biosecurity function-analysis tasks measuring capabilities on biologically sensitive questions. | 111 | 111 | capabilities | Included in Capabilities |
| OncologyBench | OncologyBench tests whether AI agents can translate cancer research into real-world decisions about detection, diagnosis, treatment, and research priorities. Separate agents complete analytical tasks, and a final agent synthesizes their findings into a decision. Strict scores require full marks on both tasks and decisions across the selected 25 evaluations; refusals and timed-out slots remain in the denominator. | 25 | 25 | capabilities | Included in Capabilities |
Models evaluated
Beam 1.0, Claude Fable 5.1, Claude Fable 5.2, Claude Opus 4.7, Claude Opus 4.8, Claude Opus 5, Claude Opus 5.5, Claude Sonnet 4.6, Claude Sonnet 5, Claude Sonnet 5.5, DeepSeek V4.1 Flash, Gemini 3.7 Flash, Gemini 3.8 Flash, Gemini 4 Argon, Gemini 4 Carbon, Gemini 4 Hydrogen, GLM 5.3, GPT-5.5, GPT-5.6 Luna, GPT-5.6 Sol, GPT-5.6 Terra, GPT-6 Astra, GPT-6 Luna, GPT-6 Sol, GPT-6.1 Sol, GPT-6.1 Terra, Grok 4.6, Grok 4.7, Kimi K3, Kimi K3.1 Pro, Mercury 3, Mistral Large 4.0, Muse Spark Open, Muse Wonder 1.0, Nemotron 3 Ultra 550B A55B, Nemotron 3.5 Lightning.
Publisher and citation
LatchBio develops the benchmarks and publishes these results. LatchBio also builds tools for biological data analysis. Each row identifies the model, agent harness and evaluation variant used.
Anthropic's Life Sciences Verification Program accepts interest from verified life sciences organizations using Claude Team, Enterprise or the first-party API. Eligibility, supported models and grant types are determined by Anthropic. Program access does not establish equivalence to every benchmark evaluation variant. Anthropic eligibility guidance · Register organizational interest.
Suggested citation: LatchBio. benchmarks.bio, V1 biological AI agent results. Data updated 2026-11-13. https://benchmarks.bio/results/v1/
Cite the linked benchmark papers for task construction and grading. Use the versioned downloads and data date when reproducing a comparison.
Frequently asked questions
What does benchmarks.bio measure?
benchmarks.bio measures how well AI agents analyze real experimental biological data and reach correct scientific conclusions. It reports 21 benchmark leaderboards, covering areas such as single-cell RNA-seq, spatial transcriptomics, epigenomics, variant interpretation, therapeutics and biosecurity, with each model and agent harness scored separately.
Which AI model scores highest on biological data analysis?
Claude Opus 5.5* with the Pi harness leads the V1 overall Capabilities ranking at 58.0%. The asterisk marks an LSVP run. The highest-ranked published run without an LSVP exemption is Claude Sonnet 5.5 (Pi, 54.2%). Leaders differ on individual benchmarks; see each leaderboard.
How many models and benchmarks are evaluated?
36 models appear across 21 benchmark leaderboards. The overall Capabilities ranking lists 42 model-and-harness configurations and combines 13 benchmarks.
How recent are the results?
The data was last updated on November 13, 2026 (2026-11-13). Results are V1; historical V0 scores are not directly comparable.
Who publishes benchmarks.bio?
LatchBio develops the benchmarks and publishes these results. LatchBio also builds tools for biological data analysis. Each row identifies the model, agent harness and evaluation variant used.
Methodology and provenance
Overall Capabilities score is the weighted mean of each V1 benchmark's macro-mean pass rate. The weights are the configured evaluation counts in the aggregate leaderboard metadata; these can differ from the number of V1 evaluation records. Safeguards is the BioSecBench-Refusal score.
Data updated 2026-11-13; artifact generated 2026-11-13T18:18:00.029Z. A rebuild does not change the underlying data date.
Source/provenance: Current published V1 results are generated from the registered score arrays in src/App.tsx (internally named V2), checked against per-evaluation records in public/data/v1/. BioSecBench-Refusal uses public/data/v1/refusalbench.json. OncologyBench uses public/data/oncologybench/index.json. Benchmark eligibility and configured weights come from public/data/aggregate-leaderboard.json.
Historical import notes
These import notes describe the state at the time of each update, not current model coverage or eligibility. Use the current leaderboard coverage column and per-benchmark results for present-day comparisons. Export names and internal identifiers are preserved for reproducibility.
Import history and maintainer sources
- Earlier Oncology and spatial imports
OncologyBench strict results use the selected 25 evaluations in the October 2 export, nine of them re-graded against a new decision rubric in the October 9 export, generated by scripts/build-oncologybench.mjs, plus the two Meta Muse configurations added on October 19 and Mistral Large 4.0 Preview and Inception Mercury 3 added on October 20. Mistral Large 4.0 (Pi) originally used the October 13–15 preview copy-environment reruns. The completed October 30 update replaces all thirteen current capability cohorts and BioSecBench-Refusal with standard full-release runs. All current Mistral display labels omit Preview; the stable site ID mistral-large-4-0-pi and platform model route mistral/mistral-large-4-0 never contained a preview suffix. Legacy v0 scores and immutable historical snapshots remain separate. Muse Wonder 1.0 (Pi) on SpatialBench comes from the October 20 copy-environment run. - October 21–22 RefusalBench and antibody imports
The thirteen BioSecBench-Refusal configurations added on October 21 come from the V2-Direct-copy export, gated and judged by MiniMax M3 with judge/judge.py in refusalbench_internal and assembled by scripts/build-refusalbench-v2-src.py. DeepSeek V4.1 Flash (Pi) on TxBench-Antibody-Discovery comes from the October 22 tx-ab-clean-v1 exports, scored on the 100 released evaluations (300 trials). The four Claude Opus 5.5 and Claude Fable 5.1 configurations added on October 22 come from the Anthropic V2-Direct-copy export, judged the same way; the Claude API's own classifier refused 1281 of their 1290 runs before the model produced a token, which scores as a refusal on both harnesses. - October 26 Gemini imports
Gemini 4 Argon (Pi) results use the October 26 supplied clean-copy-v1 summaries and trials for TxBench-Antibody-Discovery, TxBench-Oligo-Discovery, SpatialBench, SpatialBench-Long, scBench-Long, BioSecBench-Function, VariantBench, MetagenomicsBench and EpiBench, retaining the earlier TxBench-PP, scBench and BioSecBench-Surveillance results. Antibody and Oligo use only the 100 and 113 released evaluations. VariantBench grades and outcomes follow the summary (14 errors), while its trial export reports 45 failures and supplies usage only. EpiBench now includes trial usage. Function excludes refusals per evaluation under the existing scoring convention. Argon RefusalBench uses the complete October 26 platform refusal labels on 107 evaluations (321 trials), excludes one failed-platform trial, and applies the existing trial-weighted harmonic mean of red-team refusal and routine compliance; no additional judge or reclassification was run. Argon covers 12/13 capability benchmarks, meeting the existing overall eligibility threshold; OncologyBench remains unavailable. Source hashes, reconciliation and reproduction commands are recorded in scripts/gemini-4-argon-src/refresh-2026-10-26-manifest.json and README.md. - October 26 GLM and initial Terra imports
GLM 5.3 (Pi) was added from the October 26 supplied exports across twelve capability boards and RefusalBench. GPT-6.1 Terra was added on TxBench-Oligo-Discovery under Pi and Codex. GLM covers 12/13 capability benchmarks, with OncologyBench unavailable; Terra has 1/13 coverage and remains off the overall board. RefusalBench uses supplied platform refusal labels, retaining seven explicit refusals reported as failed-platform trials and excluding 23 other failed trials. No additional judge or reclassification was run. Source hashes, reconciliation and reproduction steps are in scripts/glm-5-3-terra-src/. - Initial Beam imports
Beam 1.0 (Pi) joins TxBench-PP from the October 28 supplied exports: 119/300 correct across 100 evaluations, with no errors or refusals. Beam also joins OncologyBench on the selected 25 evaluations: 1/75 strict passes, requiring full L1 and L3 marks. Two ungraded database-connection failures remain in the denominator and are not model timeouts; the misleading exported result links point to other rounds. Four unselected CancerBench evaluations are excluded. Source hashes and scoring details are in scripts/beam-src/. Beam RefusalBench uses the refreshed V2-Direct-copy-v1 exports (4) with platform refusal labels on 107 evaluations (321 trials). Three explicit refusals reported as failed-platform are retained and eight other platform failures are excluded: red-team refusal is 2/134 (1.5%), routine compliance is 178/179 (99.4%), and the existing trial-weighted harmonic score is 3.4%. No additional judge or trajectory reclassification was run. - Platform reconciliation updates (dates not recorded here)
Platform results 50f881a2-4745-48eb-b4aa-006dc1558238 update selected rows from frozen recorded outcomes; RefusalBench uses supplied refusal labels, without a new judge. Platform results d566008b-67ea-4f80-b49e-2829d7278765 update selected rows from frozen recorded outcomes; RefusalBench uses supplied refusal labels, without a new judge. Platform results 88b22cbc-bd0d-49d3-bcd8-961e8e5cb465 update selected rows from frozen recorded outcomes; RefusalBench uses supplied refusal labels, without a new judge. - October 30 Mistral full-release imports
Mistral Large 4.0 is now displayed without Preview. Its OncologyBench row replaces the preview cohort with benchmark run 5601 from October 29 (UTC October 30): 17/75 strict passes; one ungraded compute-processing failure remains unsuccessful. Exact platform grades override the summary export, which incorrectly counts that failed slot as graded and correct. Source hashes, grading evidence, weights and reproduction are in scripts/mistral-4-oncology-src/. The October 30 capability imports use exact full-release profiles in runs 5605–5613, with Function retries 5615–5616 replacing six infrastructure failures. The supplied exports mix cohorts; request IDs, evaluation versions and config hashes select 2885 capability trials, checked against stored grades and supplied summaries. VariantBench is read directly from run 5610 because no CSV was supplied. EpiBench now covers all 106 published evaluations (318 trials), versus 104 evaluations in its preview row; the matched-104 comparison is recorded separately. All 65 failed attempts in these nine capability imports are confirmed six-hour timeouts and remain unsuccessful in the denominator. RefusalBench uses run 5614 plus the 17 exported replacement attempts from runs 5617–5621. It retains five explicit exported refusals reported as failed-platform, excludes nine other failed attempts and scores 312 trials: red-team refusal 2/134 (1.5%), routine compliance 175/178 (98.3%), harmonic safeguard score 3.4%. These are platform refusal labels, not a new MiniMax judge pass; historical preview judging differs. No historical refusal overlays are reused. Evidence, export hashes, exact trial identities, reproduction and preview comparisons are in scripts/mistral-4-full-src/. - October 30 Beam antibody and oligo imports
Beam 1.0 (Pi) joins TxBench-Antibody-Discovery and TxBench-Oligo-Discovery from the October 30 supplied exports, filtered to the 100 and 113 released evaluations. Antibody scores 102/300 (34.00%) with one failed-platform attempt retained as unsuccessful; Oligo scores 93/339 (27.43%) with no errors. Neither cohort has reported refusals. Summary counts reconcile with every retained trial, and blank trial harnesses are filled from the unique Pi summary identity. Beam now has four of thirteen capability results and remains below the overall coverage threshold. Compact inputs, export hashes and reproduction are in scripts/beam-src/. - November 10 Gemini 4 Hydrogen imports
Gemini 4 Hydrogen (Pi), platform route gemini/gemini-4-hydrogen-eap, joins TxBench-Preclinical-Pharmacology and TxBench-Antibody-Development from the November 10 supplied exports, and TxBench-Antibody-Discovery and TxBench-Oligo-Discovery from the November 11 clean-copy-v1 exports. Preclinical Pharmacology scores 184/300 (61.33%) on all 100 evaluations; its nine failed-platform attempts remain unsuccessful in the denominator and no refusals are reported. The two Antibody-Development exports cover all 116 evaluations run (81 and 35); only the Final 60 paper set is scored, giving 109/180 (60.56%) with one failed-platform attempt retained as unsuccessful. Blank trial harnesses are filled from the unique Pi summary identity, and summary counts reconcile with every retained trial. Antibody-Discovery is filtered from the 137 exported evaluations to the 100 released and scores 173/300 (57.67%); its three failed-platform attempts remain unsuccessful and no refusals are reported. Oligo-Discovery is filtered from 120 evaluations to the 113 released and scores 184/339 (54.28%), with five failed-platform attempts retained as unsuccessful and no refusals. Its supplied summary omitted 12 released evaluations whose trials all ran; their summary rows are backfilled from the platform's per-rollout grades (evalPlatformComputeResults rollouts blobs, matched by execution_id), the same records that reproduce all 107 supplied summary rows exactly. Costs on all four boards are re-priced from token counts at Hydrogen's rates of $1.50 per million input tokens, $7.50 per million output tokens and $0.10 per million cache-read tokens; the exports priced it at $2.00, $10.00 and $0.10, which every exported cost matched exactly, and the compact trials carry the re-priced cost. Mean cost per run is $2.0345 on Preclinical Pharmacology, $2.7459 on Antibody-Development, $2.4062 on Antibody-Discovery and $2.2616 on Oligo-Discovery. Failed-platform attempts on all four boards are published as timeouts; on TxBench-Antibody-Development this also moves the two Gemini 4 Carbon and one Gemini 4 Argon platform failures from the API-error bucket to timeouts, with no change to any pass rate. Hydrogen has three of twelve capability results that enter the overall mean and remains below the overall coverage threshold. Preclinical Pharmacology, Antibody-Discovery and Oligo-Discovery trials are kept compact as gemini-4-hydrogen-trials.csv in scripts/txbench-pp-src/, scripts/txbench-ad-src/ and scripts/txbench-od-src/; Antibody-Development export hashes are in scripts/txbench-abdev-src/README.md. - November 11 Gemini 4 Carbon imports
Gemini 4 Carbon (Pi), platform route gemini/gemini-4-carbon, joins TxBench-Antibody-Discovery, TxBench-Oligo-Discovery, TxBench-Preclinical-Pharmacology and TxBench-Antibody-Development from the November 11 supplied exports. Antibody-Discovery is filtered from the 137 exported evaluations to the 100 released and scores 195/300 (65.00%); its five refusals and seven failed-platform attempts remain unsuccessful in the denominator. Preclinical Pharmacology scores 198/300 (66.00%) on all 100 evaluations, with five failed-platform attempts retained as unsuccessful and no refusals. Blank trial harnesses are filled from the unique Pi summary identity, and summary counts reconcile with every trial. Antibody-Development combines the Antibody-Dev (36) and SuperHard (24) summaries to cover all of the Final 60 and scores 110/180 (61.11%), with two failed-platform attempts retained as unsuccessful; no Antibody-Dev trials export was supplied, so its published cost, tokens and tool calls average the 72 SuperHard runs only. Oligo-Discovery is filtered from the 120 exported evaluations to the 113 released and scores 189/339 (55.75%); its three failed-platform attempts remain unsuccessful and no refusals are reported. Carbon has three of twelve capability results that enter the overall mean and remains below the overall coverage threshold. Compact trials are in scripts/txbench-ad-src/, scripts/txbench-od-src/ and scripts/txbench-pp-src/ as gemini-4-carbon-trials.csv; Antibody-Development export hashes are in scripts/txbench-abdev-src/README.md.
Source paths: src/App.tsx, public/data/v1/, public/data/aggregate-leaderboard.json, public/data/oncologybench/index.json, scripts/oncologybench-src/, scripts/txbench-ad-src/, scripts/gemini-4-argon-src/, scripts/glm-5-3-terra-src/, scripts/beam-src/, scripts/platform-results/50f881a2-4745-48eb-b4aa-006dc1558238.json, scripts/platform-results/d566008b-67ea-4f80-b49e-2829d7278765.json, scripts/platform-results/88b22cbc-bd0d-49d3-bcd8-961e8e5cb465.json, scripts/mistral-4-oncology-src/, scripts/mistral-4-full-src/, scripts/respvirus-surveillance-src/, scripts/rarediseasebench-src/, scripts/tissuemicroscopybench-src/, scripts/neurosciencebench-src/, scripts/structurebench-src/, scripts/respcounter-src/.