benchmarks.bio

Omics · Capabilities

VariantBench results

Agentic genetic variant discovery and interpretation tasks grounded in real biological analyses.

Published score: Current V1 evaluation set (118 evaluations) · short horizon · data updated 2026-11-13 (November 13, 2026)

Compare current model-and-harness results for VariantBench. Agentic genetic variant discovery and interpretation tasks grounded in real biological analyses. Current V1 covers 118 evaluations and 42 ranked configurations. Data updated 2026-11-13. The highest-ranked published configurations are Claude Opus 5 (Claude Code, 52.8%); Claude Sonnet 5.5 (Pi, 51.7%); Claude Sonnet 5.5 (Claude Code, 51.1%). Scores report benchmark-specific biological task performance on a 0–100 percentage scale. See the linked paper and repository for task construction, grading and uncertainty methodology. Results describe the complete model and agent harness used for each run, and are not comparable across different benchmarks or evaluation versions.

Published VariantBench scores
RankModelHarnessProviderScoreVariant
1Claude Opus 5Claude CodeAnthropic52.8%
2Claude Sonnet 5.5PiAnthropic51.7%
3Claude Sonnet 5.5Claude CodeAnthropic51.1%
4Claude Opus 5PiAnthropic50.6%
5Claude Opus 5.5PiAnthropic50.6%
6Gemini 4 ArgonPiGoogle50.3%
7Claude Opus 5.5Claude CodeAnthropic49.7%
8Claude Opus 5.5*Claude CodeAnthropic48.9%LSVP
9Claude Opus 5.5*PiAnthropic48.6%LSVP
10GPT-6 AstraPiOpenAI46.6%
11GPT-6.1 SolPiOpenAI46.0%
12Grok 4.7PiSpaceXAI45.5%
13GPT-6 AstraCodexOpenAI45.2%
14GPT-6.1 SolCodexOpenAI43.2%
15Claude Opus 4.8Claude CodeAnthropic42.4%
16Claude Opus 4.8PiAnthropic42.4%
17Claude Fable 5.1*PiAnthropic40.7%LSVP
18Grok 4.6PiSpaceXAI40.4%
19Claude Sonnet 5PiAnthropic39.8%
20Claude Sonnet 5Claude CodeAnthropic39.0%
21Grok 4.6Grok BuildSpaceXAI39.0%
22GPT-6 SolPiOpenAI38.7%
23GPT-5.6 SolPiOpenAI35.3%
24Claude Opus 4.7PiAnthropic35.0%
25GPT-6.1 TerraCodexOpenAI34.8%
26GPT-5.6 SolCodexOpenAI34.2%
27GPT-6.1 TerraPiOpenAI33.6%
28Claude Opus 4.7Claude CodeAnthropic33.0%
29Gemini 3.7 FlashPiGoogle31.4%
30Gemini 3.8 FlashPiGoogle31.4%
31Kimi K3PiMoonshot31.4%
32Claude Sonnet 4.6PiAnthropic31.1%
33DeepSeek V4.1 FlashPiDeepSeek31.1%
34GPT-5.5PiOpenAI30.5%
35Mistral Large 4.0PiMistral28.0%
36GPT-6 LunaPiOpenAI27.7%
37GPT-5.6 LunaCodexOpenAI27.1%
38Claude Sonnet 4.6Claude CodeAnthropic26.3%
39GLM 5.3PiZ.ai26.3%
40GPT-5.6 LunaPiOpenAI24.0%
41Nemotron 3 Ultra 550B A55BPiNVIDIA17.8%
42Nemotron 3.5 LightningPiNVIDIA14.4%

Methodology and provenance

Scores use a 0–100 percentage scale. See Paper PDF, Original publication, Repository for task construction, grading, aggregation, and uncertainty details.

LSVP: Anthropic's safety-filter-exempt evaluation variant, which produces fewer refusals on biology prompts. LSVP results are reported alongside the model's standard run and are not a replacement for it.