benchmarks.bio

Biosecurity · Safeguards

BioSecBench-Refusal results

Safeguards benchmark measuring refusal behavior across harmful and routine biological requests.

Published score: Full benchmark (107 evaluations) · short horizon · data updated 2026-10-09

This page reports published BioSecBench-Refusal scores from benchmarks.bio. BioSecBench-Refusal contains 107 evaluations built from real experimental data, each one graded deterministically against the biological result the original analysis reached, so a run counts as a pass only when the agent recovers that result rather than when it produces plausible-looking output. Results are reported per model and per harness rather than per model alone, because the harness an agent runs under changes its score substantially. Scores use a 0-100 percentage scale and are not comparable across benchmarks; see the linked repository and paper for task construction, grading, and confidence-interval methodology.

Published BioSecBench-Refusal scores
RankModelHarnessProviderScore
1Grok 4.6PiSpaceXAI63.0%
2Grok 4.6Grok BuildSpaceXAI59.0%
3Gemini 3.5 FlashPiGoogle51.5%
4GPT-6 AstraPiOpenAI44.6%
5Claude Sonnet 4.6Claude CodeAnthropic43.5%
6GPT-5.6 SolPiOpenAI39.5%
7GPT-5.6 SolCodexOpenAI38.2%
8GPT-5.5PiOpenAI37.1%
9Claude Opus 4.7PiAnthropic36.7%
10Claude Sonnet 4.6PiAnthropic35.4%
11GPT-5.6 TerraPiOpenAI34.3%
12Claude Opus 4.7Claude CodeAnthropic33.9%
13GPT-5.6 LunaCodexOpenAI33.3%
14GPT-5.6 TerraCodexOpenAI32.3%
15GPT-5.6 LunaPiOpenAI31.5%
16Claude Sonnet 5PiAnthropic30.1%
17Claude Opus 5PiAnthropic29.4%
18Claude Opus 5Claude CodeAnthropic28.9%
19Claude Opus 4.8PiAnthropic28.6%
20Claude Sonnet 5Claude CodeAnthropic27.4%
21GPT-5.5CodexOpenAI26.7%
22Claude Opus 4.8Claude CodeAnthropic26.1%
23GPT-6 AstraCodexOpenAI23.3%
24Grok 4.5PiSpaceXAI9.3%
25Kimi K3PiMoonshot8.6%

Methodology and provenance

Scores use a 0–100 percentage scale. See Paper PDF, Original publication, Repository for task construction, grading, aggregation, and uncertainty details.