Compare current model-and-harness results for BioSecBench-Refusal. Safeguards benchmark measuring refusal behavior across harmful and routine biological requests. Current V1 covers 107 evaluations and 31 ranked configurations. Data updated 2026-11-13. The highest-ranked published configurations are Grok 4.7 (Pi, 62.4%); Muse Wonder 1.0 (Pi, 55.3%); Gemini 3.7 Flash (Pi, 54.8%). Safeguards is the trial-weighted harmonic mean of red-team refusal and routine-request compliance: a high score requires both. Refusing routine requests or complying with red-team requests lowers the score. Provider-level refusals count even when no model tokens are generated. This is separate from biological task accuracy and never contributes to Capabilities. Results describe the complete model and agent harness used for each run, and are not comparable across different benchmarks or evaluation versions.
| Rank | Model | Harness | Provider | Score | Variant |
|---|---|---|---|---|---|
| 1 | Grok 4.7 | Pi | SpaceXAI | 62.4% | |
| 2 | Muse Wonder 1.0 | Pi | Meta | 55.3% | |
| 3 | Gemini 3.7 Flash | Pi | 54.8% | ||
| 4 | Gemini 4 Argon | Pi | 48.7% | ||
| 5 | Gemini 3.8 Flash | Pi | 47.4% | ||
| 6 | Grok 4.6 | Pi | SpaceXAI | 46.9% | |
| 7 | Grok 4.6 | Grok Build | SpaceXAI | 45.6% | |
| 8 | GPT-6 Astra | Pi | OpenAI | 40.8% | |
| 9 | GPT-6 Sol | Codex | OpenAI | 37.9% | |
| 10 | GPT-6.1 Sol | Pi | OpenAI | 37.6% | |
| 11 | GPT-6 Luna | Codex | OpenAI | 37.5% | |
| 12 | Muse Spark Open | Pi | Meta | 34.9% | |
| 13 | GPT-6.1 Terra | Codex | OpenAI | 34.2% | |
| 14 | GPT-6.1 Terra | Pi | OpenAI | 33.4% | |
| 15 | Claude Opus 5 | Claude Code | Anthropic | 31.8% | |
| 16 | GPT-6 Sol | Pi | OpenAI | 30.6% | |
| 17 | GPT-6 Luna | Pi | OpenAI | 28.6% | |
| 18 | GPT-6 Astra | Codex | OpenAI | 25.5% | |
| 19 | GPT-6.1 Sol | Codex | OpenAI | 25.0% | |
| 20 | Claude Opus 5 | Pi | Anthropic | 24.5% | |
| 21 | DeepSeek V4.1 Flash | Pi | DeepSeek | 11.3% | |
| 22 | GLM 5.3 | Pi | Z.ai | 10.9% | |
| 23 | Kimi K3 | Pi | Moonshot | 8.3% | |
| 24 | Claude Fable 5.1 | Pi | Anthropic | 5.6% | |
| 25 | Claude Opus 5.5 | Pi | Anthropic | 3.8% | |
| 26 | Beam 1.0 | Pi | Reflection | 3.4% | |
| 27 | Mistral Large 4.0 | Pi | Mistral | 3.4% | |
| 28 | Claude Fable 5.1 | Claude Code | Anthropic | 0.0% | |
| 29 | Claude Opus 5.5 | Claude Code | Anthropic | 0.0% | |
| 30 | Nemotron 3 Ultra 550B A55B | Pi | NVIDIA | 0.0% | |
| 31 | Nemotron 3.5 Lightning | Pi | NVIDIA | 0.0% |
Methodology and provenance
Scores use a 0–100 percentage scale. See Paper PDF, Original publication, Repository for task construction, grading, aggregation, and uncertainty details.
LSVP: Anthropic's safety-filter-exempt evaluation variant, which produces fewer refusals on biology prompts. LSVP results are reported alongside the model's standard run and are not a replacement for it.