APIFlow-Bench 1.0 Leaderboard | Behavior Mix | Overall
Every trial ends one of four ways: passed, failed, abstained (the model asked for clarification or declined), or no-score (runtime error). This page shows each model's mix, separating "tried and failed" from "chose not to try". 467 tasks × 24 models in this slice.
Pass rate
80.4%
Error Rate
0.0%
Trials
56,037
Tasks
467
Models
24
p95 Time
11.1m
p95 Tokens
583,536
Tool Calls Avg
20
▥Behavior Mix
24 models
passedfailedabstainedno-score
ModelPass / Fail / Abstain / ErrorScore
89.9%
89.0%
88.6%
88.4%
87.9%
87.8%
87.8%
87.6%
87.3%
87.3%
87.1%
86.7%
86.0%
86.0%
85.9%
85.5%
85.5%
85.4%
82.0%
80.9%
80.6%
68.5%
32.7%
14.5%
| Rank | Rank Spread | Model | Pass rate ± 90% CI | Pass (no abstain) | Abstain% | p95 Time | p95 Tokens | Avg Tools |
|---|---|---|---|---|---|---|---|---|
| 1 | 1 ↔ 3 | kimi-k3 | 89.9% 87.8%–91.8% | 90.3% 2,098/2,323 | 0.5% 12/2,335 | 25.4m | 439,242 | 16.7 |
| 2 | 1 ↔ 6 | claude-sonnet-4-6 | 89.0% 86.7%–91.1% | 89.1% 2,077/2,331 | 0.2% 4/2,335 | 1.7m | 122,554 | 18.9 |
| 3 | 1 ↔ 8 | claude-opus-4-8 | 88.6% 86.3%–90.8% | 90.7% 2,068/2,280 | 2.4% 55/2,335 | 1.8m | 136,460 | 12.1 |
| 4 | 2 ↔ 9 | deepseek-v4-flash | 88.4% 86.2%–90.5% | 88.6% 2,065/2,331 | 0.2% 4/2,335 | 2.2m | 229,055 | 19.6 |
| 5 | 2 ↔ 11 | kimi-k2p6 | 87.9% 85.3%–90.1% | 88.2% 2,052/2,327 | 0.3% 8/2,335 | 5.2m | 612,911 | 17 |
| 6 | 3 ↔ 11 | gpt-5.5 | 87.8% 85.4%–90.1% | 88.4% 2,051/2,320 | 0.6% 15/2,335 | 2.0m | 148,050 | 14.8 |
| 7 | 3 ↔ 12 | claude-sonnet-5 | 87.8% 85.4%–90.1% | 88.7% 2,050/2,311 | 1.0% 24/2,335 | 2.4m | 249,813 | 15.3 |
| 8 | 3 ↔ 13 | kimi-k2p7-code | 87.6% 85.2%–90.0% | 88.2% 2,046/2,319 | 0.7% 16/2,335 | 6.1m | 164,514 | 13.9 |
| 9 | 4 ↔ 13 | gpt-5.6-sol | 87.3% 84.8%–89.6% | 88.6% 2,039/2,301 | 1.5% 34/2,335 | 5.9m | 116,628 | 14.1 |
| 10 | 4 ↔ 14 | qwen3p6-plus | 87.3% 85.0%–89.6% | 87.4% 2,038/2,332 | 0.1% 2/2,334 | 6.5m | 381,981 | 16.3 |
| 11 | 5 ↔ 14 | glm-5p2 | 87.1% 84.6%–89.4% | 87.4% 2,033/2,327 | 0.3% 8/2,335 | 19.6m | 542,342 | 16.3 |
| 12 | 7 ↔ 16 | deepseek-v4-pro | 86.7% 84.2%–89.1% | 86.8% 2,024/2,332 | 0.1% 3/2,335 | 9.4m | 220,436 | 20.4 |
| 13 | 10 ↔ 18 | claude-haiku-4-5-20251001 | 86.0% 83.4%–88.3% | 86.8% 2,007/2,311 | 1.0% 24/2,335 | 1.5m | 211,661 | 16.5 |
| 13 | 10 ↔ 18 | gpt-5.6-luna | 86.0% 83.3%–88.5% | 86.5% 2,007/2,320 | 0.6% 15/2,335 | 2.7m | 68,411 | 13 |
| 15 | 11 ↔ 18 | kimi-k2p5 | 85.9% 83.1%–88.3% | 86.4% 2,005/2,320 | 0.6% 15/2,335 | 1.6m | 95,407 | 12.3 |
| 16 | 12 ↔ 18 | qwen3p7-plus | 85.5% 83.1%–87.8% | 86.6% 1,997/2,307 | 1.2% 28/2,335 | 10.5m | 542,519 | 12.1 |
| 17 | 12 ↔ 18 | minimax-m2p7 | 85.5% 82.9%–87.9% | 85.8% 1,995/2,324 | 0.4% 10/2,334 | 1.1m | 137,719 | 12.9 |
| 18 | 12 ↔ 18 | gpt-5.6-terra | 85.4% 82.7%–87.9% | 86.5% 1,993/2,304 | 1.3% 31/2,335 | 2.6m | 90,690 | 13 |
| 19 | 19 ↔ 21 | gpt-oss-120b | 82.0% 79.2%–84.5% | 82.2% 1,914/2,328 | 0.3% 7/2,335 | 50.3s | 126,916 | 13.5 |
| 20 | 19 ↔ 21 | gpt-oss-20b | 80.9% 78.4%–83.5% | 82.5% 1,889/2,289 | 1.9% 45/2,334 | 3.9m | 150,185 | 14.9 |
| 21 | 19 ↔ 21 | gpt-5-mini | 80.6% 78.0%–83.2% | 85.2% 1,881/2,207 | 5.5% 128/2,335 | 1.1m | 105,151 | 11.5 |
| 22 | 22 ↔ 22 | gpt-5.4-mini | 68.5% 65.7%–71.3% | 70.7% 1,599/2,263 | 3.1% 72/2,335 | 22.9s | 39,015 | 11 |
| 23 | 23 ↔ 23 | minimax-m3 | 32.7% 30.3%–35.1% | 54.1% 764/1,413 | 39.5% 922/2,335 | 63.5m | 14,499,727 | 151.4 |
| 24 | 24 ↔ 24 | claude-fable-5 | 14.5% 12.3%–16.7% | 14.5% 339/2,335 | 0.0% 0/2,335 | 3.1m | 56,719 | 3.3 |
Diagnostics
Loading the full trial dataset for this view…