APIFlow-Bench 1.0 Leaderboard | Authentication
How reliably a model obtains and applies credentials — picking the right auth scheme, minting and refreshing tokens, and scoping them correctly — before the actual workflow can start. 64 tasks × 24 models in this slice.
Pass rate
76.2%
Error Rate
0.0%
Trials
7,679
Tasks
64
Models
24
p95 Time
10.8m
p95 Tokens
696,140
Tool Calls Avg
19.7
| Rank | Rank Spread | Model | Pass rate ± 90% CI | Pass (no abstain) | Abstain% | p95 Time | p95 Tokens | Avg Tools |
|---|---|---|---|---|---|---|---|---|
| 1 | 1 ↔ 3 | kimi-k3 | 89.7% 83.4%–95.0% | 90.3% 287/318 | 0.6% 2/320 | 29.4m | 379,539 | 15.9 |
| 2 | 1 ↔ 8 | gpt-5.5 | 87.5% 80.9%–93.8% | 89.7% 280/312 | 2.5% 8/320 | 2.4m | 131,424 | 15.1 |
| 3 | 1 ↔ 10 | claude-sonnet-4-6 | 86.6% 80.0%–93.1% | 87.7% 277/316 | 1.3% 4/320 | 2.5m | 177,199 | 14.7 |
| 4 | 2 ↔ 13 | deepseek-v4-flash | 85.3% 78.1%–91.6% | 85.6% 273/319 | 0.3% 1/320 | 2.2m | 180,552 | 21.3 |
| 5 | 2 ↔ 13 | deepseek-v4-pro | 85.0% 77.8%–91.3% | 85.0% 272/320 | 0.0% 0/320 | 8.6m | 170,441 | 20.9 |
| 5 | 2 ↔ 13 | glm-5p2 | 85.0% 77.8%–91.6% | 85.8% 272/317 | 0.9% 3/320 | 11.6m | 270,892 | 15.6 |
| 7 | 3 ↔ 14 | gpt-5.6-sol | 84.7% 76.9%–91.6% | 87.4% 271/310 | 3.1% 10/320 | 5.8m | 94,192 | 13.7 |
| 8 | 3 ↔ 15 | kimi-k2p6 | 84.4% 77.2%–90.9% | 84.9% 270/318 | 0.6% 2/320 | 2.6m | 181,702 | 15.3 |
| 8 | 3 ↔ 14 | qwen3p6-plus | 84.4% 77.2%–91.0% | 84.6% 270/319 | 0.3% 1/320 | 2.8m | 212,156 | 14.5 |
| 10 | 3 ↔ 15 | kimi-k2p7-code | 83.8% 76.9%–90.9% | 85.6% 268/313 | 2.2% 7/320 | 4.0m | 107,011 | 14.2 |
| 11 | 5 ↔ 16 | claude-sonnet-5 | 82.8% 75.6%–89.7% | 84.9% 265/312 | 2.5% 8/320 | 1.9m | 209,378 | 15.6 |
| 11 | 5 ↔ 17 | qwen3p7-plus | 82.8% 75.3%–89.4% | 84.9% 265/312 | 2.5% 8/320 | 11.1m | 591,302 | 14 |
| 13 | 5 ↔ 17 | claude-opus-4-8 | 82.5% 75.3%–89.4% | 86.3% 264/306 | 4.4% 14/320 | 1.8m | 101,792 | 12.7 |
| 14 | 9 ↔ 18 | claude-haiku-4-5-20251001 | 80.6% 72.5%–87.8% | 81.9% 258/315 | 1.6% 5/320 | 49.7s | 134,656 | 16 |
| 14 | 8 ↔ 18 | gpt-5.6-terra | 80.6% 73.1%–88.4% | 84.0% 258/307 | 4.1% 13/320 | 3.6m | 129,238 | 13.6 |
| 16 | 9 ↔ 19 | minimax-m2p7 | 80.3% 72.0%–87.8% | 80.5% 256/318 | 0.3% 1/319 | 47.2s | 80,535 | 13.2 |
| 17 | 11 ↔ 20 | gpt-5.6-luna | 78.8% 70.3%–86.6% | 79.7% 252/316 | 1.3% 4/320 | 2.9m | 75,328 | 12.7 |
| 18 | 14 ↔ 21 | kimi-k2p5 | 77.8% 69.4%–85.6% | 79.3% 249/314 | 1.9% 6/320 | 1.3m | 71,694 | 11.9 |
| 19 | 17 ↔ 21 | gpt-oss-120b | 75.0% 67.2%–82.5% | 75.2% 240/319 | 0.3% 1/320 | 46.2s | 126,226 | 14 |
| 19 | 16 ↔ 21 | gpt-5-mini | 75.0% 66.9%–82.8% | 77.9% 240/308 | 3.8% 12/320 | 1.4m | 141,369 | 12.1 |
| 21 | 17 ↔ 21 | gpt-oss-20b | 74.4% 66.6%–82.2% | 75.8% 238/314 | 1.9% 6/320 | 3.8m | 176,259 | 15.9 |
| 22 | 22 ↔ 22 | gpt-5.4-mini | 59.4% 51.6%–67.2% | 60.9% 190/312 | 2.5% 8/320 | 21.6s | 38,400 | 11.4 |
| 23 | 23 ↔ 23 | minimax-m3 | 29.4% 23.4%–35.6% | 46.1% 94/204 | 36.3% 116/320 | 60.8m | 13,146,580 | 145.2 |
| 24 | 24 ↔ 24 | claude-fable-5 | 14.4% 8.4%–20.9% | 14.4% 46/320 | 0.0% 0/320 | 2.9m | 54,485 | 3.5 |
Diagnostics
Loading the full trial dataset for this view…