Showing 10-subtask chain — 12 tasks, 1,140 trials. Back to the default 20-subtask chain view.
APIFlow-Bench 1.0 Leaderboard
Deterministic API engineering benchmark spanning 7 capability axes, over REST APIs only (GraphQL and MCP are planned). Showing the 10-subtask chain task set: 1,140 trials across 19 models and 12 tasks covering 5 of the 7 axes — authentication, discovery, error recovery, schema, statefulness (all 7 under All tasks). Switch task set above; click any card to drill into a slice with the full interactive detail view; click any model name to see its per-task results.
☑ 1,140 trials
▤ 19 models
⛁ 12 tasks
⌖ Overall pass rate: 70.6%
Overall Ranking
Columns are grouped into metric families.
Pass rate = passed ÷ all trials;
API recovery = passed ÷ trials that hit a 4xx/5xx. Difficulty/horizon breakdowns are omitted: every task in this slice is judged Hard + Long-horizon and Long-horizon.
Per-trial details (verifier groups, checkpoints, step counts) live on the model → task → trial pages.
| Rank | Model | Scoring | Cost / efficiency | Behavioral |
|---|
| Pass rate? | API recovery? | Tool calls / trial? | Time / trial? | Tokens / trial? | Cost / trial? | Abstain %? | Refusal %? |
|---|
| 1 | minimax-m2p7 | 75.0%45/60 | 24/28 | 12.4 | 19.6s | 41.5K | $0.009 | 0.0%0/60 | 0.0%0/60 |
| 1 | claude-haiku-4-5-20251001 | 75.0%45/60 | 20/22 | 15.1 | 29.7s | 67.2K | $0.022 | 0.0%0/60 | 0.0%0/60 |
| 1 | claude-sonnet-4-6 | 75.0%45/60 | 24/24 | 10.0 | 33.0s | 37.3K | $0.027 | 0.0%0/60 | 0.0%0/60 |
| 1 | gpt-5.5 | 75.0%45/60 | 21/21 | 12.3 | 28.5s | 28.8K | $0.073 | 0.0%0/60 | 0.0%0/60 |
| 1 | qwen3p6-plus | 75.0%45/60 | 25/29 | 12.9 | 38.8s | 52.3K | $0.022 | 0.0%0/60 | 0.0%0/60 |
| 1 | deepseek-v4-flash | 75.0%45/60 | 24/25 | 16.9 | 41.4s | 56.5K | $0.006 | 0.0%0/60 | 0.0%0/60 |
| 1 | kimi-k2p6 | 75.0%45/60 | 22/25 | 13.3 | 45.2s | 55.8K | $0.028 | 0.0%0/60 | 0.0%0/60 |
| 1 | glm-5p2 | 75.0%45/60 | 23/26 | 13.0 | 2.0m | 44.5K | $0.039 | 0.0%0/60 | 0.0%0/60 |
| 1 | deepseek-v4-pro | 75.0%45/60 | 29/38 | 18.1 | 2.8m | 66.7K | $0.087 | 0.0%0/60 | 0.0%0/60 |
| 10 | claude-sonnet-5 | 73.3%44/60 | 25/28 | 14.3 | 35.6s | 70.7K | $0.032 | 0.0%0/60 | 0.0%0/60 |
| 10 | gpt-oss-20b | 73.3%44/60 | 22/25 | 14.6 | 1.4m | 67.3K | $0.003 | 0.0%0/60 | 0.0%0/60 |
| 10 | kimi-k2p7-code | 73.3%44/60 | 24/26 | 11.8 | 1.9m | 32.1K | $0.015 | 0.0%0/60 | 0.0%0/60 |
| 10 | qwen3p7-plus | 73.3%44/60 | 23/24 | 9.9 | 3.0m | 66.3K | $0.018 | 0.0%0/60 | 0.0%0/60 |
| 14 | gpt-5.6-terra | 71.7%43/60 | 25/25 | 13.5 | 1.2m | 35.3K | $0.035 | 0.0%0/60 | 0.0%0/60 |
| 15 | kimi-k2p5 | 68.3%41/60 | 24/25 | 10.5 | 28.8s | 31.9K | $0.015 | 0.0%0/60 | 0.0%0/60 |
| 15 | claude-opus-4-8 | 68.3%41/60 | 25/26 | 10.5 | 37.5s | 53.3K | $0.063 | 0.0%0/60 | 0.0%0/60 |
| 17 | gpt-oss-120b | 61.7%37/60 | 23/28 | 13.1 | 24.8s | 56.1K | $0.006 | 0.0%0/60 | 0.0%0/60 |
| 18 | gpt-5-mini | 58.3%35/60 | 16/19 | 10.6 | 33.0s | 44.2K | $0.005 | 11.7%7/60 | 0.0%0/60 |
| 19 | gpt-5.4-mini | 45.0%27/60 | 15/29 | 11.0 | 14.7s | 22.0K | $0.008 | 6.7%4/60 | 0.0%0/60 |
Slices
▥Behavior Mix
Top 10 of 19 models
passed?failed?abstained (clarify / report_blocked)?no-score?
| Rank? | Model? | Mix? | Pass rate? |
|---|
| 1 | minimax-m2p7 |
| 75.0% |
| 1 | claude-haiku-4-5-20251001 |
| 75.0% |
| 1 | claude-sonnet-4-6 |
| 75.0% |
| 1 | gpt-5.5 |
| 75.0% |
| 1 | qwen3p6-plus |
| 75.0% |
| 1 | deepseek-v4-flash |
| 75.0% |
| 1 | kimi-k2p6 |
| 75.0% |
| 1 | glm-5p2 |
| 75.0% |
| 1 | deepseek-v4-pro |
| 75.0% |
| 10 | claude-sonnet-5 |
| 73.3% |
🔐Authentication
190 trials
| Rank? | Model? | Pass rate? |
|---|
| 1 | 9 models tied at 100% | 100.0% |
| 10 | claude-sonnet-5 | 90.0% |
| 10 | deepseek-v4-flash | 90.0% |
| 10 | kimi-k2p7-code | 90.0% |
| 13 | gpt-5.6-terra | 80.0% |
| 13 | gpt-oss-20b | 80.0% |
| 15 | gpt-5-mini | 70.0% |
| 16 | kimi-k2p5 | 60.0% |
| 16 | claude-opus-4-8 | 60.0% |
| 18 | gpt-5.4-mini | 30.0% |
| 19 | gpt-oss-120b | 20.0% |
| Rank? | Model? | Pass rate? |
|---|
| 1 | deepseek-v4-flash | 64.0% |
| 1 | gpt-oss-20b | 64.0% |
| 3 | minimax-m2p7 | 60.0% |
| 3 | claude-sonnet-5 | 60.0% |
| 3 | claude-sonnet-4-6 | 60.0% |
| 3 | gpt-oss-120b | 60.0% |
| 3 | gpt-5.5 | 60.0% |
| 3 | claude-opus-4-8 | 60.0% |
| 3 | kimi-k2p5 | 60.0% |
| 3 | claude-haiku-4-5-20251001 | 60.0% |
+9 more models below — click for the full ranking of all 19.
↻Error Recovery
190 trials
| Rank? | Model? | Pass rate? |
|---|
| 1 | minimax-m2p7 | 50.0% |
| 1 | kimi-k2p5 | 50.0% |
| 1 | gpt-oss-120b | 50.0% |
| 1 | gpt-5.5 | 50.0% |
| 1 | claude-haiku-4-5-20251001 | 50.0% |
| 1 | deepseek-v4-flash | 50.0% |
| 1 | claude-sonnet-4-6 | 50.0% |
| 1 | qwen3p6-plus | 50.0% |
| 1 | claude-sonnet-5 | 50.0% |
| 1 | claude-opus-4-8 | 50.0% |
+9 more models below — click for the full ranking of all 19.
| Rank? | Model? | Pass rate? |
|---|
| 1 | 16 models tied at 100% | 100.0% |
| 17 | gpt-5.4-mini | 80.0% |
| 17 | gpt-5-mini | 80.0% |
| 17 | qwen3p7-plus | 80.0% |
| Rank? | Model? | Pass rate? |
|---|
| 1 | 18 models tied at 100% | 100.0% |
| 19 | gpt-5.4-mini | 50.0% |
Per-Model Results
Drill into any model to see per-task pass rates across its attempted tasks. This build contains aggregate results only (no per-trial pages).
