Showing 15-subtask chain — 11 tasks, 1,045 trials. Back to the default 20-subtask chain view.
APIFlow-Bench 1.0 Leaderboard
Deterministic API engineering benchmark spanning 7 capability axes, over REST APIs only (GraphQL and MCP are planned). Showing the 15-subtask chain task set: 1,045 trials across 19 models and 11 tasks covering 6 of the 7 axes — authentication, discovery, error recovery, multi-step workflow, schema, statefulness (all 7 under All tasks). Switch task set above; click any card to drill into a slice with the full interactive detail view; click any model name to see its per-task results.
☑ 1,045 trials
▤ 19 models
⛁ 11 tasks
⌖ Overall pass rate: 66.3%
Overall Ranking
Columns are grouped into metric families.
Pass rate = passed ÷ all trials;
API recovery = passed ÷ trials that hit a 4xx/5xx. Difficulty/horizon breakdowns are omitted: every task in this slice is judged Hard + Long-horizon and Long-horizon.
Per-trial details (verifier groups, checkpoints, step counts) live on the model → task → trial pages.
| Rank | Model | Scoring | Cost / efficiency | Behavioral |
|---|
| Pass rate? | API recovery? | Tool calls / trial? | Time / trial? | Tokens / trial? | Cost / trial? | Abstain %? | Refusal %? |
|---|
| 1 | glm-5p2 | 74.5%41/55 | 16/20 | 13.3 | 1.6m | 46.9K | $0.038 | 0.0%0/55 | 0.0%0/55 |
| 2 | gpt-5.5 | 72.7%40/55 | 16/21 | 11.9 | 24.8s | 26.4K | $0.069 | 0.0%0/55 | 0.0%0/55 |
| 2 | claude-sonnet-5 | 72.7%40/55 | 15/20 | 14.5 | 35.3s | 68.4K | $0.031 | 0.0%0/55 | 0.0%0/55 |
| 2 | claude-opus-4-8 | 72.7%40/55 | 16/21 | 10.6 | 35.9s | 51.2K | $0.062 | 0.0%0/55 | 0.0%0/55 |
| 2 | kimi-k2p7-code | 72.7%40/55 | 15/20 | 10.8 | 1.9m | 28.2K | $0.013 | 0.0%0/55 | 0.0%0/55 |
| 6 | deepseek-v4-flash | 70.9%39/55 | 16/22 | 17.9 | 38.8s | 52.3K | $0.005 | 0.0%0/55 | 0.0%0/55 |
| 6 | kimi-k2p6 | 70.9%39/55 | 14/19 | 12.3 | 34.9s | 41.5K | $0.020 | 0.0%0/55 | 0.0%0/55 |
| 8 | qwen3p6-plus | 69.1%38/55 | 20/25 | 11.2 | 29.3s | 42.3K | $0.018 | 0.0%0/55 | 0.0%0/55 |
| 8 | claude-sonnet-4-6 | 69.1%38/55 | 13/18 | 10.3 | 32.2s | 37.2K | $0.027 | 0.0%0/55 | 0.0%0/55 |
| 8 | gpt-oss-120b | 69.1%38/55 | 13/18 | 13.7 | 41.6s | 60.2K | $0.006 | 0.0%0/55 | 0.0%0/55 |
| 11 | qwen3p7-plus | 67.3%37/55 | 15/20 | 9.2 | 3.5m | 77.5K | $0.031 | 0.0%0/55 | 0.0%0/55 |
| 12 | gpt-5.6-terra | 65.5%36/55 | 11/16 | 12.4 | 1.2m | 30.6K | $0.032 | 0.0%0/55 | 0.0%0/55 |
| 12 | deepseek-v4-pro | 65.5%36/55 | 19/31 | 19.9 | 2.9m | 65.1K | $0.085 | 0.0%0/55 | 0.0%0/55 |
| 14 | minimax-m2p7 | 61.8%34/55 | 12/20 | 12.0 | 21.7s | 39.7K | $0.009 | 0.0%0/55 | 0.0%0/55 |
| 14 | kimi-k2p5 | 61.8%34/55 | 10/16 | 10.5 | 27.0s | 30.0K | $0.014 | 0.0%0/55 | 0.0%0/55 |
| 14 | gpt-oss-20b | 61.8%34/55 | 15/23 | 14.8 | 1.4m | 65.2K | $0.003 | 3.6%2/55 | 0.0%0/55 |
| 17 | claude-haiku-4-5-20251001 | 58.2%32/55 | 11/17 | 13.6 | 25.5s | 57.8K | $0.021 | 1.8%1/55 | 0.0%0/55 |
| 18 | gpt-5-mini | 56.4%31/55 | 10/18 | 11.5 | 30.5s | 46.4K | $0.005 | 10.9%6/55 | 0.0%0/55 |
| 19 | gpt-5.4-mini | 47.3%26/55 | 8/16 | 11.8 | 14.7s | 20.8K | $0.008 | 3.6%2/55 | 0.0%0/55 |
Slices
▥Behavior Mix
Top 10 of 19 models
passed?failed?abstained (clarify / report_blocked)?no-score?
| Rank? | Model? | Mix? | Pass rate? |
|---|
| 1 | glm-5p2 |
| 74.5% |
| 2 | gpt-5.5 |
| 72.7% |
| 2 | claude-sonnet-5 |
| 72.7% |
| 2 | claude-opus-4-8 |
| 72.7% |
| 2 | kimi-k2p7-code |
| 72.7% |
| 6 | deepseek-v4-flash |
| 70.9% |
| 6 | kimi-k2p6 |
| 70.9% |
| 8 | qwen3p6-plus |
| 69.1% |
| 8 | claude-sonnet-4-6 |
| 69.1% |
| 8 | gpt-oss-120b |
| 69.1% |
🔐Authentication
95 trials
| Rank? | Model? | Pass rate? |
|---|
| 1 | 16 models tied at 100% | 100.0% |
| 17 | gpt-oss-20b | 80.0% |
| 18 | gpt-5-mini | 60.0% |
| 19 | gpt-5.4-mini | 40.0% |
| Rank? | Model? | Pass rate? |
|---|
| 1 | gpt-oss-120b | 10.0% |
| 1 | glm-5p2 | 10.0% |
| 3 | minimax-m2p7 | 0.0% |
| 3 | qwen3p6-plus | 0.0% |
| 3 | gpt-5.4-mini | 0.0% |
| 3 | gpt-5.5 | 0.0% |
| 3 | kimi-k2p5 | 0.0% |
| 3 | claude-haiku-4-5-20251001 | 0.0% |
| 3 | gpt-5-mini | 0.0% |
| 3 | claude-opus-4-8 | 0.0% |
+9 more models below — click for the full ranking of all 19.
↻Error Recovery
190 trials
| Rank? | Model? | Pass rate? |
|---|
| 1 | 5 models tied at 100% | 100.0% |
| 6 | deepseek-v4-flash | 90.0% |
| 6 | kimi-k2p6 | 90.0% |
| 8 | claude-sonnet-4-6 | 80.0% |
| 8 | qwen3p6-plus | 80.0% |
| 10 | gpt-oss-120b | 70.0% |
| 10 | gpt-oss-20b | 70.0% |
| 10 | qwen3p7-plus | 70.0% |
| 13 | gpt-5.6-terra | 60.0% |
| 13 | deepseek-v4-pro | 60.0% |
| 15 | gpt-5.4-mini | 50.0% |
+4 more models below — click for the full ranking of all 19.
⛓Multi-step Workflow
190 trials
| Rank? | Model? | Pass rate? |
|---|
| 1 | claude-haiku-4-5-20251001 | 50.0% |
| 1 | minimax-m2p7 | 50.0% |
| 1 | gpt-5.5 | 50.0% |
| 1 | qwen3p6-plus | 50.0% |
| 1 | gpt-oss-120b | 50.0% |
| 1 | gpt-5-mini | 50.0% |
| 1 | claude-sonnet-4-6 | 50.0% |
| 1 | kimi-k2p5 | 50.0% |
| 1 | kimi-k2p6 | 50.0% |
| 1 | claude-sonnet-5 | 50.0% |
+9 more models below — click for the full ranking of all 19.
| Rank? | Model? | Pass rate? |
|---|
| 1 | 16 models tied at 100% | 100.0% |
| 17 | gpt-5.4-mini | 90.0% |
| 17 | gpt-5-mini | 90.0% |
| 17 | gpt-oss-20b | 90.0% |
| Rank? | Model? | Pass rate? |
|---|
| 1 | 16 models tied at 100% | 100.0% |
| 17 | gpt-5-mini | 90.0% |
| 17 | gpt-oss-20b | 90.0% |
| 19 | gpt-5.4-mini | 60.0% |
Per-Model Results
Drill into any model to see per-task pass rates across its attempted tasks. This build contains aggregate results only (no per-trial pages).
