APIFlow-Bench 1.0 Leaderboard
Deterministic API engineering benchmark spanning 7 capability axes, over REST APIs only (GraphQL and MCP are planned). Showing the 15-subtask chain task set: 1,320 trials across 24 models and 11 tasks covering 6 of the 7 axes — authentication, discovery, error recovery, multi-step workflow, schema, statefulness (all 7 under All tasks). Switch task set above; click any card to drill into a slice with the full interactive detail view; click any model name to see its per-task results.
Showing 15-subtask chain — 11 tasks, 1,320 trials. Back to the default 20-subtask chain view.
Overall Ranking
Columns are grouped into metric families. Pass rate = passed ÷ all trials; API recovery = passed ÷ trials that hit a 4xx/5xx. Difficulty/horizon breakdowns are omitted: every task in this slice is judged Hard + Long-horizon and Long-horizon. Per-trial details (verifier groups, checkpoints, step counts) live on the model → task → trial pages.
| Rank | Model | Scoring | Cost / efficiency | Behavioral | |||||
|---|---|---|---|---|---|---|---|---|---|
| Pass rate | API recovery | Tool calls / trial | Time / trial | Tokens / trial | Cost / trial | Abstain % | Refusal % | ||
| 1 | glm-5p2 | 74.5%41/55 | 16/20 | 13.3 | 1.6m | 46.9K | $0.038 | 0.0%0/55 | 0.0%0/55 |
| 2 | gpt-5.5 | 72.7%40/55 | 16/21 | 11.9 | 24.8s | 26.4K | $0.069 | 0.0%0/55 | 0.0%0/55 |
| 2 | claude-sonnet-5 | 72.7%40/55 | 15/20 | 14.5 | 35.3s | 68.4K | $0.031 | 0.0%0/55 | 0.0%0/55 |
| 2 | claude-opus-4-8 | 72.7%40/55 | 16/21 | 10.6 | 35.9s | 51.2K | $0.062 | 0.0%0/55 | 0.0%0/55 |
| 2 | gpt-5.6-sol | 72.7%40/55 | 15/20 | 12.4 | 2.0m | 27.2K | $0.063 | 0.0%0/55 | 0.0%0/55 |
| 2 | kimi-k2p7-code | 72.7%40/55 | 15/20 | 10.8 | 1.9m | 28.2K | $0.013 | 0.0%0/55 | 0.0%0/55 |
| 2 | kimi-k3 | 72.7%40/55 | 16/21 | 12.1 | 6.6m | 44.5K | $0.075 | 0.0%0/55 | 0.0%0/55 |
| 8 | deepseek-v4-flash | 70.9%39/55 | 16/22 | 17.9 | 38.8s | 52.3K | $0.005 | 0.0%0/55 | 0.0%0/55 |
| 8 | kimi-k2p6 | 70.9%39/55 | 14/19 | 12.3 | 34.9s | 41.5K | $0.020 | 0.0%0/55 | 0.0%0/55 |
| 10 | qwen3p6-plus | 69.1%38/55 | 20/25 | 11.2 | 29.3s | 42.3K | $0.018 | 0.0%0/55 | 0.0%0/55 |
| 10 | claude-sonnet-4-6 | 69.1%38/55 | 13/18 | 10.3 | 32.2s | 37.2K | $0.027 | 0.0%0/55 | 0.0%0/55 |
| 10 | gpt-oss-120b | 69.1%38/55 | 13/18 | 13.7 | 41.6s | 60.2K | $0.006 | 0.0%0/55 | 0.0%0/55 |
| 10 | gpt-5.6-luna | 69.1%38/55 | 14/17 | 11.7 | 1.4m | 25.8K | $0.010 | 0.0%0/55 | 0.0%0/55 |
| 14 | qwen3p7-plus | 67.3%37/55 | 15/20 | 9.2 | 3.5m | 77.5K | $0.031 | 0.0%0/55 | 0.0%0/55 |
| 15 | gpt-5.6-terra | 65.5%36/55 | 11/16 | 12.4 | 1.2m | 30.6K | $0.032 | 0.0%0/55 | 0.0%0/55 |
| 15 | deepseek-v4-pro | 65.5%36/55 | 19/31 | 19.9 | 2.9m | 65.1K | $0.085 | 0.0%0/55 | 0.0%0/55 |
| 17 | minimax-m2p7 | 61.8%34/55 | 12/20 | 12.0 | 21.7s | 39.7K | $0.009 | 0.0%0/55 | 0.0%0/55 |
| 17 | kimi-k2p5 | 61.8%34/55 | 10/16 | 10.5 | 27.0s | 30.0K | $0.014 | 0.0%0/55 | 0.0%0/55 |
| 17 | gpt-oss-20b | 61.8%34/55 | 15/23 | 14.8 | 1.4m | 65.2K | $0.003 | 3.6%2/55 | 0.0%0/55 |
| 20 | claude-haiku-4-5-20251001 | 58.2%32/55 | 11/17 | 13.6 | 25.5s | 57.8K | $0.021 | 1.8%1/55 | 0.0%0/55 |
| 21 | gpt-5-mini | 56.4%31/55 | 10/18 | 11.5 | 30.5s | 46.4K | $0.005 | 10.9%6/55 | 0.0%0/55 |
| 22 | gpt-5.4-mini | 47.3%26/55 | 8/16 | 11.8 | 14.7s | 20.8K | $0.008 | 3.6%2/55 | 0.0%0/55 |
| 23 | minimax-m3 | 16.4%9/55 | 9/55 | 190.5 | 32.7m | 5.5M | $0.38 | 52.7%29/55 | 0.0%0/55 |
| 24 | claude-fable-5 | 12.7%7/55 | 0/4 | 2.7 | 51.9s | 13.3K | $0.027 | 0.0%0/55 | 87.3%48/55 |
Slices
| Rank | Model | Mix | Pass rate |
|---|---|---|---|
| 1 | glm-5p2 | 74.5% | |
| 2 | gpt-5.5 | 72.7% | |
| 2 | claude-sonnet-5 | 72.7% | |
| 2 | claude-opus-4-8 | 72.7% | |
| 2 | gpt-5.6-sol | 72.7% | |
| 2 | kimi-k2p7-code | 72.7% | |
| 2 | kimi-k3 | 72.7% | |
| 8 | deepseek-v4-flash | 70.9% | |
| 8 | kimi-k2p6 | 70.9% | |
| 10 | qwen3p6-plus | 69.1% |
| Rank | Model | Pass rate |
|---|---|---|
| 20 | gpt-oss-20b | 80.0% |
| 21 | gpt-5-mini | 60.0% |
| 22 | gpt-5.4-mini | 40.0% |
| 23 | claude-fable-5 | 0.0% |
| 23 | minimax-m3 | 0.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 1 | gpt-oss-120b | 10.0% |
| 1 | glm-5p2 | 10.0% |
| 3 | minimax-m2p7 | 0.0% |
| 3 | qwen3p6-plus | 0.0% |
| 3 | gpt-5.4-mini | 0.0% |
| 3 | gpt-5.5 | 0.0% |
| 3 | kimi-k2p5 | 0.0% |
| 3 | claude-haiku-4-5-20251001 | 0.0% |
| 3 | gpt-5-mini | 0.0% |
| 3 | claude-opus-4-8 | 0.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 8 | deepseek-v4-flash | 90.0% |
| 8 | kimi-k2p6 | 90.0% |
| 8 | gpt-5.6-luna | 90.0% |
| 11 | claude-sonnet-4-6 | 80.0% |
| 11 | qwen3p6-plus | 80.0% |
| 13 | gpt-oss-120b | 70.0% |
| 13 | gpt-oss-20b | 70.0% |
| 13 | qwen3p7-plus | 70.0% |
| 16 | gpt-5.6-terra | 60.0% |
| 16 | deepseek-v4-pro | 60.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 1 | claude-haiku-4-5-20251001 | 50.0% |
| 1 | minimax-m2p7 | 50.0% |
| 1 | gpt-5.5 | 50.0% |
| 1 | qwen3p6-plus | 50.0% |
| 1 | gpt-oss-120b | 50.0% |
| 1 | gpt-5-mini | 50.0% |
| 1 | claude-sonnet-4-6 | 50.0% |
| 1 | kimi-k2p5 | 50.0% |
| 1 | kimi-k2p6 | 50.0% |
| 1 | claude-sonnet-5 | 50.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 20 | gpt-5.4-mini | 90.0% |
| 20 | gpt-5-mini | 90.0% |
| 20 | gpt-oss-20b | 90.0% |
| 23 | minimax-m3 | 60.0% |
| 24 | claude-fable-5 | 0.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 19 | gpt-5-mini | 90.0% |
| 19 | gpt-5.6-luna | 90.0% |
| 19 | gpt-oss-20b | 90.0% |
| 22 | gpt-5.4-mini | 60.0% |
| 23 | claude-fable-5 | 50.0% |
| 24 | minimax-m3 | 10.0% |
Per-Model Results
Drill into any model to see per-task pass rates across its attempted tasks. This build contains aggregate results only (no per-trial pages).