APIFlow-Bench 1.0 Leaderboard
Deterministic API engineering benchmark spanning 7 capability axes, over REST APIs only (GraphQL and MCP are planned). Showing the 10-subtask chain task set: 1,440 trials across 24 models and 12 tasks covering 5 of the 7 axes — authentication, discovery, error recovery, schema, statefulness (all 7 under All tasks). Switch task set above; click any card to drill into a slice with the full interactive detail view; click any model name to see its per-task results.
Showing 10-subtask chain — 12 tasks, 1,440 trials. Back to the default 20-subtask chain view.
Overall Ranking
Columns are grouped into metric families. Pass rate = passed ÷ all trials; API recovery = passed ÷ trials that hit a 4xx/5xx. Difficulty/horizon breakdowns are omitted: every task in this slice is judged Hard + Long-horizon and Long-horizon. Per-trial details (verifier groups, checkpoints, step counts) live on the model → task → trial pages.
| Rank | Model | Scoring | Cost / efficiency | Behavioral | |||||
|---|---|---|---|---|---|---|---|---|---|
| Pass rate | API recovery | Tool calls / trial | Time / trial | Tokens / trial | Cost / trial | Abstain % | Refusal % | ||
| 1 | minimax-m2p7 | 75.0%45/60 | 24/28 | 12.4 | 19.6s | 41.5K | $0.009 | 0.0%0/60 | 0.0%0/60 |
| 1 | claude-haiku-4-5-20251001 | 75.0%45/60 | 20/22 | 15.1 | 29.7s | 67.2K | $0.022 | 0.0%0/60 | 0.0%0/60 |
| 1 | claude-sonnet-4-6 | 75.0%45/60 | 24/24 | 10.0 | 33.0s | 37.3K | $0.027 | 0.0%0/60 | 0.0%0/60 |
| 1 | gpt-5.5 | 75.0%45/60 | 21/21 | 12.3 | 28.5s | 28.8K | $0.073 | 0.0%0/60 | 0.0%0/60 |
| 1 | qwen3p6-plus | 75.0%45/60 | 25/29 | 12.9 | 38.8s | 52.3K | $0.022 | 0.0%0/60 | 0.0%0/60 |
| 1 | deepseek-v4-flash | 75.0%45/60 | 24/25 | 16.9 | 41.4s | 56.5K | $0.006 | 0.0%0/60 | 0.0%0/60 |
| 1 | kimi-k2p6 | 75.0%45/60 | 22/25 | 13.3 | 45.2s | 55.8K | $0.028 | 0.0%0/60 | 0.0%0/60 |
| 1 | gpt-5.6-sol | 75.0%45/60 | 23/23 | 13.0 | 2.1m | 29.8K | $0.070 | 0.0%0/60 | 0.0%0/60 |
| 1 | glm-5p2 | 75.0%45/60 | 23/26 | 13.0 | 2.0m | 44.5K | $0.039 | 0.0%0/60 | 0.0%0/60 |
| 1 | deepseek-v4-pro | 75.0%45/60 | 29/38 | 18.1 | 2.8m | 66.7K | $0.087 | 0.0%0/60 | 0.0%0/60 |
| 11 | claude-sonnet-5 | 73.3%44/60 | 25/28 | 14.3 | 35.6s | 70.7K | $0.032 | 0.0%0/60 | 0.0%0/60 |
| 11 | gpt-oss-20b | 73.3%44/60 | 22/25 | 14.6 | 1.4m | 67.3K | $0.003 | 0.0%0/60 | 0.0%0/60 |
| 11 | kimi-k2p7-code | 73.3%44/60 | 24/26 | 11.8 | 1.9m | 32.1K | $0.015 | 0.0%0/60 | 0.0%0/60 |
| 11 | qwen3p7-plus | 73.3%44/60 | 23/24 | 9.9 | 3.0m | 66.3K | $0.018 | 0.0%0/60 | 0.0%0/60 |
| 15 | gpt-5.6-terra | 71.7%43/60 | 25/25 | 13.5 | 1.2m | 35.3K | $0.035 | 0.0%0/60 | 0.0%0/60 |
| 15 | kimi-k3 | 71.7%43/60 | 24/30 | 14.3 | 6.9m | 55.6K | $0.094 | 1.7%1/60 | 0.0%0/60 |
| 17 | kimi-k2p5 | 68.3%41/60 | 24/25 | 10.5 | 28.8s | 31.9K | $0.015 | 0.0%0/60 | 0.0%0/60 |
| 17 | claude-opus-4-8 | 68.3%41/60 | 25/26 | 10.5 | 37.5s | 53.3K | $0.063 | 0.0%0/60 | 0.0%0/60 |
| 17 | gpt-5.6-luna | 68.3%41/60 | 23/25 | 13.5 | 1.5m | 29.4K | $0.012 | 0.0%0/60 | 0.0%0/60 |
| 20 | gpt-oss-120b | 61.7%37/60 | 23/28 | 13.1 | 24.8s | 56.1K | $0.006 | 0.0%0/60 | 0.0%0/60 |
| 21 | gpt-5-mini | 58.3%35/60 | 16/19 | 10.6 | 33.0s | 44.2K | $0.005 | 11.7%7/60 | 0.0%0/60 |
| 22 | gpt-5.4-mini | 45.0%27/60 | 15/29 | 11.0 | 14.7s | 22.0K | $0.008 | 6.7%4/60 | 0.0%0/60 |
| 23 | minimax-m3 | 25.0%15/60 | 15/60 | 122.2 | 20.5m | 2.6M | $0.19 | 35.0%21/60 | 0.0%0/60 |
| 24 | claude-fable-5 | 8.3%5/60 | 0/1 | 3.0 | 53.9s | 13.6K | $0.028 | 0.0%0/60 | 91.7%55/60 |
Slices
| Rank | Model | Mix | Pass rate |
|---|---|---|---|
| 1 | minimax-m2p7 | 75.0% | |
| 1 | claude-haiku-4-5-20251001 | 75.0% | |
| 1 | claude-sonnet-4-6 | 75.0% | |
| 1 | gpt-5.5 | 75.0% | |
| 1 | qwen3p6-plus | 75.0% | |
| 1 | deepseek-v4-flash | 75.0% | |
| 1 | kimi-k2p6 | 75.0% | |
| 1 | gpt-5.6-sol | 75.0% | |
| 1 | glm-5p2 | 75.0% | |
| 1 | deepseek-v4-pro | 75.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 11 | claude-sonnet-5 | 90.0% |
| 11 | deepseek-v4-flash | 90.0% |
| 11 | kimi-k2p7-code | 90.0% |
| 11 | kimi-k3 | 90.0% |
| 15 | gpt-5.6-terra | 80.0% |
| 15 | gpt-oss-20b | 80.0% |
| 17 | gpt-5-mini | 70.0% |
| 17 | gpt-5.6-luna | 70.0% |
| 19 | kimi-k2p5 | 60.0% |
| 19 | claude-opus-4-8 | 60.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 1 | deepseek-v4-flash | 64.0% |
| 1 | gpt-oss-20b | 64.0% |
| 3 | minimax-m2p7 | 60.0% |
| 3 | claude-sonnet-5 | 60.0% |
| 3 | claude-sonnet-4-6 | 60.0% |
| 3 | gpt-oss-120b | 60.0% |
| 3 | gpt-5.5 | 60.0% |
| 3 | claude-opus-4-8 | 60.0% |
| 3 | kimi-k2p5 | 60.0% |
| 3 | claude-haiku-4-5-20251001 | 60.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 1 | minimax-m2p7 | 50.0% |
| 1 | kimi-k2p5 | 50.0% |
| 1 | gpt-oss-120b | 50.0% |
| 1 | gpt-5.5 | 50.0% |
| 1 | claude-haiku-4-5-20251001 | 50.0% |
| 1 | deepseek-v4-flash | 50.0% |
| 1 | claude-sonnet-4-6 | 50.0% |
| 1 | qwen3p6-plus | 50.0% |
| 1 | claude-sonnet-5 | 50.0% |
| 1 | claude-opus-4-8 | 50.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 20 | gpt-5.4-mini | 80.0% |
| 20 | gpt-5-mini | 80.0% |
| 20 | qwen3p7-plus | 80.0% |
| 23 | claude-fable-5 | 0.0% |
| 23 | minimax-m3 | 0.0% |
| Rank | Model | Pass rate |
|---|---|---|
| 22 | minimax-m3 | 70.0% |
| 23 | gpt-5.4-mini | 50.0% |
| 24 | claude-fable-5 | 0.0% |
Per-Model Results
Drill into any model to see per-task pass rates across its attempted tasks. This build contains aggregate results only (no per-trial pages).