APIFlow-Bench 1.0 Leaderboard
All tasksEasy
Deterministic API engineering benchmark spanning 7 capability axes, over REST APIs only (GraphQL and MCP are planned). Showing all tasks: 8,280 trials across 24 models and 69 tasks covering all 7 axes — authentication, discovery, error recovery, multi-step workflow, pagination, schema, statefulness. Switch task set above; click any card to drill into a slice with the full interactive detail view; click any model name to see its per-task results.
Showing All tasks created as easy — 69 tasks, 8,280 trials. Back to the default 20-subtask chain view.
Overall Ranking
Columns are grouped into metric families. Pass rate = passed ÷ all trials; API recovery = passed ÷ trials that hit a 4xx/5xx. Difficulty bucket is judged per task from its content and reference solve transcripts (Easy / Long-horizon / Hard / Hard + Long-horizon) — a separate taxonomy from the creation-time “Difficulty (as created)” filter, so the two don't map one-to-one. Per-trial details (verifier groups, checkpoints, step counts) live on the model → task → trial pages.
| Rank | Model | Scoring | Difficulty bucket (pass rate) | Horizon (pass rate) | Cost / efficiency | Behavioral | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pass rate | API recovery | Easy | Long-horizon | Hard | Short | Medium | Tool calls / trial | Time / trial | Tokens / trial | Cost / trial | Abstain % | Refusal % | ||
| 1 | claude-sonnet-5 | 97.4%336/345 | 90/91 | 100%195/195 | 100%75/75 | 88%66/75 | 98%231/235 | 95%105/110 | 11.4 | 37.2s | 68.1K | $0.033 | 0.0%0/345 | 0.0%0/345 |
| 2 | kimi-k3 | 97.1%335/345 | 89/92 | 99%193/195 | 100%75/75 | 89%67/75 | 97%228/235 | 97%107/110 | 11.8 | 7.0m | 69.3K | $0.098 | 0.6%2/345 | 0.0%0/345 |
| 3 | claude-opus-4-8 | 95.9%331/345 | 78/81 | 100%195/195 | 100%75/75 | 81%61/75 | 96%225/235 | 96%106/110 | 9.4 | 48.5s | 47.7K | $0.058 | 0.6%2/345 | 0.0%0/345 |
| 4 | qwen3p7-plus | 95.7%330/345 | 76/80 | 100%195/195 | 97%73/75 | 83%62/75 | 96%226/235 | 95%104/110 | 8.7 | 3.4m | 72.4K | $0.025 | 0.3%1/345 | 0.0%0/345 |
| 5 | kimi-k2p5 | 95.4%329/345 | 96/102 | 99%194/195 | 100%75/75 | 80%60/75 | 95%224/235 | 95%105/110 | 8.7 | 29.5s | 30.4K | $0.014 | 0.0%0/345 | 0.0%0/345 |
| 5 | deepseek-v4-flash | 95.4%329/345 | 90/97 | 99%194/195 | 100%75/75 | 80%60/75 | 95%224/235 | 95%105/110 | 14.7 | 46.5s | 87.7K | $0.008 | 0.0%0/345 | 0.0%0/345 |
| 5 | kimi-k2p7-code | 95.4%329/345 | 84/87 | 99%194/195 | 100%75/75 | 80%60/75 | 95%224/235 | 95%105/110 | 11.2 | 1.9m | 46.8K | $0.020 | 0.3%1/345 | 0.0%0/345 |
| 8 | claude-sonnet-4-6 | 94.8%327/345 | 63/66 | 100%195/195 | 100%75/75 | 76%57/75 | 96%225/235 | 93%102/110 | 10.5 | 49.6s | 82.7K | $0.043 | 0.0%0/345 | 0.0%0/345 |
| 8 | gpt-5.5 | 94.8%327/345 | 62/64 | 99%194/195 | 100%75/75 | 77%58/75 | 95%224/235 | 94%103/110 | 11.0 | 32.3s | 33.5K | $0.075 | 0.3%1/345 | 0.0%0/345 |
| 8 | kimi-k2p6 | 94.8%327/345 | 95/96 | 97%190/195 | 99%74/75 | 84%63/75 | 94%220/235 | 97%107/110 | 12.9 | 55.5s | 101.4K | $0.041 | 0.0%0/345 | 0.0%0/345 |
| 8 | deepseek-v4-pro | 94.8%327/345 | 90/97 | 100%195/195 | 100%75/75 | 76%57/75 | 96%225/235 | 93%102/110 | 14.3 | 2.7m | 72.6K | $0.079 | 0.0%0/345 | 0.0%0/345 |
| 12 | minimax-m2p7 | 94.5%326/345 | 120/131 | 98%191/195 | 100%75/75 | 80%60/75 | 94%220/235 | 96%106/110 | 10.2 | 27.1s | 42.2K | $0.011 | 0.0%0/345 | 0.0%0/345 |
| 13 | qwen3p6-plus | 94.2%325/345 | 82/93 | 99%193/195 | 100%75/75 | 76%57/75 | 95%223/235 | 93%102/110 | 11.1 | 55.3s | 65.1K | $0.029 | 0.0%0/345 | 0.0%0/345 |
| 13 | gpt-5.6-sol | 94.2%325/345 | 62/66 | 99%194/195 | 100%75/75 | 75%56/75 | 95%224/235 | 92%101/110 | 11.2 | 1.7m | 27.1K | $0.062 | 0.3%1/345 | 0.0%0/345 |
| 15 | gpt-5.6-luna | 93.9%324/345 | 68/72 | 99%193/195 | 100%75/75 | 75%56/75 | 94%222/235 | 93%102/110 | 10.4 | 1.2m | 22.7K | $0.010 | 0.0%0/345 | 0.0%0/345 |
| 16 | claude-haiku-4-5-20251001 | 93.6%323/345 | 107/115 | 98%191/195 | 99%74/75 | 77%58/75 | 94%221/235 | 93%102/110 | 12.5 | 33.8s | 73.1K | $0.022 | 0.6%2/345 | 0.0%0/345 |
| 17 | gpt-5.6-terra | 92.8%320/345 | 62/71 | 99%194/195 | 100%75/75 | 68%51/75 | 94%220/235 | 91%100/110 | 9.9 | 52.3s | 25.0K | $0.026 | 0.3%1/345 | 0.0%0/345 |
| 17 | glm-5p2 | 92.8%320/345 | 67/78 | 99%194/195 | 97%73/75 | 71%53/75 | 95%223/235 | 88%97/110 | 13.6 | 2.9m | 83.0K | $0.065 | 0.0%0/345 | 0.0%0/345 |
| 19 | gpt-5-mini | 91.9%317/345 | 90/101 | 95%186/195 | 99%74/75 | 76%57/75 | 92%217/235 | 91%100/110 | 9.2 | 29.4s | 37.0K | $0.005 | 3.8%13/345 | 0.0%0/345 |
| 20 | gpt-oss-120b | 91.3%315/345 | 94/103 | 95%185/195 | 95%71/75 | 79%59/75 | 91%215/235 | 91%100/110 | 11.2 | 23.8s | 47.2K | $0.005 | 0.6%2/345 | 0.0%0/345 |
| 21 | gpt-oss-20b | 90.7%313/345 | 95/101 | 93%182/195 | 93%70/75 | 81%61/75 | 91%213/235 | 91%100/110 | 12.2 | 1.3m | 53.9K | $0.003 | 1.2%4/345 | 0.0%0/345 |
| 22 | gpt-5.4-mini | 81.4%281/345 | 70/108 | 85%165/195 | 95%71/75 | 60%45/75 | 82%192/235 | 81%89/110 | 8.5 | 13.5s | 18.0K | $0.007 | 3.5%12/345 | 0.0%0/345 |
| 23 | minimax-m3 | 49.3%170/345 | 167/340 | 67%130/195 | 25%19/75 | 28%21/75 | 60%141/235 | 26%29/110 | 112.3 | 19.3m | 2.8M | $0.20 | 27.2%94/345 | 0.0%0/345 |
| 24 | claude-fable-5 | 15.9%55/345 | 3/5 | 21%41/195 | 12%9/75 | 7%5/75 | 19%45/235 | 9%10/110 | 2.5 | 51.8s | 14.0K | $0.028 | 0.0%0/345 | 85.2%294/345 |
Slices
Behavior Mix
10 models
passedfailedabstainedno-score
| Rank | Model | Mix | Pass rate |
|---|---|---|---|
| 1 | claude-sonnet-5 | 97.4% | |
| 2 | kimi-k3 | 97.1% | |
| 3 | claude-opus-4-8 | 95.9% | |
| 4 | qwen3p7-plus | 95.7% | |
| 5 | kimi-k2p5 | 95.4% | |
| 5 | deepseek-v4-flash | 95.4% | |
| 5 | kimi-k2p7-code | 95.4% | |
| 8 | claude-sonnet-4-6 | 94.8% | |
| 8 | gpt-5.5 | 94.8% | |
| 8 | kimi-k2p6 | 94.8% |
Authentication
480 trials
| Rank | Model | Pass rate |
|---|---|---|
| 1 | kimi-k3 | 90.0% |
| 2 | claude-sonnet-5 | 80.0% |
| 2 | claude-opus-4-8 | 80.0% |
| 2 | qwen3p7-plus | 80.0% |
| 2 | glm-5p2 | 80.0% |
| 6 | minimax-m2p7 | 75.0% |
| 6 | kimi-k2p5 | 75.0% |
| 6 | kimi-k2p6 | 75.0% |
| 6 | gpt-5.6-luna | 75.0% |
| 6 | deepseek-v4-flash | 75.0% |
Discovery
1,440 trials
| Rank | Model | Pass rate |
|---|---|---|
| 16 | gpt-5-mini | 98.3% |
| 16 | claude-haiku-4-5-20251001 | 98.3% |
| 16 | kimi-k2p6 | 98.3% |
| 16 | glm-5p2 | 98.3% |
| 20 | gpt-5.4-mini | 95.0% |
| 21 | gpt-oss-120b | 91.7% |
| 22 | gpt-oss-20b | 86.7% |
| 23 | minimax-m3 | 43.3% |
| 24 | claude-fable-5 | 35.0% |
Error Recovery
1,680 trials
| Rank | Model | Pass rate |
|---|---|---|
| 1 | claude-sonnet-5 | 92.9% |
| 2 | kimi-k3 | 91.4% |
| 3 | kimi-k2p6 | 90.0% |
| 4 | claude-opus-4-8 | 87.1% |
| 5 | kimi-k2p5 | 85.7% |
| 5 | gpt-oss-20b | 85.7% |
| 5 | minimax-m2p7 | 85.7% |
| 5 | deepseek-v4-flash | 85.7% |
| 5 | qwen3p7-plus | 85.7% |
| 5 | kimi-k2p7-code | 85.7% |
Multi-step Workflow
1,080 trials
| Rank | Model | Pass rate |
|---|---|---|
| 10 | kimi-k2p5 | 97.8% |
| 10 | qwen3p6-plus | 97.8% |
| 10 | deepseek-v4-flash | 97.8% |
| 10 | gpt-5.5 | 97.8% |
| 10 | gpt-5.6-luna | 97.8% |
| 10 | qwen3p7-plus | 97.8% |
| 16 | minimax-m2p7 | 95.6% |
| 16 | claude-haiku-4-5-20251001 | 95.6% |
| 16 | glm-5p2 | 95.6% |
| 19 | gpt-oss-120b | 91.1% |
Pagination
1,200 trials
| Rank | Model | Pass rate |
|---|---|---|
| 18 | qwen3p6-plus | 98.0% |
| 18 | gpt-oss-20b | 98.0% |
| 20 | minimax-m2p7 | 96.0% |
| 21 | gpt-5-mini | 94.0% |
| 22 | gpt-5.4-mini | 90.0% |
| 23 | minimax-m3 | 58.0% |
| 24 | claude-fable-5 | 12.0% |
Schema
1,320 trials
| Rank | Model | Pass rate |
|---|---|---|
| 14 | gpt-5.6-luna | 98.2% |
| 14 | gpt-5.6-sol | 98.2% |
| 14 | kimi-k2p7-code | 98.2% |
| 14 | glm-5p2 | 98.2% |
| 18 | gpt-5.6-terra | 96.4% |
| 18 | kimi-k3 | 96.4% |
| 20 | gpt-oss-120b | 92.7% |
| 21 | gpt-5-mini | 90.9% |
| 22 | gpt-5.4-mini | 81.8% |
| 23 | minimax-m3 | 61.8% |
Statefulness
1,080 trials
| Rank | Model | Pass rate |
|---|---|---|
| 18 | claude-haiku-4-5-20251001 | 97.8% |
| 18 | gpt-5-mini | 97.8% |
| 18 | claude-opus-4-8 | 97.8% |
| 18 | gpt-oss-20b | 97.8% |
| 22 | gpt-5.4-mini | 80.0% |
| 23 | minimax-m3 | 64.4% |
| 24 | claude-fable-5 | 15.6% |
Per-Model Results
Drill into any model to see per-task pass rates across its attempted tasks. This build contains aggregate results only (no per-trial pages).
claude-fable-5
345 trials15.9%
claude-haiku-4-5-20251001
345 trials93.6%
claude-opus-4-8
345 trials95.9%
claude-sonnet-4-6
345 trials94.8%
claude-sonnet-5
345 trials97.4%
deepseek-v4-flash
345 trials95.4%
deepseek-v4-pro
345 trials94.8%
glm-5p2
345 trials92.8%
gpt-5-mini
345 trials91.9%
gpt-5.4-mini
345 trials81.4%
gpt-5.5
345 trials94.8%
gpt-5.6-luna
345 trials93.9%
gpt-5.6-sol
345 trials94.2%
gpt-5.6-terra
345 trials92.8%
gpt-oss-120b
345 trials91.3%
gpt-oss-20b
345 trials90.7%
kimi-k2p5
345 trials95.4%
kimi-k2p6
345 trials94.8%
kimi-k2p7-code
345 trials95.4%
kimi-k3
345 trials97.1%
minimax-m2p7
345 trials94.5%
minimax-m3
345 trials49.3%
qwen3p6-plus
345 trials94.2%
qwen3p7-plus
345 trials95.7%