APIFlow-Bench 1.0 Leaderboard

5-subtask chainHard

Deterministic API engineering benchmark spanning 7 capability axes, over REST APIs only (GraphQL and MCP are planned). Showing the 5-subtask chain task set: 1,560 trials across 24 models and 13 tasks covering all 7 axes — authentication, discovery, error recovery, multi-step workflow, pagination, schema, statefulness. Switch task set above; click any card to drill into a slice with the full interactive detail view; click any model name to see its per-task results.

Overall pass rate83.8%Trials1,560Models24Tasks13

20-subtask chain (11)15-subtask chain (11)10-subtask chain (12)5-subtask chain (13)Solo subtasks (241)All tasks (467)

Showing 5-subtask chain — 13 tasks, 1,560 trials. Back to the default 20-subtask chain view.

Overall Ranking

Columns are grouped into metric families. Pass rate = passed ÷ all trials; API recovery = passed ÷ trials that hit a 4xx/5xx. Difficulty bucket is judged per task from its content and reference solve transcripts (Easy / Long-horizon / Hard / Hard + Long-horizon) — a separate taxonomy from the creation-time “Difficulty (as created)” filter, so the two don't map one-to-one. Difficulty/horizon breakdowns are omitted: every task in this slice is Long-horizon. Per-trial details (verifier groups, checkpoints, step counts) live on the model → task → trial pages.

Rank	Model	Scoring		Difficulty bucket (pass rate)		Cost / efficiency				Behavioral
Rank	Model	Pass rate	API recovery	Long-horizon	Hard + Long-horizon	Tool calls / trial	Time / trial	Tokens / trial	Cost / trial	Abstain %	Refusal %
1	kimi-k2p6	100.0%65/65	35/35	100%5/5	100%60/60	22.1	1.8m	193.8K	$0.088	0.0%0/65	0.0%0/65
2	deepseek-v4-flash	98.5%64/65	36/37	100%5/5	98%59/60	21.9	57.6s	101.0K	$0.009	0.0%0/65	0.0%0/65
3	claude-haiku-4-5-20251001	96.9%63/65	31/33	100%5/5	97%58/60	32.0	55.5s	333.1K	$0.058	0.0%0/65	0.0%0/65
4	kimi-k2p7-code	95.4%62/65	32/35	100%5/5	95%57/60	16.9	3.0m	73.4K	$0.032	0.0%0/65	0.0%0/65
5	kimi-k3	93.8%61/65	33/37	100%5/5	93%56/60	20.2	13.1m	164.6K	$0.20	0.0%0/65	0.0%0/65
6	kimi-k2p5	92.3%60/65	30/35	100%5/5	92%55/60	12.8	31.5s	44.6K	$0.021	0.0%0/65	0.0%0/65
6	claude-opus-4-8	92.3%60/65	26/31	100%5/5	92%55/60	13.7	45.4s	74.4K	$0.088	0.0%0/65	0.0%0/65
6	minimax-m2p7	92.3%60/65	33/38	100%5/5	92%55/60	18.3	30.6s	103.8K	$0.023	1.5%1/65	0.0%0/65
6	gpt-5.5	92.3%60/65	28/33	100%5/5	92%55/60	18.6	45.0s	68.0K	$0.14	0.0%0/65	0.0%0/65
6	gpt-5.6-sol	92.3%60/65	25/30	100%5/5	92%55/60	16.3	3.0m	48.5K	$0.11	0.0%0/65	0.0%0/65
6	deepseek-v4-pro	92.3%60/65	40/45	100%5/5	92%55/60	25.5	3.9m	111.9K	$0.13	0.0%0/65	0.0%0/65
6	glm-5p2	92.3%60/65	30/35	100%5/5	92%55/60	20.6	5.8m	243.4K	$0.15	0.0%0/65	0.0%0/65
13	gpt-5.6-terra	90.8%59/65	25/30	100%5/5	90%54/60	16.2	1.6m	50.3K	$0.050	0.0%0/65	0.0%0/65
13	qwen3p7-plus	90.8%59/65	27/32	100%5/5	90%54/60	13.0	4.7m	153.1K	$0.042	0.0%0/65	0.0%0/65
15	gpt-5.6-luna	87.7%57/65	22/30	100%5/5	87%52/60	15.0	1.8m	40.5K	$0.016	0.0%0/65	0.0%0/65
16	gpt-oss-120b	86.2%56/65	30/39	100%5/5	85%51/60	16.5	27.3s	80.5K	$0.008	0.0%0/65	0.0%0/65
16	gpt-oss-20b	86.2%56/65	34/42	100%5/5	85%51/60	17.6	1.5m	85.8K	$0.004	6.2%4/65	0.0%0/65
16	qwen3p6-plus	86.2%56/65	28/35	100%5/5	85%51/60	26.2	2.1m	224.0K	$0.089	0.0%0/65	0.0%0/65
16	claude-sonnet-4-6	86.2%56/65	21/30	100%5/5	85%51/60	77.7	4.8m	9.4M	$2.95	0.0%0/65	0.0%0/65
20	claude-sonnet-5	84.6%55/65	23/33	100%5/5	83%50/60	19.8	1.0m	118.8K	$0.059	0.0%0/65	0.0%0/65
21	gpt-5-mini	83.1%54/65	31/34	100%5/5	82%49/60	13.3	34.5s	59.4K	$0.007	13.8%9/65	0.0%0/65
22	gpt-5.4-mini	67.7%44/65	21/36	100%5/5	65%39/60	13.6	17.3s	28.3K	$0.010	4.6%3/65	0.0%0/65
23	minimax-m3	21.5%14/65	14/65	60%3/5	18%11/60	185.0	31.4m	5.3M	$0.36	53.8%35/65	0.0%0/65
24	claude-fable-5	10.8%7/65	0/4	100%5/5	3%2/60	3.4	1.1m	16.0K	$0.031	0.0%0/65	89.2%58/65

Slices

Behavior Mix

10 models

passedfailedabstainedno-score

Rank	Model	Pass rate
1	kimi-k2p6	100.0%
2	deepseek-v4-flash	98.5%
3	claude-haiku-4-5-20251001	96.9%
4	kimi-k2p7-code	95.4%
5	kimi-k3	93.8%
6	kimi-k2p5	92.3%
6	claude-opus-4-8	92.3%
6	minimax-m2p7	92.3%
6	gpt-5.5	92.3%
6	gpt-5.6-sol	92.3%

Authentication

120 trials

Rank	Model	Pass rate
18	claude-opus-4-8	80.0%
19	gpt-oss-20b	60.0%
20	gpt-5.6-luna	40.0%
21	gpt-5.4-mini	20.0%
22	claude-sonnet-5	0.0%
22	claude-fable-5	0.0%
22	minimax-m3	0.0%

Discovery

240 trials

Rank	Model	Pass rate
19	gpt-oss-120b	90.0%
19	gpt-oss-20b	90.0%
21	qwen3p6-plus	80.0%
22	gpt-5.4-mini	50.0%
23	minimax-m3	30.0%
24	claude-fable-5	20.0%

Error Recovery

600 trials

Rank	Model	Pass rate
19	gpt-oss-120b	96.0%
19	gpt-5.6-terra	96.0%
19	gpt-oss-20b	96.0%
22	gpt-5-mini	92.0%
23	minimax-m3	40.0%
24	claude-fable-5	20.0%

Multi-step Workflow

120 trials

Rank	Model	Pass rate
21	gpt-5.4-mini	80.0%
22	gpt-5-mini	0.0%
22	claude-fable-5	0.0%
22	minimax-m3	0.0%

Pagination

120 trials

Rank	Model	Pass rate
17	minimax-m2p7	80.0%
17	gpt-oss-20b	80.0%
19	gpt-5-mini	60.0%
19	qwen3p6-plus	60.0%
21	gpt-5.4-mini	40.0%
22	claude-fable-5	0.0%
22	claude-sonnet-4-6	0.0%
22	minimax-m3	0.0%

Schema

120 trials

Rank	Model	Pass rate
1	kimi-k2p6	100.0%
2	gpt-5-mini	80.0%
2	gpt-oss-20b	80.0%
2	deepseek-v4-flash	80.0%
5	claude-haiku-4-5-20251001	60.0%
6	kimi-k2p7-code	40.0%
7	claude-opus-4-8	20.0%
7	minimax-m2p7	20.0%
7	claude-sonnet-4-6	20.0%
7	kimi-k3	20.0%

Statefulness

240 trials

Rank	Model	Pass rate
18	gpt-5-mini	90.0%
18	qwen3p7-plus	90.0%
20	gpt-oss-120b	80.0%
21	gpt-5.4-mini	70.0%
21	gpt-oss-20b	70.0%
23	minimax-m3	10.0%
24	claude-fable-5	0.0%

Per-Model Results

Drill into any model to see per-task pass rates across its attempted tasks. This build contains aggregate results only (no per-trial pages).