Overview›Schema

APIFlow-Bench 1.0 Leaderboard | Schema

Whether requests and answers respect the API's data contracts — required fields, types, enums, and response shapes. 59 tasks × 24 models in this slice.

Ranking◴ Generated 2026-07-31☑ 7,080 trials▤ 24 models

Pass rate

84.0%

Error Rate

0.0%

Trials

7,080

Tasks

Models

p95 Time

10.1m

p95 Tokens

536,791

Tool Calls Avg

18.6

Rank	Rank Spread	Model	Pass rate ± 90% CI	Pass (no abstain)	Abstain%	p95 Time	p95 Tokens	Avg Tools
1	1 ↔ 9	kimi-k2p6	93.2% 88.1%–98.0%	93.9% 275/293	0.7% 2/295	8.9m	1,027,557	17.1
2	1 ↔ 11	kimi-k2p5	92.5% 86.8%–97.3%	93.2% 273/293	0.7% 2/295	1.3m	71,428	10.4
2	1 ↔ 11	deepseek-v4-flash	92.5% 86.8%–97.3%	92.5% 273/295	0.0% 0/295	1.7m	204,850	16.7
4	1 ↔ 14	claude-haiku-4-5-20251001	91.2% 85.8%–95.9%	93.1% 269/289	2.0% 6/295	2.1m	392,640	17.8
4	2 ↔ 15	qwen3p6-plus	91.2% 85.8%–96.3%	91.2% 269/295	0.0% 0/295	8.1m	641,386	17.1
6	2 ↔ 17	kimi-k2p7-code	90.8% 84.7%–96.3%	91.5% 268/293	0.7% 2/295	6.1m	230,435	12.8
6	2 ↔ 16	kimi-k3	90.8% 85.8%–95.9%	91.8% 268/292	1.0% 3/295	26.1m	453,078	15.3
8	2 ↔ 17	claude-sonnet-4-6	90.5% 84.1%–95.6%	90.5% 267/295	0.0% 0/295	1.8m	114,805	24.9
8	2 ↔ 16	claude-sonnet-5	90.5% 84.4%–96.6%	92.1% 267/290	1.7% 5/295	3.0m	297,734	13.2
10	3 ↔ 18	gpt-5.5	90.2% 83.7%–95.9%	90.5% 266/294	0.3% 1/295	2.6m	324,266	14.2
11	4 ↔ 19	gpt-5.6-luna	89.5% 82.7%–94.9%	90.1% 264/293	0.7% 2/295	2.7m	67,439	11.6
11	4 ↔ 19	deepseek-v4-pro	89.5% 82.7%–95.9%	89.8% 264/294	0.3% 1/295	12.8m	506,270	20.4
13	5 ↔ 20	gpt-5.6-sol	89.2% 82.4%–94.9%	91.0% 263/289	2.0% 6/295	5.7m	117,216	13.2
14	4 ↔ 20	claude-opus-4-8	88.8% 82.7%–94.6%	91.6% 262/286	3.1% 9/295	3.0m	185,518	12
14	4 ↔ 20	qwen3p7-plus	88.8% 82.0%–94.6%	89.7% 262/292	1.0% 3/295	10.5m	654,654	11.4
16	6 ↔ 20	gpt-5.6-terra	88.5% 81.7%–94.6%	90.0% 261/290	1.7% 5/295	2.4m	80,775	11.5
16	6 ↔ 20	glm-5p2	88.5% 81.7%–94.9%	89.7% 261/291	1.4% 4/295	17.0m	624,474	16.1
18	7 ↔ 21	minimax-m2p7	88.1% 82.0%–93.9%	88.4% 260/294	0.3% 1/295	44.2s	128,265	12.1
19	9 ↔ 21	gpt-oss-120b	87.1% 80.3%–93.2%	87.4% 257/294	0.3% 1/295	43.7s	116,941	12.4
19	11 ↔ 21	gpt-oss-20b	87.1% 81.0%–92.9%	88.6% 257/290	1.7% 5/295	3.9m	136,826	14.1
21	16 ↔ 21	gpt-5-mini	84.7% 78.0%–90.8%	89.6% 250/279	5.4% 16/295	54.4s	102,408	10.5
22	22 ↔ 22	gpt-5.4-mini	75.3% 68.5%–81.7%	76.6% 222/290	1.7% 5/295	22.2s	34,850	9.3
23	23 ↔ 23	minimax-m3	44.7% 37.3%–52.2%	69.8% 132/189	35.9% 106/295	58.0m	12,698,913	129.9
24	24 ↔ 24	claude-fable-5	11.9% 6.8%–18.0%	11.9% 35/295	0.0% 0/295	3.0m	49,433	2.6

Diagnostics

Loading the full trial dataset for this view…