Overview›Error Recovery

APIFlow-Bench 1.0 Leaderboard | Error Recovery

How a model reacts when the API pushes back — 4xx/5xx responses, rate limits, and injected faults it must diagnose and work around. 76 tasks × 24 models in this slice.

Ranking◴ Generated 2026-07-31☑ 9,120 trials▤ 24 models

Pass rate

79.8%

Error Rate

0.0%

Trials

9,120

Tasks

Models

p95 Time

10.5m

p95 Tokens

557,608

Tool Calls Avg

19.2

Rank	Rank Spread	Model	Pass rate ± 90% CI	Pass (no abstain)	Abstain%	p95 Time	p95 Tokens	Avg Tools
1	1 ↔ 7	kimi-k3	90.3% 84.7%–95.0%	91.2% 343/376	1.1% 4/380	33.9m	488,335	17.8
2	1 ↔ 8	claude-opus-4-8	89.5% 83.9%–94.5%	89.9% 340/378	0.5% 2/380	1.6m	136,253	12.2
3	1 ↔ 10	claude-sonnet-5	88.9% 83.2%–94.2%	88.9% 338/380	0.0% 0/380	2.6m	316,763	15.7
4	1 ↔ 14	deepseek-v4-flash	87.9% 82.1%–93.4%	87.9% 334/380	0.0% 0/380	2.4m	176,290	18.9
5	2 ↔ 13	claude-sonnet-4-6	87.6% 81.6%–93.2%	87.6% 333/380	0.0% 0/380	1.2m	93,265	18.4
6	2 ↔ 14	kimi-k2p7-code	87.4% 81.6%–92.9%	87.4% 332/380	0.0% 0/380	5.6m	126,759	12.9
7	3 ↔ 15	kimi-k2p6	87.1% 81.1%–93.2%	87.6% 331/378	0.5% 2/380	4.9m	547,486	16.7
7	3 ↔ 16	qwen3p7-plus	87.1% 81.1%–92.4%	87.6% 331/378	0.5% 2/380	9.0m	322,833	10.3
9	3 ↔ 16	qwen3p6-plus	86.8% 81.1%–92.4%	86.8% 330/380	0.0% 0/380	1.8m	133,919	13.9
10	3 ↔ 16	gpt-5.6-sol	86.6% 80.3%–92.6%	86.8% 329/379	0.3% 1/380	5.2m	95,191	13.5
11	5 ↔ 18	minimax-m2p7	85.8% 79.7%–91.3%	86.2% 326/378	0.5% 2/380	45.9s	123,683	12.2
12	4 ↔ 18	gpt-5.5	85.5% 78.9%–91.8%	85.8% 325/379	0.3% 1/380	2.8m	240,143	15
13	6 ↔ 19	gpt-5.6-luna	85.0% 78.4%–91.1%	85.0% 323/380	0.0% 0/380	2.6m	62,687	12.5
13	5 ↔ 19	glm-5p2	85.0% 78.4%–91.1%	85.2% 323/379	0.3% 1/380	30.1m	738,101	17.1
15	6 ↔ 20	deepseek-v4-pro	84.5% 77.6%–90.8%	84.5% 321/380	0.0% 0/380	7.5m	147,760	19.2
16	8 ↔ 20	kimi-k2p5	84.2% 77.1%–90.3%	84.2% 320/380	0.0% 0/380	1.5m	75,665	11.1
17	9 ↔ 20	gpt-5.6-terra	83.7% 77.1%–90.0%	84.1% 318/378	0.5% 2/380	2.0m	70,509	12.3
18	11 ↔ 21	claude-haiku-4-5-20251001	82.9% 75.8%–89.5%	83.1% 315/379	0.3% 1/380	1.3m	127,279	16.4
19	11 ↔ 21	gpt-oss-120b	82.6% 76.3%–88.2%	83.1% 314/378	0.5% 2/380	53.1s	119,362	12.9
20	14 ↔ 21	gpt-oss-20b	81.3% 75.3%–87.4%	83.3% 309/371	2.4% 9/380	3.0m	133,699	14
21	15 ↔ 21	gpt-5-mini	81.1% 74.2%–87.1%	86.0% 308/358	5.8% 22/380	1.1m	90,861	10.6
22	22 ↔ 22	gpt-5.4-mini	67.6% 61.0%–74.5%	69.5% 257/370	2.6% 10/380	21.8s	36,042	10.7
23	23 ↔ 23	minimax-m3	37.6% 31.6%–43.7%	59.6% 143/240	36.8% 140/380	61.6m	13,888,201	145.2
24	24 ↔ 24	claude-fable-5	8.7% 3.9%–13.7%	8.7% 33/380	0.0% 0/380	2.3m	37,919	1.9

Diagnostics

Loading the full trial dataset for this view…