arena.ai · benchmark source

Arena

Live human-preference and agent boards from Arena (LMArena). Scores are board-specific — Text Arena Score, WebDev Arena Score, and Agent net improvement are not interchangeable. The Arena product home at arena.ai/ and the long-running lmarena.ai leaderboards are the same operator (verified 2026-08-29 via arena.ai root and the LMArena research pages; see https://lmarena.ai/ for the historic domain).

Open arena.ai3 boards on this publisher

Agent systems

Agent net improvement

A net-improvement percentage reported for agent systems on the Arena Agent leaderboard.

Why it matters

It provides an agent-specific external signal without presenting a percentage improvement as an Elo rating.

Use it when

You are comparing agent-system results on the Arena Agent board and understand the metric is not a general model rating.

The limitation

It reflects the Arena Agent board and evaluation mix, not factual accuracy, cost, or enterprise safety.

Leaderboard snapshot

Agent net improvement results in a readable view

Use the tabs for the primary score and other published fields from this board. The source link remains authoritative for the live table.

Capture updated October 1 from the September 30 board. Negative net-improvement values mean the agent hurt outcomes versus baseline; mid-board order can shift as sessions accumulate and confidence intervals overlap.

Arena Agent net-improvement capture (48 rows, board dated 2026-09-30). Top: Claude Fable 5.1 (Max) (14.55%), Claude Opus 5.5 (High) (13.78%), GPT 6 Astra (Max) (12.18%).

Leader among verified rows

Claude Fable 5.1 (Max)

+14.55%

Rows shown

Net improvement

Source / snapshot date

2026-10-01

25 days old · not a live API feed

Score profile

Net improvement by published row

Higher is better in this view

Claude Fable 5.1 (Max)

+14.55%

Claude Opus 5.5 (High)

+13.78%

GPT 6 Astra (Max)

+12.18%

GPT 6 Sol (Max)

+10.65%

Claude Opus 5 (High)

+8.76%

Claude Opus 5 (Max)

+8.55%

Claude Fable 5 (High)

+8.37%

Gemini 4 Argon (High)

+7.92%

GPT 5.6 Sol (xHigh)

+7.05%

Claude Opus 4.8 (High)

+6.92%

Claude Sonnet 5 (High)

+4.81%

Hy4 preview

+4.24%

Muse Spark 1.3 (Max)

+4.19%

GPT 5.5 (xHigh)

+4.12%

Deepseek V4.1 Flash (Max)

+4.01%

Kimi K3 (Max)

+3.98%

GLM 5.2 (Max)

+3.64%

Grok 4.7 (xHigh)

+3.64%

MiMo V2.6 Pro

+3.17%

Gemini 3.8 Flash (High)

+2.96%

Qwen3.8 Max

+2.80%

GLM 5.3 (Max)

+2.57%

GPT 6 Luna (Max)

+1.68%

DeepSeek V4 Pro (High) (0813)

+1.29%

Grok 4.6 (xHigh)

+1.16%

GPT 5.5

+1.13%

Grok 4.5

+1.11%

GPT 5.4 (High)

+0.91%

GPT 5.6 Terra (xHigh)

+0.83%

GLM 5.3 Flash

-0.28%

MiMo V2.6 Flash

-0.57%

Qwen3.8 Flash Next

-0.79%

GPT 5.6 Luna (xHigh)

-1.09%

Qwen 3.8 27B

-1.61%

Gemini 3.7 Flash (High)

-1.64%

Muse Spark 1.2 (xHigh)

-2.96%

Muse Spark 1.1

-4.60%

Qwen3.7 Max

-5.20%

Hy3

-5.70%

Minimax M3

-6.71%

Mimo V2.5 Pro

-7.23%

Qwen3.7 Plus

-7.28%

Gemini 3.1 Pro Preview

-7.35%

Gemini 3.6 Flash (High)

-7.67%

Inkling Small

-9.80%

Inkling

-10.90%

Mistral Medium 3.5

-11.74%

Solar Pro 4

-15.77%

Benchmark leaderboard snapshot showing rank, score, published details, and notes
Rank	Model or system	Net improvement	Sessions	Cost / task (P50)	Output tokens / task (P50)	Other details	Note
#1	Claude Fable 5.1 (Max)Anthropic	+14.55%	14,636	$4.50	41.1K	Board: Arena Agent · Overall95% CI: ±1.92%Confirmed Success: Up 17.46% ±2.90%Praise vs Complaint: Up 33.73% ±7.89%Steerability: Up 8.95% ±2.83%Bash Recovery: Up 12.31% ±0.96%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#2	Claude Opus 5.5 (High)Anthropic	+13.78%	5,013	$1.56	25.5K	Board: Arena Agent · Overall95% CI: ±2.24%Confirmed Success: Up 13.95% ±4.25%Praise vs Complaint: Up 30.23% ±9.01%Steerability: Up 11.73% ±2.40%Bash Recovery: Up 12.69% ±0.88%Tool Hallucination: Down 0.30% ±0.04%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#3	GPT 6 Astra (Max)OpenAI	+12.18%	11,077	$2.93	26.2K	Board: Arena Agent · Overall95% CI: ±2.28%Confirmed Success: Up 12.03% ±4.03%Praise vs Complaint: Up 34.70% ±9.48%Steerability: Up 8.24% ±2.73%Bash Recovery: Up 5.59% ±1.63%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#4	GPT 6 Sol (Max)OpenAI	+10.65%	5,031	$0.90	28.3K	Board: Arena Agent · Overall95% CI: ±2.57%Confirmed Success: Up 9.87% ±4.55%Praise vs Complaint: Up 24.32% ±10.53%Steerability: Up 11.87% ±2.21%Bash Recovery: Up 6.84% ±1.99%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#5	Claude Opus 5 (High)Anthropic	+8.76%	26,501	$2.50	30.2K	Board: Arena Agent · Overall95% CI: ±1.50%Confirmed Success: Up 6.84% ±3.18%Praise vs Complaint: Up 17.81% ±5.51%Steerability: Up 6.96% ±1.93%Bash Recovery: Up 11.87% ±0.82%Tool Hallucination: Down 0.31% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#6	Claude Opus 5 (Max)Anthropic	+8.55%	21,714	$3.67	51.9K	Board: Arena Agent · Overall95% CI: ±1.57%Confirmed Success: Up 10.09% ±3.24%Praise vs Complaint: Up 14.51% ±5.84%Steerability: Up 4.76% ±2.06%Bash Recovery: Up 13.05% ±0.79%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#7	Claude Fable 5 (High)Anthropic	+8.37%	39,998	$2.05	15.1K	Board: Arena Agent · Overall95% CI: ±1.30%Confirmed Success: Up 5.50% ±2.76%Praise vs Complaint: Up 18.41% ±4.85%Steerability: Up 8.98% ±1.40%Bash Recovery: Up 8.62% ±1.33%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#8	Gemini 4 Argon (High)Google	+7.92%	3,417	$0.64	21.5K	Board: Arena Agent · Overall95% CI: ±3.24%Confirmed Success: Up 14.15% ±5.07%Praise vs Complaint: Up 27.74% ±12.81%Steerability: Up 15.88% ±2.68%Bash Recovery: Down 18.07% ±5.59%Tool Hallucination: Up 0.10% ±0.48%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#9	GPT 5.6 Sol (xHigh)OpenAI	+7.05%	33,916	$1.05	18.1K	Board: Arena Agent · Overall95% CI: ±1.38%Confirmed Success: Up 5.64% ±2.78%Praise vs Complaint: Up 21.44% ±5.14%Steerability: Up 6.20% ±1.77%Bash Recovery: Up 1.67% ±1.36%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#10	Claude Opus 4.8 (High)Anthropic	+6.92%	40,210	$1.36	18.2K	Board: Arena Agent · Overall95% CI: ±1.38%Confirmed Success: Up 4.88% ±2.77%Praise vs Complaint: Up 15.19% ±5.13%Steerability: Up 7.46% ±1.63%Bash Recovery: Up 7.09% ±1.86%Tool Hallucination: Down 0.00% ±0.20%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#11	Claude Sonnet 5 (High)Anthropic	+4.81%	31,480	$0.91	22.9K	Board: Arena Agent · Overall95% CI: ±1.72%Confirmed Success: Up 1.30% ±3.77%Praise vs Complaint: Up 10.35% ±6.26%Steerability: Up 5.38% ±2.44%Bash Recovery: Up 6.79% ±1.08%Tool Hallucination: Down 0.20% ±0.08%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#12	Hy4 previewTencent	+4.24%	53,732	$0.29	39.2K	Board: Arena Agent · Overall95% CI: ±0.85%Confirmed Success: Up 6.37% ±1.74%Praise vs Complaint: Up 7.67% ±3.20%Steerability: Up 0.92% ±1.07%Bash Recovery: Up 6.26% ±0.55%Tool Hallucination: Up 0.00% ±0.30%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#13	Muse Spark 1.3 (Max)Meta	+4.19%	61,422	$0.41	20K	Board: Arena Agent · Overall95% CI: ±0.59%Confirmed Success: Up 6.25% ±1.29%Praise vs Complaint: Up 3.27% ±2.09%Steerability: Up 5.50% ±0.75%Bash Recovery: Up 5.62% ±0.59%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#14	GPT 5.5 (xHigh)OpenAI	+4.12%	53,913	$1.21	19.7K	Board: Arena Agent · Overall95% CI: ±1.05%Confirmed Success: Down 0.44% ±2.24%Praise vs Complaint: Up 7.56% ±3.72%Steerability: Up 3.88% ±1.41%Bash Recovery: Up 9.26% ±1.16%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#15	Deepseek V4.1 Flash (Max)DeepSeek	+4.01%	95,889	$0.09	57.8K	Board: Arena Agent · Overall95% CI: ±0.50%Confirmed Success: Up 8.65% ±1.10%Praise vs Complaint: Up 2.21% ±1.83%Steerability: Up 1.19% ±0.69%Bash Recovery: Up 7.77% ±0.33%Tool Hallucination: Down 0.20% ±0.04%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#16	Kimi K3 (Max)Moonshot AI	+3.98%	129,908	$0.75	26.8K	Board: Arena Agent · Overall95% CI: ±0.53%Confirmed Success: Up 7.32% ±1.18%Praise vs Complaint: Up 6.61% ±1.89%Steerability: Up 1.42% ±0.73%Bash Recovery: Up 4.25% ±0.50%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#17	GLM 5.2 (Max)Z.ai	+3.64%	83,747	$0.75	48K	Board: Arena Agent · Overall95% CI: ±0.74%Confirmed Success: Up 5.23% ±1.54%Praise vs Complaint: Up 10.17% ±2.67%Steerability: Up 1.77% ±0.95%Bash Recovery: Up 0.68% ±0.83%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#17	Grok 4.7 (xHigh)SpaceXAI	+3.64%	10,109	$1.47	49.1K	Board: Arena Agent · Overall95% CI: ±1.57%Confirmed Success: Up 6.23% ±3.42%Praise vs Complaint: Up 5.06% ±5.73%Steerability: Up 0.37% ±2.25%Bash Recovery: Up 6.23% ±1.09%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#19	MiMo V2.6 ProXiaomi	+3.17%	8,158	$0.09	42.4K	Board: Arena Agent · Overall95% CI: ±1.64%Confirmed Success: Up 7.35% ±3.53%Praise vs Complaint: Up 3.29% ±6.15%Steerability: Up 1.98% ±2.22%Bash Recovery: Up 3.56% ±1.73%Tool Hallucination: Up 0.35% ±0.35%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#20	Gemini 3.8 Flash (High)Google	+2.96%	32,890	$0.58	44.6K	Board: Arena Agent · Overall95% CI: ±0.82%Confirmed Success: Up 7.41% ±1.82%Praise vs Complaint: Up 4.50% ±2.95%Steerability: Up 3.32% ±1.17%Bash Recovery: Down 0.61% ±0.67%Tool Hallucination: Down 0.16% ±0.09%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#21	Qwen3.8 MaxQwen	+2.80%	49,157	$0.44	25.4K	Board: Arena Agent · Overall95% CI: ±0.65%Confirmed Success: Up 5.24% ±1.46%Praise vs Complaint: Up 4.21% ±2.35%Steerability: Up 3.05% ±0.86%Bash Recovery: Up 2.65% ±0.71%Tool Hallucination: Up 1.13% ±0.21%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#22	GLM 5.3 (Max)Z.ai	+2.57%	80,704	$0.64	45.9K	Board: Arena Agent · Overall95% CI: ±0.72%Confirmed Success: Up 5.55% ±1.60%Praise vs Complaint: Up 6.40% ±2.52%Steerability: Up 0.90% ±1.04%Bash Recovery: Down 0.30% ±0.66%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#23	GPT 6 Luna (Max)OpenAI	+1.68%	12,076	$0.06	33.3K	Board: Arena Agent · Overall95% CI: ±1.41%Confirmed Success: Up 0.38% ±3.10%Praise vs Complaint: Down 1.04% ±4.97%Steerability: Up 4.35% ±1.85%Bash Recovery: Up 4.39% ±1.94%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#24	DeepSeek V4 Pro (High) (0813)DeepSeek	+1.29%	6,743	$0.38	45.4K	Board: Arena Agent · Overall95% CI: ±1.70%Confirmed Success: Down 1.23% ±4.08%Praise vs Complaint: Up 2.51% ±6.11%Steerability: Up 0.33% ±2.34%Bash Recovery: Up 4.51% ±1.25%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#25	Grok 4.6 (xHigh)SpaceXAI	+1.16%	25,563	$1.00	27.2K	Board: Arena Agent · Overall95% CI: ±1.02%Confirmed Success: Down 0.73% ±2.42%Praise vs Complaint: Down 0.49% ±3.41%Steerability: Up 1.00% ±1.38%Bash Recovery: Up 5.68% ±1.01%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#26	GPT 5.5OpenAI	+1.13%	82,103	$0.61	10K	Board: Arena Agent · Overall95% CI: ±0.97%Confirmed Success: Down 4.59% ±2.13%Praise vs Complaint: Up 4.14% ±3.16%Steerability: Up 1.20% ±1.23%Bash Recovery: Up 4.57% ±1.89%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#27	Grok 4.5SpaceXAI	+1.11%	41,479	$0.43	13.2K	Board: Arena Agent · Overall95% CI: ±0.99%Confirmed Success: Up 1.47% ±2.35%Praise vs Complaint: Down 3.97% ±3.29%Steerability: Up 2.06% ±1.45%Bash Recovery: Up 5.65% ±1.09%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#28	GPT 5.4 (High)OpenAI	+0.91%	81,278	$0.59	16.7K	Board: Arena Agent · Overall95% CI: ±0.92%Confirmed Success: Down 2.65% ±2.11%Praise vs Complaint: Down 0.66% ±3.23%Steerability: Up 5.39% ±1.16%Bash Recovery: Up 2.17% ±1.29%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#29	GPT 5.6 Terra (xHigh)OpenAI	+0.83%	22,046	$0.42	13.6K	Board: Arena Agent · Overall95% CI: ±1.25%Confirmed Success: Down 4.23% ±2.70%Praise vs Complaint: Up 4.50% ±4.21%Steerability: Up 2.49% ±1.67%Bash Recovery: Up 1.06% ±1.92%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#30	GLM 5.3 FlashZ.ai	-0.28%	76,297	$0.10	34.8K	Board: Arena Agent · Overall95% CI: ±0.48%Confirmed Success: Up 4.64% ±1.13%Praise vs Complaint: Down 0.80% ±1.60%Steerability: Down 0.22% ±0.68%Bash Recovery: Down 5.35% ±0.54%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#31	MiMo V2.6 FlashXiaomi	-0.57%	13,035	$0.04	39.5K	Board: Arena Agent · Overall95% CI: ±1.44%Confirmed Success: Up 5.44% ±2.79%Praise vs Complaint: Up 1.12% ±4.72%Steerability: Up 0.29% ±1.80%Bash Recovery: Down 5.77% ±3.43%Tool Hallucination: Up 3.92% ±0.48%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#32	Qwen3.8 Flash NextQwen	-0.79%	78,117	$0.14	46.1K	Board: Arena Agent · Overall95% CI: ±0.64%Confirmed Success: Up 3.56% ±1.51%Praise vs Complaint: Down 3.35% ±2.27%Steerability: Down 0.84% ±0.91%Bash Recovery: Down 2.48% ±0.50%Tool Hallucination: Up 0.83% ±0.10%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#33	GPT 5.6 Luna (xHigh)OpenAI	-1.09%	39,830	$0.07	15.4K	Board: Arena Agent · Overall95% CI: ±0.70%Confirmed Success: Down 2.88% ±1.65%Praise vs Complaint: Down 4.17% ±2.29%Steerability: Up 0.10% ±0.98%Bash Recovery: Up 1.17% ±1.11%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#34	Qwen 3.8 27BQwen	-1.61%	53,985	$0.31	35.9K	Board: Arena Agent · Overall95% CI: ±0.60%Confirmed Success: Up 1.51% ±1.45%Praise vs Complaint: Down 3.16% ±2.05%Steerability: Down 0.36% ±0.85%Bash Recovery: Down 5.75% ±0.60%Tool Hallucination: Up 0.30% ±0.12%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#35	Gemini 3.7 Flash (High)Google	-1.64%	51,614	$0.38	29.6K	Board: Arena Agent · Overall95% CI: ±0.76%Confirmed Success: Up 1.08% ±1.82%Praise vs Complaint: Down 4.01% ±2.49%Steerability: Down 1.03% ±1.05%Bash Recovery: Down 4.24% ±0.83%Tool Hallucination: Down 0.00% ±0.17%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#36	Muse Spark 1.2 (xHigh)Meta	-2.96%	46,284	$0.40	23K	Board: Arena Agent · Overall95% CI: ±0.69%Confirmed Success: Down 4.78% ±1.82%Praise vs Complaint: Down 11.64% ±2.09%Steerability: Down 4.86% ±1.10%Bash Recovery: Up 6.17% ±0.84%Tool Hallucination: Down 0.32% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#37	Muse Spark 1.1Meta	-4.60%	139,186	$0.48	16.3K	Board: Arena Agent · Overall95% CI: ±0.45%Confirmed Success: Down 3.91% ±1.12%Praise vs Complaint: Down 11.43% ±1.28%Steerability: Down 6.23% ±0.71%Bash Recovery: Down 1.71% ±0.82%Tool Hallucination: Down 0.30% ±0.03%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#38	Qwen3.7 MaxQwen	-5.20%	39,957	$0.20	13.6K	Board: Arena Agent · Overall95% CI: ±0.94%Confirmed Success: Down 9.07% ±2.44%Praise vs Complaint: Down 9.10% ±2.77%Steerability: Down 6.45% ±1.38%Bash Recovery: Down 1.26% ±1.42%Tool Hallucination: Up 0.14% ±0.17%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#39	Hy3Tencent	-5.70%	29,011	$0.04	23K	Board: Arena Agent · Overall95% CI: ±1.23%Confirmed Success: Down 11.47% ±2.85%Praise vs Complaint: Down 7.01% ±3.75%Steerability: Up 0.36% ±1.45%Bash Recovery: Down 7.34% ±2.87%Tool Hallucination: Up 3.06% ±1.08%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#40	Minimax M3MiniMax	-6.71%	40,771	$0.08	10.4K	Board: Arena Agent · Overall95% CI: ±0.89%Confirmed Success: Down 13.06% ±2.49%Praise vs Complaint: Down 14.31% ±2.56%Steerability: Down 6.93% ±1.46%Bash Recovery: Up 0.87% ±1.06%Tool Hallucination: Up 0.12% ±0.18%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#41	Mimo V2.5 ProXiaomi	-7.23%	44,405	$0.04	10.1K	Board: Arena Agent · Overall95% CI: ±0.91%Confirmed Success: Down 9.40% ±2.34%Praise vs Complaint: Down 11.77% ±2.66%Steerability: Down 7.43% ±1.39%Bash Recovery: Down 6.54% ±1.61%Tool Hallucination: Up 1.00% ±0.27%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#42	Qwen3.7 PlusQwen	-7.28%	21,406	$0.09	14.2K	Board: Arena Agent · Overall95% CI: ±1.47%Confirmed Success: Down 10.14% ±3.37%Praise vs Complaint: Down 11.03% ±4.07%Steerability: Down 8.36% ±1.95%Bash Recovery: Down 5.80% ±3.31%Tool Hallucination: Up 1.07% ±0.39%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#43	Gemini 3.1 Pro PreviewGoogle	-7.35%	86,482	$0.28	10.3K	Board: Arena Agent · Overall95% CI: ±0.98%Confirmed Success: Down 9.37% ±2.15%Praise vs Complaint: Down 0.90% ±2.99%Steerability: Down 4.53% ±1.38%Bash Recovery: Down 21.67% ±1.77%Tool Hallucination: Up 0.29% ±0.74%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#44	Gemini 3.6 Flash (High)Google	-7.67%	22,568	$0.29	25.6K	Board: Arena Agent · Overall95% CI: ±1.08%Confirmed Success: Down 5.56% ±2.62%Praise vs Complaint: Down 8.59% ±3.16%Steerability: Down 7.49% ±1.74%Bash Recovery: Down 17.03% ±1.98%Tool Hallucination: Down 0.29% ±0.04%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#45	Inkling SmallThinking Machines	-9.80%	13,028	$0.10	15.2K	Board: Arena Agent · Overall95% CI: ±1.42%Confirmed Success: Down 19.51% ±4.33%Praise vs Complaint: Down 21.63% ±3.68%Steerability: Down 13.12% ±2.78%Bash Recovery: Up 5.64% ±1.37%Tool Hallucination: Up 0.37% ±0.29%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#46	InklingThinking Machines	-10.90%	42,596	$0.24	11.4K	Board: Arena Agent · Overall95% CI: ±1.06%Confirmed Success: Down 20.74% ±3.01%Praise vs Complaint: Down 22.55% ±2.58%Steerability: Down 11.90% ±2.00%Bash Recovery: Up 0.69% ±1.84%Tool Hallucination: Up 0.02% ±0.12%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#47	Mistral Medium 3.5Mistral	-11.74%	10,797	$0.67	13.3K	Board: Arena Agent · Overall95% CI: ±1.50%Confirmed Success: Down 13.52% ±3.80%Praise vs Complaint: Down 16.67% ±3.93%Steerability: Down 5.72% ±2.23%Bash Recovery: Down 20.42% ±3.34%Tool Hallucination: Up 2.35% ±0.80%	— Source captured 2026-10-01 · Publisher date 2026-09-30
#48	Solar Pro 4Upstage	-15.77%	9,242	$0.05	10.8K	Board: Arena Agent · Overall95% CI: ±1.91%Confirmed Success: Down 20.84% ±5.48%Praise vs Complaint: Down 22.11% ±4.67%Steerability: Down 9.10% ±3.11%Bash Recovery: Down 25.09% ±3.90%Tool Hallucination: Up 1.71% ±0.66%	— Source captured 2026-10-01 · Publisher date 2026-09-30

Full Arena Agent net-improvement table captured 2026-10-01 from arena.ai/leaderboard/agent (48 models, 2,022,521 sessions; board dated 2026-09-30). Score signs come from the accessible net-improvement labels; rows also carry the displayed confidence interval, session count, P50 task cost and P50 output tokens. Confidence intervals overlap across much of the board, so treat close ranks as directional.

Open Arena Agent leaderboard

Evidence in this catalog

Where Agent net improvement fits

Closest verified examples we currently carry. A missing score is not a zero.

Model examples

Claude Fable 5.1 (Max)+14.55%
Published source observation captured 2026-10-01. Source-specific result; no independent rerun or universal capability verdict.
Claude Opus 5.5 (High)+13.78%
Published source observation captured 2026-10-01. Source-specific result; no independent rerun or universal capability verdict.
GPT 6 Astra (Max)+12.18%
Published source observation captured 2026-10-01. Source-specific result; no independent rerun or universal capability verdict.

Harness examples

No directly comparable harness score is verified here yet.

Open this board on Arena Agent leaderboard