artificialanalysis.ai · benchmark source
Artificial Analysis
Independent evaluations across composite intelligence, hard reasoning, science reasoning, and agentic terminal tasks. Multiple boards live under one publisher.
Tool use
Terminal-Bench 4.0
The current Terminal-Bench version (Aug 2026): 66 long, multi-step shell tasks with an 8-hour agent timeout, run through the Harbor harness.
Why it matters
It tests the loop around a model — planning, commands, file changes, and validation — on tasks that are not yet saturated.
Use it when
You are choosing a model for terminal or coding agents and want an independent, current terminal-task signal.
The limitation
These rows are Artificial Analysis's own runs (the Terminal-Bench component of Intelligence Index v4.3.2). The maintainers' tbench.ai board lists vendor agent harnesses and reports different numbers; do not mix the two, and never compare with 2.x or 3.0 scores.
Leaderboard snapshot
Terminal-Bench 4.0 results in a readable view
Use the tabs for the primary score and other published fields from this board. The source link remains authoritative for the live table.
Leader among verified rows
Claude Sonnet 5.5 (Adaptive Reasoning, Max Effort, Default Fallback)
63.6%
Rows shown
3
Success rate
Source / snapshot date
2026-10-01
25 days old · not a live API feed
Score profile
Success rate by published row
Higher is better in this view
Claude Sonnet 5.5 (Adaptive Reasoning, Max Effort, Default Fallback)Claude Opus 5.5 (Adaptive Reasoning, Max Effort, Default Fallback)Claude Opus 5.5 (Adaptive Reasoning, Xhigh Effort, Default Fallback)| Rank | Model or system | Success rate | Other details | Note |
|---|---|---|---|---|
| #1 | Claude Sonnet 5.5 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropicaa-terminal-bench-4 | 63.6% | Harness: mini-swe-agent; pass@1 averaged over three repeatsCaptured: 2026-10-01 | Leader confirmed October 1; other rows retain the September 26 capture. Source captured 2026-10-01 · Publisher/run date not disclosed |
| #2 | Claude Opus 5.5 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropicaa-terminal-bench-4 | 59.6% | View: Artificial AnalysisHarness: Artificial Analysis agent harness | — Source captured 2026-10-01 · Publisher/run date not disclosed |
| #2 | Claude Opus 5.5 (Adaptive Reasoning, Xhigh Effort, Default Fallback)Anthropicaa-terminal-bench-4 | 59.6% | View: Artificial AnalysisHarness: Artificial Analysis agent harness | — Source captured 2026-10-01 · Publisher/run date not disclosed |
Version
Terminal-Bench 4.0 (Aug 2026, 66 tasks, 8-hour agent timeout)
Runner
Artificial Analysis (independent; component of Intelligence Index v4.3.2)
Comparability
Not comparable with 2.x or 3.0 — new task set
Only three exact public Artificial Analysis headline configurations are retained. These are AA runs, not the separate tbench.ai vendor-harness board. The full AA component data require publisher access. 3 exact matching primary-source observations are ranked in this retained subset. 185 unsupported observations are excluded from ranks, charts and leader claims. Subset ranks are not positions on the complete publisher leaderboard. Exact retained receipts captured 2026-10-01; source/run dates are shown per verified row when known.
Not yet verified (185)
No exact matching primary-source observation was retained for these entries. Scores are withheld here; these entries do not affect ranks, leaders, charts or search indexing.
- GPT-6 Astra (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Astra (max)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 5.5 (Adaptive Reasoning, High Effort, Default Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Fable 5.1 (Adaptive Reasoning, Xhigh Effort, Default Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Astra (high)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 5.5 (Adaptive Reasoning, Medium Effort, Default Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Fable 5.1 (Adaptive Reasoning, High Effort, Default Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Astra (medium)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 5 (Adaptive Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 5 (Adaptive Reasoning, Xhigh Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 5 (Adaptive Reasoning, High Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Fable 5.1 (Adaptive Reasoning, Medium Effort, Default Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Sol (max)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- GLM-5.3 (max)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Astra (low)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Fable 5.1 (Adaptive Reasoning, Low Effort, Default Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Sol (max)
No exact matching primary-source row was retained; excluded from rankings.
- Qwen3.8 Max (0902)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Terra (max)
No exact matching primary-source row was retained; excluded from rankings.
- GLM-5.3 (low)
No exact matching primary-source row was retained; excluded from rankings.
- MiMo-V2.6-Pro
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 5 (Adaptive Reasoning, Medium Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Muse Spark 1.3 (max)
No exact matching primary-source row was retained; excluded from rankings.
- Step 5 Preview
No exact matching primary-source row was retained; excluded from rankings.
- GLM 5.3 Flash
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 5.5 (Adaptive Reasoning, Low Effort, Default Fallback)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Sol (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- DeepSeek V4.1 Flash (Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 5 (Adaptive Reasoning, Low Effort)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Sol (high)
No exact matching primary-source row was retained; excluded from rankings.
- Grok 4.7 (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- Qwen3.8-Flash-Next
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Sol (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- Grok 4.7 (high)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Opus 4.8 (Adaptive Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Grok 4.6 (high)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Sol (high)
No exact matching primary-source row was retained; excluded from rankings.
- Gemini 3.8 Flash (high)
No exact matching primary-source row was retained; excluded from rankings.
- Gemini 3.8 Flash (medium)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Sol (medium)
No exact matching primary-source row was retained; excluded from rankings.
- Qwen3.8 Max
No exact matching primary-source row was retained; excluded from rankings.
- Grok 4.6 (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- Muse Spark 1.3 (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- DeepSeek V4 Pro 0424 (Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.5 (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Sol (medium)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Sonnet 5 (Adaptive Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- DeepSeek V4 Pro 0813 (Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Gemini 3.7 Flash (high)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Sol (Non-reasoning)
No exact matching primary-source row was retained; excluded from rankings.
- Grok 4.6 (medium)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.5 Instant (June 2026)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Luna (max)
No exact matching primary-source row was retained; excluded from rankings.
- Kimi K3 (low)
No exact matching primary-source row was retained; excluded from rankings.
- Kimi K3 (max)
No exact matching primary-source row was retained; excluded from rankings.
- DeepSeek V4 Flash 0731 (Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- DeepSeek V4 Flash Vision (Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Luna (max)
No exact matching primary-source row was retained; excluded from rankings.
- Qwen3.8 2.4T A95B
No exact matching primary-source row was retained; excluded from rankings.
- Grok 4.5 (high)
No exact matching primary-source row was retained; excluded from rankings.
- Gemini 3.8 Flash (low)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Terra (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.5 (high)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Sol (low)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Luna (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Sonnet 5 (Adaptive Reasoning, Xhigh Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Gemini 3.6 Flash (high)
No exact matching primary-source row was retained; excluded from rankings.
- Muse Spark 1.2 (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- Gemini 3.5 Flash (high)
No exact matching primary-source row was retained; excluded from rankings.
- Muse Spark 1.1 (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- DeepSeek V4.1 Flash (Non-Reasoning)
No exact matching primary-source row was retained; excluded from rankings.
- Qwen3.8 27B (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Sonnet 5 (Adaptive Reasoning, High Effort)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.5 (medium)
No exact matching primary-source row was retained; excluded from rankings.
- Qwen3.8 27B (medium)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Luna (high)
No exact matching primary-source row was retained; excluded from rankings.
- Gemini 3.1 Pro Preview
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Luna (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- DeepSeek V4 Flash 0420 (Reasoning, High Effort)
No exact matching primary-source row was retained; excluded from rankings.
- Grok 4.6 (low)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Sonnet 5 (Adaptive Reasoning, Low Effort)
No exact matching primary-source row was retained; excluded from rankings.
- DeepSeek V4 Flash 0420 (Reasoning, Max Effort)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Luna (high)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Luna (medium)
No exact matching primary-source row was retained; excluded from rankings.
- Qwen3.8 27B (low)
No exact matching primary-source row was retained; excluded from rankings.
- Claude Sonnet 5 (Adaptive Reasoning, Medium Effort)
No exact matching primary-source row was retained; excluded from rankings.
- GLM-5.1 (Reasoning)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.4 mini (xhigh)
No exact matching primary-source row was retained; excluded from rankings.
- MiniMax-M3
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Terra (high)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Terra (low)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-6 Luna (Non-reasoning)
No exact matching primary-source row was retained; excluded from rankings.
- K2 Horizon 375B A23B
No exact matching primary-source row was retained; excluded from rankings.
- Qwen3.7 Max
No exact matching primary-source row was retained; excluded from rankings.
- Gemini 3.5 Flash-Lite
No exact matching primary-source row was retained; excluded from rankings.
- GLM-5.2 (max)
No exact matching primary-source row was retained; excluded from rankings.
- GPT-5.6 Luna (Non-reasoning)
No exact matching primary-source row was retained; excluded from rankings.
Showing 100 entries. Open the publisher source for the current complete table.
Only three exact public Artificial Analysis headline configurations are retained. These are AA runs, not the separate tbench.ai vendor-harness board. The full AA component data require publisher access. 3 exact matching primary-source observations are ranked in this retained subset. 185 unsupported observations are excluded from ranks, charts and leader claims. Subset ranks are not positions on the complete publisher leaderboard. Exact retained receipts captured 2026-10-01; source/run dates are shown per verified row when known.
Open Artificial Analysis LLM leaderboard (Terminal-Bench 4.0 column)Evidence in this catalog
Where Terminal-Bench 4.0 fits
Closest verified examples we currently carry. A missing score is not a zero.
Model examples
Published source observation captured 2026-10-01. Leader confirmed October 1; other rows retain the September 26 capture.
Published source observation captured 2026-10-01. Source-specific result; no independent rerun or universal capability verdict.
Published source observation captured 2026-10-01. Source-specific result; no independent rerun or universal capability verdict.
Harness examples
- Claude CodeTerminal agent
Harness choice moves Terminal-Bench scores materially — compare terminal agents after you narrow the model field.