Key findings
- The Claude Opus 4.8 run used agent version v1.24.0 and was submitted June 1, 2026. OpenHands' docs recommend that model first.
- The best open-weight result is MiniMax-M3 at 76.4%, across 34 SWE-bench Verified runs of the OpenHands agent.
- Against Codex on GPT-5.5, the OpenHands agent scored 78.2% to Codex's 78.0%, both submitted May 4, 2026. That gap is one issue out of 500.
- The SWE-bench team's own February 2026 run of Claude Opus 4.6, on its minimal agent, resolved 75.6%. OpenHands scored 76.8% on the same model.
- The index's top score, 95.8% with claude-fable-5, is left out of OpenHands' model recommendations because the index marks that model unavailable from its provider.
Quote an OpenHands benchmark score only with its model, agent version, and date, because the same agent resolves anywhere from 34.2% to 83.8% of SWE-bench Verified depending on the model behind it. The best result from a model you can buy is 83.8% with Claude Opus 4.8, at US$0.75 per task. All Hands AI, the company behind OpenHands, runs and publishes every result in its OpenHands Index.
What the OpenHands Index measures
The index scores each model on five benchmarks: SWE-bench, SWE-bench Multimodal, SWT-bench, Commit0, and GAIA. OpenHands' evaluation code defaults to the SWE-bench Verified test split, so the SWE-bench column is a SWE-bench Verified resolve rate.
Each result directory stores the model, the agent version, the submission time, a cost per task in US dollars, and the average runtime. It also links to the full archive of the agent's runs. The agent version matches a Software Agent SDK release, so every score names the code that produced it.
The SWE-bench Verified results to quote
The same agent spans nearly 50 points across models, so each score below carries its model, agent version, and submission date. All are SWE-bench Verified resolve rates from the index repository, with the five-benchmark average after each.
- Claude Opus 4.8: 83.8%, agent v1.24.0, submitted June 1, 2026, US$0.75 per task. Average across five benchmarks: 71.9.
- Gemini 3.5 Flash: 78.6%, agent v1.28.0, submitted June 12, 2026, US$1.53 per task. Average: 62.6.
- GPT-5.5: 78.2%, agent v1.18.1, submitted May 4, 2026, US$1.52 per task. Average: 65.9.
- MiniMax-M3 (open weights): 76.4% on the agent release Claude Opus 4.8 used, submitted June 3, 2026, US$0.17 per task. Average: 57.2.
- GLM-5.1 (open weights): 75.0%, agent v1.17.0, submitted April 21, 2026. Average: 58.2.
OpenHands against other agents on the same evaluation code
The index also runs Claude Code, Codex, and Gemini CLI through the Agent Client Protocol and stores those results separately. Two pairs ran the same model within a day of each other. On GPT-5.5, OpenHands scored 78.2% and Codex 78.0%. Both were submitted May 4, 2026, at US$1.52 and US$1.17 per task. On Claude Opus 4.6, OpenHands scored 76.8% on March 26, 2026, and Claude Code 74.4% on March 27, 2026. Both used agent version v1.15.0.
Both gaps are small. On GPT-5.5 the 0.2-point gap is one of SWE-bench Verified's 500 issues, and on Claude Opus 4.6 the 2.4-point gap is 12 issues. All Hands AI ran all four runs, so run the two agents on your own repositories before you choose between them on these numbers.
Who ran the numbers, and what they leave out
All Hands AI wrote the evaluation code and ran every index result, and GILD found no third-party reproduction of them. None of the 2026 index results appear on the public SWE-bench leaderboard at swebench.com, where the SWE-bench team marks each run it performed or checked itself. That team did check earlier OpenHands submissions, including 71.8% with GPT-5 in August 2025 and 70.4% with Claude 4 Sonnet in May 2025. Its own February 17, 2026 run of Claude Opus 4.6 on mini-SWE-agent, its minimal reference agent, resolved 75.6%, 1.2 points below OpenHands' 76.8% on the same model. ByteDance's Multi-SWE-bench paper of April 2025 ran an adapted OpenHands on 1,632 issues in seven languages. It had the highest resolve rate in five of the seven, at a higher cost than the fixed-workflow Agentless.
The newest index result is dated June 12, 2026. When GILD rechecked the repository on October 25, 2026, its last commit was a June 30 fix to cost figures, so no score covers an OpenHands release after June. Cost per task is spend on benchmark issues, so measure cost per merged pull request on your own codebase.
Review time, rework, and how often pull requests merge unchanged come only from your own repository pilot, so record all three there.
Methodology
GILD read every model directory in the OpenHands Index results repository on October 12, 2026, and rechecked its commit history on October 25, 2026. That covers 35 directories for OpenHands, 34 with a SWE-bench result, plus the Agent Client Protocol runs of Claude Code and Codex. Scores, agent versions, submission times, and costs come from each directory's scores and metadata files. The dataset default comes from OpenHands' SWE-bench evaluation code, the recommendations from its model guidance, and the meaning of available: false from the index's own commit and recommendation script. Index figures are All Hands AI's own runs. The outside figures come from the SWE-bench team's leaderboard data, read October 25, 2026. Every figure is linked in Sources.
Products in this piece
Sources
- Official documentation
- 9
- Public source
- 4
- OpenHands Index
- OpenHands Index results repository
- OpenHands Index: claude-opus-4-8 results
- OpenHands Index: GPT-5.5 results
- OpenHands Index: Codex through ACP on GPT-5.5
- OpenHands benchmarks: SWE-bench harness
- ByteDance Seed: Multi-SWE-bench, a multilingual benchmark for issue resolving (Apr 3, 2025)
- OpenHands Software Agent SDK on GitHub
- OpenHands docs: LLM overview and model recommendations
- OpenHands docs: ACP agents in Agent Canvas
- OpenHands Index results on GitHub: Add available attribute to model metadata, commit 60e48ad (Jun 16, 2026)
- SWE-bench: SWE-bench Verified leaderboard (checked Oct 25, 2026)
- Simon Willison: SWE-bench February 2026 leaderboard update (Feb 19, 2026)