Prepare each repository once with a setup script, an AGENTS.md file, and skills, then write every task with a finish line the agent can check. Make your own tests the gate for both completion and merge, so OpenHands never ends a task on its own report that the work is done.
When you need this
- Engineers have started handing OpenHands tasks, and the results come back uneven.
- Conversations stall on installing dependencies or end without a passing test run.
- You want one team standard for which tasks go to OpenHands and how its pull requests get reviewed.
Evaluation criteria
Before work: the environment
Does every conversation start with the tools and dependencies it needs?
Look forAdd .openhands/setup.sh for dependency installs and environment variables. OpenHands runs it when work starts. The default sandbox ships with Python and Node.js, so put any heavier toolchain in a custom sandbox image.
Before work: repository instructions
Where do build commands, test commands, and conventions live?
Look forPut AGENTS.md at the repository root. OpenHands loads it into every conversation as a general skill. Ask OpenHands to draft it from the README and CI workflows, then review the file. Put repeatable procedures in .agents/skills, where keywords can trigger them.
Before work: the finish line
Can the task name the files, the expected behavior, and the check that proves it?
Look forOpenHands' guidance rates three kinds of task as the best fit: a bug with reproduction steps, a test addition, and a change that follows an existing pattern. Start there. Split a large refactor or architecture change into phases. The same guidance makes good test coverage the condition for verifying any change.
During work: the model
Is the conversation running on a model strong enough for the task?
Look forUse the strongest model you can afford for planning, long tasks, debugging, and review. OpenHands advises a lower-cost LLM profile for routine edits, and profiles let an engineer switch models in the middle of a conversation.
During work: a ceiling
What stops a conversation that is going in circles?
Look forSet a maximum budget per conversation in Settings > Application. OpenHands stops at the limit and waits until an engineer tells it to continue.
Before merge: the gate
What has to pass before the agent can call the work done, and before anyone merges it?
Look forConfigure a Stop hook in .openhands/hooks.json that runs tests or linters and blocks completion until they pass. Protect the main branch and name a human reviewer for every pull request. On the OpenHands provider, an experimental critic scores each result's chance of success. In the CLI, a score below 60 percent sends the agent back to refine its work, up to 3 times by default.
Questions to ask vendors
- Which repositories have a setup script and an AGENTS.md file, and who keeps them current?
- Which task types merged last month without rework, and on which model?
- How many conversations hit their budget before finishing, and what did those tasks have in common?
- Which repeated corrections should become a skill or a Stop hook?
Red flags
- Tasks that bundle several unrelated changes into one conversation.
- No budget per conversation, so a looping task keeps spending.
- Pull requests merged on the agent's own report that tests pass, with no CI run.
- Credentials pasted into a prompt when they should be stored as secrets.
Products in this piece
Sources
- Official documentation
- 12
- Public source
- 2
- OpenHands docs: Repository customization
- OpenHands docs: General skills and AGENTS.md
- OpenHands docs: Skills
- OpenHands docs: Hooks
- OpenHands docs: When to use OpenHands
- OpenHands docs: Good vs. bad instructions
- OpenHands docs: FAQs
- OpenHands docs: LLM overview and model recommendations
- OpenHands docs: LLM settings
- OpenHands docs: Application settings
- OpenHands docs: Critic
- OpenHands docs: Secrets management
- Ancapistani on Hacker News: a microagent for every architectural decision they disagree with (Aug 7, 2025)
- Lior Strugach: OpenHands, your own AI employee or yet another GPT wrapper? (Jan 5, 2025)