Skip to main content

Research

Qodo and PR-Agent in Practice: Adoption Rates, Noise, and What Teams Kept

Qodo catches real bugs, and 8 of the 15 accounts that reported whether they kept it stayed with it. GILD read 53 public accounts to find what failed and how the teams that stayed made it work.

Reading time
4 min read
Sources
23 cited
In this piece
Qodo+17 Mixed

Key findings

  1. All 18 accounts that discuss bug catches report real ones, including a client-name leak and nine of ten seeded bugs in Convex's test of nine reviewers.
  2. Of 14 accounts that say whether they kept the reviewer, 8 kept it and 6 dropped it. Among the 6 drops, noise or quality drove 2, stopped reviews drove 2, and post-trial pricing drove 2.
  3. Comment noise is the concern raised most often, by 14 of the 18 accounts that discuss it. All 9 accounts that discuss setup or missing reviews call them a problem.
  4. Of 9 hands-on comparisons, 7 favor another reviewer, and Punt Labs measured Qodo against Copilot and made Qodo its first reviewer. Viktor Farcic and Convex still found that Qodo caught the most issues.
  5. The overall read is mixed. Team accounts and individual developers both lean positive.

Qodo catches real bugs. Of 14 accounts that reported a keep-or-drop decision, 8 kept it and 6 dropped it. The teams that kept it narrowed its rules and triggers and checked that reviews kept running, so plan both before Qodo becomes a required step on your pull requests.

Where the accounts come from

GitHub issues and pull requests make up 20 of the 53 accounts. GILD held bug reports and project threads to 16, about a third of the first 49, because issue trackers record failures. The other 4 are adoption decisions at Selenium, Punt Labs, OpenSearch and JabRef, each by a maintainer or founder of the project; OpenSearch skips hosted Qodo and runs the open source PR-Agent on Amazon Bedrock. The rest are G2 reviews, Hacker News comments, a few posts and videos, and two research papers on Beko's company-wide reviewer, which Beko built on the open source PR-Agent. G2 reviews lean positive, so the two kinds of source pull the overall read in opposite directions. GILD counted a bug report as mixed unless its author dropped Qodo. Overall, 19 accounts are positive, 20 mixed, and 11 negative, so read the themes below before that total.

What reviewers catch

Bug catches draw the most consistent praise. In Convex's test of nine reviewers on ten seeded bugs, Qodo scored 32 of 40 and caught nine primary issues. It also fell for a fake N+1 trap. Viktor Farcic found more real issues with Qodo than Copilot on the same pull request, though Copilot presented them better. Qodo caught a client-name leak for mbjorke while CodeRabbit was rate-limited. Ambrish Vadnerkar's PR-Agent setup flagged a hardcoded API key in its second week.

Generated descriptions draw praise too. Vadnerkar reports that merge request descriptions alone justified the setup. A Hacker News commenter's team values effort labels and the file-by-file walkthrough more than occasional serious catches.

Where the noise comes from

G2 reviewers report trivial issues at high severity and suggestions that miss the purpose of a change. An Indie Hackers member ranked Qodo eighth of 11 bots for hallucinations and settings that failed to reduce noise. PR-Agent users saw repeated suggestions on each push until maintainers added persistent inline comments.

Wrong rule findings cost more than nitpicks, because someone has to argue them down on every pull request. DAQiFi's firmware team and Red Hat QE each filed an issue because a Qodo rule contradicted the repository's own conventions or its ignore list.

Two products, two kinds of failure

Hosted Qodo fails at the trial's end and when the service breaks. ptr727 saw only trial-ended notices from Qodo's paid identity. don-petry counted no Qodo reviews across eight pull requests. On the PR-Agent repository, a contributor counted 195 failed hosted reviews out of 195 over about 11 days, until a maintainer fixed the cause.

Self-hosted PR-Agent fails in operation. A G2 reviewer cites setup and maintenance work. LightSpeed stayed on an older Claude model because the release pins an older LiteLLM model map. Azure DevOps users report malformed suggestions.

What the teams that kept it changed

Teams that kept the reviewer cut its scope. Fooftilly stopped comments that carried no material finding. ekson73 disabled skills and cross-repository agents after findings cited private file paths. Vadnerkar capped reviews at six findings and wrote specific extra instructions. When Victor Lyuboslavsky replaced hosted Qodo with PR-Agent, review moved from every push to pull request open.

Qodo's current settings cover each of those changes. Review Standards hold the rules, the inline severity threshold limits comments, .pr_agent.toml sets the triggers, and Relevance rates each finding against how the team handled similar ones before.

How to measure it

Pick your measures and the thresholds that keep Qodo on every pull request before the trial starts. The accounts show three in use. ptr727, don-petry, and the contributor who counted 195 failures tracked whether reviews ran. Convex scored seeded bugs. Lyuboslavsky calculated cost per review at his repository's volume. Qodo's analytics dashboard, in beta, adds a fourth by separating acted-on findings from accepted ones.

  • Reviews posted over pull requests opened, so a reviewer that has gone quiet shows up.
  • A seeded-bug test on your own stack, scored per pull request the way Convex scored its ten.
  • Accepted findings over posted findings, by severity, from Qodo's analytics.
  • Credits per review, from the billing page.

Methodology

GILD read 53 public accounts of hosted Qodo and the open source PR-Agent by engineers and teams, dated January 2025 to October 2026 and opened between October 12 and 26, 2026. They include GitHub issues and pull requests, G2 reviews on Qodo's AWS Marketplace listing, Hacker News comments, a company engineering post, and a personal post. Others are a video, an Indie Hackers post, security research, and two research papers on Beko's reviewer built on PR-Agent. GitHub issues and pull requests make up 20 of the 53, and 4 of those are adoption decisions by maintainers whose roles are stated on project pages. Every count in this piece is a count of these accounts, and the three that describe an out-of-date Qodo version stay listed and out of the counts. PR-Agent accounts cover the open source reviewer CodiumAI and Qodo built, which its community now maintains. GILD checked product behavior against Qodo's documentation on October 12, 2026.

Products in this piece

Sources

Public source
18
Official documentation
5
  1. Mike Cann, Convex Stack: Comparing 9 Code Review Tools, Which is Best? (Mar 10, 2026)Public sourcestack.convex.dev
  2. Viktor Farcic, DevOps & AI Toolkit: Better Code Reviews with AI? GitHub Copilot and Qodo Merge Tested (Jun 30, 2025)Public sourceyoutube.com
  3. Ambrish Vadnerkar: How I Added AI Code Reviews to Our GitLab Pipeline Using PR-Agent (Apr 6, 2026)Public sourcemedium.com
  4. Indie Hackers: We Ran 11 AI PR Bots, and kept 1 (Jan 29, 2026)Public sourceindiehackers.com
  5. Qodo reviews on AWS Marketplace, syndicated from G2 (newest first, read Oct 12, 2026)Public sourceaws.amazon.com
  6. timelog-extract issue #326: multi-reviewer loop with CodeRabbit and Qodo (Jul 22, 2026)Public sourcegithub.com
  7. DotNetWorkQueue issue #362: Evaluate a replacement for qodo PR review, trial ended (Sep 17, 2026)Public sourcegithub.com
  8. PRKS pull request #89: suppress empty Qodo review comments (Sep 20, 2026)Public sourcegithub.com
  9. MMPoolsV3 pull request #326: qodo is back, check it on every PR (Jul 30, 2026)Public sourcegithub.com
  10. multi-agent-os pull request #427: contain Qodo skill-insight and cross-repo leak (Sep 14, 2026)Public sourcegithub.com
  11. ptr727 ProjectTemplate issue #1465: Qodo no longer reviews this repository (Sep 9, 2026)Public sourcegithub.com
  12. DAQiFi firmware issue #1128: Qodo rule contradicts CLAUDE.md (Sep 18, 2026)Public sourcegithub.com
  13. RedHatQE mtv-api-tests issue #720: Qodo rule conflicts with the .pr_agent.toml ignore glob (Oct 8, 2026)Public sourcegithub.com
  14. PR-Agent issue #3256: Qodo review bot has failed on every recent PR since Aug 29 (Sep 10, 2026)Public sourcegithub.com
  15. PR-Agent issue #2184: Is it possible to remember code suggestions from the previous push? (Jan 27, 2026)Public sourcegithub.com
  16. PR-Agent issue #2110: Incorrect inline code suggestion formatting in Azure DevOps (Nov 20, 2025)Public sourcegithub.com
  17. LightSpeed .github issue #3535: Validate the qodo-pr-agent pilot credential boundary and activation (Sep 24, 2026)Public sourcegithub.com
  18. Victor Lyuboslavsky, fleet-edr issue #1141: Run PR-Agent from CI instead of the metered Qodo review (Sep 21, 2026)Public sourcegithub.com
  19. Qodo docs: Severity thresholdsOfficial documentationdocs.qodo.ai
  20. Qodo docs: RelevanceOfficial documentationdocs.qodo.ai
  21. Qodo docs: Review StandardsOfficial documentationdocs.qodo.ai
  22. Qodo docs: Code review analyticsOfficial documentationdocs.qodo.ai
  23. Qodo docs: Pricing & usageOfficial documentationdocs.qodo.ai

Keep reading

  1. ResearchSelf-Hosted PR-Agent or Hosted Qodo: What Each Review CostsHosted Qodo charges about US$1.67 in credits per review, while PR-Agent on your own infrastructure spends cents on model tokens. Teams that self-host report what else it costs them.
  2. Buyer guideQodo Review Rollout Playbook: Rules, Severity, and the Merge GateA Qodo reviewer is worth keeping when engineers fix what it flags. Start on one repository, write Review Standards, set severity and triggers, and decide what your merge gate waits on.
  3. Buyer guideQodo Credits & Review Spend ChecklistQodo's trial ends with no free tier to fall back on. Work out credits from your pull request count, cap overages, and have billing live before reviews stop.
Read the research