Key findings
- Self-reports run ahead of measurement. In METR's early-2025 trial, experienced developers took 19% longer with AI while believing they were 20% faster. METR's late-2025 data points to speedups of 4% to 18%, which METR calls very weak evidence.
- Developers rate AI useful or very useful for implementation (74.6%) and debugging (72.6%) far more often than for design (49.2%) or communication (35.0%), across 12,734 Stack Overflow 2026 respondents.
- Passing tests does not make a change mergeable. Maintainers in METR's March 2026 review would not merge roughly half of test-passing agent pull requests, and the automated grader scored 24 percentage points above them.
- Volume moves the bottleneck to review. Faros AI measured 98% more merged pull requests and 91% longer review time on high-adoption teams. Spotify's merged changes more than doubled, and volume outpaced some of its verification controls.
Delegate first the work a reviewer can check cheaply, because review and rework now decide whether an agent saves your team time. Controlled studies measure smaller gains than developers report. Organizations that doubled their pull request volume found that review and verification became the constraint.
What the controlled studies show
Experienced developers were slower with AI in the best-known randomized trial. METR had open-source developers work issues in their own repositories, with each issue randomly allowed or barred from AI, and tasks took longer with AI. Developers had expected a 24% speedup and afterward believed they had been 20% faster.
METR's follow-up, begun in August 2025 with 57 developers, 143 repositories, and more than 800 tasks, points to a speedup. METR calls that data very weak evidence because 30% to 50% of developers withheld tasks they did not want to do without AI. METR says the withheld tasks likely make the measured speedup smaller than the real one.
| Study | Sample | Change in time with AI (95% interval) |
|---|---|---|
| Early 2025 (Feb to Jun 2025) | 16 developers, 246 issues | +19% (+2% to +39%) |
| Late 2025, returning developers | Developers from the first study | -18% (-38% to +9%) |
| Late 2025, new developers | Newly recruited developers | -4% (-15% to +9%) |
METR, July 10, 2025 and February 24, 2026. A negative change means tasks finished faster with AI.
Which tasks pay back
Developers draw a sharp line between technical work with checkable answers and work that needs judgment about people. Stack Overflow's 2026 respondents rate implementation and debugging useful twice as often as communication. Anthropic's survey of 132 of its own engineers found debugging the most common daily use, at 55%. More than half said they could fully delegate only 0% to 20% of their work.
Large mechanical changes show the clearest organization-level wins. Spotify has applied Claude Code to about 50 migrations through Honk, its background coding agent, and its most recent Java migration across backend services took three days. Its engineers write prompts that describe a verifiable end state, one change per prompt. Expo's web team found the agent strongest on tasks the engineer already knows how to do. The same team found it still produced poorly architected solutions with surprising frequency.
| Task | Useful or very useful |
|---|---|
| Implementation | |
| Debugging and refactoring | |
| Documentation | |
| Code review | |
| Architecture exploration | |
| Design | |
| Communication |
Stack Overflow Developer Survey 2026, n = 12,734. GILD summed the published useful and very useful shares for each task.
Where the gain leaks: review and rework
In Faros AI's team telemetry, the teams that use AI most merged more pull requests and spent far longer in review. Faros found no significant correlation between AI adoption and improvement at the company level.
Rework follows the same pattern. When METR had engineers review 15 agent pull requests from 2025, none was mergeable as written. METR estimated 42 minutes on average to fix each one, and 26 minutes for those that already passed the tests. In Stack Overflow's 2025 survey, 66% named answers that are almost right as their top frustration.
Spotify shows the same pattern across a whole engineering organization. In August 2026 it merged about 17,000 changes, up from roughly 8,100 a year earlier. Its incident reviews did not find AI-authored code to be a material direct cause. Change volume grew faster than some of its verification controls could adapt.
| Measure | Increase |
|---|---|
| Average pull request size | |
| Pull requests merged | |
| Pull request review time | |
| Tasks completed | |
| Bugs per developer |
Faros AI, The AI Productivity Paradox, published July 23, 2025: 10,000+ developers, 1,255 teams, up to two years of history. Correlational, team level.
Choosing the first work to delegate
Rank candidate work by how cheaply a reviewer can confirm the result, and start at the top of that list.
- Changes with an existing check: a test suite, type checker, build, or linter that fails when the change is wrong.
- Mechanical changes repeated across many files or repositories, such as migrations and dependency upgrades, where one reviewed example sets the pattern.
- Investigation in unfamiliar code, where the engineer reads the agent's explanation and keeps the decision.
- Hold back design decisions, ambiguous tickets, and changes whose correctness only a person can judge until your review practice is settled.
Measuring it in your own organization
Measure accepted changes per engineer, review time per pull request, and the revert and defect rate on agent-assisted changes, before and after rollout, on the same teams. Do not rely on how fast developers feel, since METR's participants got even the direction of the effect wrong.
Report review load next to output. Spotify describes its 76% rise in pull request frequency as that many more pull requests to review, and its reviewers decide whether the gain holds.
Vendor dashboards each count the agent's contribution differently. Claude Code's Team and Enterprise analytics counts merged pull requests with at least one line written with Claude Code, and Anthropic calls these contribution metrics deliberately conservative. GitHub's Copilot impact dashboard groups users by adoption level and compares their pull request throughput, and GitHub asks you to treat its return-on-investment figures as directional. Cursor attributes committed lines to its Agent or Tab by matching diff signatures. Reviewer time and rework come from your own pull request data.
Methodology
Figures come from METR's randomized studies and maintainer reviews, the Stack Overflow 2025 and 2026 surveys, and Anthropic's survey of its own engineers. Others come from Faros AI's team telemetry and engineering accounts from Spotify and Expo. Dashboard details come from the analytics documentation of Claude Code, GitHub Copilot, and Cursor. Every figure is its publisher's own, linked with the date GILD opened it. The delegation ranking is GILD's recommendation.
Products in this piece
- Claude Code+75 Strong positiveAI Coding Agent by Anthropic
- Codex+55 Strong positiveAI Coding Agent by OpenAI
- Cursor+11 MixedAI Coding Agent by Anysphere
- GitHub Copilot+15 MixedAI Coding Agent by GitHub
- Gemini CLI−3 MixedAI Coding Agent by Google
- Devin+39 PositiveAI Coding Agent by Cognition
- Devin Desktop (formerly Windsurf)+17 MixedAI Coding Agent, also by Cognition
- Cline+39 PositiveAI Coding Agent by Cline
Sources
- Public source
- 11
- Official documentation
- 4
- METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (Jul 10, 2025)
- METR: We are Changing our Developer Productivity Experiment Design (Feb 24, 2026)
- METR: Research Update, Algorithmic vs. Holistic Evaluation (Aug 2025)
- METR: Many SWE-bench-Passing PRs Would Not Be Merged into Main (Mar 10, 2026)
- Stack Overflow Developer Survey 2026: AI, full data (published Oct 6, 2026)
- Stack Overflow Developer Survey 2025: AI (49,000+ responses, published Jul 29, 2025)
- Anthropic: How AI is transforming work at Anthropic (Dec 2, 2025)
- Faros AI: The AI Productivity Paradox Research Report (Jul 23, 2025)
- Spotify Engineering: Coding Is No Longer the Constraint (Jun 3, 2026)
- Spotify Engineering: AI Changed How Spotify Builds (Sep 16, 2026)
- Spotify Engineering: Context Engineering for Background Coding Agents, Part 2 (Nov 24, 2025)
- Expo: What our web team learned using Claude Code for a month (Feb 6, 2026)
- Claude Code docs: Track team usage with analytics
- GitHub Copilot docs: Viewing the Copilot impact dashboard
- Cursor docs: Team analytics