Operator Insights

What are engineering leaders learning from deploying GitHub Copilot?

13 accounts: 1 left it or turned it down, 1 switched, 2 run it beside another tool, 2 evaluated it, 1 adopted it and 6 reported by others

Left it or turned it down1

Michael ReddingPrincipal Product ManagerIndeed
Engineering organization of about 2,000 engineers. 2025 to mid-2026.
DX Engineering Enablement podcast, published Jun 29, 2026Listen to the episode (opens DX Engineering Enablement podcast in a new tab)
Observation
Indeed retired Copilot once its developers had abandoned it, with weekly use down to about 3%.
Takeaway
Seats and training do not create usage. Watch weekly active use and cancel when engineers have left on their own.
What happened
With GitHub Copilot and Sourcegraph Cody, unlimited budgets, training and leadership encouragement, Indeed's weekly AI use stalled near 25% of developers. Developers started paying for Cursor themselves; Indeed moved to agentic tools and retired Copilot and Cody, whose use had dropped to about 3% by then. Its mix by mid-2026 was mostly Claude Code.
“And then we actually have phased out Cody and Copilot. So sort of that first generation of tools we’ve actually already fully retired. Our usage when we actually did retire them was down to about 3% of people. So our users actually abandoned them before we did.”
Michael Redding
- Podcast episode (DX Engineering Enablement, published on YouTube Jun 29, 2026)
- Organization metrics reported
Deployment details
Engineering organization of about 2,000 engineers. 2025 to mid-2026.
- Podcast episode (DX Engineering Enablement, published on YouTube Jun 29, 2026)
- Organization metrics reported
Team
Indeed engineering, about 2,000 engineers
Duration
About 18 months
Workload
Product development
Product version
GitHub Copilot and Sourcegraph Cody, then agentic tools
Measurement
Weekly active use of AI tools

Switched to or from it1

Yaroslav (Yarik)Tech lead for the coding-agent rolloutSuperhuman
Superhuman engineering, formerly Grammarly. 2025 to August 2026.
Merantix AI Campus, published Aug 18, 2026Watch the talk (opens Merantix AI Campus in a new tab)
Observation
Superhuman moved on from Copilot's autocomplete to Cursor and then Claude Code.
Takeaway
Autocomplete was the first step; teams that wanted delegation moved to agents.
What happened
Superhuman's engineers first used Copilot as autocomplete about a year and a half before the August 2026 talk, switched to Cursor, and then moved to Claude Code, which the tech lead scaled across engineering.
“…a year and a half ago, we first experienced Copilot. It was very cool auto complete.”
Yaroslav (Yarik), Superhuman, at 2:52, from automatic captions
- Conference talk
- Self-reported
Deployment details
Superhuman engineering, formerly Grammarly. 2025 to August 2026.
- Conference talk
- Self-reported
Team
Superhuman engineering
Duration
About a year and a half
Workload
Product engineering
Product version
GitHub Copilot, then Cursor, then Claude Code

Run it beside another tool2

Mikhail ParakhinChief technology officerShopify
Every Shopify employee may choose a coding tool. December 2025 to April 2026.
Latent Space, published Apr 22, 2026Listen to the episode (opens Latent Space in a new tab)
Observation
At Shopify, Copilot is not shrinking but grows more slowly than command-line agents.
Takeaway
Size Copilot renewals on current usage; growth is moving to agents.
What happened
Shopify's internal usage chart shows IDE tools such as GitHub Copilot and Cursor not shrinking but growing more slowly, while command-line agents such as Claude Code and Codex run beside them and took off since December 2025.
“…tools, uh, that require IDEs such as, uh, GitHub, Copilot or Cursor, they’re not exactly shrinking, but they’re not growing as fast. Like, uh, red, red line is, is the IDE kind of tools.”
Mikhail Parakhin, Shopify, at 3:52 in the recording
- Podcast episode
- Organization usage reported
Deployment details
Every Shopify employee may choose a coding tool. December 2025 to April 2026.
- Podcast episode
- Organization usage reported
Team
Shopify, company-wide
Duration
About 5 months
Workload
Shopify's codebase
Product version
GitHub Copilot beside Cursor and command-line agents
Measurement
Internal usage by tool

37signalsAug 26, 2026Lex Fridman PodcastAt 37signals, Copilot's pull request review keeps finding real breakage behind the agents.
DHHChief technology officer37signals
DHH's own development on Omarchy and 37signals work, August 2026.
Lex Fridman Podcast, published Aug 26, 2026Listen to the episode (opens Lex Fridman Podcast in a new tab)
Observation
At 37signals, Copilot's pull request review keeps finding real breakage behind the agents.
Takeaway
Copilot review earns a place behind other vendors' agents; keep it on as the last check before merge.
What happened
DHH drives his agents in Claude Code with Codex as checker, and once they push to GitHub, Copilot's review runs beside them and keeps finding things that are legitimately broken, which he says its first version did not do.
“And then Copilot, I kid you not, has actually gotten good. Copilot keeps finding stuff that’s legitimately broken,…”
DHH, at 2:38:47 in the recording
Podcast episode
Self-reported
Deployment details
DHH's own development on Omarchy and 37signals work, August 2026.
Podcast episode
Self-reported
Team
DHH's own development
Workload
Omarchy and 37signals code
Product version
GitHub Copilot code review beside Claude Code and Codex

Evaluated it2

ZoomInfoFeb 13, 2025ZoomInfo Engineering BlogZoomInfo trialed Copilot with a pilot group before its company-wide rollout, and paced the licenses.
Ali DasdanCo-author of ZoomInfo's Copilot studyZoomInfo
Engineering organization. Over 400 developers. Rollout from September 2023.
ZoomInfo Engineering Blog, published Feb 13, 2025Read the write-up (opens ZoomInfo Engineering Blog in a new tab)
Observation
ZoomInfo trialed Copilot with a pilot group before its company-wide rollout, and paced the licenses.
Takeaway
Run a structured trial before buying every seat, and expect domain-specific logic to need close vetting even where acceptance is high.
What happened
ZoomInfo assessed Copilot with five engineers before a two-week trial with 126. It began rollout in September 2023 and issued licenses over several months. The study reports 33% suggestion acceptance, 20% line acceptance, 72% developer satisfaction, and about 20% time saved. HTML, CSS, JSON, and SQL had lower acceptance.
“The results show an average acceptance rate of 33% for suggestions and 20% for lines of code, with high developer satisfaction scores of 72%.”
ZoomInfo Engineering
Public engineering write-up
Organization metrics reported
No model named
Deployment details
Engineering organization. Over 400 developers. Rollout from September 2023.
Public engineering write-up
Organization metrics reported
No model named
Team
ZoomInfo engineering, over 400 developers
Duration
Trial July to August 2023; rollout from September 2023
Workload
TypeScript, Java, Python, and JavaScript among other languages
Product version
GitHub Copilot for Business
Measurement
Acceptance rate by suggestion and line, developer satisfaction survey
UK Department for Science, Innovation and TechnologySep 12, 2025GOV.UKThe UK public sector trialled Copilot across 50+ organisations, with a 15.8% line acceptance rate.
Government Digital ServiceUK central government digital serviceUK Department for Science, Innovation and Technology
2,500 trial licences across more than 50 UK public sector organisations, November 2024 to February 2025.
GOV.UK, published Sep 12, 2025Read the report (opens GOV.UK in a new tab)
Observation
The UK public sector trialled Copilot across 50+ organisations, with a 15.8% line acceptance rate.
Takeaway
Line acceptance and self-reported time saved measure different outcomes; track both before scaling.
What happened
The UK government's AI coding assistant trial distributed 2,500 licences across more than 50 public sector organisations, including 1,600 GitHub Copilot Business licences of which 1,100 were redeemed. Copilot telemetry showed a 15.8% average acceptance rate for suggested lines and about 418 daily active users; participants reported saving 56 minutes a working day.
“However, for GitHub Copilot, telemetry data indicated an average acceptance rate of 15.8% for suggested code lines, which is slightly lower than industry reports. Only 39% of users reported that they committed code suggested by the AICA.”
Government Digital Service
Public organization write-up
Telemetry and survey reported
Deployment details
2,500 trial licences across more than 50 UK public sector organisations, November 2024 to February 2025.
Public organization write-up
Telemetry and survey reported
Team
UK public sector, 50+ organisations
Duration
November 2024 to February 2025
Workload
Government software development
Product version
GitHub Copilot Business
Measurement
Acceptance rate, daily active users, self-reported time saved

Adopted it1

Jesse HouwingSep 18, 2026jessehouwing.netAn open-source maintainer merged 8 of 11 Copilot agent tasks and steered the design himself.
Jesse HouwingGitHub and Scrum trainer, PSScriptAnalyzer contributorXebia
His fork of PowerShell PSScriptAnalyzer, September 2026.
jessehouwing.net, published Sep 18, 2026Read the write-up (opens jessehouwing.net in a new tab)
Observation
An open-source maintainer merged 8 of 11 Copilot agent tasks and steered the design himself.
Takeaway
Agent changes pass review one by one; the engineer still has to own the overall design.
What happened
Houwing runs Copilot's coding agent on his fork: of eleven separate tasks, eight were merged and three closed as superseded. The changes could each pass review, but the agent kept adding layers unless he stopped it and set the direction.
“I ran eleven separate agent tasks on my fork; eight were merged and three were closed as superseded.”
Jesse Houwing
Public practitioner write-up
Self-reported counts
Deployment details
His fork of PowerShell PSScriptAnalyzer, September 2026.
Public practitioner write-up
Self-reported counts
Team
One maintainer
Workload
PowerShell PSScriptAnalyzer bug fixes
Product version
GitHub Copilot coding agent
Measurement
Tasks merged and superseded

Reported by others6

ThoughtworksApr 3, 2025ThoughtworksA four-developer Thoughtworks team went from 20 to 23 stories a sprint, after a slower start.
Birlla PackiasamyAuthor of the team's Copilot experimentThoughtworks
Delivery team of four developers. 10 weeks.
Its own announcement, published Apr 3, 2025Read the write-up (opens the announcement in a new tab)
Observation
A four-developer Thoughtworks team went from 20 to 23 stories a sprint, after a slower start.
Takeaway
Agree the delivery metrics and quality gates before the pilot starts, and budget for a learning dip. The gain varies widely by story.
What happened
Stakeholders required delivery metrics for the experiment. The team added CI requirements for 90% coverage and checks for code smells and security. Work slowed initially and drifted from test-driven development. Training developers to prompt for tests first corrected that drift. Delivery reached 23 stories per sprint, up from 20. Per-story results ranged from 40% gains to a 5% decline.
“Before Copilot, our team completed 20 user stories per sprint. Now, we complete 23 user stories per sprint, three more stories per sprint, a whopping 15% increase in velocity without any extra effort.”
Birlla Packiasamy
Public engineering write-up
Team metrics reported
No model named
Deployment details
Delivery team of four developers. 10 weeks.
Public engineering write-up
Team metrics reported
No model named
Team
Four developers
Duration
10 weeks
Product version
Copilot inline and chat assistance
Measurement
User stories per sprint, story cycle time, code quality gates in CI
MicrosoftJul 1, 2026arXivAt Microsoft, command-line agent use spread through peers, and adopters merged more pull requests.
Emerson Murphy-HillLead author of the rollout studyMicrosoft
Company-wide engineering rollout. January 5 to April 29, 2026. Copilot CLI and Claude Code.
arXiv, published Jul 1, 2026Read the study (opens arXiv in a new tab)
Observation
At Microsoft, command-line agent use spread through peers, and adopters merged more pull requests.
Takeaway
Make agent use visible through peers and managers, and measure outcomes beyond merged pull requests before token spend scales.
What happened
Microsoft studied its early 2026 rollout of Copilot CLI and Claude Code. Engineers were most likely to try Copilot CLI when skip-level peers used it. Retention tracked coding activity. Adopters merged about 24% more pull requests. Copilot CLI adopters showed about 2.2 times the lift of Claude Code adopters. Microsoft directed most Claude Code users to Copilot CLI after the study window.
Public engineering write-up (research paper by Microsoft, which owns GitHub)
Organization metrics reported
No model named
Deployment details
Company-wide engineering rollout. January 5 to April 29, 2026. Copilot CLI and Claude Code.
Public engineering write-up (research paper by Microsoft, which owns GitHub)
Organization metrics reported
No model named
Team
Microsoft engineering, tens of thousands of engineers
Duration
January 5 to April 29, 2026
Product version
Copilot CLI and Claude Code
Measurement
Merged pull requests per engineer against a synthetic control, adoption and retention models
David GriffithsJul 14, 2025American BankerCiti runs Copilot across 40,000 developers and added Devin for repeatable tasks.
David GriffithsChief Technology OfficerCiti
Citi's 40,000 developers, 2025.
American Banker, published Jul 14, 2025Read the report (opens American Banker in a new tab)
Observation
Citi runs Copilot across 40,000 developers and added Devin for repeatable tasks.
Takeaway
Banks keep Copilot as the assistant and bring in an agent for bounded, repeatable work, with no agent deploying code.
What happened
American Banker reported on Jul 14, 2025 that Citi runs GitHub Copilot as an assistive tool for its 40,000 developers and began giving them Devin for simple, repeatable tasks. Griffiths estimated Copilot gives regular users 5% to 15% more productivity, and the agentic approach two to 20 times faster work on certain tasks.
“Developers that regularly use the GitHub Copilot coding assistant get 5% to 15% greater productivity, Griffiths estimated. "But what we've seen with the agentic approach, for certain kinds of tasks, is you can get anywhere from twice as fast to 20 times as fast for a given task," he said.”
David Griffiths
Second-hand news report (American Banker)
Executive estimates reported
Deployment details
Citi's 40,000 developers, 2025.
Second-hand news report (American Banker)
Executive estimates reported
Team
Citi, 40,000 developers
Workload
Library upgrades, rewrites, patches, documentation and tests
Product version
GitHub Copilot beside Devin
Measurement
Estimated productivity gain
James FarisJul 29, 2026businessinsider.comDisney is dropping Copilot for US staff after employees said they rarely used it, Business Insider reported.
James FarisReporterBusiness Insider
Disney's US technology organization, August 2026.
businessinsider.com, published Jul 29, 2026Read the report (opens businessinsider.com in a new tab)
Observation
Disney is dropping Copilot for US staff after employees said they rarely used it, Business Insider reported.
Takeaway
Low weekly use is the renewal signal; agents engineers choose replace the assistant they ignore.
What happened
Business Insider reported on Jul 29, 2026, from an internal message, that Disney is dropping GitHub Copilot, Kiro and Amazon Q for US staff in August, keeping Claude Enterprise and Cursor and preparing to adopt Codex. Eight Disney tech employees said they rarely or never used Copilot.
“Eight Disney tech employees told Business Insider that they rarely or never used GitHub Copilot.”
James Faris
Second-hand news report (Business Insider, from a Disney internal message)
Employee accounts reported
Deployment details
Disney's US technology organization, August 2026.
Second-hand news report (Business Insider, from a Disney internal message)
Employee accounts reported
Team
Disney technology staff, US
Workload
Software development across Disney
Product version
GitHub Copilot, dropped
Marco ArgentiJul 25, 2025American BankerGoldman Sachs gives every developer Copilot and its teams Devin as a colleague.
Marco ArgentiChief Information OfficerGoldman Sachs
Goldman Sachs' 12,000 developers, 2025.
American Banker, published Jul 25, 2025Read the report (opens American Banker in a new tab)
Observation
Goldman Sachs gives every developer Copilot and its teams Devin as a colleague.
Takeaway
Copilot stays as the assistant while delegated work goes to an agent beside it.
What happened
American Banker reported on Jul 25, 2025 that Goldman Sachs gave its 12,000 developers GitHub Copilot in 2024 and is now giving developer teams Devin to act as a team member.
“All the developers have Github Copilot. They are getting the Devin technology. There's a shift from the developer assistant that helps you auto complete your code or generate code for a specific task, so I now have a virtual colleague that I can delegate work to.”
Marco Argenti
Second-hand news report (American Banker)
Executive statement reported
Deployment details
Goldman Sachs' 12,000 developers, 2025.
Second-hand news report (American Banker)
Executive statement reported
Team
Goldman Sachs, 12,000 developers
Duration
Since 2024
Workload
Software development
Product version
GitHub Copilot beside Devin
Suhail PatelSep 16, 2025The Pragmatic EngineerMonzo keeps investing in Copilot while it tries Cursor, Windsurf and Claude Code.
Suhail PatelLeads the platform teamsMonzo
Monzo engineering, September 2025.
The Pragmatic Engineer, published Sep 16, 2025Read the report (opens The Pragmatic Engineer in a new tab)
Observation
Monzo keeps investing in Copilot while it tries Cursor, Windsurf and Claude Code.
Takeaway
Copilot often stays as the base tool while teams trial agents around it.
What happened
The Pragmatic Engineer reported on Sep 16, 2025 that Monzo's first AI coding tool was GitHub Copilot, which came in through VS Code, and that the bank has since been trying Cursor, Windsurf and Claude Code beside it while continuing to invest in Copilot.
“The very first one was GitHub Copilot. That's going to be quite a common answer for most companies, mostly because I think GitHub played their strategy very well, as Copilot ‘sneaked’ its way into VS Code.”
Suhail Patel
Second-hand news report (The Pragmatic Engineer)
Organization practice described
Deployment details
Monzo engineering, September 2025.
Second-hand news report (The Pragmatic Engineer)
Organization practice described
Team
Monzo engineering
Workload
Banking software
Product version
GitHub Copilot beside Cursor, Windsurf and Claude Code

Left it or turned it down1

Indeed retired Copilot once its developers had abandoned it, with weekly use down to about 3%.

Switched to or from it1

Superhuman moved on from Copilot's autocomplete to Cursor and then Claude Code.

Run it beside another tool2

At Shopify, Copilot is not shrinking but grows more slowly than command-line agents.

At 37signals, Copilot's pull request review keeps finding real breakage behind the agents.

Evaluated it2

ZoomInfo trialed Copilot with a pilot group before its company-wide rollout, and paced the licenses.

The UK public sector trialled Copilot across 50+ organisations, with a 15.8% line acceptance rate.

Adopted it1

An open-source maintainer merged 8 of 11 Copilot agent tasks and steered the design himself.

Reported by others6

A four-developer Thoughtworks team went from 20 to 23 stories a sprint, after a slower start.

At Microsoft, command-line agent use spread through peers, and adopters merged more pull requests.

Citi runs Copilot across 40,000 developers and added Devin for repeatable tasks.

Disney is dropping Copilot for US staff after employees said they rarely used it, Business Insider reported.

Goldman Sachs gives every developer Copilot and its teams Devin as a colleague.

Monzo keeps investing in Copilot while it tries Cursor, Windsurf and Claude Code.