What teams building AI features on Claude API report, with every finding tied to its evidence.
Read the findingsPractitioner sentiment, teams and leaders
+25 Positive from 52 sources that take a side
How this figure is madeFrom 48 authors in 42 discussions. Run a pilot on your own prompts to measure output quality and cost per task.
Mar 2025 to Oct 2026.
93 sources from practitioners, by month and sentiment.
Teams running Claude in production praise model quality and tool use, and raise token cost at volume and behavior changes between model versions most often. The practice they most often share is running their own evaluations before adopting or switching models.
Positive, mixed and negative sources
The score counts them as 20 positive, 20 mixed and 8 negative votes, one per author. See How the numbers work.
See the evidenceBelow: 4 praise themes, 5 concerns, 2 points of agreement and 2 disagreements.
Counted for Teams and technical leaders, from 59 sources. Change group
Every capability practitioners describe positively, ranked by how many sources say so. Open one to read who said it.
Counted for Teams and technical leaders.
Positive, 16
No opinion stated
Gaurav Shrivastava, founder of Pensieve
PLUTUS 32B against Claude Sonnet 4.5 on a diagnostic queryX post
On the same diagnostic query, Pensieve's own 32B model tied Claude Sonnet 4.5 on accuracy, Claude framed compliance better, and Claude answered in 11.3 seconds against 15.7 for the tuned 32B model.
Positive
Amp Team, Amp
Opus 5.5Company blog post, Amp
Amp made Opus 5.5 its default medium model after its evals showed it solving 65% of tasks against 61% for GPT-5.6 Sol, at 10% lower cost. Amp keeps Opus 5.5 at high effort because higher settings burn tokens and score lower.
Mixed
Howard Chen, builder of the cwcode harness
DeepSeek V4 Pro at 5% the cost of Claude: what it takes to close the gapNewsletter, Howard's Newsletter
Chen's team moved its own harness to DeepSeek V4 Pro at 5% of Claude's cost, and still finds Claude ahead on long-horizon planning and first-shot UI work.
Positive
Niko Grupen, Head of Applied Research at Harvey
Opus 4.8, Now Live in HarveyCompany blog post, Harvey
On Harvey's Legal Agent Benchmark, Opus 4.8 became the first model to pass 10% under strict all-pass scoring, up from 7.1% for Opus 4.7. Harvey's evaluators praised its legal accuracy and noted it could surface judgment calls on ambiguous tasks better.
Positive
Suraj Gupta, Warp
Opus 4.7 in WarpCompany blog post, Warp
Warp made Opus 4.7 the default for its auto (genius) agent mode after its testing found the model solved harder Terminal Bench tasks and a concurrency bug Opus 4.6 could not.
Mixed
Sudeeksha Murari, AI Research & Risk Lead, Augment Code
Frontier AI at half the cost: Gemini 3.1 Pro now available in Augment CodeCompany engineering blog
Augment found Opus 4.6 better at inferring intent than Gemini 3.1 Pro, but often got equivalent output on scoped changes at higher cost. Its same-task planning comparison also exposed a detail Opus missed.
Positive
Harvey Team, Harvey
Sonnet 4.5, Now Live in HarveyCompany blog post, Harvey
Harvey measured Sonnet 4.5 at 89.55% on its BigLaw Bench suite, the highest Claude score it had recorded, and found long-context reasoning strong enough for complex legal documents such as credit agreements.
Positive
enraged_camel, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
The commenter's team uses Haiku 4.5 to turn users' natural language requests into complex structured specs for interior design and construction, with near perfect accuracy.
Positive
chriddyp, Hacker News commenter at Plotly
Comment on: Claude Haiku 5.5Hacker News comment
On Plotly's DataAnalyticsBench, Haiku 5.5 was 9x cheaper than Haiku 4.5, two letter grades better, and the fastest model, at US$0.38 for 40 questions.
Positive
axi0m, Hacker News commenter
Comment on: MAI-Code-1-FlashHacker News comment
Uses Haiku 4.5 in production on scoped tasks that Opus 4.6 generates, to optimize speed, efficiency, and cost.
Positive
gertlabs, Hacker News commenter at Gertlabs
Comment on: Claude Opus 4.7Hacker News comment
On Gertlabs' private suite, Opus 4.7 led agentic sessions with tools and tied GPT 5.4 on one-shot coding reasoning, though Anthropic's card shows context regressions.
Positive
Brian Carpio, OutcomeOps
You're Probably Using the Wrong Bedrock Model. Here's How to Tell.Engineering blog post, OutcomeOps blog
Running eight Bedrock models through the same RAG pipeline, Haiku 4.5 gave the most complete answer, beating Sonnet 4.6 at lower cost and latency.
Positive
Michael Matloka, PostHog
8 learnings from 1 year of agents: PostHog AIEngineering blog post, PostHog blog
PostHog runs its in-product agent loop on Claude Sonnet 4.5 for quality, speed, and cost, credits Claude 4 tool reliability, and avoids LLM frameworks.
"We're currently using Claude Sonnet 4.5 for the core agent loop, as it hits a sweet spot of quality, speed, and cost…"
Positive
The Cognition Team
Rebuilding Devin for Claude Sonnet 4.5: Lessons and ChallengesEngineering blog post, Cognition blog
Rebuilt on Sonnet 4.5, Devin ran 2x faster and scored 12% higher on Cognition's evals. The model also cut tasks short when it believed its context window was nearly full, which the team fixed by enabling the 1M-token beta and capping use at 200K.
No opinion stated
Kelly Hong, Anton Troynikov, and Jeff Huber, Chroma
Context Rot: How Increasing Input Tokens Impacts LLM PerformanceIndependent benchmark, Chroma Technical Report
Across 18 models, accuracy grew less reliable as input length grew; Claude models showed the widest gap between focused and full-length prompts, mostly from abstaining, and the lowest hallucination rates.
Positive
Guy Gur-Ari, co-founder of Augment Code
Claude Sonnet 4: The best model. With the best Context Engine.Company blog post, Augment Code
Augment Code measured Sonnet 4 against Claude 3.7 Sonnet. Its single-pass SWE-bench agent score rose from 60.6% to 70.6% and its valid tool-call rate from 25% to 80%. Augment made Sonnet 4 the default model on every plan.
Mixed or negative, 4
Negative
d1l, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
The commenter's team runs Haiku 4.5 for simple latency-sensitive tasks; early tests of Haiku 5.5 gave worse results, prompt leakage, and slower responses.
Mixed
faangguyindia, Hacker News commenter
Comment on: Claude Sonnet 4 now supports 1M tokens of contextHacker News comment
The commenter's Asia Pacific company could not get Claude through Vertex AI, and the commenter's own testing found Gemini 2.5 Pro had nearly closed the gap.
Mixed
Dan Shipper, CEO and cofounder of Every
Vibe Check: Claude Sonnet 4 Now Has a 1-million Token Context WindowNewsletter, Every
Testing Sonnet 4's new 1M-token window against Gemini 2.5, the author found Claude faster with fewer hallucinations, and its text and code analysis less detailed and complete.
Mixed
Dedy Kredo and Tal Ridnik, Qodo
We benchmarked GPT-4.1 vs Claude 3.7: Here's what we foundEngineering blog post, Qodo blog
On 200 real pull requests judged by o3-mini, GPT-4.1's code suggestions beat Claude 3.7 Sonnet's in 54.9% of cases, averaging 6.81 against 6.66.
Counted for Teams and technical leaders.
Positive, 10
Positive
enraged_camel, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
The commenter's team uses Haiku 4.5 to turn users' natural language requests into complex structured specs for interior design and construction, with near perfect accuracy.
Positive
Patrick Foster, Software Engineer, Mintlify
Mintlify + Claude Opus 4.6: Powering AI-native knowledge managementCompany blog post, Mintlify
On Mintlify's internal benchmarks for its docs assistant, Opus 4.6 gave answers about 20% more concise than other models, with better tool calling for searching documentation.
Positive
Michael Matloka, PostHog
8 learnings from 1 year of agents: PostHog AIEngineering blog post, PostHog blog
PostHog runs its in-product agent loop on Claude Sonnet 4.5 for quality, speed, and cost, credits Claude 4 tool reliability, and avoids LLM frameworks.
"We're currently using Claude Sonnet 4.5 for the core agent loop, as it hits a sweet spot of quality, speed, and cost…"
Positive
mparis, Hacker News commenter
Comment on: Structured outputs on the Claude Developer PlatformHacker News comment
Ran JSON-schema structured outputs on Claude through Bedrock in production for a year with good results, after OpenAI's strict validation rejected schema elements they used.
No opinion stated
Saoud Rizwan, founder of Cline
Cline: The Open Source Code Agent, with Saoud Rizwan and Nik PashPodcast episode, Latent Space
Cline's founder says Claude Sonnet 4 reached about a 4% diff edit failure rate on Cline's internal benchmarks. That is down from the 20s and 30s when fast apply launched. Claude 3.5 picked out details with 90% of its 200K window full.
Mixed
Omri Levy, Baz
How to Achieve Structured Output in Claude 3.7: Three Practical ApproachesEngineering blog post, Baz blog
Sonnet 3.7 lacked constrained decoding, so Baz produced structured output through tool calling, using LangChain and Bedrock.
Positive
Guy Gur-Ari, co-founder of Augment Code
Claude Sonnet 4: The best model. With the best Context Engine.Company blog post, Augment Code
Augment Code measured Sonnet 4 against Claude 3.7 Sonnet. Its single-pass SWE-bench agent score rose from 60.6% to 70.6% and its valid tool-call rate from 25% to 80%. Augment made Sonnet 4 the default model on every plan.
Positive
Amp Team, Amp
Amp Now Uses Claude Sonnet 4Company blog post, Amp
After weeks of internal use, Amp switched its primary model to Claude Sonnet 4, finding it faster and smarter at tool calling. Amp also found it more willing to call tools in parallel and better at recovering from errors.
Positive
gertlabs, Hacker News commenter at Gertlabs
Comment on: Claude Opus 4.7Hacker News comment
On Gertlabs' private suite, Opus 4.7 led agentic sessions with tools and tied GPT 5.4 on one-shot coding reasoning, though Anthropic's card shows context regressions.
Positive
The Cognition Team
Rebuilding Devin for Claude Sonnet 4.5: Lessons and ChallengesEngineering blog post, Cognition blog
Rebuilt on Sonnet 4.5, Devin ran 2x faster and scored 12% higher on Cognition's evals. The model also cut tasks short when it believed its context window was nearly full, which the team fixed by enabling the 1M-token beta and capping use at 200K.
Mixed or negative, 2
Negative
visioninmyblood, Hacker News commenter
Comment on: Claude Advanced Tool UseHacker News comment
Found MCP buggy and Claude's tool use underperforming in the visual domain, so the commenter's team built its own protocol for a production-grade agent.
Mixed
adidoit, Hacker News commenter at Socratify
Comment on: Structured outputs on the Claude Developer PlatformHacker News comment
Had not used Haiku in production at Socratify because it lacked structured output, and saw the tool-calling workaround as a hack.
Counted for Teams and technical leaders.
Positive
chriddyp, Hacker News commenter at Plotly
Comment on: Claude Haiku 5.5Hacker News comment
On Plotly's DataAnalyticsBench, Haiku 5.5 was 9x cheaper than Haiku 4.5, two letter grades better, and the fastest model, at US$0.38 for 40 questions.
Positive
Istasis Mishra, Member of Technical Staff, and Pramod Kalipatnapu, Cofounder, at Revefi
How Revefi Reduced Its Data Agent's Spend by 90%Engineering blog post, Revefi blog
A timestamp near the top of Raden's system prompt broke prompt caching; moving volatile content to the end lifted cache reads to 99% of input tokens.
"Earlier this year, we had to quickly reduce our data spend. We had inadvertently introduced a change that made our costs spiral out of control!"
Positive
__jl__, Hacker News commenter
Comment on: Gemini 3.5 FlashHacker News comment
Sees higher cache hit rates from OpenAI, Anthropic, and Fireworks than from Google, whose random cache misses offset its cheaper cached tokens.
Positive
Parth Malhotra, ProjectDiscovery
How We Cut LLM Costs by 59% With Prompt CachingEngineering blog post, ProjectDiscovery blog
Explicit cache breakpoints and moving dynamic content out of the prompt prefix raised Neo's cache hit rate from 7% to 84%. LLM costs fell 59%, with 9.8 billion tokens served from cache.
"Overall, caching saved 59% on LLM costs compared to what the same token volume would have cost at full input rates. Post-optimization that number is 66%, and the last 10 days are at 70%."
Positive
Amp Team, Amp
Opus 5.5Company blog post, Amp
Amp made Opus 5.5 its default medium model after its evals showed it solving 65% of tasks against 61% for GPT-5.6 Sol, at 10% lower cost. Amp keeps Opus 5.5 at high effort because higher settings burn tokens and score lower.
Positive
Liza Katz, Neradot
Case Study: Cutting LLM Cost by 36% with Prompt CachingEngineering blog post, Neradot blog
Shrinking the prompt and ordering it stable-first for prompt caching on Claude via Bedrock cut a production support agent's inference cost by 36% with no measured quality change.
Positive
axi0m, Hacker News commenter
Comment on: MAI-Code-1-FlashHacker News comment
Uses Haiku 4.5 in production on scoped tasks that Opus 4.6 generates, to optimize speed, efficiency, and cost.
Positive
Brian Carpio, OutcomeOps
You're Probably Using the Wrong Bedrock Model. Here's How to Tell.Engineering blog post, OutcomeOps blog
Running eight Bedrock models through the same RAG pipeline, Haiku 4.5 gave the most complete answer, beating Sonnet 4.6 at lower cost and latency.
Counted for Teams and technical leaders.
Positive, 8
Positive
The Cognition Team
Rebuilding Devin for Claude Sonnet 4.5: Lessons and ChallengesEngineering blog post, Cognition blog
Rebuilt on Sonnet 4.5, Devin ran 2x faster and scored 12% higher on Cognition's evals. The model also cut tasks short when it believed its context window was nearly full, which the team fixed by enabling the 1M-token beta and capping use at 200K.
Positive
chriddyp, Hacker News commenter at Plotly
Comment on: Claude Haiku 5.5Hacker News comment
On Plotly's DataAnalyticsBench, Haiku 5.5 was 9x cheaper than Haiku 4.5, two letter grades better, and the fastest model, at US$0.38 for 40 questions.
No opinion stated
Jason Li, Epoch AI
Long-context latency scales quadratically for GPT-5.6 but nearly linearly for Claude 5Independent benchmark, Epoch AI
Measuring time to first token on prompts up to one million tokens, Epoch found latency on Claude Sonnet 5 and Opus 5 grows close to linearly. GPT-5.6 latency grows quadratically with prompt length.
Positive
axi0m, Hacker News commenter
Comment on: MAI-Code-1-FlashHacker News comment
Uses Haiku 4.5 in production on scoped tasks that Opus 4.6 generates, to optimize speed, efficiency, and cost.
Positive
Brian Carpio, OutcomeOps
You're Probably Using the Wrong Bedrock Model. Here's How to Tell.Engineering blog post, OutcomeOps blog
Running eight Bedrock models through the same RAG pipeline, Haiku 4.5 gave the most complete answer, beating Sonnet 4.6 at lower cost and latency.
Positive
Michael Matloka, PostHog
8 learnings from 1 year of agents: PostHog AIEngineering blog post, PostHog blog
PostHog runs its in-product agent loop on Claude Sonnet 4.5 for quality, speed, and cost, credits Claude 4 tool reliability, and avoids LLM frameworks.
"We're currently using Claude Sonnet 4.5 for the core agent loop, as it hits a sweet spot of quality, speed, and cost…"
Mixed
Dan Shipper, CEO and cofounder of Every
Vibe Check: Claude Sonnet 4 Now Has a 1-million Token Context WindowNewsletter, Every
Testing Sonnet 4's new 1M-token window against Gemini 2.5, the author found Claude faster with fewer hallucinations, and its text and code analysis less detailed and complete.
Positive
Amp Team, Amp
Amp Now Uses Claude Sonnet 4Company blog post, Amp
After weeks of internal use, Amp switched its primary model to Claude Sonnet 4, finding it faster and smarter at tool calling. Amp also found it more willing to call tools in parallel and better at recovering from errors.
Mixed or negative, 2
Negative
d1l, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
The commenter's team runs Haiku 4.5 for simple latency-sensitive tasks; early tests of Haiku 5.5 gave worse results, prompt leakage, and slower responses.
Mixed
d1l, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
After Sonnet 5 performed far worse than Sonnet 4.6, the commenter's team waited for Sonnet 5.5 before moving customer workloads, and may take Haiku work to another provider.
Every concern practitioners raise, most raised first.
Counted for Teams and technical leaders.
Raised as a concern, 11
Mixed
coffeebeqn, Hacker News commenter
Comment on: Claude Opus 5.5Hacker News comment
The commenter's team used Opus less than it wanted because it was too expensive for general use, and juggled models until the Opus 5.5 price drop.
Mixed
Bruno Škvorc, Staff Software Engineer, Lindy
Migrating from Claude to DeepSeekEngineering blog post, Lindy
Lindy moved most of its assistant traffic from Claude Sonnet and Gemini to DeepSeek v4 Flash, served through Atlas Cloud, and cut inference costs on those routes by about 90%. It ramped up only while online evals and retention held. Sonnet still runs when a user picks it or a harder path needs it.
"We moved most managed model traffic from Claude Sonnet and Gemini paths to DeepSeek v4 Flash on Atlas Cloud. Inference costs on those routes fell by about 90%."
Mixed
Howard Chen, builder of the cwcode harness
DeepSeek V4 Pro at 5% the cost of Claude: what it takes to close the gapNewsletter, Howard's Newsletter
Chen's team moved its own harness to DeepSeek V4 Pro at 5% of Claude's cost, and still finds Claude ahead on long-horizon planning and first-shot UI work.
Negative
cedws, Hacker News commenter
Comment on: Claude Fable 5Hacker News comment
The commenter's employer is all in on Anthropic's API pricing, where under 100M tokens cost US$800 in a month, against US$10 for 600M DeepSeek tokens on a personal project.
Negative
curioussquirrel, Hacker News commenter
Comment on: Measuring Claude 4.7's tokenizer costsHacker News comment
Says Claude's tokenizers have grown less efficient since Sonnet 3.5, and non-English content costs far more, by hundreds of percent for scripts like Tamil or Japanese.
Negative
joshdev, Hacker News commenter
Comment on: Sonnet 4.6 Elevated Rate of ErrorsHacker News comment
The commenter's team found Bedrock double counting cache writes for Anthropic models with thinking enabled, alongside a much higher cache miss rate that inflated usage.
Mixed
Sudeeksha Murari, AI Research & Risk Lead, Augment Code
Frontier AI at half the cost: Gemini 3.1 Pro now available in Augment CodeCompany engineering blog
Augment found Opus 4.6 better at inferring intent than Gemini 3.1 Pro, but often got equivalent output on scoped changes at higher cost. Its same-task planning comparison also exposed a detail Opus missed.
Mixed
film42, Hacker News commenter
Comment on: Claude Sonnet 4 now supports 1M tokens of contextHacker News comment
Processing customer invoice PDFs, found Gemini 2.5 Pro far cheaper per document than Claude 4, but rated Gemini's rate limits and Vertex SDK worse than Anthropic's.
Mixed
Siddharth Kodwani, Software Engineer, GenAI Platform, DoorDash, with Swaroop Chitlur
Building a GenAI Platform at DoorDashConference talk, InfoQ
DoorDash runs the Claude API with an Amazon Bedrock fallback, and OpenAI with an Azure fallback, behind one gateway that attributes cost to each department. The platform team names rising cost at scale and model deprecations that force migrations across hundreds of use cases.
"We actually had to go understand what the fallback should be, what kind of routing should be. At one point, I remember we had OpenAI endpoint, but we also had a fallback on Azure endpoint. Or, we had Claude model API endpoint, then we had a fallback on Bedrock, so that the services can rely on us from the reliability point of view. We went in that direction. We quickly realized that as the usage is going up, then we need a way to track, because now finance got involved. Like, you guys have one API key for multiple teams, how do I track who is using what? We quickly realized we need workspaces where we have cost attribution. We can actually drill down on who is using how much. We can attribute cost back to those departments. That's what basically we did with LLM Gateway."
Mixed
Earendil Engineering
Prompt Caching In AgentsEngineering blog post, Earendil
Earendil's Pi harness follows Anthropic's five-minute API cache default, and pauses longer than that rebill the whole prefix as input.
Negative
yuanzhi1203, Hacker News commenter
Comment on: Measuring Claude 4.7's tokenizer costsHacker News comment
The commenter's team saw a model hot swap cause a 100% cache miss on sessions of up to 800K tokens, making the cache rebuild very expensive.
Discussed without concern, 1
Positive
Amp Team, Amp
Opus 5.5Company blog post, Amp
Amp made Opus 5.5 its default medium model after its evals showed it solving 65% of tasks against 61% for GPT-5.6 Sol, at 10% lower cost. Amp keeps Opus 5.5 at high effort because higher settings burn tokens and score lower.
GILD's fact checks, which don't count as sources
Checked Oct 12, 2026.Each usage tier carries a monthly spend cap. An organization can set its own lower spend limit, with per-workspace spend limits on top. A report of a surprise bill does not establish that spend controls were unavailable. Rate limits
Open them in the recordCounted for Teams and technical leaders.
Raised as a concern, 8
Negative
d1l, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
The commenter's team runs Haiku 4.5 for simple latency-sensitive tasks; early tests of Haiku 5.5 gave worse results, prompt leakage, and slower responses.
Mixed
d1l, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
After Sonnet 5 performed far worse than Sonnet 4.6, the commenter's team waited for Sonnet 5.5 before moving customer workloads, and may take Haiku work to another provider.
Negative
mchusma, Hacker News commenter
Comment on: Claude Fable 5.1 and Claude Mythos 5.1Hacker News comment
With Haiku not updated in a year and Sonnet issues this year, the commenter's team migrated workflows away from Haiku and Sonnet to other providers' models.
Mixed
Jorge Peraza, MonkeysCode
Handling Frontier Model Churn: GPT-6, Caching, and SSEEngineering blog post, DEV Community
Long thinking before the first token caused premature connection closures on Anthropic streams, and switching providers mid-session fragments the prompt cache.
Mixed
HyperL0gi, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
The commenter's team's evals and human pairwise tests still rank Haiku 4.5 first over Haiku 5.5, so they are checking whether prompts need to change for the new model.
Mixed
Siddharth Kodwani, Software Engineer, GenAI Platform, DoorDash, with Swaroop Chitlur
Building a GenAI Platform at DoorDashConference talk, InfoQ
DoorDash runs the Claude API with an Amazon Bedrock fallback, and OpenAI with an Azure fallback, behind one gateway that attributes cost to each department. The platform team names rising cost at scale and model deprecations that force migrations across hundreds of use cases.
"We actually had to go understand what the fallback should be, what kind of routing should be. At one point, I remember we had OpenAI endpoint, but we also had a fallback on Azure endpoint. Or, we had Claude model API endpoint, then we had a fallback on Bedrock, so that the services can rely on us from the reliability point of view. We went in that direction. We quickly realized that as the usage is going up, then we need a way to track, because now finance got involved. Like, you guys have one API key for multiple teams, how do I track who is using what? We quickly realized we need workspaces where we have cost attribution. We can actually drill down on who is using how much. We can attribute cost back to those departments. That's what basically we did with LLM Gateway."
Negative
yuanzhi1203, Hacker News commenter
Comment on: Measuring Claude 4.7's tokenizer costsHacker News comment
The commenter's team saw a model hot swap cause a 100% cache miss on sessions of up to 800K tokens, making the cache rebuild very expensive.
Negative
curioussquirrel, Hacker News commenter
Comment on: Measuring Claude 4.7's tokenizer costsHacker News comment
Says Claude's tokenizers have grown less efficient since Sonnet 3.5, and non-English content costs far more, by hundreds of percent for scripts like Tamil or Japanese.
Discussed without concern, 3
Mixed
gertlabs, Hacker News commenter at Gertlabs
Comment on: Claude Opus 4.7Hacker News comment
Resampling Opus 4.6 through the API during reports of degraded quality, Gertlabs found its eval performance about the same.
Positive
gertlabs, Hacker News commenter at Gertlabs
Comment on: Claude Opus 4.7Hacker News comment
On Gertlabs' private suite, Opus 4.7 led agentic sessions with tools and tied GPT 5.4 on one-shot coding reasoning, though Anthropic's card shows context regressions.
Positive
Michael Matloka, PostHog
8 learnings from 1 year of agents: PostHog AIEngineering blog post, PostHog blog
PostHog runs its in-product agent loop on Claude Sonnet 4.5 for quality, speed, and cost, credits Claude 4 tool reliability, and avoids LLM frameworks.
"We're currently using Claude Sonnet 4.5 for the core agent loop, as it hits a sweet spot of quality, speed, and cost…"
GILD's fact checks, which don't count as sources
Checked Oct 12, 2026.Anthropic gives at least 60 days' notice before retiring a publicly released model on its own platforms. Amazon Bedrock and Google Cloud set their own retirement schedules. Model deprecations
Open them in the recordCounted for Teams and technical leaders.
Raised as a concern, 5
No opinion stated
Chaithanya Bandi and coauthors, Scale AI and National University of Singapore
MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP ServersIndependent benchmark, arXiv paper
On real MCP servers, Claude Haiku 4.5 passed 40.2% of tasks, the lowest of 20 models. In the failure analysis, faulty synthesis of gathered evidence rose to 25.0% for Claude Opus 4.7.
No opinion stated
Nathan Sobo, Zed
Building Zed's agentic editing (The Changelog #640)Podcast episode, The Changelog
Zed streams edits from Claude through tool calls and found Claude mismatching the XML tags of its edit format often enough to break the parser. Prompt rules raised its eval pass rate from 60% to 95%.
Negative
visioninmyblood, Hacker News commenter
Comment on: Claude Advanced Tool UseHacker News comment
Found MCP buggy and Claude's tool use underperforming in the visual domain, so the commenter's team built its own protocol for a production-grade agent.
Mixed
adidoit, Hacker News commenter at Socratify
Comment on: Structured outputs on the Claude Developer PlatformHacker News comment
Had not used Haiku in production at Socratify because it lacked structured output, and saw the tool-calling workaround as a hack.
Mixed
Omri Levy, Baz
How to Achieve Structured Output in Claude 3.7: Three Practical ApproachesEngineering blog post, Baz blog
Sonnet 3.7 lacked constrained decoding, so Baz produced structured output through tool calling, using LangChain and Bedrock.
Discussed without concern, 2
Positive
Michael Matloka, PostHog
8 learnings from 1 year of agents: PostHog AIEngineering blog post, PostHog blog
PostHog runs its in-product agent loop on Claude Sonnet 4.5 for quality, speed, and cost, credits Claude 4 tool reliability, and avoids LLM frameworks.
"We're currently using Claude Sonnet 4.5 for the core agent loop, as it hits a sweet spot of quality, speed, and cost…"
Positive
mparis, Hacker News commenter
Comment on: Structured outputs on the Claude Developer PlatformHacker News comment
Ran JSON-schema structured outputs on Claude through Bedrock in production for a year with good results, after OpenAI's strict validation rejected schema elements they used.
Counted for Teams and technical leaders.
Raised as a concern, 3
Mixed
Jorge Peraza, MonkeysCode
Handling Frontier Model Churn: GPT-6, Caching, and SSEEngineering blog post, DEV Community
Long thinking before the first token caused premature connection closures on Anthropic streams, and switching providers mid-session fragments the prompt cache.
Negative
d1l, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
The commenter's team runs Haiku 4.5 for simple latency-sensitive tasks; early tests of Haiku 5.5 gave worse results, prompt leakage, and slower responses.
Mixed
d1l, Hacker News commenter
Comment on: Claude Haiku 5.5Hacker News comment
After Sonnet 5 performed far worse than Sonnet 4.6, the commenter's team waited for Sonnet 5.5 before moving customer workloads, and may take Haiku work to another provider.
Discussed without concern, 2
Positive
chriddyp, Hacker News commenter at Plotly
Comment on: Claude Haiku 5.5Hacker News comment
On Plotly's DataAnalyticsBench, Haiku 5.5 was 9x cheaper than Haiku 4.5, two letter grades better, and the fastest model, at US$0.38 for 40 questions.
Positive
Istasis Mishra, Member of Technical Staff, and Pramod Kalipatnapu, Cofounder, at Revefi
How Revefi Reduced Its Data Agent's Spend by 90%Engineering blog post, Revefi blog
A timestamp near the top of Raden's system prompt broke prompt caching; moving volatile content to the end lifted cache reads to 99% of input tokens.
"Earlier this year, we had to quickly reduce our data spend. We had inadvertently introduced a change that made our costs spiral out of control!"
GILD's fact checks, which don't count as sources
Checked Oct 12, 2026.Anthropic recommends Haiku 5.5 and lower effort for speed-critical work. Reducing latency
Checked Oct 25, 2026.Fast mode on Opus 5.5 costs US$8 and US$40 per million tokens. It runs on the Claude API, Microsoft Foundry deployments hosted on Azure, and Claude Models on Google Cloud (Anthropic managed). Fast mode on Opus 5 and Opus 4.8 is in pre-release on the Claude API only. Fast mode
Open them in the recordCounted for Teams and technical leaders.
Raised as a concern, 3
Mixed
Armin Ronacher, Earendil
Agent Design Is Still HardPersonal blog post, lucumr.pocoo.org
Building agents across providers, the team found Anthropic's web search tool routinely broke message history through the Vercel AI SDK, and cache management much easier through Anthropic's own SDK.
Mixed
Jorge Peraza, MonkeysCode
Handling Frontier Model Churn: GPT-6, Caching, and SSEEngineering blog post, DEV Community
Long thinking before the first token caused premature connection closures on Anthropic streams, and switching providers mid-session fragments the prompt cache.
Mixed
Dinmukhamed Mailibay, Laminar
How we instrumented Claude Agent SDK using a tiny Rust proxyEngineering blog post, Laminar blog
Teams building on the Claude Agent SDK cannot see prompts, tool calls, or subagent trajectories out of the box, so Laminar built a Rust proxy to trace them.
Discussed without concern, 2
Positive
Ian Vanagas, PostHog
What we wish we knew about building AI agentsNewsletter, Product for Engineers (PostHog newsletter)
After three harness iterations, PostHog moved its agent to the Claude Agent SDK with MCP tools and skills, and would skip a custom harness if starting again.
Mixed
film42, Hacker News commenter
Comment on: Claude Sonnet 4 now supports 1M tokens of contextHacker News comment
Processing customer invoice PDFs, found Gemini 2.5 Pro far cheaper per document than Claude 4, but rated Gemini's rate limits and Vertex SDK worse than Anthropic's.
Statements this group's sources agree on, then the ones close to agreement.
Teams run their own evaluation set before adopting or switching models, since published benchmarks and list prices do not predict results on their task.
15 supporting sourcesTeams use Claude through the API as the model behind their own agents, with tools and multi-step loops.
12 supporting sourcesClose to agreement: each needs 12 supporting sources
Teams building on Claude keep at least one other model provider in reach, for price, capacity, or task fit.
11 supporting sourcesPrompt caching pays only when prompts are built for it: stable content first, volatile content last, and hit rates monitored.
7 supporting sourcesTeams route work across model sizes: a smaller model such as Haiku for scoped, high-volume tasks and a larger model for hard reasoning.
6 supporting sourcesQuestions the sources split on, with the count on each side.
Evidence remains mixed
Own orchestration, 5 sources
Some call the API directly with their own loop and caching code, and find frameworks lag behind provider features.
Framework or SDK, 5 sources
Others build on a framework or harness such as the Claude Agent SDK, LangChain, LiteLLM, or Vercel's tooling.
Evidence remains mixed
Anthropic's API, 2 sources
Some call Anthropic's API directly, where features such as the Batch API and API credits are available.
Cloud provider, 8 sources
Others buy through Amazon Bedrock or Google Cloud for existing contracts, data terms, and procurement, and accept that platform's quotas and regions.
The three groups, then every practitioner by the context they work in, and how emphasis changes with scale.
Three groups drawn from the same sources. Choose a group to select whose experience counts. Within that group, each author counts once in the practitioner sentiment, with their latest view. Signals open on Teams and technical leaders.
59 sources. These accounts focus on cost at production volume, model changes, evaluation discipline, provider strategy, and where teams buy Claude, from the accounts of teams and companies running it in their products.
34 sources. These accounts focus on cost, caching, capacity, and day-to-day reliability for developers building on the API on their own.
93 sources. This gives the broadest read, but combines solo projects with production teams that can reach different conclusions.
93 sources from practitioners, by the context they work in. Filled rows count for this group.
Emphasizes Caching savings and per-token cost
Emphasizes Model changes, token cost, and switching providers
Emphasizes Agent tool use, evaluations, and spend
Emphasizes Model quality and long-context speed against alternatives
Emphasizes No company-wide account in GILD's search of engineering blogs, benchmarks, and GitHub; ask Anthropic for a reference customer at your scale
What changes with scale. Individual builders focus on cache savings and token cost. Team accounts focus on model changes between versions and cost at volume. More team accounts keep a second provider.
Where this group's sources were published.
Counted for Teams and technical leaders, from 59 sources. Change group
79 sources for this group, 20 of them documentation
Older evidence is retained when it still applies. Sources that depend on a retired model, a renamed tier, or a feature added since stay in the history, marked Needs a recheck.
See sources by originWho each source is, how it knows, whether it still applies and what it counts toward.
Newest first, on its own page. Every count above opens its sources there.
Read all 113 sourcesAbout this read
A running review of what teams building AI features on Claude API report, from individual builders to company-wide platforms. Each finding links to its sources, and each conflict shows both sides.
Counts cover only the sources GILD reviewed.
Choose a group to select whose experience counts. Within that group, each author counts once in the practitioner sentiment, with their latest view. GILD files each source in a group from how the account describes its author. A role or employer marked self-described is what the source states about themselves. Documentation checks facts for every group and never counts toward sentiment.
A source's date and whether it still applies are separate: a recent post can describe an old version, and an old one can still hold.
Flagged for a recheck, of the sources that take a side: teams and leaders 12 of 52, individual builders 4 of 23, all practitioners 16 of 75.
Each row opens its sources for the group you are reading.
| Finding | Teams and technical leaders | Individual builders | All practitioners | Sources |
|---|---|---|---|---|
| Current readCurrent sentiment | 22 positive, 21 mixed, 9 negative | 13 positive, 7 mixed, 3 negative | 35 positive, 28 mixed, 12 negative | Sources |
| PraiseModel Quality | 16 of 20 positive | 1 of 1 positive (under the minimum) | 17 of 21 positive | Sources |
| PraiseTool Use & Structured Output | 10 of 12 positive | 1 of 2 positive (under the minimum) | 11 of 14 positive | Sources |
| PraiseCost Efficiency | 8 of 8 positive | 8 of 10 positive | 16 of 18 positive | Sources |
| PraiseDeveloper Experience | 3 of 4 positive (under the minimum) | 1 of 3 positive (under the minimum) | 4 of 7 positive | Sources |
| PraiseStability | 0 of 1 positive (under the minimum) | 2 of 3 positive (under the minimum) | 2 of 4 positive (under the minimum) | Sources |
| PraiseSpeed | 8 of 10 positive | 3 of 5 positive | 11 of 15 positive | Sources |
| PraiseLong Context | 4 of 4 positive (under the minimum) | 0 of 0 positive (under the minimum) | 4 of 4 positive (under the minimum) | Sources |
| ConcernToken Cost | 11 of 12 raise it | 6 of 7 raise it | 17 of 19 raise it | Sources |
| ConcernModel Changes | 8 of 11 raise it | 1 of 1 raise it (under the minimum) | 9 of 12 raise it | Sources |
| ConcernCapacity & Rate Limits | 4 of 4 raise it (under the minimum) | 1 of 4 raise it (under the minimum) | 5 of 8 raise it | Sources |
| ConcernLatency | 3 of 5 raise it | 3 of 5 raise it | 6 of 10 raise it | Sources |
| ConcernData Handling | 2 of 3 raise it (under the minimum) | 1 of 2 raise it (under the minimum) | 3 of 5 raise it | Sources |
| ConcernInconsistent Results | 3 of 3 raise it (under the minimum) | 1 of 1 raise it (under the minimum) | 4 of 4 raise it (under the minimum) | Sources |
| ConcernTool Calling Failures | 5 of 7 raise it | 5 of 6 raise it | 10 of 13 raise it | Sources |
| ConcernSetup & SDK Friction | 3 of 5 raise it | 4 of 4 raise it (under the minimum) | 7 of 9 raise it | Sources |
| ConcernLong-Context Limits | 3 of 3 raise it (under the minimum) | 5 of 5 raise it | 8 of 8 raise it | Sources |
| ConsensusTeams keep a second provider | 11 supporting (below threshold) | 4 supporting (under the minimum) | 15 supporting (below threshold) | Sources |
| ConsensusCaching is designed in | 7 supporting (below threshold) | 3 supporting (under the minimum) | 10 supporting (below threshold) | Sources |
| ConsensusClaude runs their own agents | 12 supporting | 2 supporting (under the minimum) | 14 supporting (below threshold) | Sources |
| ConsensusEvaluate before switching | 15 supporting | 2 supporting (under the minimum) | 17 supporting (below threshold) | Sources |
| ConsensusRoute by task | 6 supporting (below threshold) | 3 supporting (under the minimum) | 9 supporting (below threshold) | Sources |
| DisagreementShould a team build its agent harness on the raw API or on a framework? | Own orchestration 5, Framework or SDK 5 | Own orchestration 4, Framework or SDK 2 | Own orchestration 9, Framework or SDK 7 | Sources |
| DisagreementShould a team buy Claude through Anthropic's API or through its cloud provider? | Anthropic's API 2, Cloud provider 8 | Anthropic's API 5, Cloud provider 4 | Anthropic's API 7, Cloud provider 12 | Sources |
| Current readEvidence mix | 79 sources | 54 sources | 113 sources | Sources |
GILD's checks of claims against official documentation, kept apart from the source counts.
