What teams building AI features on OpenAI API report, with every finding tied to its evidence.
Read the findingsPractitioner sentiment, teams and leaders
+3 Mixed from 65 sources that take a side
How this figure is madeFrom 62 authors in 62 discussions. Run a pilot on your own prompts to measure output quality and cost per task.
Jan 2025 to Oct 2026.
136 sources from practitioners, by month and sentiment.
Teams and leaders split close to evenly: 22 positive, 23 mixed and 20 negative. They praise model quality most, then cost efficiency. Latency leads their concerns, followed by token cost and inconsistent results, and DoorDash, Grab and Wealthsimple each run OpenAI beside other providers behind one gateway.
Positive, mixed and negative sources
The score counts them as 21 positive, 22 mixed and 19 negative votes, one per author. See How the numbers work.
See the evidenceBelow: 5 praise themes, 8 concerns, 1 point of agreement and 2 disagreements.
Counted for Teams and technical leaders, from 75 sources. Change group
Every capability practitioners describe positively, ranked by how many sources say so. Open one to read who said it.
Counted for Teams and technical leaders.
Positive, 14
Mixed
Jeffrey Rengifo and Gustavo Llermaly, Elastic
The best LLM writes correct Elasticsearch ES|QL 59% of the time. Here's what breaks the other 41%.Engineering blog post, Elasticsearch Labs
Elastic's test of 6,000 generated ES|QL queries found gpt-5.5 the most accurate of four models at 59.0% with only the schema, while gpt-5.4-mini scored 19.8%.
Positive
Amp team, Amp
GPT-5.4, The New OracleProduct engineering post, Amp Chronicle
In Amp's internal evals GPT-5.4 raised oracle response quality from 60.8% to 68.2% over GPT-5.2 and cut mean latency from about 6.7 to 4.9 minutes.
Positive
lieret, Hacker News commenter
Comment on: GPT-5 on SWE-bench: Cost and performance deep-diveHacker News comment
On SWE-bench Verified GPT-5 scored 65%, GPT-5 mini 60%, and nano 35%; mini gave up about 5 points for roughly a fifth of the cost.
Positive
Daniel Furman, Jacky Zhao, Vaibhav Kumar, Ed Sioufi, and Michele Catasta, Replit
Free the models: Harness design at the frontierEngineering blog post, Replit blog
Replit found GPT-6 Astra hands routine implementation to cheaper subagents without being told, and the GPT-6 family keeps the cache when effort changes mid-turn, which keeps Replit Agent's cost down.
Positive
EB66, Hacker News commenter
Comment on: Gemini 3.7 FlashHacker News comment
GPT-5 mini led the company's own benchmarks on cost and quality until GPT-5.6 Luna's price cut replaced it.
Positive
Lorenzo Gentile, Ploy engineering blog
Migrating a production AI agent to GPT-5.6Engineering blog post, Ploy blog
After fixing its eval harness, tool schemas, and cache keys, Ploy moved its agent from Claude Opus 4.8 to GPT-5.6 Sol: builds 2.2x faster at 27% lower cost.
"As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, no model met our bar for replacing Claude Opus. GPT-5.6 Sol did. After a head-to-head evaluation, we made it the default model for every Ploy workspace."
Positive
Artificial Analysis, independent model benchmarking service
GPT-5.6 Luna (Max) Intelligence, Performance & Price AnalysisIndependent benchmark, Artificial Analysis
Measures GPT-5.6 Luna (Max) at 127 output tokens per second, faster than the 116 average, and rates it among the leading models in intelligence at a reasonable price for its class.
Mixed
Kwindla Hultman Kramer, CEO, Daily
Voice AI & Voice Agents: An Illustrated PrimerTechnical guide, voiceaiandvoiceagents.com
Daily's CEO names GPT-4.1 and GPT-5.1 among the most used models for voice agents. He finds agents on the Realtime API cost 3 to 5 times more than GPT-4.1 agents and run slower than a tuned multi-model pipeline.
Positive
lmeyerov, Hacker News commenter
Comment on: GPT-5.4Hacker News comment
In the team's evals on cybersecurity investigations, GPT-5.2-Codex at low reasoning won: more than 2x faster with higher completion rates than higher reasoning settings.
Mixed
Zairah Mustahsan, Abel Lim, Megna Anand, Saahil Jain, and Bryan McCann, You.com
Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass CorrelationResearch paper, arXiv
On hard GAIA questions, 70% of GPT-4o's variance came from trial-to-trial randomness, making single runs unreliable; GPT-5 raised accuracy from 6.6% to 44.2% and nearly doubled consistency.
Positive
pietz, Hacker News commenter
Comment on: Claude Haiku 4.5Hacker News comment
Uses mini and nano models for every user-facing and workflow task at the company, and larger models only for internal high-complexity work such as coding.
Mixed
Amp team, Amp
GPT-5 OracleProduct engineering post, Amp Chronicle
After testing GPT-5 as a main model, Amp kept it as the oracle for planning and debugging, and left Sonnet as the main agent model because GPT-5 was less proactive.
Positive
BrandiATMuhkuh, Hacker News commenter
Comment on: ChatGPT's enterprise success against Copilot fuels OpenAI/Microsoft rivalryHacker News comment
Builds a co-pilot for architects on Azure OpenAI with retrieval and a profession-specific system prompt; a customer wants to roll it out across the company.
Positive
Dedy Kredo, Qodo
We benchmarked GPT-4.1 vs Claude 3.7: Here's what we foundEngineering blog post, Qodo blog
On 200 real pull requests judged by o3-mini, GPT-4.1's code suggestions beat Claude 3.7 Sonnet's in 54.9% of cases.
Mixed or negative, 4
Negative
carlos_ds, OpenAI Developer Forum
Please Don't Deprecate GPT-Realtime Before a True Replacement ExistsOpenAI Developer Forum post
Migrating a production voice agent to gpt-realtime-2.1-mini brought narrated instructions, skipped steps, and added latency; reverting to the deprecated model restored stability.
Mixed
Dobo Radichkov, OpenAI Developer Forum
Feedback: GPT-5 vs GPT-5-Chat-Latest for Conversational AIOpenAI Developer Forum post
For the Onsen app, gpt-5 suits Structured Outputs and gpt-5-chat-latest suits conversation, so the team now routes between them where GPT-4.1 handled both.
Negative
Prasanth Jayachandran, Isotopes AI
Beyond the leaderboard. Decision models in a real workflow.Engineering blog post, Isotopes AI blog
In Isotopes AI's own workflow suite, OpenAI's Decisions API passed 68.2% of attempts against 100% for Jev 1.13 and took about twice as long on the cases every model passed.
Negative
WebPlanning SIM-Ltd, OpenAI Developer Forum
gpt-realtime-2.1: production tool invocation regressionOpenAI Developer Forum post
In a controlled test, the older gpt-realtime completed 98% to 100% of commands and gpt-realtime-2.1 about 25%, often answering without calling a tool.
Counted for Teams and technical leaders.
Positive, 12
Positive
Cognition, the team behind Devin
GPT-6.1 Sol is now available in DevinCompany blog post, Devin blog
On Cognition's own FrontierCode benchmark GPT-6.1 Sol scored 60.4%, in line with GPT-6 Sol, while costing 44% to 57% less per task at every effort level.
Positive
dools, Hacker News commenter
Comment on: OpenAI Agents APIHacker News comment
Built an LLM gateway and agent session manager that colleagues use to ship software, and finds it cheap on API pricing with Luna or DeepSeek.
Positive
EB66, Hacker News commenter
Comment on: Gemini 3.7 FlashHacker News comment
GPT-5 mini led the company's own benchmarks on cost and quality until GPT-5.6 Luna's price cut replaced it.
Positive
__jl__, Hacker News commenter
Comment on: Advancing the price-performance frontier with GPT‑5.6Hacker News comment
For the team's production app, OpenAI is now the best provider: a reliable API with useful features and better price-performance than the open models it ran before.
Positive
madaxe_again, Hacker News commenter
Comment on: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaperHacker News comment
Assessed GPT-5.6 on the team's own workflows, moved most work from GPT-5.5 to Terra, halving cost and response times, with Luna as a cheap backstop.
Positive
dahcryn, Hacker News commenter
Comment on: Adaptive LLM routing under budget constraintsHacker News comment
Model calls are not the team's largest AI cost; GPT-4.1 is cheap enough that it skips mini models and optimizes speech-to-text spend instead.
Positive
lieret, Hacker News commenter
Comment on: GPT-5 on SWE-bench: Cost and performance deep-diveHacker News comment
On SWE-bench Verified GPT-5 scored 65%, GPT-5 mini 60%, and nano 35%; mini gave up about 5 points for roughly a fifth of the cost.
No opinion stated
Frank Hidalgo, CTO of StackAI, Asana
How we cut a browser agent's cost 76x and made it 5x faster by keeping its cache intactEngineering blog, Inside Asana
Asana cached its browser agent's history and stopped editing it on every call. On GPT-6.1 Sol the agent then cost 76x less than the original production setup, about US$0.47 per run, and read 89% of its input from cache.
"On GPT-6.1 Sol, the same agent cost 76x less and ran 5x faster, reading 89% of its input from cache."
Positive
Daniel Furman, Jacky Zhao, Vaibhav Kumar, Ed Sioufi, and Michele Catasta, Replit
Free the models: Harness design at the frontierEngineering blog post, Replit blog
Replit found GPT-6 Astra hands routine implementation to cheaper subagents without being told, and the GPT-6 family keeps the cache when effort changes mid-turn, which keeps Replit Agent's cost down.
Positive
Lorenzo Gentile, Ploy engineering blog
Migrating a production AI agent to GPT-5.6Engineering blog post, Ploy blog
After fixing its eval harness, tool schemas, and cache keys, Ploy moved its agent from Claude Opus 4.8 to GPT-5.6 Sol: builds 2.2x faster at 27% lower cost.
"As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, no model met our bar for replacing Claude Opus. GPT-5.6 Sol did. After a head-to-head evaluation, we made it the default model for every Ploy workspace."
Positive
pietz, Hacker News commenter
Comment on: Claude Haiku 4.5Hacker News comment
Uses mini and nano models for every user-facing and workflow task at the company, and larger models only for internal high-complexity work such as coding.
Positive
Kingston Duffie, OpenAI Developer Forum
My experience switching from Assistants API to Responses APIOpenAI Developer Forum post
Rewrote a production Assistants API chat app on the Responses API in about two hours; instructions and vector stores carried over, on gpt-4o-mini for cost.
Mixed or negative, 2
Mixed
anelson, Hacker News commenter
Comment on: GPT-6 Sol and LunaHacker News comment
On the team's internal evals GPT-6 Luna used more tokens and more time than GPT-5.6 Luna for the same result, which canceled its 50% lower price.
Negative
Stefan Ivanovici, Intercom
Cost of Serving LLMsEngineering research post, Fin AI research blog
Intercom calls the OpenAI and Anthropic models behind Fin expensive and slow, and its AWS benchmarks found self-hosted models under 14B parameters cheaper to serve than GPT-4.1 mini.
Counted for Teams and technical leaders.
Positive, 6
Positive
Artificial Analysis, independent model benchmarking service
GPT-5.6 Luna (Max) Intelligence, Performance & Price AnalysisIndependent benchmark, Artificial Analysis
Measures GPT-5.6 Luna (Max) at 127 output tokens per second, faster than the 116 average, and rates it among the leading models in intelligence at a reasonable price for its class.
Positive
__jl__, Hacker News commenter
Comment on: Advancing the price-performance frontier with GPT‑5.6Hacker News comment
For the team's production app, OpenAI is now the best provider: a reliable API with useful features and better price-performance than the open models it ran before.
Positive
madaxe_again, Hacker News commenter
Comment on: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaperHacker News comment
Assessed GPT-5.6 on the team's own workflows, moved most work from GPT-5.5 to Terra, halving cost and response times, with Luna as a cheap backstop.
Positive
Lorenzo Gentile, Ploy engineering blog
Migrating a production AI agent to GPT-5.6Engineering blog post, Ploy blog
After fixing its eval harness, tool schemas, and cache keys, Ploy moved its agent from Claude Opus 4.8 to GPT-5.6 Sol: builds 2.2x faster at 27% lower cost.
"As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, no model met our bar for replacing Claude Opus. GPT-5.6 Sol did. After a head-to-head evaluation, we made it the default model for every Ploy workspace."
Positive
lmeyerov, Hacker News commenter
Comment on: GPT-5.4Hacker News comment
In the team's evals on cybersecurity investigations, GPT-5.2-Codex at low reasoning won: more than 2x faster with higher completion rates than higher reasoning settings.
Positive
Amp team, Amp
GPT-5.4, The New OracleProduct engineering post, Amp Chronicle
In Amp's internal evals GPT-5.4 raised oracle response quality from 60.8% to 68.2% over GPT-5.2 and cut mean latency from about 6.7 to 4.9 minutes.
Mixed or negative, 2
Mixed
anelson, Hacker News commenter
Comment on: GPT-6 Sol and LunaHacker News comment
On the team's internal evals GPT-6 Luna used more tokens and more time than GPT-5.6 Luna for the same result, which canceled its 50% lower price.
Positive
Kingston Duffie, OpenAI Developer Forum
My experience switching from Assistants API to Responses APIOpenAI Developer Forum post
Rewrote a production Assistants API chat app on the Responses API in about two hours; instructions and vector stores carried over, on gpt-4o-mini for cost.
Counted for Teams and technical leaders.
Positive, 2
Positive
__jl__, Hacker News commenter
Comment on: Advancing the price-performance frontier with GPT‑5.6Hacker News comment
For the team's production app, OpenAI is now the best provider: a reliable API with useful features and better price-performance than the open models it ran before.
Positive
sigmoid10, Hacker News commenter
Comment on: GPT-5.2Hacker News comment
On OpenAI's highest enterprise tier, the commenter finds OpenAI a much more stable platform for high-volume API use than other providers.
Mixed or negative, 3
Negative
Foobar8568, Hacker News commenter
Comment on: OpenAI models coming to Amazon Bedrock: Interview with OpenAI and AWS CEOsHacker News comment
On Azure OpenAI deployments the team saw frequent latency spikes, up to three minutes to first byte, and escalation brought no explanation.
Negative
lastround360, OpenAI Developer Forum
Sudden increase in /v1/responses timeouts on gpt-4.1-mini (stable for months, started recently)OpenAI Developer Forum post
Responses API timeouts on gpt-4.1-mini, at a 45-second cutoff, rose from 1.28% to 13.64% of calls over four days in April 2026.
Negative
Jacob-J-Thomas, Hacker News commenter
Comment on: The Intelligence HubHacker News comment
An early enterprise chat app on Azure OpenAI hit frequent API failures and outages, and model and API changes were hard to keep up with.
Counted for Teams and technical leaders.
Positive, 1
Positive
Abiram, OpenAI Developer Forum
Decision vs execution: a pattern for using GPT safely inside RPA pipelinesOpenAI Developer Forum post
An enterprise finance team has the model return schema-checked JSON decisions while RPA bots execute them, which keeps actions auditable and retries safe.
Mixed or negative, 7
Negative
WebPlanning SIM-Ltd, OpenAI Developer Forum
gpt-realtime-2.1: production tool invocation regressionOpenAI Developer Forum post
In a controlled test, the older gpt-realtime completed 98% to 100% of commands and gpt-realtime-2.1 about 25%, often answering without calling a tool.
Positive
Lorenzo Gentile, Ploy engineering blog
Migrating a production AI agent to GPT-5.6Engineering blog post, Ploy blog
After fixing its eval harness, tool schemas, and cache keys, Ploy moved its agent from Claude Opus 4.8 to GPT-5.6 Sol: builds 2.2x faster at 27% lower cost.
"As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, no model met our bar for replacing Claude Opus. GPT-5.6 Sol did. After a head-to-head evaluation, we made it the default model for every Ploy workspace."
Negative
mparis, Hacker News commenter
Comment on: Structured outputs on the Claude Developer PlatformHacker News comment
Could not use OpenAI's structured outputs because its strict validation rejected standard JSON Schema elements the team used, which pushed it to Claude.
Mixed
hedgehog, Hacker News commenter
Comment on: Sampling and structured outputs in LLMsHacker News comment
Could not rely on OpenAI Structured Outputs in one component, and switched that component to TOML-formatted output that worked across many models.
Mixed
jbellis, Hacker News commenter at Brokk
Comment on: The unreasonable effectiveness of an LLM agent loop with tool useHacker News comment
At Brokk, some smart models such as o3 are poor at tool calling, so Gemini 2.5 Pro runs the Architect agent while GPT-4.1 and others work for search.
Mixed
Kingston Duffie, OpenAI Developer Forum
Problem with tool_choice causing caching issuesOpenAI Developer Forum post
Changing tool_choice on the final call invalidates the cached prefix and pays cache writes for the whole context; explicit caching makes the problem visible.
Negative
pimeys, Hacker News commenter
Comment on: GPT-5.6Hacker News comment
In the team's agent evals with prompts tuned per OpenAI's guide, GPT-5.6 Sol cost twice Opus 4.8 to reach the same conclusion, and Terra and Luna looped on failing tool calls.
Every concern practitioners raise, most raised first.
Counted for Teams and technical leaders.
Raised as a concern, 9
Mixed
Kwindla Hultman Kramer, CEO, Daily
Voice AI & Voice Agents: An Illustrated PrimerTechnical guide, voiceaiandvoiceagents.com
Daily's CEO names GPT-4.1 and GPT-5.1 among the most used models for voice agents. He finds agents on the Realtime API cost 3 to 5 times more than GPT-4.1 agents and run slower than a tuned multi-model pipeline.
Negative
Foobar8568, Hacker News commenter
Comment on: OpenAI models coming to Amazon Bedrock: Interview with OpenAI and AWS CEOsHacker News comment
On Azure OpenAI deployments the team saw frequent latency spikes, up to three minutes to first byte, and escalation brought no explanation.
Negative
Eduardo Stuani, OpenAI Developer Forum
Increased latency on GPT-4.1 Mini over the past few daysOpenAI Developer Forum post
Uses a large fixed, cached context to cut latency on 110K-token prompts; GPT-4.1 mini average latency rose from about 4 to 7 seconds, p90 from 7 to 12.
Negative
Prasanth Jayachandran, Isotopes AI
Beyond the leaderboard. Decision models in a real workflow.Engineering blog post, Isotopes AI blog
In Isotopes AI's own workflow suite, OpenAI's Decisions API passed 68.2% of attempts against 100% for Jev 1.13 and took about twice as long on the cases every model passed.
Mixed
anelson, Hacker News commenter
Comment on: GPT-6 Sol and LunaHacker News comment
On the team's internal evals GPT-6 Luna used more tokens and more time than GPT-5.6 Luna for the same result, which canceled its 50% lower price.
Negative
carlos_ds, OpenAI Developer Forum
Please Don't Deprecate GPT-Realtime Before a True Replacement ExistsOpenAI Developer Forum post
Migrating a production voice agent to gpt-realtime-2.1-mini brought narrated instructions, skipped steps, and added latency; reverting to the deprecated model restored stability.
Negative
dacox, Hacker News commenter
Comment on: Please don't discontinue Gemini 2.5 FlashHacker News comment
Fast, cheap models keep being deprecated by Google and OpenAI, and their replacements are reasoning models that are slower and much more expensive for the team's workloads.
Negative
lastround360, OpenAI Developer Forum
Sudden increase in /v1/responses timeouts on gpt-4.1-mini (stable for months, started recently)OpenAI Developer Forum post
Responses API timeouts on gpt-4.1-mini, at a 45-second cutoff, rose from 1.28% to 13.64% of calls over four days in April 2026.
Negative
Stefan Ivanovici, Intercom
Cost of Serving LLMsEngineering research post, Fin AI research blog
Intercom calls the OpenAI and Anthropic models behind Fin expensive and slow, and its AWS benchmarks found self-hosted models under 14B parameters cheaper to serve than GPT-4.1 mini.
Discussed without concern, 3
Positive
madaxe_again, Hacker News commenter
Comment on: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaperHacker News comment
Assessed GPT-5.6 on the team's own workflows, moved most work from GPT-5.5 to Terra, halving cost and response times, with Luna as a cheap backstop.
Positive
lmeyerov, Hacker News commenter
Comment on: GPT-5.4Hacker News comment
In the team's evals on cybersecurity investigations, GPT-5.2-Codex at low reasoning won: more than 2x faster with higher completion rates than higher reasoning settings.
Positive
Kingston Duffie, OpenAI Developer Forum
My experience switching from Assistants API to Responses APIOpenAI Developer Forum post
Rewrote a production Assistants API chat app on the Responses API in about two hours; instructions and vector stores carried over, on gpt-4o-mini for cost.
GILD's fact checks, which don't count as sources
Checked Oct 12, 2026.Fast mode, formerly Priority processing, runs up to 2.5x faster at a per-token premium. OpenAI's latency guide recommends generating fewer tokens and making fewer requests. Fast mode
Open them in the recordCounted for Teams and technical leaders.
Mixed
Kwindla Hultman Kramer, CEO, Daily
Voice AI & Voice Agents: An Illustrated PrimerTechnical guide, voiceaiandvoiceagents.com
Daily's CEO names GPT-4.1 and GPT-5.1 among the most used models for voice agents. He finds agents on the Realtime API cost 3 to 5 times more than GPT-4.1 agents and run slower than a tuned multi-model pipeline.
Negative
Stefan Ivanovici, Intercom
Cost of Serving LLMsEngineering research post, Fin AI research blog
Intercom calls the OpenAI and Anthropic models behind Fin expensive and slow, and its AWS benchmarks found self-hosted models under 14B parameters cheaper to serve than GPT-4.1 mini.
Mixed
anelson, Hacker News commenter
Comment on: GPT-6 Sol and LunaHacker News comment
On the team's internal evals GPT-6 Luna used more tokens and more time than GPT-5.6 Luna for the same result, which canceled its 50% lower price.
Mixed
pimeys, Hacker News commenter
Comment on: GPT-5.6 Sol Pricing Cut by 50% on OpenRouterHacker News comment
After evals, Kimi K3 now powers much of the team's systems in place of Opus and GPT at a fraction of the price; GPT-5.6 Sol's new pricing comes close.
Mixed
Kingston Duffie, OpenAI Developer Forum
Problem with tool_choice causing caching issuesOpenAI Developer Forum post
Changing tool_choice on the final call invalidates the cached prefix and pays cache writes for the whole context; explicit caching makes the problem visible.
Negative
pimeys, Hacker News commenter
Comment on: GPT-5.6Hacker News comment
In the team's agent evals with prompts tuned per OpenAI's guide, GPT-5.6 Sol cost twice Opus 4.8 to reach the same conclusion, and Terra and Luna looped on failing tool calls.
Negative
dacox, Hacker News commenter
Comment on: Please don't discontinue Gemini 2.5 FlashHacker News comment
Fast, cheap models keep being deprecated by Google and OpenAI, and their replacements are reasoning models that are slower and much more expensive for the team's workloads.
GILD's fact checks, which don't count as sources
Checked Oct 12, 2026.Organizations and projects have spend alerts and hard spend limits. Requests return 429 at a hard limit, though recorded spend can exceed it slightly before enforcement catches up. A surprise bill alone does not establish that spend controls were unavailable. Spend limits
Open them in the recordCounted for Teams and technical leaders.
Mixed
Zairah Mustahsan, Abel Lim, Megna Anand, Saahil Jain, and Bryan McCann, You.com
Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass CorrelationResearch paper, arXiv
On hard GAIA questions, 70% of GPT-4o's variance came from trial-to-trial randomness, making single runs unreliable; GPT-5 raised accuracy from 6.6% to 44.2% and nearly doubled consistency.
Negative
WebPlanning SIM-Ltd, OpenAI Developer Forum
gpt-realtime-2.1: production tool invocation regressionOpenAI Developer Forum post
In a controlled test, the older gpt-realtime completed 98% to 100% of commands and gpt-realtime-2.1 about 25%, often answering without calling a tool.
Negative
carlos_ds, OpenAI Developer Forum
Please Don't Deprecate GPT-Realtime Before a True Replacement ExistsOpenAI Developer Forum post
Migrating a production voice agent to gpt-realtime-2.1-mini brought narrated instructions, skipped steps, and added latency; reverting to the deprecated model restored stability.
Positive
madaxe_again, Hacker News commenter
Comment on: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaperHacker News comment
Assessed GPT-5.6 on the team's own workflows, moved most work from GPT-5.5 to Terra, halving cost and response times, with Luna as a cheap backstop.
Negative
pimeys, Hacker News commenter
Comment on: GPT-5.6Hacker News comment
In the team's agent evals with prompts tuned per OpenAI's guide, GPT-5.6 Sol cost twice Opus 4.8 to reach the same conclusion, and Terra and Luna looped on failing tool calls.
Positive
Lorenzo Gentile, Ploy engineering blog
Migrating a production AI agent to GPT-5.6Engineering blog post, Ploy blog
After fixing its eval harness, tool schemas, and cache keys, Ploy moved its agent from Claude Opus 4.8 to GPT-5.6 Sol: builds 2.2x faster at 27% lower cost.
"As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, no model met our bar for replacing Claude Opus. GPT-5.6 Sol did. After a head-to-head evaluation, we made it the default model for every Ploy workspace."
Mixed
Dobo Radichkov, OpenAI Developer Forum
Feedback: GPT-5 vs GPT-5-Chat-Latest for Conversational AIOpenAI Developer Forum post
For the Onsen app, gpt-5 suits Structured Outputs and gpt-5-chat-latest suits conversation, so the team now routes between them where GPT-4.1 handled both.
Counted for Teams and technical leaders.
Raised as a concern, 7
Negative
WebPlanning SIM-Ltd, OpenAI Developer Forum
gpt-realtime-2.1: production tool invocation regressionOpenAI Developer Forum post
In a controlled test, the older gpt-realtime completed 98% to 100% of commands and gpt-realtime-2.1 about 25%, often answering without calling a tool.
Negative
carlos_ds, OpenAI Developer Forum
Please Don't Deprecate GPT-Realtime Before a True Replacement ExistsOpenAI Developer Forum post
Migrating a production voice agent to gpt-realtime-2.1-mini brought narrated instructions, skipped steps, and added latency; reverting to the deprecated model restored stability.
Negative
dacox, Hacker News commenter
Comment on: Please don't discontinue Gemini 2.5 FlashHacker News comment
Fast, cheap models keep being deprecated by Google and OpenAI, and their replacements are reasoning models that are slower and much more expensive for the team's workloads.
Mixed
jasongill, Hacker News commenter
Comment on: New tools and features in the Responses APIHacker News comment
Found the Assistants API the best fit before the Responses API and wanted clearer timelines for its sunset and migration.
Negative
mritchie712, Hacker News commenter at Definite
Comment on: New tools and features in the Responses APIHacker News comment
Definite ran its agent on the Assistants API, expected it to die for lack of updates, and switched to the model-agnostic Pydantic AI framework.
Negative
daviding, Hacker News commenter
Comment on: New tools for building agentsHacker News comment
Porting to the Responses API is non-trivial for a team that already built history and retrieval on Chat Completions, and web search pushed it there.
Negative
Jacob-J-Thomas, Hacker News commenter
Comment on: The Intelligence HubHacker News comment
An early enterprise chat app on Azure OpenAI hit frequent API failures and outages, and model and API changes were hard to keep up with.
Discussed without concern, 1
Positive
Kingston Duffie, OpenAI Developer Forum
My experience switching from Assistants API to Responses APIOpenAI Developer Forum post
Rewrote a production Assistants API chat app on the Responses API in about two hours; instructions and vector stores carried over, on gpt-4o-mini for cost.
GILD's fact checks, which don't count as sources
Checked Oct 12, 2026.OpenAI gives generally available models at least 6 months' notice before retirement. Specialized variants get at least 3 months. Early-release models get as little as 2 weeks. The Assistants API shut down on August 26, 2026, and the Evals platform shuts down on November 30, 2026. Deprecations
Open them in the recordCounted for Teams and technical leaders.
Mixed
Ravi Raju and four coauthors
The Limits of Long-Context Reasoning in Automated Bug FixingResearch paper, arXiv (ICLR 2026 ICBINB workshop)
In an agent harness GPT-5-nano resolved up to 31% of 100 SWE-bench Verified tasks, but given the relevant files in a 64K-token context it solved none.
Mixed
Nima Esmi and coauthors, University of Groningen, University of Guilan, and TU Delft
GPT-5 vs Other LLMs in Long Short-Context PerformanceResearch paper, arXiv (FLLM 2025, IEEE)
Past 5,000 posts (about 70,000 tokens) accuracy fell for every model, to about 50 to 53% at 20,000 posts, while GPT-5 kept precision near 95% and lost-in-the-middle errors had largely disappeared.
Mixed
Daniel Chalef, founder and CEO of Zep, with Preston Rasmussen, Zep
GPT-4.1 and o4-mini: Is OpenAI Overselling Long-Context?Engineering blog post, Zep blog
On LongMemEval conversations of about 115,000 tokens, GPT-4.1 averaged 56.72%, below GPT-4o-mini's 57.87%, while o4-mini led at 72.78%; Zep concludes a 1M-token window alone does not improve recall.
Mixed
Ali Modarressi and six coauthors
NoLiMa: Long-Context Evaluation Beyond Literal MatchingResearch paper, arXiv (ICML 2025)
With no literal matches to lean on, 11 of 13 models fell below half their short-context scores at 32K tokens, and GPT-4o, one of the strongest, dropped from 99.3% to 69.7%.
Negative
carlos_ds, OpenAI Developer Forum
Please Don't Deprecate GPT-Realtime Before a True Replacement ExistsOpenAI Developer Forum post
Migrating a production voice agent to gpt-realtime-2.1-mini brought narrated instructions, skipped steps, and added latency; reverting to the deprecated model restored stability.
Mixed
Stefano Rando and seven coauthors
LongCodeBench: Evaluating Coding LLMs at 1M Context WindowsResearch paper, arXiv
At contexts up to 1M tokens, GPT-4.1 led the code question-answering task and held steady across lengths, but on bug fixing its accuracy fell from a peak of 11% to 2%.
Counted for Teams and technical leaders.
Raised as a concern, 5
Negative
elpakal, Hacker News commenter
Comment on: Decisions API is in public betaHacker News comment
The team just moved from OpenAI to Anthropic models on Amazon Bedrock to get a zero data retention guarantee.
Negative
Shane Prebenda, OpenAI Developer Forum
Zero Data Retention (ZDR) request: enterprise customer, no response after multiple submissionsOpenAI Developer Forum post
Client contracts require Zero Data Retention before production traffic; four business days after the sales form, no reply had come, blocking the launch.
Negative
Michal Stoklasa, OpenAI Developer Forum
How to become eligible for data residency endpoints?OpenAI Developer Forum post
Runs on Azure OpenAI for EU hosting but wants OpenAI's own EU data residency endpoint, since Azure lacks priority processing and vector stores and is slower; sales never answered.
Negative
Herman, OpenAI Developer Forum
NYT lawsuit and 30 days data retentionOpenAI Developer Forum post
The court-ordered retention left a Swedish company's customers out of compliance; moving to Azure OpenAI was its only option, at the cost of features such as web search.
Negative
molf, Hacker News commenter
Comment on: How we're responding to The NYT's data demands in order to protect user privacyHacker News comment
Applied for Zero Data Retention several times with no response, and says OpenAI mentions the option without approving requests in practice.
Discussed without concern, 2
Positive
alexsouthmayd, Hacker News commenter
Comment on: Launch HN: Bloomy (YC S26) - AI-powered mastery learning for K-12Hacker News comment
Bloomy uses OpenAI and Anthropic models under Zero Data Retention agreements with both, which school districts reviewed in technical due diligence.
Positive
mahmoud-almadi, Hacker News commenter at Cyberdesk
Comment on: Launch HN: Cyberdesk (YC S25) - Automate Windows legacy desktop appsHacker News comment
Cyberdesk runs healthcare automation under signed zero data retention agreements with Anthropic and OpenAI and BAAs with its customers.
GILD's fact checks, which don't count as sources
Checked Oct 12, 2026.OpenAI keeps abuse monitoring logs for up to 30 days by default. Zero Data Retention and Modified Abuse Monitoring require prior approval. Batch, Files, vector stores, and the Agents API can retain application state even with Zero Data Retention on. Data controls in the OpenAI platform
Open them in the recordCounted for Teams and technical leaders.
Raised as a concern, 4
Negative
Sai Reddy, OpenAI Developer Forum
Prompt Caching Is a Core GPT-5.6 Feature. Why Are Customers Still Reverse-Engineering It?OpenAI Developer Forum post
Measured about 96% cache hits on Azure Chat Completions and none on the Azure Responses API for GPT-5.6 Sol, and got no explanation from OpenAI or Microsoft.
Positive
Lorenzo Gentile, Ploy engineering blog
Migrating a production AI agent to GPT-5.6Engineering blog post, Ploy blog
After fixing its eval harness, tool schemas, and cache keys, Ploy moved its agent from Claude Opus 4.8 to GPT-5.6 Sol: builds 2.2x faster at 27% lower cost.
"As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, no model met our bar for replacing Claude Opus. GPT-5.6 Sol did. After a head-to-head evaluation, we made it the default model for every Ploy workspace."
Negative
Jacob-J-Thomas, Hacker News commenter
Comment on: The Intelligence HubHacker News comment
An early enterprise chat app on Azure OpenAI hit frequent API failures and outages, and model and API changes were hard to keep up with.
Negative
daviding, Hacker News commenter
Comment on: New tools for building agentsHacker News comment
Porting to the Responses API is non-trivial for a team that already built history and retrieval on Chat Completions, and web search pushed it there.
Discussed without concern, 1
Positive
Kingston Duffie, OpenAI Developer Forum
My experience switching from Assistants API to Responses APIOpenAI Developer Forum post
Rewrote a production Assistants API chat app on the Responses API in about two hours; instructions and vector stores carried over, on gpt-4o-mini for cost.
Counted for Teams and technical leaders.
Raised as a concern, 4
Negative
7thpower, Hacker News commenter
Comment on: OpenAI models coming to Amazon Bedrock: Interview with OpenAI and AWS CEOsHacker News comment
When the team used Azure OpenAI two to three years ago, throughput quotas were small and increases bureaucratic; it moved to Claude once Claude 3 shipped.
Negative
lastround360, OpenAI Developer Forum
Sudden increase in /v1/responses timeouts on gpt-4.1-mini (stable for months, started recently)OpenAI Developer Forum post
Responses API timeouts on gpt-4.1-mini, at a 45-second cutoff, rose from 1.28% to 13.64% of calls over four days in April 2026.
Negative
Jacob-J-Thomas, Hacker News commenter
Comment on: The Intelligence HubHacker News comment
An early enterprise chat app on Azure OpenAI hit frequent API failures and outages, and model and API changes were hard to keep up with.
Negative
Foobar8568, Hacker News commenter
Comment on: OpenAI models coming to Amazon Bedrock: Interview with OpenAI and AWS CEOsHacker News comment
On Azure OpenAI deployments the team saw frequent latency spikes, up to three minutes to first byte, and escalation brought no explanation.
Discussed without concern, 4
Positive
sigmoid10, Hacker News commenter
Comment on: GPT-5.2Hacker News comment
On OpenAI's highest enterprise tier, the commenter finds OpenAI a much more stable platform for high-volume API use than other providers.
Mixed
Siddharth Kodwani, Software Engineer, GenAI Platform, DoorDash
Building a GenAI Platform at DoorDashConference talk, InfoQ
DoorDash calls OpenAI's endpoint with Azure as the fallback, and the Claude API with Bedrock as its fallback, so product services stay up when one provider fails. Once finance asked who was spending what on shared keys, the gateway added workspaces that attribute usage costs to each department.
"We actually had to go understand what the fallback should be, what kind of routing should be. At one point, I remember we had OpenAI endpoint, but we also had a fallback on Azure endpoint. Or, we had Claude model API endpoint, then we had a fallback on Bedrock, so that the services can rely on us from the reliability point of view. We went in that direction. We quickly realized that as the usage is going up, then we need a way to track, because now finance got involved. Like, you guys have one API key for multiple teams, how do I track who is using what? We quickly realized we need workspaces where we have cost attribution. We can actually drill down on who is using how much. We can attribute cost back to those departments. That's what basically we did with LLM Gateway."
Positive
eis, Hacker News commenter
Comment on: Claude outage - ResolvedHacker News comment
Keeps Claude out of the product for cost and reliability, and runs OpenAI with a Gemini fallback, or the reverse depending on the use case.
Positive
__jl__, Hacker News commenter
Comment on: Advancing the price-performance frontier with GPT‑5.6Hacker News comment
For the team's production app, OpenAI is now the best provider: a reliable API with useful features and better price-performance than the open models it ran before.
GILD's fact checks, which don't count as sources
Checked Oct 12, 2026.Usage tiers upgrade automatically with credit purchases: Build at US$5, Launch at US$100, and Grow at US$500. Monthly usage limits are US$500, US$5,000, and US$200,000. Fast traffic ramps return 429 slow_down; Scale Tier and Reserved Tier offer committed capacity. Rate limits
Open them in the recordStatements this group's sources agree on, then the ones close to agreement.
Teams run their own evaluation set before adopting or switching models, since benchmarks and list prices do not predict results on their task.
15 supporting sourcesClose to agreement: each needs 15 supporting sources
Teams building on OpenAI keep another model provider available for outages or lower prices.
13 supporting sourcesTeams use OpenAI models through the API as the model behind their own agents, with tools and multi-step loops.
11 supporting sourcesPrompt caching pays only when prompts are built for it: stable content first, changing content last, and hit rates measured.
7 supporting sourcesTeams send scoped, high-volume tasks to smaller models and hard reasoning tasks to larger ones.
6 supporting sourcesQuestions the sources split on, with the count on each side.
Evidence remains mixed
OpenAI's API, 5 sources
Some sign data terms with OpenAI directly and call its API, where new models and features arrive first.
Azure or Bedrock, 7 sources
Others buy through Azure OpenAI or Amazon Bedrock for existing cloud contracts and data terms, and accept that platform's quotas and regions.
Evidence remains mixed
Standardize on OpenAI, 2 sources
Some standardize on OpenAI for reliability and the usage tier they have already reached.
Spread across providers, 8 sources
Others route between OpenAI and other providers, or move workloads to cheaper models, after their own evaluations.
The three groups, then every practitioner by the context they work in, and how emphasis changes with scale.
Three groups drawn from the same sources. Choose a group to select whose experience counts. Within that group, each author counts once in the practitioner sentiment, with their latest view. Signals open on Teams and technical leaders.
75 sources. These accounts focus on reliability, data terms, model and API changes, and where teams buy OpenAI's models, from the accounts of teams and companies running them in their products.
61 sources. These accounts focus on cost, caching, tool calling, and day-to-day reliability for developers building on the API on their own.
136 sources. This gives the broadest read, but combines solo projects with production teams that can reach different conclusions.
136 sources from practitioners, by the context they work in. Filled rows count for this group.
Emphasizes Cost efficiency and token cost
Emphasizes Latency and data retention requests
Emphasizes Tool calling, evaluations before a model switch, and company gateways that run OpenAI beside other providers
Emphasizes Realtime API cost and latency for production voice agents
What changes with scale. Individual builders focus on cost efficiency and small models. Team accounts focus more on latency, token cost and data retention terms. The largest teams on record, DoorDash, Grab and Wealthsimple, reach OpenAI through their own gateways beside other providers.
Where this group's sources were published.
Counted for Teams and technical leaders, from 75 sources. Change group
93 sources for this group, 18 of them documentation
Older evidence stays when it still applies. Accounts tied to retired models, renamed tiers, the Assistants API, or changed features remain in the history with a Needs a recheck label.
See sources by originWho each source is, how it knows, whether it still applies and what it counts toward.
Newest first, on its own page. Every count above opens its sources there.
Read all 154 sourcesAbout this read
A running review of what teams building AI features on OpenAI API report, from individual builders to technical leaders. Each finding links to its sources, and each conflict shows both sides.
Counts cover only the sources GILD reviewed.
Choose a group to select whose experience counts. Within that group, each author counts once in the practitioner sentiment, with their latest view. GILD files each source in a group from how the account describes its author. A role or employer marked self-described is what the source states about themselves. Documentation checks facts for every group and never counts toward sentiment.
A source's date and whether it still applies are separate: a recent post can describe an old version, and an old one can still hold.
Flagged for a recheck, of the sources that take a side: teams and leaders 17 of 65, individual builders 14 of 47, all practitioners 31 of 112.
Each row opens its sources for the group you are reading.
| Finding | Teams and technical leaders | Individual builders | All practitioners | Sources |
|---|---|---|---|---|
| Current readCurrent sentiment | 22 positive, 23 mixed, 20 negative | 19 positive, 19 mixed, 9 negative | 41 positive, 42 mixed, 29 negative | Sources |
| PraiseModel Quality | 14 of 18 positive | 2 of 5 positive | 16 of 23 positive | Sources |
| PraiseTool Calling & Structured Output | 1 of 8 positive | 3 of 8 positive | 4 of 16 positive | Sources |
| PraiseCost Efficiency | 12 of 14 positive | 9 of 11 positive | 21 of 25 positive | Sources |
| PraiseDeveloper Experience | 3 of 4 positive (under the minimum) | 4 of 5 positive | 7 of 9 positive | Sources |
| PraiseStability | 2 of 5 positive | 2 of 2 positive (under the minimum) | 4 of 7 positive | Sources |
| PraiseSpeed | 6 of 8 positive | 3 of 9 positive | 9 of 17 positive | Sources |
| PraiseLong Context | 0 of 5 positive | 1 of 1 positive (under the minimum) | 1 of 6 positive | Sources |
| ConcernModel & API Changes | 7 of 8 raise it | 5 of 8 raise it | 12 of 16 raise it | Sources |
| ConcernRate Limits & Reliability | 4 of 8 raise it | 3 of 7 raise it | 7 of 15 raise it | Sources |
| ConcernToken Cost | 7 of 7 raise it | 4 of 4 raise it (under the minimum) | 11 of 11 raise it | Sources |
| ConcernLatency | 9 of 12 raise it | 6 of 10 raise it | 15 of 22 raise it | Sources |
| ConcernData Handling | 5 of 7 raise it | 1 of 3 raise it (under the minimum) | 6 of 10 raise it | Sources |
| ConcernFeature Gaps | 4 of 4 raise it (under the minimum) | 0 of 0 raise it (under the minimum) | 4 of 4 raise it (under the minimum) | Sources |
| ConcernInconsistent Results | 7 of 7 raise it | 4 of 4 raise it (under the minimum) | 11 of 11 raise it | Sources |
| ConcernSetup & SDK Friction | 4 of 5 raise it | 3 of 6 raise it | 7 of 11 raise it | Sources |
| ConcernLong-Context Limits | 6 of 6 raise it | 0 of 0 raise it (under the minimum) | 6 of 6 raise it | Sources |
| ConsensusTeams keep a second provider | 13 supporting (below threshold) | 6 supporting (below threshold) | 19 supporting (below threshold) | Sources |
| ConsensusOpenAI models run their own agents | 11 supporting (below threshold) | 9 supporting (below threshold) | 20 supporting (below threshold) | Sources |
| ConsensusEvaluate before switching | 15 supporting | 7 supporting (below threshold) | 22 supporting (below threshold) | Sources |
| ConsensusRoute by task | 6 supporting (below threshold) | 2 supporting (under the minimum) | 8 supporting (below threshold) | Sources |
| ConsensusCaching is designed in | 7 supporting (below threshold) | 1 supporting (under the minimum) | 8 supporting (below threshold) | Sources |
| DisagreementShould a team call OpenAI's API directly or buy OpenAI models through its cloud provider? | OpenAI's API 5, Azure or Bedrock 7 | OpenAI's API 2, Azure or Bedrock 3 | OpenAI's API 7, Azure or Bedrock 10 | Sources |
| DisagreementShould a team move to the Responses API or stay on Chat Completions? | Responses API 3, Chat Completions 1 (under the minimum) | Responses API 3, Chat Completions 1 (under the minimum) | Responses API 6, Chat Completions 2 | Sources |
| DisagreementShould a team standardize on OpenAI or spread work across providers? | Standardize on OpenAI 2, Spread across providers 8 | Standardize on OpenAI 1, Spread across providers 2 (under the minimum) | Standardize on OpenAI 3, Spread across providers 10 | Sources |
| Current readEvidence mix | 93 sources | 79 sources | 154 sources | Sources |
GILD's checks of claims against official documentation, kept apart from the source counts.
