Every source behind the read

93 sources in Teams and technical leaders, with product documentation Showing all 154 sources

1. No opinion stated

   __jl__, Hacker News commenter

   Hacker News comment on GPT-6.1 Terra for agentic productionHacker News comment

   The author's application runs almost entirely on the OpenAI API apart from fallback models, and calls GPT-6.1 Terra a strong release for cost-sensitive agentic production work.

   "Internal evals are still running but GPT-6.1 Terra is an AWESOME release for anyone using the API in agentic production applications where cost matters. Personally, I pretty much exclusively use claude for coding work (sorry!). Our application is running almost 100% on OpenAI API (except for some fallback models). It is the best API on the market. The models are excellent particularly for any application that is cost sensitive."

   Published Oct 26, 2026self-described: An application running almost entirely on the OpenAI API, with fallback models

   Not counted

   Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   GPT-6.1 Terra

2. No opinion stated

   Andrew Hickl, CTO, Allen Institute (X profile)

   X post on OpenAI's 12-hour prompt cacheX post

   The 12-hour cache costs 2.5x the input price to write, so for agents that wake a few times a day on the same context the saving depends on the hit rate.

   Published Oct 26, 2026self-described: Agents that wake a few times a day on the same context

   Not counted

   Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

   Published on

   Developer Communities

   Who

   Technical leadership

   Evidence

   Reported

   Applies to

   Current

   Observed on

   OpenAI API 12-hour prompt cache

3. No opinion stated

   Swimming-Skirt3870, Reddit (r/OpenAI)

   Why are some GPT 5 API requests costing three times more than others?Reddit post, r/OpenAI

   A team processing documents on the GPT-5 API sees some requests cost two or three times others of similar length, and cannot tell how much of the gap is reasoning tokens.

   "Why are some GPT 5 API requests costing three times more than others? We've been using the GPT 5 API for document processing and noticed that some requests are costing two or three times more than others even when the documents are roughly the same length. At first I assumed it was just the input size but after looking at the usage numbers, that doesn't seem to explain everything cause some requests use significantly more output tokens and I'm wondering how much of that is coming from reasoning versus the actual response. It's getting harder to estimate what processing a batch of documents will cost especially when the requests aren't all behaving the same way. For anyone running GPT 5 in production, how are you figuring out what's causing these differences like are you tracking reasoning tokens separately or comparing the total cost of each request?"

   Published Oct 24, 2026self-described: Document processing on the GPT-5 API

   Not counted

   Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   GPT-5 API

4. No opinion stated

   apricot_dragon, Reddit (r/OpenAI)

   Is anyone building on the Agents API yet?Reddit post, r/OpenAI

   The author's team moved its agent backend onto the Agents API and dropped its own SDK, betting that the API's shape becomes a cross-provider standard the way Chat Completions did.

   "Is anyone building on the Agents API yet? We moved our agent backend onto the Agents API a few weeks ago, and I'm curious who else has. **Why we bet on it:** we watched Chat Completions go from one company's endpoint in 2023 to the shape almost every provider speaks now. OpenRouter, Groq, Ollama, vLLM, even Google and Anthropic for their own models. Nobody voted, developers just had code written against it and everyone followed. Agents feel like they're at the same point, and the Agents API already has the pieces a real agent needs: agents, sessions, turns, events, files, skills. **How we use it:** we dropped our own SDK and use the official openai package. Our code talks to agents and sessions in the API's shape, so changing where they run is a base URL change, not a rewrite. That alone made it worth it for us. It's still in beta, so I'm curious: * Are you using it in production or still testing? * What are you building with it? * Anything missing that's kept you on Chat Completions or the Responses API?"

   Published Oct 20, 2026self-described: An agent backend moved onto OpenAI's Agents API (beta)

   Not counted

   Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   Agents API (beta) through the official openai package

5. Negative

   Prasanth Jayachandran, Isotopes AI

   Beyond the leaderboard. Decision models in a real workflow.Engineering blog post, Isotopes AI blog

   In Isotopes AI's own workflow suite, OpenAI's Decisions API passed 68.2% of attempts against 100% for Jev 1.13 and took about twice as long on the cases every model passed.

   Published Oct 15, 2026self-described: Isotopes AI aidnn follow-up workflow, 88 scenarios run three times per model under zero data retention

   Counts toward

   • Overall signal
   • Model Quality: Mixed or negative
   • Latency: Raised as a concern
   • Evaluate before switching
   • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: OpenAI's API

   Published on

   Engineering Blogs & Reviews

   Who

   Product or platform team

   Evidence

   Measured

   Applies to

   Current

   Observed on

   OpenAI Decisions API on gpt-6-luna

6. No opinion stated

   Frank Hidalgo, CTO of StackAI, Asana

   How we cut a browser agent's cost 76x and made it 5x faster by keeping its cache intactEngineering blog, Inside Asana

   Asana cached its browser agent's history and stopped editing it on every call. On GPT-6.1 Sol the agent then cost 76x less than the original production setup, about US$0.47 per run, and read 89% of its input from cache.

   "On GPT-6.1 Sol, the same agent cost 76x less and ran 5x faster, reading 89% of its input from cache."

   Published Oct 8, 2026self-described: Asana's StackAI browser agent, tested across four frontier models in a 144-run caching study

   Counts toward

   • Cost Efficiency: Positive
   • Evaluate before switching
   • Caching is designed in

   Published on

   Engineering Blogs & Reviews

   Who

   Product or platform team

   Evidence

   Measured

   Applies to

   Current

   Observed on

   GPT-6.1 Sol through the OpenAI API, with a cache marker on the latest tool result

7. Positive

   Abiram, OpenAI Developer Forum

   Decision vs execution: a pattern for using GPT safely inside RPA pipelinesOpenAI Developer Forum post

   An enterprise finance team has the model return schema-checked JSON decisions while RPA bots execute them, which keeps actions auditable and retries safe.

   Published Oct 7, 2026self-described: Enterprise finance team adding LLM steps to UiPath and Power Automate bots

   Counts toward

   • Overall signal
   • Tool Calling & Structured Output: Positive

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

8. Negative

   elpakal, Hacker News commenter

   Comment on: Decisions API is in public betaHacker News comment

   The team just moved from OpenAI to Anthropic models on Amazon Bedrock to get a zero data retention guarantee.

   Published Oct 7, 2026self-described: Team on Amazon Bedrock

   Counts toward

   • Overall signal
   • Data Handling: Raised as a concern
   • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

   Published on

   Developer Communities

   Who

   Small product team

   Evidence

   Reported

   Applies to

   Current

9. No opinion stated

   Immediate_Menu_3695, Reddit (r/OpenAI)

   Long Responses API runs are costing us more from context than the output itselfReddit post, r/OpenAI

   On long tool-using runs a big portion of the tokens comes from context carried forward, so two successful runs of the same workflow cost very different amounts; the team weighs pruning old tool output against retries.

   Published Oct 7, 2026self-described: A multi-step, tool-using workflow on the Responses API

   Not counted

   Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   Responses API, long tool-using workflows

10. Mixed

    Siddharth Kodwani, Software Engineer, GenAI Platform, DoorDash

    Building a GenAI Platform at DoorDashConference talk, InfoQ

    DoorDash calls OpenAI's endpoint with Azure as the fallback, and the Claude API with Bedrock as its fallback, so product services stay up when one provider fails. Once finance asked who was spending what on shared keys, the gateway added workspaces that attribute usage costs to each department.

    "We actually had to go understand what the fallback should be, what kind of routing should be. At one point, I remember we had OpenAI endpoint, but we also had a fallback on Azure endpoint. Or, we had Claude model API endpoint, then we had a fallback on Bedrock, so that the services can rely on us from the reliability point of view. We went in that direction. We quickly realized that as the usage is going up, then we need a way to track, because now finance got involved. Like, you guys have one API key for multiple teams, how do I track who is using what? We quickly realized we need workspaces where we have cost attribution. We can actually drill down on who is using how much. We can attribute cost back to those departments. That's what basically we did with LLM Gateway."

    Published Oct 3, 2026self-described: DoorDash's LLM Gateway, which serves OpenAI, Anthropic and Gemini models to product teams across the company

    Counts toward

    • Overall signal
    • Rate Limits & Reliability: Discussed without concern
    • Teams keep a second provider
    • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

    Published on

    Podcasts, Talks & Videos

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API with an Azure OpenAI fallback, beside the Claude API with an Amazon Bedrock fallback

11. No opinion stated

    No_Refrigerator_8216, Reddit (r/OpenAI)

    Does anyone separate OpenAI Batch spend from normal API spend internally?Reddit post, r/OpenAI

    Batch jobs such as eval runs and new datasets add spend unrelated to product use, so the team is considering tracking and budgeting Batch apart from live API traffic.

    Published Oct 2, 2026self-described: A team running both live API traffic and Batch jobs

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Batch API beside synchronous API traffic

12. Positive

    Daniel Furman, Jacky Zhao, Vaibhav Kumar, Ed Sioufi, and Michele Catasta, Replit

    Free the models: Harness design at the frontierEngineering blog post, Replit blog

    Replit found GPT-6 Astra hands routine implementation to cheaper subagents without being told, and the GPT-6 family keeps the cache when effort changes mid-turn, which keeps Replit Agent's cost down.

    Published Sep 29, 2026self-described: Replit Agent in Max mode with GPT-6 Astra as the core loop

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Cost Efficiency: Positive
    • OpenAI models run their own agents
    • Caching is designed in

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-6 Astra

13. Positive

    Cognition, the team behind Devin

    GPT-6.1 Sol is now available in DevinCompany blog post, Devin blog

    On Cognition's own FrontierCode benchmark GPT-6.1 Sol scored 60.4%, in line with GPT-6 Sol, while costing 44% to 57% less per task at every effort level.

    Published Sep 29, 2026self-described: Devin, models graded on Cognition's FrontierCode benchmark of real engineering tasks

    Counts toward

    • Overall signal
    • Cost Efficiency: Positive
    • Evaluate before switching

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-6.1 Sol against GPT-6 Sol and GPT-5.6 Sol

14. Mixed

    Jeffrey Rengifo and Gustavo Llermaly, Elastic

    The best LLM writes correct Elasticsearch ES|QL 59% of the time. Here's what breaks the other 41%.Engineering blog post, Elasticsearch Labs

    Elastic's test of 6,000 generated ES|QL queries found gpt-5.5 the most accurate of four models at 59.0% with only the schema, while gpt-5.4-mini scored 19.8%.

    Published Sep 28, 2026self-described: Elasticsearch Labs test of 500 BIRD questions per model for generated ES|QL queries

    Counts toward

    • Overall signal
    • Model Quality: Positive

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    gpt-5.5 and gpt-5.4-mini

15. Mixed

    anelson, Hacker News commenter

    Comment on: GPT-6 Sol and LunaHacker News comment

    On the team's internal evals GPT-6 Luna used more tokens and more time than GPT-5.6 Luna for the same result, which canceled its 50% lower price.

    Published Sep 26, 2026self-described: Cybersecurity application with internal evals

    Counts toward

    • Overall signal
    • Cost Efficiency: Mixed or negative
    • Speed: Mixed or negative
    • Token Cost: Raised as a concern
    • Latency: Raised as a concern
    • Evaluate before switching

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-6 Luna against GPT-5.6 Luna

16. Negative

    WebPlanning SIM-Ltd, OpenAI Developer Forum

    gpt-realtime-2.1: production tool invocation regressionOpenAI Developer Forum post

    In a controlled test, the older gpt-realtime completed 98% to 100% of commands and gpt-realtime-2.1 about 25%, often answering without calling a tool.

    Published Sep 17, 2026self-described: WebPlanning SIM-Ltd, production voice agent

    Counts toward

    • Overall signal
    • Model Quality: Mixed or negative
    • Tool Calling & Structured Output: Mixed or negative
    • Model & API Changes: Raised as a concern
    • Inconsistent Results: Raised as a concern

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    gpt-realtime-2025-08-28 against gpt-realtime-2.1

17. No opinion stated

    authentic_developer, Reddit (r/OpenAI)

    Shipped a voice agent on the Realtime API: 7 behavioral bugs from production call logsReddit post, r/OpenAI

    Reading live call transcripts surfaced seven bugs scripted tests missed, such as the agent repeating its own instructions and not ending a call until the fourth request.

    Published Sep 14, 2026self-described: A business voice agent on the Realtime API, in production for a few months

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Realtime API

18. Positive

    dools, Hacker News commenter

    Comment on: OpenAI Agents APIHacker News comment

    Built an LLM gateway and agent session manager that colleagues use to ship software, and finds it cheap on API pricing with Luna or DeepSeek.

    Published Sep 11, 2026self-described: Internal agent app used by colleagues

    Counts toward

    • Overall signal
    • Cost Efficiency: Positive
    • Teams keep a second provider
    • OpenAI models run their own agents

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-6 Luna

19. Positive

    everlier, Hacker News commenter

    Comment on: OpenAI Agents APIHacker News comment

    The commenter's company scripts persistent agent workflows on the existing Responses and Chat Completions APIs with a simple SDK of its own.

    Published Sep 10, 2026self-described: Company workflows on the API

    Counts toward

    • Overall signal
    • OpenAI models run their own agents

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

20. Positive

    eis, Hacker News commenter

    Comment on: Claude outage - ResolvedHacker News comment

    Keeps Claude out of the product for cost and reliability, and runs OpenAI with a Gemini fallback, or the reverse depending on the use case.

    Published Sep 3, 2026self-described: Team product with a fallback provider

    Counts toward

    • Overall signal
    • Rate Limits & Reliability: Discussed without concern
    • Teams keep a second provider
    • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

21. Mixed

    pimeys, Hacker News commenter

    Comment on: GPT-5.6 Sol Pricing Cut by 50% on OpenRouterHacker News comment

    After evals, Kimi K3 now powers much of the team's systems in place of Opus and GPT at a fraction of the price; GPT-5.6 Sol's new pricing comes close.

    Published Aug 18, 2026self-described: Agent company's production systems

    Counts toward

    • Overall signal
    • Token Cost: Raised as a concern
    • Teams keep a second provider
    • Evaluate before switching
    • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

22. No opinion stated

    AccountEngineer, Reddit (r/LLMDevs)

    Our OpenAI bill tells us what we spent, not who spent itReddit post, r/LLMDevs

    Token spend tripled in six months while the invoice stays one number, and separate keys per customer show usage without dollar amounts, so the team can't prove which accounts lose money.

    Published Aug 16, 2026self-described: An AI SaaS company with about 40 customers

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API keys per customer

23. Mixed

    Kingston Duffie, OpenAI Developer Forum

    Problem with tool_choice causing caching issuesOpenAI Developer Forum post

    Changing tool_choice on the final call invalidates the cached prefix and pays cache writes for the whole context; explicit caching makes the problem visible.

    Published Aug 15, 2026self-described: Production system using function tools on GPT-5.6 through the Responses API

    Counts toward

    • Overall signal
    • Tool Calling & Structured Output: Mixed or negative
    • Token Cost: Raised as a concern
    • Caching is designed in

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-5.6, Responses API

24. Positive

    EB66, Hacker News commenter

    Comment on: Gemini 3.7 FlashHacker News comment

    GPT-5 mini led the company's own benchmarks on cost and quality until GPT-5.6 Luna's price cut replaced it.

    Published Aug 14, 2026self-described: Company in-house LLM benchmarks

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Cost Efficiency: Positive
    • Evaluate before switching

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-5 mini, then GPT-5.6 Luna

25. Negative

    carlos_ds, OpenAI Developer Forum

    Please Don't Deprecate GPT-Realtime Before a True Replacement ExistsOpenAI Developer Forum post

    Migrating a production voice agent to gpt-realtime-2.1-mini brought narrated instructions, skipped steps, and added latency; reverting to the deprecated model restored stability.

    Published Aug 4, 2026self-described: Production outbound voice agent for structured telephony calls

    Counts toward

    • Overall signal
    • Model Quality: Mixed or negative
    • Model & API Changes: Raised as a concern
    • Latency: Raised as a concern
    • Inconsistent Results: Raised as a concern
    • Long-Context Limits: Raised as a concern

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    gpt-realtime-mini, gpt-realtime-2.1-mini

26. Positive

    __jl__, Hacker News commenter

    Comment on: Advancing the price-performance frontier with GPT‑5.6Hacker News comment

    For the team's production app, OpenAI is now the best provider: a reliable API with useful features and better price-performance than the open models it ran before.

    Published Jul 30, 2026self-described: Team production app

    Counts toward

    • Overall signal
    • Cost Efficiency: Positive
    • Developer Experience: Positive
    • Stability: Positive
    • Speed: Positive
    • Rate Limits & Reliability: Discussed without concern
    • Should a team standardize on OpenAI or spread work across providers?: Standardize on OpenAI

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-5.6 Luna

27. Positive

    alexsouthmayd, Hacker News commenter

    Comment on: Launch HN: Bloomy (YC S26) - AI-powered mastery learning for K-12Hacker News comment

    Bloomy uses OpenAI and Anthropic models under Zero Data Retention agreements with both, which school districts reviewed in technical due diligence.

    Published Jul 21, 2026self-described: Bloomy, K-12 learning platform

    Counts toward

    • Overall signal
    • Data Handling: Discussed without concern
    • Teams keep a second provider
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: OpenAI's API
    • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

28. Positive

    madaxe_again, Hacker News commenter

    Comment on: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaperHacker News comment

    Assessed GPT-5.6 on the team's own workflows, moved most work from GPT-5.5 to Terra, halving cost and response times, with Luna as a cheap backstop.

    Published Jul 13, 2026self-described: Text and image extraction workflows in production

    Counts toward

    • Overall signal
    • Cost Efficiency: Positive
    • Speed: Positive
    • Latency: Discussed without concern
    • Inconsistent Results: Raised as a concern
    • Evaluate before switching
    • Route by task

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-5.5, then GPT-5.6 Terra and Luna

29. Negative

    Sai Reddy, OpenAI Developer Forum

    Prompt Caching Is a Core GPT-5.6 Feature. Why Are Customers Still Reverse-Engineering It?OpenAI Developer Forum post

    Measured about 96% cache hits on Azure Chat Completions and none on the Azure Responses API for GPT-5.6 Sol, and got no explanation from OpenAI or Microsoft.

    Published Jul 12, 2026self-described: Team running GPT-5.6 Sol on Azure OpenAI

    Counts toward

    • Overall signal
    • Developer Experience: Mixed or negative
    • Setup & SDK Friction: Raised as a concern
    • Caching is designed in
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-5.6 Sol on Azure OpenAI

30. Negative

    pimeys, Hacker News commenter

    Comment on: GPT-5.6Hacker News comment

    In the team's agent evals with prompts tuned per OpenAI's guide, GPT-5.6 Sol cost twice Opus 4.8 to reach the same conclusion, and Terra and Luna looped on failing tool calls.

    Published Jul 10, 2026self-described: Agent company running evals

    Counts toward

    • Overall signal
    • Tool Calling & Structured Output: Mixed or negative
    • Token Cost: Raised as a concern
    • Inconsistent Results: Raised as a concern
    • OpenAI models run their own agents
    • Evaluate before switching

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-5.6 Sol, Terra, and Luna

31. Negative

    dacox, Hacker News commenter

    Comment on: Please don't discontinue Gemini 2.5 FlashHacker News comment

    Fast, cheap models keep being deprecated by Google and OpenAI, and their replacements are reasoning models that are slower and much more expensive for the team's workloads.

    Published Jul 10, 2026self-described: Team fine-tuned workloads

    Counts toward

    • Overall signal
    • Model & API Changes: Raised as a concern
    • Token Cost: Raised as a concern
    • Latency: Raised as a concern

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

32. Positive

    Lorenzo Gentile, Ploy engineering blog

    Migrating a production AI agent to GPT-5.6Engineering blog post, Ploy blog

    After fixing its eval harness, tool schemas, and cache keys, Ploy moved its agent from Claude Opus 4.8 to GPT-5.6 Sol: builds 2.2x faster at 27% lower cost.

    "As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, no model met our bar for replacing Claude Opus. GPT-5.6 Sol did. After a head-to-head evaluation, we made it the default model for every Ploy workspace."

    Published Jul 9, 2026self-described: Ploy, production agent that builds and edits marketing websites

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Tool Calling & Structured Output: Mixed or negative
    • Cost Efficiency: Positive
    • Speed: Positive
    • Inconsistent Results: Raised as a concern
    • Setup & SDK Friction: Raised as a concern
    • Teams keep a second provider
    • OpenAI models run their own agents
    • Evaluate before switching
    • Caching is designed in
    • Should a team move to the Responses API or stay on Chat Completions?: Responses API

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-5.6 Sol against Claude Opus 4.8

33. Positive

    Artificial Analysis, independent model benchmarking service

    GPT-5.6 Luna (Max) Intelligence, Performance & Price AnalysisIndependent benchmark, Artificial Analysis

    Measures GPT-5.6 Luna (Max) at 127 output tokens per second, faster than the 116 average, and rates it among the leading models in intelligence at a reasonable price for its class.

    Published Jul 9, 2026self-described: Standard 10,000-input-token workload on OpenAI's API

    Counts toward, pending a recheck

    • Overall signal
    • Model Quality: Positive
    • Speed: Positive

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    GPT-5.6 Luna, max reasoning

34. Mixed

    benlyazid, OpenAI Developer Forum

    Prompt caching returns cached_tokens=0 intermittently (same static prefix, same prompt_cache_key)OpenAI Developer Forum post

    With a byte-identical static prefix and per-agent cache keys, two agents hit the cache 71% to 89% of the time while the orchestrator got none.

    Published Jul 8, 2026self-described: Multi-agent system on the OpenAI API with a per-tenant cache key

    Counts toward

    • Overall signal
    • OpenAI models run their own agents
    • Caching is designed in

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    gpt-5.4, Chat Completions

35. Mixed

    MitziMoto, Hacker News commenter

    Comment on: RubyLLM: A Ruby framework for all major AI providersHacker News comment

    Builds on the RubyLLM framework and was frustrated that it lacked native Responses API support until recently.

    Published Jun 24, 2026self-described: Team product built on RubyLLM

    Counts toward

    • Overall signal
    • Should a team move to the Responses API or stay on Chat Completions?: Responses API

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

36. Mixed

    Kwindla Hultman Kramer, CEO, Daily

    Voice AI & Voice Agents: An Illustrated PrimerTechnical guide, voiceaiandvoiceagents.com

    Daily's CEO names GPT-4.1 and GPT-5.1 among the most used models for voice agents. He finds agents on the Realtime API cost 3 to 5 times more than GPT-4.1 agents and run slower than a tuned multi-model pipeline.

    Published Jun 1, 2026Daily and the Pipecat open-source framework for production voice agents

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Token Cost: Raised as a concern
    • Latency: Raised as a concern

    Published on

    Engineering Blogs & Reviews

    Who

    Technical leadership

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-4.1, GPT-5.1, and gpt-realtime in the Realtime API

37. Negative

    Shane Prebenda, OpenAI Developer Forum

    Zero Data Retention (ZDR) request: enterprise customer, no response after multiple submissionsOpenAI Developer Forum post

    Client contracts require Zero Data Retention before production traffic; four business days after the sales form, no reply had come, blocking the launch.

    Published May 27, 2026self-described: Production platform whose client NDAs require Zero Data Retention

    Counts toward

    • Overall signal
    • Data Handling: Raised as a concern

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

38. No opinion stated

    NoTextit, Reddit (r/LLMDevs), runs platform engineering at a 150-person company

    Cross-provider API cost allocation at team scaleReddit post, r/LLMDevs

    LLM spend rose from US$8k to US$24k a month in three months. OpenAI projects gave per-project limits, but no per-person budget and no view of the Claude side, so the team is trialing a cross-provider gateway.

    Published May 19, 2026self-described: Platform engineering at a 150-person company, about 80 people using the API

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI projects beside Claude, with Portkey, LiteLLM and TokenRouter under evaluation

39. Negative

    7thpower, Hacker News commenter

    Comment on: OpenAI models coming to Amazon Bedrock: Interview with OpenAI and AWS CEOsHacker News comment

    When the team used Azure OpenAI two to three years ago, throughput quotas were small and increases bureaucratic; it moved to Claude once Claude 3 shipped.

    Published Apr 29, 2026self-described: Team that ran OpenAI models on Azure two to three years earlier

    Counts toward, pending a recheck

    • Overall signal
    • Rate Limits & Reliability: Raised as a concern
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    Azure OpenAI

40. Negative

    Foobar8568, Hacker News commenter

    Comment on: OpenAI models coming to Amazon Bedrock: Interview with OpenAI and AWS CEOsHacker News comment

    On Azure OpenAI deployments the team saw frequent latency spikes, up to three minutes to first byte, and escalation brought no explanation.

    Published Apr 29, 2026self-described: Team that ran Azure OpenAI deployments until about a year earlier

    Counts toward

    • Overall signal
    • Stability: Mixed or negative
    • Rate Limits & Reliability: Raised as a concern
    • Latency: Raised as a concern
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Azure OpenAI

41. Negative

    lastround360, OpenAI Developer Forum

    Sudden increase in /v1/responses timeouts on gpt-4.1-mini (stable for months, started recently)OpenAI Developer Forum post

    Responses API timeouts on gpt-4.1-mini, at a 45-second cutoff, rose from 1.28% to 13.64% of calls over four days in April 2026.

    Published Apr 19, 2026self-described: Mobile app calling the Responses API through a Supabase Edge Function

    Counts toward

    • Overall signal
    • Stability: Mixed or negative
    • Rate Limits & Reliability: Raised as a concern
    • Latency: Raised as a concern

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    gpt-4.1-mini, Responses API

42. Positive

    lmeyerov, Hacker News commenter

    Comment on: GPT-5.4Hacker News comment

    In the team's evals on cybersecurity investigations, GPT-5.2-Codex at low reasoning won: more than 2x faster with higher completion rates than higher reasoning settings.

    Published Mar 5, 2026self-described: Team evals on cybersecurity investigations

    Counts toward, pending a recheck

    • Overall signal
    • Model Quality: Positive
    • Speed: Positive
    • Latency: Discussed without concern
    • OpenAI models run their own agents
    • Evaluate before switching

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    GPT-5.2-Codex, low reasoning

43. Positive

    Amp team, Amp

    GPT-5.4, The New OracleProduct engineering post, Amp Chronicle

    In Amp's internal evals GPT-5.4 raised oracle response quality from 60.8% to 68.2% over GPT-5.2 and cut mean latency from about 6.7 to 4.9 minutes.

    Published Mar 5, 2026self-described: Amp coding agent, GPT-5.4 as the oracle subagent in smart and deep modes

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Speed: Positive
    • Evaluate before switching

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-5.4 as the oracle, replacing GPT-5.2

44. Mixed

    Ravi Raju and four coauthors

    The Limits of Long-Context Reasoning in Automated Bug FixingResearch paper, arXiv (ICLR 2026 ICBINB workshop)

    In an agent harness GPT-5-nano resolved up to 31% of 100 SWE-bench Verified tasks, but given the relevant files in a 64K-token context it solved none.

    Published Feb 17, 2026self-described: Academic evaluation of patch generation on SWE-bench Verified, in an agent harness and with files placed in a 64K-token context

    Counts toward

    • Overall signal
    • Long Context: Mixed or negative
    • Long-Context Limits: Raised as a concern

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Still applicable

    Observed on

    GPT-5-nano

45. Mixed

    Nima Esmi and coauthors, University of Groningen, University of Guilan, and TU Delft

    GPT-5 vs Other LLMs in Long Short-Context PerformanceResearch paper, arXiv (FLLM 2025, IEEE)

    Past 5,000 posts (about 70,000 tokens) accuracy fell for every model, to about 50 to 53% at 20,000 posts, while GPT-5 kept precision near 95% and lost-in-the-middle errors had largely disappeared.

    Published Feb 15, 2026self-described: Academic evaluation of GPT-5, GPT-4, Gemini 2.5, and Grok-4 on a 20,000-post classification set

    Counts toward

    • Overall signal
    • Long Context: Mixed or negative
    • Long-Context Limits: Raised as a concern

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Still applicable

    Observed on

    GPT-5 and GPT-4

46. Positive

    sigmoid10, Hacker News commenter

    Comment on: GPT-5.2Hacker News comment

    On OpenAI's highest enterprise tier, the commenter finds OpenAI a much more stable platform for high-volume API use than other providers.

    Published Dec 12, 2025self-described: Team on OpenAI's highest enterprise tier

    Counts toward

    • Overall signal
    • Stability: Positive
    • Rate Limits & Reliability: Discussed without concern
    • Should a team standardize on OpenAI or spread work across providers?: Standardize on OpenAI

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

47. Mixed

    Zairah Mustahsan, Abel Lim, Megna Anand, Saahil Jain, and Bryan McCann, You.com

    Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass CorrelationResearch paper, arXiv

    On hard GAIA questions, 70% of GPT-4o's variance came from trial-to-trial randomness, making single runs unreliable; GPT-5 raised accuracy from 6.6% to 44.2% and nearly doubled consistency.

    Published Dec 7, 2025self-described: You.com research team running 64 trials per question on GAIA and FRAMES

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Inconsistent Results: Raised as a concern
    • Evaluate before switching

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Still applicable

    Observed on

    GPT-4o and GPT-5 with web search

48. Negative

    mparis, Hacker News commenter

    Comment on: Structured outputs on the Claude Developer PlatformHacker News comment

    Could not use OpenAI's structured outputs because its strict validation rejected standard JSON Schema elements the team used, which pushed it to Claude.

    Published Nov 14, 2025self-described: Team running structured outputs in production

    Counts toward

    • Overall signal
    • Tool Calling & Structured Output: Mixed or negative
    • Feature Gaps: Raised as a concern
    • Teams keep a second provider

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

49. Positive

    pietz, Hacker News commenter

    Comment on: Claude Haiku 4.5Hacker News comment

    Uses mini and nano models for every user-facing and workflow task at the company, and larger models only for internal high-complexity work such as coding.

    Published Oct 15, 2025self-described: Company workflows: extraction, conversion, translation, evaluation

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Cost Efficiency: Positive
    • Route by task

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    GPT-5 mini and nano

50. Negative

    Michal Stoklasa, OpenAI Developer Forum

    How to become eligible for data residency endpoints?OpenAI Developer Forum post

    Runs on Azure OpenAI for EU hosting but wants OpenAI's own EU data residency endpoint, since Azure lacks priority processing and vector stores and is slower; sales never answered.

    Published Oct 13, 2025self-described: AI web agent company with about 70 larger customers in Europe

    Counts toward

    • Overall signal
    • Data Handling: Raised as a concern
    • Feature Gaps: Raised as a concern
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: OpenAI's API

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

51. Mixed

    Amp team, Amp

    GPT-5 OracleProduct engineering post, Amp Chronicle

    After testing GPT-5 as a main model, Amp kept it as the oracle for planning and debugging, and left Sonnet as the main agent model because GPT-5 was less proactive.

    Published Oct 1, 2025self-described: Amp coding agent, GPT-5 as the oracle subagent beside Sonnet as the main agent model

    Counts toward, pending a recheck

    • Overall signal
    • Model Quality: Positive
    • Route by task
    • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    GPT-5 as the oracle, replacing o3

52. Mixed

    hedgehog, Hacker News commenter

    Comment on: Sampling and structured outputs in LLMsHacker News comment

    Could not rely on OpenAI Structured Outputs in one component, and switched that component to TOML-formatted output that worked across many models.

    Published Sep 23, 2025self-described: Team with its own prompt templates and schema definitions

    Counts toward

    • Overall signal
    • Tool Calling & Structured Output: Mixed or negative

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

53. Negative

    Stefan Ivanovici, Intercom

    Cost of Serving LLMsEngineering research post, Fin AI research blog

    Intercom calls the OpenAI and Anthropic models behind Fin expensive and slow, and its AWS benchmarks found self-hosted models under 14B parameters cheaper to serve than GPT-4.1 mini.

    Published Sep 10, 2025self-described: Intercom's Fin customer support agent, self-hosted open-weight models compared with provider APIs

    Counts toward, pending a recheck

    • Overall signal
    • Cost Efficiency: Mixed or negative
    • Token Cost: Raised as a concern
    • Latency: Raised as a concern

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    GPT-4.1 and GPT-4.1 mini

54. Positive

    dahcryn, Hacker News commenter

    Comment on: Adaptive LLM routing under budget constraintsHacker News comment

    Model calls are not the team's largest AI cost; GPT-4.1 is cheap enough that it skips mini models and optimizes speech-to-text spend instead.

    Published Sep 2, 2025self-described: Team running AI features in production

    Counts toward

    • Overall signal
    • Cost Efficiency: Positive

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    GPT-4.1

55. Positive

    mahmoud-almadi, Hacker News commenter at Cyberdesk

    Comment on: Launch HN: Cyberdesk (YC S25) - Automate Windows legacy desktop appsHacker News comment

    Cyberdesk runs healthcare automation under signed zero data retention agreements with Anthropic and OpenAI and BAAs with its customers.

    Published Aug 14, 2025self-described: Cyberdesk, healthcare desktop automation

    Counts toward

    • Overall signal
    • Data Handling: Discussed without concern
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: OpenAI's API

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Still applicable

56. Mixed

    Dobo Radichkov, OpenAI Developer Forum

    Feedback: GPT-5 vs GPT-5-Chat-Latest for Conversational AIOpenAI Developer Forum post

    For the Onsen app, gpt-5 suits Structured Outputs and gpt-5-chat-latest suits conversation, so the team now routes between them where GPT-4.1 handled both.

    Published Aug 10, 2025self-described: Onsen, AI mental health companion app

    Counts toward

    • Overall signal
    • Model Quality: Mixed or negative
    • Inconsistent Results: Raised as a concern
    • Route by task

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    gpt-5, gpt-5-chat-latest, gpt-4.1

57. Positive

    lieret, Hacker News commenter

    Comment on: GPT-5 on SWE-bench: Cost and performance deep-diveHacker News comment

    On SWE-bench Verified GPT-5 scored 65%, GPT-5 mini 60%, and nano 35%; mini gave up about 5 points for roughly a fifth of the cost.

    Published Aug 8, 2025self-described: SWE-bench team evaluating models with a minimal agent

    Counts toward, pending a recheck

    • Overall signal
    • Model Quality: Positive
    • Cost Efficiency: Positive
    • OpenAI models run their own agents

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    GPT-5, GPT-5 mini, GPT-5 nano

58. Negative

    Jacob-J-Thomas, Hacker News commenter

    Comment on: The Intelligence HubHacker News comment

    An early enterprise chat app on Azure OpenAI hit frequent API failures and outages, and model and API changes were hard to keep up with.

    Published Jul 23, 2025self-described: Enterprise chat app on Azure OpenAI, two years earlier

    Counts toward, pending a recheck

    • Overall signal
    • Stability: Mixed or negative
    • Model & API Changes: Raised as a concern
    • Rate Limits & Reliability: Raised as a concern
    • Feature Gaps: Raised as a concern
    • Setup & SDK Friction: Raised as a concern
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    Azure OpenAI, 2023

59. Positive

    BrandiATMuhkuh, Hacker News commenter

    Comment on: ChatGPT's enterprise success against Copilot fuels OpenAI/Microsoft rivalryHacker News comment

    Builds a co-pilot for architects on Azure OpenAI with retrieval and a profession-specific system prompt; a customer wants to roll it out across the company.

    Published Jun 25, 2025self-described: Co-pilot product for architects (AEC)

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    Azure OpenAI with retrieval

60. Negative

    Herman, OpenAI Developer Forum

    NYT lawsuit and 30 days data retentionOpenAI Developer Forum post

    The court-ordered retention left a Swedish company's customers out of compliance; moving to Azure OpenAI was its only option, at the cost of features such as web search.

    Published Jun 16, 2025self-described: Swedish company whose customers need retention under 30 days

    Counts toward, pending a recheck

    • Overall signal
    • Data Handling: Raised as a concern
    • Feature Gaps: Raised as a concern
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: OpenAI's API

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Needs a recheck

61. Negative

    molf, Hacker News commenter

    Comment on: How we're responding to The NYT's data demands in order to protect user privacyHacker News comment

    Applied for Zero Data Retention several times with no response, and says OpenAI mentions the option without approving requests in practice.

    Published Jun 6, 2025self-described: Business API customer applying for Zero Data Retention

    Counts toward, pending a recheck

    • Overall signal
    • Data Handling: Raised as a concern

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Needs a recheck

62. Mixed

    jasongill, Hacker News commenter

    Comment on: New tools and features in the Responses APIHacker News comment

    Found the Assistants API the best fit before the Responses API and wanted clearer timelines for its sunset and migration.

    Published May 21, 2025self-described: Team product on the Assistants API

    Counts toward, pending a recheck

    • Overall signal
    • Model & API Changes: Raised as a concern

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    Assistants API

63. Negative

    mritchie712, Hacker News commenter at Definite

    Comment on: New tools and features in the Responses APIHacker News comment

    Definite ran its agent on the Assistants API, expected it to die for lack of updates, and switched to the model-agnostic Pydantic AI framework.

    Published May 21, 2025self-described: Definite, data platform agent

    Counts toward, pending a recheck

    • Overall signal
    • Model & API Changes: Raised as a concern
    • Teams keep a second provider
    • OpenAI models run their own agents

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    Assistants API

64. Mixed

    jbellis, Hacker News commenter at Brokk

    Comment on: The unreasonable effectiveness of an LLM agent loop with tool useHacker News comment

    At Brokk, some smart models such as o3 are poor at tool calling, so Gemini 2.5 Pro runs the Architect agent while GPT-4.1 and others work for search.

    Published May 15, 2025self-described: Brokk, search and architect agents

    Counts toward, pending a recheck

    • Overall signal
    • Tool Calling & Structured Output: Mixed or negative
    • Teams keep a second provider
    • OpenAI models run their own agents
    • Route by task

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    o3 and GPT-4.1

65. Mixed

    Harvey Team, Harvey

    Expanding Harvey's Model OfferingsCompany blog post, Harvey blog

    Harvey added Anthropic and Google models to supplement its existing OpenAI models and routes each legal task to the best model, chosen with its own BigLaw Bench evaluations.

    Published May 13, 2025self-described: Harvey legal AI platform, adding Anthropic and Google models beside its OpenAI models

    Counts toward

    • Overall signal
    • Teams keep a second provider
    • Evaluate before switching
    • Route by task
    • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI models beside Anthropic models on AWS Bedrock and Google models on Vertex

66. Mixed

    Stefano Rando and seven coauthors

    LongCodeBench: Evaluating Coding LLMs at 1M Context WindowsResearch paper, arXiv

    At contexts up to 1M tokens, GPT-4.1 led the code question-answering task and held steady across lengths, but on bug fixing its accuracy fell from a peak of 11% to 2%.

    Published May 12, 2025self-described: Academic coding benchmark of code questions and bug fixes on real repositories at contexts up to 1M tokens

    Counts toward, pending a recheck

    • Overall signal
    • Long Context: Mixed or negative
    • Long-Context Limits: Raised as a concern

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    GPT-4.1 and GPT-4o

67. Negative

    Eduardo Stuani, OpenAI Developer Forum

    Increased latency on GPT-4.1 Mini over the past few daysOpenAI Developer Forum post

    Uses a large fixed, cached context to cut latency on 110K-token prompts; GPT-4.1 mini average latency rose from about 4 to 7 seconds, p90 from 7 to 12.

    Published Apr 30, 2025self-described: Latency-critical application with 110K-token cached prompts

    Counts toward

    • Overall signal
    • Latency: Raised as a concern
    • Caching is designed in

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Still applicable

    Observed on

    GPT-4.1 mini

68. Mixed

    Daniel Chalef, founder and CEO of Zep, with Preston Rasmussen, Zep

    GPT-4.1 and o4-mini: Is OpenAI Overselling Long-Context?Engineering blog post, Zep blog

    On LongMemEval conversations of about 115,000 tokens, GPT-4.1 averaged 56.72%, below GPT-4o-mini's 57.87%, while o4-mini led at 72.78%; Zep concludes a 1M-token window alone does not improve recall.

    Published Apr 17, 2025self-described: Zep, a memory service for AI agents, benchmarking OpenAI models on LongMemEval

    Counts toward, pending a recheck

    • Overall signal
    • Long Context: Mixed or negative
    • Long-Context Limits: Raised as a concern
    • Evaluate before switching

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    GPT-4.1 and o4-mini, full 115,000-token context

69. Positive

    Dedy Kredo, Qodo

    We benchmarked GPT-4.1 vs Claude 3.7: Here's what we foundEngineering blog post, Qodo blog

    On 200 real pull requests judged by o3-mini, GPT-4.1's code suggestions beat Claude 3.7 Sonnet's in 54.9% of cases.

    Published Apr 13, 2025self-described: Qodo code review product

    Counts toward, pending a recheck

    • Overall signal
    • Model Quality: Positive
    • Evaluate before switching

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    GPT-4.1

70. Positive

    Kingston Duffie, OpenAI Developer Forum

    My experience switching from Assistants API to Responses APIOpenAI Developer Forum post

    Rewrote a production Assistants API chat app on the Responses API in about two hours; instructions and vector stores carried over, on gpt-4o-mini for cost.

    Published Mar 22, 2025self-described: Production RAG chat application with tens of thousands of users

    Counts toward

    • Overall signal
    • Cost Efficiency: Positive
    • Developer Experience: Positive
    • Speed: Mixed or negative
    • Model & API Changes: Discussed without concern
    • Latency: Discussed without concern
    • Setup & SDK Friction: Discussed without concern
    • Should a team move to the Responses API or stay on Chat Completions?: Responses API

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Still applicable

    Observed on

    gpt-4o-mini, Assistants API to Responses API

71. Mixed

    k9294, Hacker News commenter

    Comment on: Ask HN: AI infrastructure in production - what is your tech stack?Hacker News comment

    A small startup dropped LangChain for the OpenAI SDK behind its own gateway, which handles cost tracking, metered billing, and model fallbacks.

    Published Mar 17, 2025self-described: Startup with a few thousand monthly active users

    Counts toward

    • Overall signal
    • Teams keep a second provider
    • OpenAI models run their own agents

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    OpenAI SDK behind a custom gateway

72. Negative

    daviding, Hacker News commenter

    Comment on: New tools for building agentsHacker News comment

    Porting to the Responses API is non-trivial for a team that already built history and retrieval on Chat Completions, and web search pushed it there.

    Published Mar 12, 2025self-described: Team assistant built on Chat Completions

    Counts toward, pending a recheck

    • Overall signal
    • Model & API Changes: Raised as a concern
    • Setup & SDK Friction: Raised as a concern
    • Should a team move to the Responses API or stay on Chat Completions?: Chat Completions

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Needs a recheck

73. Mixed

    Siddharth Pandey and Wenbo Wei, Grab engineering

    Grab AI Gateway: Connecting Grabbers to multiple GenAI providersEngineering blog, Grab Tech

    Grab reaches OpenAI and Azure OpenAI as separate routes behind one AI Gateway, where teams set only a base URL. Grab counts more than 300 use cases onboarded since the gateway started in 2023, with many in production (Feb 19, 2025).

    Published Feb 19, 2025self-described: Grab's company AI Gateway, a set of reverse proxies to Azure, OpenAI, AWS and other providers

    Counts toward

    • Overall signal
    • Teams keep a second provider
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock
    • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    OpenAI and Azure OpenAI as separate provider routes behind one gateway

74. Mixed

    Ali Modarressi and six coauthors

    NoLiMa: Long-Context Evaluation Beyond Literal MatchingResearch paper, arXiv (ICML 2025)

    With no literal matches to lean on, 11 of 13 models fell below half their short-context scores at 32K tokens, and GPT-4o, one of the strongest, dropped from 99.3% to 69.7%.

    Published Feb 7, 2025self-described: Academic long-context benchmark of 13 models, with questions and needles that share minimal wording

    Counts toward, pending a recheck

    • Overall signal
    • Long Context: Mixed or negative
    • Long-Context Limits: Raised as a concern

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    GPT-4o

75. Mixed

    Mandy Gu, author of the InfoQ article on Wealthsimple's platform

    Gen AI productivity tools at WealthsimpleEngineering article, InfoQ

    Wealthsimple's gateway began with OpenAI and Cohere, added Gemini for its 1-million-token context and then Amazon Bedrock, and in September 2024 shipped a v2 API that mirrors OpenAI's API specification so teams code against one shape.

    "We took some time in September of 2024 to ship v2 of our API, which did mirror OpenAI's API specs. We learned that as the GenAI industry matures, it's important to think about what the right standards and integrations are."

    Published Feb 6, 2025self-described: Wealthsimple's internal LLM gateway, which started with OpenAI and Cohere and later added Gemini and Amazon Bedrock

    Counts toward

    • Overall signal
    • Developer Experience: Positive
    • Teams keep a second provider
    • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    OpenAI and Cohere, then Gemini and Amazon Bedrock, behind one gateway

76. No opinion stated

    Sonofpeace0001, X

    X post on OpenAI API 429 errors during capacity issuesX post

    On October 26, 2026 the author saw 429 errors tied to OpenAI capacity issues; longer waits between retries and smaller tasks meant a failed call cost one step while the rest of the run continued.

    Published Oct 26, 2026self-described: An agent builder running multi-step work on the OpenAI API

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API 429 errors

77. No opinion stated

    South_Traffic_72, Reddit (r/OpenAIDev)

    One compromised project, entire OpenAI account closedReddit post, r/OpenAIDev

    After one of about 20 client projects was compromised, OpenAI closed the whole account for abuse, cutting API access for every project plus ChatGPT and Codex; an appeal had no answer after four days.

    Published Oct 23, 2026self-described: A developer building about 20 client projects on the OpenAI API

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API account shared across client projects

78. No opinion stated

    Tarun122, Reddit (r/OpenAI)

    Using OpenAI Realtime API with a RobotReddit post, r/OpenAI

    The author drives a robot arm by voice through gpt-realtime-2.1, giving the model seven tools while simulation-trained policies handle the motion.

    "Using OpenAI Realtime API with a Robot I hooked up the OpenAI Realtime API (gpt-realtime-2.1 over WebRTC) to a small robot arm I built, so I can just talk to it. In the video it plays tic-tac-toe with me, puts a cube in a bowl, picks its favourite toy car and hands a cube back to me. How it works: * The voice model gets only a handful of tools: look, reach, grab, drop, gripper, park and stop. Reach and grab aren't hand-coded moves. They're policies I trained in simulation, so the model decides *what* to do and the trained skills decide *how*. * Fresh images from both cameras (top view and wrist) go into the conversation with every user turn, so it always sees the current board. It reads the tic-tac-toe grid itself from the images. * To pick a target, a vision model puts a point on the object in the top camera image, and the reach or grab policy goes to that point."

    Published Oct 23, 2026self-described: A personal robot arm driven by voice

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    gpt-realtime-2.1 over WebRTC

79. No opinion stated

    keithmifsud, Reddit (r/OpenAI)

    API Credit expiry - another yearReddit post, r/OpenAI

    Credits bought on November 18, 2025 expire on November 1, 2026, and the author asks OpenAI for an extension before buying more for the team.

    "API Credit expiry - another year Is there any way to extend the API credit expiry date? Also, if I purchased these on 18th November 2025, why will they expire on the 1st November 2026? I stopped using ChatGPT last year because of this crap. I literally just subscribed a week ago as I was purchasing plans for my team. ChatGPT is almost good now (work mode times outafter 11mins though and never managed to complete a job thus far - even with additional credits and resets. However, it is okay for small things. I'll ban OpenAI for good at every level of the org group if I don't get at least a month's extension on the API credits from the 17th Nov!"

    Published Oct 21, 2026self-described: Prepaid OpenAI API credits bought for a team

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API prepaid credits

80. No opinion stated

    jkos123, Reddit (r/OpenAI)

    Reply on API billing errors, r/OpenAIReddit comment, r/OpenAI

    The author says OpenAI overcharged them US$1,950 the month before, refunded it as credits, and now shows a billing error against those credits.

    "I just got it too! Last month they overcharged me by $1,950...got it refunded in credits...now $1,000 worth of credits with this error...hopefully they get their billing department sorted out here soon...kind of a mess."

    Published Oct 20, 2026self-described: OpenAI API account billing

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API billing

81. No opinion stated

    ChoiceAnalyst1599, Reddit (r/OpenAI)

    Reply on ChatGPT credits and API billing, r/OpenAIReddit comment, r/OpenAI

    Credits bought in ChatGPT do not fund the API, which bills separately on the platform site; the author advises starting on a cheaper model before moving to GPT-6.

    "I went through same headache few weeks ago. The credits you buy in ChatGPT are only for the web chat interface, not for API. Is two completely separate billing systems which makes zero sense but thats how they set it up. For API you need to go to the platform site and add payment method there, minimum is like 5 dollars I think. Also the top comment is right about costs, I burned through 3 dollars just experimenting with some queries without even noticing. The pricing page is confusing but you should look at it before doing anything serious. If you just want to test the API maybe start with a cheaper model first, get comfortable with how the requests work, then switch to GPT6 when you know what you are doing. The documentation has some basic curl examples you can copy paste directly into terminal."

    Published Oct 19, 2026self-described: Setting up OpenAI API billing

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API platform billing

82. No opinion stated

    deadstrobe5, GitHub issue author

    openai-python issue #4090: streaming helpers re-parse accumulated JSONGitHub issue, openai/openai-python

    The SDK's streaming helpers re-parse the whole accumulated JSON on every delta, so parse time grows with the square of output size: 141 ms for a 70 KB tool argument in the author's test.

    "Streaming helpers re-parse the whole accumulated JSON on every delta (quadratic in output size) `ChatCompletionStreamState._accumulate_chunk` (`src/openai/lib/streaming/chat/_completions.py`) calls `from_json(bytes(..., "utf-8"), partial_mode=True)` on the full accumulated `message.content` (with a `response_format` model) and on the full accumulated arguments of strict function tools, on every delta. Streaming one structured output or tool call therefore costs O(n²) in its size. Here is the parse cost alone, for a tool argument that writes a file, in 10-byte deltas (CPython 3.14, jiter, median of 5): | arguments | deltas | time in `from_json` | |---|---|---| | 4 KB | 502 | 0.7 ms | | 20 KB | 2,499 | 12.6 ms | | 70 KB | 8,739 | 141 ms | Because `partial_mode=True` drops an unterminated trailing string, a del…"

    Published Oct 16, 2026self-described: Streaming structured outputs and strict tool calls in openai-python

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

    Observed on

    openai-python streaming helpers

83. Mixed

    epolanski, Hacker News commenter

    Comment on: Why isn't the industry freaking out about DeepSeek 4.1 Flash?Hacker News comment

    Virtually all of the commenter's clients buy models through Bedrock or Azure with zero data retention, and none would contract with OpenAI directly.

    Published Oct 9, 2026self-described: Consultant describing client procurement

    Counts toward

    • Overall signal
    • Data Handling: Raised as a concern
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

84. Positive

    Farmadupe, Hacker News commenter

    Comment on: Claude Haiku 5.5Hacker News comment

    On a classification workload GPT-6 Luna used about 7.9M input tokens against 11.9M for Haiku 5.5 and cost US$0.31 against US$0.42 synchronously.

    Published Oct 8, 2026self-described: Own classification workload

    Counts toward

    • Overall signal
    • Cost Efficiency: Positive
    • Evaluate before switching

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-6 Luna, Batch and synchronous

85. Mixed

    Topfi, Hacker News commenter

    Comment on: Claude Haiku 5.5Hacker News comment

    Measured Haiku 5.5 at about 131 tokens per second on OpenRouter, roughly twice GPT-6 Luna's throughput with lower latency.

    Published Oct 7, 2026self-described: Own model comparisons

    Counts toward

    • Overall signal
    • Speed: Mixed or negative
    • Latency: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-6 Luna via OpenRouter

86. Positive

    mrbungie, Hacker News commenter

    Comment on: Claude Haiku 5.5Hacker News comment

    Comparing low-tier model prices for zero-shot and few-shot tasks, the commenter chose GPT-5.6 Luna because other rates made no sense.

    Published Oct 7, 2026self-described: Own zero-shot and few-shot tasks

    Counts toward

    • Overall signal
    • Cost Efficiency: Positive
    • Teams keep a second provider

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-5.6 Luna

87. No opinion stated

    Suitable-Season-4847, Reddit (r/OpenAI)

    Has anyone managed to get EU data residency on their API account?Reddit post, r/OpenAI

    The author's launch waits on EU data residency and abuse monitoring approval, and a request to OpenAI's sales team had no reply at the time of posting.

    Published Oct 5, 2026self-described: A UK SaaS platform with public sector customers

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API EU data residency and abuse monitoring controls

88. Positive

    bob1029, Hacker News commenter

    Comment on: The OpenAI Decisions API needs a confidence you can trustHacker News comment

    Now uses the Responses API with clients, since it is required for non-zero reasoning effort on the latest models, and finds Luna at high reasoning close to ideal.

    Published Oct 2, 2026self-described: Client projects on the API

    Counts toward

    • Overall signal
    • Model Quality: Positive
    • Should a team move to the Responses API or stay on Chat Completions?: Responses API

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-5.6 Luna and later via the Responses API

89. No opinion stated

    zdurm, GitHub issue author

    openai-python issue #4020: Responses request transform CPU costGitHub issue, openai/openai-python

    With a long input and many tools, the SDK's request transform takes about 28 ms of event-loop CPU per request, by the author's measurement.

    "With a long `input` and many `tools`, the transform takes about 28ms of CPU on the event loop per request."

    Published Oct 2, 2026self-described: The openai-python SDK on the Responses API

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    openai-python, Responses API

90. Positive

    buckwheatmilk, Hacker News commenter

    Comment on: GPT-6 Sol and LunaHacker News comment

    A fine-tuned GPT-4.1-mini had no good successor until GPT-6 Luna, which matches its accuracy at competitive pricing; GPT-5-mini was slower and more expensive.

    Published Sep 23, 2026self-described: Own structured-output task

    Counts toward

    • Overall signal
    • Tool Calling & Structured Output: Positive
    • Cost Efficiency: Positive
    • Speed: Mixed or negative
    • Model & API Changes: Raised as a concern
    • Inconsistent Results: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Fine-tuned GPT-4.1-mini, GPT-5-mini, GPT-6 Luna

91. No opinion stated

    snow_mac, Hacker News commenter

    Show HN reply on a podcast-archive pipeline's costsHacker News comment

    Transcribing and indexing 230 episodes, about 145 hours of audio, cost the author about US$33 on OpenAI's transcription and embedding APIs; unbounded async calls hit rate limits first.

    "I wanted to query the entire archive of a podcast I listen to without paying for their coaching tier. Built a lightweight pipeline to ingest, summarize, index, and answer questions with citations back to the source audio. Totals: - 230 episodes (~145 hours of audio, 1.29M words) - Total cost: ~$33 using OpenAI's transcription and embedding APIs - Pipeline: Audio chunking -> Whisper transcription -> Structured summaries -> Semantic index -> Retrieval with source citations A few engineering details from the post-mortem: 1. Concurrency traps: Initially went with unbounded async calls, which quickly triggered ra…"

    Published Sep 23, 2026self-described: A personal pipeline that transcribes, summarizes and indexes 230 podcast episodes

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

    Observed on

    OpenAI transcription and embedding APIs

92. Negative

    z2, Hacker News commenter

    Comment on: OpenAI Agents APIHacker News comment

    Burned by the Assistants API retirement, the commenter swore off heavily stateful model APIs, including the new Agents API.

    Published Sep 11, 2026self-described: Own model integrations

    Counts toward

    • Overall signal
    • Model & API Changes: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

93. Positive

    baalimago, Hacker News commenter

    Comment on: OpenAI Agents APIHacker News comment

    Has had great success with the OpenAI Agents SDK, building sandbox, Slack, and knowledge integrations while limiting what is exposed to OpenAI.

    Published Sep 11, 2026self-described: Own agent with sandbox and Slack integrations

    Counts toward

    • Overall signal
    • Developer Experience: Positive
    • Setup & SDK Friction: Discussed without concern
    • OpenAI models run their own agents

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI Agents SDK

94. Mixed

    SyneRyder, Hacker News commenter

    Comment on: GPT-5.6 Sol Pricing Cut by 50% on OpenRouterHacker News comment

    Reports Azure-hosted GPT-5.6 Sol performing far worse than OpenAI's own endpoint on OpenRouter, citing about 87% uptime on Azure US.

    Published Aug 18, 2026self-described: Own routing through OpenRouter

    Counts toward

    • Overall signal
    • Speed: Positive
    • Rate Limits & Reliability: Raised as a concern
    • Latency: Discussed without concern
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: OpenAI's API

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    GPT-5.6 Sol via OpenRouter, OpenAI and Azure endpoints

95. Positive

    xmonkee, Hacker News commenter

    Comment on: GPT-5.6 Sol Pricing Cut by 50% on OpenRouterHacker News comment

    Keeps sensitive work on OpenAI and Anthropic because the commenter has zero data retention agreements with both.

    Published Aug 17, 2026self-described: Own work with sensitive data

    Counts toward

    • Overall signal
    • Data Handling: Discussed without concern
    • Teams keep a second provider
    • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: OpenAI's API

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

96. Positive

    dannyw, Hacker News commenter

    Comment on: Advancing the price-performance frontier with GPT‑5.6Hacker News comment

    Separates overloads on Codex subscriptions from the pay-per-token API, which the commenter finds rock stable.

    Published Jul 31, 2026self-described: Own pay-per-token API use

    Counts toward

    • Overall signal
    • Stability: Positive
    • Rate Limits & Reliability: Discussed without concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

97. No opinion stated

    ancestralsimulation, Reddit (r/OpenAIDev)

    OpenAI says I must switch to prepaid API billing, but the option is missingReddit post, r/OpenAIDev

    OpenAI told the author to move from monthly to prepaid billing by July 24, 2026, but the prepaid controls never appeared, so they are creating a backup API account to keep production apps running.

    Published Jul 18, 2026self-described: Several production mobile apps on one API organization

    Not counted

    Its author's ties to the vendor have not been checked. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenAI API billing, monthly to prepaid

98. Negative

    Ben B, OpenAI Developer Forum

    GPT-5.6: usage.output_tokens is ~9x actual generationOpenAI Developer Forum post

    Across about 23,000 stored reasoning responses, found output tokens over-counted and billed, at about 2x on o1, o3-mini, and gpt-5.4-nano for months.

    Published Jul 11, 2026self-described: ChessBench, a benchmark that runs chess games between models

    Counts toward

    • Overall signal
    • Token Cost: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-5.6, o1, o3-mini, gpt-5.4-nano

99. Positive

    XCSme, Hacker News commenter

    Comment on: GPT-5.6Hacker News comment

    Found GPT-5.6 good and cheap enough to replace GPT-5.3-Codex for tool calling in n8n workflows, and published benchmark results.

    Published Jul 9, 2026self-described: Own n8n automations and benchmark site

    Counts toward

    • Overall signal
    • Tool Calling & Structured Output: Positive
    • Cost Efficiency: Positive

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

    Observed on

    GPT-5.6

100. Mixed

     SwellJoe, Hacker News commenter

     Comment on: GPT-5.6 Sol Ultra will be in CodexHacker News comment

     On the commenter's own benchmarks, GPT-5.5 Pro cost nearly US$23 per task against US$1.12 for GPT-5.5, about 20x, though its token price is 6x.

     Published Jul 6, 2026self-described: Own benchmarks

     Counts toward

     • Overall signal
     • Token Cost: Raised as a concern
     • Evaluate before switching

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Measured

     Applies to

     Current

     Observed on

     GPT-5.5 and GPT-5.5 Pro via the API

101. Negative

     HyperL0gi, Hacker News commenter

     Comment on: Previewing GPT-5.6 Sol: a next-generation modelHacker News comment

     GPT-5 mini retires in December, and its replacement GPT-5.4 mini costs US$0.75 and US$4.50 per million tokens against US$0.25 and US$2, forcing a pricier upgrade.

     Published Jun 26, 2026self-described: Own workloads on GPT-5 mini

     Counts toward

     • Overall signal
     • Model & API Changes: Raised as a concern
     • Token Cost: Raised as a concern
     • Inconsistent Results: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Current

     Observed on

     GPT-5 mini, GPT-5.4 mini, GPT-5.4 nano

102. Positive

     sibidharan, Hacker News commenter

     Comment on: Ask HK: How are you building AI apps today?Hacker News comment

     Moved all earlier Completions and Responses projects to the Agents SDK after the Assistants API was retired, and finds it stable, with sessions, streaming, and cache tracking built in.

     Published May 29, 2026self-described: EdTech product with AI learning tools

     Counts toward

     • Overall signal
     • Developer Experience: Positive
     • Model & API Changes: Discussed without concern
     • Setup & SDK Friction: Discussed without concern
     • OpenAI models run their own agents
     • Should a team move to the Responses API or stay on Chat Completions?: Responses API

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Current

     Observed on

     OpenAI Agents SDK

103. Mixed

     AndreiOnyx, r/LLMDevs

     Calories & Macros LLM estimates from text (simple meals) comparison between frontier labsReddit post

     Benchmarked nine models with the app's production prompt and JSON schema; GPT-5.4 Mini and Nano were fastest at about 1.5 to 2.5 seconds but less accurate.

     Published Apr 7, 2026self-described: Calorie tracking app, benchmark before choosing a model

     Counts toward

     • Overall signal
     • Model Quality: Mixed or negative
     • Latency: Discussed without concern
     • Evaluate before switching

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Measured

     Applies to

     Current

     Observed on

     GPT-5.4, GPT-5.4 mini, GPT-5.4 nano

104. Negative

     eloisius, Hacker News commenter

     Comment on: The first 40 months of the AI eraHacker News comment

     While helping a client build agentic features, switched from the OpenAI Agents SDK to Agno, finding the Agents library messy and inconsistent.

     Published Mar 29, 2026self-described: Consulting on a client's agentic features

     Counts toward

     • Overall signal
     • Developer Experience: Mixed or negative
     • Setup & SDK Friction: Raised as a concern
     • OpenAI models run their own agents

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Current

     Observed on

     OpenAI Agents SDK

105. Mixed

     ilaksh, Hacker News commenter

     Comment on: GPT‑5.4 Mini and NanoHacker News comment

     GPT-5.4 mini and nano are fast enough for the commenter's voice applications but do not follow instructions well enough for the task.

     Published Mar 17, 2026self-described: Own voice applications

     Counts toward

     • Overall signal
     • Model Quality: Mixed or negative
     • Speed: Positive
     • Latency: Discussed without concern
     • Inconsistent Results: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Current

     Observed on

     GPT-5.4 mini and nano

106. Negative

     Rapzid, Hacker News commenter

     Comment on: GPT‑5.4 Mini and NanoHacker News comment

     Expected GPT-5.4 mini and nano to succeed the GPT-4.1 models, but their latency is far above the 400 to 600 ms the commenter gets now.

     Published Mar 17, 2026self-described: Own latency-sensitive workloads

     Counts toward

     • Overall signal
     • Model & API Changes: Discussed without concern
     • Latency: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Current

     Observed on

     GPT-4.1 models, GPT-5.4 mini and nano

107. Negative

     pscanf, Hacker News commenter

     Comment on: GPT‑5.4 Mini and NanoHacker News comment

     Preferred GPT for agentic applications a year ago but now finds it constantly misusing tools, and reaches for Claude for tool use and Gemini Flash for price and latency.

     Published Mar 17, 2026self-described: Own agentic applications

     Counts toward

     • Overall signal
     • Tool Calling & Structured Output: Mixed or negative
     • Teams keep a second provider
     • OpenAI models run their own agents
     • Route by task

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Current

108. Mixed

     Kunal Ganglani, independent AI engineer

     5 LLM APIs Tested for Latency: Real Data [2026]Independent benchmark, personal blog

     GPT-4.1 mini took about 2,400 ms to first token against 597 ms for Claude Haiku 4.5. GPT-4.1 streamed a strong 125 tokens per second, but its p99 latency can spike 3 to 5 times above p50 at peak hours.

     Published Mar 7, 2026Streaming benchmark from one Toronto server, plus a year of production workloads across three providers

     Counts toward, pending a recheck

     • Overall signal
     • Speed: Mixed or negative
     • Latency: Raised as a concern
     • Evaluate before switching

     Published on

     Engineering Blogs & Reviews

     Who

     Individual builder

     Evidence

     Measured

     Applies to

     Needs a recheck

     Observed on

     GPT-4.1 and GPT-4.1 mini

109. Positive

     Dumky de Wilde, independent developer

     LLMs Are Cheap Enough to Change How You Work With DataPersonal blog post, dumky.net

     Backfilled about 60,000 job postings through OpenAI's Batch API for US$10 in total, with batches often finishing in 10 to 15 minutes.

     Published Dec 21, 2025self-described: Backfill of job postings for selectfrom.work

     Counts toward

     • Overall signal
     • Cost Efficiency: Positive
     • Speed: Positive

     Published on

     Engineering Blogs & Reviews

     Who

     Individual builder

     Evidence

     Measured

     Applies to

     Still applicable

     Observed on

     OpenAI Batch API, Responses endpoint

110. Negative

     mips_avatar, Hacker News commenter

     Comment on: Gemini 3 Flash: Frontier intelligence built for speedHacker News comment

     Runs their own evals on GPT-5 with low thinking, GPT-4.1, and the mini model for low-latency apps, and finds Gemini 3 Flash ahead of them.

     Published Dec 17, 2025self-described: Own applications that need low latency

     Counts toward

     • Overall signal
     • Speed: Mixed or negative
     • Latency: Raised as a concern
     • Setup & SDK Friction: Raised as a concern
     • Evaluate before switching
     • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

111. Positive

     consumer451, Hacker News commenter

     Comment on: Claude Opus 4.5Hacker News comment

     Across side projects, found Azure-hosted GPT-4.1 relatively rock solid compared with Gemini 2.5 Pro's random resource-exhausted errors.

     Published Nov 24, 2025self-described: Side projects on several model APIs

     Counts toward

     • Overall signal
     • Stability: Positive
     • Rate Limits & Reliability: Discussed without concern
     • Teams keep a second provider
     • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     GPT-4.1 on Azure

112. Positive

     j_filipe, Hacker News commenter

     Comment on: Show HN: I built CostLens SDK to cut my AI bills by routing to cheaper modelsHacker News comment

     Routing requests to a cheaper model first and upgrading only when quality checks fail cut the commenter's OpenAI bill from about US$400 to US$120 a month.

     Published Nov 16, 2025self-described: Own production apps

     Not counted

     Its author builds and sells CostLens, a cost-routing layer that sits on top of the OpenAI API. Kept in the record; it takes part in no figure.

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Measured

     Applies to

     Still applicable

113. Positive

     rcarmo, Hacker News commenter

     Comment on: GPT-5.1: A smarter, more conversational ChatGPTHacker News comment

     Has used OpenAI models through Azure OpenAI with OpenWebUI for about a year, to keep chats and context on their own side.

     Published Nov 13, 2025self-described: Personal OpenWebUI setup on Azure OpenAI

     Counts toward

     • Overall signal
     • Data Handling: Discussed without concern
     • Should a team call OpenAI's API directly or buy OpenAI models through its cloud provider?: Azure or Bedrock

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

114. Mixed

     Topfi, Hacker News commenter

     Comment on: Claude Haiku 4.5Hacker News comment

     Early testing found GPT-5 models using more input tokens than the commenter considers justifiable, compared with Haiku 4.5.

     Published Oct 16, 2025self-described: Own model evaluations

     Counts toward

     • Overall signal
     • Speed: Mixed or negative
     • Long Context: Positive
     • Token Cost: Raised as a concern
     • Evaluate before switching

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     GPT-5 models

115. Mixed

     tempusalaria, Hacker News commenter

     Comment on: Claude Haiku 4.5Hacker News comment

     For low-latency calls with long prompts and many parallel requests, found OpenAI cached the shared prefix inconsistently.

     Published Oct 16, 2025self-described: Own production workloads, details withheld

     Counts toward

     • Overall signal
     • Cost Efficiency: Mixed or negative
     • Latency: Discussed without concern
     • Caching is designed in

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

116. Negative

     Zhu Liang, creator of 16x Eval

     GPT-5 Coding Evaluation: Underwhelming Performance Given the HypeIndependent benchmark, 16x Eval blog

     GPT-5 averaged 7.46 out of 10 on six human-rated coding tasks, below GPT-4.1's 8.08, and the author notes developers reporting inconsistent real-world performance after launch.

     Published Aug 15, 2025self-described: 16x Eval's own set of six coding tasks, run through the OpenAI API at default settings

     Counts toward, pending a recheck

     • Overall signal
     • Model Quality: Mixed or negative
     • Inconsistent Results: Raised as a concern
     • Evaluate before switching

     Published on

     Engineering Blogs & Reviews

     Who

     Individual builder

     Evidence

     Measured

     Applies to

     Needs a recheck

     Observed on

     GPT-5, medium reasoning effort, OpenAI API defaults

117. Positive

     energy123, Hacker News commenter

     Comment on: GPT-5 for DevelopersHacker News comment

     Finds pay-per-use cost-effective for small-context work: about US$0.03 to US$0.04 per coding prompt on GPT-5 with Flex pricing.

     Published Aug 8, 2025self-described: Own coding questions on the API

     Counts toward

     • Overall signal
     • Cost Efficiency: Positive

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     GPT-5, Flex processing

118. Mixed

     potatolicious, Hacker News commenter

     Comment on: GPT-5 for DevelopersHacker News comment

     Working on tool-heavy systems, finds models hallucinating fake tools, or answering without making the tool call, a major problem, and hopes GPT-5's training fixes it.

     Published Aug 7, 2025self-described: Tool-heavy systems at work

     Counts toward, pending a recheck

     • Overall signal
     • Tool Calling & Structured Output: Mixed or negative
     • OpenAI models run their own agents

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

119. Positive

     Simon Willison, independent developer and creator of the LLM command-line tool

     GPT-5: Key characteristics, pricing and model cardPersonal blog post, simonwillison.net

     After two weeks as his daily driver, finds GPT-5 rarely screws up and calls its pricing aggressively competitive: US$1.25 per million input tokens and US$10 per million output tokens.

     Published Aug 7, 2025self-described: Two weeks of daily use of GPT-5 through the API and his own LLM tool

     Counts toward, pending a recheck

     • Overall signal
     • Model Quality: Positive
     • Cost Efficiency: Positive

     Published on

     Engineering Blogs & Reviews

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

     Observed on

     GPT-5, GPT-5 mini, GPT-5 nano

120. Positive

     bayesianbot, Hacker News commenter

     Comment on: Claude Opus 4.1Hacker News comment

     Runs agentic work on o3 with about 95% of tokens cached and Flex processing on top, which brings input to about US$0.25 per million tokens.

     Published Aug 6, 2025self-described: Own agentic workloads

     Counts toward, pending a recheck

     • Overall signal
     • Cost Efficiency: Positive
     • Teams keep a second provider
     • OpenAI models run their own agents

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Measured

     Applies to

     Needs a recheck

     Observed on

     o3 with Flex processing

121. Mixed

     druskacik, Hacker News commenter

     Comment on: Batch Mode in the Gemini API: Process More for LessHacker News comment

     Used OpenAI's Batch API for some time, then moved to Mistral's batch API because Mistral Small was cheaper for the same use case.

     Published Jul 11, 2025self-described: Own batch workloads

     Counts toward

     • Overall signal
     • Cost Efficiency: Mixed or negative
     • Teams keep a second provider
     • Should a team standardize on OpenAI or spread work across providers?: Spread across providers

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     OpenAI Batch API

122. Mixed

     johnroy, OpenAI Developer Forum

     Experience and opinion on 4 APIs (assistant, chat completions, responses, realtime)OpenAI Developer Forum post

     After 18 months on Assistants, Chat Completions, Responses, and Realtime, found the first three too slow and prefers Realtime over WebRTC with OpenAI's examples.

     Published Jul 3, 2025self-described: Startup founder building on four OpenAI APIs over 18 months

     Counts toward, pending a recheck

     • Overall signal
     • Developer Experience: Positive
     • Speed: Mixed or negative
     • Model & API Changes: Raised as a concern
     • Latency: Raised as a concern
     • Setup & SDK Friction: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

123. Mixed

     jacob019, Hacker News commenter

     Comment on: OpenAI dropped the price of o3 by 80%Hacker News comment

     Reasoning models use more tokens and their latency is annoying, so the commenter keeps GPT-4.1 for day-to-day work.

     Published Jun 10, 2025self-described: Own day-to-day API use

     Counts toward, pending a recheck

     • Overall signal
     • Latency: Raised as a concern
     • Route by task

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

     Observed on

     o3 and GPT-4.1

124. Positive

     alasano, Hacker News commenter

     Comment on: New tools and features in the Responses APIHacker News comment

     Started an MVP on the Assistants API and migrated to the Responses API easily, handling a few more things in the app.

     Published May 21, 2025self-described: MVP product

     Counts toward, pending a recheck

     • Overall signal
     • Model & API Changes: Discussed without concern
     • Setup & SDK Friction: Discussed without concern
     • Should a team move to the Responses API or stay on Chat Completions?: Responses API

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

125. Positive

     PeterStuer, Hacker News commenter

     Comment on: Ask HN: How much better are AI IDEs vs. copy pasting into chat apps?Hacker News comment

     Stays on OpenAI partly because the commenter has worked up to a high usage tier in production, while another provider still throttles.

     Published May 8, 2025self-described: Own production systems on the API

     Counts toward

     • Overall signal
     • Rate Limits & Reliability: Discussed without concern
     • Should a team standardize on OpenAI or spread work across providers?: Standardize on OpenAI

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

126. Positive

     simonw (Simon Willison), Hacker News commenter

     Comment on: Google Gemini has the worst LLM APIHacker News comment

     Prefers OpenAI's automatic prompt caching, which saves money on traffic spikes with no planning, over explicit breakpoints and hourly cache storage fees elsewhere.

     Published May 4, 2025self-described: Own LLM tools and apps

     Counts toward, pending a recheck

     • Overall signal
     • Cost Efficiency: Positive
     • Developer Experience: Positive

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

127. Mixed

     ericlmtn, Hacker News commenter

     Comment on: Show HN: Daily Jailbreak - Prompt Engineer's WordleHacker News comment

     A daily prompt game hit OpenAI's 30,000 tokens-per-minute rate limit as players arrived, so some attempts errored.

     Published Apr 27, 2025self-described: Daily prompt game

     Counts toward

     • Overall signal
     • Rate Limits & Reliability: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

128. Mixed

     Wowfunhappy, Hacker News commenter

     Comment on: OpenAI releases image generation in the APIHacker News comment

     Reached usage tier 4 after a 30-day wait and US$1,000 in credit purchases, which opened the image API's lower moderation setting.

     Published Apr 24, 2025self-described: Own API account

     Counts toward, pending a recheck

     • Overall signal
     • Rate Limits & Reliability: Discussed without concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

129. Mixed

     jumploops, Hacker News commenter at Magic Loops

     Comment on: OpenAI o3 and o4-miniHacker News comment

     In the commenter's own tests, o3-mini did well on some technical problems but still struggled with long chains of tool calls in a pure agent loop.

     Published Apr 16, 2025self-described: Own agent tests

     Counts toward, pending a recheck

     • Overall signal
     • Tool Calling & Structured Output: Mixed or negative
     • OpenAI models run their own agents

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

     Observed on

     o3-mini

130. Mixed

     pcwelder, Hacker News commenter

     Comment on: GPT-4.1 in the APIHacker News comment

     In quick tests GPT-4.1 struggled with an agentic loop and needed tool calls forced, though its tool use beat Gemini 2.5 Pro's handling of JSON schemas.

     Published Apr 14, 2025self-described: Own agent tests

     Counts toward, pending a recheck

     • Overall signal
     • Tool Calling & Structured Output: Mixed or negative

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

     Observed on

     GPT-4.1

131. Positive

     Simon Willison, independent developer and creator of the LLM command-line tool

     GPT-4.1: Three new million token input models from OpenAI, including their cheapest model yetPersonal blog post, simonwillison.net

     Calls the 1 million input token context a really big deal and estimates GPT-4.1 nano could describe 4,310 photos for just under a dollar.

     Published Apr 14, 2025self-described: Testing the GPT-4.1 family through his llm-openai plugin on launch day

     Not counted

     Its author already has two counted records on this product. Kept in the record; it takes part in no figure.

     Published on

     Engineering Blogs & Reviews

     Who

     Individual builder

     Evidence

     Measured

     Applies to

     Needs a recheck

     Observed on

     GPT-4.1, GPT-4.1 mini, GPT-4.1 nano

132. Mixed

     simonw (Simon Willison), Hacker News commenter

     Comment on: OpenAI's o1-pro now available via APIHacker News comment

     o1-pro was the first model available only through the Responses API, so tools built on Chat Completions, including the commenter's LLM tool, had to add support.

     Published Mar 19, 2025self-described: Own open-source LLM tool

     Not counted

     Its author already has two counted records on this product. Kept in the record; it takes part in no figure.

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     o1-pro

133. Positive

     sagarpatil, Hacker News commenter

     Comment on: New tools for building agentsHacker News comment

     Has run several agents built only on function calling and structured outputs in production for more than a year.

     Published Mar 12, 2025self-described: Several agents in production

     Counts toward

     • Overall signal
     • Tool Calling & Structured Output: Positive
     • OpenAI models run their own agents

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

134. Mixed

     bob1029, Hacker News commenter

     Comment on: New tools for building agentsHacker News comment

     Sees API churn as unhelpful for real products; vendor-managed state proved inadequate, so the commenter uses only Chat Completions with structured outputs and keeps agent state local.

     Published Mar 11, 2025self-described: Own products built on the API

     Counts toward

     • Overall signal
     • Model & API Changes: Raised as a concern
     • OpenAI models run their own agents
     • Should a team move to the Responses API or stay on Chat Completions?: Chat Completions

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

135. Negative

     sippeangelo, Hacker News commenter

     Comment on: New tools for building agentsHacker News comment

     Finds function calling still hallucinates parameters and schemas and breaks down beyond about five simple functions.

     Published Mar 11, 2025self-described: Own applications

     Counts toward, pending a recheck

     • Overall signal
     • Tool Calling & Structured Output: Mixed or negative

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

136. Mixed

     ilaksh, Hacker News commenter

     Comment on: OpenAI O3-MiniHacker News comment

     On usage tier 5 the commenter still had no API access to o3-mini at launch, since it rolled out only to select tier 3 to 5 customers.

     Published Jan 31, 2025self-described: Own API account on usage tier 5

     Counts toward, pending a recheck

     • Overall signal
     • Rate Limits & Reliability: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

     Observed on

     o3-mini, usage tier 5