GILD verdict
Standardize on the OpenAI API when GPT models belong inside your own product and you want new models and features on release day, and keep a second route as DoorDash and Grab do with Azure OpenAI.
GPT-6.1 Sol is its best value: the Artificial Analysis Intelligence Index v5.0 (Oct 27, 2026) scores it 2 points below GPT-6 Astra at about a fifth of Astra's cost per task. Before launch, confirm the usage tier, set a project hard spend limit and retention terms, and plan for model retirements.
OpenAI API
AI Model API by OpenAI
+3 MixedFit and limits
Who it fits
- You are building AI features or agents into your own product or internal tools, and have engineers to own the integration.
- You want OpenAI's newest models and API features as they ship, such as mid-turn steering and asynchronous tool calling, which Amazon Bedrock does not offer.
- Your workloads repeat long prompts or can wait, where cache reads (10% of the input rate, 5% on GPT-6.1 Sol and GPT-6.1 Terra) and the 50% Batch and Flex discounts apply.
- High-volume work fits GPT-6 Luna, at US$0.10 input and US$0.50 output per million tokens. Simon Willison moved the public Datasette Agent to Luna on its launch day (Sep 22, 2026).
- You run your own gateway with a second route: DoorDash falls back to Azure OpenAI when OpenAI's endpoint fails, and Wealthsimple's gateway API mirrors OpenAI's request shape so teams can swap providers behind it.
Where it falls short
- You buy model capability and own the product around it, so evaluations and retries are your team's work. OpenAI's Evals platform shuts down on November 30, 2026.
- Monthly usage limits rise with total credit purchases, from US$500 on Build to US$200,000 on Grow; beyond that, capacity comes through Scale Tier, Reserved Tier, or an enterprise agreement.
- Zero Data Retention needs OpenAI's approval, and Batch, Files, vector stores, Conversations, and the Agents API keep application state even with it on.
- The platform changes on a published schedule. The Assistants API shut down on August 26, 2026, and older GPT-5 and o3 snapshots retire on December 11, 2026. Self-serve fine-tuning is closed to organizations that have not used it.
- A newer model can break a working agent. WebPlanning SIM-Ltd's voice agent fell from 98% to 100% successful commands to about 25% when only the model changed to gpt-realtime-2.1, because it skipped the tool calls and answered on its own (Sep 17, 2026).
Implementation reality
Choose the model per task
Test at least two models on your own evaluation set and compare cost per completed task. OpenAI recommends GPT-6 Astra for demanding reasoning and coding. GPT-6.1 Sol targets near-Astra work at lower cost, while GPT-6 Luna targets focused, high-volume tasks. The Artificial Analysis Intelligence Index v5.0 (Oct 27, 2026) backs that split: Astra scores 57 at US$6.24 per task, Sol 55 at US$1.27, and Luna 28 at US$0.15. Reasoning effort and output length affect the bill more than list price.
Design for the cache
Put tool definitions and stable instructions before changing content. Caching is on by default. On GPT-5.6 and later, writes cost 1.25x the input rate for the default 30-minute cache or 2.5x for a 12-hour cache, and reads cost 0.1x, or 0.05x on GPT-6.1 Sol and GPT-6.1 Terra. Check the prompt caching dashboard. A prefix that changes every call pays the write premium without the read discount. Hit rates vary in production. Requesty's gateway served 36% of OpenAI input tokens from cache in April 2026. One Azure customer measured 96% hits on Chat Completions and none on Responses for GPT-5.6 Sol.
Plan capacity and spend
Usage tiers set monthly usage limits and per-model rate limits; on Build, GPT-6 Astra allows 5,000 requests and 1,000,000 tokens per minute. Set an organization or project hard spend limit below your tier's limit. Once traffic passes 1 million tokens per minute, grow it by no more than 50% every 15 minutes to avoid 429 slow_down errors.
Settle retention and region
By default OpenAI keeps abuse monitoring logs for up to 30 days and does not train on API data. Zero Data Retention or Modified Abuse Monitoring requires approval. Regional processing costs 10% more on models released since March 5, 2026, and every region other than the US also requires abuse monitoring controls and a Modified Retention amendment.
What to measure in your pilot
- Track output and reasoning tokens per completed task on GPT-6 Luna for two weeks; if retries lift its cost per task above a quarter of GPT-6.1 Sol's, move that workload to Sol.
- Log cached tokens per request on each endpoint you call during week one; a stable prefix that hits below 50% points to a prompt-order or endpoint problem to fix before scale.
- GILD found no named team's cost split between the Agents SDK and the hosted Agents API; run one agent both ways for two weeks and compare total cost and failed runs.
Sources
- Official documentation
- 19
- Public source
- 11
- OpenAI API docs: Models
- OpenAI API docs: Pricing
- OpenAI API docs: GPT-6 Astra
- OpenAI API docs: GPT-6.1 Sol
- OpenAI API docs: GPT-6.1 Terra
- OpenAI API docs: GPT-6 Luna
- OpenAI API docs: Prompt caching
- OpenAI API docs: Batch API
- OpenAI API docs: Flex processing
- OpenAI API docs: Fast mode
- OpenAI API docs: Rate limits
- OpenAI API docs: Spend limits
- OpenAI API docs: Data controls in the OpenAI platform
- OpenAI API docs: Deprecations
- OpenAI API docs: Evaluation best practices
- OpenAI API docs: Agents
- OpenAI API docs: Agents API
- OpenAI API docs: MCP servers
- OpenAI API docs: OpenAI on Amazon Bedrock
- Artificial Analysis: LLM Leaderboard (Oct 28, 2026)
- Artificial Analysis: Changelog (Oct 28, 2026)
- Artificial Analysis: GPT-6 Astra analysis (Oct 28, 2026)
- Artificial Analysis: GPT-6.1 Sol analysis (Oct 28, 2026)
- Artificial Analysis: GPT-6 Luna analysis (Oct 28, 2026)
- Requesty: Prompt-cache hit rate per provider, April 2026 (May 9, 2026)
- OpenAI Developer Forum: Prompt caching on Azure GPT-5.6 Sol, 96% hits on Chat Completions and none on Responses (Jul 12, 2026)
- InfoQ: Building a GenAI Platform at DoorDash, talk by Swaroop Chitlur and Siddharth Kodwani (Oct 3, 2026)
- InfoQ: Mandy Gu on Wealthsimple's gen AI platform and LLM gateway (Feb 6, 2025)
- Grab Engineering: Grab AI Gateway, connecting Grabbers to multiple GenAI providers (Feb 19, 2025)
- Ploy: Migrating a production AI agent to GPT-5.6 (Jul 9, 2026)