Before Claude API traffic reaches production, pick each model from your own evaluations, order every prompt so its stable prefix caches, and set spend limits and retention. Anthropic's documentation and the public accounts of teams running Claude agree on that order.
When you need this
- A feature or agent built on Claude is moving from prototype to production traffic.
- The monthly API bill grows faster than usage, and nobody can say which workload drives it.
- A model you depend on is deprecated, or a new model ships and the team has to decide whether to switch.
Evaluation criteria
Setup: one workspace per workload
Can you see and cap spend and rate limits for each product or team?
Look forGive each workload a workspace with its own API keys, rate limits, and spend limit (up to 100 workspaces per organization by default). Keep the organization spend limit below your tier's monthly cap: US$500 on Start, US$1,000 on Build, US$200,000 on Scale.
Model choice: evaluations per task
Which model passes your own test set at the lowest cost per completed task?
Look forA held-out test set run on two adjacent models, with effort tuned before switching, as Anthropic recommends. OutcomeOps found Haiku 4.5 gave a more complete answer than Sonnet 4.6 on its retrieval workload. Qodo found GPT-4.1 ahead of Claude 3.7 Sonnet on 200 pull requests.
Cost control: build for the cache
Does the stable part of every prompt sit before anything that changes per request?
Look forTool definitions and the system prompt first, timestamps and per-user context last, and cache reads tracked per workload. Revefi lifted cache reads to 99% of input tokens by moving one timestamp. A Bedrock user paid US$37,901.73 when caching failed without an error.
Rollout: batch what can wait
Which workloads can accept results within 24 hours?
Look forEvaluations, backfills, and bulk classification on the Message Batches API at 50% off, where most batches finish within an hour. Interactive agent turns stay synchronous, because one builder measured 90 to 120 seconds per turn through batch.
Governance: retention before launch
Does your retention arrangement cover every feature and model you call?
Look forZero data retention enabled per organization where you need it, with the features outside it mapped: Batch, the Files API, code execution, the MCP connector, and Managed Agents. Fable or Mythos models used only where 30-day retention is acceptable.
Review practice: re-run evaluations on every model change
What happens to your feature when a model you use changes or retires?
Look forModel identifiers in one configuration file, evaluations re-run before every switch, and retirement dates tracked against Anthropic's 60-day minimum notice. In their own tests, one team on Hacker News found Sonnet 5 worse than Sonnet 4.6 on complex work, and another found Haiku 5.5 trailing Haiku 4.5. Newer models also reject older request shapes. Per Anthropic's release notes, Sonnet 5.5 returns a 400 error on forced tool use (tool_choice any or tool). Haiku 5.5 can return a 400 error on budget_tokens, assistant prefill, temperature, top_p, top_k, and computer_20250124. API version 2026-10-21, the default in Python SDK 2.0.0 and TypeScript SDK 1.0.0, removes temperature, top_p, and top_k on every model.
Questions to ask vendors
- Which usage tier is our organization on, and what does a move to Scale or a Custom arrangement require?
- Which of the features we use are eligible for zero data retention, and which retain data?
- Is any model we use deprecated or scheduled for retirement, and what is the recommended replacement?
- If we resell or proxy Claude to our own customers, do we send a user profile with each request? That way an Anthropic restriction or ban applies to one end customer and spares the rest of our organization.
- Does a private offer or committed-use discount apply on the platform we plan to buy through?
Red flags
- One API key and one workspace shared by every product, so nobody can trace spend or rate limits to a workload.
- No evaluation set, so benchmark headlines or list price decide a model switch.
- A timestamp, user ID, or tool list that changes per request placed at the top of the prompt.
- Retry logic that resends immediately on 529 overloaded errors with no backoff.
- No organization spend limit below the tier's monthly cap.
Products in this piece
Sources
- Official documentation
- 11
- Public source
- 7
- Claude API docs: Workspaces
- Claude API docs: Rate limits
- Claude API docs: Choosing a model
- Claude API docs: Define success criteria and build evaluations
- Claude API docs: Prompt caching
- Claude API docs: Batch processing
- Claude API docs: API and data retention
- Claude API docs: Model deprecations
- Claude API docs: Errors
- Claude API docs: Claude Platform release notes
- Claude API docs: Versioning
- OutcomeOps: You're Probably Using the Wrong Bedrock Model (Apr 1, 2026)
- Qodo: We benchmarked GPT-4.1 vs Claude 3.7 (Apr 13, 2025)
- Revefi: How Revefi Reduced Its Data Agent's Spend by 90% (Sep 23, 2026)
- Hacker News: US$38k AWS Bedrock bill caused by a simple prompt caching miss (Apr 28, 2026)
- Eran Sandler: Batch API is terrible for one agent. It might be great for a fleet. (Apr 27, 2026)
- Hacker News comment on moving customer workloads from Sonnet 4.6 to Sonnet 5.5 (Oct 7, 2026)
- Hacker News comment on evals and pairwise tests of Haiku 5.5 (Oct 7, 2026)