Key findings
- Ai2 switched its OLMo Playground API to a TogetherAI engine in April 2024 and removed it in November 2024, with no reason stated.
- MLCommons' benchmark scoring broke when Together stopped supporting one annotator model in May 2025 and when another returned a non-serverless error in March 2026.
- GoodParty.org moved its non-streaming LLM calls off Together in June 2026 after incidents it traced to its own fallback model and credit balance.
- Developers report tool-calling errors on Together through Kilo Code, opencode, LangChain and LiteLLM.
Three named organizations ran Together AI in their own code: Ai2 replaced its InferD serving for its OLMo playground in 2024, MLCommons scores a safety benchmark with annotator models it calls there, and GoodParty.org moved its non-streaming LLM calls to one other provider in 2026. Developers in public threads like open models billed per token on a host that runs them itself, and report tool-calling errors, retired serverless models and prepaid credit problems.
Named organizations
Ai2's pull request of April 16, 2024 says the team implemented TogetherAI as a replacement for InferD, and notes that Together returned the logprob only for the chosen token while streaming. Ai2 removed the integration on November 11, 2024 without stating a reason.
MLCommons engineers filed that Together no longer supported Mixtral-8x22B-Instruct-v0.1 in May 2025, and that LlamaGuard-2-8b had become non-serverless in March 2026. GoodParty.org's pull request of June 26, 2026 migrated its LLM layer off Together onto Anthropic to run one provider.
What developers report
Practitioners choose Together to run open models such as GLM-5.3 on a US host billed per token. One measured gpt-oss-120b on Together and Cerebras, where Cerebras answered sooner and cost about 2.1 times as much per long-prompt run. Others report coding agents that fail to call tools, serverless models deprecated by email notice, and out-of-credits notices while using free models.
Methodology
Built on Nov 1, 2026 from records a second reviewer checked against stored copies of each page: three deployment accounts from Ai2's, MLCommons' and GoodParty.org's own GitHub repositories, and the public practitioner accounts behind Together AI's Signals page. Vendor case studies are excluded. Together AI's documentation is cited for its own terms.
Products in this piece
Sources
- Public source
- 11
- Official documentation
- 4
- Ai2, allenai/olmo-api pull request #102: Add TogetherAIEngine and switch to it (Apr 16, 2024)
- MLCommons, modelbench issue #1045: Find replacement for missing annotator model (May 20, 2025)
- GoodParty.org, omni pull request #435: Migrate the LLM layer fully off Together AI onto Anthropic (Jun 26, 2026)
- Sam Mirazi: Same model, two services, gpt-oss-120b on Cerebras and Together (Oct 25, 2026)
- Hacker News, joshheitzman: Kilo Code edits on together.ai (Sep 17, 2026)
- langchain-ai/langchain-together issue #19: gpt-oss-120b fails once tools are added (Aug 20, 2025)
- OpenHands/software-agent-sdk issue #1899: move qwen3-coder-next from together.ai to OpenRouter (Feb 4, 2026)
- earendil-works/pi issue #6132: Together.ai models to be deprecated July 10 (Jun 28, 2026)
- earendil-works/pi issue #5506: MiniMax-M2.5 doesn't work with serverless inference (Jun 8, 2026)
- r/LocalLLaMA, Wild_King_1035: out-of-credits email while using a free model (Jul 16, 2025)
- r/LocalLLaMA, slippery: Ephemeral chat on together.ai with Zero Data Retention (Feb 7, 2026)
- Together AI docs: Serverless models
- Together AI docs: Privacy and security
- Together AI docs: Billing and credits
- Together AI docs: Serverless rate limits