Together AI Signals
What teams building AI features on Together AI report, with every finding tied to its evidence.
Read the findingsWhat the evidence says
Practitioner sentiment, all practitioners
−49 Negative from 40 sources that take a side
How this figure is madeNegative overall signal, net 49 points negative
From 35 authors in 36 discussions. Run a pilot on your own prompts at expected concurrency to measure time to first token, output speed, error rate and cost per million tokens.
- Analyzed
- 46 total sources
- Top praise
- For all practitioners, no praise theme reaches 5 sources from 3 authors.
- Top concern
- Tool Calling & API Compatibility (9 of 9 sources) See concern evidence
Jan 2025 to Oct 2026.
41 sources from practitioners, by month and sentiment.
The read leans negative. Developers like pay-per-token access to open models on a US host and its speed against some hosts, and report tool-calling errors, retired serverless models, outages and credit problems.
Positive, mixed and negative sources
The score counts them as 5 positive, 8 mixed and 22 negative votes, one per author. See How the numbers work.
See the evidenceBelow: 3 concerns.
Findings
Counted for All practitioners, from 41 sources. Change group
What practitioners praise
Every capability practitioners describe positively, ranked by how many sources say so. Open one to read who said it.
For all practitioners, no capability reaches 5 sources from 3 authors.
What concerns them
Every concern practitioners raise, most raised first.
Tool Calling & API CompatibilityAll 9 sources on it raise it as a concernTool calls, structured output and request parameters that fail or behave differently on Together than on other hosts.
Counted for All practitioners.
Negative
Josh Heitzman (joshheitzman), software engineer and creator of the FluxImp agent harness
Hacker News comment on running DeepSeek-V4-Flash-0731 through Kilo Code and his own harnessHacker News comment
With DeepSeek-V4-Flash-0731 on together.ai, Kilo Code had a lot of trouble making successful edits, while the same model used the tools of their own harness just fine. On DeepInfra the two harnesses showed little difference.
Negative
Branko Radovanović (brankoradovanovic-mcom), Multicom d.o.o.
bug: max_output_tokens setting doesn't work with Together.aiGitHub issue, janhq/jan
In Jan, every prompt to Together.ai with a Max Output Tokens value fails with Bad Request, whatever the value, while other parameters work. His own diagnosis is that Together.ai doesn't expect the parameter and rejects the request.
Negative
Juan Pedro Michelini Jorge (juanmichelini), MTS @ Openhands: Evals, benchmarks, leaderboards, harness
Change model qwen/qwen3-coder-next from together.ai to openrouterGitHub issue, OpenHands/software-agent-sdk
The OpenHands evals team tried Qwen3-Coder-Next through LiteLLM's together_ai route and got LiteLLM's error that the model provided by together.ai does not support tools, so the team moved the model to OpenRouter.
Negative
Roman Vashurin (rvashurin), research engineer at MBZUAI
seed must not be set when best_of is greater than 1GitHub issue, togethercomputer/together-python
Asking Together's chat API for several completions with a seed set returned an input validation error: seed must not be set when best_of is greater than 1.
Negative
wleonie, collaborator on debatelab/evidence-seeker
Fix access to Llama-3.3-70B-Instruct modelGitHub issue, debatelab/evidence-seeker
Llama-3.3-70B-Instruct, with Together.ai as the inference provider, returned a 422 error, grammar is not valid: failed to compile grammar, at four steps of the evidence-seeker workflow.
Negative
David Else (David-Else), software developer, Else Web Development
Agent seems unable to do anything using together.aiGitHub issue comment, anomalyco/opencode
With Qwen3 Coder and Kimi K2 on together.ai, the opencode agent said it would act and then did nothing, and DeepSeek R1 returned raw tool-call tokens. An opencode contributor thought Together was not parsing out tool calls in a compatible manner, and he asked for another host.
Negative
Sagar (sagar-catomz), GitHub
Cannot use GPT-OSS-120b with ChatTogether and tools for create_react_agentGitHub issue, langchain-ai/langchain-together
create_react_agent with ChatTogether on gpt-oss-120b works without tools, and once tools are added it fails with a 400 error, The decoder prompt cannot be empty.
Negative
Jarett (JarettForzano), Vals AI
Llama 4 Scout on together_ai returns tool calls in the message textGitHub issue, BerriAI/litellm
LiteLLM's model list says Llama 4 Scout supports tool calling, but on together_ai the tool call came back as text inside the message content, with tool_calls empty, and he believed it was consistent. Another commenter called it a Together bug.
Negative
Shivanshu Gupta (Shivanshu-Gupta), NLP and AI researcher, Adobe on his GitHub profile
JSON mode on Llama 3.1 and 3.3 70B Turbo returns invalid JSONGitHub issue, togethercomputer/together-python
With JSON mode on Meta-Llama-3.1-70B-Instruct-Turbo and Llama-3.3-70B-Instruct-Turbo, the model regularly failed to produce valid JSON, often adding random tokens toward the end. A Together contributor said the issue was with the API and asked him to open a support ticket.
GILD's fact checks, which don't count as sources
Checked Nov 1, 2026.Together's API is compatible with the OpenAI REST API and SDKs across chat, completions, vision, image generation, text-to-speech and embeddings. An app using the OpenAI client points at Together by changing the API key and base URL. OpenAI compatibility
Open them in the recordPython SDK & Integration PackagesAll 5 sources on it raise it as a concernThe together Python SDK's install size, client errors and command-line bugs, and integration packages that lag their frameworks.
Counted for All practitioners.
Negative
Santiago Martinez Bragado (smartinezbragado), Traza
Release a langchain-together 1.0 without dependency conflictsGitHub issue, langchain-ai/langchain-together
langchain-together 0.3.x pinned langchain-openai below 0.4, so Traza's workspace could not resolve once it needed langchain-openai 1.1. The package in the langchain-ai organization lagged LangChain 1.x, and he asked for a 1.0 release.
Negative
Matthias Geihs (matthiasgeihs), cryptography engineer and systems architect at Consensys
The together library adds over 100MB of dependenciesGitHub issue, togethercomputer/together-python
Matthias Geihs found that adding the together library added over 100MB in dependencies after his small shared VMs ran out of memory, and replaced it with a REST call. In February 2026 a Together contributor said the 2.0 SDK should be much smaller.
Negative
Venkat (venkatasg), assistant professor of computer science at Ithaca College
Unclosed client session errors from the async client after a few batch callsGitHub issue, togethercomputer/together-python
On Build tier 3, having spent US$100, Venkat kept hitting unclosed client session errors after only a couple of batch calls with the async client. In February 2026 a Together contributor said its 2.0 SDK reworks the HTTP client and should solve the issue.
Negative
Steve Cosman (scosman), building Kiln (kiln.tech), formerly Apple Photos AI
Remove the pyarrow dependency from the together Python SDKGitHub issue, togethercomputer/together-python
Steve Cosman found the together Python SDK required pyarrow, which he says is about 80MB, for a parquet check most users never need, and asked for it to be removed.
Mixed
Carl Silva (silvacarl2), Sphinx Medical Technologies
together fine-tuning list crashes with a validation errorGitHub issue, togethercomputer/together-python
together fine-tuning list crashed with a pydantic validation error on the data the API returned. Two days later a Together contributor said it was a bug in the API, now resolved with no client update, and he replied that their stuff rocks.
Errors, Rate Limits & Outages5 of 6 sources on it raise it as a concernStream errors, cut-off answers, 429s at peak traffic, and outages.
Counted for All practitioners.
Raised as a concern, 5
Mixed
karan R., G2 reviewer
G2 review of Together AI on AWS Marketplace by karan R.G2 review on AWS Marketplace
Praises Together's latency and quick access to newly launched models for voice agents, still below Cerebras. The API's requests-per-minute limit is harming their business: their traffic comes in bursts, and at peak times they run into 429 errors.
Negative
GT95, Reddit (r/openrouter)
OpenRouter keeps routing calls to failing providerReddit post, r/openrouter
Running DeepSeek V4.1 Flash through OpenRouter in the Hermes agent, the work stopped several times with API call failed after 3 retries: Upstream error from Together: Stream error: h2 protocol error.
Negative
Mo Moustafa, AI Product Engineer for Startups, runs Olly
So you want to use OpenRouter? (hollow completions)Engineering blog post, mmoustafa.com
Running Olly on open models through OpenRouter, he saw one host return 200 responses with null content, null reasoning and no usage object in July 2026, and a month later Together did the same on the DeepSeek 0731 checkpoint.
Negative
HDBaseT, Hacker News
Hacker News comment on together.ai outages and latencyHacker News comment
One commenter's impression that together.ai seems fairly unreliable, with constant outages or higher than normal latency. The comment states no use of their own.
Negative
Infamous_Sector_6411, Reddit (r/LocalLLaMA)
Having a problem with the together.ai apiReddit post, r/LocalLLaMA
Bought €15 of Together credits to power AnythingLLM. Whenever they try a more complex prompt the model abruptly stops, the same through the aichat CLI, with max_tokens set high, and they ask whether OpenRouter would have been the better pick.
Discussed without concern, 1
Positive
abnormal_human, Reddit (r/LocalLLaMA)
Reply on truncated answers from the together.ai apiReddit comment, r/LocalLLaMA
abnormal_human says they have pushed more than 1 billion tokens (1B+ tokens) through Together across all kinds of use cases without running into the cut-off answers, and does not think the API is broken.
GILD's fact checks, which don't count as sources
Checked Nov 1, 2026.Together's serverless performance is best-effort. When demand is high, Together may limit requests, and you may receive 429 Too Many Requests or 503 Service Unavailable responses. Serverless rate limits
Checked Nov 1, 2026.On Together, purchased rate limits under an active Enterprise or Scale contract stay in place until the contract expires, and when a model is in especially high demand Together may apply custom rate limits or access restrictions to that model. Serverless rate limits
Open them in the record
Where the evidence agrees
Statements this group's sources agree on, then the ones close to agreement.
For all practitioners, no statement reaches 5 sources from 3 authors.
Where it disagrees
Questions the sources split on, with the count on each side.
For all practitioners, no question reaches 5 sources from 3 authors, with 2 on each side.
The evidence
Whose experience this is
The three groups, then every practitioner by the context they work in, and how emphasis changes with scale.
Three groups drawn from the same sources. Choose a group to select whose experience counts. Within that group, each author counts once in the practitioner sentiment, with their latest view. Signals open on All practitioners. Teams and leaders have 1 accounts; GILD prints their sentiment from 20.
Teams and technical leaders
1 sources. This group counts the accounts of teams and companies running AI features in production, and the leaders choosing their models and providers.
Individual builders
40 sources. This group counts developers building on the platform on their own.
All practitioners
41 sources. This gives the broadest read, but combines solo projects with production teams that can reach different conclusions.
41 sources from practitioners, by the context they work in. Filled rows count for this group.
- Individual builderDevelopers building on the platform on their own40 sources, counted here
Emphasizes Tool-calling errors in coding agents, retired serverless models, and prepaid credits
- Small product team2 to 10 engineers, or an unnamed startup teamNo sources from this setting yet
- Product or platform team11 to 50 engineers, or a named company's team1 sources, counted here
Emphasizes An evaluation team moving a model off Together after a tools error
- Technical leadershipLeaders setting model and platform choices across teamsNo sources from this setting yet
- Organization-wideEnterprise or company-wide AI platformNo sources from this setting yet
What changes with scale. Individual developers use Together for open models billed per token and report tool-calling errors, retired models and credit problems. The one team account moved a model to another host after a tools error.
Where the sources come from
Where this group's sources were published.
Counted for All practitioners, from 41 sources. Change group
46 sources for this group, 5 of them documentation
- Developer Communities35, 76% of 46 sources
- Practitioner Blogs & Reviews6, 13% of 46 sources
- Product Documentation5, 11% of 46 sources
Earlier evidence remains when it still applies. Records about a model Together no longer serves, or a bug a later release may have fixed, carry Still applicable only where the account still informs a buyer.
See sources by originThe record
Who each source is, how it knows, whether it still applies and what it counts toward.
Newest first, on its own page. Every count above opens its sources there.
Read all 46 sourcesAbout this read
What changed, and how we count
What changed
- Nov 1, 2026: first edition, built only from records a second reviewer checked against stored copies of each page on Nov 1, 2026.
- Every account links to its public source and names its author and publication date.
- A finding shows only when at least 5 sources from 3 different authors discuss it. A disagreement also requires at least 2 sources on each side.
- Counts practitioners who describe their own use of Together, plus two commenters' stated views with no use of their own. Together's own customer stories are excluded.
- Checks the pricing, rate-limit, data-handling and integration findings against Together's documentation on the review date.
Method
A running review of what teams building AI features on Together AI report, from individual builders to product and platform teams. Each finding links to its sources, and each conflict shows both sides.
Counts cover only the sources GILD reviewed.
Who counts
- Teams and technical leaders
- Small product team, Product or platform team, Technical leadership, Organization-wide
- Individual builders
- Individual builder
- All practitioners
- Everyone in the teams and leaders and individual builders groups
Choose a group to select whose experience counts. Within that group, each author counts once in the practitioner sentiment, with their latest view. GILD files each source in a group from how the account describes its author. A role or employer marked self-described is what the source states about themselves. Documentation checks facts for every group and never counts toward sentiment.
How the numbers work
- Overall signal. Positive votes minus negative ones, per 100 votes from the sources that take a side. Each author counts once, with their latest view, whoever they are and wherever they posted. The older weighted reading, where accounts GILD matched to a named person count three times when GILD lists the author among technical leaders and twice otherwise, is shown beside it as a second view. Where an author wrote more than one source, the line beside the split gives the votes, one per author, so the score follows from it. When one author's sources from the same day disagree, that author counts as mixed, and a half point rounds away from zero. Sources with no opinion are listed and not counted. Strong positive is +50 or higher, positive +20 to +49, mixed −19 to +19, negative −20 to −49, strong negative −50 or lower.
- Weighted reading, second view. Counted the older way, where an account GILD matched to a named person counts two or three times, all practitioners read −50 strong negative (method sentiment/S1). GILD printed that number as "GILD score" until October 29, 2026; most of those matches rest on a name or a self-described role.
- Unscored. A group gets a score only with 20 sources from 15 authors. Below that, one source moves the score about 5 points, so the read says "Unscored until 20 sources from 15 authors" and shows its counts.
- Independence. A source whose author works for the vendor, for one of its partners or programs, or for a competitor, and a page the vendor published or ran, is listed with that tie and counts toward nothing. One author counts on at most two records per product. The line under the score says how many authors and how many separate discussions the counted sources come from: comments in one thread are one discussion.
- Praise and concerns. How many of the sources that discuss a theme praise it or raise it.
- Consensus. A statement joins when 20% of the group's sources support it and fewer contradict it. Contradicting sources are counted separately. Below that line a statement shows as building. At 40% or more it reads Strong agreement; below, Recurring point.
- Disagreements. The count on each side. GILD names no winner and gives no share of all users.
- Minimum evidence. A finding needs 5 sources from 3 authors; a disagreement also needs 2 on each side. Smaller counts stay in the table below, marked under the minimum.
Labels on a source
A source's date and whether it still applies are separate: a recent post can describe an old version, and an old one can still hold.
- Documented
- Stated in official documentation.
- Reported
- A practitioner's account of their own use.
- Measured
- Backed by the source's own measurements, such as a comparative test or tracked team metrics.
- GILD analysis
- GILD's check of a claim against official documentation.
- Current
- Describes the product as it is now.
- Still applicable
- Older, and the behavior or lesson still holds.
- Needs a recheck
- May depend on a plan, version or setting that changed since it was written. It keeps counting until GILD rechecks it, since the rest of the account usually holds, and its card says so.
- Out of date
- No longer accurate. Kept in the record; counts toward nothing current.
Flagged for a recheck, of the sources that take a side: teams and leaders 0 of 1, individual builders 0 of 39, all practitioners 0 of 40.
Limits
- These figures describe reviewed sources; they do not represent all Together AI customers. Most sources are Hacker News comments, Reddit posts and GitHub issues, and bug reports tilt toward problems. The two G2 reviews shown on Together's AWS Marketplace listing were posted the same day as most reviews on that page, so they count as one publisher. Reddit authors' profiles could not be retrieved, so their identity stays unstated unless a page they control names it. Accounts published before 2025 are left out.
- Sources run from January 2025 to October 2026 and name the model they used where they state one: DeepSeek V4 Flash and V4.1 Flash, GLM-5.3 and GLM-5.3-Flash, gpt-oss-120b, Llama 3.3 70B and others. One names a Together usage tier (Build tier 3), and none names a contract.
Every count, and the sources behind it
15 findings for 3 groups
Each row opens its sources for the group you are reading.
| Finding | Teams and technical leaders | Individual builders | All practitioners | Sources |
|---|---|---|---|---|
| Current readCurrent sentiment | 0 positive, 0 mixed, 1 negative | 7 positive, 7 mixed, 25 negative | 7 positive, 7 mixed, 26 negative | Sources |
| PraiseOpen Models Behind One API | 0 of 0 positive (under the minimum) | 2 of 2 positive (under the minimum) | 2 of 2 positive (under the minimum) | Sources |
| PraisePay-per-Token Pricing | 0 of 0 positive (under the minimum) | 3 of 3 positive (under the minimum) | 3 of 3 positive (under the minimum) | Sources |
| PraiseSpeed Against Other Hosts | 0 of 0 positive (under the minimum) | 3 of 3 positive (under the minimum) | 3 of 3 positive (under the minimum) | Sources |
| PraiseUS Hosting of Open Models | 0 of 0 positive (under the minimum) | 2 of 2 positive (under the minimum) | 2 of 2 positive (under the minimum) | Sources |
| PraiseZero Data Retention | 0 of 0 positive (under the minimum) | 1 of 1 positive (under the minimum) | 1 of 1 positive (under the minimum) | Sources |
| PraiseFine-Tuning & GPU Clusters | 0 of 0 positive (under the minimum) | 1 of 1 positive (under the minimum) | 1 of 1 positive (under the minimum) | Sources |
| ConcernTool Calling & API Compatibility | 1 of 1 raise it (under the minimum) | 8 of 8 raise it | 9 of 9 raise it | Sources |
| ConcernErrors, Rate Limits & Outages | 0 of 0 raise it (under the minimum) | 5 of 6 raise it | 5 of 6 raise it | Sources |
| ConcernModel Retirements & Serverless Availability | 0 of 0 raise it (under the minimum) | 3 of 3 raise it (under the minimum) | 3 of 3 raise it (under the minimum) | Sources |
| ConcernPrepaid Credits, Refunds & Free Models | 0 of 0 raise it (under the minimum) | 4 of 4 raise it (under the minimum) | 4 of 4 raise it (under the minimum) | Sources |
| ConcernPython SDK & Integration Packages | 0 of 0 raise it (under the minimum) | 5 of 5 raise it | 5 of 5 raise it | Sources |
| ConcernPrice & Speed Against Other Hosts | 0 of 0 raise it (under the minimum) | 3 of 3 raise it (under the minimum) | 3 of 3 raise it (under the minimum) | Sources |
| ConcernOutput Quality & Quantization | 0 of 0 raise it (under the minimum) | 3 of 3 raise it (under the minimum) | 3 of 3 raise it (under the minimum) | Sources |
| Current readEvidence mix | 6 sources | 45 sources | 46 sources | Sources |
Fact checks
GILD's checks of claims against official documentation, kept apart from the source counts.
- Open Models Behind One API. Together's serverless catalog lists chat models by API model string, including zai-org/GLM-5.3, openai/gpt-oss-120b and moonshotai/Kimi-K3. Serverless and dedicated model inference support different sets of models. Serverless models, checked Nov 1, 2026
- Open Models Behind One API. Together's provisioned throughput reserves capacity for a selected model or model family on its managed inference infrastructure, for GLM-5.3, GLM-5.3 Flash, DeepSeek V4.1 Flash, GLM-5.2 and Kimi K3, with a reliability commitment that at least 99% of eligible requests complete successfully each month. Provisioned throughput, checked Nov 1, 2026
- Open Models Behind One API. Together's API is compatible with the OpenAI REST API and SDKs across chat, completions, vision, image generation, text-to-speech and embeddings. An app using the OpenAI client points at Together by changing the API key and base URL. OpenAI compatibility, checked Nov 1, 2026
- Open Models Behind One API. The Vercel AI SDK's @ai-sdk/togetherai provider gives it native access to Together AI models. Vercel AI SDK quickstart, checked Nov 1, 2026
- Open Models Behind One API. Together's third-party integrations page lists LiteLLM: LiteLLM 1.100.0 and later ships official Together AI support, with a model registry that syncs against Together's serverless catalog daily. Third-party integrations, checked Nov 1, 2026
- Open Models Behind One API. Together's third-party integrations page lists LangChain's langchain-together package, which provides chat models and embeddings for Together. Third-party integrations, checked Nov 1, 2026
- Pay-per-Token Pricing. Together's serverless models are called through a shared per-token API with no provisioning, no replicas to size and no minimum cost. Serverless, provisioned throughput and dedicated model inference all use the same inference APIs, so apps work on any deployment mode without code changes. Serverless models and inference overview, checked Nov 1, 2026
- Pay-per-Token Pricing. Together's batch API runs many independent inference requests asynchronously from a single uploaded JSONL file. Together bills each successful response in the output file, failed requests aren't billed, and small batches (under 1,000 requests) typically finish in minutes within a 24-hour maximum window. Batch inference, checked Nov 1, 2026
- Speed Against Other Hosts. Together's dedicated model inference serves a model on reserved hardware with no hard rate limits beyond the selected hardware's capacity and the autoscaling bounds, and can serve a model fine-tuned from a supported base model. Dedicated endpoints overview, checked Nov 1, 2026
- US Hosting of Open Models. Models from third-party authors such as DeepSeek, Qwen and Mistral that Together hosts run on Together's own infrastructure and do not call out to the model author. DeepSeek models are hosted in Together's North America data centers. Privacy and security, checked Nov 1, 2026
- US Hosting of Open Models. For regions beyond the US serverless endpoint, Together says to use a dedicated endpoint and pin its hardware to that region. Placement profiles control the regions where deployments may be scheduled, and each instance type lists its regions. Serverless overview, checked Nov 1, 2026
- Zero Data Retention. By default Together stores prompts and responses and may use them for product improvements. Organization admins can turn storage off to enable zero data retention. Data sharing for training other models is opt-in and not enabled by default. Privacy and security, checked Nov 1, 2026
- Zero Data Retention. Some Together models are offered as passthrough: Together forwards prompts and responses directly to the upstream provider, and data is handled under that provider's own data policy. A separate organization-level toggle controls passthrough models, independent of the training opt-in. Privacy and security, checked Nov 1, 2026
- Fine-Tuning & GPU Clusters. Together AI fine-tuning handles the full lifecycle of data upload, training, hosting and inference on a dedicated endpoint, with low-rank adapter (LoRA) training as one approach and full fine-tuning, which updates every weight, as the other. Fine-tuning overview, checked Nov 1, 2026
- Fine-Tuning & GPU Clusters. Together GPU clusters provide dedicated multi-node GPU infrastructure for training, fine-tuning and other large-scale AI workloads. Clusters provision in minutes with InfiniBand networking and persistent shared storage. Every cluster runs on Kubernetes. GPU clusters overview, checked Nov 1, 2026
- Tool Calling & API Compatibility. Together's API is compatible with the OpenAI REST API and SDKs across chat, completions, vision, image generation, text-to-speech and embeddings. An app using the OpenAI client points at Together by changing the API key and base URL. OpenAI compatibility, checked Nov 1, 2026
- Errors, Rate Limits & Outages. Together's serverless performance is best-effort. When demand is high, Together may limit requests, and you may receive 429 Too Many Requests or 503 Service Unavailable responses. Serverless rate limits, checked Nov 1, 2026
- Errors, Rate Limits & Outages. On Together, purchased rate limits under an active Enterprise or Scale contract stay in place until the contract expires, and when a model is in especially high demand Together may apply custom rate limits or access restrictions to that model. Serverless rate limits, checked Nov 1, 2026
- Model Retirements & Serverless Availability. Together's serverless catalog lists chat models by API model string, including zai-org/GLM-5.3, openai/gpt-oss-120b and moonshotai/Kimi-K3. Serverless and dedicated model inference support different sets of models. Serverless models, checked Nov 1, 2026
- Prepaid Credits, Refunds & Free Models. Together's zero-balance cut-off does not guarantee an exact spending cap: metering has a short delay, so some usage can arrive after the balance reaches zero and the customer remains responsible for it. API keys do not currently support independent spending limits, and organization and project budgets and spend alerts are planned features that are not currently available to configure. Billing spend controls, checked Nov 1, 2026