Key findings
- 28 of the 35 counted sources that discuss trace-level debugging are positive, and every counted team and leadership source that discusses it praises it.
- All 11 counted sources that discuss setup praise it. 8 raise LangSmith's ties to LangChain as a concern.
- The UI and SDK draw the most concerns, raised in 23 counted sources, followed by price in 12 and self-hosting or data control in 8.
- 16 of the 25 counted sources that describe a platform choice picked another tool, and 9 chose or kept LangSmith. Among teams and leaders, 8 of 9 chose another tool. Langfuse drew six of them, and a seventh planned self-hosted Langfuse, then moved to MLflow on October 9, 2026.
- In the one independent test on record, Parlance Labs gave LangSmith Engine 100 hand-labeled production traces in July 2026; it surfaced too few issues to score, and LangSmith's chat agent caught 79.5% of the labeled failures.
- The largest deployments come from customer talks at LangChain's Interrupt conference, which LangChain published on its own channel, so they are listed and not counted. Toyota runs more than 50 agents on its ToyotaGPT platform. Lyft, Cisco, Chime, Etsy, LATAM Airlines, Coinbase, and Spring Health describe production agents they trace and evaluate in LangSmith.
Teams praise LangSmith's trace-level debugging, and setup rarely stops anyone. Yet more of the accounts that describe a platform choice picked another tool than stayed, citing self-hosting, where trace data may live and LangSmith's ties to LangChain. GILD read English-language public accounts published from January 2025 through October 2026.
What tracing found
Positive accounts name a failure that traces exposed. LATAM Airlines traced Concierge from launch. Of messages classified out of scope, 13% of total traffic, 95% were valid questions about check-in, baggage, and LATAM Pass benefits. The traces showed a gap in the product, and the model was working. Toyota's AI engineering lead reads each query's route and tool calls in the trace and skips log searches. A solo developer uses traces across a 10-node LangGraph RAG pipeline to distinguish parsing, retrieval, reranking, and generation failures.
Traces miss some failures. One developer found every turn of a verification flow looked fine in LangSmith while the whole session failed. A B2B team saw every tool call return 200 while the agent pulled the wrong context for a client's data. A team whose agents read extracted document fields saw normal traces over bad input.
How teams evaluate with it
Detailed accounts include reviewers outside engineering. Spring Health clinicians write rubrics and review guardrail flags in annotation queues. Non-technical staff tune prompts and start evaluations. Chime's legal and compliance partners grade copilot answers beside the LLM judge's verdict. Lyft's operations team labels production traces that judges fail.
Others tie evaluation to release. Etsy built a CLI on the LangSmith APIs that generates simulated datasets, splits them, and aligns its judge before launch, and took its gifting assistant to beta in six weeks. Weber Shandwick checked a re-architected agent against its LangSmith quality baselines before moving other agents to the new design, and one startup runs LangSmith evals in GitHub CI.
Parlance Labs, which states no vendor ties, tested LangSmith Engine in July 2026. On 100 production traces labeled by a domain expert, LangSmith Engine surfaced too few issues to score, so the authors scored LangSmith's chat agent: 79.5% recall and 77.5% precision, against 91.0% precision for Arize's Alyx. The chat agent cited the exact message behind each real failure, yet several wrong findings were marked high severity.
Individual developers are less happy with evaluation. 6 of the 12 who discuss it are critical. In Hamel Husain's panel, where three evals tools completed the same assignment, LangSmith's annotation queue and trace-to-playground flow drew praise and its UI felt cluttered. One developer's sales agent skipped a required question for two days while its LangSmith evals stayed green. Two developers export results to build the comparisons they need.
Where it hurts
UI and SDK problems lead complaints, often in LangChain repository issues. A Python client stopped tracing until restart. Another report found memory growth only with tracing enabled. One OpenAI Agents SDK run appeared as three traces, and a Playground message box took about 60 seconds to add. On Reddit, one developer measured LangSmith at about 2 GB of their app's 4 to 4.5 GB of memory. Other issues report token and cost totals that read 0 or double the real figure, and streamed Anthropic replies logged twice.
Pricing complaints follow trace volume and retention. A forum user's extended-retention limit blocked new experiments. One developer archives to their own S3 storage to avoid longer-retention fees. Reddit users describe the cost as too high for side projects. Several accounts predate the September 2026 reduction from 400 to 180 days of SaaS extended retention.
Why teams choose another platform, or stay
Self-hosting and distance from LangChain are the reasons given for leaving. A small team that spent three months comparing Galileo, Arize, Langfuse, and LangSmith picked Langfuse because LangSmith is tied to the LangChain way of building and self-hosting needs Enterprise. NonBioS, which had removed LangChain from its code, chose on-prem Langfuse. digithings.ai planned self-hosted Langfuse to replace LangSmith across its LangGraph workloads, then dropped that plan on October 9, 2026 for Cloudflare-native tracing with MLflow, keeping LangSmith until a parity run. One developer ruled LangSmith out for multi-agent tracing because the self-serve plans are cloud-only. In October 2026 Cue Labs retired LangSmith for Langfuse to keep customer messages in its own boundary, and an insurance platform's team did the same for its EU-resident cluster. GAIA ran a self-hosted Langfuse beside LangSmith from May 2026 after its development workspace kept hitting the monthly trace limit.
Teams that stay value tracing and evals, with fewer services to operate. Apollo.io keeps every product eval in LangSmith and counted that as a reason to rebuild its assistant on LangChain's Deep Agents. Kyle Rush's team chose LangSmith over Langfuse because Rush already knew it. Another developer replaced self-hosted Langfuse with hosted LangSmith and cut a local agent template from about 47 containers to 14. monday.com runs an LLM judge in CI before each Feed agent deploy and keeps one LangSmith trace per infrastructure-agent run.
What to settle before choosing
Answer these questions with your own traffic before a pilot ends.
- Must traces stay in your own network? If so, price LangSmith Enterprise against self-hosted Langfuse or Phoenix.
- Is the application built on LangChain or LangGraph, or will it be traced through wrappers and OpenTelemetry?
- How many traces a month, how many runs each, and what share must be kept past 14 days?
- Which failures will become dataset examples, and who outside engineering reviews them in annotation queues?
- What does the application do when the tracing service errors or slows down?
Methodology
GILD opened 80 sources on October 12, 2026, 8 on October 25, 2026, 6 on October 26, 2026, and 16 on October 28, 2026. All were published between January 2025 and October 2026. They include Hacker News comments, Reddit posts and comments from r/LangChain, r/LLMDevs and r/AI_Agents read through the Arctic Shift archive, and GitHub issues and pull requests. Others are LangChain Forum posts, named engineering and personal blogs, an independent evaluation, Product Hunt reviews, and recorded talks, including customer talks at LangChain's Interrupt conference. Counts in this piece cover the 97 of those 110 sources that are independent of LangChain and count toward GILD's figures; the other 13, mostly talks on LangChain's own channel, are listed and not counted. All were written by people who chose to post in public. GILD checked product behavior against LangChain's documentation on October 12, 2026.
Products in this piece
Sources
- Public source
- 37
- Official documentation
- 5
- Nico Venegas and Claudio Urbina Lara (LATAM Airlines): How LATAM Airlines Built Intelligent Agents in Aviation, Interrupt 26 (Jun 30, 2026)
- Ravi Chandu Ummadisetti and Kordel France (Toyota): Building ToyotaGPT, Interrupt 26 (Jul 15, 2026)
- r/LangChain comment on attributing failures in a 10-node LangGraph RAG pipeline with LangSmith traces (Jul 26, 2026)
- Hacker News comment on turn-level traces missing a broken flow (Mar 4, 2026)
- r/LangChain comment on successful and failed runs looking identical in LangSmith traces (Jun 30, 2026)
- r/LangChain comment on normal-looking traces over bad extracted fields (Jul 19, 2026)
- Marissa Saunders (Spring Health): The Safety Net Behind the Conversation, LangSmith for Mental Health AI, Interrupt NYC 2026 (Sep 29, 2026)
- Philipp Comans (Chime): Make Legal Write Your Evals, Building Jade, Interrupt 26 (Jun 22, 2026)
- Nick Ung (Lyft): How Lyft Builds Evals That Actually Matter in Production, Interrupt 26 (Jun 15, 2026)
- Derrick Kondo (Etsy): The Etsy Gifting Assistant, From Prototype to Production, Interrupt 26 (Jun 8, 2026)
- Dotan Limon (Weber Shandwick) and Jordan Kamm (Focused): From One-Off Agents to an Agent Factory, Interrupt NYC 2026 (Sep 29, 2026)
- Hannah Craighead on Product Hunt: Review of LangSmith (Jan 27, 2025)
- r/LangChain: Testing LangChain agents beyond LangSmith eval scripts (May 30, 2026)
- r/LangChain comment on exporting LangSmith results for paired query comparisons (Oct 8, 2026)
- langsmith-sdk issue #1630: client silently stops posting traces (Apr 3, 2025)
- langsmith-sdk issue #2009: memory growth with tracing on (Sep 10, 2025)
- langsmith-sdk issue #1619: duplicate traces through the OpenAI Agents SDK (Mar 31, 2025)
- langsmith-sdk issue #2719: extreme lagging on the LangSmith Playground (Apr 15, 2026)
- r/LangChain comment measuring LangSmith at about 2 GB of a 4 to 4.5 GB app's memory (Sep 9, 2026)
- LangChain Forum: Not able to delete not used Extended Data Retention traces (Jul 29, 2025)
- Aviad Rozenhek on dev.to: Keep Every LangSmith Trace Without the 10x Retention Bill (Aug 21, 2026)
- r/LangChain: Anyone here using LangGraph with o11y that is not LangSmith? (Jul 30, 2026)
- r/LLMDevs: Spent 3 months evaluating Galileo, Arize, Langfuse and LangSmith for production LLM monitoring (May 30, 2026)
- NonBioS: The NonBioS LLM Observability Pick (Mar 5, 2025)
- digithings issue #4928: self-hosted Langfuse on Cloudflare, cut LangSmith (Oct 1, 2026)
- autonomous-dev issue #1452: self-hosted OTel observability for multi-agent tracing, not LangSmith (Aug 6, 2026)
- Aniruddha Laud and Anshul Pahwa (Apollo.io): From Supervisor to Deep Agents, Interrupt NYC 2026 (Sep 29, 2026)
- Kyle Rush on Product Hunt: Review of LangSmith (Jun 22, 2025)
- Titus Lim, pull request #14: cut local stack from about 47 containers to 14, Langfuse to LangSmith (Sep 3, 2026)
- Carlos Pereira (Cisco): Observing And Testing CX Agents, Interrupt 26 (Jun 10, 2026)
- Evan Kormos (Coinbase): How Coinbase Builds Developer Support Agents, Interrupt 26 (Jul 20, 2026)
- Parlance Labs: Do Automated Evals Work? (Jul 11, 2026)
- Cue Labs, cue-app pull request #513: consolidate LLM observability on Langfuse, retire LangSmith (Oct 9, 2026)
- retrieva-backend pull request #94: drop LangSmith, Langfuse as the single LLMOps backbone (Oct 11, 2026)
- GAIA pull request #720: self-hosted Langfuse alongside LangSmith (May 28, 2026)
- monday.com Engineering: Building a Robust Harness for agent in production, Feed agent case study (Sep 9, 2026)
- monday.com Engineering: How We Gave AI Agents the Keys to Production and Still Sleep at Night (Oct 26, 2026)
- LangSmith docs: Usage and billing (retention tiers and rate limits)
- LangSmith docs: Data purging for compliance
- LangSmith docs: Self-hosted LangSmith
- LangSmith docs: Evaluation concepts
- LangSmith docs: Annotation queues