LangSmith Signals

Counted for All practitioners.

Positive, 28

1. Positive

   Kamie Shami-Schnitzer, feedAgent builder, monday.com

   Building a Robust Harness for agent in production: Feed agent case-studyCompany engineering blog, engineering.monday.com

   monday.com wraps each feedAgent run in LangSmith's traceable(), which records every intermediate step and shows which tools ran and what they returned; its evals run on internal monday.com tooling.

   "We use an LLM as a judge with specific metrics for each part, and run it on the CI to make sure we have no regressions before deploying to production."

   Published Sep 9, 2026self-described: monday.com feedAgent, a deep agent that curates a personal activity feed for each workspace member

2. Mixed

   Aviad Rozenhek, Developer and maintainer of the open-source langsmith-cli

   Keep Every LangSmith Trace Without the 10 Retention Billdev.to article

   Calls LangSmith excellent for live debugging but warns extended retention costs 10x a base trace and online evaluators can silently upgrade retention, so archives traces to private S3.

   Published Aug 21, 2026self-described: Operates LangSmith for live debugging of production AI systems; built an S3/Parquet archive workflow for traces

3. Positive

   True-Homework6226, Reddit user

   Comment on: How are you actually debugging complex LangChain agents?Reddit comment, r/LangChain

   Debugs agents by opening the full run in LangSmith, finding where a tool call or context diverged, and searching backward from there.

   Published Aug 10, 2026self-described: Multi-step LangChain agents

4. Positive

   ambujsystems, Reddit user

   Comment on: My 10-Node Agentic RAG Architecture: Combining LangGraph, Cohere, Pinecone & MCP for Dense Legal ParsingReddit comment, r/LangChain

   Relies on LangSmith traces to tell whether a failure came from parsing, retrieval, reranking, or generation in a 10-node LangGraph RAG pipeline.

   Published Jul 26, 2026self-described: Solo, 10-node LangGraph RAG pipeline on free infrastructure

5. Positive

   dataplayer, Reddit user, self-described

   Comment on: We tested LangGraph against 8 other frameworks in 2026Reddit comment, r/LangChain

   Asks how other agent frameworks could produce metrics and traces as good as LangSmith's.

   Published Jul 24, 2026

6. Positive

   Nick Ung and Akshay Sharma, data science manager and machine learning engineer, Lyft

   Build Evals That Actually MatterConference talk with transcript, AI Engineer World's Fair 2026

   Lyft reads each AI Assist run's reasoning, help articles and tool calls in LangSmith traces, sends failed traces to an annotation queue where product managers and QA label them, and measures each LLM judge's precision and recall against about 100 hand-labeled examples.

   Published Jul 19, 2026self-described: Lyft AI Assist customer-support agents, with LLM judges checked against about 100 hand-labeled pass and fail examples

7. Mixed

   Antaripa Saha and Hamel Husain, AI evals consultants, Parlance Labs

   Do Automated Evals Work?Independent evaluation, Parlance Labs blog

   Parlance Labs scored LangSmith's chat agent at 79.5% recall and 77.5% precision against 39 hand-labeled failures, after LangSmith Engine surfaced too few issues to score. LangSmith had the best interface for reading single traces, and several of its wrong findings were marked high severity.

   "But it surfaced only a small set of issues, far fewer than a full review turns up, so we could not use it for the 100-trace comparison. We ran the scored experiment with LangSmith’s chat agent instead."

   Published Jul 11, 2026self-described: 100 production traces from Nurture Boss's apartment-leasing agent, hand-labeled with a domain expert into 39 failures, then given to Braintrust Loop, Arize Alyx, LangSmith and three coding agents

8. Positive

   Daniel Cordeiro, developer

   Building an AI Billing Assistant: Integrating LangChain ReAct Agents with Spring Boot Microservicesdev.to article

   LangSmith traces each ReAct loop's tool calls, reasoning and timings, which they call invaluable for spotting and debugging agent misbehavior.

   Published Jun 8, 2026self-described: Telecom billing support agent: LangChain ReAct agent in FastAPI calling Spring Boot services

9. Positive

   Stefano Casafranca, Independent developer (maker of the open-source Build_Script)

   How I Built an Autonomous PR Agent with SerpApi, LangGraph, and LangSmithdev.to article

   Traces every graph run in LangSmith with tags and metadata, letting them retrace why the agent ranked a thread or drafted an odd comment.

   Published Apr 30, 2026self-described: Solo; autonomous Reddit-marketing agent on LangGraph and GitHub Actions, two weeks in production

10. Positive

    vivek, Independent developer

    I Tried to Build an Alexa with Real Memory: Here's What I Learned After 3 Months of Failuredev.to article

    Lists LangSmith in their stack for tracing and debugging the LangGraph graph and calls it essential.

    Published Mar 5, 2026self-described: Solo personal-assistant project with LangGraph, Groq, Pinecone, Postgres and Redis over three months

11. Positive

    Mesterniz, Hacker News commenter

    Comment on: Show HN: How to fix AI Agents at the component levelHacker News comment

    Built a four-component agent on LangGraph and LangSmith to track per-component latency, intermediate state, and which component caused a bad answer.

    Published Dec 12, 2025self-described: Multi-component agent built for a write-up

12. Positive

    Maryam Warraich, Product Hunt

    Review of LangSmith by Maryam WarraichProduct Hunt review

    Says LangSmith's debugging and iteration tools made it easier to take their LLM projects to production and saved a lot of trial and error.

    Published Oct 5, 2025self-described: Own LLM projects taken from idea to production

13. Positive

    Dan Bulteel, Maker of Meet-Ting

    Review of LangSmith by Dan BulteelProduct Hunt review

    Building Ting without LangSmith felt like guesswork; after adopting it the team traces problems through the full prompt path and moves faster.

    Published Jul 16, 2025self-described: Ting scheduling assistant, early-stage product team

14. Mixed

    Dan Mason, Principal, AI, Stride

    Case Study + Deep Dive: Telemedicine Support Agents with LangGraph/MCPConference workshop with transcript, AI Engineer World's Fair 2025

    Mason reviews every agent conversation and its tool calls in LangSmith and keeps eval datasets there. He runs the evals in his own promptfoo harness to avoid lock-in, and says LangSmith's displayed costs ignored prompt caching.

    Published Jun 22, 2025LangGraph agents Stride built to guide telemedicine patients through at-home treatment by text message

15. Positive

    John Capobianco, YouTube creator building AI agents

    LangSmith: Observability for AI AgentsVideo, YouTube

    Walks through adding @traceable and the OpenAI wrapper to their LangGraph agent, praising waterfalls with latency, tokens and cost, and noting it cost them nothing as an individual.

    "…wrapper that wraps around our open AI client so we're just simply going to wrap our openai client and we're going to make traceables at traceable so what does this look like in the code I'm not going to show you myv file but my…"

    Published Mar 9, 2025self-described: Own LangGraph agent project instrumented with @traceable and the OpenAI client wrapper

16. Positive

    Pablo Sebastian Martinez, Production Engineering team, monday.com

    How We Gave AI Agents the Keys to Production and Still Sleep at NightCompany engineering blog, engineering.monday.com

    monday.com's Production Engineering team gives every Infra agent run one parent trace in LangSmith as its audit trail, so the team can reconstruct any run after an incident.

    "Each run has one parent trace in LangSmith and JSON logs to stdout, and every call that crosses a boundary is traced so the run can be reconstructed afterwards."

    Published Oct 26, 2026self-described: monday.com Infra agents platform, where agents resolve infrastructure tickets and approve production pull requests

17. Positive

    Dan Haggerty, agents engineer at NVIDIA

    Add configurable LangSmith tags to Compass telemetryGitHub pull request

    Added LangSmith tags to NeMo Compass's OpenTelemetry root traces so the engineer can filter traces by issue and compare runs before and after a code change; a live check loaded 160 traces with the tags stored on the root.

    Published Oct 22, 2026self-described: NVIDIA NeMo Compass, whose development traces reach LangSmith over OpenTelemetry

18. Mixed

    Unlucky_Emu6094, Reddit user

    Comment on: When you change a RAG pipeline, how do you know which queries actually regressed?Reddit comment, r/LangChain

    Likes LangSmith for traces but exports results to a spreadsheet for paired query comparisons, because LangSmith does not lead them to a screen built for that comparison.

    Published Oct 8, 2026self-described: RAG pipeline changes

19. Mixed

    Tyler Norlund (tnorlund), Developer, maintainer of a personal portfolio repo

    Remove the paid-LangSmith wiring now that the subscription is cancelledGitHub issue

    After canceling LangSmith Plus, strips the paid bulk-export wiring in favor of native S3 traces. Keeps the SDK and opt-in tracing of a fraction of runs on a free org for debugging.

    Published Sep 28, 2026self-described: Personal portfolio with LangGraph receipt and QA agents on AWS

20. Positive

    Titus Lim (tituslhy), Data scientist (GitHub profile)

    Cut local stack from ~47 containers to 14; Langfuse → LangSmithGitHub pull request

    Swaps self-hosted Langfuse and its ClickHouse/MinIO containers for hosted LangSmith tracing, shrinking a local agent template stack from about 47 containers to 14.

    Published Sep 3, 2026self-described: Agent template repo (AG-UI / A2UI / A2A) on LangGraph

21. Positive

    Folarin Akinloye, AI engineer in London

    Evaluating Agents with LangSmith: A Complete GuidePersonal blog post, folarin.dev

    Akinloye traces every agent from day one and saves failed runs to a dataset. Before each prompt, model, or tool change, Akinloye runs an experiment against the last one and ships only when nothing important regresses.

    Published Jun 19, 2026self-described: Agents built with LangChain and LangGraph, traced from day one and tested against datasets before each change

22. Positive

    fxwin, Hacker News commenter, self-described

    Comment on: Apache Burr: Build reliable AI agents and applicationsHacker News comment

    Says LangChain's one environment variable for tracing gives a UI to browse every trace with almost no effort, which a hand-rolled setup cannot match.

    Published Jun 10, 2026

23. Mixed

    Total_Listen_4289, Reddit user

    Spent 3 months evaluating Galileo, Arize, Langfuse and LangSmith for production LLM monitoring. Notes for anyone doing the same.Reddit post, r/LLMDevs

    After a three-month evaluation the small team found LangSmith smoothest inside LangChain, but chose Langfuse over its LangChain ties and Enterprise-only self-hosting.

    Published May 30, 2026self-described: Small team shipping agents

24. Positive

    Balachander Keelapudi, Balachander's Substack

    Tracing Claude Code with LangSmith: Full Observability for Your AI Coding AgentNewsletter, Balachander's Substack

    Wired Claude Code into LangSmith with the official plugin in minutes, gaining traces of LLM calls, tool executions and subagent runs that are otherwise invisible.

    Published Apr 4, 2026self-described: Tracing Claude Code sessions, tool calls and subagents with the official LangSmith plugin

25. Mixed

    Aviad Rozenhek, Developer and author of langsmith-cli

    LangSmith CLI: Not Just Efficient, Actually Better Than MCPdev.to article

    Values LangSmith's traces, datasets and evals but found daily use via the web UI and MCP server clunky, measuring the MCP tools at 16,100 tokens of always-loaded context.

    Published Jan 15, 2026self-described: Debugging and monitoring production LLM runs from the terminal and from Claude Code

26. Positive

    Juha Pellotsalo, Senior engineer (self-described)

    Reflections on building my first LangGraph projectdev.to article

    Tracing turned on with just an API key and was the easiest usable tracing UI they found; they call tracing a must-have and would start evals in LangSmith.

    Published Dec 22, 2025self-described: Solo learning project: multi-agent newsroom workflow in LangGraph

27. Positive

    Kyle Rush, Maker of Astrid: Personal Shopping Agent

    Review of LangSmith by Kyle RushProduct Hunt review

    The team became LangSmith power users for traces, datasets, annotation queues and evals, and picked it over Langfuse because of their prior experience and speed.

    Published Jun 22, 2025self-described: Astrid personal shopping agent; traces, datasets, annotation queues and evals

28. Mixed

    fazlerocks, Hacker News poster

    Ask HN: What tools are you using for AI evals? Everything feels half-bakedHacker News post (Ask HN)

    Evaluated LangSmith for production evals: strong tracing, but evals feel secondary, US$0.50 per 1k traces after the free tier adds up, and the UI slows with large datasets.

    Published Jun 5, 2025self-described: Team running LLMs in production for content generation, customer support and code review assistance

Mixed or negative, 7

1. Negative

   Daniel Payne, Developer (self-described long-time programmer)

   Why I think Vercel EVE Web interface is so much better than Langsmith for development.Medium article

   Prefers a custom Eve web channel for development because debugging in LangSmith meant digging through raw JSON inputs and outputs, which felt too manual.

   Published Sep 16, 2026self-described: Solo app that values building-material photos with a Vercel Eve agent; previously used LangGraph and LangSmith

2. Mixed

   automation_experto, Reddit user

   Comment on: DoubtReddit comment, r/LangChain

   Says LangSmith showed normal-looking traces while agents reasoned over bad extracted fields, so the real bug sat upstream of the trace.

   Published Jul 19, 2026self-described: Agents built on document extraction

3. Mixed

   Previous_Net_1154, Reddit user

   Comment on: 6 things I learned deploying AI agents for B2B clients (the hard way)Reddit comment, r/LangChain

   LangSmith showed every LLM and tool call, but a run that retrieved the wrong client context looked identical to a good one at the trace level.

   Published Jun 30, 2026self-described: B2B client agents in production

4. Mixed

   MickeyShmueli, Hacker News commenter

   Comment on: Launch HN: Cekura (YC F24) - Testing and monitoring for voice and chat AI agentsHacker News comment

   On a verification flow, every turn looked fine in LangSmith while the overall flow was broken, so turn-level traces missed a session-level failure.

   Published Mar 4, 2026self-described: Support and verification agents in production

5. Mixed

   BasRizk, developer filing a GitHub issue

   [TypeScript] CRITICAL BUG: AI message duplicated in traces when using Anthropic streaming with finalMessage()GitHub issue

   With the Anthropic wrapper and finalMessage(), every streamed assistant reply appeared twice in LangSmith traces, reproduced on every run.

   Published Feb 7, 2026self-described: TypeScript app tracing Anthropic streaming calls

6. Mixed

   zhangjunkang (Z-Jr10-K), Developer, SprintRay

   Issue: LangSmith tracing does not capture subagent tool calls in Claude Agent SDKGitHub issue

   Reports that LangSmith's Claude Agent SDK integration records only the main agent and omits tool calls made by subagents.

   Published Oct 27, 2025self-described: Claude Agent SDK app with subagents and MCP servers

7. Mixed

   Tejasri Teyam, developer filing a GitHub issue

   Bug: First turn intermittently missing from LangSmith Trajectory viewGitHub issue

   The Trajectory view sometimes starts at the second turn of a multi-turn trace while the stats panel still counts every turn, so the first turn cannot be reviewed there.

   Published Sep 25, 2026self-described: Multi-turn LangGraph application traced to LangSmith Cloud

Counted for All practitioners.

Positive, 13

1. Positive

   Patrick Porche Jr, product engineer building AI tools with clinicians and operations teams

   Move the pipeline onto LangGraph, LangChain and LangSmith; remove LogfireGitHub pull request

   Replaced Logfire with LangSmith tracing and moved the pipeline's evals from pydantic-evals to LangSmith datasets and experiments, sending only allowlisted keys so patient data stays out of traces.

   Published Sep 21, 2026self-described: Medical-necessity review pipeline on Anthropic and Google models, with a patient-data boundary

2. Positive

   Akele Reed and Dave Revere, Principal AI Engineer and Staff AI Engineer, SonderMind

   Evals Driven-Development: Engineering a Mental Health AI Coach Ethically & SafelyConference talk with transcript, AI Engineer World's Fair 2026

   SonderMind clinicians annotate production conversations in a LangSmith annotation queue, and a script turns each annotation into a typed eval that every prompt or model change must pass in CI.

   "A clinical SME reviews a production conversation in LangSmith, marks the missed disclosure with an expected observation, and the script writes it to the eval set."

   Published Jul 25, 2026Sonder, SonderMind's AI mental health coach, with guardrail evals written from clinician annotations

3. Positive

   Nick Ung and Akshay Sharma, data science manager and machine learning engineer, Lyft

   Build Evals That Actually MatterConference talk with transcript, AI Engineer World's Fair 2026

   Lyft reads each AI Assist run's reasoning, help articles and tool calls in LangSmith traces, sends failed traces to an annotation queue where product managers and QA label them, and measures each LLM judge's precision and recall against about 100 hand-labeled examples.

   Published Jul 19, 2026self-described: Lyft AI Assist customer-support agents, with LLM judges checked against about 100 hand-labeled pass and fail examples

4. Positive

   Folarin Akinloye, AI engineer in London

   Evaluating Agents with LangSmith: A Complete GuidePersonal blog post, folarin.dev

   Akinloye traces every agent from day one and saves failed runs to a dataset. Before each prompt, model, or tool change, Akinloye runs an experiment against the last one and ships only when nothing important regresses.

   Published Jun 19, 2026self-described: Agents built with LangChain and LangGraph, traced from day one and tested against datasets before each change

5. Positive

   Aishwarya Srinivasan, Aishwarya Srinivasan (Substack)

   Setting up Evaluations for LLMsNewsletter, Aishwarya Srinivasan (Substack)

   Uses LangSmith as the orchestration layer for datasets, traces and evaluators, describing it as a full-lifecycle evaluation platform for automated regression pipelines.

   Published Mar 31, 2026self-described: Hands-on eval pipeline: reference datasets, heuristic checks and LLM-as-judge (Gemini grading Claude) orchestrated in LangSmith

6. Positive

   Andrew Larsen, engineer

   From Tests to Experiments: Evaluating LangGraph Agents with LangSmith and PytestMedium article

   Added LangSmith's pytest integration so static eval datasets and untracked results become registered datasets and experiments tracked over time, without changing how tests are written.

   Published Dec 28, 2025self-described: Production support agent built with LangGraph; pytest evaluators extended with LangSmith experiments

7. Positive

   Lysia Li and five co-authors, Harvey

   How Agentic Search Unlocks Legal Research Intelligence at HarveyCompany engineering blog, harvey.ai

   Harvey sends the OpenTelemetry traces of its OpenAI Agents SDK search agent into LangSmith and runs offline evaluations there on queries its legal experts write.

   "Centralized evaluation infrastructure that we built using the OpenAI Agent SDK's OpenTelemetry traces enables us to run offline evaluations in LangSmith. We track metrics like hallucination, tool recall, retrieval recall, formatting, and answer quality."

   Published Dec 10, 2025self-described: Harvey Agentic Search for legal research, built on the OpenAI Agents SDK

8. Positive

   Maitar Asher, Head of Engineering, Tavily

   Evaluating AI Search: A Practical Framework for Augmented AI SystemsConference talk with transcript, AI Engineer World's Fair 2025

   Asher reports that Tavily tracks its offline evaluation runs in LangSmith and calls it a great observability tool for comparing performance over time.

   Published Jul 29, 2025Tavily's open-source LangGraph agent that builds fresh eval datasets for web search from live documents

9. Positive

   Kyle Rush, Maker of Astrid: Personal Shopping Agent

   Review of LangSmith by Kyle RushProduct Hunt review

   The team became LangSmith power users for traces, datasets, annotation queues and evals, and picked it over Langfuse because of their prior experience and speed.

   Published Jun 22, 2025self-described: Astrid personal shopping agent; traces, datasets, annotation queues and evals

10. Mixed

    Hannah Craighead, startup team member

    Review of LangSmith by Hannah CraigheadProduct Hunt review

    LangSmith evals in GitHub CI/CD catch regressions, but the UI gets tricky with large datasets or long experiment histories and filters are not kept in the URL.

    Published Jan 27, 2025self-described: Startup; ongoing LLM evaluations with LangSmith evals in GitHub CI/CD

11. Positive

    MarkAZhang, developer of a GAIA benchmark research agent

    Migrate observability from Langfuse to LangSmithGitHub pull request

    Moved a LangGraph research agent's tracing, datasets and five evaluators from Langfuse to LangSmith, with a one-time script that ports existing Langfuse datasets.

    Published Jun 30, 2026self-described: LangGraph research agent scored on the GAIA benchmark with five evaluators

12. Mixed

    Total_Listen_4289, Reddit user

    Spent 3 months evaluating Galileo, Arize, Langfuse and LangSmith for production LLM monitoring. Notes for anyone doing the same.Reddit post, r/LLMDevs

    After a three-month evaluation the small team found LangSmith smoothest inside LangChain, but chose Langfuse over its LangChain ties and Enterprise-only self-hosting.

    Published May 30, 2026self-described: Small team shipping agents

13. Mixed

    rafio77, Reddit user

    I ran langfuse, langsmith and helicone in prod for a month and only one of them stuckReddit post, r/LLMDevs

    Ran three tools in production for a month: LangSmith was the most complete once wired in, but the team is mostly outside LangChain and kept Langfuse.

    Published May 20, 2026self-described: Team running agents in production, mostly outside LangChain

Mixed or negative, 9

1. Mixed

   Unlucky_Emu6094, Reddit user

   Comment on: When you change a RAG pipeline, how do you know which queries actually regressed?Reddit comment, r/LangChain

   Likes LangSmith for traces but exports results to a spreadsheet for paired query comparisons, because LangSmith does not lead them to a screen built for that comparison.

   Published Oct 8, 2026self-described: RAG pipeline changes

2. Mixed

   Antaripa Saha and Hamel Husain, AI evals consultants, Parlance Labs

   Do Automated Evals Work?Independent evaluation, Parlance Labs blog

   Parlance Labs scored LangSmith's chat agent at 79.5% recall and 77.5% precision against 39 hand-labeled failures, after LangSmith Engine surfaced too few issues to score. LangSmith had the best interface for reading single traces, and several of its wrong findings were marked high severity.

   "But it surfaced only a small set of issues, far fewer than a full review turns up, so we could not use it for the 100-trace comparison. We ran the scored experiment with LangSmith’s chat agent instead."

   Published Jul 11, 2026self-described: 100 production traces from Nurture Boss's apartment-leasing agent, hand-labeled with a domain expert into 39 failures, then given to Braintrust Loop, Arize Alyx, LangSmith and three coding agents

3. Mixed

   itsmeAki, Reddit user

   Testing LangChain agents beyond LangSmith eval scripts: what's actually catching prod issues?Reddit post, r/LangChain

   Says LangSmith evals catch prompt regressions but stayed green for two days while the agent skipped a required question after a prompt change.

   Published May 30, 2026self-described: Sales-qualification chat agent in production

4. Mixed

   jdudekBC, Developer (issue author)

   Feature Request: Custom Input Rendering for Annotation QueuesGitHub issue

   Builds golden eval datasets with LangSmith annotation queues, but HTML inputs are hard to review since custom rendering replaces the editable output pane annotators need.

   Published Mar 18, 2026self-described: Team building golden evaluation datasets with annotators

5. Mixed

   Hamed Karbasi (hamedkarbasi-haypp), Developer (issue author)

   Issue: LangSmith UI doesn't show the datasets created by SDKGitHub issue

   Reports that datasets and experiments created with the Python SDK never appear in the Datasets and Experiments pages, though direct links work.

   Published Feb 27, 2026self-described: Datasets and experiments created with the Python SDK

6. Negative

   Bob (hubdev), Forum user

   Not able to delete not used Extended Data Retention tracesLangChain Forum post

   Prompt experiments with a code evaluator auto-convert traces to Extended Data Retention, hitting usage limits that block new experiments; deleting traces does not lower the count.

   Published Jul 29, 2025self-described: Comparing prompts in LangSmith experiments against a dataset

7. Mixed

   Yurii Venher (yurii-ve), Developer, Wix

   [Bug] Eval experiment table is empty if OTEL_ENABLED environment variable is setGitHub issue

   Reports that running evaluate on Vercel AI SDK calls with OpenTelemetry enabled creates an empty experiment, though results exist in code.

   Published Jul 11, 2025self-described: Evaluations of Vercel AI SDK calls through OpenTelemetry

8. Mixed

   fazlerocks, Hacker News poster

   Ask HN: What tools are you using for AI evals? Everything feels half-bakedHacker News post (Ask HN)

   Evaluated LangSmith for production evals: strong tracing, but evals feel secondary, US$0.50 per 1k traces after the free tier adds up, and the UI slows with large datasets.

   Published Jun 5, 2025self-described: Team running LLMs in production for content generation, customer support and code review assistance

9. Mixed

   h-portia, Reddit user

   Best evaluation platform for LLMs/agentic frameworks for error analysis?Reddit post, r/LLMDevs

   Finds LangSmith handy for high-level eval results and drill-downs, but hard for spotting failure patterns across groups, so builds own charts from its Python library.

   Published Feb 6, 2025self-described: Team running LLM output evals in LangSmith, Python

GILD's fact checks, which don't count as sources

Checked Oct 12, 2026.LangSmith documents offline evaluation on datasets and online evaluation on production traces, with human, code, LLM-as-judge, decision model, and pairwise evaluators. Its documentation says LLM-as-judge evaluators need careful review of scores and prompt tuning. Evaluation concepts

Counted for All practitioners.

1. Positive

   regnull, Reddit user

   I open-sourced a multi-bot runtime that uses LangMem and LangSmithReddit post, r/LangChain

   Built a self-hosted multi-bot runtime on LangMem and reports that LangSmith tracing works out of the box.

   Published Sep 14, 2026self-described: Solo, open-source self-hosted multi-bot platform

2. Positive

   Titus Lim (tituslhy), Data scientist (GitHub profile)

   Cut local stack from ~47 containers to 14; Langfuse → LangSmithGitHub pull request

   Swaps self-hosted Langfuse and its ClickHouse/MinIO containers for hosted LangSmith tracing, shrinking a local agent template stack from about 47 containers to 14.

   Published Sep 3, 2026self-described: Agent template repo (AG-UI / A2UI / A2A) on LangGraph

3. Positive

   Juan Cruz Boullosa Gutierrez, contributor to a LangGraph Discord bot

   Replace custom tracing with LangSmith observabilityGitHub pull request

   Deleted the bot's own trace recorder, span helpers and readers in favor of LangSmith's LangGraph auto-instrumentation set by environment variables, with every input and output hidden and only allowlisted metadata sent.

   Published Jul 16, 2026self-described: LangGraph Discord bot with direct-message and crisis paths, traced with inputs and outputs hidden

4. Positive

   fxwin, Hacker News commenter, self-described

   Comment on: Apache Burr: Build reliable AI agents and applicationsHacker News comment

   Says LangChain's one environment variable for tracing gives a UI to browse every trace with almost no effort, which a hand-rolled setup cannot match.

   Published Jun 10, 2026

5. Positive

   Consistent_Wash_276, Reddit user

   LangSmith for local tracingReddit post, r/LangChain

   New to LangGraph and LangSmith, came away pretty impressed with LangSmith tracing after a two-hour first session.

   Published Jun 7, 2026self-described: Solo, first LangGraph project

6. Positive

   Markus Eisele, Java developer and newsletter author

   Trace a Quarkus LangChain4j App in LangSmithNewsletter, The Main Thread (Substack)

   Java teams can get LangSmith traces over plain OTLP with no custom instrumentation, though they would add sampling or redaction before shipping full prompts from production.

   Published May 30, 2026self-described: Example Quarkus LangChain4j support assistant on Ollama exporting OpenTelemetry traces to LangSmith

7. Positive

   Balachander Keelapudi, Balachander's Substack

   Tracing Claude Code with LangSmith: Full Observability for Your AI Coding AgentNewsletter, Balachander's Substack

   Wired Claude Code into LangSmith with the official plugin in minutes, gaining traces of LLM calls, tool executions and subagent runs that are otherwise invisible.

   Published Apr 4, 2026self-described: Tracing Claude Code sessions, tool calls and subagents with the official LangSmith plugin

8. Positive

   Juha Pellotsalo, Senior engineer (self-described)

   Reflections on building my first LangGraph projectdev.to article

   Tracing turned on with just an API key and was the easiest usable tracing UI they found; they call tracing a must-have and would start evals in LangSmith.

   Published Dec 22, 2025self-described: Solo learning project: multi-agent newsroom workflow in LangGraph

9. Positive

   Hugo Santana, Builder of Applied, author of The Applied Report

   My AI Agents Were Working, but I Had No VisibilityNewsletter, Agent-led Growth (Substack)

   Connected non-LangChain agents to LangSmith without rebuilding them and now sees historical traces, token spend per agent, failures and latency.

   Published Jul 7, 2026self-described: Solo; custom multi-agent system (Applied) sending run summaries to LangSmith

10. Positive

    Folarin Akinloye, AI engineer in London

    Evaluating Agents with LangSmith: A Complete GuidePersonal blog post, folarin.dev

    Akinloye traces every agent from day one and saves failed runs to a dataset. Before each prompt, model, or tool change, Akinloye runs an experiment against the last one and ships only when nothing important regresses.

    Published Jun 19, 2026self-described: Agents built with LangChain and LangGraph, traced from day one and tested against datasets before each change

11. Mixed

    adhishthite, Developer (issue author)

    [langsmith]: LangSmith traces from Claude Code do not expose cost, token usage, or detailed model metadataGitHub issue

    Tracing Claude Code into LangSmith works, but traces show only the model name with no token usage or cost, so top-cost traces and model spend cannot be audited.

    Published Dec 21, 2025self-described: Tracing Claude Code sessions into LangSmith

GILD's fact checks, which don't count as sources

Checked Oct 12, 2026.LangChain and LangGraph code traces with LANGSMITH_TRACING=true and an API key. Other code uses the traceable wrapper, the OpenAI, Anthropic, and Gemini wrappers, or OpenTelemetry spans. Tracing quickstart

Counted for All practitioners.

Positive, 4

1. Positive

   Hugo Santana, Builder of Applied, author of The Applied Report

   My AI Agents Were Working, but I Had No VisibilityNewsletter, Agent-led Growth (Substack)

   Connected non-LangChain agents to LangSmith without rebuilding them and now sees historical traces, token spend per agent, failures and latency.

   Published Jul 7, 2026self-described: Solo; custom multi-agent system (Applied) sending run summaries to LangSmith

2. Positive

   Kanishk Gupta, Builder of Promptius

   Review of LangSmith by Kanishk GuptaProduct Hunt review

   Credits the LangChain, LangGraph and LangSmith combination for building Promptius, with LangSmith giving strong observability and cost tracking.

   Published Aug 16, 2025self-described: Promptius, built over five months on LangChain, LangGraph and LangSmith

3. Mixed

   Capable-Purpose9911, Reddit user

   Comment on: What is the best AI observability tool in 2026?Reddit comment, r/LangChain

   Finds LangSmith good for analytics but measured it using about 2 GB of a 4 to 4.5 GB app's memory.

   Published Sep 9, 2026self-described: Small app on a 4 to 4.5 GB RAM host

4. Positive

   John Capobianco, YouTube creator building AI agents

   LangSmith: Observability for AI AgentsVideo, YouTube

   Walks through adding @traceable and the OpenAI wrapper to their LangGraph agent, praising waterfalls with latency, tokens and cost, and noting it cost them nothing as an individual.

   "…wrapper that wraps around our open AI client so we're just simply going to wrap our openai client and we're going to make traceables at traceable so what does this look like in the code I'm not going to show you myv file but my…"

   Published Mar 9, 2025self-described: Own LangGraph agent project instrumented with @traceable and the OpenAI client wrapper

Mixed or negative, 10

1. Mixed

   Gabriel Gouvêa, software engineer at movedot

   Amazon Bedrock Claude Sonnet 5 US geo profile is priced at the global rateGitHub issue

   Gabriel Gouvêa found LangSmith pricing Bedrock's US-geo Claude Sonnet 5 runs at the global rate of US$2.00 per million input tokens, where AWS lists US$2.20, so cost totals for those runs read low.

   Published Jul 28, 2026self-described: Hosted LangSmith cost tracking for Claude Sonnet 5 runs through Amazon Bedrock's US cross-region profile

2. Mixed

   FeeVirtual, Reddit user

   langsmith + testmu running together. prod regressions still slipping. what’s the missing layer?Reddit post, r/AI_Agents

   Traces in LangSmith, adversarial evals and CI rubric checks all missed three production failures: a tool's changed response format, an eval set three months stale, and memory that drifted across long sessions.

   Published Jul 2, 2026self-described: Production agents traced in LangSmith, with an agent-to-agent adversarial eval tool and custom rubrics in CI

3. Mixed

   poushkar, Reddit user, self-described

   Are there any tools out there automatically suggesting possible agent improvements?Reddit post, r/LLMDevs

   Finds LangSmith Insights still rough for spotting agent failure patterns because it needs setup and a model choice first.

   Published Jun 20, 2026

4. Mixed

   Daniel (dast-draftwise), Developer, Draftwise

   Issue: Messy Error Logging in LangSmithGitHub issue

   Reports that each failed retry under wrap_openai becomes its own error trace, so error dashboards count failures for requests that succeeded.

   Published Jan 6, 2026self-described: Draftwise, OpenAI calls wrapped with wrap_openai and retries

5. Mixed

   adhishthite, Developer (issue author)

   [langsmith]: LangSmith traces from Claude Code do not expose cost, token usage, or detailed model metadataGitHub issue

   Tracing Claude Code into LangSmith works, but traces show only the model name with no token usage or cost, so top-cost traces and model spend cannot be audited.

   Published Dec 21, 2025self-described: Tracing Claude Code sessions into LangSmith

6. Mixed

   ryanlohyr, developer filing a GitHub issue

   Issue: Cost showing up as 0 despite having tokensGitHub issue

   LangSmith showed a cost of 0 for every run that called OpenAI models, though the models are on its pricing list and the traces carry token counts.

   Published Sep 16, 2025self-described: Runs with several OpenAI calls traced to LangSmith

7. Mixed

   Shane S (ssmithra), Forum user

   Self-hosted: Langsmith is counting base64 encoded files/images as string tokens, causing massively incorrect token costsLangChain Forum post

   On self-hosted LangSmith, base64 images are counted as string tokens, producing wildly wrong costs in a UI the team relies on for cost management.

   Published Jul 29, 2025self-described: Self-hosted LangSmith, moving into multi-modal inputs

8. Mixed

   yurii-ve, developer filing a GitHub issue

   Tracing with AI SDK (by Vercel) creates trace with wrong (doubled) total tokens countGitHub issue

   Following LangSmith's Vercel AI SDK guide, the parent trace reported 22 tokens for a call that used 11.

   Published Jun 23, 2025self-described: Vercel AI SDK app calling Gemini on Vertex, traced through LangSmith's exporter

9. Mixed

   Karolis Mariunas (kmariunas), Developer (issue author)

   Duplicate traces through OpenAI agents SDKGitHub issue

   Reports one OpenAI Agents SDK run appearing as three LangSmith traces with different IDs, which made the team's dashboards untrustworthy.

   Published Mar 31, 2025self-described: OpenAI Agents SDK app traced to LangSmith

10. Mixed

    André Menezes (menezesandre), AI engineer

    Issue: Client silently stops posting traces to the API (auto_batch_tracing=True, DISABLE_RUN_COMPRESSION=False)GitHub issue

    Reports that the Python client sometimes stops sending traces with no error logged while the app keeps running, and only a restart fixes it.

    Published Apr 3, 2025self-described: Python application traced with auto-batch tracing and compression

Counted for All practitioners.

Raised as a concern, 23

1. Mixed

   Tejasri Teyam, developer filing a GitHub issue

   Bug: First turn intermittently missing from LangSmith Trajectory viewGitHub issue

   The Trajectory view sometimes starts at the second turn of a multi-turn trace while the stats panel still counts every turn, so the first turn cannot be reviewed there.

   Published Sep 25, 2026self-described: Multi-turn LangGraph application traced to LangSmith Cloud

2. Negative

   MysteriousPlan6, Reddit user

   Comment on: We built a LangGraph integration for our agent runtimeReddit comment, r/LangChain

   Says LangSmith costs a lot to show a few nested spans, and that the trace view can crash the browser tab on multi-agent loops.

   Published Sep 23, 2026self-described: Multi-agent LangGraph workflows

3. Mixed

   JamesDun, developer filing a GitHub issue

   claude_agent_sdk integration: receive_messages() is not instrumented, traces silently lost for the full-stream consumption pathGitHub issue

   LangSmith's Claude Agent SDK integration recorded no traces at all when an app read the SDK's continuous message stream, the path long-lived multi-turn sessions use, and gave no warning or error.

   Published Sep 20, 2026self-described: Interactive multi-turn sessions on the Claude Agent SDK, traced through LangSmith's integration

4. Negative

   Daniel Payne, Developer (self-described long-time programmer)

   Why I think Vercel EVE Web interface is so much better than Langsmith for development.Medium article

   Prefers a custom Eve web channel for development because debugging in LangSmith meant digging through raw JSON inputs and outputs, which felt too manual.

   Published Sep 16, 2026self-described: Solo app that values building-material photos with a Vercel Eve agent; previously used LangGraph and LangSmith

5. Mixed

   Capable-Purpose9911, Reddit user

   Comment on: What is the best AI observability tool in 2026?Reddit comment, r/LangChain

   Finds LangSmith good for analytics but measured it using about 2 GB of a 4 to 4.5 GB app's memory.

   Published Sep 9, 2026self-described: Small app on a 4 to 4.5 GB RAM host

6. Mixed

   ErzErzHerzog, Developer (issue author)

   Extreme lagging on Langsmith PlaygroundGitHub issue

   Reports that opening a 40k-token LLM call in the Playground takes over a minute and that each new message box takes about 60 seconds.

   Published Apr 15, 2026self-described: Debriefing live agent runs in the Playground

7. Mixed

   jdudekBC, Developer (issue author)

   Feature Request: Custom Input Rendering for Annotation QueuesGitHub issue

   Builds golden eval datasets with LangSmith annotation queues, but HTML inputs are hard to review since custom rendering replaces the editable output pane annotators need.

   Published Mar 18, 2026self-described: Team building golden evaluation datasets with annotators

8. Mixed

   Hamed Karbasi (hamedkarbasi-haypp), Developer (issue author)

   Issue: LangSmith UI doesn't show the datasets created by SDKGitHub issue

   Reports that datasets and experiments created with the Python SDK never appear in the Datasets and Experiments pages, though direct links work.

   Published Feb 27, 2026self-described: Datasets and experiments created with the Python SDK

9. Mixed

   Aviad Rozenhek, Developer and author of langsmith-cli

   LangSmith CLI: Not Just Efficient, Actually Better Than MCPdev.to article

   Values LangSmith's traces, datasets and evals but found daily use via the web UI and MCP server clunky, measuring the MCP tools at 16,100 tokens of always-loaded context.

   Published Jan 15, 2026self-described: Debugging and monitoring production LLM runs from the terminal and from Claude Code

10. Mixed

    dast-draftwise, developer filing a GitHub issue

    Issue: Bulk Export takes very long time with small number of tracesGitHub issue

    A bulk export of about 600 traces stayed in Running for over an hour, which the author says makes exporting their 50,000+ traces unworkable.

    Published Oct 29, 2025self-described: Exporting LangSmith traces to S3 as Parquet for a data pipeline

11. Mixed

    henriquejsb, Developer (issue author)

    Issue: Memory leaksGitHub issue

    In a long-running LangGraph app, memory grows significantly only when LangSmith tracing is on; tracemalloc diffs point at the SDK's serialization code.

    Published Sep 10, 2025self-described: Long-running Python application on LangGraph and LangChain

12. Negative

    Ahmad Ilaiwi (Ilaiwi), Forum user

    Langsmith is downLangChain Forum post

    Could not load any LangSmith projects during an outage and says LangSmith had been very slow for them recently.

    Published Sep 9, 2025self-described: Team using LangSmith projects

13. Negative

    staletofu, Hacker News commenter, self-described

    Comment on: Ask HN: GCP Outage?Hacker News comment

    Reports being unable to log in to LangSmith with Google during a Google Cloud outage.

    Published Jul 18, 2025

14. Mixed

    erwan-png, developer filing a GitHub issue

    Failed to send compressed multipart ingest: Connection error caused failure to POST https://api.smith.langchain.com/runs/multipart in LangSmith APIGitHub issue

    After upgrading the LangSmith SDK, every LLM call's trace failed to upload and stayed pending, while a project on an older SDK version kept working until it was upgraded too.

    Published Jun 4, 2025self-described: LangGraph app on Ollama and Groq models traced to LangSmith

15. Mixed

    André Menezes (menezesandre), AI engineer

    Issue: Client silently stops posting traces to the API (auto_batch_tracing=True, DISABLE_RUN_COMPRESSION=False)GitHub issue

    Reports that the Python client sometimes stops sending traces with no error logged while the app keeps running, and only a restart fixes it.

    Published Apr 3, 2025self-described: Python application traced with auto-batch tracing and compression

16. Mixed

    Hannah Craighead, startup team member

    Review of LangSmith by Hannah CraigheadProduct Hunt review

    LangSmith evals in GitHub CI/CD catch regressions, but the UI gets tricky with large datasets or long experiment histories and filters are not kept in the URL.

    Published Jan 27, 2025self-described: Startup; ongoing LLM evaluations with LangSmith evals in GitHub CI/CD

17. Mixed

    BasRizk, developer filing a GitHub issue

    [TypeScript] CRITICAL BUG: AI message duplicated in traces when using Anthropic streaming with finalMessage()GitHub issue

    With the Anthropic wrapper and finalMessage(), every streamed assistant reply appeared twice in LangSmith traces, reproduced on every run.

    Published Feb 7, 2026self-described: TypeScript app tracing Anthropic streaming calls

18. Negative

    NeutralCrane, Hacker News commenter, self-described

    Comment on: Open SWE by LangChainHacker News comment

    Prefers Langfuse: open source, fewer bugs, and cheaper, since LangSmith bills per seat and per trace, charges more for retention past 14 days, and caps it at 400 days even self-hosted.

    Published Aug 11, 2025

19. Negative

    NeutralCrane, Hacker News commenter, self-described

    Comment on: Open SWE by LangChainHacker News comment

    Calls LangSmith usable but not great, prefers Langfuse, and says LangSmith more or less requiring LangChain is the biggest knock against it.

    Published Aug 8, 2025

20. Mixed

    Dan Mason, Principal, AI, Stride

    Case Study + Deep Dive: Telemedicine Support Agents with LangGraph/MCPConference workshop with transcript, AI Engineer World's Fair 2025

    Mason reviews every agent conversation and its tool calls in LangSmith and keeps eval datasets there. He runs the evals in his own promptfoo harness to avoid lock-in, and says LangSmith's displayed costs ignored prompt caching.

    Published Jun 22, 2025LangGraph agents Stride built to guide telemedicine patients through at-home treatment by text message

21. Mixed

    fazlerocks, Hacker News poster

    Ask HN: What tools are you using for AI evals? Everything feels half-bakedHacker News post (Ask HN)

    Evaluated LangSmith for production evals: strong tracing, but evals feel secondary, US$0.50 per 1k traces after the free tier adds up, and the UI slows with large datasets.

    Published Jun 5, 2025self-described: Team running LLMs in production for content generation, customer support and code review assistance

22. Mixed

    Karolis Mariunas (kmariunas), Developer (issue author)

    Duplicate traces through OpenAI agents SDKGitHub issue

    Reports one OpenAI Agents SDK run appearing as three LangSmith traces with different IDs, which made the team's dashboards untrustworthy.

    Published Mar 31, 2025self-described: OpenAI Agents SDK app traced to LangSmith

23. Mixed

    h-portia, Reddit user

    Best evaluation platform for LLMs/agentic frameworks for error analysis?Reddit post, r/LLMDevs

    Finds LangSmith handy for high-level eval results and drill-downs, but hard for spotting failure patterns across groups, so builds own charts from its Python library.

    Published Feb 6, 2025self-described: Team running LLM output evals in LangSmith, Python

Discussed without concern, 2

1. Mixed

   Hamel Husain, AI evals consultant and course instructor (co-written with Shreya Shankar)

   AI Evals: Everything You Need to KnowPersonal blog post, hamel.dev

   Names LangSmith among the eval vendors they meet most, sees little feature difference, often builds custom tools on top, and says vendor prompt management adds indirection.

   Published Sep 18, 2026self-described: Advising client teams on eval tooling and vendor selection

2. Positive

   shcallaway, Hacker News commenter

   Comment on: LangChain raises US$125M to build the platform for agent engineeringHacker News comment

   A year-long LangGraph Platform (now Deployments) user found it rough early, with poor monorepo support, but says benefits outweighed costs and the product has since matured.

   Published Oct 21, 2025self-described: Agents on LangGraph TypeScript SDK hosted on LangGraph Platform / Deployments for about a year

Counted for All practitioners.

Raised as a concern, 12

1. Negative

   Hackerjurassicpark, Reddit user, self-described

   Comment on: How will LangChain continue making money?Reddit comment, r/LangChain

   Calls LangSmith very expensive and says they know a few organizations moving to open-source alternatives.

   Published Oct 9, 2026

2. Mixed

   Tyler Norlund (tnorlund), Developer, maintainer of a personal portfolio repo

   Remove the paid-LangSmith wiring now that the subscription is cancelledGitHub issue

   After canceling LangSmith Plus, strips the paid bulk-export wiring in favor of native S3 traces. Keeps the SDK and opt-in tracing of a fraction of runs on a free org for debugging.

   Published Sep 28, 2026self-described: Personal portfolio with LangGraph receipt and QA agents on AWS

3. Negative

   MysteriousPlan6, Reddit user

   Comment on: We built a LangGraph integration for our agent runtimeReddit comment, r/LangChain

   Says LangSmith costs a lot to show a few nested spans, and that the trace view can crash the browser tab on multi-agent loops.

   Published Sep 23, 2026self-described: Multi-agent LangGraph workflows

4. Mixed

   Aviad Rozenhek, Developer and maintainer of the open-source langsmith-cli

   Keep Every LangSmith Trace Without the 10 Retention Billdev.to article

   Calls LangSmith excellent for live debugging but warns extended retention costs 10x a base trace and online evaluators can silently upgrade retention, so archives traces to private S3.

   Published Aug 21, 2026self-described: Operates LangSmith for live debugging of production AI systems; built an S3/Parquet archive workflow for traces

5. Negative

   icekarim, developer of the momo personal assistant

   feat: migrate observability from LangSmith to Langfuse Cloud (v4 SDK)GitHub pull request

   When the LangSmith trial ended the assistant had no observability, so the developer moved tracing and a 16-item eval dataset to Langfuse Cloud's free Hobby tier.

   Published Aug 5, 2026self-described: Solo assistant on Anthropic models deployed on Cloud Run, with scheduled briefing jobs

6. Negative

   type-hinter, Reddit user

   Anyone here using Lang graph with o11y that is not Langsmith?Reddit post, r/LangChain

   Finds LangSmith too expensive for experimental LangGraph projects and self-hosted Langfuse too much work, so asks for other tracing tools.

   Published Jul 30, 2026self-described: Solo, toy and experimental LangGraph projects

7. Mixed

   Aryan Randeriya, founder and CEO of Experience Inc., which builds GAIA

   feat: self-hosted Langfuse alongside LangSmithGitHub pull request

   GAIA added a self-hosted Langfuse beside LangSmith because LangSmith is paid and its development workspace kept hitting the monthly trace limit; LangSmith kept receiving the same traces in parallel during the move.

   Published May 28, 2026self-described: GAIA, an open-source personal AI assistant whose agents, sub-agents and MCP tools are traced in LangSmith

8. Negative

   Bladerunner_7_, Reddit user, self-described

   Are there any genuinely good open-source alternatives to LangSmith right now?Reddit post, r/LangChain

   Looks for open-source LangSmith alternatives because the more useful monitoring features get restrictive once you reach the paywall.

   Published May 15, 2026

9. Mixed

   stingraycharles, Hacker News commenter, self-described

   Comment on: Critical vulnerability in LangChain - CVE-2025-68664Hacker News comment

   Finds LangSmith's prompt engineering workflow okay-ish but laborious, quickly expensive, and only workable for one-turn prompts; multi-turn never works.

   Published Dec 27, 2025

10. Negative

    NeutralCrane, Hacker News commenter, self-described

    Comment on: Open SWE by LangChainHacker News comment

    Prefers Langfuse: open source, fewer bugs, and cheaper, since LangSmith bills per seat and per trace, charges more for retention past 14 days, and caps it at 400 days even self-hosted.

    Published Aug 11, 2025

11. Negative

    Bob (hubdev), Forum user

    Not able to delete not used Extended Data Retention tracesLangChain Forum post

    Prompt experiments with a code evaluator auto-convert traces to Extended Data Retention, hitting usage limits that block new experiments; deleting traces does not lower the count.

    Published Jul 29, 2025self-described: Comparing prompts in LangSmith experiments against a dataset

12. Mixed

    fazlerocks, Hacker News poster

    Ask HN: What tools are you using for AI evals? Everything feels half-bakedHacker News post (Ask HN)

    Evaluated LangSmith for production evals: strong tracing, but evals feel secondary, US$0.50 per 1k traces after the free tier adds up, and the UI slows with large datasets.

    Published Jun 5, 2025self-described: Team running LLMs in production for content generation, customer support and code review assistance

Discussed without concern, 2

1. Negative

   Andrew Kaszubski (akaszubski), Developer, maintainer of the autonomous-dev Claude Code plugin

   feat: self-hosted OTel observability (Phoenix/Langfuse) for multi-agent tracing - not LangSmithGitHub issue

   Rejects LangSmith for multi-agent tracing because self-serve is cloud-only, self-hosting is an Enterprise add-on, and traces go to LangChain infra; picks self-hosted Phoenix or Langfuse.

   Published Aug 6, 2026self-described: Local-first multi-agent coding harness on Claude Code / claude-agent-sdk

2. Positive

   John Capobianco, YouTube creator building AI agents

   LangSmith: Observability for AI AgentsVideo, YouTube

   Walks through adding @traceable and the OpenAI wrapper to their LangGraph agent, praising waterfalls with latency, tokens and cost, and noting it cost them nothing as an individual.

   "…wrapper that wraps around our open AI client so we're just simply going to wrap our openai client and we're going to make traceables at traceable so what does this look like in the code I'm not going to show you myv file but my…"

   Published Mar 9, 2025self-described: Own LangGraph agent project instrumented with @traceable and the OpenAI client wrapper

GILD's fact checks, which don't count as sources

Checked Oct 12, 2026.Plus is US$39 per seat per month with 10,000 base traces included. LangChain's pricing page lists 0.005 LSU per further base trace and 0.0025 LSU to upgrade a trace to extended retention, at US$1 per LSU. Base traces are kept 14 days and extended traces 180 days, and online evaluators and automation rules upgrade the traces they touch unless each opts out. Usage and billing

Counted for All practitioners.

Raised as a concern, 8

1. Mixed

   Berniebiloxi, Developer (issue author)

   Import time: langchain_core.runnables eagerly imports the langsmith SDK via tracer modulesGitHub issue

   Measured that importing core LangChain primitives eagerly loads the whole langsmith SDK, adding roughly 85 to 100 ms of cold start even when nothing is traced.

   Published Sep 6, 2026self-described: CLIs, serverless entry points and test collection importing langchain_core

2. Mixed

   rafio77, Reddit user

   I ran langfuse, langsmith and helicone in prod for a month and only one of them stuckReddit post, r/LLMDevs

   Ran three tools in production for a month: LangSmith was the most complete once wired in, but the team is mostly outside LangChain and kept Langfuse.

   Published May 20, 2026self-described: Team running agents in production, mostly outside LangChain

3. Negative

   Pager_dot, Reddit user, self-described

   How to integrate Langchain and Trace with OpenTelemetry without using LangSmithReddit post, r/LangChain

   Feels they fight LangChain's abstractions and walled garden, and wants to trace with OpenTelemetry and skip LangSmith.

   Published May 11, 2026

4. Negative

   threecheese, Hacker News commenter

   Comment on: Show HN: Sim - Apache-2.0 n8n alternativeHacker News comment

   Likes LangGraph's abstractions but feels LangSmith is creeping further into LangChain and LangGraph, with AWS-like lock-in vibes that make them hesitant to invest.

   Published Dec 12, 2025self-described: Builds agents with LangGraph workflows

5. Negative

   NeutralCrane, Hacker News commenter, self-described

   Comment on: Open SWE by LangChainHacker News comment

   Calls LangSmith usable but not great, prefers Langfuse, and says LangSmith more or less requiring LangChain is the biggest knock against it.

   Published Aug 8, 2025

6. Negative

   senko, Hacker News commenter

   Comment on: Use the Gemini API with OpenAI Fallback in TypeScriptHacker News comment

   Says LangChain's docs and tutorials now work as content marketing for LangGraph Platform and LangSmith, so they built their own thin wrapper instead.

   Published Apr 7, 2025self-described: Solo, own thin LLM wrapper

7. Negative

   Amit, NonBioS author

   The NonBioS LLM Observability PickCompany blog post, NonBioS.ai

   Ruled LangSmith out because it seemed tightly coupled to LangChain, which they had removed, and chose open-source, on-prem Langfuse for data governance.

   "Langsmith appeared popular, but we had encountered challenges with Langchain from the same company, finding it overly complex for previous NonBioS tooling. We rewrote our systems to remove dependencies on Langchain and chose not to proceed with Langsmith as it seemed strongly coupled with Langchain."

   Published Mar 5, 2025self-described: NonBioS AI Engineer product; runs consume over a million tokens and about a hundred inference calls in five minutes, with sensitive customer data

8. Mixed

   Total_Listen_4289, Reddit user

   Spent 3 months evaluating Galileo, Arize, Langfuse and LangSmith for production LLM monitoring. Notes for anyone doing the same.Reddit post, r/LLMDevs

   After a three-month evaluation the small team found LangSmith smoothest inside LangChain, but chose Langfuse over its LangChain ties and Enterprise-only self-hosting.

   Published May 30, 2026self-described: Small team shipping agents

Discussed without concern, 1

1. Positive

   Markus Eisele, Java developer and newsletter author

   Trace a Quarkus LangChain4j App in LangSmithNewsletter, The Main Thread (Substack)

   Java teams can get LangSmith traces over plain OTLP with no custom instrumentation, though they would add sampling or redaction before shipping full prompts from production.

   Published May 30, 2026self-described: Example Quarkus LangChain4j support assistant on Ollama exporting OpenTelemetry traces to LangSmith

GILD's fact checks, which don't count as sources

Checked Oct 12, 2026.LangSmith documents tracing without LangChain through its SDK wrappers and OpenTelemetry. LangSmith Deployment runs agents built with Google ADK, the Claude Agent SDK, Strands, CrewAI, and AutoGen as well as LangGraph. LangSmith Deployment

Counted for All practitioners.

Raised as a concern, 8

1. Negative

   Andre Kanmegne, on andrelair-platform, his personal homelab project

   refactor(observability): drop LangSmith, Langfuse as the single LLMOps backboneGitHub pull request

   Removed LangSmith from the RAG backend of his personal homelab platform, which the author calls an external US cloud service sending traces out of an EU-resident cluster, and moved its five LangChain call sites to the Langfuse instance already running in the cluster.

   Published Oct 11, 2026self-described: RAG and contract-analysis services on a self-hosted Kubernetes cluster with EU data residency and outbound traffic denied by default

2. Negative

   obarton, contributor to Cue Labs' cue-app repository

   Consolidate LLM observability on Langfuse; retire LangSmithGitHub pull request

   Cue Labs retired its LangSmith tracing and moved production tracing and evals to Langfuse, citing data residency for customer messages, personal data and one vendor standard, and kept the option to self-host Langfuse so that data stays inside its own boundary.

   Published Oct 9, 2026self-described: Cue Labs' customer-support agents for Instagram direct messages, chat and SMS, built on LangChain and LangGraph

3. Negative

   pallasaisrujan28, developer of an enterprise agent framework

   feat: replace LangSmith with self-hosted LangfuseGitHub pull request

   Replaced LangSmith with a self-hosted Langfuse inside the project's EKS cluster in eu-west-2, so traces never leave the cluster.

   Published Aug 25, 2026self-described: Agent framework deployed on Amazon EKS in eu-west-2

4. Negative

   Andrew Kaszubski (akaszubski), Developer, maintainer of the autonomous-dev Claude Code plugin

   feat: self-hosted OTel observability (Phoenix/Langfuse) for multi-agent tracing - not LangSmithGitHub issue

   Rejects LangSmith for multi-agent tracing because self-serve is cloud-only, self-hosting is an Enterprise add-on, and traces go to LangChain infra; picks self-hosted Phoenix or Langfuse.

   Published Aug 6, 2026self-described: Local-first multi-agent coding harness on Claude Code / claude-agent-sdk

5. No opinion stated

   Folarin Akinloye, AI engineer in London

   How to Evaluate Agents Locally with Arize PhoenixPersonal blog post, folarin.dev

   Akinloye, who also wrote a LangSmith evaluation guide, finds LangSmith more polished inside LangChain and LangGraph but picks Phoenix when evaluation data must stay on the team's own machines.

   Published Jul 12, 2026self-described: Agent evaluation when traces cannot leave the team's own infrastructure

6. Mixed

   Total_Listen_4289, Reddit user

   Spent 3 months evaluating Galileo, Arize, Langfuse and LangSmith for production LLM monitoring. Notes for anyone doing the same.Reddit post, r/LLMDevs

   After a three-month evaluation the small team found LangSmith smoothest inside LangChain, but chose Langfuse over its LangChain ties and Enterprise-only self-hosting.

   Published May 30, 2026self-described: Small team shipping agents

7. Mixed

   Aryan Randeriya, founder and CEO of Experience Inc., which builds GAIA

   feat: self-hosted Langfuse alongside LangSmithGitHub pull request

   GAIA added a self-hosted Langfuse beside LangSmith because LangSmith is paid and its development workspace kept hitting the monthly trace limit; LangSmith kept receiving the same traces in parallel during the move.

   Published May 28, 2026self-described: GAIA, an open-source personal AI assistant whose agents, sub-agents and MCP tools are traced in LangSmith

8. Negative

   Amit, NonBioS author

   The NonBioS LLM Observability PickCompany blog post, NonBioS.ai

   Ruled LangSmith out because it seemed tightly coupled to LangChain, which they had removed, and chose open-source, on-prem Langfuse for data governance.

   "Langsmith appeared popular, but we had encountered challenges with Langchain from the same company, finding it overly complex for previous NonBioS tooling. We rewrote our systems to remove dependencies on Langchain and chose not to proceed with Langsmith as it seemed strongly coupled with Langchain."

   Published Mar 5, 2025self-described: NonBioS AI Engineer product; runs consume over a million tokens and about a hundred inference calls in five minutes, with sensitive customer data

Discussed without concern, 4

1. Negative

   Chris (chrizefan), Owner/approver of tooling decisions at digithings.ai

   epic: digitrace rename + self-hosted Langfuse on Cloudflare (cut LangSmith)GitHub issue

   Approved an epic to replace LangSmith with self-hosted Langfuse on Cloudflare, then on October 9, 2026 switched the plan to Cloudflare-native tracing with MLflow, keeping LangSmith until a dual-run parity window proves the new path.

   Published Oct 1, 2026self-described: digithings.ai LangGraph workloads (house-run, OpenWiki, digiquant-baseline)

2. Mixed

   Aviad Rozenhek, Developer and maintainer of the open-source langsmith-cli

   Keep Every LangSmith Trace Without the 10 Retention Billdev.to article

   Calls LangSmith excellent for live debugging but warns extended retention costs 10x a base trace and online evaluators can silently upgrade retention, so archives traces to private S3.

   Published Aug 21, 2026self-described: Operates LangSmith for live debugging of production AI systems; built an S3/Parquet archive workflow for traces

3. Positive

   Markus Eisele, Java developer and newsletter author

   Trace a Quarkus LangChain4j App in LangSmithNewsletter, The Main Thread (Substack)

   Java teams can get LangSmith traces over plain OTLP with no custom instrumentation, though they would add sampling or redaction before shipping full prompts from production.

   Published May 30, 2026self-described: Example Quarkus LangChain4j support assistant on Ollama exporting OpenTelemetry traces to LangSmith

4. Negative

   NeutralCrane, Hacker News commenter, self-described

   Comment on: Open SWE by LangChainHacker News comment

   Prefers Langfuse: open source, fewer bugs, and cheaper, since LangSmith bills per seat and per trace, charges more for retention past 14 days, and caps it at 400 days even self-hosted.

   Published Aug 11, 2025

GILD's fact checks, which don't count as sources

Checked Oct 12, 2026.Self-hosted LangSmith is an add-on to the Enterprise plan. LangSmith Cloud runs in GCP US, EU, and APAC regions and an AWS US region, and the SDK can hide or mask inputs and outputs before they are sent. Self-hosted LangSmith

Counted for All practitioners.

Raised as a concern, 4

1. Negative

   Andrew Kaszubski (akaszubski), Developer, maintainer of the autonomous-dev Claude Code plugin

   feat: self-hosted OTel observability (Phoenix/Langfuse) for multi-agent tracing - not LangSmithGitHub issue

   Rejects LangSmith for multi-agent tracing because self-serve is cloud-only, self-hosting is an Enterprise add-on, and traces go to LangChain infra; picks self-hosted Phoenix or Langfuse.

   Published Aug 6, 2026self-described: Local-first multi-agent coding harness on Claude Code / claude-agent-sdk

2. Negative

   Pager_dot, Reddit user, self-described

   How to integrate Langchain and Trace with OpenTelemetry without using LangSmithReddit post, r/LangChain

   Feels they fight LangChain's abstractions and walled garden, and wants to trace with OpenTelemetry and skip LangSmith.

   Published May 11, 2026

3. Negative

   threecheese, Hacker News commenter

   Comment on: Show HN: Sim - Apache-2.0 n8n alternativeHacker News comment

   Likes LangGraph's abstractions but feels LangSmith is creeping further into LangChain and LangGraph, with AWS-like lock-in vibes that make them hesitant to invest.

   Published Dec 12, 2025self-described: Builds agents with LangGraph workflows

4. Negative

   NeutralCrane, Hacker News commenter, self-described

   Comment on: Open SWE by LangChainHacker News comment

   Prefers Langfuse: open source, fewer bugs, and cheaper, since LangSmith bills per seat and per trace, charges more for retention past 14 days, and caps it at 400 days even self-hosted.

   Published Aug 11, 2025

Discussed without concern, 2

1. Mixed

   Dan Mason, Principal, AI, Stride

   Case Study + Deep Dive: Telemedicine Support Agents with LangGraph/MCPConference workshop with transcript, AI Engineer World's Fair 2025

   Mason reviews every agent conversation and its tool calls in LangSmith and keeps eval datasets there. He runs the evals in his own promptfoo harness to avoid lock-in, and says LangSmith's displayed costs ignored prompt caching.

   Published Jun 22, 2025LangGraph agents Stride built to guide telemedicine patients through at-home treatment by text message

2. Negative

   Amit, NonBioS author

   The NonBioS LLM Observability PickCompany blog post, NonBioS.ai

   Ruled LangSmith out because it seemed tightly coupled to LangChain, which they had removed, and chose open-source, on-prem Langfuse for data governance.

   "Langsmith appeared popular, but we had encountered challenges with Langchain from the same company, finding it overly complex for previous NonBioS tooling. We rewrote our systems to remove dependencies on Langchain and chose not to proceed with Langsmith as it seemed strongly coupled with Langchain."

   Published Mar 5, 2025self-described: NonBioS AI Engineer product; runs consume over a million tokens and about a hundred inference calls in five minutes, with sensitive customer data