Every source behind the read

125 sources in All practitioners, with product documentation Showing all 125 sources

1. Positive

   Pablo Sebastian Martinez, Production Engineering team, monday.com

   How We Gave AI Agents the Keys to Production and Still Sleep at NightCompany engineering blog, engineering.monday.com

   monday.com's Production Engineering team gives every Infra agent run one parent trace in LangSmith as its audit trail, so the team can reconstruct any run after an incident.

   "Each run has one parent trace in LangSmith and JSON logs to stdout, and every call that crosses a boundary is traced so the run can be reconstructed afterwards."

   Published Oct 26, 2026self-described: monday.com Infra agents platform, where agents resolve infrastructure tickets and approve production pull requests

   Counts toward

   • Overall signal
   • Trace-Level Debugging: Positive
   • Agents in production need tracing

   Published on

   Engineering Blogs & Reviews

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

2. Positive

   Dan Haggerty, agents engineer at NVIDIA

   Add configurable LangSmith tags to Compass telemetryGitHub pull request

   Added LangSmith tags to NeMo Compass's OpenTelemetry root traces so the engineer can filter traces by issue and compare runs before and after a code change; a live check loaded 160 traces with the tags stored on the root.

   Published Oct 22, 2026self-described: NVIDIA NeMo Compass, whose development traces reach LangSmith over OpenTelemetry

   Counts toward

   • Overall signal
   • Trace-Level Debugging: Positive
   • Useful without LangChain

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Measured

   Applies to

   Current

   Observed on

   OpenTelemetry export with LangSmith's span tag attribute, no LangChain

3. Negative

   Andre Kanmegne, on andrelair-platform, his personal homelab project

   refactor(observability): drop LangSmith, Langfuse as the single LLMOps backboneGitHub pull request

   Removed LangSmith from the RAG backend of his personal homelab platform, which the author calls an external US cloud service sending traces out of an EU-resident cluster, and moved its five LangChain call sites to the Langfuse instance already running in the cluster.

   Published Oct 11, 2026self-described: RAG and contract-analysis services on a self-hosted Kubernetes cluster with EU data residency and outbound traffic denied by default

   Counts toward

   • Overall signal
   • Self-Hosting & Data Control: Raised as a concern
   • Open-source tools are the alternative
   • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

   Published on

   Developer Communities

   Who

   Individual builder

   Evidence

   Reported

   Applies to

   Current

   Observed on

   LangSmith callbacks beside a self-hosted Langfuse

4. Negative

   Hackerjurassicpark, Reddit user, self-described

   Comment on: How will LangChain continue making money?Reddit comment, r/LangChain

   Calls LangSmith very expensive and says they know a few organizations moving to open-source alternatives.

   Published Oct 9, 2026

   Counts toward

   • Overall signal
   • Seat & Trace Pricing: Raised as a concern
   • Open-source tools are the alternative

   Published on

   Developer Communities

   Who

   Individual builder

   Evidence

   Reported

   Applies to

   Current

5. Negative

   obarton, contributor to Cue Labs' cue-app repository

   Consolidate LLM observability on Langfuse; retire LangSmithGitHub pull request

   Cue Labs retired its LangSmith tracing and moved production tracing and evals to Langfuse, citing data residency for customer messages, personal data and one vendor standard, and kept the option to self-host Langfuse so that data stays inside its own boundary.

   Published Oct 9, 2026self-described: Cue Labs' customer-support agents for Instagram direct messages, chat and SMS, built on LangChain and LangGraph

   Counts toward

   • Overall signal
   • Self-Hosting & Data Control: Raised as a concern
   • Open-source tools are the alternative
   • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

   Published on

   Developer Communities

   Who

   Small product team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   LangSmith tracing turned on by environment variables only

6. Mixed

   Unlucky_Emu6094, Reddit user

   Comment on: When you change a RAG pipeline, how do you know which queries actually regressed?Reddit comment, r/LangChain

   Likes LangSmith for traces but exports results to a spreadsheet for paired query comparisons, because LangSmith does not lead them to a screen built for that comparison.

   Published Oct 8, 2026self-described: RAG pipeline changes

   Counts toward

   • Overall signal
   • Trace-Level Debugging: Positive
   • Datasets & Evaluations: Mixed or negative

   Published on

   Developer Communities

   Who

   Individual builder

   Evidence

   Reported

   Applies to

   Current

7. Negative

   Chris (chrizefan), Owner/approver of tooling decisions at digithings.ai

   epic: digitrace rename + self-hosted Langfuse on Cloudflare (cut LangSmith)GitHub issue

   Approved an epic to replace LangSmith with self-hosted Langfuse on Cloudflare, then on October 9, 2026 switched the plan to Cloudflare-native tracing with MLflow, keeping LangSmith until a dual-run parity window proves the new path.

   Published Oct 1, 2026self-described: digithings.ai LangGraph workloads (house-run, OpenWiki, digiquant-baseline)

   Counts toward

   • Overall signal
   • Self-Hosting & Data Control: Discussed without concern
   • Open-source tools are the alternative
   • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

   Published on

   Developer Communities

   Who

   Technical leadership

   Evidence

   Reported

   Applies to

   Current

   Observed on

   with LangGraph

8. Positive

   Aniruddha Laud and Anshul Pahwa, AI engineers, Apollo.io

   From Supervisor to Deep Agents: Re-Architecting a Production GTM Assistant Without Breaking TrustConference talk, Interrupt NYC 2026 (LangChain YouTube channel)

   Apollo.io runs its observability and every product eval in LangSmith, and named that investment as one of three reasons it rebuilt its assistant on LangChain's Deep Agents.

   "We wanted some capabilities that are specific to Apollo. That's why we use this thing. And Lang-native. We are heavily invested in LangSmith in terms of our observability stack. All the evals that we introduced to our product are tied to LangSmith. So all three…"

   Published Sep 29, 2026self-described: Apollo.io AI assistant for go-to-market teams, in general availability

   Not counted

   Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

   Published on

   Podcasts, Talks & Videos

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

9. Positive

   Dotan Limon and Jordan Kamm, Chief Product and Technology Officer, Weber Shandwick; agent engineer, Focused

   From One-Off Agents to an Agent FactoryConference talk, Interrupt NYC 2026 (LangChain YouTube channel)

   Weber Shandwick checked a re-architected production agent against its LangSmith quality baselines before moving other agents to the new design.

   "And as an agent engineer, I basically live in LangSmith. It is my bread and butter, and having the tools available to not only observe production traffic, but measure against baselines and like be confident that I can tell Dotan that yes, we re-architected this…"

   Published Sep 29, 2026self-described: Weber Shandwick client agents built with Focused, one reusable architecture per brand

   Not counted

   Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

   Published on

   Podcasts, Talks & Videos

   Who

   Technical leadership

   Evidence

   Reported

   Applies to

   Current

10. Positive

    Marissa Saunders, Senior Director of Engineering, Spring Health

    The Safety Net Behind the Conversation: LangSmith for Mental Health AIConference talk, Interrupt NYC 2026 (LangChain YouTube channel)

    Clinicians write rubrics and review guardrail flags in LangSmith annotation queues, and non-technical staff tune prompts and start evaluation runs without an engineer.

    "You need a set of rubrics or standards so that you can iterate quickly on, is this good? And you have to enable them to give iterative feedback and work in annotation queues. Now, tools like LangSmith have helped us make this really accessible, not…"

    Published Sep 29, 2026self-described: Spring Health, member-facing AI agent for mental health care

    Not counted

    Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

    Published on

    Podcasts, Talks & Videos

    Who

    Technical leadership

    Evidence

    Reported

    Applies to

    Current

11. Mixed

    Tyler Norlund (tnorlund), Developer, maintainer of a personal portfolio repo

    Remove the paid-LangSmith wiring now that the subscription is cancelledGitHub issue

    After canceling LangSmith Plus, strips the paid bulk-export wiring in favor of native S3 traces. Keeps the SDK and opt-in tracing of a fraction of runs on a free org for debugging.

    Published Sep 28, 2026self-described: Personal portfolio with LangGraph receipt and QA agents on AWS

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Seat & Trace Pricing: Raised as a concern
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Plus plan cancelled; free Developer org kept for opt-in tracing

12. Mixed

    Tejasri Teyam, developer filing a GitHub issue

    Bug: First turn intermittently missing from LangSmith Trajectory viewGitHub issue

    The Trajectory view sometimes starts at the second turn of a multi-turn trace while the stats panel still counts every turn, so the first turn cannot be reviewed there.

    Published Sep 25, 2026self-described: Multi-turn LangGraph application traced to LangSmith Cloud

    Counts toward, pending a recheck

    • Overall signal
    • Trace-Level Debugging: Mixed or negative
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    LangSmith Cloud Trajectory view, with LangGraph

13. Negative

    MysteriousPlan6, Reddit user

    Comment on: We built a LangGraph integration for our agent runtimeReddit comment, r/LangChain

    Says LangSmith costs a lot to show a few nested spans, and that the trace view can crash the browser tab on multi-agent loops.

    Published Sep 23, 2026self-described: Multi-agent LangGraph workflows

    Counts toward

    • Overall signal
    • Seat & Trace Pricing: Raised as a concern
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

14. Mixed

    Maverick (mavericksea-ai), builder of Driftproof, an evaluation reliability product

    Issue: evaluate_comparative with randomize_order records concurrent comparators' scores against the wrong runsGitHub issue

    With two comparative evaluators and randomized order, evaluate_comparative recorded 15 to 19 of 40 preferences on the wrong run in the author's reproduction, with no error and plausible-looking scores.

    Published Sep 23, 2026self-described: Pairwise LLM-judge experiments with two comparative evaluators and randomized order

    Not counted

    Its author builds Driftproof, an evaluation reliability product that competes with LangSmith's evaluators. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    langsmith 0.14.0 evaluate_comparative

15. Positive

    Patrick Porche Jr, product engineer building AI tools with clinicians and operations teams

    Move the pipeline onto LangGraph, LangChain and LangSmith; remove LogfireGitHub pull request

    Replaced Logfire with LangSmith tracing and moved the pipeline's evals from pydantic-evals to LangSmith datasets and experiments, sending only allowlisted keys so patient data stays out of traces.

    Published Sep 21, 2026self-described: Medical-necessity review pipeline on Anthropic and Google models, with a patient-data boundary

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Positive
    • Strongest with LangGraph
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith tracing and experiments replacing Logfire and pydantic-evals, with LangGraph

16. Mixed

    JamesDun, developer filing a GitHub issue

    claude_agent_sdk integration: receive_messages() is not instrumented, traces silently lost for the full-stream consumption pathGitHub issue

    LangSmith's Claude Agent SDK integration recorded no traces at all when an app read the SDK's continuous message stream, the path long-lived multi-turn sessions use, and gave no warning or error.

    Published Sep 20, 2026self-described: Interactive multi-turn sessions on the Claude Agent SDK, traced through LangSmith's integration

    Counts toward, pending a recheck

    • Overall signal
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    langsmith 0.8.18 to 0.13.0 with claude-agent-sdk 0.2.144 and Claude Sonnet 4.5

17. Mixed

    Hamel Husain, AI evals consultant and course instructor (co-written with Shreya Shankar)

    AI Evals: Everything You Need to KnowPersonal blog post, hamel.dev

    Names LangSmith among the eval vendors they meet most, sees little feature difference, often builds custom tools on top, and says vendor prompt management adds indirection.

    Published Sep 18, 2026self-described: Advising client teams on eval tooling and vendor selection

    Counts toward

    • Overall signal
    • Playground & Prompt Versioning: Mixed or negative
    • UI & SDK Reliability: Discussed without concern

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

18. Negative

    Daniel Payne, Developer (self-described long-time programmer)

    Why I think Vercel EVE Web interface is so much better than Langsmith for development.Medium article

    Prefers a custom Eve web channel for development because debugging in LangSmith meant digging through raw JSON inputs and outputs, which felt too manual.

    Published Sep 16, 2026self-described: Solo app that values building-material photos with a Vercel Eve agent; previously used LangGraph and LangSmith

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Mixed or negative
    • UI & SDK Reliability: Raised as a concern
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Compared against a custom Vercel Eve web channel

19. Positive

    regnull, Reddit user

    I open-sourced a multi-bot runtime that uses LangMem and LangSmithReddit post, r/LangChain

    Built a self-hosted multi-bot runtime on LangMem and reports that LangSmith tracing works out of the box.

    Published Sep 14, 2026self-described: Solo, open-source self-hosted multi-bot platform

    Counts toward

    • Overall signal
    • Quick Setup: Positive

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

20. Mixed

    Capable-Purpose9911, Reddit user

    Comment on: What is the best AI observability tool in 2026?Reddit comment, r/LangChain

    Finds LangSmith good for analytics but measured it using about 2 GB of a 4 to 4.5 GB app's memory.

    Published Sep 9, 2026self-described: Small app on a 4 to 4.5 GB RAM host

    Counts toward

    • Overall signal
    • Production Monitoring: Positive
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

21. Positive

    Kamie Shami-Schnitzer, feedAgent builder, monday.com

    Building a Robust Harness for agent in production: Feed agent case-studyCompany engineering blog, engineering.monday.com

    monday.com wraps each feedAgent run in LangSmith's traceable(), which records every intermediate step and shows which tools ran and what they returned; its evals run on internal monday.com tooling.

    "We use an LLM as a judge with specific metrics for each part, and run it on the CI to make sure we have no regressions before deploying to production."

    Published Sep 9, 2026self-described: monday.com feedAgent, a deep agent that curates a personal activity feed for each workspace member

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Agents in production need tracing

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Evals run on monday.com's internal tooling

22. Mixed

    Berniebiloxi, Developer (issue author)

    Import time: langchain_core.runnables eagerly imports the langsmith SDK via tracer modulesGitHub issue

    Measured that importing core LangChain primitives eagerly loads the whole langsmith SDK, adding roughly 85 to 100 ms of cold start even when nothing is traced.

    Published Sep 6, 2026self-described: CLIs, serverless entry points and test collection importing langchain_core

    Counts toward

    • Overall signal
    • LangChain Coupling: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

23. Positive

    Titus Lim (tituslhy), Data scientist (GitHub profile)

    Cut local stack from ~47 containers to 14; Langfuse → LangSmithGitHub pull request

    Swaps self-hosted Langfuse and its ClickHouse/MinIO containers for hosted LangSmith tracing, shrinking a local agent template stack from about 47 containers to 14.

    Published Sep 3, 2026self-described: Agent template repo (AG-UI / A2UI / A2A) on LangGraph

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Quick Setup: Positive
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Hosted LangSmith replacing self-hosted Langfuse

24. Negative

    pallasaisrujan28, developer of an enterprise agent framework

    feat: replace LangSmith with self-hosted LangfuseGitHub pull request

    Replaced LangSmith with a self-hosted Langfuse inside the project's EKS cluster in eu-west-2, so traces never leave the cluster.

    Published Aug 25, 2026self-described: Agent framework deployed on Amazon EKS in eu-west-2

    Counts toward

    • Overall signal
    • Self-Hosting & Data Control: Raised as a concern
    • Open-source tools are the alternative
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

25. Mixed

    Aviad Rozenhek, Developer and maintainer of the open-source langsmith-cli

    Keep Every LangSmith Trace Without the 10 Retention Billdev.to article

    Calls LangSmith excellent for live debugging but warns extended retention costs 10x a base trace and online evaluators can silently upgrade retention, so archives traces to private S3.

    Published Aug 21, 2026self-described: Operates LangSmith for live debugging of production AI systems; built an S3/Parquet archive workflow for traces

    Counts toward, pending a recheck

    • Overall signal
    • Trace-Level Debugging: Positive
    • Seat & Trace Pricing: Raised as a concern
    • Self-Hosting & Data Control: Discussed without concern
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    Base 14-day vs extended 400-day trace retention

26. Positive

    True-Homework6226, Reddit user

    Comment on: How are you actually debugging complex LangChain agents?Reddit comment, r/LangChain

    Debugs agents by opening the full run in LangSmith, finding where a tool call or context diverged, and searching backward from there.

    Published Aug 10, 2026self-described: Multi-step LangChain agents

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Agents in production need tracing

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

27. Negative

    Andrew Kaszubski (akaszubski), Developer, maintainer of the autonomous-dev Claude Code plugin

    feat: self-hosted OTel observability (Phoenix/Langfuse) for multi-agent tracing - not LangSmithGitHub issue

    Rejects LangSmith for multi-agent tracing because self-serve is cloud-only, self-hosting is an Enterprise add-on, and traces go to LangChain infra; picks self-hosted Phoenix or Langfuse.

    Published Aug 6, 2026self-described: Local-first multi-agent coding harness on Claude Code / claude-agent-sdk

    Counts toward

    • Overall signal
    • Seat & Trace Pricing: Discussed without concern
    • Self-Hosting & Data Control: Raised as a concern
    • Closed Source & Vendor Dependence: Raised as a concern
    • Open-source tools are the alternative
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenTelemetry

28. Negative

    icekarim, developer of the momo personal assistant

    feat: migrate observability from LangSmith to Langfuse Cloud (v4 SDK)GitHub pull request

    When the LangSmith trial ended the assistant had no observability, so the developer moved tracing and a 16-item eval dataset to Langfuse Cloud's free Hobby tier.

    Published Aug 5, 2026self-described: Solo assistant on Anthropic models deployed on Cloud Run, with scheduled briefing jobs

    Counts toward

    • Overall signal
    • Seat & Trace Pricing: Raised as a concern
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith trial, then Langfuse Cloud Hobby

29. Negative

    type-hinter, Reddit user

    Anyone here using Lang graph with o11y that is not Langsmith?Reddit post, r/LangChain

    Finds LangSmith too expensive for experimental LangGraph projects and self-hosted Langfuse too much work, so asks for other tracing tools.

    Published Jul 30, 2026self-described: Solo, toy and experimental LangGraph projects

    Counts toward

    • Overall signal
    • Seat & Trace Pricing: Raised as a concern
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

30. Mixed

    Gabriel Gouvêa, software engineer at movedot

    Amazon Bedrock Claude Sonnet 5 US geo profile is priced at the global rateGitHub issue

    Gabriel Gouvêa found LangSmith pricing Bedrock's US-geo Claude Sonnet 5 runs at the global rate of US$2.00 per million input tokens, where AWS lists US$2.20, so cost totals for those runs read low.

    Published Jul 28, 2026self-described: Hosted LangSmith cost tracking for Claude Sonnet 5 runs through Amazon Bedrock's US cross-region profile

    Counts toward, pending a recheck

    • Overall signal
    • Production Monitoring: Mixed or negative

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    Hosted LangSmith model price map, Claude Sonnet 5 on Amazon Bedrock

31. Positive

    ambujsystems, Reddit user

    Comment on: My 10-Node Agentic RAG Architecture: Combining LangGraph, Cohere, Pinecone & MCP for Dense Legal ParsingReddit comment, r/LangChain

    Relies on LangSmith traces to tell whether a failure came from parsing, retrieval, reranking, or generation in a 10-node LangGraph RAG pipeline.

    Published Jul 26, 2026self-described: Solo, 10-node LangGraph RAG pipeline on free infrastructure

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Strongest with LangGraph

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

32. Positive

    Akele Reed and Dave Revere, Principal AI Engineer and Staff AI Engineer, SonderMind

    Evals Driven-Development: Engineering a Mental Health AI Coach Ethically & SafelyConference talk with transcript, AI Engineer World's Fair 2026

    SonderMind clinicians annotate production conversations in a LangSmith annotation queue, and a script turns each annotation into a typed eval that every prompt or model change must pass in CI.

    "A clinical SME reviews a production conversation in LangSmith, marks the missed disclosure with an expected observation, and the script writes it to the eval set."

    Published Jul 25, 2026Sonder, SonderMind's AI mental health coach, with guardrail evals written from clinician annotations

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Positive
    • Evals gate prompt and model changes

    Published on

    Podcasts, Talks & Videos

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

33. Positive

    dataplayer, Reddit user, self-described

    Comment on: We tested LangGraph against 8 other frameworks in 2026Reddit comment, r/LangChain

    Asks how other agent frameworks could produce metrics and traces as good as LangSmith's.

    Published Jul 24, 2026

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Strongest with LangGraph
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

34. Positive

    Evan Kormos, Engineering manager, Coinbase

    How Coinbase Builds Developer Support Agents | Interrupt 26Conference talk, Interrupt 26 (LangChain YouTube channel)

    Coinbase's developer support team traces its agents in self-hosted LangSmith, links triage messages to traces, and mines traces with the LangSmith CLI and Claude Code.

    "Slack channels, new MCP channels, generative UX, a lot is possible with this setup. A huge unlock for us was including Python services as first-class infrastructure. This was the team's first service of this type. We paired it with self-hosted LangSmith…"

    Published Jul 20, 2026Coinbase developer support team, Discord and Slack support agents

    Not counted

    Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

    Published on

    Podcasts, Talks & Videos

    Who

    Technical leadership

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Self-hosted

35. Mixed

    automation_experto, Reddit user

    Comment on: DoubtReddit comment, r/LangChain

    Says LangSmith showed normal-looking traces while agents reasoned over bad extracted fields, so the real bug sat upstream of the trace.

    Published Jul 19, 2026self-described: Agents built on document extraction

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Mixed or negative

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

36. Positive

    Nick Ung and Akshay Sharma, data science manager and machine learning engineer, Lyft

    Build Evals That Actually MatterConference talk with transcript, AI Engineer World's Fair 2026

    Lyft reads each AI Assist run's reasoning, help articles and tool calls in LangSmith traces, sends failed traces to an annotation queue where product managers and QA label them, and measures each LLM judge's precision and recall against about 100 hand-labeled examples.

    Published Jul 19, 2026self-described: Lyft AI Assist customer-support agents, with LLM judges checked against about 100 hand-labeled pass and fail examples

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Datasets & Evaluations: Positive
    • Agents in production need tracing

    Published on

    Podcasts, Talks & Videos

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

37. Positive

    GeneralMongoose5979, Reddit user

    Comment on: Anyone else feel like LangChain is amazing for a demo, but a nightmare for production?Reddit comment, r/LangChain

    Has run LangSmith and LangGraph in production at modest volume and reports both have performed quite well.

    Published Jul 17, 2026self-described: LangGraph agents in production at low volume

    Counts toward

    • Overall signal
    • Strongest with LangGraph
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

38. Positive

    Juan Cruz Boullosa Gutierrez, contributor to a LangGraph Discord bot

    Replace custom tracing with LangSmith observabilityGitHub pull request

    Deleted the bot's own trace recorder, span helpers and readers in favor of LangSmith's LangGraph auto-instrumentation set by environment variables, with every input and output hidden and only allowlisted metadata sent.

    Published Jul 16, 2026self-described: LangGraph Discord bot with direct-message and crisis paths, traced with inputs and outputs hidden

    Counts toward

    • Overall signal
    • Quick Setup: Positive
    • Strongest with LangGraph
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith auto-instrumentation with LANGSMITH_HIDE_INPUTS and LANGSMITH_HIDE_OUTPUTS, with LangGraph

39. Positive

    Ravi Chandu Ummadisetti and Kordel France, Head of agent AI and product research, and head of AI engineering, Toyota

    Building ToyotaGPT: 50+ Production Agents, One Config File, Zero Architecture ReviewsConference talk, Interrupt 26 (LangChain YouTube channel)

    Toyota's enterprise AI team builds 50+ production agents on LangGraph and uses LangSmith to watch every agent and find the cause of a failing tool call without reading logs.

    "…going well? You allocate resources to bolster another part of the manufacturing line that's dwindling. And LangSmith is the literal embodiment of an Andon board. We can see observability over all of our agents in real time. Understand what tool calls aren't…"

    Published Jul 15, 2026self-described: ToyotaGPT, Toyota's enterprise agent platform with 50+ agents in production

    Not counted

    Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

    Published on

    Podcasts, Talks & Videos

    Who

    Organization-wide

    Evidence

    Reported

    Applies to

    Current

40. No opinion stated

    Folarin Akinloye, AI engineer in London

    How to Evaluate Agents Locally with Arize PhoenixPersonal blog post, folarin.dev

    Akinloye, who also wrote a LangSmith evaluation guide, finds LangSmith more polished inside LangChain and LangGraph but picks Phoenix when evaluation data must stay on the team's own machines.

    Published Jul 12, 2026self-described: Agent evaluation when traces cannot leave the team's own infrastructure

    Counts toward

    • Self-Hosting & Data Control: Raised as a concern
    • Open-source tools are the alternative
    • Does LangSmith earn its place outside the LangChain ecosystem?: Most useful inside LangChain

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith compared with self-hosted Arize Phoenix

41. Mixed

    Antaripa Saha and Hamel Husain, AI evals consultants, Parlance Labs

    Do Automated Evals Work?Independent evaluation, Parlance Labs blog

    Parlance Labs scored LangSmith's chat agent at 79.5% recall and 77.5% precision against 39 hand-labeled failures, after LangSmith Engine surfaced too few issues to score. LangSmith had the best interface for reading single traces, and several of its wrong findings were marked high severity.

    "But it surfaced only a small set of issues, far fewer than a full review turns up, so we could not use it for the 100-trace comparison. We ran the scored experiment with LangSmith’s chat agent instead."

    Published Jul 11, 2026self-described: 100 production traces from Nurture Boss's apartment-leasing agent, hand-labeled with a domain expert into 39 failures, then given to Braintrust Loop, Arize Alyx, LangSmith and three coding agents

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Datasets & Evaluations: Mixed or negative

    Published on

    Engineering Blogs & Reviews

    Who

    Small product team

    Evidence

    Measured

    Applies to

    Current

    Observed on

    LangSmith Engine and LangSmith's chat agent, July 2026

42. Positive

    Hugo Santana, Builder of Applied, author of The Applied Report

    My AI Agents Were Working, but I Had No VisibilityNewsletter, Agent-led Growth (Substack)

    Connected non-LangChain agents to LangSmith without rebuilding them and now sees historical traces, token spend per agent, failures and latency.

    Published Jul 7, 2026self-described: Solo; custom multi-agent system (Applied) sending run summaries to LangSmith

    Counts toward

    • Overall signal
    • Quick Setup: Positive
    • Production Monitoring: Positive
    • Agents in production need tracing
    • Useful without LangChain
    • Does LangSmith earn its place outside the LangChain ecosystem?: Works well beyond LangChain

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    without LangChain, custom agents sending traces after each routine

43. Mixed

    FeeVirtual, Reddit user

    langsmith + testmu running together. prod regressions still slipping. what’s the missing layer?Reddit post, r/AI_Agents

    Traces in LangSmith, adversarial evals and CI rubric checks all missed three production failures: a tool's changed response format, an eval set three months stale, and memory that drifted across long sessions.

    Published Jul 2, 2026self-described: Production agents traced in LangSmith, with an agent-to-agent adversarial eval tool and custom rubrics in CI

    Counts toward

    • Overall signal
    • Production Monitoring: Mixed or negative
    • Agents in production need tracing

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

44. Mixed

    Previous_Net_1154, Reddit user

    Comment on: 6 things I learned deploying AI agents for B2B clients (the hard way)Reddit comment, r/LangChain

    LangSmith showed every LLM and tool call, but a run that retrieved the wrong client context looked identical to a good one at the trace level.

    Published Jun 30, 2026self-described: B2B client agents in production

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Mixed or negative
    • Agents in production need tracing

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

45. Positive

    MarkAZhang, developer of a GAIA benchmark research agent

    Migrate observability from Langfuse to LangSmithGitHub pull request

    Moved a LangGraph research agent's tracing, datasets and five evaluators from Langfuse to LangSmith, with a one-time script that ports existing Langfuse datasets.

    Published Jun 30, 2026self-described: LangGraph research agent scored on the GAIA benchmark with five evaluators

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Positive
    • Strongest with LangGraph
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith evaluate() with a one-time dataset migration script

46. Positive

    Nico Venegas and Claudio Urbina Lara, LATAM Airlines AI team

    How LATAM Airlines Built Intelligent Agents in Aviation | Interrupt 2026Conference talk, Interrupt 26 (LangChain YouTube channel)

    LATAM traced its Concierge agent in LangSmith from day one and used the traces to find that 95% of messages classed as out of scope were real passenger needs.

    "Python problem. That's normal. That's okay." But, um but when you But, still 13% was a lot. So, uh we used LangSmith again, and we dig into those conversation to understand what the user were asking. And we realized that 95% of…"

    Published Jun 30, 2026self-described: LATAM Concierge, a LangGraph travel agent with 4,000 daily users

    Not counted

    Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

    Published on

    Podcasts, Talks & Videos

    Who

    Product or platform team

    Evidence

    Measured

    Applies to

    Current

47. Positive

    Philipp Comans, Software engineer, Chime

    Make Legal Write Your Evals: Building Jade, Chime's Financial Copilot | Interrupt 26Conference talk, Interrupt 26 (LangChain YouTube channel)

    Chime scores compliance-risk evals for its Jade copilot in LangSmith, and legal partners mark each response pass or fail next to the LLM judge's verdict.

    "So here's what this might look like in LangSmith, right? We get a result for each question and agent response pair, right? Fail or pass. And we can calculate a pass rate in percent for each risk dataset. In this case, we have a pass…"

    Published Jun 22, 2026self-described: Jade, Chime's AI financial copilot

    Not counted

    Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

    Published on

    Podcasts, Talks & Videos

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

48. Mixed

    poushkar, Reddit user, self-described

    Are there any tools out there automatically suggesting possible agent improvements?Reddit post, r/LLMDevs

    Finds LangSmith Insights still rough for spotting agent failure patterns because it needs setup and a model choice first.

    Published Jun 20, 2026

    Counts toward

    • Overall signal
    • Production Monitoring: Mixed or negative

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

49. Positive

    Folarin Akinloye, AI engineer in London

    Evaluating Agents with LangSmith: A Complete GuidePersonal blog post, folarin.dev

    Akinloye traces every agent from day one and saves failed runs to a dataset. Before each prompt, model, or tool change, Akinloye runs an experiment against the last one and ships only when nothing important regresses.

    Published Jun 19, 2026self-described: Agents built with LangChain and LangGraph, traced from day one and tested against datasets before each change

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Datasets & Evaluations: Positive
    • Quick Setup: Positive
    • Evals gate prompt and model changes

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith tracing, datasets, evaluators, and experiments

50. Positive

    Nick Ung, Leads data science and ML for safety and customer care at Lyft

    How Lyft Builds Evals That Actually Matter in Production | Interrupt 26Conference talk, YouTube (LangChain channel, Interrupt 26)

    Lyft's AI Assist team uses LangSmith to capture simulated and production trajectories, host LLM-judge evaluators, and auto-route traces failing the judge to annotation queues for its operations team.

    "…in production, we're using LangSmith to help us capture these nice trajectories. This becomes really nice training data for our models later on, as well as a good interface for our operations team who are manually annotating our data. Evaluators, this is how we…"

    Published Jun 15, 2026self-described: Lyft AI Assist customer-care agents (LangChain, LangGraph, LangSmith), about 270,000 AI interactions per month

    Not counted

    Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

    Published on

    Podcasts, Talks & Videos

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    with LangGraph; offline simulator plus online evals and annotation queues

51. Positive

    fxwin, Hacker News commenter, self-described

    Comment on: Apache Burr: Build reliable AI agents and applicationsHacker News comment

    Says LangChain's one environment variable for tracing gives a UI to browse every trace with almost no effort, which a hand-rolled setup cannot match.

    Published Jun 10, 2026

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Quick Setup: Positive
    • Agents in production need tracing

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

52. Positive

    Carlos Pereira, Cisco Customer Experience team (as described in the video)

    Observing And Testing CX Agents | Interrupt 26Conference talk, YouTube (LangChain channel, Interrupt 26)

    Cisco CX captures production traces in LangSmith, has a triage agent pull thumbs-down and error traces through LangSmith's MCP into Jira, and treats LangSmith as business-critical infrastructure.

    "So all thumbs down, all the errors, all the low confidence classifications, everything, we have production traces that we capture via LangSmith. And then we built the triage agent. What the triage agent is, is basically a deep research agent that runs in a browser,…"

    Published Jun 10, 2026self-described: Cisco CX support agents, about 16 million interactions a year; a triage agent pulls LangSmith traces via MCP and opens Jira issues

    Not counted

    Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

    Published on

    Podcasts, Talks & Videos

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith MCP server, LangChain deep agents, Splunk monitoring of LangSmith

53. Positive

    Daniel Cordeiro, developer

    Building an AI Billing Assistant: Integrating LangChain ReAct Agents with Spring Boot Microservicesdev.to article

    LangSmith traces each ReAct loop's tool calls, reasoning and timings, which they call invaluable for spotting and debugging agent misbehavior.

    Published Jun 8, 2026self-described: Telecom billing support agent: LangChain ReAct agent in FastAPI calling Spring Boot services

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    with LangChain

54. Positive

    Derrick Kondo, Engineer, GenAI Enablement team, Etsy

    The Etsy Gifting Assistant: From Prototype to Production | Interrupt 26Conference talk, Interrupt 26 (LangChain YouTube channel)

    Etsy built an internal CLI on the LangSmith APIs for simulated datasets, data splits, and judge alignment, deploys through LangSmith Deployment, and reached beta in six weeks.

    "No Terraform required, no handwritten pipelines. All one has to specify is a YAML file for specifying the resources for the deployment. The build system uses LangSmith deployment APIs to deploy in a scalable and reliable way. Taking a step back, it took us six…"

    Published Jun 8, 2026self-described: Etsy gifting assistant, about three senior engineers and one designer

    Not counted

    Published on LangChain's own YouTube channel from its Interrupt conference. Kept in the record; it takes part in no figure.

    Published on

    Podcasts, Talks & Videos

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith Deployment

55. Positive

    Consistent_Wash_276, Reddit user

    LangSmith for local tracingReddit post, r/LangChain

    New to LangGraph and LangSmith, came away pretty impressed with LangSmith tracing after a two-hour first session.

    Published Jun 7, 2026self-described: Solo, first LangGraph project

    Counts toward

    • Overall signal
    • Quick Setup: Positive
    • Strongest with LangGraph

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

56. Mixed

    Total_Listen_4289, Reddit user

    Spent 3 months evaluating Galileo, Arize, Langfuse and LangSmith for production LLM monitoring. Notes for anyone doing the same.Reddit post, r/LLMDevs

    After a three-month evaluation the small team found LangSmith smoothest inside LangChain, but chose Langfuse over its LangChain ties and Enterprise-only self-hosting.

    Published May 30, 2026self-described: Small team shipping agents

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Datasets & Evaluations: Positive
    • Self-Hosting & Data Control: Raised as a concern
    • LangChain Coupling: Raised as a concern
    • Open-source tools are the alternative
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

57. Mixed

    itsmeAki, Reddit user

    Testing LangChain agents beyond LangSmith eval scripts: what's actually catching prod issues?Reddit post, r/LangChain

    Says LangSmith evals catch prompt regressions but stayed green for two days while the agent skipped a required question after a prompt change.

    Published May 30, 2026self-described: Sales-qualification chat agent in production

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Mixed or negative
    • Evals gate prompt and model changes

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

58. Positive

    Markus Eisele, Java developer and newsletter author

    Trace a Quarkus LangChain4j App in LangSmithNewsletter, The Main Thread (Substack)

    Java teams can get LangSmith traces over plain OTLP with no custom instrumentation, though they would add sampling or redaction before shipping full prompts from production.

    Published May 30, 2026self-described: Example Quarkus LangChain4j support assistant on Ollama exporting OpenTelemetry traces to LangSmith

    Counts toward

    • Overall signal
    • Quick Setup: Positive
    • Self-Hosting & Data Control: Discussed without concern
    • LangChain Coupling: Discussed without concern
    • Useful without LangChain
    • Does LangSmith earn its place outside the LangChain ecosystem?: Works well beyond LangChain

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenTelemetry (OTLP) from Java, no LangChain Python

59. Mixed

    Aryan Randeriya, founder and CEO of Experience Inc., which builds GAIA

    feat: self-hosted Langfuse alongside LangSmithGitHub pull request

    GAIA added a self-hosted Langfuse beside LangSmith because LangSmith is paid and its development workspace kept hitting the monthly trace limit; LangSmith kept receiving the same traces in parallel during the move.

    Published May 28, 2026self-described: GAIA, an open-source personal AI assistant whose agents, sub-agents and MCP tools are traced in LangSmith

    Counts toward

    • Overall signal
    • Seat & Trace Pricing: Raised as a concern
    • Self-Hosting & Data Control: Raised as a concern
    • Open-source tools are the alternative
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith's monthly unique-trace limit reached on the development workspace

60. Mixed

    rafio77, Reddit user

    I ran langfuse, langsmith and helicone in prod for a month and only one of them stuckReddit post, r/LLMDevs

    Ran three tools in production for a month: LangSmith was the most complete once wired in, but the team is mostly outside LangChain and kept Langfuse.

    Published May 20, 2026self-described: Team running agents in production, mostly outside LangChain

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Positive
    • LangChain Coupling: Raised as a concern
    • Open-source tools are the alternative
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

61. Negative

    Bladerunner_7_, Reddit user, self-described

    Are there any genuinely good open-source alternatives to LangSmith right now?Reddit post, r/LangChain

    Looks for open-source LangSmith alternatives because the more useful monitoring features get restrictive once you reach the paywall.

    Published May 15, 2026

    Counts toward

    • Overall signal
    • Seat & Trace Pricing: Raised as a concern
    • Open-source tools are the alternative

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

62. Negative

    Pager_dot, Reddit user, self-described

    How to integrate Langchain and Trace with OpenTelemetry without using LangSmithReddit post, r/LangChain

    Feels they fight LangChain's abstractions and walled garden, and wants to trace with OpenTelemetry and skip LangSmith.

    Published May 11, 2026

    Counts toward

    • Overall signal
    • LangChain Coupling: Raised as a concern
    • Closed Source & Vendor Dependence: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

63. Negative

    shivnathtathe, Hacker News poster; author of opensmith

    Show HN: I built an open-source, local-first alternative to LangSmithHacker News post (Show HN)

    Built an open-source, local-first tracer (SQLite, localhost dashboard, no accounts) because LangSmith requires a cloud account just to view their own traces.

    Published May 8, 2026self-described: Solo, Python LLM apps

    Not counted

    Its author built opensmith, an alternative to LangSmith, and the post launches it. Kept in the record; it takes part in no figure.

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

64. Positive

    Stefano Casafranca, Independent developer (maker of the open-source Build_Script)

    How I Built an Autonomous PR Agent with SerpApi, LangGraph, and LangSmithdev.to article

    Traces every graph run in LangSmith with tags and metadata, letting them retrace why the agent ranked a thread or drafted an odd comment.

    Published Apr 30, 2026self-described: Solo; autonomous Reddit-marketing agent on LangGraph and GitHub Actions, two weeks in production

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Strongest with LangGraph

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    with LangGraph

65. Mixed

    ErzErzHerzog, Developer (issue author)

    Extreme lagging on Langsmith PlaygroundGitHub issue

    Reports that opening a 40k-token LLM call in the Playground takes over a minute and that each new message box takes about 60 seconds.

    Published Apr 15, 2026self-described: Debriefing live agent runs in the Playground

    Counts toward

    • Overall signal
    • Playground & Prompt Versioning: Mixed or negative
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

66. Positive

    Balachander Keelapudi, Balachander's Substack

    Tracing Claude Code with LangSmith: Full Observability for Your AI Coding AgentNewsletter, Balachander's Substack

    Wired Claude Code into LangSmith with the official plugin in minutes, gaining traces of LLM calls, tool executions and subagent runs that are otherwise invisible.

    Published Apr 4, 2026self-described: Tracing Claude Code sessions, tool calls and subagents with the official LangSmith plugin

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Quick Setup: Positive
    • Agents in production need tracing
    • Useful without LangChain

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith Claude Code plugin, no LangChain

67. Positive

    Aishwarya Srinivasan, Aishwarya Srinivasan (Substack)

    Setting up Evaluations for LLMsNewsletter, Aishwarya Srinivasan (Substack)

    Uses LangSmith as the orchestration layer for datasets, traces and evaluators, describing it as a full-lifecycle evaluation platform for automated regression pipelines.

    Published Mar 31, 2026self-described: Hands-on eval pipeline: reference datasets, heuristic checks and LLM-as-judge (Gemini grading Claude) orchestrated in LangSmith

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Positive
    • Evals gate prompt and model changes

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

68. Mixed

    jdudekBC, Developer (issue author)

    Feature Request: Custom Input Rendering for Annotation QueuesGitHub issue

    Builds golden eval datasets with LangSmith annotation queues, but HTML inputs are hard to review since custom rendering replaces the editable output pane annotators need.

    Published Mar 18, 2026self-described: Team building golden evaluation datasets with annotators

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Mixed or negative
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Annotation queues, datasets

69. Positive

    vivek, Independent developer

    I Tried to Build an Alexa with Real Memory: Here's What I Learned After 3 Months of Failuredev.to article

    Lists LangSmith in their stack for tracing and debugging the LangGraph graph and calls it essential.

    Published Mar 5, 2026self-described: Solo personal-assistant project with LangGraph, Groq, Pinecone, Postgres and Redis over three months

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Strongest with LangGraph

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    with LangGraph

70. Mixed

    MickeyShmueli, Hacker News commenter

    Comment on: Launch HN: Cekura (YC F24) - Testing and monitoring for voice and chat AI agentsHacker News comment

    On a verification flow, every turn looked fine in LangSmith while the overall flow was broken, so turn-level traces missed a session-level failure.

    Published Mar 4, 2026self-described: Support and verification agents in production

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Mixed or negative

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

71. Mixed

    Hamed Karbasi (hamedkarbasi-haypp), Developer (issue author)

    Issue: LangSmith UI doesn't show the datasets created by SDKGitHub issue

    Reports that datasets and experiments created with the Python SDK never appear in the Datasets and Experiments pages, though direct links work.

    Published Feb 27, 2026self-described: Datasets and experiments created with the Python SDK

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Mixed or negative
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

72. Mixed

    BasRizk, developer filing a GitHub issue

    [TypeScript] CRITICAL BUG: AI message duplicated in traces when using Anthropic streaming with finalMessage()GitHub issue

    With the Anthropic wrapper and finalMessage(), every streamed assistant reply appeared twice in LangSmith traces, reproduced on every run.

    Published Feb 7, 2026self-described: TypeScript app tracing Anthropic streaming calls

    Counts toward, pending a recheck

    • Overall signal
    • Trace-Level Debugging: Mixed or negative
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    LangSmith TypeScript SDK wrapAnthropic with Claude Sonnet 4.5

73. Mixed

    Aviad Rozenhek, Developer and author of langsmith-cli

    LangSmith CLI: Not Just Efficient, Actually Better Than MCPdev.to article

    Values LangSmith's traces, datasets and evals but found daily use via the web UI and MCP server clunky, measuring the MCP tools at 16,100 tokens of always-loaded context.

    Published Jan 15, 2026self-described: Debugging and monitoring production LLM runs from the terminal and from Claude Code

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • UI & SDK Reliability: Raised as a concern

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Current

    Observed on

    LangSmith MCP server vs CLI used from Claude Code

74. Mixed

    Daniel (dast-draftwise), Developer, Draftwise

    Issue: Messy Error Logging in LangSmithGitHub issue

    Reports that each failed retry under wrap_openai becomes its own error trace, so error dashboards count failures for requests that succeeded.

    Published Jan 6, 2026self-described: Draftwise, OpenAI calls wrapped with wrap_openai and retries

    Counts toward

    • Overall signal
    • Production Monitoring: Mixed or negative
    • Useful without LangChain

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    wrap_openai

75. Positive

    Andrew Larsen, engineer

    From Tests to Experiments: Evaluating LangGraph Agents with LangSmith and PytestMedium article

    Added LangSmith's pytest integration so static eval datasets and untracked results become registered datasets and experiments tracked over time, without changing how tests are written.

    Published Dec 28, 2025self-described: Production support agent built with LangGraph; pytest evaluators extended with LangSmith experiments

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Positive
    • Evals gate prompt and model changes
    • Strongest with LangGraph

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    LangSmith pytest integration, with LangGraph

76. Mixed

    stingraycharles, Hacker News commenter, self-described

    Comment on: Critical vulnerability in LangChain - CVE-2025-68664Hacker News comment

    Finds LangSmith's prompt engineering workflow okay-ish but laborious, quickly expensive, and only workable for one-turn prompts; multi-turn never works.

    Published Dec 27, 2025

    Counts toward

    • Overall signal
    • Playground & Prompt Versioning: Mixed or negative
    • Seat & Trace Pricing: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    Prompt Hub / prompt templates

77. Positive

    Juha Pellotsalo, Senior engineer (self-described)

    Reflections on building my first LangGraph projectdev.to article

    Tracing turned on with just an API key and was the easiest usable tracing UI they found; they call tracing a must-have and would start evals in LangSmith.

    Published Dec 22, 2025self-described: Solo learning project: multi-agent newsroom workflow in LangGraph

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Quick Setup: Positive
    • Agents in production need tracing

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    with LangGraph

78. Mixed

    adhishthite, Developer (issue author)

    [langsmith]: LangSmith traces from Claude Code do not expose cost, token usage, or detailed model metadataGitHub issue

    Tracing Claude Code into LangSmith works, but traces show only the model name with no token usage or cost, so top-cost traces and model spend cannot be audited.

    Published Dec 21, 2025self-described: Tracing Claude Code sessions into LangSmith

    Counts toward

    • Overall signal
    • Quick Setup: Positive
    • Production Monitoring: Mixed or negative
    • Useful without LangChain

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    Claude Code tracing integration, without LangChain

79. Negative

    threecheese, Hacker News commenter

    Comment on: Show HN: Sim - Apache-2.0 n8n alternativeHacker News comment

    Likes LangGraph's abstractions but feels LangSmith is creeping further into LangChain and LangGraph, with AWS-like lock-in vibes that make them hesitant to invest.

    Published Dec 12, 2025self-described: Builds agents with LangGraph workflows

    Counts toward

    • Overall signal
    • LangChain Coupling: Raised as a concern
    • Closed Source & Vendor Dependence: Raised as a concern
    • Does LangSmith earn its place outside the LangChain ecosystem?: Most useful inside LangChain

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    with LangGraph

80. Positive

    Mesterniz, Hacker News commenter

    Comment on: Show HN: How to fix AI Agents at the component levelHacker News comment

    Built a four-component agent on LangGraph and LangSmith to track per-component latency, intermediate state, and which component caused a bad answer.

    Published Dec 12, 2025self-described: Multi-component agent built for a write-up

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Agents in production need tracing
    • Strongest with LangGraph

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

81. Positive

    Lysia Li and five co-authors, Harvey

    How Agentic Search Unlocks Legal Research Intelligence at HarveyCompany engineering blog, harvey.ai

    Harvey sends the OpenTelemetry traces of its OpenAI Agents SDK search agent into LangSmith and runs offline evaluations there on queries its legal experts write.

    "Centralized evaluation infrastructure that we built using the OpenAI Agent SDK's OpenTelemetry traces enables us to run offline evaluations in LangSmith. We track metrics like hallucination, tool recall, retrieval recall, formatting, and answer quality."

    Published Dec 10, 2025self-described: Harvey Agentic Search for legal research, built on the OpenAI Agents SDK

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Positive
    • Useful without LangChain
    • Does LangSmith earn its place outside the LangChain ecosystem?: Works well beyond LangChain

    Published on

    Engineering Blogs & Reviews

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

82. Mixed

    dast-draftwise, developer filing a GitHub issue

    Issue: Bulk Export takes very long time with small number of tracesGitHub issue

    A bulk export of about 600 traces stayed in Running for over an hour, which the author says makes exporting their 50,000+ traces unworkable.

    Published Oct 29, 2025self-described: Exporting LangSmith traces to S3 as Parquet for a data pipeline

    Counts toward, pending a recheck

    • Overall signal
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    LangSmith bulk export API

83. Mixed

    zhangjunkang (Z-Jr10-K), Developer, SprintRay

    Issue: LangSmith tracing does not capture subagent tool calls in Claude Agent SDKGitHub issue

    Reports that LangSmith's Claude Agent SDK integration records only the main agent and omits tool calls made by subagents.

    Published Oct 27, 2025self-described: Claude Agent SDK app with subagents and MCP servers

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Mixed or negative
    • Useful without LangChain

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    Claude Agent SDK

84. Positive

    shcallaway, Hacker News commenter

    Comment on: LangChain raises US$125M to build the platform for agent engineeringHacker News comment

    A year-long LangGraph Platform (now Deployments) user found it rough early, with poor monorepo support, but says benefits outweighed costs and the product has since matured.

    Published Oct 21, 2025self-described: Agents on LangGraph TypeScript SDK hosted on LangGraph Platform / Deployments for about a year

    Counts toward

    • Overall signal
    • UI & SDK Reliability: Discussed without concern
    • Strongest with LangGraph
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    LangGraph Platform (now Deployments), LangGraph TypeScript SDK

85. Positive

    Maryam Warraich, Product Hunt

    Review of LangSmith by Maryam WarraichProduct Hunt review

    Says LangSmith's debugging and iteration tools made it easier to take their LLM projects to production and saved a lot of trial and error.

    Published Oct 5, 2025self-described: Own LLM projects taken from idea to production

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Playground & Prompt Versioning: Positive

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

86. Positive

    Hamel Husain, AI evals consultant and course instructor, with panelists Shreya Shankar and Bryan Bischof

    Selecting The Right AI Evals ToolIndependent evaluation, hamel.dev blog

    In Hamel Husain's side-by-side panel of three evals tools, LangSmith's trace-to-playground workflow and annotation queue drew praise, while its UI felt cluttered and side-by-side prompt comparison was hard.

    Published Oct 1, 2025self-described: Panel of data scientists watching LangSmith, Braintrust, and Arize Phoenix complete the same evals homework

    Not counted

    Its assessment comes from a course session in which LangChain's CEO walked through LangSmith. Kept in the record; it takes part in no figure.

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    LangSmith as of mid-2025

87. Mixed

    ryanlohyr, developer filing a GitHub issue

    Issue: Cost showing up as 0 despite having tokensGitHub issue

    LangSmith showed a cost of 0 for every run that called OpenAI models, though the models are on its pricing list and the traces carry token counts.

    Published Sep 16, 2025self-described: Runs with several OpenAI calls traced to LangSmith

    Counts toward, pending a recheck

    • Overall signal
    • Production Monitoring: Mixed or negative

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    OpenAI models listed on LangSmith's model pricing page

88. Mixed

    henriquejsb, Developer (issue author)

    Issue: Memory leaksGitHub issue

    In a long-running LangGraph app, memory grows significantly only when LangSmith tracing is on; tracemalloc diffs point at the SDK's serialization code.

    Published Sep 10, 2025self-described: Long-running Python application on LangGraph and LangChain

    Counts toward

    • Overall signal
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Still applicable

    Observed on

    with LangGraph

89. Negative

    Ahmad Ilaiwi (Ilaiwi), Forum user

    Langsmith is downLangChain Forum post

    Could not load any LangSmith projects during an outage and says LangSmith had been very slow for them recently.

    Published Sep 9, 2025self-described: Team using LangSmith projects

    Counts toward

    • Overall signal
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

90. Positive

    Kanishk Gupta, Builder of Promptius

    Review of LangSmith by Kanishk GuptaProduct Hunt review

    Credits the LangChain, LangGraph and LangSmith combination for building Promptius, with LangSmith giving strong observability and cost tracking.

    Published Aug 16, 2025self-described: Promptius, built over five months on LangChain, LangGraph and LangSmith

    Counts toward

    • Overall signal
    • Production Monitoring: Positive
    • Strongest with LangGraph

    Published on

    Engineering Blogs & Reviews

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    with LangChain and LangGraph

91. Negative

    NeutralCrane, Hacker News commenter, self-described

    Comment on: Open SWE by LangChainHacker News comment

    Prefers Langfuse: open source, fewer bugs, and cheaper, since LangSmith bills per seat and per trace, charges more for retention past 14 days, and caps it at 400 days even self-hosted.

    Published Aug 11, 2025

    Counts toward, pending a recheck

    • Overall signal
    • Seat & Trace Pricing: Raised as a concern
    • Self-Hosting & Data Control: Discussed without concern
    • UI & SDK Reliability: Raised as a concern
    • Closed Source & Vendor Dependence: Raised as a concern
    • Open-source tools are the alternative
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

92. Negative

    NeutralCrane, Hacker News commenter, self-described

    Comment on: Open SWE by LangChainHacker News comment

    Calls LangSmith usable but not great, prefers Langfuse, and says LangSmith more or less requiring LangChain is the biggest knock against it.

    Published Aug 8, 2025

    Counts toward

    • Overall signal
    • UI & SDK Reliability: Raised as a concern
    • LangChain Coupling: Raised as a concern
    • Open-source tools are the alternative
    • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool
    • Does LangSmith earn its place outside the LangChain ecosystem?: Most useful inside LangChain

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

93. Negative

    Bob (hubdev), Forum user

    Not able to delete not used Extended Data Retention tracesLangChain Forum post

    Prompt experiments with a code evaluator auto-convert traces to Extended Data Retention, hitting usage limits that block new experiments; deleting traces does not lower the count.

    Published Jul 29, 2025self-described: Comparing prompts in LangSmith experiments against a dataset

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Mixed or negative
    • Seat & Trace Pricing: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    Experiments with code evaluators; Extended Data Retention limits

94. Mixed

    Shane S (ssmithra), Forum user

    Self-hosted: Langsmith is counting base64 encoded files/images as string tokens, causing massively incorrect token costsLangChain Forum post

    On self-hosted LangSmith, base64 images are counted as string tokens, producing wildly wrong costs in a UI the team relies on for cost management.

    Published Jul 29, 2025self-described: Self-hosted LangSmith, moving into multi-modal inputs

    Counts toward

    • Overall signal
    • Production Monitoring: Mixed or negative

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    self-hosted, version 0.10.93

95. Positive

    Maitar Asher, Head of Engineering, Tavily

    Evaluating AI Search: A Practical Framework for Augmented AI SystemsConference talk with transcript, AI Engineer World's Fair 2025

    Asher reports that Tavily tracks its offline evaluation runs in LangSmith and calls it a great observability tool for comparing performance over time.

    Published Jul 29, 2025Tavily's open-source LangGraph agent that builds fresh eval datasets for web search from live documents

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Positive

    Published on

    Podcasts, Talks & Videos

    Who

    Technical leadership

    Evidence

    Reported

    Applies to

    Current

96. Negative

    staletofu, Hacker News commenter, self-described

    Comment on: Ask HN: GCP Outage?Hacker News comment

    Reports being unable to log in to LangSmith with Google during a Google Cloud outage.

    Published Jul 18, 2025

    Counts toward

    • Overall signal
    • UI & SDK Reliability: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

97. Positive

    Dan Bulteel, Maker of Meet-Ting

    Review of LangSmith by Dan BulteelProduct Hunt review

    Building Ting without LangSmith felt like guesswork; after adopting it the team traces problems through the full prompt path and moves faster.

    Published Jul 16, 2025self-described: Ting scheduling assistant, early-stage product team

    Counts toward

    • Overall signal
    • Trace-Level Debugging: Positive
    • Agents in production need tracing

    Published on

    Engineering Blogs & Reviews

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

98. Mixed

    Yurii Venher (yurii-ve), Developer, Wix

    [Bug] Eval experiment table is empty if OTEL_ENABLED environment variable is setGitHub issue

    Reports that running evaluate on Vercel AI SDK calls with OpenTelemetry enabled creates an empty experiment, though results exist in code.

    Published Jul 11, 2025self-described: Evaluations of Vercel AI SDK calls through OpenTelemetry

    Counts toward

    • Overall signal
    • Datasets & Evaluations: Mixed or negative
    • Useful without LangChain

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    OpenTelemetry

99. Mixed

    yurii-ve, developer filing a GitHub issue

    Tracing with AI SDK (by Vercel) creates trace with wrong (doubled) total tokens countGitHub issue

    Following LangSmith's Vercel AI SDK guide, the parent trace reported 22 tokens for a call that used 11.

    Published Jun 23, 2025self-described: Vercel AI SDK app calling Gemini on Vertex, traced through LangSmith's exporter

    Counts toward, pending a recheck

    • Overall signal
    • Production Monitoring: Mixed or negative

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Measured

    Applies to

    Needs a recheck

    Observed on

    LangSmith AISDKExporter with gemini-2.5-flash

100. Positive

     Kyle Rush, Maker of Astrid: Personal Shopping Agent

     Review of LangSmith by Kyle RushProduct Hunt review

     The team became LangSmith power users for traces, datasets, annotation queues and evals, and picked it over Langfuse because of their prior experience and speed.

     Published Jun 22, 2025self-described: Astrid personal shopping agent; traces, datasets, annotation queues and evals

     Counts toward

     • Overall signal
     • Trace-Level Debugging: Positive
     • Datasets & Evaluations: Positive
     • When practitioners weigh LangSmith against another platform, which do they keep?: Chose or kept LangSmith

     Published on

     Engineering Blogs & Reviews

     Who

     Small product team

     Evidence

     Reported

     Applies to

     Still applicable

101. Mixed

     Dan Mason, Principal, AI, Stride

     Case Study + Deep Dive: Telemedicine Support Agents with LangGraph/MCPConference workshop with transcript, AI Engineer World's Fair 2025

     Mason reviews every agent conversation and its tool calls in LangSmith and keeps eval datasets there. He runs the evals in his own promptfoo harness to avoid lock-in, and says LangSmith's displayed costs ignored prompt caching.

     Published Jun 22, 2025LangGraph agents Stride built to guide telemedicine patients through at-home treatment by text message

     Counts toward

     • Overall signal
     • Trace-Level Debugging: Positive
     • UI & SDK Reliability: Raised as a concern
     • Closed Source & Vendor Dependence: Discussed without concern
     • Strongest with LangGraph

     Published on

     Podcasts, Talks & Videos

     Who

     Technical leadership

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     Datasets kept in LangSmith; evals run through promptfoo

102. Mixed

     fazlerocks, Hacker News poster

     Ask HN: What tools are you using for AI evals? Everything feels half-bakedHacker News post (Ask HN)

     Evaluated LangSmith for production evals: strong tracing, but evals feel secondary, US$0.50 per 1k traces after the free tier adds up, and the UI slows with large datasets.

     Published Jun 5, 2025self-described: Team running LLMs in production for content generation, customer support and code review assistance

     Counts toward, pending a recheck

     • Overall signal
     • Trace-Level Debugging: Positive
     • Datasets & Evaluations: Mixed or negative
     • Seat & Trace Pricing: Raised as a concern
     • UI & SDK Reliability: Raised as a concern
     • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool

     Published on

     Developer Communities

     Who

     Small product team

     Evidence

     Reported

     Applies to

     Needs a recheck

103. Mixed

     erwan-png, developer filing a GitHub issue

     Failed to send compressed multipart ingest: Connection error caused failure to POST https://api.smith.langchain.com/runs/multipart in LangSmith APIGitHub issue

     After upgrading the LangSmith SDK, every LLM call's trace failed to upload and stayed pending, while a project on an older SDK version kept working until it was upgraded too.

     Published Jun 4, 2025self-described: LangGraph app on Ollama and Groq models traced to LangSmith

     Counts toward, pending a recheck

     • Overall signal
     • UI & SDK Reliability: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Needs a recheck

     Observed on

     langsmith 0.3.42 compared with 0.2.11

104. Negative

     senko, Hacker News commenter

     Comment on: Use the Gemini API with OpenAI Fallback in TypeScriptHacker News comment

     Says LangChain's docs and tutorials now work as content marketing for LangGraph Platform and LangSmith, so they built their own thin wrapper instead.

     Published Apr 7, 2025self-described: Solo, own thin LLM wrapper

     Counts toward

     • Overall signal
     • LangChain Coupling: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

105. Mixed

     André Menezes (menezesandre), AI engineer

     Issue: Client silently stops posting traces to the API (auto_batch_tracing=True, DISABLE_RUN_COMPRESSION=False)GitHub issue

     Reports that the Python client sometimes stops sending traces with no error logged while the app keeps running, and only a restart fixes it.

     Published Apr 3, 2025self-described: Python application traced with auto-batch tracing and compression

     Counts toward

     • Overall signal
     • Production Monitoring: Mixed or negative
     • UI & SDK Reliability: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

106. Mixed

     Karolis Mariunas (kmariunas), Developer (issue author)

     Duplicate traces through OpenAI agents SDKGitHub issue

     Reports one OpenAI Agents SDK run appearing as three LangSmith traces with different IDs, which made the team's dashboards untrustworthy.

     Published Mar 31, 2025self-described: OpenAI Agents SDK app traced to LangSmith

     Counts toward

     • Overall signal
     • Production Monitoring: Mixed or negative
     • UI & SDK Reliability: Raised as a concern
     • Useful without LangChain

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     OpenAI Agents SDK

107. Positive

     John Capobianco, YouTube creator building AI agents

     LangSmith: Observability for AI AgentsVideo, YouTube

     Walks through adding @traceable and the OpenAI wrapper to their LangGraph agent, praising waterfalls with latency, tokens and cost, and noting it cost them nothing as an individual.

     "…wrapper that wraps around our open AI client so we're just simply going to wrap our openai client and we're going to make traceables at traceable so what does this look like in the code I'm not going to show you myv file but my…"

     Published Mar 9, 2025self-described: Own LangGraph agent project instrumented with @traceable and the OpenAI client wrapper

     Counts toward

     • Overall signal
     • Trace-Level Debugging: Positive
     • Production Monitoring: Positive
     • Seat & Trace Pricing: Discussed without concern
     • Useful without LangChain

     Published on

     Podcasts, Talks & Videos

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     @traceable and wrap_openai with LangGraph; free tier

108. Negative

     Amit, NonBioS author

     The NonBioS LLM Observability PickCompany blog post, NonBioS.ai

     Ruled LangSmith out because it seemed tightly coupled to LangChain, which they had removed, and chose open-source, on-prem Langfuse for data governance.

     "Langsmith appeared popular, but we had encountered challenges with Langchain from the same company, finding it overly complex for previous NonBioS tooling. We rewrote our systems to remove dependencies on Langchain and chose not to proceed with Langsmith as it seemed strongly coupled with Langchain."

     Published Mar 5, 2025self-described: NonBioS AI Engineer product; runs consume over a million tokens and about a hundred inference calls in five minutes, with sensitive customer data

     Counts toward

     • Overall signal
     • Self-Hosting & Data Control: Raised as a concern
     • LangChain Coupling: Raised as a concern
     • Closed Source & Vendor Dependence: Discussed without concern
     • Open-source tools are the alternative
     • When practitioners weigh LangSmith against another platform, which do they keep?: Chose another tool
     • Does LangSmith earn its place outside the LangChain ecosystem?: Most useful inside LangChain

     Published on

     Engineering Blogs & Reviews

     Who

     Product or platform team

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     Evaluated and passed over

109. Mixed

     h-portia, Reddit user

     Best evaluation platform for LLMs/agentic frameworks for error analysis?Reddit post, r/LLMDevs

     Finds LangSmith handy for high-level eval results and drill-downs, but hard for spotting failure patterns across groups, so builds own charts from its Python library.

     Published Feb 6, 2025self-described: Team running LLM output evals in LangSmith, Python

     Counts toward

     • Overall signal
     • Datasets & Evaluations: Mixed or negative
     • UI & SDK Reliability: Raised as a concern

     Published on

     Developer Communities

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

110. Mixed

     Hannah Craighead, startup team member

     Review of LangSmith by Hannah CraigheadProduct Hunt review

     LangSmith evals in GitHub CI/CD catch regressions, but the UI gets tricky with large datasets or long experiment histories and filters are not kept in the URL.

     Published Jan 27, 2025self-described: Startup; ongoing LLM evaluations with LangSmith evals in GitHub CI/CD

     Counts toward

     • Overall signal
     • Datasets & Evaluations: Positive
     • UI & SDK Reliability: Raised as a concern
     • Evals gate prompt and model changes

     Published on

     Engineering Blogs & Reviews

     Who

     Individual builder

     Evidence

     Reported

     Applies to

     Still applicable

     Observed on

     Python SDK, evals in CI/CD