GILD verdict
Choose Braintrust when many reviewers work from an eval agent's findings, and LangSmith when your agents run on LangGraph.
- The workflow
- an AI team choosing where release evals and production traces live once agents ship to users
- The decisive evidence
- Reported4 of 6 named teams adopted one; What teams report below sets out each account.
- MeasuredOn 100 hand-labelled production traces, Parlance Labs found on Jul 11, 2026 that Braintrust Loop had the highest recall of three platforms, while LangSmith Engine surfaced too few issues to score.1
- DocumentedBraintrust Pro costs US$249 a month with unlimited users, and LangSmith Plus costs US$39 per seat plus 0.005 LSU per base trace past 10,000.615
- GILD analysisOne test on one assistant decides the recall question, so it is unresolved for other workloads. GILD's calculation from published rates puts 200,000 traces a month for 10 users at US$474 on Braintrust Pro and US$1,840 on LangSmith Plus with 180-day retention.
- The tradeoff
- Test Loop on your own labelled traces before you rely on its recall. Braintrust bills by volume; LangSmith hosts, traces and caps LangGraph agents on one platform. Braintrust has no hard spend cap and does not host agents; LangSmith stops usage at a limit you set and hosts LangGraph agents, but its seats and per-trace fees cost more as headcount and volume grow.1691516
- What would change the call
- The call moves when a second team publishes a head-to-head on its own traces, or when LangSmith Engine's findings are scored on a labelled set like Parlance Labs' 100 traces.
The one head-to-head on record favors Braintrust for finding failures. In Parlance Labs' July 2026 test on 100 hand-labelled traces, Braintrust Loop had the highest recall of three platforms, while LangSmith Engine surfaced too few issues and its chat agent stood in. Fintool and Dropbox gate releases on Braintrust experiments; monday.com and Harvey run their agent evals in LangSmith. At 200,000 traces a month for 10 users, GILD's calculation puts Braintrust Pro at US$474 and LangSmith Plus at US$1,840. Only LangSmith hosts your agents.
What teams report
Evaluated 2
Parlance Labs
Parlance Labs masked the hand labels on 100 production traces from an apartment-leasing assistant and asked Braintrust Loop, Arize Alyx and LangSmith to find the failures (Jul 11, 2026). Loop had the highest recall of the three.
“Braintrust had the highest recall of the three platforms and was easy to use. We could upload our traces, perform the auto-evaluation, and export its findings seamlessly.”
Antaripa Saha and Hamel Husain, Evaluation practitioners and co-authors, Parlance LabsParlance Labs, , write-up Parlance LabsOn LangSmith
In the same test LangSmith Engine surfaced too few issues to score against the 100 labelled traces, so the authors ran the scored experiment with LangSmith's chat agent.
“But it surfaced only a small set of issues, far fewer than a full review turns up, so we could not use it for the 100-trace comparison. We ran the scored experiment with LangSmith’s chat agent instead.”
Antaripa Saha and Hamel Husain, Evaluation practitioners and co-authors, Parlance LabsParlance Labs, , write-up
Adopted 4
FintoolOn Braintrust
Fintool uses Braintrust to evaluate every model, prompt and skill change against about 2,000 test cases and blocks a pull request when the score drops more than 5% (Jan 24, 2026).
“We use Braintrust for experiment tracking. Every model change, every prompt change, every skill change gets evaluated against a test set.”
Nicolas Bustamante, Founder, Fintoolnicolasbustamante.com, , write-up monday.comOn LangSmith
monday.com's Feed agent runs LLM-judge evals in CI before each deploy and traces every invocation in LangSmith (Sep 9, 2026).
“We use an LLM as a judge with specific metrics for each part, and run it on the CI to make sure we have no regressions before deploying to production.”
Kamie Shami-Schnitzer, monday.com engineering, monday.commonday.com Engineering, , write-up DropboxOn Braintrust
Dropbox's Dash team adopted Braintrust for datasets, scorers and experiments, and runs about 150 judged queries on every pull request in under 10 minutes (Oct 2, 2025).
“…we turned to internal datasets by collecting production logs from Dropbox employees dogfooding Dash.”
Ranjitha Gurunath Kulkarni, Dropbox Dash engineer, DropboxDropbox Tech blog, , write-up HarveyOn LangSmith
Harvey runs offline evaluations of its agentic legal search in LangSmith on OpenTelemetry traces from the OpenAI Agents SDK (Dec 10, 2025).
“Centralized evaluation infrastructure that we built using the OpenAI Agent SDK's OpenTelemetry traces enables us to run offline evaluations in LangSmith. We track metrics like hallucination, tool recall, retrieval recall, formatting, and answer quality.”
Lysia Li and co-authors, Harvey engineering, HarveyHarvey blog, , write-up
Choosing between Braintrust and LangSmith
Choose Braintrust if
- An eval agent must find as many failures as it can in a batch of traces: Braintrust Loop had the highest recall in Parlance Labs' 100-trace test (Jul 11, 2026).
- Many people need access and the bill must follow volume: every plan has unlimited users, and GILD's 200,000-trace workload comes to US$474 a month on Pro against US$1,840 on LangSmith Plus for 10 seats.
- Evals gate every release, as at Fintool, which blocks a pull request when its score drops more than 5% (Jan 24, 2026).
- You want traces in an EU data plane on a self-serve plan, or a data plane in your own cloud that Braintrust operates for you.
Choose LangSmith if
- Your agents run on LangGraph or LangChain, as monday.com's service agents do, and you want Studio and LangSmith Deployment on the same platform as your traces.
- Your team has few seats and keeps most traces 14 days: below seven seats, LangSmith Plus costs less than Braintrust Pro's US$249 before usage.
- You want a monthly usage limit that stops spend at a cap.
Side by side
| Dimension | Braintrust | LangSmith |
|---|---|---|
| What teams reportThe only named head-to-head GILD found is Parlance Labs' 100-trace test on one apartment-leasing assistant. Every system in it missed failures that needed business context. | Parlance Labs, Jul 2026: Loop had the highest recall of three platforms on 100 hand-labelled traces, with upload, review and export in one pass. Fintool evaluates every model, prompt and skill change against a Braintrust test set (Jan 2026), and Dropbox Dash runs about 150 judged queries per pull request (Oct 2025).123 | Parlance Labs, Jul 2026: LangSmith Engine surfaced too few issues for the 100-trace comparison, so the authors scored its chat agent. monday.com gates Feed agent deploys on LLM-judge evals in CI and traces each run in LangSmith (Sep 2026); Harvey runs offline evals of its legal search in LangSmith (Dec 2025).145 |
| PlansList prices in US dollars. Braintrust charges a flat platform fee; LangSmith charges per seat. | Starter US$0 and Pro US$249 a month, both with unlimited users; Enterprise custom, invoiced annually6 | Developer US$0 for 1 seat with 5,000 base traces a month; Plus US$39 per seat per month with 10,000 base traces a month; Enterprise custom15 |
| Billing unitA long agent trace costs one LangSmith trace however many steps it has, while on Braintrust it costs its size in bytes plus every score written to it. | Processed data, every byte ingested and measured at ingestion, plus each score recorded; built-in models draw on monthly model credits | A trace: one execution of the application with every run inside it, up to 25,000 runs |
| Usage price | Starter: 1 GB and 10,000 scores included, then US$4 per GB and US$2.50 per 1,000 scores. Pro: 5 GB and 50,000 scores included, then US$3 per GB and US$1.50 per 1,000 scores6 | 0.005 LSU per base trace past the included volume, US$5 per 1,000, plus 0.0025 LSU to upgrade a trace to extended retention; 1 LSU is US$11516 |
| 200,000 traces a month, 10 usersGILD's calculation from published rates, Oct 28, 2026. Inputs: 200,000 traces a month, 10 spans, 2 model calls and 1 code score per trace, 20 KB per trace, 10 users, at least 30 days retained, no judge calls. | US$474 a month on Pro: US$249, plus US$225 for 150,000 scores past the 50,000 included, at US$1.50 per 1,000. The 4 GB of data stays inside the 5 GB included.67 | US$1,840 a month on Plus: US$390 for 10 seats, US$950 for 190,000 base traces past the 10,000 included, and US$500 to extend all 200,000 traces to 180-day retention, since base traces keep only 14 days.1516 |
| Retention | Logs 14 days on Starter and 30 days on Pro, extendable to 180 days at US$0.50 per GB per month. Up to 365 days on Enterprise. Experiments up to 365 days on every plan | Base traces 14 days and extended traces 180 days, chosen per tracing project |
| Spend control | Spend alerts by email or Slack at up to three thresholds; no hard spending limit | Self-configured monthly usage limits that return HTTP 429 once reached |
| Hosting | Braintrust-run SaaS with a US or EU data plane, fixed at sign-up. Enterprise adds BYOC and a self-hosted data plane on AWS, GCP, or Azure. The control plane always runs at Braintrust | LangChain's cloud in GCP US, EU, and APAC and AWS US; on Enterprise, hybrid with a SaaS control plane and self-hosted data plane, or full self-hosting on Kubernetes |
| Agent hosting | Not offered; Braintrust runs only the prompts, scorers, and functions you deploy to it | LangSmith Deployment, with 1 free Small Serverless deployment on Plus and further deployments metered by resources consumed |
| SSO and access | Google sign-in on Starter and Pro; SAML and OIDC SSO and custom permission groups on Enterprise | Google and GitHub sign-in on Developer and Plus; custom SSO, ABAC, and RBAC on Enterprise |
| Training on your data | The MSA states Braintrust will not use customer data to train AI models | The pricing FAQ states LangSmith does not use your data to train models |
| Compliance | SOC 2 Type II; report access, BAA, and audit logs on Enterprise; click-through DPA on Pro | SOC 2 Type 2 certified, HIPAA compliant, and GDPR compliant; DPA through support |
What practitioners report, side by side
Each product's figures come from its own practitioners, for the group you pick.
Overall signal
All practitioners rate Braintrust +38, 21 points above LangSmith at +17. Most of the gap is in praise, with 27 of Braintrust's 51 sources positive against 35 of LangSmith's 96. Braintrust has 7 negative and LangSmith 20.
The score counts the authors from all practitioners that take a side, one vote each with their latest view, and is how far positive votes lead negative ones, per 100 votes, so Braintrust's +38 rests on votes split 25 positive, 16 mixed and 7 negative and LangSmith's +17 on 35, 38 and 19.
A lead of a fifth of the votes reads positive and a lead of half strong positive, with the same lines below zero. The dots show each source, positive, mixed, negative. Run a pilot on one production app to measure which failures it surfaces and how long tracing them takes.
Footprint on the AI Application Platform stack
They differ on two of 9 layers. On Orchestration, Braintrust touches it and LangSmith supports it. On Cloud deployment, Braintrust touches it and LangSmith supports it. Coverage is each product's placement on this stack, as its product page shows it. Core is a job the product owns; Supports and Touches are jobs it helps with. Coverage says nothing about how well a product does them.
| Layer | Braintrust | LangSmith |
|---|---|---|
| Build | ||
| Orchestration | Touches | Supports |
| Operate | ||
| Cloud deployment | Touches | Supports |
The 7 layers where they match
| Layer | Braintrust | LangSmith |
|---|---|---|
| Model | ||
| Model APIs | Touches | Touches |
| Model gateway | Supports | Supports |
| Build | ||
| Tool use | Supports | Supports |
| Data | ||
| Retrieval | Touches | Touches |
| Evaluate | ||
| Evaluation | Core | Core |
| Operate | ||
| Observability | Core | Core |
| Guardrails | Touches | Touches |
What practitioners praise
Among all practitioners, LangSmith alone has three praise themes that reach 5 sources from 3 authors. Datasets & Evaluations and Quick Setup lead, at 13 of 22 and 11 of 11. Both carry Trace-Level Debugging. It is praised in 9 of the 11 Braintrust sources that discuss it, against 28 of 35 for LangSmith. Datasets, Experiments & Scorers and Instrumentation & SDK Coverage reach 5 sources from 3 authors only for Braintrust, at 23 of 25 and 4 of 6. LangSmith needs a pilot on the same production traffic before you choose on either.
| Praised for | Braintrust | LangSmith |
|---|---|---|
| Trace-Level Debugging | 9 of 11 | 28 of 35 |
| Datasets, Experiments & Scorers | 23 of 25 | Not a theme in its sources |
| Instrumentation & SDK Coverage | 4 of 6 | Not a theme in its sources |
| Datasets & Evaluations | Not a theme in its sources | 13 of 22 |
| Quick Setup | Not a theme in its sources | 11 of 11 |
| Production Monitoring | Not a theme in its sources | 4 of 14 |
Each count is the sources that praise a theme, or raise it as a concern, out of those that discuss it. A row pairs the two products when both carry a theme under the same or an equivalent name.
A count ending in "so far" has under 5 sources from 3 authors among all practitioners. "Not a theme in its sources" means GILD found no theme of that name in that product's sources.
What practitioners raise as concerns
Among all practitioners, LangSmith alone has five concern themes that reach 5 sources from 3 authors. UI & SDK Reliability and Seat & Trace Pricing lead, at 23 of 25 and 12 of 14. Braintrust alone has four concern themes that reach 5 sources from 3 authors. SDK & Ingestion Reliability and Data Security & Control lead, at 9 of 9 and 6 of 6.
| Raised as a concern | Braintrust | LangSmith |
|---|---|---|
| SDK & Ingestion Reliability | 9 of 9 | Not a theme in its sources |
| Data Security & Control | 6 of 6 | Not a theme in its sources |
| Build It Yourself | 4 of 11 | Not a theme in its sources |
| Usage-Based Cost | 1 of 5 | Not a theme in its sources |
| UI & SDK Reliability | Not a theme in its sources | 23 of 25 |
| Seat & Trace Pricing | Not a theme in its sources | 12 of 14 |
| LangChain Coupling | Not a theme in its sources | 8 of 9 |
| Self-Hosting & Data Control | Not a theme in its sources | 8 of 12 |
| Closed Source & Vendor Dependence | Not a theme in its sources | 4 of 6 |
20 sources
- Parlance Labs: Auto-evals with Braintrust Loop, Arize Alyx and LangSmith (Jul 11, 2026)
- Nicolas Bustamante (Fintool): Lessons from Building AI Agents for Financial Services (Jan 24, 2026)
- Dropbox Tech blog: A practical blueprint for evaluating conversational AI at scale (Oct 2, 2025)
- monday.com Engineering: Building a Robust Harness for agent in production, Feed agent case-study (Sep 9, 2026)
- Harvey blog: How Agentic Search Unlocks Legal Research Intelligence at Harvey (Dec 10, 2025)
- Braintrust pricing
- Braintrust docs: Plans and limits
- Braintrust docs: Billing FAQ
- Braintrust docs: Monitor usage and spend alerts
- Braintrust docs: Configure data retention
- Braintrust docs: Deployment options (SaaS, BYOC, self-hosted)
- Braintrust docs: Organizations and data plane region
- Braintrust docs: Security
- Braintrust Master Subscription Agreement
- LangSmith plans and pricing
- LangSmith docs: Usage and billing (retention tiers and limits)
- LangSmith docs: Cloud (SaaS) architecture and regions
- LangSmith docs: Regions FAQ
- LangSmith docs: Self-hosted LangSmith
- LangSmith docs: Evaluation concepts