GILD verdict
Choose Arize when a person acts on every flagged failure or also monitors ML models, and LangSmith for debugging single traces.
- The workflow
- an AI team picking the platform that will trace, score and watch its production agents
- The decisive evidence
- Reported3 of 6 named teams adopted one; What teams report below sets out each account.
- MeasuredOn 100 hand-labeled production traces, Parlance Labs found on Jul 11, 2026 that 91.0% of Arize Alyx's flags were valid against 77.5% for LangSmith's chat agent, which caught more failures, 79.5% against 74.4%. LangSmith Engine surfaced too few issues to score.1
- DocumentedArize AX logs predictions and actuals for classification, regression, ranking and object detection models, and LangSmith Deployment hosts agents on Agent Servers in LangChain's cloud, hybrid or self-hosted.1519
- GILD analysisThe AX bill at volume is unresolved. GILD's calculation from published rates puts 200,000 traces a month for 10 users, kept 30 days, at US$1,840 on LangSmith Plus, while AX Pro's US$50 base fee leaves 1,950,000 spans unpriced until Arize quotes a rate.
The one shared test on record splits the pair. In Parlance Labs' July 2026 study of 100 hand-labeled traces, LangSmith's chat agent caught 79.5% of the failures and Arize Alyx 74.4%, while 91.0% of Alyx's flags were valid against 77.5%; LangSmith Engine surfaced too few issues to score. Arize also watches classic ML models, and LangSmith also hosts the agents it traces. Both take OpenTelemetry traces from other frameworks.
What teams report
Left or turned down 1
NonBioS
NonBioS ruled out LangSmith for its ties to LangChain and looked at Arize Phoenix only briefly, put off by its naming and website, before choosing Langfuse (Mar 5, 2025).
“We looked at Phoenix Arize very briefly, but the naming was a bit confusing. Is it Phoenix, or is it Arize? The website also seemed challenging from a usability perspective, and we were left wondering if that would translate to the tool as well.”
Amit, Author of the NonBioS engineering post, NonBioSNonBioS, , write-up
Evaluated 2
Parlance LabsOn LangSmith
Parlance Labs gave Braintrust Loop, Arize Alyx and LangSmith the same 100 masked production traces (Jul 11, 2026). LangSmith Engine surfaced far fewer issues than a full review, so the authors scored LangSmith's chat agent instead, at 79.5% recall and 77.5% precision.
“But it surfaced only a small set of issues, far fewer than a full review turns up, so we could not use it for the 100-trace comparison. We ran the scored experiment with LangSmith’s chat agent instead.”
Antaripa Saha and Hamel Husain, AI evals consultants and co-authors, Parlance LabsParlance Labs, , write-up Parlance LabsOn Arize
In the same test Alyx caught 74.4% of the hand-labeled failures, fewer than LangSmith's chat agent, but 91.0% of its flags were valid and only three were wrong.
“Arize traded recall for precision. Alyx missed more of the human-labeled failures than Loop, but 91 percent of its flags were valid and only three were wrong.”
Antaripa Saha and Hamel Husain, AI evals consultants and co-authors, Parlance LabsParlance Labs, , write-up
Adopted 3
WayfairOn Arize
Wayfair runs Wilma, a LangGraph-based agent that flags supplier-transfer tickets, and monitors it in Arize with trace-level views and alerts on unusual shifts in key metrics, beside offline evaluation and QA review (Jan 15, 2026).
“Beyond human feedback, Wilma’s performance is continuously monitored through offline evaluation, quality assurance team reviews and observability tooling in Arize. Arize enables trace-level visibility into flagging behavior, supports rapid debugging and provides automated alerts for anomalous shifts in key metrics…”
Luming Wang and coauthors, Machine learning engineers, WayfairWayfair Tech Blog, , write-up HarveyOn LangSmith
Harvey sends the OpenTelemetry traces of its OpenAI Agents SDK legal search agent into LangSmith and runs its offline evaluations there (Dec 10, 2025).
“Centralized evaluation infrastructure that we built using the OpenAI Agent SDK's OpenTelemetry traces enables us to run offline evaluations in LangSmith. We track metrics like hallucination, tool recall, retrieval recall, formatting, and answer quality.”
Lysia Li and five co-authors, Engineering, HarveyHarvey, , write-up PagerDutyOn Arize
PagerDuty runs Arize monitors on its Insights Agent that open PagerDuty incidents when a metric such as tool-selection performance breaches its threshold, and its SRE Agent triages them (Jul 21, 2026).
“It has multiple monitors running in Arize that trigger PagerDuty incidents when breached.”
PagerDuty engineering, Engineering team, PagerDutyPagerDuty Engineering, , write-up
Choosing between LangSmith and Arize
Choose LangSmith if
- People will read single traces to debug: Parlance Labs found LangSmith had the best interface for that, and its chat agent caught 79.5% of 39 hand-labeled failures against Alyx's 74.4% (Jul 11, 2026).
- Your agents run on LangGraph, and you want Studio and LangSmith Deployment next to your traces.
- You want to price traces one by one, keeping most for 14 days and paying more only for the ones you keep 180 days.
- Your team wants LangChain to host its agents.
Choose Arize if
- A person acts on every failure the eval agent flags, so flags must be right: 91.0% of Alyx's were valid in Parlance Labs' test, against 77.5% for LangSmith's chat agent.
- You also run classification, regression, ranking, or object detection models and want drift and performance monitoring in the same platform.
- You want unlimited users on the free and US$50 Pro plans.
- You want a self-hosted starting point: Arize Phoenix, which Arize calls open source, is free to self-host under the Elastic License 2.0, which bars offering it as a managed service, the reason bbv moved its AI Hub to Langfuse in February 2026.
Side by side
| Dimension | LangSmith | Arize |
|---|---|---|
| What teams reportNamed teams' own pages. Parlance Labs' test, below, is the only one that ran both on the same traces. | Harvey runs offline evals in LangSmith on OpenTelemetry traces from the OpenAI Agents SDK (Dec 10, 2025). monday.com keeps one parent LangSmith trace per Infra agent run so any run can be reconstructed (Oct 26, 2026). NonBioS ruled LangSmith out for its LangChain ties (Mar 5, 2025).237 | Wayfair monitors Wilma, a LangGraph-based ticket-flagging agent, with trace-level views and alerts in Arize (Jan 15, 2026). PagerDuty runs Arize monitors that open PagerDuty incidents on its Insights Agent (Jul 21, 2026). NonBioS found Phoenix's naming and website confusing and passed (Mar 5, 2025).457 |
| Eval agent on 100 production tracesParlance Labs, Jul 11, 2026: one apartment-leasing assistant, 39 failures labeled by a domain expert, the same prompt for every system. The authors state no vendor affiliation or sponsorship. | Engine surfaced too few issues to score, so the authors used LangSmith's chat agent: 79.5% recall, 77.5% precision, 20 discoveries and 9 false positives. Best interface for reading single traces; several wrong findings marked high severity.1 | Alyx: 74.4% recall, 91.0% precision, 19 discoveries and 3 false positives. Early runs read only the first 200 characters of each trace and hit compaction and timeouts, which Arize fixed once reported.1 |
| 200,000 traces a month, 10 users, kept 30 daysGILD's calculation from published rates, Oct 28, 2026. Inputs: 200,000 traces a month, 10 spans and 1 code score per trace, 20 KB per trace, 10 users, at least 30 days retrievable, no judge calls, 1 LSU at US$1. | US$1,840 a month on Plus: US$390 for 10 seats, US$950 for 190,000 base traces past the 10,000 included, and US$500 to move all 200,000 to the 180-day tier, since base traces last 14 days.810 | US$50 AX Pro base fee, plus 1,950,000 spans past the 50,000 included at a rate Arize no longer publishes, so the total needs a quote. Self-hosted Phoenix on one 4-vCPU, 8 GiB VM: about US$48 a month in infrastructure, before operations time.171821 |
| PlansList prices in US dollars. LangSmith charges per seat; Arize AX plans include unlimited users. | Developer US$0 for 1 seat with 5,000 base traces a month; Plus US$39 per seat per month with 10,000 base traces a month; Enterprise custom8 | AX Free US$0; AX Pro US$50 a month; AX Enterprise custom17 |
| Included volumeA trace holds many spans: an agent run with 10 spans is 1 LangSmith trace and 10 Arize spans. | 5,000 base traces a month on Developer, 10,000 on Plus8 | 25,000 spans and 1 GB a month on Free; 50,000 spans and 10 GB a month on Pro1718 |
| Usage price | 0.005 LSU per base trace past the included volume, or US$5 per 1,000, plus 0.0025 LSU, or US$2.50 per 1,000, to upgrade a trace to extended retention; 1 LSU is US$18 | Pro includes 50,000 spans and 10 GB a month, and Arize's pricing page lists no overage rate (Oct 28, 2026); Enterprise volume is custom1724 |
| Retention | Base traces 14 days, extended traces 180 days, chosen per tracing project; online evaluators and automation rules upgrade the traces they touch unless you opt out10 | 15 days on Free, 30 days on Pro with a self-serve 13-month add-on, custom on Enterprise1822 |
| Classic ML models | Not covered; LangSmith traces LLM applications and agents16 | Predictions and actuals for classification, regression, ranking, and object detection, with drift, performance, and data quality monitoring19 |
| Self-hosting | Enterprise add-on, on Kubernetes13 | AX Enterprise on Kubernetes; Phoenix free to self-host under the Elastic License 2.0, which bars offering it as a managed service; bbv left Phoenix for that reason (Dec 22, 2025)62021 |
| Cloud regions | GCP us-central1, europe-west4, and australia-southeast1, and AWS us-east-21112 | US, EU, or Canada |
| Compliance | SOC 2 Type 2 certified, HIPAA compliant, and GDPR compliant; DPA on request12 | SOC 2 Type II and GDPR from Pro; HIPAA, SSO, and audit logs on Enterprise17 |
| Ownership | LangChain, which also maintains LangChain, LangGraph, and Deep Agents9 | Dynatrace, which completed its acquisition of Arize on October 1, 2026 and keeps Arize AX and Phoenix available23 |
What practitioners report, side by side
Each product's figures come from its own practitioners, for the group you pick.
Overall signal
All practitioners rate Arize +35, 18 points above LangSmith at +17. Most of the gap is in complaints, with 20 of LangSmith's 96 sources negative against 6 of Arize's 53.
The score counts the authors from all practitioners that take a side, one vote each with their latest view, and is how far positive votes lead negative ones, per 100 votes, so LangSmith's +17 rests on votes split 35 positive, 38 mixed and 19 negative and Arize's +35 on 24, 22 and 6.
A lead of a fifth of the votes reads positive and a lead of half strong positive, with the same lines below zero. The dots show each source, positive, mixed, negative. Run a pilot on one production app to measure which failures it surfaces and how long tracing them takes.
Footprint on the AI Application Platform stack
They differ on five of 9 layers. Coverage is each product's placement on this stack, as its product page shows it. Core is a job the product owns; Supports and Touches are jobs it helps with. Coverage says nothing about how well a product does them.
| Layer | LangSmith | Arize |
|---|---|---|
| Model | ||
| Model gateway | Supports | Touches |
| Build | ||
| Orchestration | Supports | Touches |
| Tool use | Supports | Touches |
| Operate | ||
| Cloud deployment | Supports | Touches |
| Guardrails | Touches | Supports |
The 4 layers where they match
| Layer | LangSmith | Arize |
|---|---|---|
| Model | ||
| Model APIs | Touches | Touches |
| Data | ||
| Retrieval | Touches | Touches |
| Evaluate | ||
| Evaluation | Core | Core |
| Operate | ||
| Observability | Core | Core |
What practitioners praise
Among all practitioners, Free, Simple Self-Hosting and OpenTelemetry & OpenInference reach 5 sources from 3 authors only for Arize, at 18 of 19 and 12 of 16. LangSmith needs a pilot on the same production traffic before you choose on either. Of the three themes both carry, Production Monitoring differs most. It is praised in 4 of the 14 LangSmith sources that discuss it, against 4 of 5 for Arize. Quick Setup reaches 5 sources from 3 authors only for LangSmith, at 11 of 11. Arize needs a pilot on the same production traffic before you choose on it.
| Praised for | LangSmith | Arize |
|---|---|---|
| Trace-Level Debugging | 28 of 35 | 13 of 14 |
| Datasets & Evaluations | 13 of 22 | 6 of 8 |
| Production Monitoring | 4 of 14 | 4 of 5 |
| Quick Setup | 11 of 11 | Not a theme in its sources |
| Free, Simple Self-Hosting | Not a theme in its sources | 18 of 19 |
| OpenTelemetry & OpenInference | Not a theme in its sources | 12 of 16 |
Each count is the sources that praise a theme, or raise it as a concern, out of those that discuss it. A row pairs the two products when both carry a theme under the same or an equivalent name.
A count ending in "so far" has under 5 sources from 3 authors among all practitioners. "Not a theme in its sources" means GILD found no theme of that name in that product's sources.
What practitioners raise as concerns
Among all practitioners, LangSmith alone has five concern themes that reach 5 sources from 3 authors. UI & SDK Reliability and Seat & Trace Pricing lead, at 23 of 25 and 12 of 14. Arize alone has four concern themes that reach 5 sources from 3 authors. Scale & Self-Host Operations and UI, Features & Reliability lead, at 11 of 11 and 8 of 8.
| Raised as a concern | LangSmith | Arize |
|---|---|---|
| UI & SDK Reliability | 23 of 25 | Not a theme in its sources |
| Seat & Trace Pricing | 12 of 14 | Not a theme in its sources |
| LangChain Coupling | 8 of 9 | Not a theme in its sources |
| Self-Hosting & Data Control | 8 of 12 | Not a theme in its sources |
| Closed Source & Vendor Dependence | 4 of 6 | Not a theme in its sources |
| Scale & Self-Host Operations | Not a theme in its sources | 11 of 11 |
| UI, Features & Reliability | Not a theme in its sources | 8 of 8 |
| Access Control & Multi-Team Use | Not a theme in its sources | 4 of 5 |
| Source-Available License | Not a theme in its sources | 3 of 5 |
24 sources
- Parlance Labs: Do Automated Evals Work? (Jul 11, 2026)
- Harvey: How Agentic Search Unlocks Legal Research Intelligence at Harvey (Dec 10, 2025)
- monday.com Engineering: How We Gave AI Agents the Keys to Production and Still Sleep at Night (Oct 26, 2026)
- Wayfair Tech Blog: Flag Early, Fix Faster, Real-Time Intervention for Supplier Transfers with Wilma (Jan 15, 2026)
- PagerDuty Engineering: PagerDuty for AI, How the SRE Agent Triages AI Incidents (Jul 21, 2026)
- bbv AI Hub pull request #851: replace Arize Phoenix with an open-source alternative (Dec 22, 2025)
- NonBioS: The NonBioS LLM Observability Pick (Mar 5, 2025)
- LangSmith plans and pricing
- LangSmith platform
- LangSmith docs: Usage and billing (retention tiers and rate limits)
- LangSmith docs: Cloud (SaaS) architecture and regions
- LangSmith docs: Regions FAQ
- LangSmith docs: Self-hosted LangSmith
- LangSmith docs: Trace with OpenTelemetry
- LangSmith docs: LangSmith Deployment
- LangSmith docs: Observability
- Arize AX pricing
- Arize AX docs: Pricing and usage
- Arize docs: ML models (predictions, drift, and performance)
- Arize AX docs: Self-hosted
- Arize Phoenix docs: License
- Arize AX docs: Release notes (13-month retention on Pro, Jul 10, 2026)
- Dynatrace: Dynatrace completes acquisition of Arize (Oct 1, 2026)
- Arize: Compare Arize