Arize AICompanyArize AIArize AI develops AI observability and evaluation infrastructure for teams building and operating machine-learning systems, LLM applications, multimodal systems, and agents.ArizeOpen companyAI Observability
Arize
Arize AX and Phoenix trace and evaluate LLM apps and agents over OpenTelemetry. Managed AX adds online evals and monitors, plus Signal. Phoenix is free to self-host under the Elastic License 2.0. Arize has been part of Dynatrace since October 2026.
The 30-Second Take
Start on Arize Phoenix self-hosted, and move to managed Arize AX when production monitoring matters.
Both use OpenTelemetry, so the same instrumentation moves between them. PagerDuty and Uber run Arize AX in production.
- Best fit
- Teams that want OpenTelemetry-based tracing and evals they can self-host on Phoenix and move to managed Arize AX when production monitoring matters.
- Main constraint
- Phoenix leaves the database and access rules to your team.
- First move
- Instrument one agent with OpenInference and send its spans to self-hosted Phoenix. Estimate AX Pro overages from that traffic before production.
Why
Watch the trade: Phoenix leaves the database and access rules to your team, and AX allowances are small. AX Pro includes 50,000 spans a month, with no overage rate on Arize's pricing page.
27 practitioners, 78 sources reviewed, Updated Oct 30, 2026How this is measured
- Operating stack
- AI Application Platform
Comparisons
How does Arize compare with the alternatives?
Arize vs LangSmith
- AI Observability
Pick LangSmith for LangGraph agents you want traced and hosted by one vendor. Pick Arize when the same team also watches classic ML models, or when traces must stay on your own servers through free Phoenix.
Arize vs Langfuse
- AI Observability
Pick Langfuse when your license policy needs MIT or you want cloud usage billed in units, where Arize AX bills by spans and GB. Pick Arize when the same team also monitors classic ML models. Phoenix, under the Elastic License 2.0, is Arize's free self-hosted tracer.
Arize vs Braintrust
- AI Observability
- AI Costs
On Parlance Labs' 100 production traces, Braintrust Loop caught the most failures and 91% of Arize Alyx's flags were valid. Pick Arize to self-host free on Phoenix; pick Braintrust for Pro overage rates you can price, US$474 a month at 200,000 traces in GILD's calculation.
Operator Insights
What are teams learning from building on Arize and running it in production?
Jérémie Doucy (jdoucy)Contributor to the Bayes Impact platformBayes Impact
Bayes Impact's nonprofit platform, deployed with a Helm chart that bundles Phoenix.
GitHub, bayesimpact/bayes-platform, published Read the write-up (opens GitHub, bayesimpact/bayes-platform in a new tab)
Observation
Bayes Impact merged a switch of its platform's LLM tracing from Langfuse to Arize Phoenix, exporting plain OpenTelemetry through a collector.
- Takeaway
- Emit plain OpenTelemetry and let a collector map it, so the trace backend becomes a setting. Harden a bundled Phoenix with sign-in and PostgreSQL before production.
- What happened
- A pull request merged Sep 29, 2026 deleted a 673-line Langfuse exporter and the Langfuse packages. The platform now emits standard OTLP, and an OpenTelemetry Collector maps the spans to OpenInference for Phoenix. The plan keeps Langfuse up, read-only, for 30 days, since Phoenix deletes traces after 30 days. The pull request notes that the Phoenix bundled in its Helm chart is not production ready, with no authentication and SQLite by default, and that the Cloud Run production environment has no gateway and exports nothing with this version.
“…the platform emits standard OTLP and knows no trace backend. The OpenTelemetry Collector does the mapping to OpenInference and sends the LLM traces to Phoenix.”
jdoucy - Public engineering write-up (GitHub pull request)
- Team metrics reported
Deployment details
Bayes Impact's nonprofit platform, deployed with a Helm chart that bundles Phoenix.
- Public engineering write-up (GitHub pull request)
- Team metrics reported
- Team
- Bayes Impact platform engineers
- Duration
- Planned: Langfuse kept read-only for 30 days after the Sep 29, 2026 merge
- Workload
- LLM traces from the Bayes Impact platform
- Product version
- Phoenix bundled in the platform's Helm chart, fed by an OpenTelemetry Collector
- Measurement
- 673 lines of Langfuse exporter removed
mfundnContributor, bbvch-ai AI Hubbbv
bbvch-ai AI Hub, a product bbv wants to offer as SaaS, traced with OpenInference instrumentation.
GitHub, bbvch-ai/aihub-core, published Read the write-up (opens GitHub, bbvch-ai/aihub-core in a new tab)
Observation
bbv's AI Hub team dropped Arize Phoenix for Langfuse in February 2026, citing Phoenix's experiment complexity and limited cost tracking.
- Takeaway
- If your team runs experiments from a UI and needs cost per call, test Phoenix and Langfuse on those jobs first: bbv gave up Phoenix's programmatic evaluation API to get both in Langfuse.
- What happened
- mfundn opened pull request #907 on Feb 10, 2026, and it merged on Feb 12. It replaced the Phoenix dataset API with Langfuse and moved tracing to OpenTelemetry and Langfuse spans. The decision record written for the review gives Phoenix's experiment complexity, limited cost tracking and development focus as the context, and Langfuse's UI-driven experiments, native cost tracking and Swiss data sovereignty as the drivers, at the cost of a programmatic evaluation API.
“Replace Phoenix dataset API with Langfuse…”
mfundn - Public engineering write-up (GitHub pull request)
- No team metrics reported
Deployment details
bbvch-ai AI Hub, a product bbv wants to offer as SaaS, traced with OpenInference instrumentation.
- Public engineering write-up (GitHub pull request)
- No team metrics reported
- Team
- bbv AI Hub developers
- Duration
- Opened Feb 10, 2026; merged Feb 12, 2026
- Workload
- Agent tracing and eval datasets for the AI Hub
- Product version
- Self-hosted Phoenix, replaced by self-hosted Langfuse
Buying and deployment
What a team building on it should verify before production traffic grows.
Sourced from Arize documentation
Signals
What is the current evidence saying about Arize?
Named teams run Arize AX in production at PagerDuty, Wayfair, Booking.com and Uber, and builders praise Phoenix's self-hosting while reporting trouble at volume. Each finding links to its public sources.
Practitioner sentiment, teams and leaders
+26 Positive net
of 27 sources that take a side, each author counted once. Small sample.
How this figure is madeCurrent sentiment
Positive, mixed and negative sources
Team accounts lean positive, with 11 positive, 12 mixed and 4 negative among 27 counted. PagerDuty, Wayfair, Booking.com and Uber run Arize AX in production, while Elastic, Apify and bbv left Phoenix in 2026. All 6 discussing self-hosting praise it. All 6 discussing volume report problems. Six compare Langfuse. Platform decisions split between 9 keeping or choosing Arize and 6 moving to Langfuse, Elasticsearch, or no tracer.
Run a pilot on one production app to measure which failures it surfaces and how long tracing them takes.
Evidence field. All 78 sources, one mark each.
Arrange by
Each mark is one source: positive, mixed, negative, no opinion stated, or documentation (not counted).
| Group | Sources | Positive | Mixed | Negative | No opinion stated and not counted in the score | Documentation, checked and not counted |
|---|---|---|---|---|---|---|
| Positive | 11 | 11 | 0 | 0 | 0 | 0 |
| Mixed | 12 | 0 | 12 | 0 | 0 | 0 |
| Negative | 4 | 0 | 0 | 4 | 0 | 0 |
| Not counted | 2 | 0 | 0 | 0 | 2 | 0 |
| Documentation | 14 | 0 | 0 | 0 | 0 | 14 |
| Outside this group | 35 | 14 | 10 | 2 | 9 | 0 |
| Quarter | Sources |
|---|---|
| Q1 2025 | 3 |
| Q2 2025 | 2 |
| Q3 2025 | 11 |
| Q4 2025 | 7 |
| Q1 2026 | 6 |
| Q2 2026 | 5 |
| Q3 2026 | 21 |
| Q4 2026 | 9 |
| Documentation, checked | 14 |
By sentiment
14 documentation pages check facts and are not counted; 35 sit outside this group.
Hover a mark for a quick look, click it or press Enter to open its source.Tap a mark to open its source, then step to its neighbors with Previous and Next.
What practitioners praise
Top praise of 3. Each mark is a source on it; filled marks praise it.
Most common concerns
Each mark is a source on it; filled marks raise it.
All 78 sources behind these findings
- PromptVersion.format() fails on developer messages for Anthropic and Googlejaeminbird, self-hosted Phoenix user (issue author) (self-described: Local, self-hosted Phoenix with arize-phoenix-client from main), GitHub issue, published Oct 28, 2026
- Phoenix's simple architecture for tracing, other tools for evalsnvmdbljstm, Hacker News commenter (self-described: Has used only Phoenix for tracing), Hacker News comment, published Oct 28, 2026
- Phoenix project page slows to seconds per load at 2.4 million root spansschlaepf, Phoenix self-hoster (issue author) (self-described: Self-hosted Phoenix on PostgreSQL 16 with about 2.4 million root spans), GitHub issue, published Oct 26, 2026
- Experiment results lose their evaluator definition when every attempt throwsjuliendemarque-tulip, Phoenix user (issue author) (self-described: Phoenix experiments whose runs can fail before returning a result), GitHub issue, published Oct 25, 2026
- Datadog or Arize as the SaaS choice at workaiexplorations, builder of Praval agents (self-described: Agent observability at the author's workplace), Post on X, published Oct 22, 2026
- Arize, Langfuse and Braintrust all intimidating at firstGanesh Iyer, AI payments builder and creator of the Nura family assistant (self-described: Tried the observability tools for his own agents), Post on X, published Oct 13, 2026
- Arize AX: Pricing and usageArize AI (Official), Product documentation, checked Oct 12, 2026
- Arize AX: Run online evals on tracesArize AI (Official), Product documentation, checked Oct 12, 2026
- Arize AX: SignalArize AI (Official), Product documentation, checked Oct 12, 2026
- Arize AX: SSO & RBACArize AI (Official), Product documentation, checked Oct 12, 2026
- Arize AX: Mask and redact sensitive dataArize AI (Official), Product documentation, checked Oct 12, 2026
- Arize AX: Self-hostedArize AI (Official), Product documentation, checked Oct 12, 2026
- Arize AX: Migrate from Phoenix to Arize AXArize AI (Official), Product documentation, checked Oct 12, 2026
- Phoenix: LicenseArize AI (Official), Product documentation, checked Oct 12, 2026
- Phoenix: Self-hostingArize AI (Official), Product documentation, checked Oct 12, 2026
- Phoenix: ArchitectureArize AI (Official), Product documentation, checked Oct 12, 2026
- Phoenix: PrivacyArize AI (Official), Product documentation, checked Oct 12, 2026
- Phoenix: Data retentionArize AI (Official), Product documentation, checked Oct 12, 2026
- Phoenix: Access control (RBAC)Arize AI (Official), Product documentation, checked Oct 12, 2026
- Phoenix: Arize Phoenix or Arize AXArize AI (Official), Product documentation, checked Oct 12, 2026
- First Phoenix trace showed a coding agent retrying a failing test 9 timesErik Fiala, founder building Breakflare and Everchat (self-described: Tracing his own Claude Code loops with self-hosted Phoenix), Post on X, published Oct 8, 2026
- Introducing AgentHub: Booking.com Platform for Modular Travel AI AgentsMoran Beladev, Booking.com (with Hadas Harush and Chana Ross) (self-described: AgentHub, Booking.com's platform that gives product teams configurable travel agents with shared monitoring), Engineering blog post, Booking.com ML & DS Blog, published Oct 2, 2026
- Self-hosted Phoenix and Langfuse side by side over OTLPbathon, Reddit commenter on r/mlops (self-described: Team projects with self-hosted models, tracing every call, tool call and agent-to-agent call), Reddit comment, r/mlops, published Oct 1, 2026
- feat: enable Phoenix tracing in production deploymentsGennaro Baratta (GennaroBaratta), Developer on the FREE project (FREE, self-hosted production deployment with GPU inference), GitHub pull request, published Sep 29, 2026
- feat(observability): export traces over OTLP and link them to Phoenix instead of LangfuseJérémie Doucy (jdoucy), Jérémie Doucy, contributor to the Bayes Impact platform (self-described: Bayes Impact platform, staging and production), GitHub pull request, published Sep 29, 2026
- chore(evals): Delete the Phoenix eval runnerJiří Spilka (jirispilka), Engineer at Apify (self-described: Apify MCP server evals), GitHub pull request, published Sep 14, 2026
- Can the image dependencies for an offline/air-gapped Phoenix deployment be simplified?ToWorld, Developer (discussion author) (self-described: Team deploying Phoenix in a fully air-gapped environment), GitHub discussion, published Sep 9, 2026
- fix: disable Phoenix tracing in production, gated behind PHOENIX_ENABLEDDanilo Henrique (danhenriquex), Developer (self-described: Production chatbot built on Google ADK), GitHub pull request, published Sep 8, 2026
- feat(phoenix): add Arize Phoenix as a second OTLP trace sinkJacob (JacobPEvans-personal), Developer, self-hosted homelab (self-described: Self-hosted Proxmox stack with Cribl fanning OTLP traces to Langfuse and Phoenix), GitHub pull request, published Sep 3, 2026
- Separating tool-call cost from tool-output cost in PhoenixNormal-Blueberry-385, Reddit poster on r/AI_Agents (self-described: Cost and token observability for a Python LangChain application on Phoenix and OpenTelemetry), Reddit post, r/AI_Agents, published Sep 3, 2026
- Add backend OpenTelemetry tracing with PhoenixRafael Pierre (rafaelpierre), Developer (self-described: Sherlock, a Strands agent backend on AWS), GitHub pull request, published Sep 1, 2026
- Wants reusable Phoenix interface componentsJohn Zabroski, founder of Simmaculate and FluentMigrator maintainer (self-described: Building his own eval viewer), Post on X, published Aug 31, 2026
- spike(observability): compare Traceway, Tempo, Langfuse, Phoenix, SigNoz and Grafana Agent Observabilitymashkovd, Developer on mctl's platform team (mctl agent platform choosing an OTLP backend), GitHub issue, published Aug 26, 2026
- feat: self-hosted OTel observability (Phoenix/Langfuse) for multi-agent tracing - not LangSmithakaszubski, Developer (issue author) (self-described: Personal multi-agent coding harness on Claude Code), GitHub issue, published Aug 6, 2026
- feat(evals): host Phoenix on Cloud Run and Cloud SQLTapan Chugh (chughtapan), Developer (self-described: moltzap evals, dashboards that outlive an ephemeral GKE cluster), GitHub pull request, published Aug 6, 2026
- feat(observability): standardise on Phoenix; restore Langfuse v4 as fallbackJordi Martinez (jordimo), Jordi Martinez, developer (self-described: Own infrastructure, run locally and on an AWS host), GitHub pull request, published Aug 6, 2026
- [kbn-evals] Remove Arize Phoenix executor and @arizeai/phoenix-client dependencyarturoliduena, Developer on Elastic's Kibana evaluation framework (Kibana's LLM evaluation framework), GitHub pull request, published Jul 29, 2026
- Comment on: A CLI that shows which sub-agent burned your tokens in one runrrkher059, Hacker News commenter (self-described: Built a CLI over OpenInference spans), Hacker News comment, published Jul 29, 2026
- Building Closed-Loop Evals for a Multimodal Agent at Uber ScaleJai Chopra, Sr. Product Manager, Uber (with Soumya Gupta, Sr. Software Engineer) (Uber's computer-vision team, running an agent that edits Uber Eats merchants' food photos), Conference talk with transcript, AI Engineer World's Fair 2026, published Jul 24, 2026
- PagerDuty for AI: How the SRE Agent Triages AI IncidentsPagerDuty Engineering team (self-described: PagerDuty's Insights Agent, a chat assistant for operational questions, watched by Arize monitors), Engineering blog post, PagerDuty engineering blog, published Jul 21, 2026
- [ENHANCEMENT]: Support attaching images to non-LLM spans (TOOL/CHAIN) without faking LLM message attributesrobbert-gijsels, Developer (issue author) (self-described: Team tracing a computer-vision pipeline with OpenTelemetry and OpenInference), GitHub issue, published Jul 15, 2026
- Evaluate Agents Locally with Arize Phoenix (Hands-On Guide)Folarin Akinloye, independent developer (self-described: Tracing and judging their own agent locally), Personal blog post, folarin.dev, published Jul 12, 2026
- Auto-evals: Braintrust Loop, Arize Alyx and LangSmith on the same 100 tracesAntaripa Saha and Hamel Husain, Parlance Labs (self-described: 100 hand-labelled traces from a production apartment-leasing assistant), Independent evaluation, Parlance Labs blog, published Jul 11, 2026
- From Traditional ML to AI Agents: How Booking.com Scales AI Observability With Arize AIAmir Bitaraf, Booking.com (with Shahaf Veber) (self-described: Booking.com's shared observability platform for traditional ML models and LLM agents; the authors describe a partnership with Arize), Engineering blog post, Booking.com ML & DS Blog, published Jul 10, 2026
- Add Arize Phoenix integration for LLM tracing and evalsLaurent Bouis (lbouis), Developer (self-described: raffy-research, report generation with LLM-as-a-judge evals), GitHub pull request, published Jun 19, 2026
- Spent 3 months evaluating Galileo, Arize, Langfuse and LangSmith for production LLM monitoring. Notes for anyone doing the same.Total_Listen_4289, Reddit user, r/LLMDevs (self-described: Small team shipping agents, evaluating observability for production), Reddit post, published May 30, 2026
- Automating and Optimizing Financial Signal Discovery with Multi-Agent SystemsPeihan Huo, Hanyue He, and Ioana Boier of NVIDIA, with Yang Shen of Huatai Securities (Multi-agent signal discovery workflow on NeMo Agent Toolkit for quantitative research), Engineering blog post, NVIDIA Technical Blog, published May 21, 2026
- chore: remove Arize Phoenix (telemetry / observability) from VSSzac-wang-nv, Developer on NVIDIA's Video Search and Summarization blueprint (NVIDIA AI Blueprint shipped with Docker Compose and Helm), GitHub pull request, published May 20, 2026
- fix: replace arize-phoenix with arize-phoenix-otel to fix OOM (500s on /api/traces)Venkata Subbarao Gorantla (gvsrusa), Developer (self-described: Agent service on Cloud Run sending traces to Phoenix), GitHub pull request, published May 9, 2026
- BulkInserter opens up to 7 DB sessions per loop iteration, exhausting connection pool under loadDavid O'Connell (doconnell2020), David O'Connell, GitHub profile lists Telnyx (Self-hosted Phoenix on PostgreSQL with 2 replicas), GitHub issue, published Mar 25, 2026
- Replace Langfuse with Phoenix, add LiteLLM proxy, add Snowflake Labs MCPMichael Svendsen (thesven), Michael Svendsen, developer (self-described: Own self-hosted agent stack with LibreChat and LiteLLM), GitHub pull request, published Mar 4, 2026
- [BUG]: Error fetching GraphQL query ProjectPageHeaderQuery when opening project with >10k spans (costSummary None)Lim Ghim Boon (GhimBoon), Lim Ghim Boon, GitHub profile lists Intel (Self-hosted Phoenix 12.33.1 on PostgreSQL behind nginx), GitHub issue, published Feb 5, 2026
- Slow
traceandspansearch performance when searching byspans.attributesoginskis, Developer (discussion author) (self-described: Team self-hosting Phoenix on Kubernetes with Cloud SQL PostgreSQL, about 100,000 LLM interactions a day), GitHub discussion, published Feb 4, 2026 - Comment on: Show HN: A MitM proxy to see what your LLM tools are sendingvitorbaptistaa, Hacker News commenter (self-described: Tracing their own Claude Code sessions), Hacker News comment, published Jan 29, 2026
- Flag Early, Fix Faster: Real-Time Intervention for Supplier Transfers with WilmaFarshid Bahrami, Wayfair (with Atharva Deo, Roger Bock, Ying Jiang, and Luming Wang) (self-described: Wayfair's Wilma, a LangGraph agent that flags supplier transfer tickets for associates), Engineering blog post, Wayfair tech blog, published Jan 15, 2026
- fix(aihub): Replace Arize Phoenix with open source alternativeJoel Barmettler (joelbarmettlerUZH), Developer at bbv, bbvch-ai AI Hub (bbvch-ai AI Hub, a product the team wants to offer as SaaS), GitHub pull request, published Dec 22, 2025
- [ENHANCEMENT]: Group-Based Multi-Tenancy for Data Isolationmetalshanked, Developer (issue author) (self-described: Planning one central Phoenix instance for several internal teams), GitHub issue, published Dec 8, 2025
- A pragmatic guide to LLM evals for devsHamel Husain, former machine learning engineer at Airbnb and GitHub, guest author (self-described: NurtureBoss, an apartment leasing assistant, reviewing agent traces), Newsletter guest post, The Pragmatic Engineer, published Dec 2, 2025
- [BUG]: Transient error StatusCode.RESOURCE_EXHAUSTEDhanweidong, Developer (issue author) (self-described: Self-hosted Phoenix receiving OpenInference traces over gRPC), GitHub issue, published Oct 27, 2025
- [ENHANCEMENT]: Apache-2.0 (or similar) relicensing for
arize-phoenix-clientwillkill07, Developer (issue author) (self-described: Open-source projects integrating with Phoenix), GitHub issue, published Oct 13, 2025 - Evaluating AI Agents with DeepEval and Arize Phoenix: Lessons from Our Integration JourneyValentin Velev, Data Scientist, Codify (Codify AG, Swiss software and AI consultancy, evaluating its own agents), Engineering blog post, Codify blog, published Oct 6, 2025
- Selecting The Right AI Evals ToolHamel Husain, AI evals consultant and course instructor (self-described: Three-person panel that watched LangSmith, Braintrust, and Arize Phoenix complete the same eval assignment), Independent evaluation, hamel.dev, published Oct 1, 2025
- Comment on: LLM Observability in the Wild, Why OpenTelemetry Should Be the Standarddat_attack, Hacker News commenter (self-described: Hacker News thread on OpenTelemetry for LLM observability), Hacker News comment, published Sep 29, 2025
- Comment on: LLM Observability in the Wild, Why OpenTelemetry Should Be the Standardjkisiel, Hacker News commenter (self-described: Small startup choosing its LLM observability stack), Hacker News comment, published Sep 28, 2025
- Arize integration for .NETCommercial_Artist632, Reddit user, r/LLMDevs (self-described: Arize AX tracing, moving from a Python codebase to .NET), Reddit post, published Sep 28, 2025
- Comment on: LLM Observability in the Wild, Why OpenTelemetry Should Be the StandardCuriouslyC, Hacker News commenter (self-described: Hacker News thread on OpenTelemetry for LLM observability), Hacker News comment, published Sep 27, 2025
- Comment on: LLM Observability in the Wild, Why OpenTelemetry Should Be the Standard7thpower, Hacker News commenter (self-described: Hacker News thread on OpenTelemetry for LLM observability), Hacker News comment, published Sep 27, 2025
- Self-hosted Phoenix as the one observability tool in a lean agent stackProdigyManlet, machine learning engineer on r/mlops (self-described: Building agents with PydanticAI or Google's ADK, paying only for the basics), Reddit comment, r/mlops, published Sep 20, 2025
- Expand RBAC To Limit Access To Projectsring-pete, Developer (discussion author) (self-described: One Phoenix instance serving development, QA, and production projects), GitHub discussion, published Sep 19, 2025
- [BUG]: Prebuilt Evaluators break for Experiments.taoari, developer (self-described: Self-hosted Phoenix 11.29.0, dataset experiments), GitHub issue, published Aug 28, 2025
- PagerDuty + Arize: Building End-to-End Observability for AI Agents in ProductionEveraldo Aguiar, Senior Engineering Manager, Applied AI, PagerDuty (co-written with Antonio Correia and with Arize's Rich Young) (PagerDuty Advance agents: Insights, Shift, and SRE agents), Engineering blog post, PagerDuty engineering blog, published Aug 13, 2025
- [BUG]: DeadlockDetectedErrorluismarquezgft, Developer (issue author) (self-described: Self-hosted Phoenix on OpenShift with LangChain instrumentation), GitHub issue, published Jul 31, 2025
- Old logs are getting deleted when the number of log request is highvignesh-arivazhagan, Developer (discussion author) (self-described: Team building a high-throughput LLM application), GitHub discussion, published Jul 17, 2025
- The RAG Stack We Landed On After 37 FailsJonathan Fernandes, independent AI engineer (self-described: RAG systems for financial clients, deployed with Docker Compose), Conference talk with transcript, AI Engineer World's Fair 2025, published Jun 3, 2025
- Add support for Arize Phoenix for LLM observability and evaluationyurif-freitas, Open WebUI user (issue author) (self-described: Team running Phoenix alongside Open WebUI), GitHub issue, published May 16, 2025
- Authorization beyond Authenticationlucasalvarezlacasa, Developer (issue author) (self-described: Self-hosted Phoenix behind Keycloak), GitHub issue, published Mar 5, 2025
- The NonBioS LLM Observability PickNonBioS team (self-described: NonBioS, building an AI software engineer that handles sensitive customer data), Company blog post, NonBioS, published Mar 5, 2025
- Optimizing LLMs in Insurance with DSPy: Beyond Manual TuningJeronim Morina, AI and MLOps Engineer, AXA (AXA Germany's data team, building a DSPy pipeline on its internal Secure GPT platform), Conference talk with transcript, AI Engineer World's Fair 2024, video published February 2025, published Feb 16, 2025
Feb 2025 to Oct 2026.
64 sources from practitioners, by month and sentiment.
Where It Fits
Where does Arize belong in the stack?
Arize is Core across Evaluate and Operate and touches Model, Build, and Data.
- Business area
- Product & Engineering
- Function
- Software Development
- Operating stack
- AI Application Platform
- Primary role
- LLM & Agent Observability and Evaluation
Model
Model APIs: Touches
Model APIs, touches. Arize records application calls to OpenAI, Anthropic, Google, Bedrock, and other providers. Its playground and LLM judges use your configured provider keys. It supplies no models.
Model gateway: Touches
Model gateway, touches. AX connects to gateways such as LiteLLM, OpenRouter, and TrueFoundry for playground and judge calls. Arize traces those calls. The gateway controls routing and spend limits.
Build
Orchestration: Touches
Orchestration, touches. OpenInference auto-instruments LangGraph, LangChain, LlamaIndex, CrewAI, DSPy, the Vercel AI SDK, and other frameworks. The framework builds and runs the agent.
Tool use: Touches
Tool use, touches. Arize records tool arguments and outputs. Phoenix evaluates tool selection and invocation. The application executes the tools.
Data
Retrieval: Touches
Retrieval, touches. Retriever spans show returned documents and evaluators score relevance. The vector store handles search and indexing, including embeddings.
Evaluate
Evaluation: Core
Evaluation, core. Phoenix and AX evaluate prompt and model changes using datasets and experiments. They support LLM judges and code evaluators, plus human annotations. AX adds agent-as-a-judge and labeling queues, with online production evals and CI templates.
Operate
Observability: Core
Observability, core. Arize traces LLM apps and agents over OpenTelemetry. AX combines sessions and token-cost tracking with dashboards, online evals, monitors, and alerts. Phoenix self-hosts the same tracing.
Cloud deployment: Touches
Cloud deployment, touches. AX runs as SaaS in US, EU, or Canada regions, or on Kubernetes under Enterprise. Phoenix runs on Docker or Kubernetes. Neither hosts your application or models.
Guardrails: Supports
Guardrails, supports. Arize Guards screen user inputs and LLM outputs at runtime. Failed messages trigger corrective actions such as a default response, a retry, or blocking the output.
Market mapThe layers Arize covers, and who else owns each one.
Evaluation & testing
Observability & tracing
Guardrails & AI security
- Prisma AIRS
- NeMo Guardrails
Model APIs
Model gateways & routing
- LiteLLM
- OpenRouter
- Portkey
- Vercel AI Gateway
4 more layers on the full map. Core, Supports and Touches describe coverage. They say nothing about quality.