Key findings
- Of 17 sources discussing self-hosting, 16 praise running Phoenix in their own infrastructure from one container.
- Trace-level debugging draws praise in 9 of 10 sources discussing it, including NVIDIA's quantitative research agents. Datasets and evaluations draw praise in 7 of 9, including PagerDuty's regression experiments and Elastic's first Kibana eval framework.
- All 11 sources discussing scale and self-host operations raise concerns. UI and feature gaps appear in 8 sources, access control in 4, and the Elastic License in 3.
- Of the 16 sources that describe a platform choice, 9 chose or kept Arize and 7 chose another tool. Langfuse is the alternative named most often, in 10 sources.
Teams choose Arize Phoenix because it self-hosts for free and speaks OpenTelemetry, and the trouble they report starts at production volume. Pick Phoenix if your team can run PostgreSQL at your span volume and can accept one set of roles across every project. Otherwise buy managed Arize AX, which PagerDuty and Wayfair run in production, or follow the teams that moved to Langfuse.
What teams trace and evaluate
PagerDuty traces its Insights, Shift, and SRE agents in AX. It turns golden questions into dataset experiments, scores production on relevance, groundedness, and tool selection, and pages the agent team when a monitor breaches. Wayfair's Wilma agent clears about 65% of supplier-transfer tickets. Wayfair monitors it with human feedback, offline evaluation, and Arize traces and alerts.
Bayes Impact replaced its Langfuse exporter with plain OpenTelemetry and a collector that maps spans to OpenInference for self-hosted Phoenix. Elastic first ran Kibana evals on Phoenix datasets and experiments, then built its own storage in Elasticsearch. Smaller projects send AI SDK spans to Phoenix Cloud and record LLM-judge scores as experiments. NVIDIA's financial services architects trace long-running signal discovery agents in Phoenix to see reasoning and token use in real time.
Why teams self-host
Practitioners choose Phoenix because it needs less infrastructure. One measured a Phoenix container at about 400 MiB against 1,848 MiB for four Langfuse v4 containers. Another replaced Langfuse's ClickHouse, Redis, and MinIO services with one Phoenix container. A startup reported that Phoenix was the only tool that worked out of the box. A homelab operator made Phoenix the default for OTLP ingestion with Langfuse as a tested fallback, and named Elastic License 2.0 as one of its real trade-offs.
OpenTelemetry support draws praise in 10 of 14 sources, and several value spans in the portable OpenInference format. The 4 critical accounts describe hard instrumentation, a heavy Claude Code setup, image support limited to LLM spans, and gRPC export failures with no explanation. That last problem led one developer to turn off production tracing.
Where self-hosted Phoenix hurts
Most concerns come from bug reports and feature requests in Arize's own repository. Users report slow attribute searches at about 100,000 interactions a day and PostgreSQL deadlocks that lost traces. A bulk inserter exhausted a connection pool across two replicas. Projects above 10,000 spans failed to load, and high traffic pushed out older traces. Other accounts describe rejected large exports, and a container that ran out of memory because the team installed full Phoenix and skipped the tracing client. Codify had to query Phoenix's database by hand to reread past experiment results, and set experiment concurrency to 1 to avoid errors.
Arize closed the deadlock report as fixed in August 2025, and a maintainer endorsed adding PostgreSQL indexes on the span attributes a team searches. The connection-pool, large-project, and large-export issues were still open on October 25, 2026, so load-test those paths before production.
Phoenix roles apply to the whole instance, so a viewer sees every project, and authentication stays off until an administrator turns it on. Teams that share an instance raise access control for that reason. Per-project roles sit in Arize's open roadmap epic, Phoenix issue #11646, which had no ship date when GILD checked it that day. AX already offers organizations and spaces with custom roles.
Why teams choose another tool, or stay
Teams that left name fit more often than failure. NVIDIA's blueprint team removed Phoenix because it added a container and dependencies the team did not want to maintain. Elastic moved evaluation storage into Elasticsearch. The bbvch-ai AI Hub team replaced Phoenix with Langfuse for tracing and datasets. Apify deleted its Phoenix eval runner once Langfuse-based gates took its place. bbv proposed Langfuse because Phoenix's Elastic License blocks including it in a SaaS product. NonBioS found the naming confusing on a first look, and a Hacker News commenter found Phoenix clunky to instrument next to Langfuse.
Teams that stay cite self-hosting, OpenTelemetry support, or production evals. They include Bayes Impact, an Open WebUI team, the startup for which only Phoenix worked out of the box, developers who replaced Langfuse with Phoenix or run both, and PagerDuty on AX.
What to settle before choosing
Five questions decide between Phoenix, Arize AX, and the alternatives. Answer them with your own traffic before a pilot ends.
- Must traces stay in your own network, and who will run the database if they do?
- How many spans a month will production send, and what overage rate will Arize put in writing for AX Pro?
- Will one instance serve several teams that must not see each other's traces?
- Does your license policy accept the Elastic License 2.0?
- Will your instrumentation follow OpenInference, the OpenTelemetry GenAI conventions, or both?
| Theme | Sources | Share of sources |
|---|---|---|
| Free, simple self-hosting (praise) | 16 of 17 | |
| Trace-level debugging (praise) | 9 of 10 | |
| OpenTelemetry & OpenInference (praise) | 10 of 14 | |
| Datasets & evaluations (praise) | 7 of 9 | |
| Scale & self-host operations (concern) | 11 of 11 | |
| UI, features & reliability (concern) | 8 of 8 | |
| Access control & multi-team use (concern) | 4 of 5 |
GILD count of 47 public sources, March 2025 to September 2026, opened Oct 12 and Oct 25, 2026
Methodology
GILD read 47 public sources about Arize Phoenix and AX, 42 on October 12, 2026 and 5 more on October 25, published between March 2025 and September 2026. They come from Hacker News, Reddit, GitHub issues, pull requests and discussions, engineering and personal blogs, an independent tool evaluation, a newsletter guest post, and a recorded conference talk. GILD searched Hacker News through Algolia, Reddit through Arctic Shift, and GitHub directly. GitHub issues and discussions make up about a third of the set at most, because they lean toward problems. Sixteen of those records report a problem. Counts describe these 47 sources. Arize's own blog, customer stories, and comparison pages are left out. PagerDuty co-wrote its account with an Arize solutions architect. GILD checked product behavior against Arize's docs on October 12, 2026, and checked issue status, the access-control roadmap, and AX pricing on October 25, 2026.
Products in this piece
Sources
- Public source
- 26
- Official documentation
- 5
- PagerDuty engineering blog: PagerDuty + Arize, building end-to-end observability for AI agents in production (Aug 13, 2025)
- Wayfair tech blog: Flag Early, Fix Faster, real-time intervention for supplier transfers with Wilma (Jan 15, 2026)
- NVIDIA Technical Blog: Automating and optimizing financial signal discovery with multi-agent systems (May 21, 2026)
- Valentin Velev (Codify): Evaluating AI agents with DeepEval and Arize Phoenix, lessons from our integration journey (Oct 6, 2025)
- bayesimpact/bayes-platform pull request #877: export traces over OTLP and link them to Phoenix instead of Langfuse (Sep 29, 2026)
- apify/apify-mcp-server pull request #1376: delete the Phoenix eval runner (Sep 14, 2026)
- bbvch-ai/aihub-core pull request #851: replace Arize Phoenix with an open source alternative (Dec 22, 2025)
- elastic/kibana pull request #281472: remove the Arize Phoenix executor from kbn-evals (Jul 29, 2026)
- jordimo/infra pull request #15: standardise on Phoenix, keep Langfuse v4 as fallback (Aug 6, 2026)
- dryvist/ansible-proxmox-ai pull request #646: add Arize Phoenix as a second OTLP trace sink (Sep 3, 2026)
- Hacker News comment: Phoenix the only tool that worked out of the box for a small startup (Sep 28, 2025)
- Phoenix issue #10622: OpenTelemetry GenAI semantic conventions support (Dec 13, 2025)
- Phoenix discussion #11233: slow attribute search at about 100,000 interactions a day (Feb 4, 2026)
- Phoenix issue #8879: PostgreSQL deadlocks and lost traces on OpenShift (Jul 31, 2025)
- Phoenix issue #12358: bulk inserter exhausts the connection pool under load (Mar 25, 2026)
- Phoenix issue #11243: projects with more than 10,000 spans fail to load (Feb 5, 2026)
- Phoenix discussion #8625: older traces lost under high request volume (Jul 17, 2025)
- Phoenix issue #10021: RESOURCE_EXHAUSTED when exporting large traces (Oct 27, 2025)
- Phoenix issue #10504: group-based multi-tenancy for data isolation (Dec 8, 2025)
- Phoenix discussion #9565: limit access to projects (Sep 19, 2025)
- NVIDIA video-search-and-summarization pull request #583: remove Arize Phoenix from VSS (May 20, 2026)
- bbvch-ai/aihub-core pull request #907: replace Arize Phoenix with Langfuse (Feb 10, 2026)
- NonBioS: The NonBioS LLM Observability Pick (Mar 5, 2025)
- Hacker News comment: Phoenix clunky, happier with Langfuse (Sep 27, 2025)
- Folarin Akinloye: Evaluate agents locally with Arize Phoenix (Jul 12, 2026)
- Phoenix docs: Architecture
- Phoenix docs: Access control (RBAC)
- Phoenix docs: Self-hosting
- Arize AX docs: Run online evals on traces
- Arize AX pricing
- Phoenix issue #11646: Custom RBAC & Roles roadmap epic, open (opened Feb 20, 2026)