GILD verdict
Choose Datadog Agent Observability when your operators already work in Datadog and want each agent run traced under the same trace ID as the services, hosts, and user sessions around it.
- The workflow
- A team running LLM features or agents in production on services Datadog APM already traces, with SREs and AI engineers sharing one incident view.610
- The decisive evidence
- DocumentedDatadog meters only LLM spans, one per call to a model provider, and leaves tool, retrieval, and workflow spans unbilled. Pro starts at US$160 a month for 100,000 LLM spans billed annually.610
- DocumentedThe SDKs for Python, Node.js, and Java auto-instrument OpenAI, Anthropic, Bedrock, LangChain, LangGraph, LiteLLM, and other libraries, and OpenTelemetry spans that follow the GenAI 1.37+ or OpenInference conventions arrive without the SDK.1213
- ReportedA Source.ag engineer reported on Jun 16, 2026 that nested LangChain and Anthropic spans made Datadog's cost rollup overcount spend about 2.2 times, and Arize's customer story of Aug 17, 2026 says Handshake's internal LLM service sends every call to both Datadog and Arize AX.12
- DocumentedDatadog's LLM Observability product URL was redirecting to Agent Observability by Mar 22, 2026, and the Agent Observability page was first archived on Jun 10, 2026, the day after DASH 2026; GILD found no dated rename announcement.458
- GILD analysisPublic bug reports on this Hub's Signals describe nested LangChain and Anthropic spans that overcounted one team's spend about 2.2 times, LiteLLM fallbacks that emit two LLM spans per call, and streamed Amazon Nova calls recorded with zero tokens. No public account measures how often span counts or token totals drift on a stack that has none of these defects, so compare a week of Datadog totals with your model invoices before relying on its cost views.
- The tradeoff
- Traces keep 15 days unless you buy a retention add-on through Datadog's account team, judge evaluations bill tokens to your own model account, and the product does not run on Datadog's US government sites.1618
- What would change the call
- The call changes if your AI engineers need a separate evaluation workspace or a tracer you host, or if your team does not run Datadog for anything else, since correlation with APM is the main reason to pick it.6
Datadog Agent Observability
AI Observability by Datadog
+32 PositiveFit and limits
Who it fits
- Your services already send APM traces to Datadog, and you want model calls in the same trace and trace ID as the request, the host, and the user session.
- Agent runs make many tool and retrieval calls, which Datadog does not bill, so the meter follows model calls alone.
- You want one product to trace production, run LLM-as-a-judge and code evaluators, and gate releases on offline experiments over versioned datasets.
- Your security team already governs Datadog through SAML single sign-on, roles, and Audit Trail, and wants AI traces under the same controls.
- Operators and AI engineers read the same calls in different tools: Arize's customer story of Aug 17, 2026 says Handshake's LLM Orca service sends every call to both Datadog and Arize AX.
Where it falls short
- Span counts and token totals depend on the tracer. Public reports from 2026 describe nested integrations and LiteLLM fallbacks that emit two LLM spans per call, and streamed Nova calls recorded with zero tokens.
- Traces keep 15 days on every plan by default. Longer retention is an add-on arranged through Datadog's account team, and it is not offered on the Free tier.
- Custom LLM judges run on the model account you connect, so your provider bills the judge tokens, and Datadog states that it does not support data residency for an OpenAI judge account.
- Agent Observability is not supported on the US1-FED and US2-FED government sites, and customers under Datadog's BAA may use third-party generative AI services only when they are listed as HIPAA-eligible.
- Prompts that must stay on your own servers: Datadog runs only as SaaS on its own sites, and NonBioS chose self-hosted Langfuse over it and a dozen other tools for that reason (Mar 5, 2025).
Implementation reality
Choose the instrumentation path
Use the Datadog SDK for Python, Node.js, or Java (Go is experimental) to auto-instrument supported libraries. Teams already on OpenTelemetry can send spans that follow the GenAI 1.37+ or OpenInference conventions, OpenLLMetry 0.47+, or Langfuse attributes without the SDK.
Mask before export
Write span processors in the SDK to redact inputs and outputs or drop spans before they leave the application. Sensitive Data Scanner then scans what arrives, with 1 GB of scanning included per 10,000 LLM spans. Restrict each application's traces to chosen teams with the ml_app tag.
Count spans before you commit
Trace a week of real traffic on Free, count LLM spans per run, and compare token totals with your provider invoices. Set a trace sampling rate in the SDK if you need fewer billed spans; sampling is supported from ddtrace 4.12.0, dd-trace 5.110.0, and dd-trace-java 1.66.0.
Connect judges and gate releases
Connect an OpenAI, Azure OpenAI, Anthropic, Bedrock, Vertex AI, or OpenAI-compatible account for bring-your-own-key judges. Build versioned datasets from production traces and run offline experiments on each prompt or model change before release.
What to verify before you buy
Billing and retention
- Billing term: US$160, US$200, or US$240 a month for Pro, and US$3.50, US$4.20, or US$5 per 10,000 further LLM spans.
- Retention: which retention each application needs, and the add-on price for that term on Datadog's pricing list.
Identity and access
- SAML: teammates sign in with the credentials in your Active Directory, LDAP, or other identity store configured with a SAML Identity Provider. Setup requires Datadog Administrator access, and Datadog's docs say to reach out to support if SAML is not enabled on your account.
- SCIM: provisions and deprovisions users automatically, in sync with your identity provider. Datadog lists it in the Infrastructure Pro, Infrastructure Enterprise, and Startup plans, so ask how a contract with Agent Observability alone provisions users.
- Team access: the ml_app tag restricts each AI application's data to chosen teams.
- Audit Trail: shows who is using Datadog within your organization and in what context. A user with Audit Trail Write permission must enable it under Audit Trail Settings.
Data location and HIPAA
- Site: which Datadog site holds your data.
- Judge accounts: judge evaluations run on the model account you connect, so check whether those accounts meet your residency and HIPAA terms.
- HIPAA list: Datadog Legal's HIPAA-Eligible Services list, which names Agent Observability (formerly LLM Observability), was last updated July 23, 2026.
What to measure in your pilot
- No public account measures span-count drift on a clean stack; compare Datadog's LLM span and token totals with your provider invoices weekly for a month, and file any gap over 5% with Datadog support.
- Tracer fixes ship often; pin the ddtrace or dd-trace version you validated and retest span counts on a staging agent before each upgrade.
- Datadog published no dated rename notice: the old product URL redirected by Mar 22, 2026 and the Agent Observability page was first archived Jun 10, 2026; track the product under both names in release notes and SDK changelogs until the llm_observability paths and DD_LLMOBS settings change.
Sources
- Public source
- 5
- Official documentation
- 21
- dd-trace-js issue #8936: two LLM spans per call overcount spend about 2.2x (Jun 16, 2026)
- Arize customer story: How Handshake deployed and scaled 15+ LLM use cases in under 6 months (Aug 17, 2026)
- NonBioS: The NonBioS LLM observability pick (Mar 5, 2025)
- Internet Archive: Datadog's LLM Observability product URL redirecting to Agent Observability (Mar 22, 2026)
- Internet Archive: first capture of Datadog's Agent Observability page (Jun 10, 2026)
- Datadog pricing: Agent Observability
- Datadog pricing list
- Datadog Agent Observability product page (titled Agent Observability | LLM Observability)
- Datadog blog: DASH 2026, guide to Datadog's newest AI announcements (Jun 9, 2026)
- Datadog docs: Agent Observability
- Datadog docs: Agent Observability SDK reference
- Datadog docs: Automatic instrumentation for Agent Observability
- Datadog docs: OpenTelemetry instrumentation for Agent Observability
- Datadog docs: Agent Observability evaluations
- Datadog docs: LLM-as-a-judge evaluations
- Datadog docs: Connect an LLM provider for bring-your-own-key evaluations
- Datadog docs: Agent Observability offline experiments
- Datadog docs: Agent Observability data governance
- Datadog docs: Getting started with Datadog sites
- Datadog docs: Single sign-on with SAML
- Datadog docs: User provisioning with SCIM
- Datadog docs: Datadog Audit Trail
- Datadog docs: HIPAA compliance
- Datadog Trust Hub: platform security
- Datadog Trust Center: compliance certifications
- Datadog Legal: HIPAA-Eligible Services