Every source behind the read

38 sources in Teams and technical leaders, with product documentation Showing all 63 sources

1. Mixed

   roroghost17, Bifrost contributor

   feat(datadog): add otel_llmobs_conversion and llmobs_gateway_modeGitHub pull request (Bifrost)

   Adds Datadog plugin settings to the Bifrost gateway, with Datadog's OpenTelemetry-to-LLM Observability conversion off by default so Datadog does not duplicate spans Bifrost already sends.

   Published Oct 17, 2026self-described: Bifrost AI gateway's Datadog plugin

   Not counted

   The author contributes the Datadog plugin of the Bifrost AI gateway. Kept in the record; it takes part in no figure.

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   Bifrost Helm chart 2.1.45

2. Mixed

   Anurag Ojha, engineer at Mastra (GitHub profile)

   [BUG]: llmobs.flush() is documented as blocking but returns before anything is sentGitHub issue

   llmobs.flush() returns before sending, so a short-lived process that flushes and exits loses its spans and evaluations; the Mastra integration waits two seconds as a workaround.

   "`llmobs.flush()` is a blocking function that submits all buffered Agent Observability data to the Datadog backend."

   Published Oct 14, 2026self-described: Building the @mastra/datadog integration for short-lived processes

   Not counted

   The author works at Mastra and builds its @mastra/datadog integration. Kept in the record; it takes part in no figure.

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   dd-trace 5.120.0, Node.js 24

3. Mixed

   Ino de Bruijn, cBioPortal and Memorial Sloan Kettering (GitHub profile)

   Promote #168 to main: fix Datadog LLM Observability tool spansGitHub pull request (cBioPortal)

   Fixes the cBioPortal MCP server after its Datadog LLM Observability tool spans dropped to zero, because it called an LLMObs method that does not exist in ddtrace 4.13.

   Published Oct 12, 2026self-described: cBioPortal MCP server in production

   Counts toward

   • Overall signal
   • Missing or Dropped Traces: Raised as a concern
   • Where should a team keep its LLM and agent traces?: Keep them in Datadog

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   ddtrace 4.13

4. Mixed

   Kyle Purdon, engineer at Semgrep (GitHub profile)

   [BUG]: Generated dd tracestate exceeds 256-character limit and is rejected by OpenTelemetryGitHub issue

   LLM Observability propagation metadata pushes Datadog's tracestate past the W3C 256-character limit, so OpenTelemetry warns and drops the vendor trace state.

   "We encountered this with LLM Observability propagation metadata…"

   Published Oct 5, 2026self-described: LLM Observability propagation alongside the OpenTelemetry API

   Counts toward

   • Overall signal
   • OpenTelemetry Intake: Mixed or negative
   • Missing or Dropped Traces: Raised as a concern

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   ddtrace 4.14.0 and 4.15.4, opentelemetry-api 1.42.1

5. Mixed

   Christopher Krah, engineer at UiPath (GitHub profile)

   [BUG]: Anthropic streamed spans report 1h cache writes as ephemeral_5m_input_tokensGitHub issue

   Streamed Anthropic calls that write a 1-hour prompt cache are recorded as 5-minute cache writes, while the same call without streaming splits them correctly, so cache cost is misreported.

   "A streamed `messages.create` that writes a 1-hour prompt cache gets `ephemeral_1h_input_tokens: 0` and the whole write under `ephemeral_5m_input_tokens`."

   Published Oct 4, 2026self-described: Streamed Anthropic calls with 1-hour prompt caching

   Counts toward

   • Overall signal
   • Double-Counted Spans & Wrong Token Totals: Raised as a concern

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   ddtrace 4.15.4, anthropic 1.11.0

6. Positive

   cflocke, engineer at codecentric AG (GitHub profile)

   fix(datadog): include DD_TAGS on LLM Observability spansGitHub pull request (LiteLLM)

   Adds global DD_TAGS to LiteLLM's Datadog LLM Observability spans so operators can attribute LLM usage to a cost center, verified live against Datadog.

   Published Sep 30, 2026self-described: LiteLLM proxy operators attributing LLM usage by cost center

   Counts toward

   • Overall signal
   • Auto-Instrumentation of Libraries: Positive
   • Where should a team keep its LLM and agent traces?: Keep them in Datadog

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   LiteLLM datadog_llm_observability callback

7. Positive

   Caleb Evans, engineer at Airbyte (GitHub profile)

   feat(mcp): default to Datadog when LLM Observability is enabledGitHub pull request (PyAirbyte)

   Makes PyAirbyte's hosted MCP server pick the native Datadog tracing backend by default whenever LLM Observability is enabled. Merged the same day.

   Published Sep 30, 2026self-described: PyAirbyte's hosted MCP server

   Counts toward

   • Overall signal
   • Auto-Instrumentation of Libraries: Positive
   • Where should a team keep its LLM and agent traces?: Keep them in Datadog

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   DD_LLMOBS_ENABLED tracing backend

8. Positive

   Minh Cung (cungminh2710), GitHub user

   Instrument AI SDK with Datadog LLM Observability and add content-redaction telemetryGitHub pull request

   Adds Datadog LLM Observability for server-side AI SDK calls with a fail-closed redaction policy, so only workflow names, token counts, and latency leave the application.

   Published Sep 24, 2026self-described: Server-side Vercel AI SDK calls with customer content

   Counts toward

   • Overall signal
   • Auto-Instrumentation of Libraries: Positive
   • Where should a team keep its LLM and agent traces?: Keep them in Datadog

   Published on

   Developer Communities

   Who

   Small product team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   Vercel AI SDK with Datadog LLM Observability

9. Mixed

   Zach Hammer, engineer at SeatGeek (GitHub profile)

   [BUG]: LLMObs: errors creating openai spans - missing span kind in span contextGitHub issue

   LLM spans for aborted or truncated streamed OpenAI Responses calls are dropped entirely, orphaning child spans and losing token metrics.

   Published Aug 18, 2026self-described: Streamed OpenAI Responses calls that end early

   Counts toward

   • Overall signal
   • Double-Counted Spans & Wrong Token Totals: Raised as a concern
   • Missing or Dropped Traces: Raised as a concern

   Published on

   Developer Communities

   Who

   Product or platform team

   Evidence

   Reported

   Applies to

   Current

   Observed on

   ddtrace 4.9.7, Python 3.13

10. Mixed

    Tomas Alabes, engineer at Source.ag (GitHub profile)

    LLMObs: langchain + modelcontextprotocol-sdk integrations double-capture MCP tool callsGitHub issue

    Every MCP tool call is captured twice, with the full tool payload on each span, which in the team's production app tripled storage for tool results and doubled error counts. Closed as completed in August 2026.

    "…every MCP tool call is captured **twice** in LLM Observability, with the full tool input/output payload stored on each span:…"

    Published Jul 17, 2026self-described: Production Node.js agent calling MCP tools through LangChain

    Counts toward, pending a recheck

    • Overall signal
    • Double-Counted Spans & Wrong Token Totals: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 6.3.0, @modelcontextprotocol/sdk 1.29.0

11. Positive

    QuickNick123, Reddit user, r/devops, self-described engineer at an S&P 500 company

    Comment on: My CTO forced a microservices migration for our tiny user baseReddit comment

    The company uses nearly every Datadog product, LLM observability included, and at its scale the Datadog bill is small next to cloud spend.

    Published Jul 16, 2026self-described: Large company running most Datadog products

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Organization-wide

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog LLM observability with APM, logs, and SIEM

12. Positive

    Kentaro Wakayama, engineer at Coder Society (GitHub profile)

    Add Datadog LLM Observability GenAI spansGitHub pull request

    Fans OTLP traces out to Datadog LLM Observability and emits OpenTelemetry GenAI 1.37+ attributes so Datadog classifies agent, chat, and tool spans correctly.

    Published Jul 9, 2026self-described: Veryfront agent runtime exporting over OTLP

    Counts toward

    • Overall signal
    • OpenTelemetry Intake: Positive
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenTelemetry GenAI 1.37+ attributes

13. Mixed

    Tomas Alabes, engineer at Source.ag (GitHub profile)

    LLMObs: LangChain + provider (anthropic) integrations emit two llm spans per call, double-counting costGitHub issue

    LangChain and the Anthropic SDK integration each emit an LLM span for the same model call, so Datadog's cost rollup overcounted the team's spend about 2.2 times. Datadog closed the issue as completed in June 2026.

    "…cost rollup sums both layers and overcounts spend (~2.2× in our case)…"

    Published Jun 16, 2026self-described: Node.js agent on LangChain over the Anthropic SDK

    Counts toward, pending a recheck

    • Overall signal
    • Double-Counted Spans & Wrong Token Totals: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 5.106, Node.js 22

14. Positive

    DifficultyIcy454, Reddit user, r/FinOps

    Comment on: GenAI is the first cost line my allocation playbook completely falls apart onReddit comment

    Traces every model call in Datadog LLM Observability to track model and workload efficiency, retry loops, and prompt cache hit rates, and calls it a big improvement for cost allocation.

    Published Jun 11, 2026self-described: FinOps team allocating GenAI spend

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog LLM Observability

15. Positive

    Gaofei Zhao (dippindots), cBioPortal contributor

    feat: add dd-trace for Datadog LLM ObservabilityGitHub pull request (cBioPortal)

    Adds dd-trace to cBioPortal's LibreChat image so Anthropic and OpenAI SDK calls are auto-instrumented into Datadog LLM Observability with no code changes.

    Published Jun 9, 2026self-described: cBioPortal's LibreChat deployment

    Counts toward

    • Overall signal
    • Auto-Instrumentation of Libraries: Positive
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    dd-trace loaded through NODE_OPTIONS

16. Mixed

    Rita Agafonova (rita-aga), GitHub user

    Disable Datadog OTel LLMObs duplicate conversionGitHub pull request

    Turns off Datadog's OpenTelemetry-to-LLMObs conversion in production so the direct LLMObs API export is the only source, after both paths produced duplicate spans.

    Published Jun 9, 2026self-described: Production agent exporting to Datadog through both OTLP and the LLMObs API

    Counts toward

    • Overall signal
    • OpenTelemetry Intake: Mixed or negative
    • Double-Counted Spans & Wrong Token Totals: Raised as a concern

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog OTel to LLMObs conversion

17. Mixed

    Troy Rhinehart, engineer at Attentive (GitHub profile)

    [BUG]: LLM Obs anthropic plugin mutates caller's options.messages arrayGitHub issue

    The Anthropic plugin inserted a system message into the caller's messages array while tagging a span, so retries and multi-turn loops got 400 errors from Anthropic. Fixed and closed three days later.

    "The LLM Obs plugin for `@anthropic-ai/sdk` mutates the caller's `options.messages` array when tagging the input on a span."

    Published Apr 25, 2026self-described: Anthropic tool runners and multi-turn loops in Node.js

    Counts toward, pending a recheck

    • Overall signal
    • Tracer Breaks the Application: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 5.98.0

18. Mixed

    nijave, Reddit user, r/AI_Agents

    Comment on: AI governance isn't failing because we lack regulationReddit comment

    The team uses Datadog's AI and LLM Observability mainly because it already runs Datadog, finds it helpful, and notes other tools may be better or cheaper.

    Published Apr 15, 2026self-described: Team already on Datadog for other monitoring

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive
    • Datadog Pricing & Bill Growth: Raised as a concern
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog AI and LLM Observability

19. Negative

    RavenPack's Bigdata.com engineering team

    70% of our AI agent failures had nothing to do with AIEngineering blog post, RavenPack on Medium

    A version mismatch between Datadog's tracer and LangChain crashed the team's agent in some edge cases, so the observability tooling built to debug failures was causing them.

    Published Jan 29, 2026self-described: Production financial research agent on LangChain

    Counts toward

    • Overall signal
    • Tracer Breaks the Application: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Engineering Blogs & Press

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    Datadog tracing with LangChain

20. Mixed

    Ian Kaplan, engineer at Unify (GitHub profile)

    [BUG]: LLM Observability should not throw an unhandled exception if the agent is unavailable.GitHub issue

    When the Datadog agent is unreachable, LLM Observability can throw an unhandled exception that crashes the whole Node.js process, which the author calls wrong for non-critical telemetry. Closed as completed in April 2026.

    "If the exception is not caught by a top-level uncaughtException handler, it crashes the entire Node.js process."

    Published Jan 28, 2026self-described: Node.js service with LLM Observability enabled

    Counts toward, pending a recheck

    • Overall signal
    • Tracer Breaks the Application: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 5.82, Node.js 24.9

21. Positive

    RasheedaDeals, Reddit user, r/devops

    Comment on: Anyone else finding observability for LLM workloads is a completely different beast?Reddit comment

    The team moved its LLM observability into Datadog because it already ran the main stack there, and correlating LLM traces with infrastructure metrics made debugging much easier.

    Published Jan 9, 2026self-described: Team that already ran Datadog for its main stack

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog LLM observability

22. Mixed

    Mustafa Akin, engineer at JumpCloud (GitHub profile)

    dd-trace-js issue 7146: ToolLoopAgent in AI SDK 6 and LLM Observability tracesGitHub issue

    Regular streamText and object calls were captured, but the AI SDK 6 ToolLoopAgent was not traced in LLM Observability. Closed as completed in March 2026.

    "However it is not being captured in LLM Observability."

    Published Dec 19, 2025self-described: Agent on AI SDK 6 ToolLoopAgent

    Counts toward, pending a recheck

    • Overall signal
    • Auto-Instrumentation of Libraries: Mixed or negative
    • Missing or Dropped Traces: Raised as a concern

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 5.81.0

23. Mixed

    SigmoidGrindset, Reddit user, r/LocalLLaMA

    Comment on: LiteLLM started breaking down for us past 300 RPS, what are folks using in prod?Reddit comment

    The team uses APM with propagated trace headers, log correlation, and LLMObs, and stays on Datadog because its platform is deeply integrated with it, even though another tool looked more advanced.

    Published Aug 6, 2025self-described: Team running an LLM gateway in production

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog APM, log correlation, and LLMObs

24. Mixed

    Bert Stewart, head of the global command center at Thomson Reuters, quoted by TechTarget

    Datadog AI agent observability, security seek to boost trustTrade press article, TechTarget

    Called LLM Observability Experiments valuable because building model comparison yourself takes a lot of infrastructure, and said unclear pricing of new Datadog AI features will decide whether they reach production.

    Published Jun 12, 2025self-described: Enterprise operations team assessing Datadog's new AI features

    Counts toward

    • Overall signal
    • Evaluations & Experiments: Positive
    • Datadog Pricing & Bill Growth: Raised as a concern

    Published on

    Engineering Blogs & Press

    Who

    Technical leadership

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    LLM Observability Experiments at DASH 2025

25. Positive

    Kosuke Oya, engineer at PLAID (GitHub profile)

    [FEATURE]: LLMObs - Instrumentation for mastraGitHub feature request

    Runs Mastra in production and asks for LLM Observability instrumentation so its traces show in an LLM-specific view, with plain APM as the fallback it considered.

    Published Apr 27, 2025self-described: Mastra agents running in production

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    dd-trace 5.49.0

26. Mixed

    lucasiscovici, GitHub user filing an issue

    [BUG]: LLM Observability Spans Not Appearing When Payload > 1MBGitHub issue

    When a payload to the LLM Observability intake exceeded 1 MB, no LLM spans appeared and the links between APM and LLM spans were missing, although the tracer and agent allow 5 MB. Datadog closed the issue as completed in August 2025.

    "…exceeds 1MB, the corresponding LLM Obs spans are not correctly ingested."

    Published Apr 23, 2025self-described: The team's LLM app sending spans to Datadog's EU intake

    Counts toward, pending a recheck

    • Overall signal
    • Missing or Dropped Traces: Raised as a concern

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    ddtrace 3.2.1 on Python 3.12, EU site

27. No opinion stated

    Amit, author on the NonBioS engineering blog

    The NonBioS LLM observability pickCompany engineering blog, NonBioS

    NonBioS listed Datadog among about a dozen LLM observability offerings, narrowed the choice to Langfuse and Helicone, and chose Langfuse, because it needed stable open-source software it could run on its own premises. It describes no use of Datadog, so it states no sentiment.

    "We narrowed down our choices to Langfuse and Helicone. Both appeared solid, but we ultimately chose Langfuse."

    Published Mar 5, 2025self-described: NonBioS choosing an observability tool for its AI software engineering agent

    Counts toward

    • Where should a team keep its LLM and agent traces?: Use another tool

    Published on

    Engineering Blogs & Press

    Who

    Small product team

    Evidence

    Reported

    Applies to

    Still applicable

28. Mixed

    TJ Koblentz (tjk), GitHub user filing an issue

    [BUG]: LLMObs agentLessEnabled does not mean no agent?GitHub issue

    With the Datadog agent reached over a Unix socket, dd-trace could not send LLM Observability span events, because the LLMObs writer expected an HTTP agent proxy. Datadog closed the issue as completed in September 2025.

    "Cannot send LLMObs events using our agent configuration."

    Published Feb 5, 2025self-described: The team's Node.js services on Amazon Linux, reaching the Datadog agent over a Unix socket

    Counts toward, pending a recheck

    • Overall signal
    • Missing or Dropped Traces: Raised as a concern

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 5.35.0 on Node.js 22.13.1

29. Mixed

    Ryuichi Watanabe (ryuichi1208), GitHub user filing an issue

    [BUG]: Traces Dropped After 5 Retries When Sending to Local Agent ProxyGitHub issue

    After an upgrade to ddtrace 2.19.2, LLM Observability traces failed to reach the local agent proxy and were dropped after 5 retries, while traces to other destinations arrived; 2.18 versions worked. Datadog closed the issue as completed in April 2025.

    "The traces fail to be sent to the local agent proxy (http://localhost:8126) after 5 retry attempts, leading to the traces being dropped."

    Published Jan 22, 2025self-described: The team's WebSocket service on gunicorn with uvicorn workers

    Counts toward, pending a recheck

    • Overall signal
    • Missing or Dropped Traces: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Product or platform team

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    ddtrace 2.19.2 on Python 3.11

30. Positive

    aiexplorations, X user who builds the Praval agent framework

    Reply on agent tracing at workPost on X

    Says that at their workplace Datadog, Arize, or a similar SaaS is the best option for agent traces.

    "Same here at my workplace. Datadog, Arize or similar SaaS are the best options there."

    Published Oct 22, 2026self-described: Agent tracing at the author's workplace

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive
    • Where should a team keep its LLM and agent traces?: Keep them in Datadog

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog for agent traces

31. Positive

    Ofek Shaked, AI engineer (X profile)

    Reply to @datadogdevs on Agent Observability traces over MCPPost on X

    Says giving Claude Code or Cursor access to Agent Observability traces through Datadog's MCP server is useful for debugging agent loops, and asks whether exact tool-call payloads are exposed.

    "Giving Claude Code/Cursor access to Agent Observability traces via Datadog’s MCP is useful for debugging agent loops."

    Published Oct 21, 2026self-described: Debugging agent loops with Claude Code and Cursor

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Agent Observability through Datadog's MCP server

32. Mixed

    pcbeingused333, GitHub user filing an issue

    LLM Observability: streamed Amazon Nova invoke_model calls report 0 tokensGitHub issue

    Streamed Amazon Nova calls produce LLM spans with 0 input, output, and total tokens, because the tracer reads counts that Nova always reports as zero.

    "The LLM span has `input_tokens: 0` and `output_tokens: 0`, while `metadata.usage` reported 4 and 20."

    Published Oct 17, 2026self-described: Amazon Bedrock Nova streaming through the botocore integration

    Counts toward

    • Overall signal
    • Double-Counted Spans & Wrong Token Totals: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    ddtrace 4.16.0

33. Mixed

    SierraJY, GitHub user filing an issue

    [BUG]: vllm integration raises AttributeError 'EngineCoreOutput' object has no attribute 'num_cached_tokens' with vLLM 0.20.0 and laterGitHub issue

    With the vLLM integration on, requests to vLLM 0.20.0 and later fail with HTTP 500 errors raised by the tracer, because a field it reads was removed upstream.

    Published Oct 3, 2026self-described: vLLM served through Single Step Instrumentation

    Counts toward

    • Overall signal
    • Tracer Breaks the Application: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    ddtrace 4.15.1; vLLM 0.27.1 and 0.28.0

34. Mixed

    Phillipe Silva (phillipehsantos), GitHub user filing an issue

    [BUG]: litellm integration: fallbacks parameter creates duplicate LLM spans (one call to two spans with identical usage)GitHub issue

    A LiteLLM call with the fallbacks parameter emits two LLM spans with identical usage when the primary model succeeds, so cost and token totals double, and so does the billed span count.

    "…produces TWO LLM-kind spans with identical usage on the success path."

    Published Sep 29, 2026self-described: LiteLLM acompletion calls with fallbacks

    Counts toward

    • Overall signal
    • Double-Counted Spans & Wrong Token Totals: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    ddtrace 4.15.2, litellm 1.97.0

35. Mixed

    Rafael G. Fernandes (rafagfe), GitHub user filing an issue

    [Bug]: Datadog LLM Obs embedding spans lack top-level model_name, causing Partial costGitHub issue (LiteLLM)

    Embedding spans from LiteLLM's Datadog callback lack a model name, so Datadog cannot price them and marks whole traces as partial cost, while LLM spans price correctly.

    Published Sep 17, 2026self-described: LiteLLM proxy sending to Datadog LLM Observability

    Counts toward

    • Overall signal
    • Double-Counted Spans & Wrong Token Totals: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LiteLLM proxy v1.90.3

36. Mixed

    Jesse Mills (jrmils89), GitHub user filing an issue

    MonitorFormulaAndFunctionEventsDataSource missing llm_observability: multi-query llm-observability alert monitors cannot be createdGitHub issue

    The Go API client, and the Terraform provider generated from it, reject the llm_observability data source, so multi-query LLM Observability alert monitors cannot be created as code.

    Published Sep 8, 2026self-described: Managing Datadog monitors with Terraform

    Counts toward

    • Overall signal
    • Tracer Breaks the Application: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    datadog-api-client-go master; terraform-provider-datadog 4.20.0

37. Mixed

    Jonathan van Westendorp, GitHub user filing an issue

    [BUG]: google_adk LLMObs agent span records every streaming partial as a separate messageGitHub issue

    In streaming mode the Google ADK integration records every partial chunk as its own message, so an answer streamed in 40 chunks shows as 41 messages and the span grows toward the size limit.

    "…an answer streamed in 40 chunks becomes 41 messages in the span, which makes the conversation view unreadable and inflates the span payload toward the event size limit."

    Published Aug 6, 2026self-described: Google ADK agents in streaming mode

    Counts toward

    • Overall signal
    • Double-Counted Spans & Wrong Token Totals: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    ddtrace 4.12.2, google-adk 2.5.0

38. Mixed

    Felipe Paiva (felipepaivawebmotors), GitHub user filing an issue

    MCP integration broken with mcp 2.0.0 and later: ModuleNotFoundError: No module named 'mcp.shared.session'GitHub issue

    The MCP integration fails to patch on mcp 2.0.0 because it imports a removed module, so MCP tool-call tracing is silently disabled.

    "…fails to patch when the `mcp` SDK is on `2.0.0+`…"

    Published Aug 6, 2026self-described: MCP servers on mcp 2.0.0 and fastmcp 4

    Counts toward

    • Overall signal
    • Missing or Dropped Traces: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    ddtrace 4.8.2 to 4.13.0rc3, mcp 2.0.0

39. Mixed

    Dan Aronson (AronsonDan), GitHub user filing an issue

    Anthropic AsyncMessageStream.get_final_message()/get_final_text() bypass TracedAsyncStream, so the span never finishesGitHub issue

    Consuming an Anthropic stream with get_final_message left the span open forever, so nothing reached APM or LLM Observability and no error was logged. Closed as completed in September 2026.

    Published Jul 21, 2026self-described: Async Anthropic streams consumed with get_final_message

    Counts toward, pending a recheck

    • Overall signal
    • Missing or Dropped Traces: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    ddtrace with the Anthropic integration

40. Negative

    vasilii-kurochkin, GitHub user filing an issue

    [BUG]: ai plugin leaks heap on every AI SDK call: LLMObs sub-plugin retains every tool() object foreverGitHub issue

    The LLMObs plugin for the Vercel AI SDK kept every tool object for the life of the process, leaking about 11 MB per chat stream and crashing the production process daily at its heap limit. Closed as completed in September 2026.

    Published Jul 9, 2026self-described: Production chat service on the Vercel AI SDK

    Counts toward, pending a recheck

    • Overall signal
    • Tracer Breaks the Application: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 5.80.0, Node.js 24.16.0

41. Positive

    jeann1977, Reddit user, r/devops

    LiteLLM is becoming much more than just an LLM gateway. Has anyone actually deployed its Datadog integration in production?Reddit post

    Tested LiteLLM's Datadog integration and found it covers tracing, LLM observability, cost, prompt redaction, and gateway versus model latency more completely than expected.

    Published Jun 29, 2026self-described: Testing LiteLLM's Datadog integration before production

    Counts toward

    • Overall signal
    • Auto-Instrumentation of Libraries: Positive

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LiteLLM Datadog integration in a test environment

42. Mixed

    Julien Perrin (jpr175), GitHub user filing an issue

    [BUG]: invoke_model_with_response_stream token metrics are all 0 in LLMObs tracesGitHub issue

    Streamed Claude Haiku 4.5 calls on Bedrock produced LLM spans with every token metric at 0, although the stream carries input and output counts, so cost estimates were skipped. The issue is still open.

    Published Jun 26, 2026self-described: Claude Haiku 4.5 on Amazon Bedrock, called with invoke_model_with_response_stream

    Counts toward

    • Overall signal
    • Double-Counted Spans & Wrong Token Totals: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    ddtrace 4.10.4 tracing Claude Haiku 4.5 through the Bedrock Runtime client

43. Mixed

    Daniel Polistchuck (dpolistwm), GitHub user filing an issue

    [BUG]: google_adk LLMObs drops agent span (missing span kind) when a Gemini response Part is unhandledGitHub issue

    The Google ADK integration intermittently dropped the agent span when Gemini returned an unhandled response part, orphaning child LLM spans. Closed as completed in July 2026.

    "The `google_adk` LLM Observability integration intermittently drops the agent/request span entirely…"

    Published Jun 22, 2026self-described: Google ADK agents on Gemini

    Counts toward, pending a recheck

    • Overall signal
    • Missing or Dropped Traces: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    ddtrace 4.10.5

44. Mixed

    Daniel Bernardes, GitHub user filing an issue

    [BUG]: Circular import in llmobs._experiment when anthropic is installed (ddtrace 4.6.0 and later)GitHub issue

    From ddtrace 4.6.0, a circular import in the experiments module stopped the OpenAI integration from enabling whenever anthropic was installed; 4.5.x was unaffected. Closed as completed in July 2026.

    "This creates a circular import when the `anthropic` package is installed and ddtrace auto-patches `openai` or `botocore` at startup:…"

    Published Apr 15, 2026self-described: Python app with anthropic installed and auto-patching on

    Counts toward, pending a recheck

    • Overall signal
    • Tracer Breaks the Application: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    ddtrace 4.6.0 to 4.7.1

45. Mixed

    Sirish Dwa (sdwa0), GitHub user filing a feature request

    [FEATURE]: Graceful SIGTERM handling and Configurable shutdown timeout for LLMObs writersGitHub feature request

    Buffered LLMObs spans are dropped when pods restart or deploy, leaving broken traces; the request for graceful shutdown was closed as not planned in September 2026.

    "Currently, LLMObs writers are silently killed on SIGTERM without flushing buffered spans."

    Published Mar 25, 2026self-described: Python services on pods that restart during deployments

    Counts toward

    • Overall signal
    • Missing or Dropped Traces: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    ddtrace llmobs writers

46. Mixed

    WowSoWholesome, Reddit user, r/LangChain

    Comment on: What do people use for tracing and observabilityReddit comment

    Uses LangSmith for tracing and is weighing whether to move to Datadog and its evaluations instead.

    Published Mar 19, 2026self-described: LangSmith user weighing Datadog's evals

    Counts toward

    • Overall signal
    • Evaluations & Experiments: Positive
    • Where should a team keep its LLM and agent traces?: Use another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    LangSmith, with Datadog under consideration

47. Mixed

    Daniel, DevOps engineer and author of the Signals & Codes newsletter

    Datadog AI Observability: What the Getting Started Guide Doesn't CoverPersonal newsletter post, Signals & Codes

    Generic APM spans did not appear in the AI Observability views, so the configured evaluators processed nothing until the code switched to native LLM Observability span types.

    Published Mar 19, 2026self-described: Moving an agentic LLM app from plain APM tracing to Agent Observability

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Mixed or negative
    • Evaluations & Experiments: Mixed or negative
    • Test each tracer release before trusting it

    Published on

    Engineering Blogs & Press

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog LLM Observability SDK and evaluators

48. Positive

    Daniel, DevOps engineer and author of the Signals & Codes newsletter

    Datadog's hallucination evaluatorPersonal newsletter post, Signals & Codes

    The hallucination evaluator needed only context and question variables annotated on an LLM span, which made hallucination detection simple to validate.

    Published Mar 16, 2026self-described: Testing hallucination detection without a full retrieval pipeline

    Counts toward

    • Overall signal
    • Evaluations & Experiments: Positive

    Published on

    Engineering Blogs & Press

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog hallucination evaluator

49. Mixed

    Isaac Sherrill (imsherrill), GitHub user filing an issue

    Anthropic streaming calls don't capture input/output content in LLM ObservabilityGitHub issue

    Auto-instrumentation captured inputs and outputs for non-streaming Anthropic calls, but streaming calls showed no content. Closed as completed in March 2026.

    "…streaming calls produce spans with empty input/output in LLM Observability."

    Published Feb 28, 2026self-described: Anthropic SDK streaming in Node.js

    Counts toward, pending a recheck

    • Overall signal
    • Auto-Instrumentation of Libraries: Positive
    • Missing or Dropped Traces: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 5.87.0

50. Positive

    kolaak, GitHub user filing a content request

    Add Datadog LLM Observability tutorial for AgentCore Runtime with Strands AgentsGitHub issue (AWS AgentCore)

    Proposes a tutorial for sending AgentCore and Strands Agents traces to Datadog LLM Observability, citing its trace views for prompts, responses, token usage, and model performance.

    Published Feb 13, 2026self-described: Amazon Bedrock AgentCore Runtime with Strands Agents

    Counts toward

    • Overall signal
    • OpenTelemetry Intake: Positive

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    OpenTelemetry GenAI 1.37+ spans

51. Negative

    platypii, Hacker News commenter

    Comment on: Ask HN: Where are you keeping your LLM logs?Hacker News comment

    Wary of putting LLM logs in Datadog after past runaway Datadog bills, since the LLM logs are much larger than the APM logs.

    Published Jan 9, 2026self-described: Choosing where to store LLM logs

    Counts toward

    • Overall signal
    • Datadog Pricing & Bill Growth: Raised as a concern
    • Where should a team keep its LLM and agent traces?: Use another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Current

    Observed on

    Datadog for LLM logs

52. Negative

    Dmitry (vilmar-hillow), GitHub user filing an issue

    [BUG]: ddtrace breaks applications with OpenAI usageGitHub issue

    ddtrace 4.0.0 crashed application startup when an OpenAI client used a function as its API key, even with LLM Observability not enabled. Fixed two days later.

    "…even with no LLM observability explicitly enabled."

    Published Dec 10, 2025self-described: Python app creating an OpenAI client with Azure identity

    Counts toward, pending a recheck

    • Overall signal
    • Tracer Breaks the Application: Raised as a concern
    • Test each tracer release before trusting it

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    ddtrace 4.0.0, openai 2.9.0

53. Mixed

    Scott Prue (prescottprue), GitHub user filing an issue

    [BUG]: LLMObs for ai-sdk not capturing traceGitHub issue

    LLM Observability captured no traces for Vercel AI SDK calls in a Next.js app while other dd-trace traces arrived. Closed as completed in July 2026.

    "…but I'm not seeing LLM traces be captured. Other traces are captured just fine so setup of dd-trace is working."

    Published Oct 22, 2025self-described: Next.js app on the Vercel AI SDK

    Counts toward, pending a recheck

    • Overall signal
    • Auto-Instrumentation of Libraries: Mixed or negative
    • Missing or Dropped Traces: Raised as a concern

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Needs a recheck

    Observed on

    dd-trace 5.63.0, Node.js 22.20.0

54. Mixed

    Pictti, Reddit user, r/learnmachinelearning

    Datadog LLM observability alternativesReddit post

    Finds Datadog solid for dashboards and infrastructure monitoring but a generalist that is not built for LLM work, with pricing that can creep up, and asks for alternatives.

    Published Apr 3, 2025self-described: Looking for an alternative after using Datadog for LLM observability

    Counts toward

    • Overall signal
    • One Trace With APM and Infrastructure: Positive
    • Datadog Pricing & Bill Growth: Raised as a concern
    • Where should a team keep its LLM and agent traces?: Use another tool

    Published on

    Developer Communities

    Who

    Individual builder

    Evidence

    Reported

    Applies to

    Still applicable

    Observed on

    Datadog LLM Observability