Every source GILD analyzed, newest first. Each says who the source is, how it knows, whether it still applies and which findings it counts toward. Open a count to see only its sources.
Source
55 sources in Teams and technical leaders, with product documentation Showing all 73 sources
Negative
Braintrust ruled out because self-hosting is Enterprise onlyReddit post
Shortlisted Braintrust, Arize Phoenix and Orq AI to replace Langfuse, and ruled Braintrust out because its self-hosting needs an Enterprise plan the team could not justify at its stage.
Positive
Living Evals: an eval suite that grows from productionEngineering blog post, Rox
Rox has Devin pull about 350 conversations a week from Braintrust traces and turn failures into evals. The first sweep covered 67 behaviors where 142 hand-written evals had covered 34, and production bugs now show up as failing evals in the weekly pull request.
"Bugs surface before anyone reports them. When Rox gets something wrong in production, it shows up as a failing eval in that week's pull request."
Not counted
Rox published the post in partnership with Cognition and Braintrust. Kept in the record; it takes part in no figure.
Mixed
feat(observability): Braintrust refuse-to-enable until compliance affirmed (audit C2/H7)GitHub pull request
Made Braintrust tracing default-off until a signed DPA, 30-day retention and a tested deletion cascade are confirmed, since it sends member Q&A and user IDs to a US processor.
Positive
feat(evals): link Braintrust experiments to their CI run and the PRGitHub pull request
Linked each Braintrust experiment to the GitHub Actions run and pull requests that produced it, shown on an experiment running Codex on GPT-6 Luna. The team can go from a trace to its CI logs and from a run to its source.
Mixed
Remove the Braintrust integrationGitHub pull request
Prepared, but did not merge, a change removing Braintrust in case the team drops it, listing the losses: all LLM tracing, prompt loading and the prompt A/B eval harness.
Counts toward
Mixed
Question: why does a self-hosted deploy's daemon contact www.braintrust.dev on cold start?GitHub issue
On their self-hosted deployment, the bt daemon contacted the public braintrust.dev domain on cold start, which their network and security review did not expect.
Mixed
chore(eval-tests): upgrade eval deps, weekly evals, standalone Braintrust CLI CLOUDP-450203GitHub pull request
Proposes moving the MongoDB MCP server's Braintrust evals from every push to weekly runs plus labeled pull requests. Also proposes calling the standalone Braintrust CLI by its direct path because the SDK bundles a conflicting bt binary.
Negative
Remove Braintrust reporting and implicit judge gateway integrationGitHub issue
Removed Braintrust reporting and its implicit judge gateway from Ultrafuzz after cutting paid CI, so evaluations run locally without a Braintrust account.
Positive
feat: trace simulations to Braintrust instead of the GigaFlow SDKGitHub pull request
Proposes, in an open pull request, replacing the team's own SDK with Braintrust tracing for agent simulations, verified on 9 runs and 210 spans whose rewards matched when queried back.
Positive
The same evals score production traffic, not only experimentsReddit comment
Disputes that Braintrust is only for experiments: the team scores production traffic with the same evals, which catches quality changes after a release, and that link is why it kept Braintrust.
Mixed
The Agno integration logged the full agent object, including the provider API key, into tool-span metadata; the team found a real OpenAI key in its own project logs.
Positive
Our PM runs prompt changes against the dataset without an engineerReddit comment
Says Braintrust has worked for the team because, once evals are set up, its product manager can change a prompt, run it against the dataset and review the regressions in the UI without an engineer.
Mixed
The Flue adapter logged the whole conversation on every turn, so the SDK's background logger grew quadratically and crash-looped their production agent on Cloudflare Workers.
Positive
The best evals we've written came from production failuresReddit post
The team manages its traces and runs its regression tests in Braintrust, and every new model must pass the cases drawn from production failures before the averages are trusted. Curating which failures join the suite is still hard.
Positive
feat(evals): bump braintrust to 3.x; first-party BraintrustSpanProcessor OTEL sinkGitHub pull request
Proposes upgrading Stagehand's evals from Braintrust 0.4 to 3.x, because the old pin blocked Braintrust's first-party OpenTelemetry span processor, which captures AI SDK spans better than a generic OTLP exporter.
Positive
Auto-evals: Braintrust Loop, Arize Alyx and LangSmith on the same 100 tracesIndependent evaluation, Parlance Labs blog
Parlance Labs masked hand labels on 100 production traces and asked each platform's agent to find the failures. Braintrust Loop had the highest recall of the three and ran from upload to export in one pass; LangSmith Engine surfaced too few issues to score.
"Braintrust had the highest recall of the three platforms and was easy to use. We could upload our traces, perform the auto-evaluation, and export its findings seamlessly."
Not counted
Its author already has two counted records on this product. Kept in the record; it takes part in no figure.
No opinion stated
How Braintrust uses AI agents, evals, and CI to ship better softwarePodcast episode, How I AI
Braintrust's CEO describes treating evals as the product spec, writing scorers so an agent can improve a prompt in a playground, and turning a designer's taste into a repeatable eval.
Not counted
Its guest is Braintrust's founder and CEO. Kept in the record; it takes part in no figure.
Positive
Combine Skills and MCP to Close the Context GapConference talk with transcript, AI Engineer Europe 2026
Supabase graded task completeness on Braintrust while testing coding agents with and without its MCP server and skill, and Rodrigues calls Braintrust a very cool product.
Positive
Shipping complex AI applications: Braintrust & TrainlineConference talk (co-presented with Braintrust), AI Engineer Europe 2026 (YouTube)
Trainline evaluates its travel assistant offline and online before model and feature changes, and its product managers inspect traces themselves and skip asking engineers for logs.
Not counted
It was a workshop co-presented with Braintrust. Kept in the record; it takes part in no figure.
Positive
Replacing 12K LoC with a 200 LoC SkillConference talk with transcript, AI Engineer Europe 2026
Gomes wrote evals for Cursor's worktree skill in Braintrust, running tasks through the headless Cursor CLI with scorers that check where the agent made changes. Gomes reports that writing them was easy for someone early in writing evals.
Positive
feat(observability): replace Arize with Braintrust OTel + exact wire logging via Finch telemetryGitHub issue
Plans to replace Arize with Braintrust over OTLP, mapping spans to the OpenTelemetry GenAI conventions and logging the exact payload sent to the LLM.
Mixed
Comment on: Ask HN: How are people doing AI evals these days?Hacker News comment
Says the value came from running evals against a dataset and different models; Braintrust makes that easy, but the commenter would now vibe-code the same functionality themselves.
Positive
Comment on: Ask HN: How are people doing AI evals these days?Hacker News comment
Rates Braintrust highly: datasets were easy to set up, and evals showed whether prompts were working, which mattered most for classification that has to stay consistent.
Mixed
Regular 500 responses from /logs3GitHub issue
Their eval pipeline regularly hit 500 errors and failed fetches from the Braintrust log ingestion endpoint while running evals.
Mixed
Significant latency regression between TypeScript SDKs v0.4.10 and v1.0.0GitHub issue
Upgrading the TypeScript SDK from 0.4.10 to 1.0.0 slowed eval runs of a few hundred tasks from 1.5 to 5 minutes, traced to a sequential log upload loop.
Positive
chore(sc-41159): Add Braintrust evaluation cronjob infrastructureGitHub pull request
Proposes, in an open pull request, production jobs that back up Braintrust logs to Google Cloud Storage weekly and use Claude to tag logs into Braintrust datasets daily.
Positive
Lessons from Building AI Agents for Financial ServicesPersonal blog post, nicolasbustamante.com
Fintool tracks experiments in Braintrust and evaluates every model or prompt change against about 2,000 test cases. A pull request is blocked when the eval score drops more than 5%, and auto-filed production errors link to Braintrust traces.
"We auto-file GitHub issues for production errors. Error happens, issue gets created with full context: conversation ID, user info, traceback, links to Braintrust traces and Temporal workflows."
Mixed
Remote Evals (examples and support)GitHub issue
As a Braintrust customer, their org wanted remote evals in its Rails app; the Ruby SDK lacked them, so the author built support modeled on the Python SDK and offered it upstream.
Positive
feat: implement remote Braintrust evals for all HolmesGPT testsGitHub pull request
Proposes, in an open pull request, exposing HolmesGPT's roughly 53 pytest LLM evals as remote Braintrust evals so the team can run them from the playground.
Positive
Migrate to Braintrust-native eval frameworkGitHub pull request
Replaced about 2,000 lines of custom eval code with Braintrust's Eval() for 28 routing and attachment test cases, for stored and versioned results, side-by-side experiment comparison, and regression tracking.
Mixed
Node.js SDK has broken/bad bundlingGitHub issue
The Node.js SDK bundle published to npm was broken because OpenTelemetry modules were not marked external, so the team patched it locally.
Positive
BOLT - How Mura wrote an in-house LLM Eval FrameworkEngineering blog post, James's Newsletter (Substack)
After trialing Vellum, W&B Weave and Patronus, Mura built its BOLT eval CLI on Braintrust Datasets and Experiments, praising experiment visualization while owning its domain logic.
"We ended up using Braintrust’s Datasets and Experiments functionality as some of the core primitives in BOLT."
Positive
A practical blueprint for evaluating conversational AI at scaleEngineering blog post, Dropbox Tech blog
Dash adopted Braintrust once scattered eval artifacts became unmanageable. Every pull request runs about 150 judged queries in under 10 minutes, merges are blocked on any red-line miss, and one click opens the trace behind a score.
"…we turned to internal datasets by collecting production logs from Dropbox employees dogfooding Dash."
Positive
Evals Are Not Unit TestsConference talk with transcript, AI Engineer World's Fair 2025
Recommends adding evals to CI because Braintrust reports each change's improvements and regressions across the whole dataset, and says the v0 team runs its evals at least daily.
No opinion stated
Turning Fails into Features: Zapier's Hard-Won Eval LessonsConference talk with transcript, AI Engineer World's Fair 2025
Reading one Braintrust experiment screen at a time made aggregate regressions hard to explain. The team pointed a reasoning model at two runs through Braintrust MCP to compare Gemini Pro with Claude before deploying.
Positive
How to build world-class AI products: Sarah Sachs (AI lead @ Notion) & Carlos Esteban (Braintrust)Conference talk (co-presented with Braintrust), AI Engineer World's Fair 2025 (YouTube)
Notion's AI team spends about 10% of its time prompting and 90% on evals and inspecting usage in Braintrust. Specialists curate datasets and write per-example judges, though evals do not run on every CI change.
"…ultimately we spend maybe 10% of our time prompting and 90% of our time looking at evals and iterating on our evals and looking at our usage and brain trust and that I believe is the right balance of work in order to know that…"
Not counted
It was co-presented with Braintrust staff. Kept in the record; it takes part in no figure.
Positive
The Web Browser Is All You NeedConference talk with transcript, AI Engineer World's Fair 2025
Klein distrusts public web-agent benchmarks and says Browserbase runs its own evals with Braintrust's support, which lets the team pick the best model for each web task or website.
Negative
Ask HN: What tools are you using for AI evals? Everything feels half-bakedHacker News post
Evaluated Braintrust among several eval tools and found it an interesting approach that feels early-stage, with sparse documentation and limited integrations; the team still relies on custom scripts.
Positive
test(max): Migrate insight deepevals to BraintrustGitHub pull request
Moved PostHog's Max AI insight evals from DeepEval to Braintrust and added LLM classifiers that score whether a generated plan answers the question and whether the final query follows the plan.
Positive
Review of BraintrustProduct Hunt review
Says Braintrust evals transformed AI development at Airtable, raising confidence within weeks and giving the team the feedback loop to ship reliable AI features faster.
No opinion stated
Unit-testing AI agents in the browserEngineering blog post, Browserbase blog
Stagehand's act, extract, and observe evals guard its pull requests against regressions. The team runs about 10 tests per category three times per model and published the Braintrust runs with chat traces for every eval.
Counts toward
No opinion stated
Arize, Langfuse and Braintrust all intimidating at firstPost on X
Tried Arize, Langfuse, Braintrust and others and found almost all of them intimidating at the start; Langfuse was the easiest to get going.
"I have tried out all the existing tools like Arize, langfuse, braintrust etc and almost all of them are quite intimidating to use in the beginning (langfuse was the easier one to get started)."
"I have tried out all the existing tools like Arize, langfuse, braintrust etc and almost all of them are quite intimidating to use in the beginning (langfuse was the easier one to get started)."
Negative
Used Braintrust and did not enjoy it, built custom evals insteadReddit comment
Says they used Braintrust and did not enjoy the experience, and have been building their own offline observability and evals instead.
Positive
Braintrust LLM observability: spans for every model call, off by defaultGitHub pull request
Added Braintrust spans for every model call, off by default, because cost logs showed what each call cost but not what the model said.
Mixed
AI Evals: Everything You Need to KnowPersonal blog post, hamel.dev
Names Braintrust among the eval vendors Husain meets most, sees little feature difference between them, often builds custom tools on top, and warns vendor prompt management adds indirection.
Positive
feat(homelab): tail-sample LLM traces into Braintrust and repoint birmelGitHub pull request
Rolled out OpenTelemetry export that forwards selected whole LLM traces from their homelab gateway into per-service Braintrust projects, with an allowlist as a byte-budget guard.
Mixed
Eval::Runner's per-case force_flush adds 10-25s+ per case even when no scorer/classifier uses trace:GitHub issue
The Ruby eval runner flushes spans after every case, which the author measured at 10 to 25+ seconds per case on a 139-case suite that never used trace data.
Negative
Migrate ThoughtForm observability to LangfuseGitHub pull request
Replaced Braintrust with Langfuse and OpenTelemetry so tracing, evals and prompt iteration sit in one place, with prompt fallbacks still reviewed in GitHub.
Positive
Back model benchmark with Braintrust, drop in-house scoringGitHub pull request
Replaced an in-house benchmark scoring engine with Braintrust Eval, keeping only the CLI-agent adapters and handing scoring, experiments and comparison to Braintrust.
Negative
Retire Braintrust for LangfuseGitHub pull request
Retired Braintrust for Langfuse across evals, telemetry, scripts and workflows, then reran nine eval suites against Langfuse.
Negative
feat(photo-curation): local eval store + runner (replace Braintrust write path)GitHub issue
After exhausting Braintrust write credits on a photo-judge eval, the author decided to replace Braintrust entirely with a local SQLite store and runner.
Mixed
Standalone-api / brainstore v2.1.1 ships starlette < 1.0.1 (CVE-2026-48710)GitHub issue
The self-hosted data-plane images pinned by the latest Helm chart still shipped a vulnerable starlette version a week after the upstream fix.
Mixed
setup_adk() does not trace tool calls in nested sub-agentsGitHub issue
Tool calls inside nested Google ADK sub-agents were not traced, so experiment summaries showed zero tool calls even when tools ran.
Positive
Comment on: Ask HN: Where are you keeping your LLM logs?Hacker News comment
Points to instrumenting model SDKs and sending traces over OpenTelemetry to a provider such as Braintrust, the one the commenter knows, so a team can skip building LLM logging itself.
Positive
LangFuse Telemetry not working, switch to BraintrustGitHub issue
With Langfuse tracing broken in both their CLI agent and Next.js app, the author concluded after research to switch to Braintrust, which has a free tier and logs local and deployed runs.
Mixed
Ask HN: What do u use for agent/agentic evals?Hacker News post
Looking at MLflow and Braintrust for agent evals, finds it hard to compare across agent versions or A/B test agents and MCP tools.
Positive
Selecting The Right AI Evals ToolIndependent evaluation, hamel.dev
A panel of eval practitioners rated Braintrust well for its clean UI, readable trace view, and human review workflow, and questioned its Loop auto-scorer and proprietary BTQL query language.
Positive
Evals your LLM driven Agents (Experiments and Lessons Learned) with BraintrustReddit post
Ran weekend experiments on Braintrust with an email-management agent, then turned its logs into automated evaluations that catch regressions and re-run selected cases, and shared the code.
Mixed
If you log a span instance, your API key is included in the logGitHub issue
Found that logging a span object writes the Braintrust API key into the log through the span's internal state.
Pages from Braintrust, used to check facts for every group.
Documents Starter, Pro, and Enterprise: included processed data, scores, retention, and model credits, on-demand rates, and which features and deployment options each plan includes.
States that processed data is measured at ingestion, so deleting data does not lower it. It also states there are no hard spending limits and that plans bought before March 16, 2026 keep legacy features.
Documents the four usage dimensions, retention storage billed on the highest daily amount retained, and spend alerts by email or Slack.
Documents 14-day retention on Starter, 30 days extendable to 180 on Pro, up to 365 days on Enterprise, and Enterprise retention automations that delete data.
Compares SaaS, BYOC, self-hosted, and non-standard self-hosted deployments, and states that BYOC and self-hosting require Enterprise.
Documents the split between the Braintrust-run control plane and the data plane that holds traces, datasets, and prompts, and where each kind of data is stored.
Documents SSO, RBAC, encryption, isolated code execution, BYOC and self-hosted data residency, SOC 2 Type II, BAAs, and DPAs.
Documents SDK masking functions and span customizers that remove values before upload, and ingestion redaction for a limited set of customers.
Documents experiments as immutable records of an evaluation run, created in code, in the UI, or in CI, and compared across runs.
Documents the GitHub Action and bt eval command that run evaluations on pull requests and report scores back.
Documents online scoring rules that run scorers on production traces in the background, on a percentage of matching traces set per rule.
Documents automatic instrumentation and wrappers for AI providers and frameworks across the Braintrust SDKs.
Documents sending OpenTelemetry traces and logs to Braintrust through an OTLP exporter, a span processor, or a Collector.
Documents the Gateway, a unified API to AI providers, free to use and without an SLA on the hosted endpoint, which is not yet generally available.
Source index
Every source GILD read, with the original address and the context used to count it. Open a finding to filter the full cards above. On smaller screens the index lists each source once by name, cohort and date; its full record above carries the address, context and findings.
| Source | Original URL | Author and context | Who | Published | Review date | Applies | Findings |
|---|---|---|---|---|---|---|---|
| Braintrust ruled out because self-hosting is Enterprise onlyID sf-6 | Developer Communities | www.reddit.com/ | CollectionPure7752, Reddit user, r/mlopsself-described: An early-stage team replacing Langfuse after letting non-engineers edit prompts | Small product team | Published Oct 22, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Living Evals: an eval suite that grows from productionID sf-2 | Engineering Blogs & Reviews | www.rox.com/ | Brian Xu, engineer on the Agents team at Roxself-described: Rox's Agents team, with Devin writing evals from Braintrust production traces each week | Product or platform team | Published Oct 14, 2026 | Assessment reviewed Oct 30, 2026 | Current | Not counted. Rox published the post in partnership with Cognition and Braintrust. |
| Arize, Langfuse and Braintrust all intimidating at firstID sf-3 | Developer Communities | x.com/ | Ganesh Iyer, AI payments builder and creator of the Nura family assistantself-described: Tried the observability tools for his own agents | Individual builder | Published Oct 13, 2026 | Assessment reviewed Oct 30, 2026 | Current | Listed, no finding position |
| feat(observability): Braintrust refuse-to-enable until compliance affirmed (audit C2/H7)ID dc-36 | Developer Communities | github.com/ | mdenner1234, Developer (pull request author)self-described: Tech Fleet network, member Q&A assistant | Product or platform team | Published Oct 10, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| feat(evals): link Braintrust experiments to their CI run and the PRID dc-35 | Developer Communities | github.com/ | Matt Rossman (mattrossman), AI tooling at Supabaseself-described: Supabase evals repository, experiments run from GitHub Actions | Product or platform team | Published Oct 8, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Remove the Braintrust integrationID dc-34 | Developer Communities | github.com/ | dealvz, Developer (pull request author)self-described: GoodParty.org omni platform: chat, briefings, PMF runner | Product or platform team | Published Oct 5, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Used Braintrust and did not enjoy it, built custom evals insteadID sf-5 | Developer Communities | www.reddit.com/ | Omrinachmani, Reddit user, r/LangChainself-described: A developer running custom offline observability and evals after trying Braintrust | Individual builder | Published Sep 27, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Question: why does a self-hosted deploy's daemon contact www.braintrust.dev on cold start?ID dc-32 | Developer Communities | github.com/ | LuizNascimentoTR, Developer (issue author)self-described: Team running a self-hosted Braintrust deployment | Small product team | Published Sep 25, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| chore(eval-tests): upgrade eval deps, weekly evals, standalone Braintrust CLI CLOUDP-450203ID dc-33 | Developer Communities | github.com/ | Nima Taheri (nima-taheri-mongodb), Contributor to MongoDB's MCP serverself-described: MongoDB MCP server, eval tests in CI | Product or platform team | Published Sep 25, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Remove Braintrust reporting and implicit judge gateway integrationID dc-31 | Developer Communities | github.com/ | aviggiano, Developer (issue author)self-described: Ultrafuzz, Monad developers' fuzzing project | Product or platform team | Published Sep 22, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Braintrust LLM observability: spans for every model call, off by defaultID dc-30 | Developer Communities | github.com/ | punitarani, Developer (pull request author)self-described: Personal research project calling models through OpenRouter | Individual builder | Published Sep 21, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| AI Evals: Everything You Need to KnowID ir-03 | Engineering Blogs & Reviews | hamel.dev/ | Hamel Husain, AI evals consultant and course instructorself-described: Advising client teams on eval tooling and vendor selection | Individual builder | Published Sep 18, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| feat: trace simulations to Braintrust instead of the GigaFlow SDKID dc-29 | Developer Communities | github.com/ | giga-james, Developer (pull request author)self-described: GigaFlow, tau2 agent simulations | Small product team | Published Sep 17, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| feat(homelab): tail-sample LLM traces into Braintrust and repoint birmelID dc-28 | Developer Communities | github.com/ | shepherdjerred, Developer (pull request author)self-described: Homelab services with an OpenTelemetry gateway | Individual builder | Published Sep 10, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| The same evals score production traffic, not only experimentsID sf-8 | Developer Communities | www.reddit.com/ | Cheap-Ad-2360, Reddit user, r/LangChainself-described: A team scoring production traffic with its release evals | Product or platform team | Published Sep 9, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| FunctionCall tool-span instrumentation for Agno leaks the full Agent/Team object (incl. model.api_key) into span metadataID dc-27 | Developer Communities | github.com/ | paulosuzart, Developer (issue author)self-described: Team tracing Agno agents with MCP tools | Small product team | Published Sep 3, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Eval::Runner's per-case force_flush adds 10-25s+ per case even when no scorer/classifier uses trace:ID dc-26 | Developer Communities | github.com/ | satanas, Developer (issue author)self-described: Ruby eval suite | Individual builder | Published Aug 19, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Our PM runs prompt changes against the dataset without an engineerID sf-7 | Developer Communities | www.reddit.com/ | Low_Dust_9566, Reddit user, r/mlopsself-described: A team whose product manager changes prompts and reviews regressions | Product or platform team | Published Aug 19, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Flue adapter: logging the full conversation-so-far per turn makes background-logger retention O(turns²), OOMs 128 MB Workers isolatesID dc-25 | Developer Communities | github.com/ | dknecht, Developer (issue author)self-described: Production agent on Cloudflare Workers Durable Objects | Small product team | Published Aug 11, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Migrate ThoughtForm observability to LangfuseID dc-24 | Developer Communities | github.com/ | adambelton, Developer (pull request author)self-described: ThoughtForm, a personal app with five runtime prompts | Individual builder | Published Aug 7, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| The best evals we've written came from production failuresID sf-4 | Developer Communities | www.reddit.com/ | Disastrous_Injury561, Reddit user, r/mlopsself-described: A team that turns odd production traces into regression tests before each model upgrade | Product or platform team | Published Aug 5, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Back model benchmark with Braintrust, drop in-house scoringID dc-23 | Developer Communities | github.com/ | time-attack, Developer (pull request author)self-described: GStack model benchmark driving Claude, Codex, and Gemini CLI agents | Individual builder | Published Jul 21, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| feat(evals): bump braintrust to 3.x; first-party BraintrustSpanProcessor OTEL sinkID dc-22 | Developer Communities | github.com/ | Miguel (miguelg719), Contributor to Browserbase's Stagehand evalsself-described: Browserbase Stagehand, browser-agent evals | Product or platform team | Published Jul 13, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Auto-evals: Braintrust Loop, Arize Alyx and LangSmith on the same 100 tracesID sf-1 | Engineering Blogs & Reviews | parlance-labs.com/ | Antaripa Saha and Hamel Husain, Parlance Labsself-described: 100 hand-labelled traces from a production apartment-leasing assistant | Small product team | Published Jul 11, 2026 | Assessment reviewed Oct 30, 2026 | Current | Not counted. Its author already has two counted records on this product. |
| Retire Braintrust for LangfuseID dc-21 | Developer Communities | github.com/ | createsomethingtoday, Developer (pull request author)self-described: Create Something monorepo: evals, telemetry, prompts, and MCP services | Individual builder | Published Jul 6, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| How Braintrust uses AI agents, evals, and CI to ship better softwareID sf-9 | Podcasts, Talks & Videos | podcasts.apple.com/ | Ankur Goyal, founder and CEO of Braintrust, on How I AI with Claire Voself-described: Braintrust's own engineering team, using coding agents, evals and CI | Technical leadership | Published Jun 15, 2026 | Assessment reviewed Oct 30, 2026 | Current | Not counted. Its guest is Braintrust's founder and CEO. |
| feat(photo-curation): local eval store + runner (replace Braintrust write path)ID dc-20 | Developer Communities | github.com/ | julianken, Developer (issue author)self-described: Photo-judge eval for a personal bird-sighting system | Individual builder | Published Jun 12, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Standalone-api / brainstore v2.1.1 ships starlette < 1.0.1 (CVE-2026-48710)ID dc-19 | Developer Communities | github.com/ | joel-vaz, Developer (issue author)self-described: Self-hosted data plane on the Helm chart | Individual builder | Published May 28, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Combine Skills and MCP to Close the Context GapID l2-04 | Podcasts, Talks & Videos | ai.engineer/ | Pedro Rodrigues, AI tooling engineer, SupabaseSupabase testing coding agents with its MCP server and Agent Skills | Product or platform team | Published May 15, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Shipping complex AI applications: Braintrust & TrainlineID tc-02 | Podcasts, Talks & Videos | www.youtube.com/ | Oussama Hafferssas and Mayank Soni, Engineers at Trainlineself-described: Trainline travel assistant: refunds, train changes, and handover to support | Product or platform team | Published May 1, 2026 | Assessment reviewed Oct 30, 2026 | Current | Not counted. It was a workshop co-presented with Braintrust. |
| Replacing 12K LoC with a 200 LoC SkillID l2-03 | Podcasts, Talks & Videos | ai.engineer/ | David Gomes, Engineer on coding agents, CursorCursor team replacing its worktree feature with an agent skill | Product or platform team | Published Apr 30, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| feat(observability): replace Arize with Braintrust OTel + exact wire logging via Finch telemetryID dc-18 | Developer Communities | github.com/ | BlueHotDog, Developer (issue author)self-described: Frontman, agent platform on Elixir | Small product team | Published Apr 10, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| setup_adk() does not trace tool calls in nested sub-agentsID dc-17 | Developer Communities | github.com/ | sm-stripe, Developer (issue author)self-described: Google ADK pipeline with nested sub-agents | Individual builder | Published Mar 24, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Comment on: Ask HN: How are people doing AI evals these days?ID dc-16 | Developer Communities | news.ycombinator.com/ | mierz00, Hacker News commenterself-described: Team using an LLM for classification | Small product team | Published Mar 17, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Comment on: Ask HN: How are people doing AI evals these days?ID dc-15 | Developer Communities | news.ycombinator.com/ | mierz00, Hacker News commenterself-described: Team using an LLM for classification | Small product team | Published Mar 11, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Regular 500 responses from /logs3ID dc-14 | Developer Communities | github.com/ | CodingCanuck, Developer (issue author)self-described: Team eval pipeline on the TypeScript SDK | Small product team | Published Feb 27, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Significant latency regression between TypeScript SDKs v0.4.10 and v1.0.0ID dc-13 | Developer Communities | github.com/ | CodingCanuck, Developer (issue author)self-described: Team eval pipeline on the TypeScript SDK | Small product team | Published Feb 19, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| chore(sc-41159): Add Braintrust evaluation cronjob infrastructureID dc-12 | Developer Communities | github.com/ | yodem, Developer (pull request author)self-described: Sefaria, AI chatbot in production | Product or platform team | Published Feb 4, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Lessons from Building AI Agents for Financial ServicesID l2-02 | Engineering Blogs & Reviews | nicolasbustamante.com/ | Nicolas Bustamante, Founder of FintoolFintool, AI agents for financial research and equity analysis | Technical leadership | Published Jan 24, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Remote Evals (examples and support)ID dc-11 | Developer Communities | github.com/ | gabe-kent, Developer at a Braintrust customer (issue author)self-described: Customer organization with a main Rails app | Small product team | Published Jan 13, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Comment on: Ask HN: Where are you keeping your LLM logs?ID dc-10 | Developer Communities | news.ycombinator.com/ | barbazoo, Hacker News commenterself-described: Hacker News thread: Where are you keeping your LLM logs? | Individual builder | Published Jan 9, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| feat: implement remote Braintrust evals for all HolmesGPT testsID dc-09 | Developer Communities | github.com/ | aantn, Maintainer (pull request author)self-described: HolmesGPT, an open-source SRE agent, with about 53 LLM evals | Product or platform team | Published Jan 3, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| LangFuse Telemetry not working, switch to BraintrustID dc-08 | Developer Communities | github.com/ | fyang0507, Developer (issue author)self-described: Personal Notion agent, CLI and Next.js app | Individual builder | Published Jan 1, 2026 | Assessment reviewed Oct 30, 2026 | Current | |
| Migrate to Braintrust-native eval frameworkID dc-07 | Developer Communities | github.com/ | Nick Sullivan (TechNickAI), Builder at Carmenta Collectiveself-described: Carmenta, evals for model routing and file handling | Product or platform team | Published Dec 11, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| Ask HN: What do u use for agent/agentic evals?ID dc-06 | Developer Communities | news.ycombinator.com/ | hhthrowaway1230, Hacker News commenterself-described: Evaluating agents | Individual builder | Published Nov 25, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| Node.js SDK has broken/bad bundlingID dc-05 | Developer Communities | github.com/ | ai-simple-ai, Developer (issue author)self-described: Team patching the SDK locally | Small product team | Published Nov 20, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| BOLT - How Mura wrote an in-house LLM Eval FrameworkID ir-02 | Engineering Blogs & Reviews | mackey.substack.com/ | James Mackey, Co-founder of Muraself-described: Mura, seed-stage startup automating billing for commercial HVAC providers | Technical leadership | Published Oct 21, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| A practical blueprint for evaluating conversational AI at scaleID s2-01 | Engineering Blogs & Reviews | dropbox.tech/ | Ranjitha Gurunath Kulkarni, Ameya Bhatawdekar, and Gonzalo Garcia, Dropbox Dash engineersDropbox Dash, a conversational AI search product | Product or platform team | Published Oct 2, 2025 | Assessment reviewed Oct 30, 2026 | Current | |
| Selecting The Right AI Evals ToolID s2-02 | Engineering Blogs & Reviews | hamel.dev/ | Hamel Husain, AI evals consultant and course instructorself-described: Three-person panel that watched LangSmith, Braintrust, and Arize Phoenix complete the same eval assignment | Individual builder | Published Oct 1, 2025 | Assessment reviewed Oct 30, 2026 | Current | |
| Evals your LLM driven Agents (Experiments and Lessons Learned) with BraintrustID dc-04 | Developer Communities | www.reddit.com/ | AIForOver50Plus, Reddit user, r/LLMDevsself-described: Weekend experiments with an email management agent | Individual builder | Published Sep 28, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| If you log a span instance, your API key is included in the logID dc-03 | Developer Communities | github.com/ | Austin Tackaberry (austintackaberry), Co-founder and CTO at Levenzaself-described: Levenza, logging with the Braintrust TypeScript SDK | Individual builder | Published Sep 7, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| Evals Are Not Unit TestsID s2-05 | Podcasts, Talks & Videos | ai.engineer/ | Ido Pesok, engineer on v0, VercelVercel v0 team running evals in CI and on a daily schedule | Product or platform team | Published Aug 6, 2025 | Assessment reviewed Oct 30, 2026 | Current | |
| Turning Fails into Features: Zapier's Hard-Won Eval LessonsID s2-06 | Podcasts, Talks & Videos | ai.engineer/ | Rafal Wilinski, Tech Lead for Zapier Agents, with Vitor BaloccoZapier Agents team comparing models on Braintrust experiments | Product or platform team | Published Jun 30, 2025 | Assessment reviewed Oct 30, 2026 | Current | |
| How to build world-class AI products: Sarah Sachs (AI lead @ Notion) & Carlos Esteban (Braintrust)ID tc-01 | Podcasts, Talks & Videos | www.youtube.com/ | Sarah Sachs, Engineering lead, AI modeling, at Notionself-described: Notion AI team: engineers, data specialists, PMs and designers | Product or platform team | Published Jun 27, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | Not counted. It was co-presented with Braintrust staff. |
| The Web Browser Is All You NeedID l2-01 | Podcasts, Talks & Videos | ai.engineer/ | Paul Klein IV, Founder and CEO, BrowserbaseBrowserbase, browser infrastructure for AI agents | Technical leadership | Published Jun 20, 2025 | Assessment reviewed Oct 30, 2026 | Current | |
| Ask HN: What tools are you using for AI evals? Everything feels half-bakedID dc-02 | Developer Communities | news.ycombinator.com/ | fazlerocks, Hacker News commenterself-described: Team running LLMs in production for content generation, customer support, and code review assistance | Small product team | Published Jun 5, 2025 | Assessment reviewed Oct 30, 2026 | Needs a recheck | |
| test(max): Migrate insight deepevals to BraintrustID dc-01 | Developer Communities | github.com/ | Michael Matloka (Twixes), Engineer at PostHogself-described: PostHog Max AI, evals for generated insight queries | Product or platform team | Published May 6, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| Review of BraintrustID ir-01 | Engineering Blogs & Reviews | www.producthunt.com/ | Howie Liu, Product Hunt reviewer, Airtableself-described: Airtable AI Assistant | Product or platform team | Published Apr 9, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| Unit-testing AI agents in the browserID s2-04 | Engineering Blogs & Reviews | www.browserbase.com/ | Anirudh Kamath and Sean McGuire, Browserbase Stagehand teamself-described: Stagehand browser-agent SDK, model comparison on its eval suite | Product or platform team | Published Apr 9, 2025 | Assessment reviewed Oct 30, 2026 | Still applicable | |
| Plans and limitsID pd-01 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Billing FAQID pd-02 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Monitor usageID pd-03 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Configure data retentionID pd-04 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Deployment optionsID pd-05 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| ArchitectureID pd-06 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| SecurityID pd-07 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Protect sensitive dataID pd-08 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| ExperimentsID pd-09 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Run experiments in CI/CDID pd-10 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Score production tracesID pd-11 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Trace LLM callsID pd-12 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Send OpenTelemetry traces and logsID pd-13 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
| Use the Braintrust GatewayID pd-14 | Product Documentation | www.braintrust.dev/ | BraintrustOfficial | Not counted (documentation) | No date stated | Documentation checked Oct 12, 2026 | Current | Verification only |
