Skip to main content

CrewAI vs OpenAI Agents SDK: Role-Based Crews or a Lightweight Agent Loop?

No named team on record runs both. Harvey evaluates its legal search agent on the Agents SDK's traces, PwC and Gelato report CrewAI gains in CrewAI's own case studies, and EpicStaff and Aurelio Labs each left one for code of their own.

Reading time
11 min read
Sources
42 cited
In this piece

GILD verdict

Choose the OpenAI Agents SDK for a small loop on OpenAI models whose traces feed evals, and CrewAI for specialist roles across vendors.

The workflow
A Python team choosing the framework its first production agent workflow will live on, between role-based crews and a small model-driven loop.1530
The decisive evidence
  • Reported5 of 8 named teams adopted one; What teams report below sets out each account.
  • ReportedHarvey's engineers wrote on Dec 10, 2025 that they run offline evals of their legal search agent in LangSmith on the Agents SDK's OpenTelemetry traces, and that the eval data lifted tool selection precision from near zero to 0.8 to 0.9.1
  • DocumentedCrewAI requires chromadb 1.5.9, which GitHub advisory GHSA-f4j7-r4q5-qw2c rates critical with no patched version, and its agents fall back to gpt-4.1-mini in code. The Agents SDK defaults to gpt-5.6-luna and sends traces with model and tool inputs and outputs to OpenAI unless the exporter is replaced.1021223135
  • GILD analysisNo named team on record runs both, so each side's case rests on its own accounts. At 10,000 runs a month of 20,000 input and 2,000 output tokens on gpt-5.6-luna, model tokens cost about US$64 on either framework, and the platform bill stays unresolved because AMP above 50 executions needs an Enterprise quote that CrewAI does not publish.
The tradeoff
Harvey's evals run on Agents SDK traces; a CrewAI team must pin its dependencies and make its tools safe to retry. CrewAI adds roles, flows, and a governed platform at the cost of more framework and dependencies to pin, which EpicStaff stopped carrying in September 2026. The Agents SDK keeps the code small, and its 0.Y.Z versioning lets minor releases break public interfaces.1615161922303136
What would change the call
The call moves to the Agents SDK when the team settles on OpenAI models or must leave Agent Builder before November 30, 2026, and to CrewAI when SSO, audit logs of review decisions, or VPC deployment become requirements.252740

The named teams split by side. Harvey built the offline evals for its agentic legal search on the Agents SDK's OpenTelemetry traces, and Lark's sandbox runtime defaults to an SDK agent. On CrewAI, PwC and Gelato report gains in CrewAI's case studies, an AB InBev engineer hit an async guardrail crash, and EpicStaff deleted its vendored CrewAI in September 2026. CrewAI runs models from any vendor and requires a ChromaDB release under a critical advisory; the Agents SDK defaults to gpt-5.6-luna and sends traces to OpenAI.

What teams report

Failures and incidents 1

  1. AB InBevOn CrewAI

    On crewai 1.15.18, async agents that an output guardrail rejected crashed when they should have retried, because the retry ran through the synchronous executor inside the event loop (GitHub issue, Sep 3, 2026).

    “AttributeError: 'coroutine' object has no attribute 'get'…”
    Jucelio Quentino, Developer, AB InBevGitHub, crewAIInc/crewAI, , post

Left or turned down 1

  1. PydanticOn CrewAI

    When Pydantic began building AI features into Logfire, Colvin looked at the frameworks built on Pydantic, CrewAI among them, found them short on type safety, and built Pydantic AI, which now competes with CrewAI (Software Engineering Daily, Dec 4, 2025).

    “Obviously, LangChain, LangGraph, Crew AI, LlamaIndex, all of these guys used Pydantic. And I assumed there was going to be a good one that I could go and use. Started looking at them and was super disappointed by what I found.”
    Samuel Colvin, Founder, PydanticSoftware Engineering Daily, , around 13:08

Evaluated 1

  1. IBM ConsultingOn CrewAI

    CrewAI's undated case study says IBM Consulting's federal team had two CrewAI pilots for eligibility decisions running inside US agencies by 2025, on watsonx, with Flows deciding when rules or agents run.

    “By 2025 his team had two CrewAI pilots running inside federal agencies…”
    Narayanan Alavandar, Quoted in CrewAI's case study, IBM ConsultingCrewAI, , write-upVendor customer story

Adopted 5

  1. HarveyOn OpenAI Agents SDK

    Harvey built centralized evaluation infrastructure on the Agents SDK's OpenTelemetry traces and scores its agentic legal search offline in LangSmith; the eval data lifted tool selection precision from near zero to 0.8 to 0.9 (Dec 10, 2025).

    “Centralized evaluation infrastructure that we built using the OpenAI Agent SDK's OpenTelemetry traces enables us to run offline evaluations in LangSmith. We track metrics like hallucination, tool recall, retrieval recall, formatting, and answer quality.”
    Lysia Li, Varun Nair, Aaron Stern and co-authors, Harvey engineering, HarveyHarvey blog, , write-up

  2. LarkOn OpenAI Agents SDK

    Lark open-sourced the sandbox runtime it built for its test-generating agents; by default it runs an Agents SDK agent with shell and web search tools, and one flag switches it to a Claude Code agent (Mar 18, 2026).

    “By default it runs an OpenAI Agents SDK agent with shell + web search tools.”
    Vijit and Jack, Builders of Runtime Use (Show HN authors), LarkHacker News, , post

  3. PwCOn CrewAI

    CrewAI's undated case study says PwC's CrewAI agents generate and validate proprietary-language code and draft long specifications, and reports code-generation accuracy rising from 10% to 70% with no method stated.

    “Crew-powered agents boosted code-generation accuracy from 10% to 70%, slashed turnaround time on complex documents, and supplied granular data to prove ROI.”
    Jacob Wilson, Quoted in CrewAI's case study, PwCCrewAI, , write-upVendor customer story
  4. GelatoCrewAI's undated case study says Gelato runs CrewAI agents that write, test and deploy carrier-integration code, and reports carrier onboarding falling from 5 days to 10 minutes.
    “Deployed logistics agents that generate, test, and deploy carrier‑integration code, shrinking onboarding from 5 days to 10 minutes.”
    Styrbjörn Holmberg, Quoted in CrewAI's case study, GelatoCrewAI, , write-upVendor customer story
  5. Octagon AIOctagon AI open-sourced an MCP server of simulated venture investors on the Agents SDK, citing multi-agent reasoning, handoffs and tool use out of the box (Apr 26, 2025).
    “It’s powered by OpenAI’s new Agents SDK, so you get true agentic workflows out of the box:…”
    ag_21, Show HN author, Octagon AIHacker News, , post

Choosing between CrewAI and OpenAI Agents SDK

Choose CrewAI if

  • Your work maps to specialist roles with rules around them, as IBM Consulting's federal eligibility pilots let Flows decide when rules or agents run, per CrewAI's 2025 case study.
  • You want a managed platform with SSO, role-based access control, and deployment in your own VPC.
  • You run models from several vendors, and your team can pin CrewAI's dependencies, ChromaDB among them, and make every tool with side effects safe to retry.

Choose OpenAI Agents SDK if

  • Your agents run mainly on OpenAI models and you want their traces as eval input, as Harvey exports them over OpenTelemetry to LangSmith (Dec 10, 2025).
  • A small loop with handoffs and guardrails covers the work, in TypeScript as well as Python.
  • You are moving workflows off Agent Builder, which OpenAI retires on November 30, 2026 and replaces with the SDK for workflows kept as code.

Side by side

DimensionCrewAIOpenAI Agents SDK
What teams reportNo named team on record runs both. CrewAI's figures come from its own case studies, which state no method.PwC reports code-generation accuracy rising from 10% to 70%, and Gelato carrier onboarding falling from 5 days to 10 minutes, in CrewAI's case studies with no method stated. An AB InBev engineer's async agents crashed on their first guardrail rejection (Sep 3, 2026), and EpicStaff deleted its vendored CrewAI, 180,251 lines (Sep 4, 2026)561213Harvey runs offline evals of its legal search agent on the SDK's traces, and its eval data lifted tool selection precision from near zero to 0.8 to 0.9 (Dec 10, 2025). Lark's sandbox runtime and Coinbase's AgentKit ship on it; Aurelio Labs left it for its own async framework (Oct 8, 2025)1234
DesignCrews of role-based agents with goals and tools run tasks, inside Flows that hold state and route with start, listen, and router steps1516A small set of primitives: agents with instructions and tools, handoffs, and guardrails, in a built-in loop that runs until the task is complete30
License and languagesLatest releases on each project's GitHub repository, checked Oct 28, 2026.MIT, free; Python 3.10 to 3.13 only (crewai 1.15.32, Oct 27, 2026)2829MIT, free; Python (openai-agents 0.24.0, Oct 15, 2026) and TypeScript (@openai/agents 0.21.0, Oct 20, 2026); 0.Y.Z versioning in which a minor release may break public interfaces363739
ModelsAn agent with no model set reads MODEL, MODEL_NAME or OPENAI_MODEL_NAME, then falls back to gpt-4.1-mini in the library's code, while the Agents docs still say gpt-4; you choose the model per agent as provider/model-id and bring your own key for each provider; native SDKs for OpenAI, Anthropic, Gemini, Azure, Bedrock, and Snowflake Cortex, and LiteLLM for the rest192021Default model gpt-5.6-luna with reasoning effort none when an agent sets no model; Responses API for OpenAI models; other providers through Chat Completions-compatible endpoints or the LiteLLM and any-llm adapters31
DependenciesChecked against each project's pyproject.toml on GitHub and the GitHub advisory database, Oct 28, 2026.Requires chromadb~=1.5.9 on Oct 28, 2026, a range GitHub advisory GHSA-f4j7-r4q5-qw2c rates critical for pre-authentication code injection with no patched version; Headroom and ai-agent-assembly removed CrewAI from their lockfiles over ChromaDB in 2026891022Depends on openai, pydantic, mcp, websockets, and a few smaller packages, with security floors set on transitive dependencies such as starlette and urllib3 (pyproject, Oct 28, 2026)38
State and durabilityFlow state with @persist, SQLite by default; checkpoints let a crew, flow, or agent resume after a failure or fork, skipping completed tasks and rerunning the rest, tools included1617Sessions keep history in SQLite, Redis, SQLAlchemy, MongoDB, Dapr, or OpenAI's Conversations API; durable execution through Temporal, Restate, or DBOS integrations, Temporal's generally available since Mar 23, 2026113233
Human reviewThe @human_feedback decorator, from CrewAI 1.8.0, pauses a Flow for review and routes on the answer; AMP adds email routing and immutable audit logs of decisions1825Tools that need approval pause the run, and the run state is serialized so it can resume after a decision34
Telemetry and tracingAnonymous usage telemetry on by default, off with CREWAI_DISABLE_TELEMETRY or OTEL_SDK_DISABLED; AMP tracing off with CREWAI_TRACING_ENABLED=false; AMP exports traces to your OpenTelemetry collector2324Tracing on by default, to OpenAI's traces dashboard, with model and tool inputs and outputs included; off with OPENAI_AGENTS_DISABLE_TRACING=1; unavailable to organizations under Zero Data Retention; Harvey exports the traces over OpenTelemetry to LangSmith135
Hosting and platform priceList prices in US dollars, checked Oct 28, 2026. CrewAI publishes no per-execution rate.Your own service, or CrewAI AMP metered per month in workflow executions: Basic at US$0 a month up to 50 executions, Enterprise custom with included executions sized to the workflow and flexible overage, on CrewAI cloud, a dedicated VPC, or your infrastructure2627A library you run in your own service; spend is the model and tool usage it calls, at OpenAI API prices. OpenAI's hosted Agents API is a separate product in public beta3042
Cost at a stated workloadGILD's calculation from list prices checked Oct 28, 2026. The inputs are stated assumptions that nobody measured; cached input at US$0.02 per million lowers the total, and prompts above 272,000 input tokens cost more.GILD's calculation for a workload of 10,000 agent runs a month, each with 20,000 input and 2,000 output tokens, all uncached, on gpt-5.6-luna at US$0.20 per million input and US$1.20 per million output tokens: about US$64 a month in model tokens, US$40 input plus US$24 output, when CrewAI agents use the same model. On AMP each run is a workflow execution, so 10,000 runs pass Basic's maximum of 50 and need an Enterprise quote with overage; self-hosting the library adds your own infrastructure instead1927GILD's calculation for the same workload of 10,000 agent runs a month, each with 20,000 input and 2,000 output tokens, all uncached, on gpt-5.6-luna at US$0.20 per million input and US$1.20 per million output tokens, the SDK's default model: about US$64 a month, US$40 for 200 million input tokens plus US$24 for 20 million output tokens, with the MIT library free and hosting on your own infrastructure314142
When to chooseChoose CrewAI if the work maps to specialist agents with roles, you run models from several vendors, or you want a managed platform with SSO and VPC deployment, and you can pin its dependencies152227Choose the OpenAI Agents SDK if agents run mainly on OpenAI models and a small loop with handoffs, guardrails, sessions, and built-in tracing covers the work, in Python or TypeScript3035

What practitioners report, side by side

Each product's figures come from its own practitioners, for the group you pick.

CrewAICrewAI−7 MixedHow this figure is made
OpenAI Agents SDKOpenAI−17 MixedHow this figure is made
Whose sources to count
Compare CrewAI with
  • OpenAI Agents SDK

Overall signal

All practitioners rate CrewAI −7, 10 points above OpenAI Agents SDK at −17. Most of the gap is in complaints, with 21 of OpenAI Agents SDK's 50 sources negative against 9 of CrewAI's 30.

The score counts the authors from all practitioners that take a side, one vote each with their latest view, and is how far positive votes lead negative ones, per 100 votes, so OpenAI Agents SDK's −17 rests on votes split 11 positive, 16 mixed and 19 negative.

Each of CrewAI's authors wrote one source here, so each source is one vote. A lead of a fifth of the votes reads positive and a lead of half strong positive, with the same lines below zero. The dots show each source, positive, mixed, negative. Build one pilot agent to measure the glue code and debugging time the framework adds.

Footprint on the AI Application Platform stack

They differ on two of 9 layers. On Retrieval, CrewAI supports it and OpenAI Agents SDK touches it. On Cloud deployment, CrewAI supports it and OpenAI Agents SDK touches it. Coverage is each product's placement on this stack, as its product page shows it. Core is a job the product owns; Supports and Touches are jobs it helps with. Coverage says nothing about how well a product does them.

Layers where CrewAI and OpenAI Agents SDK differ
LayerCrewAIOpenAI Agents SDK
Data
RetrievalSupportsTouches
Operate
Cloud deploymentSupportsTouches
The 7 layers where they match
Layers where CrewAI and OpenAI Agents SDK match
LayerCrewAIOpenAI Agents SDK
Model
Model APIsTouchesTouches
Model gatewayTouchesTouches
Build
OrchestrationCoreCore
Tool useSupportsSupports
Evaluate
EvaluationTouchesTouches
Operate
ObservabilitySupportsSupports
GuardrailsSupportsSupports

What practitioners praise

Among all practitioners, Fast to a Working Crew and Crews in Real Work reach 5 sources from 3 authors only for CrewAI, at 10 of 10 and 7 of 7. OpenAI Agents SDK needs a pilot on the same agent task before you choose on either. A Small Set of Primitives and Agents Built and Shipped reach 5 sources from 3 authors only for OpenAI Agents SDK, at 11 of 11 and 9 of 9. CrewAI needs a pilot on the same agent task before you choose on either.

What practitioners praise, All practitioners
Praised forCrewAIOpenAI Agents SDK
Fast to a Working Crew10 of 10Not a theme in its sources
Crews in Real Work7 of 7Not a theme in its sources
A Small Set of PrimitivesNot a theme in its sources11 of 11
Agents Built and ShippedNot a theme in its sources9 of 9

Each count is the sources that praise a theme, or raise it as a concern, out of those that discuss it. A row pairs the two products when both carry a theme under the same or an equivalent name.

A count ending in "so far" has under 5 sources from 3 authors among all practitioners. "Not a theme in its sources" means GILD found no theme of that name in that product's sources.

What practitioners raise as concerns

Among all practitioners, CrewAI alone has five concern themes that reach 5 sources from 3 authors. Async, Memory & Checkpoint Defects and Coarse Control Over Agents lead, at 10 of 10 and 9 of 9. OpenAI Agents SDK alone has five concern themes that reach 5 sources from 3 authors. Runtime Defects in Production and Latency & Run Overhead lead, at 14 of 14 and 7 of 7.

What practitioners raise as concerns, All practitioners
Raised as a concernCrewAIOpenAI Agents SDK
Async, Memory & Checkpoint Defects10 of 10Not a theme in its sources
Coarse Control Over Agents9 of 9Not a theme in its sources
Defaults, Telemetry & Dependencies9 of 9Not a theme in its sources
Token Burn & Usage Accounting7 of 7Not a theme in its sources
Tools That Run Twice or Not at All7 of 7Not a theme in its sources
Runtime Defects in ProductionNot a theme in its sources14 of 14
Latency & Run OverheadNot a theme in its sources7 of 7
OpenAI-First Design & Adapter GapsNot a theme in its sources7 of 8
Hosting & Storage Left to the TeamNot a theme in its sources5 of 5
Tracing Defaults & Export GapsNot a theme in its sources4 of 5
CrewAI SignalsOpenAI Agents SDK Signals

42 sources

  1. Harvey blog: How agentic search unlocks legal research intelligence at Harvey (Dec 10, 2025)Public sourceharvey.ai
  2. Show HN by Lark's founders: Runtime Use, whose default agent runs on the OpenAI Agents SDK (Mar 18, 2026)Public sourcenews.ycombinator.com
  3. coinbase/agentkit pull request #550: OpenAI Agents SDK support in Python (Mar 11, 2025)Public sourcegithub.com
  4. Hacker News comment by Aurelio Labs' graphai maintainer on leaving LangChain, LangGraph and the Agents SDK (Oct 8, 2025)Public sourcenews.ycombinator.com
  5. CrewAI issue #7252: kickoff_async guardrail retry crashes, filed by an AB InBev engineer (Sep 3, 2026)Public sourcegithub.com
  6. EpicStaff pull request #768: delete vendored CrewAI and mem0 libraries (Aug 20, 2026, merged Sep 4, 2026)Public sourcegithub.com
  7. Software Engineering Daily: Pydantic AI with Samuel Colvin (Dec 4, 2025)Public sourcesoftwareengineeringdaily.com
  8. Headroom pull request #3870: drop the crewai extra to remove chromadb from the lockfile (Sep 30, 2026)Public sourcegithub.com
  9. ai-agent-assembly pull request #133: remove crewai to fix a critical chromadb vulnerability (Jul 7, 2026)Public sourcegithub.com
  10. GitHub advisory GHSA-f4j7-r4q5-qw2c: ChromaDB pre-authentication code injection, critical (May 18, 2026)Public sourcegithub.com
  11. Temporal blog: Production-ready agents with the OpenAI Agents SDK and Temporal, generally available Mar 23, 2026 (Jul 30, 2025)Public sourcetemporal.io
  12. CrewAI customer story: PwC accelerates enterprise-scale GenAI adoption with CrewAI (undated, read Oct 28, 2026)Official documentationcrewai.com
  13. CrewAI customer story: Gelato accelerates fulfillment via agentic integration (undated, read Oct 28, 2026)Official documentationcrewai.com
  14. CrewAI customer story: IBM automates federal eligibility with agents (undated, read Oct 28, 2026)Official documentationcrewai.com
  15. CrewAI docs: Introduction (Crews and Flows)Official documentationdocs.crewai.com
  16. CrewAI docs: Flows (state and @persist)Official documentationdocs.crewai.com
  17. CrewAI docs: CheckpointingOfficial documentationdocs.crewai.com
  18. CrewAI docs: Human feedback in FlowsOfficial documentationdocs.crewai.com
  19. CrewAI docs: LLMs (model setup and providers)Official documentationdocs.crewai.com
  20. CrewAI docs: Agents (the llm attribute and its default)Official documentationdocs.crewai.com
  21. CrewAI source: constants.py, DEFAULT_LLM_MODEL gpt-4.1-miniOfficial documentationgithub.com
  22. CrewAI source: pyproject.toml, which requires chromadb~=1.5.9Official documentationgithub.com
  23. CrewAI docs: TelemetryOfficial documentationdocs.crewai.com
  24. CrewAI AMP docs: OpenTelemetry ExportOfficial documentationdocs-platform.crewai.com
  25. CrewAI AMP docs: Flow HITL Management (audit logs)Official documentationdocs-platform.crewai.com
  26. CrewAI AMP docs: Deploy to AMPOfficial documentationdocs-platform.crewai.com
  27. CrewAI pricing (AMP Basic and Enterprise, executions, overage)Official documentationcrewai.com
  28. CrewAI on GitHub (MIT license, Python versions, telemetry)Official documentationgithub.com
  29. CrewAI releases on GitHub (1.15.32, Oct 27, 2026)Official documentationgithub.com
  30. OpenAI Agents SDK docs: OverviewOfficial documentationopenai.github.io
  31. OpenAI Agents SDK docs: Models (default model gpt-5.6-luna, other providers)Official documentationopenai.github.io
  32. OpenAI Agents SDK docs: SessionsOfficial documentationopenai.github.io
  33. OpenAI Agents SDK docs: Running agents (durable execution integrations)Official documentationopenai.github.io
  34. OpenAI Agents SDK docs: Human in the loopOfficial documentationopenai.github.io
  35. OpenAI Agents SDK docs: TracingOfficial documentationopenai.github.io
  36. OpenAI Agents SDK docs: Release process (0.Y.Z versioning)Official documentationopenai.github.io
  37. OpenAI Agents SDK on GitHub (MIT license, v0.24.0, Oct 15, 2026)Official documentationgithub.com
  38. OpenAI Agents SDK source: pyproject.toml dependenciesOfficial documentationgithub.com
  39. OpenAI Agents SDK for TypeScript docs: Overview (@openai/agents 0.21.0, Oct 20, 2026)Official documentationopenai.github.io
  40. OpenAI: Introducing AgentKit, with the Jun 3, 2026 update retiring Agent Builder on Nov 30, 2026Official documentationopenai.com
  41. OpenAI API docs: GPT-5.6 Luna model page (pricing)Official documentationdevelopers.openai.com
  42. OpenAI API docs: PricingOfficial documentationdevelopers.openai.com
Read the research