Skip to main content

BerriAIAI Gateway

LiteLLM

BerriAI's self-hosted gateway and Python SDK expose 100+ model providers through one OpenAI-format API. The MIT-licensed core includes virtual keys, budgets, rate limits, fallbacks, and spend tracking. Enterprise adds SSO and audit logs, plus support.

The 30-Second Take

Pick LiteLLM if prompts and provider keys must stay in your own network and someone will patch it.

BerriAI's MIT gateway puts one OpenAI-format API in front of 100+ providers, with virtual keys and budgets. Okta and Zalando run it as their company model gateway.

Best fit
Platform teams giving many apps and engineers self-hosted access to several providers.
Main constraint
You run the database and Redis, track weekly releases, and patch a proxy with a long advisory record.
First move
Run the Docker image pinned by digest to 1.106.5 or a later patched release, with Postgres for one team and fail-closed budgets.

Why

Watch the supply chain: malicious releases reached PyPI twice in 2026, and official Docker tags carried the malware once. One upgrade also stopped a team's usage billing for 6.5 days, so alert when spend falls to zero too.

Practitioner sentiment, teams and leaders+8 Mixed

36 practitioners, 137 sources reviewed, Updated Nov 1, 2026How this is measured

Read the GILD Review
Save a product your organization uses.Keep up with LiteLLM without adding it to your stack.Visit Website
Operating stack
AI Application Platform

Comparisons

How does LiteLLM compare with the alternatives?

  1. LiteLLM vs OpenRouter

    • AI Gateways

    A developer left LiteLLM for OpenRouter over outages, and an RTFM engineer kept LiteLLM in front of OpenRouter after its 429 errors. On 1 million Sonnet 5.5 requests a month, GILD prices LiteLLM at US$4,024 and OpenRouter's Standard credits at US$4,220, before staff time.

  2. LiteLLM vs Portkey

    • AI Gateways

    TU Vienna rejected both over enterprise-gated SSO and built its own gateway; a Portkey cache bug took a production platform down for all its customers. Pick LiteLLM for free key budgets in a gateway you run, and Portkey, now Palo Alto Networks' Prisma AIRS AI Gateway, for hosted observability priced by recorded logs.

Operator Insights

What are teams learning from building on LiteLLM and running it in production?

See all 8 operator accounts

Buying and deployment

What a team building on it should verify before production traffic grows.

Sourced from BerriAI documentation

How you buy it

Free, MIT

Self-hosting is free with keys, budgets, and spend tracking. Enterprise bills yearly by capacity.

Source 1, litellm.ai, LiteLLM pricing: LiteLLM pricingSource 2, BerriAI, Enterprise: EnterpriseSource 5, GitHub, LICENSE (GitHub): LICENSE (GitHub)

Details

Self-hosting costs US$0 under MIT outside the enterprise directory, including virtual keys, budgets, rate limits, fallbacks, and spend tracking. Enterprise licenses bill annually by gateway request capacity, architecture, and support, with no per-token charge. Buyers purchase directly, through AWS or Azure Marketplace, or through a reseller. A 30-day trial key is free.

Cost drivers

Provider tokens

Each provider bills at its rates. Plan 1 vCPU and 4 GiB per gateway pod.

Source 1, litellm.ai, LiteLLM pricing: LiteLLM pricingSource 6, BerriAI, Production best practices: Production best practicesSource 8, BerriAI, Budgets and rate limits: Budgets and rate limitsSource 2, BerriAI, Enterprise: Enterprise

Details

Each provider bills tokens at its rates. BerriAI recommends 1 vCPU and 4 GiB per gateway pod, with Postgres for keys and spend and Redis across instances. Enterprise adds SSO beyond 5 users, audit logs, and tag budgets. Round-the-clock support SLAs cost extra.

Where it runs

Your infrastructure

Run it in your cloud. Prompts stay off BerriAI; Enterprise reports successful request counts.

Source 2, BerriAI, Enterprise: EnterpriseSource 7, docs.litellm.ai, Production deployment: Production deploymentSource 1, litellm.ai, LiteLLM pricing: LiteLLM pricingSource 9, BerriAI, Data privacy and security: Data privacy and securitySource 10, docs.litellm.ai, Billable request metering: Billable request metering

Details

LiteLLM runs in your cloud or data center using signed Docker images, a Helm chart, or Terraform for AWS and GCP. Enterprise supports air-gapped deployments. Self-hosted LiteLLM keeps prompts and telemetry off BerriAI servers. Enterprise metering sends a count of successful requests.

What to verify

March 24, 2026

Two PyPI releases carried credential-stealing malware. Only the latest four minor lines get patches.

Source 12, api.github.com, GitHub Advisory Database: reviewed litellm advisories (API): GitHub Advisory Database: reviewed litellm advisories (API)Source 4, docs.litellm.ai, Security update, March 2026: Security update, March 2026Source 11, docs.litellm.ai, Security incident, v1.107.0 (Oct 21, 2026): Security incident, v1.107.0 (Oct 21, 2026)Source 13, api.github.com, Advisory GHSA-w4c7-34wj-hqm8 (GitHub API): Advisory GHSA-w4c7-34wj-hqm8 (GitHub API)Source 3, BerriAI, Version support: Version supportSource 8, BerriAI, Budgets and rate limits: Budgets and rate limits

Details

GitHub lists 44 reviewed litellm advisories as of October 28, 2026, including 7 critical ones. PyPI versions 1.82.7 and 1.82.8 carried credential-stealing malware on March 24, 2026. On October 21, 2026, PyPI 1.107.0 and the Docker tags v1.107.0, latest, main-latest, and main-stable did too. Pin 1.106.5 by digest. Only the four latest weekly minor lines receive patches. Budgets fail open without a database.

Signals

What is the current evidence saying about LiteLLM?

What platform teams at Okta, Zalando and FutureSearch and individual developers on Reddit, Hacker News and GitHub report publicly about running LiteLLM, with each finding linked to its sources.

Whose sources count

Practitioner sentiment, teams and leaders

+8 Mixed net

of 37 sources that take a side, each author counted once.

How this figure is made

Current sentiment

Positive, mixed and negative sources

The score counts them as 9 positive, 21 mixed and 6 negative votes, one per author.

Teams and leaders are mixed, and most of the mixed accounts are bug reports. Okta and Zalando run LiteLLM as the company gateway, and teams on Reddit cap each team key with a daily hard stop. Critics report budgets that track spend but do not block requests, and team keys that get messy past a handful of teams. Every team and leadership source that discusses stability raises it. On performance at scale, 7 of 9 raise concerns, and 5 of the 9 who describe a gateway choice picked another tool or their own code.

Run a pilot on one service's model traffic to measure added latency, failed fallbacks and how closely its spend records match each provider's bill.

Evidence field. All 137 sources, one mark each.

Arrange by

Each mark is one source: positive, mixed, negative, no opinion stated, or documentation (not counted).

By sentiment, Teams and technical leaders. Every source GILD read, grouped as the field draws it.
GroupSourcesPositiveMixedNegativeNo opinion stated and not counted in the scoreDocumentation, checked and not counted
Positive10100000
Mixed21021000
Negative600600
Not counted10000100
Documentation12000012
Outside this group7828212270
Every source GILD read, by the quarter it was published, as the field's time axis counts it. Documentation is dated by the day it was checked.
QuarterSources
Q1 20254
Q2 202512
Q3 202513
Q4 202511
Q1 202631
Q2 202623
Q3 202622
Q4 20269
Documentation, checked12

By sentiment

12 documentation pages check facts and are not counted; 78 sit outside this group.

Hover a mark for a quick look, click it or press Enter to open its source.Tap a mark to open its source, then step to its neighbors with Previous and Next.

What practitioners praise

Top praise of 4. Each mark is a source on it; filled marks praise it.

  1. Virtual Keys, Budgets & Spend Tracking7 of 10 sources on it praise it

Most common concerns

Top concern of 3. Each mark is a source on it; filled marks raise it.

  1. Stability & Code QualityAll 11 sources on it raise it as a concern
All 137 sources behind these findings
  1. Comment on: If you do, which ai gateway do you use and why?anitamaxwynnn69, Reddit commenter (self-described: Local AI setup that tracks token use and shares one key with a few people), Reddit comment, r/LocalLLaMA, published Oct 24, 2026
  2. Comment on: If you do, which ai gateway do you use and why?SamSausages, Reddit commenter (self-described: Personal setup that switches between model accounts when usage limits run out), Reddit comment, r/LocalLLaMA, published Oct 23, 2026
  3. Comment on: If you do, which ai gateway do you use and why?colin_colout, Reddit commenter (self-described: Personal local AI setup behind a gateway), Reddit comment, r/LocalLLaMA, published Oct 23, 2026
  4. Comment on: If you do, which ai gateway do you use and why?OneAnxiety2523, Reddit commenter (self-described: Personal projects tinkering with local and hosted models), Reddit comment, r/LocalLLaMA, published Oct 23, 2026
  5. Comment on: LiteLLM Compromised again - 1.107.0. PyPI Installs a Credential Stealer and BackdoorPsychologicalFlan877, Reddit commenter (self-described: A team whose CI gates new package versions on release age), Reddit comment, r/devops, published Oct 22, 2026
  6. litellm 1.107.0 compromised: cook.pth credential stealer and sitecustomize RCE loaderLidorMachluf, GitHub user who reported the compromise (self-described: Analysis of the 1.107.0 PyPI wheels and Docker images), GitHub issue, published Oct 21, 2026
  7. MCP gateway DCR rejects Gemini app connectors: redirect_uris must be a list of 1 to 3 URIsmadsbergstroem, GitHub user (self-described: LiteLLM's MCP gateway serving a custom connector in the Gemini app), GitHub issue, BerriAI/litellm, published Oct 15, 2026
  8. EnterpriseBerriAI (Official), Product documentation, checked Oct 12, 2026
  9. Budgets and rate limitsBerriAI (Official), Product documentation, checked Oct 12, 2026
  10. Spend trackingBerriAI (Official), Product documentation, checked Oct 12, 2026
  11. Load balancingBerriAI (Official), Product documentation, checked Oct 12, 2026
  12. Production best practicesBerriAI (Official), Product documentation, checked Oct 12, 2026
  13. Release cycleBerriAI (Official), Product documentation, checked Oct 12, 2026
  14. Version supportBerriAI (Official), Product documentation, checked Oct 12, 2026
  15. Docker image securityBerriAI (Official), Product documentation, checked Oct 12, 2026
  16. Shared responsibilityBerriAI (Official), Product documentation, checked Oct 12, 2026
  17. SSO for the Admin UIBerriAI (Official), Product documentation, checked Oct 12, 2026
  18. Rust AI Gateway (beta)BerriAI (Official), Product documentation, checked Oct 12, 2026
  19. Data privacy and securityBerriAI (Official), Product documentation, checked Oct 12, 2026
  20. Comment on: LLM gateway - pass through Vs unifiedPerfectStephen, Reddit commenter (self-described: A team running a unified-schema gateway in front of several model providers), Reddit comment, r/mlops, published Oct 11, 2026
  21. Comment on: Ask HN: What do you run on a US$5 VPS that's worth keeping online 24/7?K0IN, Hacker News commenter (self-described: Personal VPS), Hacker News comment, published Oct 9, 2026
  22. Comment on: How do you set LLM budgets per team?DevWorkflowBuilder, Reddit commenter (self-described: A team whose weekend test script ate half a month's model budget), Reddit comment, r/devops, published Sep 29, 2026
  23. Comment on: Litelm: LiteLLM Without the Bloatznpy, Hacker News commenter (self-described: LiteLLM at work), Hacker News comment, published Sep 12, 2026
  24. Comment on: Litelm: LiteLLM Without the Bloatblazarquasar, Hacker News commenter (self-described: Ran LiteLLM and Bifrost side by side), Hacker News comment, published Sep 12, 2026
  25. Comment on: Litelm: LiteLLM Without the Bloatsantiagobasulto, Hacker News commenter (self-described: Built own provider-agnostic agent layer), Hacker News comment, published Sep 12, 2026
  26. Comment on: Litelm: LiteLLM Without the Bloatmpyne, On a team that adopted LiteLLM (as stated) (self-described: Organization rolling out AI with LiteLLM as its gateway), Hacker News comment, published Sep 12, 2026
  27. litellm-ops: operations toolkit and proxy-versus-direct latency benchmarkphenethol, LLM infrastructure intern at the Ruđer Bošković Institute (self-described: The Ruđer Bošković Institute's self-hosted LiteLLM proxy, with users, budgets, and rate limits), Independent benchmark, GitHub repository, published Sep 12, 2026
  28. Comment on: Litelm: LiteLLM Without the Bloat9dev, Hacker News commenter (self-described: Platform tracking token spend per customer), Hacker News comment, published Sep 11, 2026
  29. Comment on: Litelm: LiteLLM Without the Bloattestycool, Hacker News commenter (self-described: LiteLLM on a Hetzner server via Coolify), Hacker News comment, published Sep 11, 2026
  30. Building a Local-First AI Coding Agent with Open Tools and Adaptive Routingraphiki, Writer for Worldline's technology blog (self-described: Laptop setup with Ollama, OpenCode and LiteLLM), Engineering blog post, Technology at Worldline (dev.to), published Sep 9, 2026
  31. AnyLLM and LiteLLM return successful empty output for truncated responsesCoralGarden52, GitHub user (self-described: OpenAI Agents SDK agents calling providers through the LiteLLM adapter), GitHub issue, openai/openai-agents-python, published Sep 6, 2026
  32. [Bug]: Vertex AI Gemini + explicit context caching - usage.prompt_tokens intermittently reports only the fresh (non-cached) token countfranciscobispo, GitHub user (self-described: A LiteLLM proxy calling Vertex AI Gemini with explicit context caching, with cost reported from the usage fields), GitHub issue, BerriAI/litellm, published Sep 1, 2026
  33. LiteLLM AI Gateway: Routing, API Keys, Budgets, and Guardrails for Local and Cloud ModelsPaul Zuradzki, Developer (self-described: Self-hosted gateway in front of local GPUs and cloud models for a coding agent), Video, YouTube, published Aug 26, 2026
  34. Comment on: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)geek_at, Hacker News commenter (self-described: Self-hosted router with model aliases), Hacker News comment, published Aug 25, 2026
  35. Agentic Engineering at Zalando: a snapshotBartosz Ocytko, Executive Principal Engineer, Zalando (Zalando's company-wide LLM proxy, run by its ML platform team since January 2024), Engineering blog post, Zalando Engineering Blog, published Aug 14, 2026
  36. Comment on: Managing AI Coding Costs at Scale827a, Engineer at a startup (as stated) (self-described: Startup engineering team using Azure credits), Hacker News comment, published Aug 8, 2026
  37. Comment on: Is Hermes Agent unusable without a model router?Swapnoneel, Hacker News commenter (self-described: Model router for the Hermes agent), Hacker News comment, published Aug 3, 2026
  38. LiteLLM: OpenRouter Integration and Fallbacks ConfigurationArseny Zinchenko, DevOps engineer and author of the RTFM blog (self-described: A company LiteLLM gateway that already handles requests from all of the team's services, with OpenRouter added behind it), Engineering blog post, RTFM, published Jul 31, 2026
  39. [Bug]: Noma v2 guardrail ignores streaming_end_of_stream_only and streaming_sampling_rateyryzhan-vitech, GitHub user running the Noma guardrail on a LiteLLM proxy (self-described: A LiteLLM proxy with the Noma v2 guardrail scanning streamed responses), GitHub issue, BerriAI/litellm, published Jul 28, 2026
  40. Comment on: Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTAaftbit, Hacker News commenter (self-described: Running LiteLLM), Hacker News comment, published Jul 22, 2026
  41. v3 rate limiter double-counts team per-model limits (model_per_team): effective RPM/TPM is half configuredalmogx3, GitHub user running a self-hosted proxy (self-described: Self-hosted LiteLLM proxy with per-team model rate limits), GitHub issue, published Jul 21, 2026
  42. Comment on: An LLM API gateway with budget-limited tokens for your LLM API keysendlessroad1991, Author of a budget-limited key gateway (as stated) (self-described: Personal serverless gateway on Cloudflare Workers), Hacker News comment, published Jul 20, 2026
  43. Comment on: Kimi K2.7 Code is generally available in GitHub Copilotlbreakjai, At a company that rolled out LiteLLM (as stated) (self-described: Company-wide rollout replacing per-seat AI plans), Hacker News comment, published Jul 2, 2026
  44. Comment on: TrueFoundry vs LiteLLM vs Portkey for Internal LLM Access?Own_Anywhere9206, Reddit commenter (self-described: A company that ran LiteLLM internally for several teams for a few months), Reddit comment, r/devops, published Jun 18, 2026
  45. Comment on: TrueFoundry vs LiteLLM vs Portkey for Internal LLM Access?Jony_Dony, Reddit commenter (self-described: Teams enforcing different model access per environment through one gateway), Reddit comment, r/devops, published Jun 18, 2026
  46. Comment on: TrueFoundry vs LiteLLM vs Portkey for Internal LLM Access?theexplorer1997, Reddit commenter (self-described: An internal playground and trial setup on LiteLLM), Reddit comment, r/devops, published Jun 18, 2026
  47. 5 Pitfalls I Hit Running LiteLLM Proxy in Production (with a 1-page survival map)Hanlin Xiang, Developer running an LLM gateway (as stated) (self-described: 18-channel gateway with Redis and PostgreSQL serving agents and internal tools for six months), dev.to article, published Jun 17, 2026
  48. Comment on: Kimi K2.7-Code: open-source coding model with better token efficiencynobleach, Hacker News commenter (self-described: Organization gateway offering Kimi), Hacker News comment, published Jun 12, 2026
  49. Comment on: Kimi K2.7-Code: open-source coding model with better token efficiencyjwbron, Hacker News commenter (self-described: Claude Code on open models), Hacker News comment, published Jun 12, 2026
  50. Comment on: Running Claude Code Offline on an M3 Pro with Qwen3.6msradam, Hacker News commenter (self-described: Internal models at work behind a LiteLLM proxy), Hacker News comment, published Jun 11, 2026
  51. LiteLLM made us accidentally make our product free for a weekJack Wildman, Senior Software Engineer, FutureSearch (FutureSearch's research-agent platform, which bills users from LiteLLM spend logs), Engineering blog post, FutureSearch blog, published Jun 10, 2026
  52. Comment on: Ask HN: How to manage AI APIs for SaaS application?mmarian, Running the LibreThinker service (as stated) (self-described: LibreThinker, a service with about 1,000 users), Hacker News comment, published May 20, 2026
  53. Comment on: LLMCap - A proxy that hard-stops LLM API calls when you hit a dollar capkgeist, Hacker News commenter (self-described: Self-hosted gateway at work), Hacker News comment, published May 19, 2026
  54. Comment on: LLMCap - A proxy that hard-stops LLM API calls when you hit a dollar caplukewarm707, Hacker News commenter (self-described: Self-hosted LiteLLM locally for privacy), Hacker News comment, published May 19, 2026
  55. Comment on: AI subscriptions are a ticking time bomb for enterpriseintothemild, Hacker News commenter (self-described: Company pilot: self-hosted Qwen, Gemma, and DeepSeek behind LiteLLM), Hacker News comment, published May 17, 2026
  56. Heavy RAM usage over some timeSanady, GitHub user at a firm running LiteLLM on Kubernetes (self-described: Two-pod Kubernetes deployment with Redis caching and 4 GB pods), GitHub issue, published May 14, 2026
  57. [Bug]: Budget enforcement bypassed in v1.82.3 for key/user max_budget despite spend exceeding max_budgetlin0703, GitHub user (self-described: Fresh v1.82.3 deployment compared with v1.81.0), GitHub issue, published Apr 28, 2026
  58. US$38k AWS Bedrock bill caused by a simple prompt caching missZephyr0x, Hacker News commenter (self-described: Own coding-agent workflow: Droid to LiteLLM to Claude Opus 4.6 on Amazon Bedrock), Hacker News post, published Apr 28, 2026
  59. Comment on: Show HN: GoModel - an open-source AI gateway in Gosmcleod, Hacker News commenter (self-described: Has deployed LiteLLM Proxy in several places), Hacker News comment, published Apr 21, 2026
  60. LiteLLM | Stop Over Spending on LLMs | Build Smart AI Gateway | Demo - LiteLLM & OpenCodeKamal Krishna Bhatt, Developer (self-described: LiteLLM gateway routing OpenCode traffic across NVIDIA, OpenAI and Gemini), Video, YouTube, published Apr 18, 2026
  61. [Feature]: unlimited SSO Login with Standard Planescon1004, GitHub user (self-described: Company evaluating LiteLLM), GitHub issue, published Apr 15, 2026
  62. LiteLLM (Technology Radar, Languages & Frameworks)Thoughtworks delivery teams, Technology Radar (self-described: Thoughtworks client teams that run LiteLLM as the AI gateway for enterprise applications), Technology Radar entry, Thoughtworks, published Apr 15, 2026
  63. Comment on: N-Day-Bench - Can LLMs find real vulnerabilities in real codebases?sacrelege, Running a small inference provider (as stated) (self-described: Small private datacenter serving inference customers), Hacker News comment, published Apr 14, 2026
  64. Comment on: Reallocating US$100/Month Claude Code Spend to Zed and OpenRoutertherealpygon, Hacker News commenter (self-described: LiteLLM instance for internal users), Hacker News comment, published Apr 9, 2026
  65. Comment on: Reallocating US$100/Month Claude Code Spend to Zed and OpenRoutercobertos, Hacker News commenter (self-described: Evaluated LiteLLM for Anthropic models and MCP), Hacker News comment, published Apr 9, 2026
  66. Comment on: The Claude Code Leaklukewarm707, Hacker News commenter (self-described: Tried a local LiteLLM install), Hacker News comment, published Apr 2, 2026
  67. Focus: Stabilize core features and fix accumulated bugs/UX issuesbabanetworx, GitHub user (self-described: Proxy and UI user), GitHub issue, published Mar 31, 2026
  68. Sync convert_url_to_base64() blocks asyncio event loop, causing pod health check failures and mass restartsruettiger, GitHub user running LiteLLM on GKE (self-described: GKE Autopilot production deployment with 16 replicas), GitHub issue, published Mar 30, 2026
  69. TIL: uv Settings I Changed after LiteLLMDan O'Leary, Independent developer (self-described: Several personal projects that use LiteLLM), Personal blog post, Antisimplistic, published Mar 28, 2026
  70. Comment on: My minute-by-minute response to the LiteLLM malware attackStrauXX, Hacker News commenter (self-described: Evaluated LiteLLM for a project), Hacker News comment, published Mar 27, 2026
  71. LiteLLM supply chain compromise: what happened, impact on Giskard, and lessons learnedMatteo Dora and David Berenstein, Giskard (self-described: Giskard, AI red-teaming company whose services and open-source packages depend on LiteLLM), Engineering blog post, Giskard blog, published Mar 25, 2026
  72. LiteLLM Hack: Were You One of the 47,000?Daniel Hnyk, FutureSearch (Analysis of PyPI download logs and the 2,337 packages that depend on LiteLLM), Engineering blog post, FutureSearch blog, published Mar 25, 2026
  73. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedsschueller, Hacker News commenter (self-described: Sharing several LLMs across a set of users), Hacker News comment, published Mar 24, 2026
  74. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedchillfox, Hacker News commenter (self-described: Former LiteLLM user), Hacker News comment, published Mar 24, 2026
  75. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised0fflineuser, Hacker News commenter (self-described: Homelab proxy in Docker Compose), Hacker News comment, published Mar 24, 2026
  76. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedte_chris, Hacker News commenter (self-described: Evaluated LiteLLM source), Hacker News comment, published Mar 24, 2026
  77. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedabhikul0, Hacker News commenter (self-described: Local Python environment with browser-use), Hacker News comment, published Mar 24, 2026
  78. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedsantiagobasulto, Leading a team of developers (as stated) (self-described: Team of less senior developers), Hacker News comment, published Mar 24, 2026
  79. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedjerieljan, Hacker News commenter (self-described: Long-time LiteLLM proxy user), Hacker News comment, published Mar 24, 2026
  80. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedtreefarmer, Hacker News commenter (self-described: Former LiteLLM proxy user), Hacker News comment, published Mar 24, 2026
  81. Comment on: Malicious litellm_init.pth in litellm 1.82.8 PyPI package - credential stealerkingreflex, Hacker News commenter (self-described: Team running LiteLLM Proxy from Helm charts), Hacker News comment, published Mar 24, 2026
  82. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedddp26, Leading the FutureSearch team (as stated) (self-described: FutureSearch, research product using LiteLLM), Hacker News comment, published Mar 24, 2026
  83. litellm 1.82.8 Supply Chain Attack on PyPI (March 2026)Callum McMahon, Research Scientist at FutureSearch (self-described: FutureSearch, research product that uses LiteLLM across model providers), Engineering blog post, FutureSearch blog, published Mar 24, 2026
  84. LiteLLM Supply Chain Attack: What Happened, Who's Affected, and What You Should Do Right NowNimrod Lahav, Comet engineering (author of Comet's incident write-up) (self-described: Comet, whose Opik product depends on LiteLLM; audit of 50+ repositories and every developer machine), Engineering blog post, Comet blog, published Mar 24, 2026
  85. Malicious litellm_init.pth in litellm 1.82.8: credential stealerSimon Willison, independent developer and Datasette creator (self-described: Link-blog note on the March 2026 PyPI compromise), Personal blog post, simonwillison.net, published Mar 24, 2026
  86. Comment on: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromisedmathis-l, Hacker News commenter (self-described: Following CrewAI's dependency on LiteLLM during the March 2026 compromise), Hacker News comment, published Mar 24, 2026
  87. Comment on: Show HN: key-carousel - Key rotation for LLM agentsomrimaya, Hacker News commenter (self-described: Team that built key rotation for LLM agents), Hacker News comment, published Mar 7, 2026
  88. Comment on: Why Developers Keep Choosing Claude over Every Other AIalexsmirnov, Hacker News commenter (self-described: Claude Code across several providers with an evaluation loop), Hacker News comment, published Feb 26, 2026
  89. How Okta Deploys and Secures AI for Its EmployeesDennis Henry, Productivity Architect, Okta (self-described: Okta's company-wide AI gateway, run in its private cloud for developers calling model APIs and employees using an internal web UI), Engineering blog post, Okta blog, published Feb 26, 2026
  90. [Performance] LiteLLM Overhead and Performance in Productionjklinkenberg, GitHub user (self-described: Proxy in front of self-hosted GPU models, 4 vCPU and 8 GB VM), GitHub issue, published Feb 12, 2026
  91. Comment on: Gemini 3 Deep Thinkraybb, Hacker News commenter (self-described: Calling Gemini Deep Think), Hacker News comment, published Feb 12, 2026
  92. Updating the team/key as enterprise feature in v1.81..6.rc.1?pstanisl, GitHub user (self-described: Self-hosted proxy), GitHub issue, published Feb 5, 2026
  93. Comment on: Kimi K2.5 Technical Report [pdf]zeroxfe, Hacker News commenter (self-described: Claude Code with Kimi through a proxy), Hacker News comment, published Jan 30, 2026
  94. Comment on: Show HN: A MitM proxy to see what your LLM tools are sendingRoark66, Hacker News commenter (self-described: Recording coding assistant traffic), Hacker News comment, published Jan 29, 2026
  95. [Bug]: Significant performance regression after upgrading from 1.80.5 to 1.81.x (UI + API slowness)phanisarman, GitHub user (self-described: Gateway for 50 internal users and 200 models at about 10 RPS, with Postgres and Langfuse OTel callbacks), GitHub issue, published Jan 28, 2026
  96. Comment on: The Agentic AI Handbook: Production-Ready Patternsphotios, Hacker News commenter (self-described: Previously at a bank where only Copilot was approved), Hacker News comment, published Jan 21, 2026
  97. Comment on: Anthropic blocks third-party use of Claude Code subscriptionsmlrtime, Hacker News commenter (self-described: Claude Code with cheaper models), Hacker News comment, published Jan 9, 2026
  98. One API to Rule Them All? LiteLLM in ProductionAlina Dallmann, Speaker at PyData Berlin 2025 (self-described: Privacy-preserving university RAG chatbot for students and employees), Conference talk, PyData Berlin 2025 (PyData YouTube channel), published Nov 23, 2025
  99. Comment on: Control LLM Spend and Access with any-LLM-gatewayManouchehri, Hacker News commenter (self-described: Long-time LiteLLM user), Hacker News comment, published Nov 20, 2025
  100. Comment on: Control LLM Spend and Access with any-LLM-gatewayeverlier, Hacker News commenter (self-described: Debugging LiteLLM with Ollama), Hacker News comment, published Nov 19, 2025
  101. Comment on: Control LLM Spend and Access with any-LLM-gatewaysothatsit, Hacker News commenter (self-described: Team running LiteLLM, looking to switch), Hacker News comment, published Nov 19, 2025
  102. Comment on: Control LLM Spend and Access with any-LLM-gatewaysmcleod, Hacker News commenter (self-described: Deployed LiteLLM Proxy in a number of locations), Hacker News comment, published Nov 19, 2025
  103. Comment on: 5 Things to Try with Gemini 3 Pro in Gemini CLImalnourish, Hacker News commenter (self-described: Roo Code user), Hacker News comment, published Nov 18, 2025
  104. [Feature]: Provide non-profit licensesmenardorama, GitHub user (self-described: Non-profit and research use), GitHub issue, published Nov 12, 2025
  105. Build Production AI Agents in 2025 [With LiteLLM]Owain Lewis, AI engineering educator (self-described: Plain-Python agents built on the LiteLLM SDK), Video, YouTube, published Nov 7, 2025
  106. Comment on: Claude Code on the weblibraryofbabel, Hacker News commenter (self-described: Comparing coding models under Claude Code), Hacker News comment, published Oct 20, 2025
  107. [Bug]: Litellm Proxy availability concerns. Cascading failures during high load to providersbjornjee, ML Ops team member (as stated) (self-described: Kubernetes deployment, 2 to 5 replicas, about 60 QPS average and 500 QPS peak), GitHub issue, published Oct 14, 2025
  108. [Bug]: This feature is only available for LiteLLM Enterprise users error while trying to update a virtual keyLLukas22, GitHub user (self-described: Self-hosted proxy without a license), GitHub issue, published Oct 6, 2025
  109. Comment on: Building your own CLI coding agent with Pydantic-AIbinalpatel, Hacker News commenter (self-described: Hobby coding agent CLI), Hacker News comment, published Aug 28, 2025
  110. Comment on: Building your own CLI coding agent with Pydantic-AIziftface, Hacker News commenter (self-described: Agentic application), Hacker News comment, published Aug 28, 2025
  111. Comment on: Show HN: AgentGuard - Auto-kill AI agents before they burn through your budgetsamsk, Hacker News commenter (self-described: Agent spend control), Hacker News comment, published Jul 31, 2025
  112. Comment on: Ollama's new appjerieljan, Hacker News commenter (self-described: Personal Open WebUI setup), Hacker News comment, published Jul 31, 2025
  113. Comment on: Show HN: AgentGuard - Auto-kill AI agents before they burn through your budgettherealpygon, Hacker News commenter (self-described: Local proxy for trying editors and tools), Hacker News comment, published Jul 31, 2025
  114. Comment on: Cerebras launches Qwen3-235B, achieving 1.5k tokens per secondsneilan1, Hacker News commenter (self-described: Aider on Cerebras through a local proxy), Hacker News comment, published Jul 23, 2025
  115. Comment on: Show HN: Any-LLM - lightweight and open-source router to access any LLM Providerdlojudice, Hacker News commenter (self-described: Development environment), Hacker News comment, published Jul 22, 2025
  116. Comment on: Show HN: Any-LLM - Lightweight router to access any LLM Providerdelijati, Hacker News commenter (self-described: Experimenting with the library), Hacker News comment, published Jul 22, 2025
  117. Comment on: Show HN: Any-LLM - Lightweight router to access any LLM ProviderSzpadel, Hacker News commenter (self-described: Personal AI gateway), Hacker News comment, published Jul 22, 2025
  118. Comment on: Show HN: Any-LLM - Lightweight router to access any LLM ProviderAMeckes, Hacker News commenter, posting any-llm (self-described: Team that built its own lightweight LLM router), Hacker News comment, published Jul 22, 2025
  119. Introducing any-llm: A unified API to access any LLM providerNathan Brake, Mozilla.ai (self-described: Mozilla.ai, which built its own provider library after reviewing LiteLLM), Engineering blog post, Mozilla.ai blog, published Jul 22, 2025
  120. [Bug]: Heavy RAM Usage over timeTSJasonH, GitHub user (self-described: Team testing an application through the proxy and pass-through routes), GitHub issue, published Jul 17, 2025
  121. Show HN: Aqueduct AI Gateway, a self-hostable AI gateway without the "taxes"meffmadd, member of a small TU Vienna team and author of the Aqueduct gateway (self-described: A small TU Vienna team providing an OpenAI-compatible API to university researchers), Hacker News post (Show HN), published Jul 1, 2025
  122. Comment on: Launch HN: Reducto Studio (YC W24) - Build accurate document pipelines, fastbitdribble, Founder of DocRouter (as stated) (self-described: DocRouter, open-source document extraction product), Hacker News comment, published Jun 25, 2025
  123. LiteLLM + Gemini 2.5 Pro: cached_tokens=None crashes Agents SDK with Pydantic int-validation errorruidazeng (Ruida Zeng), GitHub user (self-described: An OpenAI Agents SDK agent running Gemini through LiteLLM), GitHub issue, openai/openai-agents-python, published May 26, 2025
  124. litellm fails to block requests over end-user budget when user header usedsammcj, GitHub user (self-described: LiteLLM proxy with per-user budgets), GitHub issue, published May 23, 2025
  125. Comment on: An upgraded dev experience in Google AI Studionprateem, Hacker News commenter (self-described: SaaS product with fallbacks across Claude, Gemini and OpenAI), Hacker News comment, published May 22, 2025
  126. Comment on: The unreasonable effectiveness of an LLM agent loop with tool usejbellis, Building the Brokk coding tool (as stated) (self-described: Brokk, AI coding product), Hacker News comment, published May 15, 2025
  127. Comment on: A critical look at MCPstuaxo, Hacker News commenter (self-described: Django app on several model backends), Hacker News comment, published May 11, 2025
  128. Comment on: Hypermode Model Router Preview - OpenRouter Alternativejbellis, Hacker News commenter, building Brokk (self-described: Brokk, an AI code tool), Hacker News comment, published May 8, 2025
  129. Comment on: AI Meets WinDBGnicovank, Hacker News commenter, co-author of the project (self-described: Debugger assistant that relies on function calling), Hacker News comment, published May 7, 2025
  130. Comment on: Gemini 2.5 Pro Preview: even better coding performancesimple10, Hacker News commenter (self-described: Local LiteLLM with Langfuse in front of Cursor), Hacker News comment, published May 6, 2025
  131. Comment on: Google Gemini has the worst LLM APImiki123211, Hacker News commenter (self-described: Structured output code across providers), Hacker News comment, published May 4, 2025
  132. Comment on: Show HN: Memex is a Claude Code alternative built on Rust+Tauri for vibe codingehsanu1, Hacker News commenter (self-described: Employer requires LLM use through its own provider contracts), Hacker News comment, published Apr 29, 2025
  133. Comment on: Show HN: Memex is a Claude Code alternative built on Rust+Tauri for vibe codinguyzeqzer, Hacker News commenter (self-described: Workplace that routes coding tools through LiteLLM), Hacker News comment, published Apr 29, 2025
  134. Comment on: xAI has acquired X, xAI now valued at US$80Bdimitri-vs, Hacker News commenter (self-described: Team app using several providers), Hacker News comment, published Mar 28, 2025
  135. Comment on: We chose LangGraph to build our coding agentscosman, Maintainer of the open-source Kiln app (as stated) (self-described: Kiln, bring-your-own-key app across 13 providers), Hacker News comment, published Mar 25, 2025
  136. I'm changing how I use AI (Open WebUI + LiteLLM)NetworkChuck (Chuck Keith), creator of the NetworkChuck YouTube channel (self-described: Shared AI hub for a team and family on Open WebUI), Video, YouTube, published Mar 13, 2025
  137. Comment on: Chatbox: Cross-platform desktop client for ChatGPT, Claude and other LLMswkat4242, Hacker News commenter (self-described: Personal setup: Open WebUI with Claude, OpenAI and Groq), Hacker News comment, published Jan 25, 2025

Jan 2025 to Oct 2026.

125 sources from practitioners, by month and sentiment.

The full readExplore all LiteLLM Signals5 praise themes, 5 concerns, 5 points of agreement and 2 disagreements across all three groups, with all 137 sources open to inspect.

Where It Fits

Where does LiteLLM belong in the stack?

Open Where It Fits: the Stack Field and the full map

LiteLLM is Core across Model, supports Build and Operate, and touches Data and Evaluate.

Business area
Product & Engineering
Function
Software Development
Operating stack
AI Application Platform
Primary role
Self-Hosted AI Gateway & Model Router
LiteLLM beside the rest of its stackThe Stack Field, every product on every layer
FootprintLiteLLM across the AI Application Platform stack. Bar height is its role on each layer.

  1. Model

    1. Model APIs: Touches

      Model APIs, touches. LiteLLM translates calls to OpenAI, Anthropic, Bedrock, Vertex AI, Azure, and 100+ other providers into one OpenAI-format API. It supplies no models. Providers bill the tokens.

    2. Model gateway: Core

      Model gateway, core. LiteLLM's MIT-licensed proxy supplies virtual keys with budgets and rate limits per key, user, team, and organization. It adds load balancing, retries, fallbacks, and spend tracking. Enterprise features require a license key.

  2. Build

    1. Orchestration: Touches

      Orchestration, touches. The Python SDK runs Claude Code and Codex agent harnesses against any model. Agent Gateway registers and invokes A2A agents. A framework such as LangGraph controls the agent's steps.

    2. Tool use: Supports

      Tool use, supports. MCP Gateway puts MCP servers behind the same keys and permissions as models. Agents reach tools through that endpoint, while the tools and integrations run elsewhere.

  3. Data

    1. Retrieval: Touches

      Retrieval, touches. LiteLLM routes /embeddings and /rerank calls to providers such as OpenAI, Cohere, and Voyage AI, with cost tracking. It stores and searches no documents.

  4. Evaluate

    1. Evaluation: Touches

      Evaluation, touches. Logging callbacks send requests and responses to tools such as Langfuse and LangSmith for tracing and evaluation. Those tools own datasets and scoring.

  5. Operate

    1. Observability: Supports

      Observability, supports. Per-request spend logs and Admin UI usage pages show cost and latency. Prometheus metrics, OpenTelemetry, and team-level logging callbacks also record calls. An observability tool handles agent-step tracing and analysis.

    2. Cloud deployment: Touches

      Cloud deployment, touches. Your team runs LiteLLM with Postgres and Redis on Kubernetes, ECS, or VMs. Official Docker images, a Helm chart, or Terraform modules handle deployment. Provider or cloud accounts control model capacity and quotas.

    3. Guardrails: Supports

      Guardrails, supports. LiteLLM proxy guardrails run on chat and message routes, with providers such as Presidio, Bedrock Guardrails, OpenAI Moderation, and its built-in content filter.

Market mapThe layers LiteLLM covers, and who else owns each one.

  1. Model gateways & routing

    CoreModel

  2. Tool use & integrations

    SupportsBuild

  3. Observability & tracing

    SupportsOperate

  4. Guardrails & AI security

    SupportsOperate

    • Prisma AIRS
    • NeMo Guardrails

  5. Model APIs

    TouchesModel

4 more layers on the full map. Core, Supports and Touches describe coverage. They say nothing about quality.

More on LiteLLM

  1. GILD ReviewGILD's evaluation, with the buying facts
  2. Research2 pieces
  3. Guides3 guides
  4. Operator Insights8 operator accounts
  5. Where It FitsOn the AI Application Platform stack
  6. Related Products6 products