LiteLLM Gateway Rollout Playbook

Look forRun the official image pinned by digest to 1.106.5 or a later patched release, never a floating tag such as latest or main-stable. BerriAI says dependency pins kept that image clear of the March 2026 PyPI compromise. On October 21, 2026, its own pipeline signed and published malicious 1.107.0 images, so verify the cosign signature and check that the provenance names BerriAI/litellm as the source repository. Set LITELLM_MASTER_KEY and LITELLM_SALT_KEY from a secret manager before startup. Never change the salt key once models are stored.

Look forRun Postgres for keys, teams, and spend. Add Redis when you run more than one instance. BerriAI recommends 1 vCPU and 4 GiB per pod for requests and limits, with batched spend writes every 60 seconds. Set the database connection cap to maxReplicas times workers. Under BerriAI's shared responsibility model, your team owns database health and gateway uptime.

Look forMap business units to organizations and groups to teams. Give each person or app a virtual key with allowed models, max_budget, and budget_duration. Keep provider keys in the gateway, so revoking one virtual key cuts that user off from every model.

Look forSSO is free for up to 5 users from v1.76.0. Past that, SSO needs an Enterprise license, as do SCIM and audit logs. One team hit the 5-user limit during its evaluation. Without a license, issue keys from your own internal tool through LiteLLM's key management API.

Look forSet each tool's base URL and key to the gateway and name models by purpose so the backend can change. Test each tool first. Developers report Claude Code failing through LiteLLM on new Anthropic headers, and Codex working only with the Responses API turned off.

Look forConfigure multiple deployments under one model name. Set RPM and TPM limits per deployment, with retries and a fallback model. Use Redis to share routing state across instances. Force each fallback in a test. One team reported readiness probes failing across Kubernetes pods when a provider returned 429s.

Look forLiteLLM ships a stable minor about weekly and supports the four latest minor lines. Pin a line, take its patches, and load-test each move to a new line against fixed baselines in CI. Pfizer's load test caught v1.89.2 halving throughput with an unchanged error rate, and a GitHub issue reports a 50-user gateway slowing across the UI and API after a minor upgrade.