How a Supply‑Chain Poisoning of LiteLLM Exposed Critical AI API Secrets – and What to Do

On March 24, 2026 a supply‑chain poisoning attack targeted LiteLLM, a popular open‑source AI‑model proxy framework with over 95 million monthly downloads. Malicious versions v1.82.7 and v1.82.8 were published on PyPI, embedding a hidden litellm_init.pth file that executes silently when the package is installed.

Attack Timeline

Mar 24 : Security monitoring first detected anomalous activity; two suspicious LiteLLM releases appeared on PyPI.

Mar 25 : Multiple security vendors issued alerts confirming that versions v1.82.7 and v1.82.8 contain malicious code.

After discovery : PyPI removed the affected packages; the LiteLLM team halted new releases and began a full investigation.

Technical Details of the Poisoning

The threat group “TeamPCP” first compromised the CI/CD pipeline of the open‑source scanner Trivy, stole the PyPI publishing credentials of the LiteLLM maintainers, and injected the payload into the release tarball, bypassing normal code review.

The payload consists of two parts: litellm_init.pth – a Python .pth file that is automatically loaded by the interpreter at startup, requiring no explicit import. When a developer runs pip install litellm, the file is executed and silently exfiltrates data.

Modified proxy/proxy_server.py – additional malicious logic that further hides the exfiltration.

Once activated, the code scans the host environment for a wide range of secrets and sends them to the attacker’s server models.litellm.cloud using a hybrid encryption scheme: the data are first encrypted with AES‑256‑CBC, the symmetric key is derived via PBKDF2, and the key itself is encrypted with RSA‑4096 before transmission over HTTPS.

Collected secrets include:

Environment variables and .env files containing API keys for OpenAI, Anthropic, Azure, etc.

SSH private keys.

Cloud provider credentials (AWS, Azure, GCP).

Kubernetes secrets, Docker configs.

Git configuration ( .gitconfig) and stored credentials.

Shell history files that may contain plaintext passwords.

Encrypted cryptocurrency wallets and mnemonic phrases.

Why LiteLLM Became the Prime Target

LiteLLM acts as a unified proxy for multiple large‑model APIs, meaning it must handle every API key supplied by developers, typically stored in local .env files or environment variables. By compromising the proxy itself, attackers gain unrestricted access to all those keys, effectively turning the proxy into an unlocked vault.

Countermeasure: Alibaba Cloud AI Gateway

Alibaba Cloud proposes an AI Gateway that moves key management and API routing to a managed cloud service, eliminating the need for a locally deployed third‑party proxy. The gateway stores credentials in a KMS‑backed vault, authenticates clients via JWT, and never exposes static keys to the client environment.

Key security benefits include:

Zero exposure of API keys on the client side.

No requirement to install third‑party packages, removing the supply‑chain entry point.

Built‑in AI security guardrail that scans requests and responses for API keys, PII, malicious URLs, jailbreak prompts, and other policy violations.

End‑to‑end encryption, real‑time threat detection, and compliance checks.

Full traffic observability, automatic fallback, load‑aware routing, and serverless auto‑scaling.

For organizations already using LiteLLM, the article lists immediate remediation steps:

Check the installed version; if it is v1.82.7 or v1.82.8, roll back to a safe version (e.g., v1.82.6).

Search the Python site‑packages directory for litellm_init.pth and proxy/proxy_server.py and delete any modified files.

Rotate all compromised credentials – API keys, SSH keys, cloud service keys, and database passwords.

Audit network logs for outbound connections to models.litellm.cloud.

Run a comprehensive security scan to detect any persistent backdoors.

Consider migrating to a managed AI gateway to eliminate the local proxy attack surface.

Future Outlook

The LiteLLM incident demonstrates a new “one‑fish‑multiple‑bites” supply‑chain threat model, where attackers compromise upstream dependencies (e.g., Trivy) to poison downstream projects. As large‑model applications proliferate, similar attacks are expected to become more sophisticated and harder to detect, underscoring the need for architecture‑level security controls rather than reliance on cautious package installation.

In summary, protecting AI workloads requires moving secret management to dedicated cloud services, employing real‑time AI‑specific security guardrails, and ensuring full observability of traffic – a strategy embodied by Alibaba Cloud’s AI Gateway.