How AI Hallucinations Fuel Fake NPM Package Attacks and What You Can Do

Background

Large language models like ChatGPT often produce hallucinations —fabricated information that looks plausible. When developers rely on AI‑generated code or package recommendations, they may unknowingly trust these false answers, including bogus URLs or package names.

Attack Technique: AI Package Hallucination

Attackers exploit this by identifying AI‑generated package references that do not actually exist. After the AI suggests a non‑existent npm package, the attacker quickly registers the corresponding domain or publishes a malicious package under that name. Subsequent AI responses will then return the attacker‑controlled link, leading unsuspecting users to install malware.

Experimental Procedure

The researchers queried ChatGPT for a Node.js package (e.g., arangodb ) and asked for the npm install command. ChatGPT returned a package name that did not exist in the npm registry. Repeating the query for additional packages yielded dozens of such fake entries (over 50 fake npm packages from 200+ Node.js questions and over 100 fake Python packages from 227 questions).

To demonstrate the full attack chain, they:

Asked ChatGPT for a non‑existent npm package.

Published a malicious package with that name to the npm registry.

Configured the package’s preinstall script to execute node index.js on the victim’s machine.

In index.js, collected system information (hostname, module paths, etc.) and sent it to a remote server.

When a user runs npm install arangodb, the malicious pre‑install script runs silently, harvesting data without the user noticing.

Mitigation Recommendations

Because the attack hinges on trusting AI‑generated suggestions, the authors advise:

Never install a package solely based on an AI response; verify the package’s existence on the official registry.

Check the package’s release date, GitHub repository, star count, and download statistics before installing.

Use package‑verification tools or lockfiles to ensure only vetted dependencies are added.

By maintaining a healthy skepticism toward AI‑generated code and performing basic due‑diligence, developers can reduce the risk of supply‑chain compromises caused by AI hallucinations.