When AI Code Assistants Leak Fake IDs: What GitHub Copilot’s Slip Reveals

AI Code Completion Generates a Fake ID

GitHub Copilot, powered by the Codex model (an upgrade of GPT‑3), recently produced a Chinese identity‑card number when a user typed the name of Bilibili CEO Chen Rui. The number looked plausible but contained obvious errors (the birth year was 1988 instead of 1978), confirming it was synthetic data.

This incident raised alarm about the possibility of AI tools leaking personal information.

Why Does This Happen?

Copilot’s underlying language model is trained on massive amounts of public internet data, which inevitably includes personal details such as names, addresses, and ID numbers. During generation the model can “remember” fragments of its training set and unintentionally “spit out” that information.

GitHub’s CEO Nat Friedman has stated that any private data produced by Copilot is fabricated, synthesized from the training corpus, not retrieved from a real database.

Broader Risks and Controversies

Beyond privacy concerns, Copilot has faced criticism for copying code without proper licensing, generating biased or offensive outputs, and being offered as a paid service despite being trained on publicly available repositories.

The Free Software Foundation has protested the tool’s licensing model, and developers have voiced worries that sensitive data may still slip through.

Industry observers, including Xiaomi’s Vice President Cui Baoqiu, advise users to anonymize personal data and remain vigilant about AI‑driven privacy risks.

Overall, the episode highlights the need for clearer standards and safeguards when training and deploying large language models for code assistance.

https://twitter.com/DeltonDing/status/1423651446340259840

https://venturebeat.com/2021/07/08/openai-warns-ai-behind-githubs-copilot-may-be-susceptible-to-bias/

https://www.infoworld.com/article/3627319/github-copilot-is-unacceptable-and-unjust-says-free-software-foundation.html