When AI Code Completion Leaks Fake ID Numbers: Copilot’s Privacy Risks

AI auto‑completion generated a fake ID number, causing surprise among users.

GitHub Copilot displayed a fabricated ID for Bilibili CEO Chen Rui after the user entered his name.

Observers quickly pointed out that the ID was false: the birth year and checksum were incorrect, showing the data was synthetic.

What the model eats, it may spit out

GitHub Copilot is powered by the Codex model, an upgraded version of GPT‑3 that can understand both code and natural language.

To interpret comments, Codex undergoes language training similar to GPT‑3, and during generation it can randomly exhibit characteristics of its training data.

Large language models are trained on massive public datasets that inevitably contain personal sensitive information such as names, addresses, and ID numbers.

Consequently, the model may “remember” such data and reproduce it when prompted, creating synthetic personal details.

GitHub CEO Nat Friedman has responded that any privacy‑related output from Copilot is fake, synthesized from training data rather than extracted from real records.

Continuous controversy surrounding GitHub Copilot

Since its launch, Copilot has faced criticism for copying source code without proper licensing, using public code repositories while being a paid product, and generating biased or unethical content.

The Free Software Foundation (FSF) has protested that Copilot requires paid IDEs, infringing on user rights.

GitHub has expressed openness to discussion and aims to help establish standards for training AI models.

Experts, including Xiaomi’s open‑source committee chair, warn users to protect their privacy and anonymize personal data.

Overall, the incident reveals privacy risks in AI‑driven code assistants and underscores the need for careful handling of sensitive information.