Does GitHub Copilot Really Copy-Paste Code? An In-Depth AI Analysis

Introduction

GitHub Copilot, built on OpenAI’s Codex algorithm, was trained on terabytes of public code from GitHub and English language examples. It claims to analyze strings, comments, function names, and existing code to generate matching snippets, supporting languages such as Python, JavaScript, TypeScript, Ruby, and Go.

Performance on LeetCode

Early users tested Copilot on LeetCode problems and reported that it consistently passed the test cases with near‑real‑time generation, leading some to claim that AI may write code better than humans. However, skeptics noted that the generated comments and templates closely resembled existing LeetCode solutions, suggesting possible prior exposure.

Copy‑Paste Allegations

Critics accused Copilot of simply “copy‑pasting” well‑known code, such as the fast inverse square‑root algorithm (magic number 0x5f3759df) and even GPL‑licensed snippets. GitHub responded that direct copying occurs in at most 0.1% of suggestions, and most output is original.

Case Classification

Albert Ziegler, a GitHub researcher, collected 453,780 Copilot suggestions for Python and categorized them into five groups:

Category 1 – Near‑duplicate suggestions after a prior accepted suggestion.

Category 2 – Long, repetitive sequences (e.g., repeated <p> tags).

Category 3 – Standard lists such as natural numbers, primes, or Greek letters.

Category 4 – Common solutions for low‑entropy tasks (e.g., using BeautifulSoup to parse Wikipedia lists).

Category 5 – Cases that match training data closely enough to be considered “recitation”.

Statistical Findings

After removing Category 1, 185 suggestions remained. Of these, 144 fell into Categories 2‑4, leaving 41 cases in Category 5, which Albert regards as true code recitation. Most of these 41 cases appeared in over 100 different files, indicating frequent reuse.

Conclusion

Albert concludes that while Copilot can occasionally copy exact code fragments, this behavior is rare and usually involves widely used snippets. He recommends that the UI indicate when a suggestion originates from training data, allowing users to attribute or reject the code accordingly.