GPT-4 Shows Early Signs of Artificial General Intelligence: Insights from the "Sparks of AGI" Paper

The article reports on a 154‑page Microsoft research paper called Sparks of Artificial General Intelligence: Early Experiments with GPT‑4 , which claims that GPT‑4 can be regarded as an early version of artificial general intelligence (AGI) because of its breadth and depth across many tasks.

Meta AI chief Yann LeCun expressed skepticism, noting the massive data and compute requirements of large models and questioning their learning efficiency, while the Microsoft paper counters this view by presenting extensive evaluations.

The paper highlights GPT‑4’s multimodal abilities, stating that it can understand and generate text, code, images, music, and even perform complex reasoning without special prompting. Examples include generating JavaScript code to draw Kandinsky‑style images, creating SVG graphics of objects, and composing melodies in ABC notation.

Quantitatively, GPT‑4 is reported to have roughly one trillion parameters—about six times larger than GPT‑3’s 175 billion—bringing its scale close to that of a squirrel brain, and the authors suggest continued rapid growth could soon match human brain size.

Extensive benchmarks show GPT‑4 outperforming previous models on tasks such as LeetCode programming problems, mathematics datasets (GSM8K, MATH), and high‑school to graduate‑level problem solving, often reaching or surpassing human‑level performance.

Despite these strengths, the paper acknowledges limitations: the underlying next‑token prediction architecture lacks planning, working memory, and global reasoning, leading to poor performance on simple arithmetic with larger numbers and occasional hallucinations.

Critics like Gary Marcus have challenged the AGI claim, arguing that the paper provides insufficient transparency about training data and architecture, and that the model’s shortcomings—especially in reliability and reasoning—remain unresolved.

The research team includes prominent Microsoft researchers such as Sébastien Bubeck, Ronen Eldan, Yin Tat Lee, and Li Yuan‑zhi, and the original, unredacted LaTeX source reveals an earlier title "First Contact with AGI".

References to the original arXiv preprint (2303.12712) and related online discussions are provided at the end of the article.