This comprehensive survey examines the foundations, tokenization techniques, model architectures, training paradigms, evaluation benchmarks, and open challenges of multimodal next‑token prediction (MMNTP), offering researchers a clear roadmap for future advances in multimodal AI.
The article argues that a large language model's ability to compress data through next‑token prediction reflects its intelligence, reviews theoretical and empirical evidence linking compression efficiency to model scale, and proposes a circuit‑competition framework to explain emergent capabilities, in‑context learning, and fine‑tuning effects.
This article examines the debate over whether large language models truly possess intelligence, arguing that next‑token prediction functions as a form of lossless data compression whose efficiency reflects intelligence, and it surveys research on knowledge extraction, neuron semantics, circuit competition, scaling effects, and the broader philosophical implications of GPT as a mirror of the world’s parameters.
