Microsoft’s Phi‑3 Mini: The Smallest LLM That Beats GPT‑3.5 on iPhone

On April 23, Microsoft announced the open‑source Phi‑3 series, a family of lightweight large language models (LLMs) that it claims are the most powerful and cost‑effective small language models (SLMs) currently available.

The smallest model, Phi‑3‑mini, has only 3.8 billion parameters but outperforms much larger models in benchmark tests, surpassing Meta’s Llama 3 8B and even beating GPT‑3.5 Turbo when compared with the larger Phi‑3‑small and Phi‑3‑medium variants.

Phi‑3‑mini’s memory footprint is tiny; after compression it occupies about 1.8 GB and can run on an iPhone 14 with the A16 Bionic chip at roughly 12 tokens per second, enabling fully offline inference. Microsoft says the operating cost is roughly one‑tenth that of comparable performance models.

The model is designed for scenarios with limited network resources, low latency requirements, or tight cost constraints, making it attractive for edge devices and offline applications. Microsoft cites an Indian agritech partnership where Phi‑3 helps farmers obtain localized advice without internet connectivity.

While the series excels in reasoning, coding, and multilingual tasks, its factual knowledge lags behind larger models on benchmarks such as TriviaQA. Microsoft mitigates this by integrating search‑engine‑based retrieval‑augmented generation (RAG) to supplement missing facts.

Phi‑3 uses a transformer architecture with 4 K and 128 K context windows—the first open‑source SLM to support a 128 K window. Training involved a curated 3.3 trillion‑token dataset comprising high‑quality web documents, educational material, code, and synthetic data, followed by instruction fine‑tuning and RLHF.

According to Microsoft executives, the reduced size and cost (up to ten times cheaper than comparable models) open new possibilities for AI‑enabled devices, including upcoming AI‑PCs and edge AI initiatives.