AI Frontier Lectures

Nov 4, 2026 · Artificial Intelligence

SpecPilot: 10.87x LLM Inference Speedup via Cost-Model Guided Speculative Decoding

NVIDIA, UCB, Princeton, and Stanford introduce SpecPilot, a cost-model guided speculative decoding system that achieves 10.87x speedup over autoregressive decoding for Qwen3.8-27B on RTX PRO 6000, using 12-bit weight format, tree verification, and two-level verification to optimize per-token verification and batch scheduling.