Machine Learning Algorithms & Natural Language Processing

Oct 19, 2026 · Artificial Intelligence

AdaFlash: Adaptive Speculative Decoding with Online Self-Improvement for High-Concurrency LLM Inference

Nanjing University and Huawei's Foundation Model team introduce AdaFlash, an adaptive speculative decoding framework that enables diffusion draft models to self-improve online via on-policy distillation and dynamic verification length prediction, achieving up to 5.32x end-to-end speedup on Qwen3-8B and 66% higher throughput under high concurrency.