ME-Decoding: Reframing LLM Decoding as Ensemble Pruning with Semantic Redundancy Control
Renmin University researchers propose ME-Decoding, a method that selects candidate tokens by jointly optimizing probability confidence and semantic diversity via Mahalanobis distance, outperforming probability-only truncation on reasoning and open-ended generation across multiple models and temperatures with only 3% latency overhead.
