Code Archaeology
Jan 19, 2026 · Artificial Intelligence
LLM Inference Optimization: Metrics, Model & Service Techniques
This article systematically covers LLM inference optimization across performance metrics (TTFT, TPOT, throughput, goodput, MFU, MBU), model-level techniques (quantization, distillation, pruning, speculative decoding, attention optimizations), and service-level strategies (continuous batching, prefill/decode separation, prompt caching, parallelism) to improve latency, throughput, and cost-efficiency.
Inference OptimizationKV cacheLLM inference
0 likes · 14 min read
