Skip to content
Tagged articles

TPOT

3 articles · Page 1 of 1
Code Archaeology
Code Archaeology
Jan 19, 2026 · Artificial Intelligence

LLM Inference Optimization: Metrics, Model & Service Techniques

This article systematically covers LLM inference optimization across performance metrics (TTFT, TPOT, throughput, goodput, MFU, MBU), model-level techniques (quantization, distillation, pruning, speculative decoding, attention optimizations), and service-level strategies (continuous batching, prefill/decode separation, prompt caching, parallelism) to improve latency, throughput, and cost-efficiency.

Inference OptimizationKV CacheLLM inference
0 likes · 14 min read
LLM Inference Optimization: Metrics, Model & Service Techniques
Baidu Geek Talk
Baidu Geek Talk
Jan 15, 2025 · Artificial Intelligence

Understanding Large Model Inference Engines and Reducing Token Interval (TPOT)

Large‑model inference engines convert prompts into responses via a Prefill stage and an autoregressive Decoder, measured by TTFT and TPOT, and Baidu’s AIAK suite improves TPOT by separating tokenization, using static slot scheduling, and asynchronous execution, cutting token‑interval latency from ~35 ms to ~14 ms and boosting GPU utilization to about 75 % while also leveraging quantization and speculative execution for higher throughput.

AI accelerationGPU utilizationLarge Model Inference
0 likes · 10 min read
Understanding Large Model Inference Engines and Reducing Token Interval (TPOT)
Python Programming Learning Circle
Python Programming Learning Circle
Oct 25, 2022 · Artificial Intelligence

Genetic Algorithms: Theory, Steps, and Practical Implementation with TPOT for Data Science

This article introduces genetic algorithms, explains their biological inspiration, details each step of the algorithm, demonstrates solving the knapsack problem, and provides a complete Python implementation using the TPOT library for feature selection and regression on the Big Mart Sales dataset.

PythonTPOTfeature selection
0 likes · 19 min read
Genetic Algorithms: Theory, Steps, and Practical Implementation with TPOT for Data Science