0 views collected around this technical thread.
Microsoft Research introduced BitNet b1.58 2B4T, a native 1‑bit large language model with 2 billion parameters trained on 4 trillion tokens, achieving only 0.4 GB non‑embedding memory, 0.028 J decoding energy, and 29 ms CPU latency while matching full‑precision performance.
Baidu's Search Architecture team details how its deep‑learning models have evolved to deliver direct answer results via semantic embeddings, describes a massive online inference pipeline that rewrites queries, ranks relevance, and classifies types, and outlines optimization techniques—including data I/O, CPU/GPU balancing, pruning, quantization, and distillation—to achieve high‑throughput, low‑latency search.
The article presents the background, challenges, and objectives of Bilibili's AI services, introduces the self‑developed InferX inference framework with its quantization and sparsity optimizations, details OCR‑specific enhancements, and describes how integrating InferX with Nvidia Triton dramatically improves throughput, latency, and GPU utilization.
This article presents an in‑depth overview of NLP techniques used in voice assistants, covering the end‑to‑end conversational AI pipeline, intent and slot modeling, multi‑turn dialog management, model deployment pipelines, quantization methods, and self‑learning strategies for continuous improvement.
This article reviews neural‑network model quantization, explaining why quantization is needed, detailing forward‑ and backward‑propagation issues, presenting three main mitigation strategies, discussing subsequent pruning, performance‑recovery techniques, and outlining future research avenues in efficient machine learning.