Tagged articles
INT8 quantization
15 articles · Page 1 of 1
HyperFlow H3 Flash compresses MiniMax-H3 from 33B to 14B parameters and 49 sampling steps to 6, reducing weight size to 7.2GB, enabling video generation with synchronized audio on consumer GPUs with 24GB VRAM while maintaining near-original quality and gaining rapid community adoption via ComfyUI integration.
ComfyUIGGUFHyperFlow
0 likes · 9 min read

FreeVideo enables the 33B MiniMax H3 video model to run on 8GB VRAM via weight streaming and INT8 quantization, but 120 community benchmarks reveal 10-second clips take 10–30 minutes on 8GB cards, making the real trade-off API cost savings versus substantial generation time and SSD wear.
Apache 2.0ComfyUIFreeVideo
0 likes · 21 min read

The author uses a $20 Gemini Pro subscription to access Google Colab's RTX PRO 6000 96GB GPU, deploys the MiniMax H3 video generation model with INT8 quantization, and benchmarks Base 20-step versus Turbo 8-step inference, achieving a 2.5× speedup with similar visual quality at 54GB peak VRAM.
ComfyUIGPU benchmarkingGemini Pro
0 likes · 9 min read

This tutorial explains FastMetal, a local text-to-video solution for Apple Silicon Macs that uses DMD2-distilled Wan models with INT8 quantization to generate 5-second videos in 30 seconds offline, covering installation, model selection, and advanced usage.
Apple SiliconDMD2FastMetal
0 likes · 19 min read

This article details Huawei's xDeepServe system for deploying massive Mixture-of-Experts models on the CloudMatrix384 supernode, covering the XCCL communication library, FlowServe decentralized serving engine, Transformerless execution architecture with prefill-decode and MoE-Attention decoupling, and hierarchical fault tolerance, achieving 2400 tokens/s per chip at 50ms TPOT.
Ascend 910CCloudMatrix384EPLB
0 likes · 18 min read

Huawei's CloudMatrix384 integrates 384 Ascend 910 NPUs with a unified bus network and decoupled PDC service architecture to achieve 4.45 tokens/s/TFLOPS prefill efficiency on DeepSeek-R1 671B, outperforming H100 baselines through fused MoE communication, INT8 quantization, and heterogeneous pipelining.
Ascend 910CloudMatrix384Expert Parallelism
0 likes · 20 min read


21CTOAug 11, 2026 · Artificial Intelligence
The article details how antirez rebuilt a zero‑dependency, high‑throughput MiniMax‑H3 video/audio inference engine entirely in C for Apple Silicon, achieving 22‑frame 512×512 video rendering in about 3.5 seconds on an M5 Max, and explains the engineering tricks, benchmark results, adjustable quality settings, terminal UI, community feedback, and upcoming sparse‑attention support.
Apple SiliconC++INT8 quantization
0 likes · 7 min read

This article details the technical challenges of adapting the open‑source vLLM inference framework to Baidu's Kunlun XPU, outlines four major performance bottlenecks, and presents a multi‑dimensional optimization roadmap—including custom plugins, operator fusion, INT8 quantization, and CUDA‑Graph techniques—that together boost throughput by up to 8% and narrow the gap with leading GPU hardware.
CUDA GraphINT8 quantizationKunlun XPU
0 likes · 13 min read

The article outlines a comprehensive, repeatable regression testing framework for TensorRT inference pipelines, covering engine build validation, functional correctness against golden outputs, performance monitoring, common pitfalls, and CI/CD integration to ensure model updates remain both fast and reliable.
INT8 quantizationMLOpsPerformance Regression
0 likes · 12 min read

This article explains how Baidu's new GLM-5 large model is adapted to the Kunlun P800 XPU, detailing the async reinforcement learning framework Slime, optimization techniques like INT8 quantization and tensor‑parallelism, and provides step‑by‑step deployment commands using the open‑source vLLM‑Kunlun plugin.
AI accelerationGLM-5INT8 quantization
0 likes · 6 min read

Meituan’s search and recommendation team converted the FP8‑only DeepSeek‑R1 model to INT8 by first casting weights to BF16 and then applying block‑wise or channel‑wise quantization, which preserves GSM8K and MMLU accuracy while delivering 33% to 50% higher throughput on A100‑80G GPUs, and they released the SGLang‑based inference scripts and quantized weights publicly, enabling deployment on older NVIDIA hardware without accuracy loss.
DeepSeek-R1GPU deploymentINT8 quantization
0 likes · 11 min read

This article explores the rapid evolution of Baidu's semantic search models, the large GPU consumption they entail, and how extensive INT8 quantization, sensitivity analysis, calibration data augmentation, hyper‑parameter auto‑tuning, and advanced methods like Quantization‑Aware Training and SmoothQuant dramatically improve inference performance while preserving business metrics.
Deep LearningERNIEINT8 quantization
0 likes · 17 min read

Baidu applied large‑scale INT8 quantization to its ERNIE search semantic models, achieving over 25% inference speedup with less than 1% degradation in relevance metrics by selectively quantizing less‑sensitive fully‑connected layers, using automated calibration, hyper‑parameter tuning, and techniques such as QAT and SmoothQuant, while paving the way for even lower‑bit quantization and token pruning.
ERNIEINT8 quantizationSemantic Search
0 likes · 15 min read

This technical walkthrough explains how Swin Transformer training and inference can be dramatically accelerated on NVIDIA GPUs by using Nsight Systems profiling, mixed‑precision tensor‑core kernels, Apex‑based and custom CUDA operator fusion, half2 vectorization, register‑array caching, and INT8 quantization, achieving up to 2.85× training and 7.34× inference speedups while preserving model accuracy.
GPU performanceINT8 quantizationNsight Profiling
0 likes · 23 min read

iQIYI optimized a 4K video super-resolution model using TensorRT, employing split of graph, operator fusion, custom CUDA kernels, and int8 quantization, achieving tenfold speedup (≈180 ms per 1080p frame) and demonstrating deep customization potential for large‑scale production.
INT8 quantizationTensorRTiQIYI
0 likes · 17 min read
