Tagged articles
FP8
40 articles · Page 1 of 1
llama.cpp 1.0.0 removes 35 deprecated functions from its public C API, adopts semantic versioning where only breaking API changes trigger major versions, adds FP8 KV cache support, MoE CPU offloading, new model integrations including Kimi-K3.1-Pro and drama-3, and provides pre-built binaries for 150+ architecture variants across major platforms.
C APIFP8KV Cache
0 likes · 5 min read

vLLM's PR #63199 introduces a dedicated FP8 W8A8 Blockscale GEMM kernel for AMD's gfx1151 (RDNA 3.5) architecture, accelerating specific small-batch matrix multiplications by 1.70–2.54x and improving end-to-end generation throughput by ~6.8% on Ryzen AI MAX+ 395, though the PR remains in draft and covers only a subset of FP8 operations.
AMDFP8GEMM
0 likes · 8 min read

vLLM 0.32.0 introduces ROCm Segmented Attention backend and W4A16 kernel optimizations for AMD Ryzen AI MAX+ 395 (gfx1151), delivering up to 2.5x attention throughput and 22% TTFT reduction, though FP8 support remains limited without native hardware acceleration.
AMD Ryzen AI MAX+ 395FP8LLM Inference Optimization
0 likes · 9 min read

The author covers Alibaba Qwen team's upcoming Qwen4 family with open-source plans, surveys recent global AI model releases, notes a slowdown in Chinese model updates, shares hands-on testing of Qwen3.8-27B quantization variants on A100 and 4090 GPUs, and recommends FP8 for dual-4090 and GGUF UD-Q4+ for single-4090 setups.
FP8GGUFLLM Deployment
0 likes · 4 min read

Alibaba Cloud's Panjiu AL64 super node integrates the Zhenwu V900 AI chip, ICN Switch, Panmai smart NIC, and Zhenyue SSD controller to enable 500,000-GPU cluster scaling via a three-tier interconnect architecture and liquid-cooled orthogonal design.
AI super-nodeAlibaba CloudFP4
0 likes · 9 min read

Moore Threads unveils the MTT C256 supernode at WAIC 2026, packing 256 GPUs into a single Scale-up network layer with compute-switch integration, breaking the 64-GPU limit. The MTT S5000 chip delivers 95,920 tok/s FP8 throughput, native FP8 KV Cache, and CUDA-source-level MUSA compatibility for heterogeneous prefill/decode inference.
FP8MTT C256MTT S5000
0 likes · 6 min read

The article details how the OrcaRouter team applied abliteration to Qwen3.8‑27B, removed the model's refusal direction, re‑quantized it to FP8, benchmarked unchanged general abilities but near‑zero refusal rates, and provides download links and deployment instructions for red‑team and safety research.
AbliterationFP8Qwen3.8
0 likes · 10 min read


21CTOAug 4, 2026 · Artificial Intelligence
DeepSeek has begun IPO preparation targeting a 2027 listing, possibly as early as year‑end, backed by a $1.5 billion financing round that lifts its valuation to $71 billion, while its founder retains roughly 78% of equity and the company showcases a self‑developed, cost‑efficient AI stack.
AIDeepSeekDualPipe
0 likes · 7 min read

The article provides a thorough cost analysis for locally deploying the GLM‑5.2 large language model, detailing hardware configurations, FP8 and BF16 precision options, single‑node versus dual‑node setups, memory requirements, and why regulated finance firms are the primary candidates for such an investment.
AI infrastructureBF16FP8
0 likes · 7 min read

This guide explains how to evaluate INT8, FP8, and INT4 quantization strategies for large language models on NVIDIA GPUs, covering precision trade‑offs, memory consumption, kernel support, KV‑Cache considerations, and detailed deployment, testing, and rollback procedures to ensure performance and quality.
FP8GPU deploymentINT4
0 likes · 48 min read

MiniMax Sparse Attention (MSA) is an open‑source library that delivers high‑performance dense and block‑sparse attention operators for NVIDIA SM100 GPUs by combining a Jinja‑based csrc JIT stack with a Cutlass Python DSL (CuTe‑DSL), enabling low‑precision quantization, paging support, and seamless migration from dense code.
AI kernelsCuTe-DSLCutlass
0 likes · 5 min read

The 2026 analysis shows China's GPU market capturing a growing share of the $1.12 trillion global AI GPU market, with Huawei Ascend leading at 44%, domestic firms leveraging 7nm processes, Chiplet and FP8 breakthroughs, while Nvidia and AMD face increasing competition from Chinese players expanding into inference, edge and enterprise segments.
AI chipsChinaChiplet
0 likes · 5 min read

The article analyzes Huawei's Ascend AI chip evolution from the 910C baseline through the 950 series' low‑precision FP8/FP4 breakthrough to the 960/970 generation’s 8 PFLOPS performance, highlighting architectural innovations, memory and interconnect upgrades, scenario‑specific models, and a cost advantage over competing solutions.
AI chipAscendFP8
0 likes · 6 min read

The article details how DeepSeek V4 became the first top‑level large model to run on Huawei's Ascend 950 PR chip, delivering up to 2.87× the performance of Nvidia H20, cutting inference cost by up to 90%, and spurring a booming domestic AI‑chip ecosystem and supply‑chain surge.
AI chip performanceAI inferenceCANN Next
0 likes · 10 min read

DeepSeek‑V4, a 1.6‑trillion‑parameter MoE model with mixed‑precision attention, is benchmarked on three accelerators—NVIDIA H100, Huawei Ascend 910C, and Ascend 950PR—showing that the 950PR delivers the lowest per‑token cost in both Prefill and Decode phases, while the H100 offers the highest raw performance at a far greater price.
FP8Huawei Ascend 950PRMoE
0 likes · 8 min read

The article compiles key information on DeepSeek V4, covering Ollama's one‑click launch, the model's FP4/FP8 mixed‑precision quantization, size reductions, high local deployment costs, recent benchmark rankings, and the accompanying stock price movements in both China and the US.
AI benchmarksFP4FP8
0 likes · 5 min read

The newly released Qwen3.6-27B dense multimodal model, at just 27 B parameters, surpasses the 397 B flagship on most encoding benchmarks, offers up to 1 M token context, supports FP8 quantization, and can be deployed locally via vLLM, SGLang or Transformers with modest hardware.
27BDense ModelFP8
0 likes · 12 min read

The article walks through upgrading vLLM to 0.17.0, configuring Docker containers for 4090 GPUs, comparing FP8 and 4‑bit quantization of Qwen3.5 35B and 27B models, and presents detailed performance numbers and script parameters that reveal trade‑offs in memory usage and throughput.
4-bit quantizationDockerFP8
0 likes · 7 min read

Qwen3.5‑397B‑A17B, the newly open‑sourced multimodal giant, combines a 400‑billion‑parameter sparse MoE architecture with FP8 pipelines and an asynchronous RL framework to deliver GPT‑5.2‑level capabilities, 60% lower memory usage, up to 19× higher throughput, and extensive image, video, and agent support, while outlining its deployment requirements and API pricing.
AI inferenceFP8Multimodal Model
0 likes · 11 min read

This article walks through practical steps to speed up LTX‑2 AI video generation—enabling the NVFP4 model, updating NVIDIA drivers and CUDA, using FP8 text encoders, and applying a custom prompt‑optimizing assistant—showing memory savings, sub‑minute rendering at 1280×720, and noticeable quality gains.
AI Video GenerationFP8LTX-2
0 likes · 11 min read

AWS's new Trainium3 chip, built on a 3nm process with FP8 performance up to 2.52 PFLOPs, promises massive compute gains, lower costs, and a new cloud‑centric AI ecosystem, challenging Nvidia's dominance and reshaping the AI hardware market.
3nmAI hardwareAWS
0 likes · 12 min read

This comprehensive analysis examines the strategic importance, technical challenges, innovation pathways, and market landscape of domestic AI chips in China, highlighting key players, regional clusters, core applications such as intelligent computing, autonomous driving, and robotics, and projecting future industry bottlenecks and opportunities.
AI chipsAutonomous DrivingChina semiconductor
0 likes · 18 min read

The article explains how DeepSeek’s Lightning Indexer acts as a memory‑filtering expert that computes index scores, selects the top‑k relevant tokens, and maps a compact formula to FP8 kernel code, reducing attention complexity from 128K to 2048 tokens for massive sequences.
DeepSeekFP8Lightning Indexer
0 likes · 7 min read


AntTechOct 9, 2025 · Artificial Intelligence
Ling-1T, a trillion‑parameter flagship non‑thinking model, combines 50 billion active parameters per token, 128 K context, Evo‑CoT reasoning, and FP8 mixed‑precision training to achieve state‑of‑the‑art performance on complex reasoning, code generation, and multimodal tasks while outlining its architecture, benchmarks, limitations, and future roadmap.
AIFP8LLM
0 likes · 11 min read

Baidu’s Baige 5.0 AI Computing Platform introduces FP8 mixed‑precision training, MoE‑aware distributed strategies, adaptive parallelism, and a three‑tier KV‑Cache, delivering over 30% training speedup and 50% inference throughput gains while keeping token latency under half a second for large‑scale models.
AIFP8MoE
0 likes · 16 min read

This article explains how integers and decimals are stored using fixed‑point and floating‑point representations, introduces the 8‑bit FP8 formats (E4M3 and E5M2) used in modern AI hardware, and traces the evolution from classic FP32/FP64 to the latest ultra‑compact numeric types.
AI hardwareFP8Fixed-Point
0 likes · 8 min read

DeepSeek‑V3.1 introduces the UE8M0 FP8 Scale precision, cutting memory usage by up to 75% and enabling next‑generation Chinese chips such as Ascend 910B to run 128K context models efficiently, while the ecosystem rapidly adopts FP8, yet challenges in IP autonomy and software maturity remain before global competitiveness is achieved.
AI hardwareDeepSeekFP8
0 likes · 10 min read

This article explains the meaning of UE8M0 by introducing fixed‑point (INT8) and floating‑point representations, showing how integers and decimals are stored in binary, describing the limitations of fixed‑point, the advantages of floating‑point scientific notation, and detailing the emerging FP8 formats such as E4M3 and E5M2 used in modern AI hardware.
AI hardwareFP8Fixed-Point
0 likes · 8 min read

The article explains the UE8M0 FP8 precision format, its MXFP8 origins, how it reduces bandwidth and power consumption, and why Chinese AI chip makers like Cambricon, HaiGuang and Moore Threads are rapidly adopting it, signaling a shift toward domestic AI hardware independence.
AI hardwareChinese chipsDeepSeek
0 likes · 10 min read

This article provides a detailed technical analysis of DeepSeek‑V3, covering its MOE architecture, the novel Multi‑head Latent Attention (MLA) mechanism, the DualPipe pipeline‑parallel algorithm, mixed‑precision FP8 training, and the Multi‑Token Prediction (MTP) inference improvements that together boost performance and efficiency.
DeepSeekDualPipeFP8
0 likes · 44 min read

This article presents a comprehensive benchmark of DeepGEMM, Cutlass, and Triton on NVIDIA H20 and H800 GPUs, analyzing TFLOPS, bandwidth, latency, and speedup across various matrix sizes, and concludes which library is optimal for different workload scenarios.
CUDADeepGEMMFP8
0 likes · 15 min read

This article provides a detailed technical analysis of FP8 training, comparing Nvidia’s TransformerEngine approach with DeepSeek V3’s novel scheme, and examines how block‑wise scaling, high‑precision accumulation, and vector length and correlation affect quantization error and signal‑to‑noise ratio in large‑language‑model training.
DeepSeekFP8LLM
0 likes · 20 min read

The article provides an in‑depth overview of DeepSeek’s large language model, detailing its mixture‑of‑experts and Transformer foundations, novel attention mechanisms, load‑balancing, multi‑token prediction, FP8 mixed‑precision training, and various training regimes such as knowledge distillation and reinforcement learning.
DeepSeekFP8MLA
0 likes · 18 min read

Thirteen engineers praise DeepSeek’s open‑source, reinforcement‑learning‑driven architecture—using FP8 storage and SFT‑free training—to deliver GPT‑4‑level reasoning at one‑twentieth the cost, enabling single‑GPU deployment, lowering barriers for academia and startups, and prompting notable market reactions that could democratize advanced AI.
AI cost reductionDeepSeekFP8
0 likes · 9 min read

DeepGEMM is an open‑source FP8‑precision GEMM library that delivers up to 1350 TFLOPS on NVIDIA Hopper GPUs, offering JIT‑compiled, lightweight code (~300 lines) for dense and MoE matrix multiplication, with easy deployment, configurable environment variables, and performance advantages over CUTLASS for large AI models.
AI accelerationDeepGEMMFP8
0 likes · 7 min read

The article explains the performance benefits of using FP8 for large‑model training, outlines three main categories of FP8‑related issues such as loss spikes, divergence, and downstream metric gaps, and introduces a dedicated FP8 debug tool with metrics like MSE, cosine similarity, underflow, and overflow to help diagnose and resolve these problems.
AIDebuggingFP8
0 likes · 9 min read

The article examines DeepSeek‑V3’s architecture and training pipeline, highlighting its use of MLA and a highly granular MoE design, pioneering FP8 mixed‑precision training, fine‑grained per‑tile quantization, advanced parallelism strategies, and inference optimizations such as PD separation and NanoFlow to achieve unprecedented efficiency on limited GPU resources.
DeepSeek-V3FP8Inference Optimization
0 likes · 10 min read

This article introduces the open‑source Pai‑Megatron‑Patch tool from Alibaba Cloud, explains its non‑intrusive patch architecture, enumerates supported models and features such as weight conversion, Flash‑Attention 2.0, FP8 training with Transformer Engine, and provides detailed command‑line examples for model conversion, pre‑training, supervised fine‑tuning, inference, and RLHF reinforcement learning pipelines.
FP8LLMMegatron
0 likes · 19 min read

This article introduces Pai‑Megatron‑Patch, an open‑source tool from Alibaba Cloud that streamlines large language model (LLM) training, weight conversion, FP8 mixed‑precision acceleration, and reinforcement‑learning workflows, providing detailed architecture, key features, code examples, and step‑by‑step usage instructions.
FP8LLM trainingMegatron
0 likes · 19 min read

The article provides a detailed technical analysis of NVIDIA's Hopper‑based H100 GPU, covering its 4 nm process, 800 billion transistors, GPC/TPC hierarchy, new FP8 Tensor Cores, Transformer engine, Tensor Memory Accelerator, and the resulting six‑fold performance jump over the previous A100 generation.
AI accelerationFP8GPU architecture
0 likes · 8 min read
