Tagged articles
FP8 quantization
11 articles · Page 1 of 1
Peking University researchers release OmniEdu, an open-source K-12 educational foundation model family (4B, 9B, 27B) built on Qwen3.5/3.8, featuring curriculum alignment, diagnostic reasoning, and pedagogical scaffolding, with full paper, weights, training data, code, and evaluation toolkit, outperforming base models on education benchmarks while retaining general capabilities.
FP8 quantizationK-12 educationOmniEdu
0 likes · 14 min read

This comprehensive guide covers production deployment and performance tuning of LLM inference services using vLLM and TGI, including hardware planning, Docker/Kubernetes orchestration, key parameter optimization, monitoring with Prometheus, autoscaling with KEDA, troubleshooting common failures, and security hardening.
CUDA GraphDockerFP8 quantization
0 likes · 99 min read

Community clues suggest Alibaba's Qwen 4 is near release, with inference optimizations in vLLM and llama.cpp, architectural details like a 27B dense model with 5120 hidden dim, 24 query heads, 4 KV heads, 4 residual streams, MTP, and QSA sparse attention, while anonymous Arena models Kalmia and Kerria are speculated as Qwen 4 variants showing strong code and 3D generation.
AlibabaArena benchmarkFP8 quantization
0 likes · 7 min read

Researchers from Peking University, Tsinghua University, and Alibaba introduce SparkDiffusion, a unified framework combining sparse attention, few-step distillation, and FP8 quantization to achieve 265× acceleration for DiT video generation on a single RTX 5090, overcoming the high-sparsity trap that degrades quality at extreme sparsity.
DiTFP8 quantizationacceleration
0 likes · 12 min read

DeepSeek-V4.1-Flash open-sourced with CED architecture and 8B/16B activated parameters outperforms its 1.6T predecessor V4-Pro on coding benchmarks, approaches GPT-6 Astra on DeepSWE, but requires 510GB FP8 weights needing 8×H200 for full-context local deployment; author tests across six harnesses finding Claude Code/Codex integration near peak performance.
AI model evaluationCED architectureDeepSWE
0 likes · 8 min read

DiffusionGemma, Google DeepMind’s 26B MoE model that generates 256‑token blocks via diffusion, achieves over 1000 tokens per second on H100/H200 GPUs, offers FP8 and NVFP4 quantized versions with near‑lossless accuracy, and can be deployed locally with vLLM Docker images, though it incurs higher first‑token latency and limited concurrency.
26B modelDiffusionGemmaFP8 quantization
0 likes · 10 min read

This article walks through deploying DeepSeek‑V4‑Flash on a server with two NVIDIA H20 GPUs (96 GB each), detailing model download, Docker image preparation, launch script tweaks, memory compression via FP8 and expert parallelism, and reports observed concurrency limits and token‑per‑second speeds, including a test that disables the model's thinking mode.
DockerFP8 quantizationGPU deployment
0 likes · 6 min read

DeepSeek has released TileKernels, a GPU kernel library written in the TileLang DSL, that targets H100/H200/B200 GPUs and claims to approach hardware limits in compute intensity and memory bandwidth, offering MoE routing, FP8/FP4 quantization, and dual‑language PyTorch references for deep‑learning engineers.
FP8 quantizationGPU optimizationLLM training
0 likes · 9 min read

This article walks through the DeepSeek‑V3.2‑Exp inference codebase, detailing its MoE architecture, Multi‑Head Latent Attention, FP8 quantization, custom CUDA kernels, and 8‑GPU NCCL‑based distributed execution from initialization through prefill and decode stages.
CUDAFP8 quantizationLLM
0 likes · 9 min read

DeepSeek‑Prover‑V2‑671B, a 671 billion‑parameter AI model released on Hugging Face, dramatically advances formal mathematical theorem proving with MoE architecture, FP8 quantization, 163 k token context, superior performance over GPT‑4 Turbo and other models, and broad implications for research and industry.
AIDeepSeekFP8 quantization
0 likes · 11 min read

This guide explains how to prepare an Alibaba Cloud GPU instance, install Docker and NVIDIA tools, pull or build a container image, and run the FP8‑quantized DeepSeek‑R1‑Distill‑Qwen‑32B model using vLLM and OpenWebUI for both offline and online inference.
DeepSeekFP8 quantizationGPU
0 likes · 18 min read
