BestHub
Sign in
Tagged articles

NVFP4

12 articles · Page 1 of 1
Old Zhang's AI Learning
Old Zhang's AI Learning
Oct 7, 2026 · Artificial Intelligence

AI-Generated Inference Engine Beats vLLM by 90% at 1,792 tokens/s

Baseten engineers used Claude Code to auto-generate a custom inference engine (VibeQwen) for Qwen-3.6-35B-A3B on B200, achieving 1,792 tokens/s decode (90% faster than vLLM), 12ms TTFT, and 10,307 tokens/s throughput at concurrency 32, via MetaInfer's LLM-as-Compiler loop with accuracy oracle and reusable optimizations, also speeding up SAM 3.1 by 50% in two days.

AI-generated codeB200 GPUClaude Code
0 likes · 11 min read
AI-Generated Inference Engine Beats vLLM by 90% at 1,792 tokens/s
Old Zhang's AI Learning
Old Zhang's AI Learning
Aug 23, 2026 · Artificial Intelligence

Qwen3.8-27B Quantization Selection Guide: Match Your Hardware to the Right GGUF Version

This guide analyzes Unsloth's Dynamic v3.0 quantizations of Qwen3.8-27B, showing Mean KLD divergence across versions, recommending UD-Q4_K_XL for 24GB GPUs, detailing hardware requirements, sampling parameters for thinking modes, and explaining why 4-bit is the baseline for tool use while 8-bit offers diminishing returns.

Dynamic v3.0GGUFMean KLD
0 likes · 10 min read
Qwen3.8-27B Quantization Selection Guide: Match Your Hardware to the Right GGUF Version
Old Zhang's AI Learning
Old Zhang's AI Learning
Jul 11, 2026 · Artificial Intelligence

Unsloth’s Dynamic NVFP4 Makes Qwen3.6 Run 2.5× Faster Than NVIDIA’s Official Quantization

Unsloth’s Dynamic NVFP4 quantization (W4A4) lets Qwen3.6‑27B run up to 2.5× faster on Blackwell GPUs while keeping near‑BF16 accuracy, adds FP8 KV‑Cache calibration, provides detailed hardware requirements, benchmark tables, and step‑by‑step deployment guides via vLLM, SGLang or Unsloth Studio.

Blackwell GPUDynamic QuantizationFP8 KV cache
0 likes · 13 min read
Unsloth’s Dynamic NVFP4 Makes Qwen3.6 Run 2.5× Faster Than NVIDIA’s Official Quantization
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 29, 2026 · Artificial Intelligence

How Nvidia’s NVFP4 Cuts GLM‑5.2 Deployment Cost by Half

Semgrep’s benchmark shows open‑source GLM‑5.2 matching Claude’s performance while costing only $0.17 per vulnerability, and Nvidia’s NVFP4 quantization halves the model’s memory footprint with virtually unchanged accuracy, making local deployment on 8‑GPU systems far more affordable.

AI deploymentGLM-5.2Model Quantization
0 likes · 11 min read
How Nvidia’s NVFP4 Cuts GLM‑5.2 Deployment Cost by Half
Old Zhang's AI Learning
Old Zhang's AI Learning
May 31, 2026 · Artificial Intelligence

Qwen3.6-35B-A3B NVFP4: A Stable, Highly Compressed Quantized Model

NVIDIA's NVFP4 quantization reduces Qwen3.6-35B-A3B's memory footprint by threefold with almost no accuracy loss, offers plug‑and‑play deployment via vLLM, and outperforms other 4‑bit formats on Hopper/Blackwell GPUs, making it a practical choice for production AI workloads.

MoENVFP4Qwen3.6-35B-A3B
0 likes · 13 min read
Qwen3.6-35B-A3B NVFP4: A Stable, Highly Compressed Quantized Model
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 18, 2026 · Artificial Intelligence

NVIDIA Nemotron 3 Super: 7× Faster Than Qwen3.5 – Inside Hybrid Mamba‑Attention, LatentMoE, and MTP

NVIDIA’s Nemotron 3 Super, a 120.6 B‑parameter flagship model supporting 1 M‑token context, combines Hybrid Mamba‑Attention, LatentMoE, and Multi‑Token Prediction to achieve up to 7.5× higher inference throughput than Qwen3.5 while matching or surpassing its accuracy across a range of benchmarks.

Hybrid Mamba-AttentionLatentMoEMTP
0 likes · 11 min read
NVIDIA Nemotron 3 Super: 7× Faster Than Qwen3.5 – Inside Hybrid Mamba‑Attention, LatentMoE, and MTP
Old Zhang's AI Learning
Old Zhang's AI Learning
Mar 13, 2026 · Artificial Intelligence

Nvidia’s New OpenClaw‑Optimized Model Cracks Top‑5 on PinchBench – Free to Use

Nvidia’s open‑source Nemotron‑3‑Super model achieves an 85.6% success rate on the PinchBench OpenClaw benchmark, ranking in the top five (the only open‑source entry), and the article explains its architecture, quantization, training pipeline, performance numbers, usage options, and practical limitations.

AI coding agentMoENVFP4
0 likes · 10 min read
Nvidia’s New OpenClaw‑Optimized Model Cracks Top‑5 on PinchBench – Free to Use
AI Cyberspace
AI Cyberspace
Jan 26, 2026 · Artificial Intelligence

How NVFP4 Quantization Supercharges LLM Inference on NVIDIA DGX

This article explains the NVFP4 4‑bit floating‑point quantization technique, shows how to deploy Qwen3‑30B‑A3B models with TensorRT‑LLM and vLLM, compares performance across NVFP4, AWQ and INT8 quantizations, and provides practical profiling commands for NVIDIA DGX systems.

LLMNVFP4NVIDIA DGX
0 likes · 23 min read
How NVFP4 Quantization Supercharges LLM Inference on NVIDIA DGX
Design Hub
Design Hub
Jan 9, 2026 · Artificial Intelligence

LTX‑2 Acceleration Secrets: Boost Speed, Stability, and Visual Quality

This article walks through practical steps to speed up LTX‑2 AI video generation—enabling the NVFP4 model, updating NVIDIA drivers and CUDA, using FP8 text encoders, and applying a custom prompt‑optimizing assistant—showing memory savings, sub‑minute rendering at 1280×720, and noticeable quality gains.

AI video generationFP8LTX-2
0 likes · 11 min read
LTX‑2 Acceleration Secrets: Boost Speed, Stability, and Visual Quality