Skip to content
Tagged articles

quantization

202 articles · Page 1 of 3
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 16, 2026 · Artificial Intelligence

CPU-Only 284B LLM at 9 Tokens/sec: Why Memory Capacity Now Trumps GPU

The llama-modchip project demonstrates running a 284B parameter MoE model on a 64GB RAM CPU-only machine via aggressive quantization to 56.6GB and kernel optimizations, but achieves only 9 tokens/sec decoding speed, revealing that memory capacity—not GPU—is becoming the key bottleneck for local large language model inference, with practical implications for Mac memory sizing.

CPU inferenceMac hardwareMoE
0 likes · 11 min read
CPU-Only 284B LLM at 9 Tokens/sec: Why Memory Capacity Now Trumps GPU
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 15, 2026 · Artificial Intelligence

4× DGX Spark vs M5 Ultra: Why Higher Bandwidth Mac Loses on GLM-5.3-Flash

A developer's benchmark shows four NVIDIA DGX Spark units interconnected via 200Gb/s networking outperform a single 512GB Mac Studio M5 Ultra on GLM-5.3-Flash inference across prefill, code generation, and concurrent workloads, despite the Mac's higher 1.23TB/s memory bandwidth, due to software stack differences, KV cache compression, and batching efficiency.

GLM-5.3-FlashKV CacheLLM inference
0 likes · 13 min read
4× DGX Spark vs M5 Ultra: Why Higher Bandwidth Mac Loses on GLM-5.3-Flash
DataFunSummit
DataFunSummit
Nov 15, 2026 · Artificial Intelligence

284B DeepSeek on 64GB CPU: Quantization Works, 96GB Xeon Fails on AVX-512

The llama-modchip project runs a 284B-parameter DeepSeek-V4-Flash model on a 64GB RAM CPU-only machine via aggressive quantization to 1.71 bits/weight (56.56 GiB), achieving 9–10 tokens/sec, but a 96GB Xeon server fails due to custom GGUF tensor types and missing AVX-512, highlighting hardware-software compatibility gaps.

AVX-512CPU inferenceDeepSeek-V4-Flash
0 likes · 13 min read
284B DeepSeek on 64GB CPU: Quantization Works, 96GB Xeon Fails on AVX-512
Thought Artisan
Thought Artisan
Nov 12, 2026 · Artificial Intelligence

From Recency to Semantic Retrieval: Building On-Device Memory for Android AI Assistant

The article details the evolution of a memory system for an Android screen-reading AI assistant, from naive recency-based injection to on-device semantic retrieval using embeddings, covering research, lexical vs embedding benchmarks, model selection (bge-small-zh), runtime comparison (MNN vs LiteRT), quantization choices, and a fallback strategy ensuring zero regression.

AndroidMNNRAG
0 likes · 13 min read
From Recency to Semantic Retrieval: Building On-Device Memory for Android AI Assistant
DataFunTalk
DataFunTalk
Nov 8, 2026 · Artificial Intelligence

DeepSeek-V4-Flash on 64GB CPU: Why 96GB Xeon Can't Load It

The article explains how llama-modchip runs 284B DeepSeek-V4-Flash on a 64GB RAM CPU-only machine via aggressive quantization (1.71 bits/weight), while a 96GB Xeon server fails due to tensor format mismatch and missing AVX-512. It breaks down inference latency for long contracts (~10 minutes for 20k tokens) and proposes three evaluation metrics: time to first token, key-field extraction accuracy, and total processing time.

AVX-512CPU inferenceDeepSeek-V4-Flash
0 likes · 12 min read
DeepSeek-V4-Flash on 64GB CPU: Why 96GB Xeon Can't Load It
Tech Freedom Circle
Tech Freedom Circle
Nov 4, 2026 · Artificial Intelligence

Production-Grade KV Cache Optimization: Memory Budgeting, PagedAttention, Quantization & Offloading

This article provides a comprehensive guide to production-grade KV Cache optimization for LLM inference, covering memory budget formulas, PagedAttention paging mechanism, prefix caching, FP8/NVFP4 quantization, hierarchical offloading, and eviction strategies with concrete deployment priorities.

KV CacheLLM inferenceOffloading
0 likes · 32 min read
Production-Grade KV Cache Optimization: Memory Budgeting, PagedAttention, Quantization & Offloading
AI Frontier Lectures
AI Frontier Lectures
Nov 4, 2026 · Artificial Intelligence

SpecPilot: 10.87x LLM Inference Speedup via Cost-Model Guided Speculative Decoding

NVIDIA, UCB, Princeton, and Stanford introduce SpecPilot, a cost-model guided speculative decoding system that achieves 10.87x speedup over autoregressive decoding for Qwen3.8-27B on RTX PRO 6000, using 12-bit weight format, tree verification, and two-level verification to optimize per-token verification and batch scheduling.

12-bit weightsLLM inferenceNVIDIA
0 likes · 5 min read
SpecPilot: 10.87x LLM Inference Speedup via Cost-Model Guided Speculative Decoding
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 2, 2026 · Artificial Intelligence

Why 70B Struggles on 64GB MacBook Pro: 5 Deployment Iron Laws

This guide details five iron laws for deploying large language models on a 64GB MacBook Pro with M5 Pro chip, covering memory budgeting (45GB usable), model tier recommendations (32B smooth, 70B borderline), inference engine selection, speedup techniques like speculative decoding, and API serving best practices.

Apple SiliconLLM DeploymentMacBook Pro M5 Pro
0 likes · 10 min read
Why 70B Struggles on 64GB MacBook Pro: 5 Deployment Iron Laws
Lao Guo's Learning Space
Lao Guo's Learning Space
Oct 31, 2026 · Artificial Intelligence

552B DeepSeek on M5 Ultra 256GB: 3883 Prefill, 92 Decode Tok/s – Local Full-Scale LLM Achieved

Community developers benchmarked the 552B DeepSeek V4.1-Flash model at 3bpw quantization on a Mac Studio M5 Ultra with 256GB unified memory, achieving 3883 tok/s prefill, 92 tok/s decode, and up to 163 tok/s with speculative decoding, demonstrating that full-scale LLMs can run locally on consumer hardware.

Apple SiliconDeepSeekLLM inference
0 likes · 8 min read
552B DeepSeek on M5 Ultra 256GB: 3883 Prefill, 92 Decode Tok/s – Local Full-Scale LLM Achieved
Lao Guo's Learning Space
Lao Guo's Learning Space
Oct 23, 2026 · Artificial Intelligence

128GB vs 256GB: Which LLMs Actually Fit on RTX Spark and Mac Studio M5 Ultra?

The article compares unified memory capacities of NVIDIA RTX Spark (128GB) and Apple Mac Studio M5 Ultra (256GB) for local LLM inference, calculating model fit at Q4 quantization (0.5 bytes/parameter), categorizing models by size, noting Spark's clustering via ConnectX-7 for scaling, and concluding capacity needs depend on target model size.

ConnectX-7LLM inferenceMac Studio M5 Ultra
0 likes · 6 min read
128GB vs 256GB: Which LLMs Actually Fit on RTX Spark and Mac Studio M5 Ultra?
Machine Heart
Machine Heart
Oct 22, 2026 · Artificial Intelligence

Flint-40B-A2B: AI-Built MoE Coding Model Hits 125 tok/s on MacBook Pro

Recursive_SI's first model Flint-40B-A2B, a 40B MoE model activating 2.5B parameters per token, was built in under four months using the automated Eureka Machine system. It achieves state-of-the-art local coding performance on benchmarks like SWE-Bench while running at 125+ tokens/sec on MacBook Pro M4 Pro and 150+ tokens/sec on NVIDIA RTX Spark, with open weights on Hugging Face under Apache 2.0.

Apple SiliconEureka MachineFlint-40B-A2B
0 likes · 10 min read
Flint-40B-A2B: AI-Built MoE Coding Model Hits 125 tok/s on MacBook Pro
Alibaba Cloud Native
Alibaba Cloud Native
Oct 20, 2026 · Artificial Intelligence

Inference Frameworks Demystified: Core Concepts, Frameworks & Cluster Deployment for LLMs

This comprehensive guide explains inference frameworks for large language models, covering core concepts like Prefill/Decode and KV Cache, comparing major open-source frameworks (vLLM, SGLang, TensorRT-LLM), detailing request processing flow from scheduling to sampling, and identifying who benefits from understanding inference infrastructure.

KV CacheLLM servingMoE
0 likes · 46 min read
Inference Frameworks Demystified: Core Concepts, Frameworks & Cluster Deployment for LLMs
liandk
liandk
Oct 13, 2026 · Artificial Intelligence

Mac Local LLM Deployment Pitfalls: Ollama Memory Optimization & Model Selection Guide

This article details real-world pitfalls when deploying large language models locally on Mac using Ollama, covering memory overflow causes, key parameter tuning (num_ctx, num_parallel, num_gpu), quantization level selection, model recommendations for coding vs chat, and monitoring strategies to avoid swap-induced performance degradation.

LLM DeploymentMacOllama
0 likes · 14 min read
Mac Local LLM Deployment Pitfalls: Ollama Memory Optimization & Model Selection Guide
Old Zhang's AI Learning
Old Zhang's AI Learning
Oct 11, 2026 · Artificial Intelligence

Qwen4-Max & Qwen4-27B Coming: Open-Source Plans & Qwen3.8-27B Deployment Insights

The author covers Alibaba Qwen team's upcoming Qwen4 family with open-source plans, surveys recent global AI model releases, notes a slowdown in Chinese model updates, shares hands-on testing of Qwen3.8-27B quantization variants on A100 and 4090 GPUs, and recommends FP8 for dual-4090 and GGUF UD-Q4+ for single-4090 setups.

FP8GGUFLLM Deployment
0 likes · 4 min read
Qwen4-Max & Qwen4-27B Coming: Open-Source Plans & Qwen3.8-27B Deployment Insights
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 7, 2026 · Artificial Intelligence

VeriLoop E2 Advances Riemann Hypothesis Zero Proportion to 67.35% with Verifiable AI

Tsinghua researchers open-source VeriLoop E2, a post-trained model using verifiable recurrence to achieve 67.35% zero proportion lower bound for Riemann hypothesis, while setting new benchmarks in code, math, and physics reasoning with external verification.

Post-TrainingQwen 3.8-27BRiemann hypothesis
0 likes · 40 min read
VeriLoop E2 Advances Riemann Hypothesis Zero Proportion to 67.35% with Verifiable AI
Ops Community
Ops Community
Oct 7, 2026 · Artificial Intelligence

Multi-GPU LLM Inference: Step-by-Step Deployment Guide for Memory-Constrained Environments

This comprehensive tutorial covers multi-GPU inference for large language models, detailing VRAM estimation formulas, parallel strategies (tensor, pipeline, data), GPU interconnects (NVLink, PCIe), framework comparisons (vLLM, DeepSpeed, Transformers), quantization techniques (GPTQ, AWQ, bitsandbytes), performance benchmarking, and production deployment with Docker and Kubernetes.

DeepSpeedGPU memory optimizationLLM Deployment
0 likes · 32 min read
Multi-GPU LLM Inference: Step-by-Step Deployment Guide for Memory-Constrained Environments
Ops Development & AI Practice
Ops Development & AI Practice
Sep 26, 2026 · Artificial Intelligence

Local LLM Hardware Truths: MoE, CPU/GPU/NPU, and 32GB Mac Mini Tuning

This article explains why CPUs struggle with MoE model prefill but excel at decode, compares dense, SSM, MLA, and BitNet architectures, evaluates CPU, NVIDIA GPU, Apple UMA, and NPU hardware for local LLM inference, and provides a practical tuning guide for running 32B models on a 32GB Mac mini with MLX and Ollama.

Apple SiliconKV CacheLLM inference
0 likes · 21 min read
Local LLM Hardware Truths: MoE, CPU/GPU/NPU, and 32GB Mac Mini Tuning
Lao Guo's Learning Space
Lao Guo's Learning Space
Sep 23, 2026 · Artificial Intelligence

Mac mini 16GB to Studio 256GB: Which Mac Runs Your LLMs Best?

This article analyzes Mac mini and Mac Studio configurations for local LLM inference, showing how unified memory capacity determines which models fit and memory bandwidth dictates token generation speed, with real-world benchmarks for 8B to 235B models across M6, M5 Pro, M5 Max, and M5 Ultra chips.

Apple SiliconLLM inferenceMac Studio
0 likes · 13 min read
Mac mini 16GB to Studio 256GB: Which Mac Runs Your LLMs Best?
Lao Guo's Learning Space
Lao Guo's Learning Space
Sep 23, 2026 · Artificial Intelligence

M5 Ultra Mac Studio: The Dream Machine for Local AI Agents?

Federico Viticci's four-day deep test of the M5 Ultra Mac Studio (256GB unified memory) reveals 1.2TB/s bandwidth, 2.5x faster prefill, up to 93.5% faster long-context generation, and superior concurrency for 24/7 local AI agent workflows at near-zero cost, outperforming RTX 5090 in usability.

ConcurrencyM5 UltraMac Studio
0 likes · 12 min read
M5 Ultra Mac Studio: The Dream Machine for Local AI Agents?
Lao Guo's Learning Space
Lao Guo's Learning Space
Sep 21, 2026 · Artificial Intelligence

M5 Ultra 256GB: Which LLMs Fit? 27B to 200B+ Capacity Analysis

The article analyzes Apple M5 Ultra 256GB unified memory capacity for local LLM inference, showing Q4-quantized models from 27B to 200B+ fit, but KV cache overhead for long contexts reduces headroom; it compares with multi-GPU setups and advises on purchase decisions.

Apple SiliconKV CacheLLM inference
0 likes · 8 min read
M5 Ultra 256GB: Which LLMs Fit? 27B to 200B+ Capacity Analysis
Cambridge Mofang Notes
Cambridge Mofang Notes
Sep 4, 2026 · Artificial Intelligence

Model Distillation: Teaching Small Models to Learn from Large Ones

This article explains model distillation through a question rewriting example, detailing how teacher models provide demonstrations to train smaller student models, the differences between distillation, fine-tuning, and quantization, and practical pitfalls like data quality and student capacity limits.

data qualityfine-tuningknowledge distillation
0 likes · 19 min read
Model Distillation: Teaching Small Models to Learn from Large Ones
Cambridge Mofang Notes
Cambridge Mofang Notes
Sep 3, 2026 · Artificial Intelligence

AI Model Types, Quantization & File Formats: A Complete Guide

This article explains the four key dimensions of AI models—purpose, modality, quantization, and file format—covering model categories like LLMs, vision, audio, embedding, and reranker models, multimodal concepts, quantization trade-offs (FP16, Q4, Q8), and formats such as Safetensors, GGUF, and ONNX, with a practical checklist for model selection.

Embedding ModelsGGUFMultimodal Models
0 likes · 27 min read
AI Model Types, Quantization & File Formats: A Complete Guide
DataFunTalk
DataFunTalk
Sep 1, 2026 · Artificial Intelligence

Cracking the Last Mile of Edge Agents: Quantization, Instruction Sets, and Deployment

The article examines how large‑model agents can finally run on phones, PCs, cars and IoT devices by tackling three engineering layers—unlocking compute with Arm SME2, slimming models through ultra‑low‑bit quantization, and deploying with ExecuTorch and Windows on Arm—presented at the free Arm Create 2026 events.

Arm SME2ExecuTorchMNN
1 likes · 3 min read
Cracking the Last Mile of Edge Agents: Quantization, Instruction Sets, and Deployment
ThinkingAgent
ThinkingAgent
Aug 24, 2026 · Artificial Intelligence

Why Quantization and KV‑Cache Are Key to High‑Performance LLM Inference

The article analyzes why the same LLM can exhibit vastly different cost, speed, and concurrency across inference systems, showing that KV‑cache memory management, continuous batching, PagedAttention, quantization trade‑offs, and speculative decoding together determine real‑world throughput and latency.

KV CacheLLM inferencecontinuous batching
0 likes · 39 min read
Why Quantization and KV‑Cache Are Key to High‑Performance LLM Inference
Old Zhang's AI Learning
Old Zhang's AI Learning
Aug 23, 2026 · Artificial Intelligence

Qwen3.8-27B Quantization Selection Guide: Match Your Hardware to the Right GGUF Version

This guide analyzes Unsloth's Dynamic v3.0 quantizations of Qwen3.8-27B, showing Mean KLD divergence across versions, recommending UD-Q4_K_XL for 24GB GPUs, detailing hardware requirements, sampling parameters for thinking modes, and explaining why 4-bit is the baseline for tool use while 8-bit offers diminishing returns.

Dynamic v3.0GGUFMean KLD
0 likes · 10 min read
Qwen3.8-27B Quantization Selection Guide: Match Your Hardware to the Right GGUF Version
AI Engineer Programming
AI Engineer Programming
Aug 21, 2026 · Artificial Intelligence

Essential Concepts and Terminology for Deploying Large Language Models Locally

This article walks through the core concepts needed before deploying a large language model on‑premises, covering weight precision, quantization methods, model packaging formats, inference engines, GPU memory considerations, KV‑cache sizing, sampling strategies, optional extensions such as LoRA and RAG, and a step‑by‑step decision workflow to match hardware, model, and deployment goals.

KV CacheLLMdeployment
0 likes · 21 min read
Essential Concepts and Terminology for Deploying Large Language Models Locally
AntTech
AntTech
Aug 20, 2026 · Artificial Intelligence

Ling-3.0-flash: Open-Source LLM Designed for Real-World Deployment

Ling-3.0-flash is a newly open‑sourced 124B‑parameter MoE model that offers multiple quantized versions, API, single‑machine private deployment, and high‑performance GPU inference exceeding 1100 tokens/s, with detailed benchmarks, optimization techniques, and real‑world use‑case analyses for agents, coding, and sensitive data processing.

Agent applicationsLLMLing-3.0-flash
0 likes · 15 min read
Ling-3.0-flash: Open-Source LLM Designed for Real-World Deployment
AI Architecture Path
AI Architecture Path
Aug 19, 2026 · Artificial Intelligence

Needle2: 45M‑Parameter Open‑Source Edge Agent – 14 MB Model, 28 MB Runtime for Offline Tool Calls

Needle2 tackles the twin constraints of cloud‑dependency and heavyweight local models by offering a 45 M‑parameter, 14 MB single‑file model that runs in a constant ~28 MB memory footprint, specializing in tool calling, device control, and JSON extraction on ultra‑light hardware such as Raspberry Pi, VR headsets, and ESP32‑S3, while providing detailed architecture, benchmark, deployment, and fine‑tuning guidance.

LoRANeedle2SAN architecture
0 likes · 15 min read
Needle2: 45M‑Parameter Open‑Source Edge Agent – 14 MB Model, 28 MB Runtime for Offline Tool Calls
Old Zhang's AI Learning
Old Zhang's AI Learning
Aug 17, 2026 · Artificial Intelligence

How Many GPUs Does Kimi K3 Need? Self‑Hosting vs API Cost Comparison

The article breaks down Kimi K3’s 2.8‑trillion‑parameter architecture, explains its 4‑bit MXFP4 quantization, calculates the ~1.4 TB memory requirement, shows that 8‑GPU clusters (e.g., NVIDIA B300 or AMD MI350X) are needed for self‑hosting, and compares these costs with the per‑token API pricing, highlighting when each option is economical.

API costDigitalOceanGPU requirements
0 likes · 14 min read
How Many GPUs Does Kimi K3 Need? Self‑Hosting vs API Cost Comparison
Old Zhang's AI Learning
Old Zhang's AI Learning
Aug 14, 2026 · Artificial Intelligence

Why Qwen3.8-27B Is the World’s New Favorite Open‑Source LLM and How to Deploy It Locally

The article introduces Qwen3.8-27B, a dense multimodal LLM with up to 256K tokens (extendable to 1M), highlights its benchmark gains over previous Qwen models, discusses model size, quantization options, and provides step‑by‑step instructions for local deployment using vLLM, Docker, and LMStudio.

Qwen3.8-27Bbenchmarkdeployment
0 likes · 8 min read
Why Qwen3.8-27B Is the World’s New Favorite Open‑Source LLM and How to Deploy It Locally
DataFunTalk
DataFunTalk
Aug 11, 2026 · Artificial Intelligence

Meta Open‑Sources Muse Glimmer: Packing a 30B Model into 24 GB VRAM for Always‑On Local Agents

Meta’s Muse Glimmer is a 30‑billion‑parameter, open‑source dense model engineered to run continuously on consumer‑grade hardware with as little as 24 GB VRAM, using 4‑bit quantization and DFlash speculative decoding to retain multimodal, long‑context and agentic capabilities while achieving up to 233 tok/s throughput.

Muse Glimmeragentic benchmarkslocal agents
0 likes · 10 min read
Meta Open‑Sources Muse Glimmer: Packing a 30B Model into 24 GB VRAM for Always‑On Local Agents
SuanNi
SuanNi
Aug 11, 2026 · Artificial Intelligence

How Meta’s Open‑Source 30B Muse Glimmer Agent Runs on Your PC

Meta’s newly open‑sourced 30‑billion‑parameter Muse Glimmer agent model runs on a single consumer‑grade GPU, outperforms Gemma‑4 and Qwen‑3.6 on multiple Agent benchmarks, uses a perception encoder for multimodal input, and fits into a 20 GB memory envelope through quantization and a lightweight drafter.

LLMMuse Glimmeragent model
0 likes · 7 min read
How Meta’s Open‑Source 30B Muse Glimmer Agent Runs on Your PC
Machine Heart
Machine Heart
Aug 10, 2026 · Artificial Intelligence

How MiniMax H3’s Open‑Source Week Is Rewriting the DeepSeek Story for Video AI

MiniMax H3’s open‑source release sparked a week of explosive community activity, with rapid ecosystem integration, aggressive quantization to fit consumer GPUs, speed‑up LoRAs, diverse creative workflows, and industry attention that together illustrate how a powerful video model can reshape the AI landscape.

AI communityComfyUIMiniMax H3
0 likes · 18 min read
How MiniMax H3’s Open‑Source Week Is Rewriting the DeepSeek Story for Video AI
Data Party THU
Data Party THU
Aug 8, 2026 · Artificial Intelligence

Memory-Efficient Algorithms for Large Language Model Inference

The article reviews Coleman Hooper's 2026 Berkeley PhD thesis, which shows that LLM inference is increasingly limited by memory bandwidth and capacity, and proposes a four‑pronged approach—weight quantization, KV‑cache quantization, selective context loading, and multipole attention—to dramatically improve memory efficiency and throughput.

AttentionKV CacheLLM
0 likes · 12 min read
Memory-Efficient Algorithms for Large Language Model Inference
Old Zhang's AI Learning
Old Zhang's AI Learning
Aug 4, 2026 · Artificial Intelligence

The “Best” Qwen3.6-27B Variant: A God‑Level Model for Local Deployment

The community‑fine‑tuned Qwen3.6-27B‑Fable‑Fusion‑711 model combines multi‑stage fine‑tuning, model fusion and uncensored processing, delivers a 0.711 ARC‑C score that surpasses the original on six of seven benchmarks, and offers a rich set of GGUF quantizations with detailed performance guidance for local deployment.

AIGGUFQwen3.6-27B
0 likes · 10 min read
The “Best” Qwen3.6-27B Variant: A God‑Level Model for Local Deployment
DataFunSummit
DataFunSummit
Aug 1, 2026 · Artificial Intelligence

From Text to Images: Building Multimodal Product Search with Elasticsearch Serverless

This article presents a detailed technical guide on constructing a multimodal product search system that combines text and image queries using embedding vectors, hybrid dense‑sparse models, vector similarity metrics, and Elasticsearch Serverless’s built‑in vector support, while also covering quantization techniques, hybrid search fusion, and best‑practice deployment architectures on Alibaba Cloud.

AI search platformElasticsearch ServerlessMultimodal Search
0 likes · 22 min read
From Text to Images: Building Multimodal Product Search with Elasticsearch Serverless
PaperAgent
PaperAgent
Aug 1, 2026 · Artificial Intelligence

How to Run DeepSeek‑V4‑Flash Locally on a 100 GB Server: Best‑Practice Guide

The article details the release of DeepSeek‑V4‑Flash‑0731, explains how its 284 B‑parameter, 13 B‑activated model can run losslessly on a machine with only 169 GB RAM using Unsloth’s UD‑Q8_K_XL quantization, compares quantization quality, and provides step‑by‑step deployment instructions via Unsloth Studio and llama.cpp.

AI model deploymentDeepSeek-V4-FlashLossless Quantization
0 likes · 7 min read
How to Run DeepSeek‑V4‑Flash Locally on a 100 GB Server: Best‑Practice Guide
AI Engineering
AI Engineering
Aug 1, 2026 · Artificial Intelligence

Running DeepSeek V4 Flash 284B Locally – Performance Beats V4 Pro

DeepSeek V4 Flash 0731, a 284‑billion‑parameter model with 13 B active weights and a 1 M context window, can run locally using Unsloth's lossless GGUF quantizations on machines with 128‑169 GB memory, and its benchmark scores surpass the V4 Pro preview.

AI agentDeepSeekV4-Flash
0 likes · 5 min read
Running DeepSeek V4 Flash 284B Locally – Performance Beats V4 Pro
AI Programming Lab
AI Programming Lab
Jul 28, 2026 · Artificial Intelligence

How a Team Ran the Open‑Source Kimi K3 Model on 80 RTX 5090 GPUs

The Kimi K3 model weights were released on HuggingFace (1.56 TB total), featuring mixed attention, Attention Residuals, and a Stable LatentMoE that together cut scaling cost by 2.5×, and a detailed cost‑benefit analysis shows how 80 consumer‑grade RTX 5090 cards can run the full 2.8‑trillion‑parameter model with 20 tok/s throughput, while highlighting memory‑saving quantization, KV‑cache design, and the steep price gap versus professional GPUs.

AI model deploymentKimi K3RTX 5090
0 likes · 9 min read
How a Team Ran the Open‑Source Kimi K3 Model on 80 RTX 5090 GPUs
AI Programming Lab
AI Programming Lab
Jul 21, 2026 · Artificial Intelligence

How Kimi K3 and Qwen3.8‑Max Reach 2T+ Parameters: The Evolution of Chinese LLM Architecture

The article examines how Chinese LLMs such as Kimi K3 (2.8 T) and Qwen 3.8‑Max (2.4 T) achieved 2‑trillion‑parameter scales by progressively decoupling parameters from compute with Mixture‑of‑Experts, optimizing attention, and introducing training tricks like Muon and low‑precision quantization, tracing four years of architectural advances.

Model Scalingattention mechanismslarge language models
0 likes · 11 min read
How Kimi K3 and Qwen3.8‑Max Reach 2T+ Parameters: The Evolution of Chinese LLM Architecture
MaGe Linux Operations
MaGe Linux Operations
Jul 16, 2026 · Artificial Intelligence

How to Choose Between INT8, FP8, and INT4 Quantization for Large Models

This guide explains how to evaluate INT8, FP8, and INT4 quantization strategies for large language models on NVIDIA GPUs, covering precision trade‑offs, memory consumption, kernel support, KV‑Cache considerations, and detailed deployment, testing, and rollback procedures to ensure performance and quality.

FP8GPU deploymentINT4
0 likes · 48 min read
How to Choose Between INT8, FP8, and INT4 Quantization for Large Models
Old Zhang's AI Learning
Old Zhang's AI Learning
Jul 8, 2026 · Artificial Intelligence

Deploying the Quantized DeepSeek‑V4‑Flash Locally: Performance, Benchmarks, and Tips

This article walks through DeepSeek‑V4‑Flash, a lightweight 284B MoE model with 13B active parameters and 1‑million‑token context, explains its three thinking modes, presents official benchmark results showing Flash can match Pro with higher reasoning budget, and provides step‑by‑step local deployment instructions using Unsloth’s GGUF quantization and llama.cpp fixes.

AIDeepSeek-V4-FlashGGUF
0 likes · 13 min read
Deploying the Quantized DeepSeek‑V4‑Flash Locally: Performance, Benchmarks, and Tips
AntData
AntData
Jul 8, 2026 · Databases

Deep Dive into SINDI: High‑Performance Sparse Vector Search Index for VSAG

The article explains SINDI, a sparse‑vector retrieval index designed for VSAG, detailing its window‑based weighted inverted lists, multi‑layer pruning, quantization and reranking techniques that achieve dramatically faster index construction, smaller index size, and high query throughput compared with traditional inverted or graph indexes.

Maximum Inner Product SearchSparse VectorVSAG
0 likes · 21 min read
Deep Dive into SINDI: High‑Performance Sparse Vector Search Index for VSAG
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Jul 6, 2026 · Artificial Intelligence

Running GLM‑5.2 on AMD: 2,626 Tokens/s at Half Nvidia’s Cost

Wafer demonstrates that the open‑source GLM‑5.2 model can run on AMD’s MI355X GPU at 2,626 tokens per second with single‑stream throughput of 213 tokens per second, achieving comparable performance to Nvidia’s B200 at less than half the cost, by applying MXFP4 quantization, selecting the sglang engine, and fixing naming and kernel issues.

AMD MI355XGLM-5.2cost
0 likes · 9 min read
Running GLM‑5.2 on AMD: 2,626 Tokens/s at Half Nvidia’s Cost
DataFunSummit
DataFunSummit
Jul 6, 2026 · Artificial Intelligence

From Text to Images: Building Multi‑Modal Product Search with Elasticsearch Serverless

The article explains how modern e‑commerce search is evolving from simple keyword matching to multi‑modal retrieval, outlines a generic architecture that combines text and image embeddings, describes vector similarity metrics and quantization techniques, and demonstrates how Elasticsearch Serverless and Alibaba Cloud AI Search Platform enable a low‑cost, fully managed end‑to‑end multi‑modal product search solution.

ElasticsearchEmbeddingMulti-modal Search
0 likes · 21 min read
From Text to Images: Building Multi‑Modal Product Search with Elasticsearch Serverless
ThinkingAgent
ThinkingAgent
Jul 5, 2026 · Artificial Intelligence

Building L1 AI Infra: Model Gateways, Smart Routing, and High‑Performance Inference Engines

The article presents a comprehensive, production‑ready guide for the L1 layer of AI infrastructure, detailing how model gateways unify calls, intelligent routing selects the optimal model, inference engines maximize GPU throughput, and quantization and KV‑Cache techniques dramatically cut costs while maintaining performance.

AI infrastructureKV CacheLLM routing
0 likes · 25 min read
Building L1 AI Infra: Model Gateways, Smart Routing, and High‑Performance Inference Engines
Black & White Path
Black & White Path
Jun 30, 2026 · Artificial Intelligence

A 27B Red‑Team AI Model That Runs on Just 12 GB VRAM

The BugTraceAI CORE Ultra 27B model, fine‑tuned on 2,541 real vulnerability reports, generates fully functional Nuclei templates, CVE PoCs, webshell bypasses, JWT cracking tools, and kernel exploits with a 0 % rejection rate, and its quantized Q4 version runs on a single 24 GB GPU, making advanced red‑team automation accessible.

BugTraceAIGPULLM
0 likes · 7 min read
A 27B Red‑Team AI Model That Runs on Just 12 GB VRAM
AI Tech Publishing
AI Tech Publishing
Jun 29, 2026 · Artificial Intelligence

Understanding the Full LLM Inference Pipeline: From Tokenization to Streaming Output

The article walks through every stage of LLM inference—from tokenization and embedding, through transformer layers, prefill (compute‑bound) and decode (memory‑bound) phases, KV‑cache management, attention redesign, quantization, and serving infrastructure—explaining how each step consumes time and resources and how to identify bottlenecks.

Attention optimizationDecodeKV Cache
0 likes · 14 min read
Understanding the Full LLM Inference Pipeline: From Tokenization to Streaming Output
Raymond Ops
Raymond Ops
Jun 27, 2026 · Artificial Intelligence

vLLM Quantized Inference: Loading AWQ/GPTQ Models and Optimizing GPU Memory

This article provides a step‑by‑step guide on using vLLM to load AWQ and GPTQ quantized large language models, covering environment setup, calibration data preparation, model quantization, deployment scripts, performance benchmarking, accuracy checks, best‑practice recommendations, and troubleshooting tips for GPU memory optimization.

AWQGPTQGPU memory optimization
0 likes · 45 min read
vLLM Quantized Inference: Loading AWQ/GPTQ Models and Optimizing GPU Memory
21CTO
21CTO
Jun 27, 2026 · Artificial Intelligence

Large vs Small Language Models: An Apple‑Centric Technical Comparison

The article analyses how deployment targets, inference economics, and training budgets drive divergent design choices for large (LLM) and small (SLM) Transformer‑based language models, covering architecture tweaks, data‑centric training methods, quantisation, KV‑cache management, and hybrid routing strategies for production systems.

Hybrid InferenceInference Optimizationlarge language models
0 likes · 16 min read
Large vs Small Language Models: An Apple‑Centric Technical Comparison
AI Programming Lab
AI Programming Lab
Jun 26, 2026 · Industry Insights

How to Land a Role in a Cutting‑Edge AI Lab Amid DeepSeek’s Hiring Surge

The article breaks down DeepSeek’s massive hiring push, outlines the core traits DeepMind’s Vlad Feinberg says are essential, and provides a step‑by‑step roadmap—including kernel work, scaling‑law studies, and open‑source contributions—to help aspiring engineers break into top AI research labs.

AI labsDeepSeekjob preparation
0 likes · 10 min read
How to Land a Role in a Cutting‑Edge AI Lab Amid DeepSeek’s Hiring Surge
DataFunSummit
DataFunSummit
Jun 25, 2026 · Cloud Computing

From Text to Images: Building Multi‑Modal Product Search with Elasticsearch Serverless

The article walks through the evolution of e‑commerce search from simple keyword matching to multi‑modal retrieval, explains a generic architecture that fuses text and image embeddings, details core techniques such as dense, sparse and hybrid models, vector similarity metrics, quantization methods like SQ and BBQ, and demonstrates how Elasticsearch Serverless provides a server‑less, cost‑effective platform to implement the end‑to‑end solution.

AIElasticsearchEmbedding
0 likes · 21 min read
From Text to Images: Building Multi‑Modal Product Search with Elasticsearch Serverless
Raymond Ops
Raymond Ops
Jun 22, 2026 · Artificial Intelligence

Elastic Deployment and GPU Scheduling for Large‑Model Inference with vLLM on Kubernetes

This article presents a detailed, step‑by‑step analysis of deploying the high‑performance vLLM inference engine on Kubernetes, covering GPU memory management, tensor parallelism, quantization choices, continuous batching, and automated scaling with HPA/KEDA to achieve low latency and high throughput for large language models.

DockerGPU schedulingKubernetes
0 likes · 49 min read
Elastic Deployment and GPU Scheduling for Large‑Model Inference with vLLM on Kubernetes
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 19, 2026 · Artificial Intelligence

Gemma‑4‑12B‑v2 (Fable 5 Clone) Achieves 3.5× Telecom Benchmark Boost

The author reproduces Anthropic’s Fable 5 using Gemma‑4‑12B‑v2, showing a 3.5× improvement on the telecom tau2‑bench versus the base model, details the agentic, coding, and general training data, compares quantization sizes, provides llama.cpp launch commands, and notes speed gains from speculative MTP decoding and current limitations.

Agentic AIFable 5Gemma-4-12B
0 likes · 9 min read
Gemma‑4‑12B‑v2 (Fable 5 Clone) Achieves 3.5× Telecom Benchmark Boost
DataFunSummit
DataFunSummit
Jun 18, 2026 · Artificial Intelligence

From Text to Images: Building Multimodal Product Search with Elasticsearch Serverless

This article examines the evolution of e‑commerce search from simple keyword matching to multimodal, cross‑modal retrieval, explains the core embedding and vector‑search technologies, compares dense, sparse and hybrid models, and demonstrates how Elasticsearch Serverless and Alibaba Cloud AI Search Platform enable a low‑cost, serverless, high‑performance end‑to‑end multimodal product search solution.

AI search platformBBQElasticsearch Serverless
0 likes · 21 min read
From Text to Images: Building Multimodal Product Search with Elasticsearch Serverless
ThinkingAgent
ThinkingAgent
Jun 17, 2026 · Artificial Intelligence

Why a 7B Model Can Outperform a 70B Model: The Power of Knowledge Distillation

The article explains how knowledge distillation lets a small LLM learn from a much larger teacher model, achieving near‑teacher performance while cutting inference cost, latency and memory, and provides a step‑by‑step guide, benchmark results, advanced on‑policy techniques, common pitfalls and best‑practice recommendations.

LLMLlama-3On-Policy Distillation
0 likes · 13 min read
Why a 7B Model Can Outperform a 70B Model: The Power of Knowledge Distillation
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 1, 2026 · Artificial Intelligence

Opus‑Distilled Qwen3.5‑Coder Scores 100/100 Tool Calls, 1.4‑2.2× Faster with MTP, 128K Context on Consumer GPU

The article introduces Qwopus3.5‑4B‑Coder‑MTP‑GGUF, a 4‑billion‑parameter agent model fine‑tuned for code debugging, tool calling, and structured reasoning, explains its novel Trace Inversion, high‑quality trajectory data, and Curriculum SFT training, details MTP acceleration, benchmark results, quantization options, and step‑by‑step local deployment instructions.

AgentGGUFMTP
0 likes · 10 min read
Opus‑Distilled Qwen3.5‑Coder Scores 100/100 Tool Calls, 1.4‑2.2× Faster with MTP, 128K Context on Consumer GPU
Old Zhang's AI Learning
Old Zhang's AI Learning
May 31, 2026 · Artificial Intelligence

Qwen3.6-35B-A3B NVFP4: A Stable, Highly Compressed Quantized Model

NVIDIA's NVFP4 quantization reduces Qwen3.6-35B-A3B's memory footprint by threefold with almost no accuracy loss, offers plug‑and‑play deployment via vLLM, and outperforms other 4‑bit formats on Hopper/Blackwell GPUs, making it a practical choice for production AI workloads.

MoENVFP4Qwen3.6-35B-A3B
0 likes · 13 min read
Qwen3.6-35B-A3B NVFP4: A Stable, Highly Compressed Quantized Model
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
May 29, 2026 · Artificial Intelligence

How Alibaba Cloud Milvus Achieves 20× Faster Billion‑Scale Vector Search with DiskANN and RaBitQ

Alibaba Cloud Milvus combines DiskANN graph indexing with the RaBitQ quantization algorithm, delivering over 20× higher QPS, sub‑10% P99 latency, 29% lower memory usage and more than 98% recall on a 100 million‑vector, 768‑dimensional benchmark, while also cutting index build time from 20 h to about 6 h.

DiskANNMilvusRaBitQ
0 likes · 7 min read
How Alibaba Cloud Milvus Achieves 20× Faster Billion‑Scale Vector Search with DiskANN and RaBitQ
DataFunSummit
DataFunSummit
May 27, 2026 · Artificial Intelligence

From Text to Images: Building Multi‑Modal Product Search with Elasticsearch Serverless

This article walks through a complete multi‑modal product search solution that transforms textual and visual product data into embeddings, leverages dense, sparse and hybrid models, applies vector similarity and quantization techniques such as SQ and BBQ, and demonstrates how Elasticsearch Serverless provides a serverless, cost‑effective, auto‑scaling backbone for end‑to‑end retrieval.

AI Search Open PlatformElasticsearch ServerlessEmbedding
0 likes · 22 min read
From Text to Images: Building Multi‑Modal Product Search with Elasticsearch Serverless
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
May 22, 2026 · Artificial Intelligence

20‑Year‑Old Transformer Co‑author Open‑Sources a 218‑Billion‑Parameter Model

Cohere’s Command A+ model, built by Transformer co‑author Aidan Gomez and backed by Nick Frosst, packs 218 billion parameters but activates only 25 billion at inference, uses a lossless 4‑bit quantization scheme, offers native citation support, runs on a single B200 or two H100 GPUs, and is released under an Apache 2.0 license, marking a major shift toward truly open‑source, enterprise‑ready large language models.

AIApache 2.0Cohere
0 likes · 12 min read
20‑Year‑Old Transformer Co‑author Open‑Sources a 218‑Billion‑Parameter Model
Old Zhang's AI Learning
Old Zhang's AI Learning
May 17, 2026 · Artificial Intelligence

Why DeepSeek V4 Flash’s Quantized Model Is Gaining Traction

The DeepSeek V4 Flash quantized GGUF model and the dedicated ds4 inference engine, both released by antirez, offer dramatically reduced activation parameters, massive 1‑million‑token context windows, aggressive KV‑cache compression and hardware‑specific quantizations that enable smooth local inference on high‑memory Macs and CUDA machines, while sacrificing generality for performance.

DS4DeepSeek-V4-FlashGGUF
0 likes · 11 min read
Why DeepSeek V4 Flash’s Quantized Model Is Gaining Traction
DataFunSummit
DataFunSummit
May 15, 2026 · Artificial Intelligence

From Text to Images: Building Multimodal Product Search with Elasticsearch Serverless

The article analyzes the shift from keyword‑based to multimodal e‑commerce search, outlines a generic architecture that combines text and image embedding with vector retrieval, and demonstrates how Elasticsearch Serverless and Alibaba Cloud AI Search platform enable a low‑cost, scalable, and high‑performance product search solution.

AI searchElasticsearchEmbedding
0 likes · 20 min read
From Text to Images: Building Multimodal Product Search with Elasticsearch Serverless
SuanNi
SuanNi
May 13, 2026 · Artificial Intelligence

How MiniCPM-V 4.6 Achieves Lightning‑Fast Multimodal AI on Smartphones (Open‑Source)

MiniCPM-V 4.6 combines a SigLIP2 visual encoder with a Qwen3.5 LLM, cuts FLOPs by over 50%, lowers token cost up to 43×, scores 13 on the Artificial Analysis Intelligence Index, and runs with 75 ms first‑token latency on 3136×3136 images across iOS, Android and HarmonyOS, all with fully open‑source code and extensive quantization support.

MiniCPM-Vbenchmarkmobile inference
0 likes · 6 min read
How MiniCPM-V 4.6 Achieves Lightning‑Fast Multimodal AI on Smartphones (Open‑Source)
Lao Guo's Learning Space
Lao Guo's Learning Space
May 12, 2026 · Artificial Intelligence

Demystifying the Core Technologies Behind ChatGPT, GPT‑4, and DeepSeek

This article breaks down the key algorithms that power large‑language models—Transformer, Mixture‑of‑Experts, Flash Attention, KV‑Cache, Multi‑Token Prediction, quantization, Chain‑of‑Thought and Retrieval‑Augmented Generation—explaining how each contributes to the performance of ChatGPT, GPT‑4 and DeepSeek.

Flash AttentionKV CacheRetrieval-Augmented Generation
0 likes · 10 min read
Demystifying the Core Technologies Behind ChatGPT, GPT‑4, and DeepSeek
DataFunSummit
DataFunSummit
May 7, 2026 · Artificial Intelligence

From Text to Images: Building Multimodal Product Search with Elasticsearch Serverless

This article walks through a complete multimodal product search solution, explaining how embedding and vector retrieval technologies—combined with Elasticsearch Serverless and Alibaba Cloud AI Search—enable image‑based and semantic queries, detailing the architecture, key algorithms, quantization tricks, and practical deployment steps.

AI searchElasticsearchEmbedding
0 likes · 22 min read
From Text to Images: Building Multimodal Product Search with Elasticsearch Serverless
Machine Heart
Machine Heart
May 6, 2026 · Artificial Intelligence

Turning Your Mac into a Private AI Workstation with Cider and Mano‑P

The article analyzes how Ollama's shift to Apple’s MLX framework unlocks major speed gains on M5‑class Macs, then introduces the open‑source Cider inference accelerator and Mano‑P visual agent, detailing their quantization modes, benchmark results, hardware constraints, and how together they enable fast, offline private AI on macOS.

Apple SiliconCiderMLX
0 likes · 15 min read
Turning Your Mac into a Private AI Workstation with Cider and Mano‑P
Architects' Tech Alliance
Architects' Tech Alliance
May 1, 2026 · Artificial Intelligence

How DeepSeek V4 Triggers a Global AI Price War with OpenAI

DeepSeek V4’s open‑source 1 M‑token MoE model delivers benchmark scores of MMLU 88.7, C‑Eval 92.1 and HumanEval 69.5, while its 4‑bit AWQ quantization, PagedAttention memory management and FlashAttention acceleration cut inference costs and latency, prompting rivals such as Anthropic, OpenAI, Baidu and Huawei to slash prices and boost efficiency in a fierce market battle.

AI EfficiencyMoEdeepseek-v4
0 likes · 9 min read
How DeepSeek V4 Triggers a Global AI Price War with OpenAI
AI Engineer Programming
AI Engineer Programming
Apr 25, 2026 · Artificial Intelligence

Quantization Across Signal Processing, AI Inference, and RAG Vector Search

This article explains how quantization—originating from signal processing—reduces precision to save resources, details its application to neural network weights and activations via PTQ, QAT, GPTQ, AWQ, and SmoothQuant, and shows how vector quantization enables fast, memory‑efficient retrieval in large‑scale RAG systems.

AWQGPTQLLM
0 likes · 19 min read
Quantization Across Signal Processing, AI Inference, and RAG Vector Search
Woodpecker Software Testing
Woodpecker Software Testing
Apr 24, 2026 · Artificial Intelligence

Practical Guide to Optimizing Large Model Performance in Production

This guide details how enterprises can move large language models from lab to production by defining specific SLI/SLO metrics, diagnosing hidden bottlenecks such as tokenizer latency, and applying four quantifiable optimization levers that dramatically improve latency, throughput, and cost efficiency.

GPU optimizationLoRAcontinuous batching
0 likes · 6 min read
Practical Guide to Optimizing Large Model Performance in Production
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 19, 2026 · Artificial Intelligence

Qwen3.6-35B: 4‑bit Quantization, DFlash Speedup, Claude Opus Distillation

The article reviews three optimization paths for the Qwen3.6‑35B model—four‑bit AWQ quantization variants, the DFlash speculative decoding accelerator, and a Claude Opus‑based distillation—detailing their implementation steps, benchmark results, and guidance on selecting the best version for different hardware and performance needs.

AIDFlashQwen3.6
0 likes · 11 min read
Qwen3.6-35B: 4‑bit Quantization, DFlash Speedup, Claude Opus Distillation
DataFunSummit
DataFunSummit
Apr 19, 2026 · Artificial Intelligence

How to Build a Multimodal Product Search Engine with Embedding and Vector Retrieval on Elasticsearch Serverless

This article explains a complete multimodal product search solution that combines text and image embeddings, dense, sparse, and hybrid models, vector similarity metrics, and Elasticsearch Serverless features such as dense_vector, sparse_vector, hybrid search, quantization, and RRF ranking to achieve fast, accurate, and cost‑effective retrieval.

AIElasticsearchEmbedding
0 likes · 20 min read
How to Build a Multimodal Product Search Engine with Embedding and Vector Retrieval on Elasticsearch Serverless
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 18, 2026 · Artificial Intelligence

How to Run MiniMax‑M2.7 on Mac: Comparing Two Quantization Paths

This article explains why standard uniform quantization fails for the 228‑billion‑parameter MiniMax‑M2.7 MoE model on macOS, and compares two practical solutions—JANGTQ + MLX Studio with 2‑bit mixed‑precision achieving 91.5 % MMLU using 56.5 GB, and LM Studio + GGUF which is easier but requires at least 138 GB RAM and yields lower accuracy.

JANGTQLM StudioMLX Studio
0 likes · 8 min read
How to Run MiniMax‑M2.7 on Mac: Comparing Two Quantization Paths
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 12, 2026 · Artificial Intelligence

How to Deploy MiniMax-M2.7 Quantized Models Locally on macOS and Linux

This guide explains the 22 GGUF quantized versions of MiniMax-M2.7 released by Unsloth, compares their accuracy and size, recommends the UD‑Q4_K_XL model for best quality‑to‑size trade‑off, and provides step‑by‑step instructions for local deployment via Unsloth Studio, llama.cpp, API server, or the MLX native solution, along with important pitfalls and performance‑tuning tips.

Dynamic 2.0MLXMiniMax-M2.7
0 likes · 14 min read
How to Deploy MiniMax-M2.7 Quantized Models Locally on macOS and Linux
AI Tech Publishing
AI Tech Publishing
Apr 9, 2026 · Artificial Intelligence

Engineering‑Focused Guide to Training and Inference of Large Language Models

This article walks engineers through the full LLM stack—from tokenization and positional encoding to transformer blocks, efficient fine‑tuning, quantization, and production‑grade inference techniques such as KV‑cache, FlashAttention, PagedAttention, continuous batching, and speculative decoding—highlighting trade‑offs, toolchains, and practical workflow steps.

AttentionLLMLoRA
0 likes · 13 min read
Engineering‑Focused Guide to Training and Inference of Large Language Models
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Apr 8, 2026 · Artificial Intelligence

Dissecting Gemma‑4’s Architecture and Training Choices: A Technical Comparison with Qwen‑3 and GLM‑5

This article breaks down every architectural and training decision behind Gemma‑4—KV sharing, p‑RoPE, per‑layer embeddings, and a dual‑path MoE + dense MLP—while contrasting its efficiency and performance with Qwen‑3 and GLM‑5 across benchmarks, quantization strategies, and RL pipelines.

GLM-5Gemma 4LLM Architecture
0 likes · 23 min read
Dissecting Gemma‑4’s Architecture and Training Choices: A Technical Comparison with Qwen‑3 and GLM‑5
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
Apr 8, 2026 · Artificial Intelligence

Unlocking 8‑Hour Autonomous Coding: GLM‑5.1’s Leap with Kunlun XPU

The open‑source GLM‑5.1 model, adapted to Baidu Baige's Kunlun XPU via the vLLM‑Kunlun Plugin, delivers record‑breaking SWE‑bench scores, eight‑hour autonomous coding, long‑context handling up to 64K tokens, and scalable deployment across tens of thousands of chips, showcasing end‑to‑end AI acceleration.

GLM-5.1Kunlun XPUlong context
0 likes · 8 min read
Unlocking 8‑Hour Autonomous Coding: GLM‑5.1’s Leap with Kunlun XPU