Skip to content
Tagged articles

memory bandwidth

26 articles · Page 1 of 1
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 15, 2026 · Artificial Intelligence

4× DGX Spark vs M5 Ultra: Why Higher Bandwidth Mac Loses on GLM-5.3-Flash

A developer's benchmark shows four NVIDIA DGX Spark units interconnected via 200Gb/s networking outperform a single 512GB Mac Studio M5 Ultra on GLM-5.3-Flash inference across prefill, code generation, and concurrent workloads, despite the Mac's higher 1.23TB/s memory bandwidth, due to software stack differences, KV cache compression, and batching efficiency.

GLM-5.3-FlashKV CacheLLM inference
0 likes · 13 min read
4× DGX Spark vs M5 Ultra: Why Higher Bandwidth Mac Loses on GLM-5.3-Flash
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 9, 2026 · Industry Insights

M7 Base 240GB/s vs M5 Pro 307GB/s: Apple's AI Bandwidth Claim Debunked

Analysis of Apple Silicon rumors shows M7 base model's rumored 240GB/s memory bandwidth is lower than shipping M5 Pro's 307GB/s, while M7 Ultra won't arrive until 2028 — an 18-month gap with no verified specs — making current M5 Max/Ultra the only proven high-bandwidth options for large LLM inference today.

Apple SiliconLLM inferenceM5 Pro
0 likes · 11 min read
M7 Base 240GB/s vs M5 Pro 307GB/s: Apple's AI Bandwidth Claim Debunked
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 6, 2026 · Artificial Intelligence

M6 Launch Day: 9.1 vs 10.6 tok/s — Should You Upgrade 16GB to 32GB for Local LLMs?

Real-world tests show M6 MacBook Pro hits 9.1 tok/s on Qwen3.8-27B 4-bit, reaching 86% of the 10.6 tok/s bandwidth ceiling; the article explains why the dual neural engine doesn't accelerate token generation, debunks the 8-model limit as a software filter, and provides a three-tier buying guide based on memory bandwidth and model size.

Apple SiliconBuying GuideLLM inference
0 likes · 13 min read
M6 Launch Day: 9.1 vs 10.6 tok/s — Should You Upgrade 16GB to 32GB for Local LLMs?
Leo Talks Systems Thinking
Leo Talks Systems Thinking
Nov 6, 2026 · Industry Insights

How Large Models Are Redefining Chip Architecture: Bandwidth, Energy, and the Gamble on Future Algorithms

This article analyzes how large language models are reverse-defining chip architecture through three fundamental principles: memory bandwidth bottlenecks dominate over compute, data movement energy costs dwarf computation, and algorithmic shifts rewrite hardware specs, framing three industry strategies as risk-pricing bets on algorithmic stability.

AI chip architectureRoofline modelautomotive AI
0 likes · 22 min read
How Large Models Are Redefining Chip Architecture: Bandwidth, Energy, and the Gamble on Future Algorithms
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 2, 2026 · Artificial Intelligence

Why 70B Struggles on 64GB MacBook Pro: 5 Deployment Iron Laws

This guide details five iron laws for deploying large language models on a 64GB MacBook Pro with M5 Pro chip, covering memory budgeting (45GB usable), model tier recommendations (32B smooth, 70B borderline), inference engine selection, speedup techniques like speculative decoding, and API serving best practices.

Apple SiliconLLM DeploymentMacBook Pro M5 Pro
0 likes · 10 min read
Why 70B Struggles on 64GB MacBook Pro: 5 Deployment Iron Laws
Lao Guo's Learning Space
Lao Guo's Learning Space
Oct 28, 2026 · Artificial Intelligence

M6 Mac Memory Tiers: Max LLM Size & Speed for 16/24/32GB (Real Tests)

This article analyzes Apple M6 Mac memory configurations (16GB/153GB/s, 24GB/170GB/s, 32GB/170GB/s) for local LLM inference, using official specs and community benchmarks to determine which models (7B–32B) fit each tier, their token speeds, and buying recommendations for different user needs.

Apple M6Apple SiliconBuying Guide
0 likes · 8 min read
M6 Mac Memory Tiers: Max LLM Size & Speed for 16/24/32GB (Real Tests)
Ops Development & AI Practice
Ops Development & AI Practice
Sep 26, 2026 · Artificial Intelligence

Mac mini M6 32GB LLM Inference: Real TPS, Bandwidth Limits & Hybrid Cloud Strategy

This analysis dissects the 32GB Mac mini M6's true LLM inference capabilities, revealing 170 GB/s unified memory bandwidth limits, GPU compute constraints versus AMD Strix Halo, measured tokens-per-second for 8B–32B models, and a cost-driven argument for hybrid local-cloud deployment over expensive local-only workstations.

Apple SiliconLLM inferenceM6 chip
0 likes · 21 min read
Mac mini M6 32GB LLM Inference: Real TPS, Bandwidth Limits & Hybrid Cloud Strategy
Lao Guo's Learning Space
Lao Guo's Learning Space
Sep 23, 2026 · Artificial Intelligence

Mac mini 16GB to Studio 256GB: Which Mac Runs Your LLMs Best?

This article analyzes Mac mini and Mac Studio configurations for local LLM inference, showing how unified memory capacity determines which models fit and memory bandwidth dictates token generation speed, with real-world benchmarks for 8B to 235B models across M6, M5 Pro, M5 Max, and M5 Ultra chips.

Apple SiliconLLM inferenceMac Studio
0 likes · 13 min read
Mac mini 16GB to Studio 256GB: Which Mac Runs Your LLMs Best?
Lao Guo's Learning Space
Lao Guo's Learning Space
Sep 22, 2026 · Industry Insights

M5 Ultra 64 vs 80 Core GPU: What ¥9750 Buys – Not Faster LLM Decoding

The article compares Apple M5 Ultra 64-core and 80-core GPU variants, revealing that the ¥9750 price difference adds 16 GPU cores, 6 CPU cores, and 512GB memory support, but identical 1.2TB/s memory bandwidth means LLM decoding speed remains unchanged; the upgrade only benefits GPU-intensive tasks like 3D rendering and video effects.

3D renderingApple M5 UltraBuying Guide
0 likes · 10 min read
M5 Ultra 64 vs 80 Core GPU: What ¥9750 Buys – Not Faster LLM Decoding
Refining Core Development Skills
Refining Core Development Skills
Aug 25, 2026 · Fundamentals

NVIDIA Maxwell Architecture: How SM Partitioning Drove 20x FP32 Growth in a Decade

The article analyzes NVIDIA's Maxwell GM200 architecture, detailing how splitting each SM into four independent processing blocks improved scheduling efficiency and core utilization, boosting FP32 performance to 6.84 TFLOPS on the Tesla M40 — a 20x increase over 10 years — while highlighting limited FP64 capability and memory bandwidth scaling challenges.

FP32 performanceGM200GPU architecture
0 likes · 12 min read
NVIDIA Maxwell Architecture: How SM Partitioning Drove 20x FP32 Growth in a Decade
Machine Heart
Machine Heart
May 16, 2026 · Artificial Intelligence

Why More Compute Can't Fix LLM Inference Lag and Why RL Leads to Overtraining

In a deep interview, former Google TPU architect Reiner Pope explains that low‑concurrency fast‑mode services trade higher fees for faster streaming but are limited by memory‑bandwidth bottlenecks, that optimal concurrency balances compute and memory costs, and that pipeline‑parallel sparse expert models and reinforcement‑learning fine‑tuning introduce new inefficiencies and overtraining risks.

LLMOvertrainingPipeline Parallelism
0 likes · 7 min read
Why More Compute Can't Fix LLM Inference Lag and Why RL Leads to Overtraining
AI Frontier Lectures
AI Frontier Lectures
Jan 12, 2026 · Industry Insights

Why LLM Inference Hits a Memory Wall – Four Hardware Research Directions

The article analyses the challenges of large‑language‑model inference, highlighting memory bandwidth and interconnect as the primary bottlenecks, and presents four research opportunities—high‑bandwidth flash, processing‑near‑memory, 3D memory‑logic stacking, and low‑latency interconnect—while evaluating current Nvidia solutions and proposing integrated architectural approaches.

3D stackingAI hardware researchLLM inference
0 likes · 22 min read
Why LLM Inference Hits a Memory Wall – Four Hardware Research Directions
Architects' Tech Alliance
Architects' Tech Alliance
Mar 31, 2025 · Industry Insights

GPGPU vs ASIC: Who Wins the AI Compute Race?

This article analyzes the trade‑offs between GPGPU and ASIC for AI workloads, covering precision, compute density, power efficiency, memory bandwidth, interconnect technologies like NVLink, and the strategic reasons why leading firms are investing in custom AI chips.

AI chipsASICGPGPU
0 likes · 8 min read
GPGPU vs ASIC: Who Wins the AI Compute Race?
Architects' Tech Alliance
Architects' Tech Alliance
Mar 30, 2025 · Industry Insights

Why Memory, Not Compute, Is the Bottleneck for Next‑Gen AI Chips

The article analyzes the rapid growth of AI model memory and compute demands, the slow increase of chip memory capacity, and argues that memory bandwidth and energy consumption, rather than raw compute, will dominate AI chip design, emphasizing multi‑tenancy, DSA flexibility, and data‑flow optimization.

AI chipsDSAMulti-tenancy
0 likes · 7 min read
Why Memory, Not Compute, Is the Bottleneck for Next‑Gen AI Chips
Architects' Tech Alliance
Architects' Tech Alliance
Mar 13, 2025 · Fundamentals

How Memory Bandwidth and Latency Shape CPU Performance

The article explains how CPU computation latency arises from memory speed, bandwidth, and access delays, detailing the relationships among memory, bandwidth, and latency, and examines key factors such as clock frequency, pipelining, parallelism, cache hit rate, and signal propagation distances that together determine overall system performance.

CPUcomputer architecturelatency
0 likes · 9 min read
How Memory Bandwidth and Latency Shape CPU Performance
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Mar 24, 2021 · Cloud Computing

LIBRA and CARE: Memory Bandwidth Management and Fault‑Tolerance Innovations Presented at HPCA 2021

The article reviews two HPCA 2021 papers from Alibaba Cloud—LIBRA, a dynamic memory‑bandwidth management framework that boosts data‑center utilization, and CARE, a cache‑based fault‑tolerance architecture that delivers near‑Chipkill reliability with minimal overhead—while also highlighting future research directions in ML systems, quantum computing, and cache computing.

HPCA2021Resource Utilizationcloud computing
0 likes · 4 min read
LIBRA and CARE: Memory Bandwidth Management and Fault‑Tolerance Innovations Presented at HPCA 2021