Skip to content
Tagged articles

Roofline model

6 articles · Page 1 of 1
Leo Talks Systems Thinking
Leo Talks Systems Thinking
Nov 6, 2026 · Industry Insights

How Large Models Are Redefining Chip Architecture: Bandwidth, Energy, and the Gamble on Future Algorithms

This article analyzes how large language models are reverse-defining chip architecture through three fundamental principles: memory bandwidth bottlenecks dominate over compute, data movement energy costs dwarf computation, and algorithmic shifts rewrite hardware specs, framing three industry strategies as risk-pricing bets on algorithmic stability.

AI chip architectureRoofline modelautomotive AI
0 likes · 22 min read
How Large Models Are Redefining Chip Architecture: Bandwidth, Energy, and the Gamble on Future Algorithms
Architects' Tech Alliance
Architects' Tech Alliance
Nov 6, 2026 · Artificial Intelligence

Harvard GPU Lecture Notes: Roofline Analysis of LLM Inference on Hopper & Blackwell

Harvard lecture notes explain GPU hardware principles for LLMs, using the Roofline model to contrast compute-bound prefill vs. memory-bound decode phases, detailing Hopper/Blackwell architectures, memory hierarchy, multi-GPU scaling, and comparing NVIDIA with AMD MI355X, Google TPU Ironwood, and AWS Trainium3 accelerators.

AI AcceleratorsBlackwell ArchitectureGPU
0 likes · 9 min read
Harvard GPU Lecture Notes: Roofline Analysis of LLM Inference on Hopper & Blackwell
Architects' Tech Alliance
Architects' Tech Alliance
Aug 29, 2026 · Artificial Intelligence

How to Decode AI Chip Compute Units and Avoid Common Pitfalls

The article explains why a single GPU can show multiple compute figures, clarifies the meanings of FLOPS, TOPS, precision levels and structured sparsity, demonstrates calculations with RTX 5090 and H100, and provides practical guidance for interpreting theoretical peaks versus real‑world performance.

AIFLOPSGPU
0 likes · 13 min read
How to Decode AI Chip Compute Units and Avoid Common Pitfalls
Machine Heart
Machine Heart
May 16, 2026 · Artificial Intelligence

Why More Compute Can't Fix LLM Inference Lag and Why RL Leads to Overtraining

In a deep interview, former Google TPU architect Reiner Pope explains that low‑concurrency fast‑mode services trade higher fees for faster streaming but are limited by memory‑bandwidth bottlenecks, that optimal concurrency balances compute and memory costs, and that pipeline‑parallel sparse expert models and reinforcement‑learning fine‑tuning introduce new inefficiencies and overtraining risks.

LLMOvertrainingPipeline Parallelism
0 likes · 7 min read
Why More Compute Can't Fix LLM Inference Lag and Why RL Leads to Overtraining
SuanNi
SuanNi
Mar 4, 2026 · Artificial Intelligence

How to Fit Large Language Models into Cars and Robots: A Hardware‑Aware Scaling Law

This article presents a hardware‑aware co‑design framework for edge‑deployed large language models, revealing a scaling law that balances model accuracy and inference latency, and demonstrates how Pareto‑optimal architectures can be discovered quickly using roofline analysis and systematic search on devices like NVIDIA Jetson Orin.

AI inferenceEdge ComputingPareto Optimization
0 likes · 15 min read
How to Fit Large Language Models into Cars and Robots: A Hardware‑Aware Scaling Law