What’s the Real Difference Between GPU and CPU? Why Mining and LLMs Prefer GPUs

Interview Focus Points

Architecture depth: Interviewers want to see if you understand why CPUs have few strong cores and GPUs have many weak cores, i.e., the trade‑off between latency and throughput.

Parallel computing awareness: Identify tasks that suit GPUs versus those that must stay on CPUs.

Scenario transfer: Recognise the commonality of massive, repeatable computation in both mining and large‑model workloads.

Core Answer

In one sentence: CPU is a handful of “old professors”, GPU is tens of thousands of “elementary students”. Professors solve complex problems one at a time; students solve simple addition‑subtraction tasks en masse, giving GPUs far higher aggregate throughput.

Key Comparison (re‑formatted from the original table)

Core count: CPU – a few to a few dozen cores (consumer CPUs up to ~32 cores); GPU – thousands to tens of thousands of CUDA cores (e.g., RTX 4090 has 16,384).

Single‑core capability: CPU – very strong (complex logic, branch prediction, out‑of‑order execution); GPU – weak (simple arithmetic, high parallelism).

Transistor allocation: CPU – ~30 % ALU, ~70 % control logic & cache; GPU – ~90 % ALU, ~10 % control & cache.

Design goal: CPU – low latency (finish a single task quickly); GPU – high throughput (process massive numbers of tasks simultaneously).

Memory bandwidth: CPU – DDR5 ~100 GB/s; GPU – HBM3 ~3 TB/s (order‑of‑magnitude higher).

Typical workloads: CPU – OS scheduling, complex business logic, databases, branch‑heavy code; GPU – matrix operations, graphics rendering, hash calculations, neural‑network training.

Deep Analysis

1. Architecture Comparison: How Transistors Are Distributed

The CPU core packs extensive control units: branch predictors, out‑of‑order engines, deep pipelines (15‑20 stages), and large multi‑level caches, all to make a single task run as fast as possible. This consumes most transistors, leaving less than 30 % for arithmetic logic units (ALU).

The GPU core takes the opposite approach: it minimizes control logic and cache, dedicating about 90 % of transistors to ALUs. A streaming multiprocessor (SM) can hold 128 or more ALUs, and a whole GPU contains thousands of them. Each ALU is simple, but the sheer number yields massive parallel throughput.

Fundamentally, CPU pursues “do one thing extremely fast” (low latency), while GPU pursues “do many things at once” (high throughput). They complement each other: the CPU schedules, the GPU executes the bulk work.

2. Why Mining Uses GPUs

Mining (both Bitcoin’s SHA‑256 and Ethereum’s Ethash) repeatedly computes independent hashes: "compute hash → check difficulty → earn reward". Because each hash attempt is independent, the workload is perfectly parallel.

Each hash calculation is independent; millions of attempts can run simultaneously.

CPU mining: An 8‑core CPU can compute only eight hashes in parallel, yielding a few tens of millions of hashes per second—economically unviable.

GPU mining: Thousands of ALUs compute hundreds of millions of hashes per second, giving a 10‑100× speedup over CPUs.

ASICs: Specialized hardware that hard‑wires the hash algorithm, delivering several orders of magnitude higher efficiency; thus Bitcoin mining has largely moved to ASICs, while GPUs remain popular for algorithms designed to resist ASICs (e.g., Ethereum).

In short, mining is a "hash‑rate race" where higher parallel throughput wins.

3. Why Large‑Model Training Relies on GPUs

Training large language models is dominated by matrix multiplication (MAC). For example, GPT‑3 (175 B parameters) requires roughly 3 × 10²³ FLOPs per forward‑plus‑backward pass. Each element of the result matrix C = A × B is computed independently, making the operation embarrassingly parallel. C = A × B GPU advantages:

Tensor Cores: Introduced with NVIDIA Volta (V100), they perform 4×4 matrix‑multiply‑accumulate in a single cycle, delivering 8‑16× higher throughput than regular CUDA cores. Newer cards (H100, B200) support FP8 and INT4, reaching hundreds of TFLOPs per card.

High‑bandwidth memory: HBM3/HBM3e provides ~3 TB/s bandwidth, over 30× faster than DDR5, preventing ALU stalls during massive data movement.

CUDA ecosystem: Decade‑long software stack (CUDA, cuDNN, cuBLAS) integrates with PyTorch, TensorFlow, and other AI frameworks, giving GPUs a decisive software advantage.

These factors make GPUs the sweet spot for "massively parallel, compute‑intensive" AI workloads.

4. Are CPUs Obsolete?

No. CPUs remain indispensable in the AI pipeline:

Data preprocessing: Image decoding, tokenisation, and augmentation are I/O‑ and logic‑heavy, best handled by CPUs.

Inference for small models: Quantised models (INT8/INT4) can run efficiently on CPUs with modern instruction sets (Intel AMX, ARM SME), especially when latency requirements are modest.

Scheduling and coordination: Multi‑GPU training requires gradient synchronisation, inter‑node communication, and task orchestration, all managed by CPUs.

The guiding rule: if a task is massively parallel, uses simple arithmetic, and processes large data volumes, choose GPU; otherwise, CPU may be more appropriate.

High‑Frequency Interview Follow‑Ups

Why can’t CPUs be built with thousands of cores? Cache‑coherency overhead explodes, memory bandwidth cannot feed many cores, and power/thermal limits cap practical core counts to a few dozen (or a few hundred with NUMA clusters).

What is the GPU programming model? Can Java use it? NVIDIA’s CUDA (C/C++) is dominant. Java options include JCuda (JNI), TornadoVM (JIT to GPU), and DJL (Deep Java Library) for inference, though production AI still favours Python + PyTorch.

Must large‑model inference run on GPUs? Not always; quantised small models can run on CPUs with low latency, but high‑QPS, long‑context services typically need GPU acceleration.

Summary

CPU and GPU have complementary roles: CPUs act as the “commanders” handling control‑intensive, low‑latency tasks, while GPUs are the “troops” executing massive parallel workloads. Mining and large‑model training both fall into the "massive, parallel, compute‑intensive" category, perfectly matching the GPU’s high‑throughput design. Understanding the latency‑vs‑throughput trade‑off, transistor allocation, and ecosystem support lets you answer interview questions confidently and choose the right processor for a given workload.