Mac Studio M5: 128GB vs 256GB for Local LLMs — Memory Ceiling & Speed Trade-offs

Unified Memory: The Only Ceiling

On Apple Silicon, CPU and GPU share the same physical memory — there is no separate VRAM and no "VRAM copy" bottleneck. The maximum model size you can run is strictly limited by the unified memory capacity. Mac Studio M5 offers two relevant tiers: M5 Max caps at 128GB, M5 Ultra at 256GB (higher tiers exist but are not the focus here). Whether you can run a 70B or 405B model depends entirely on this memory number, not the chip marketing name.

How Much Memory Does a Model Actually Consume?

During local inference, a model occupies: Weights + KV Cache (attention cache) + System Overhead (macOS itself uses ~12–16GB; we estimate 14GB).

Weight size is determined by parameter count and quantization precision. 4-bit (Q4) is the most common "good enough" tier; 8-bit (Q8) is near-lossless but doubles the footprint. The table below (estimated for 32k context, 14GB system overhead) shows real-world memory usage for mainstream models:

Key takeaway: 128GB tops out at 70B Q4; 235B and above require 256GB. Note that MoE giants like DeepSeek R1 671B need ~400GB even at Q4 — far beyond 256GB — and are not covered further.

The Real Ceiling of 128GB

7B–32B models run comfortably , with room for very long contexts and even running two or three small models simultaneously for multi-agent experiments.

70B Q4 is very comfortable : ~40GB weights + ~8GB KV + 14GB system ≈ 62GB, leaving half the memory for context and headroom.

70B Q8 can run (~85GB total), but context and headroom become tight; long documents easily hit the ceiling.

120B–235B Q4 (weights ~70–130GB) basically won't fit on 128GB ; even if they squeeze in, KV cache for long context will overflow.

In short: 128GB is the comfort zone for ≤70B models and a dead zone for 100B+.

What 256GB Unlocks

235B models (e.g., Qwen3-235B MoE) Q4 run stably : ~130GB weights + KV + system, still leaving ample space for 100k+ token contexts.

405B Q4 (~235GB) becomes runnable — context must be constrained, but "runnable" is a step from zero to one.

True multi-model parallelism : a 70B primary + a 32B coder + an embedding model can all stay resident, eliminating load/unload cycles in agentic workflows.

For RAG, long-document analysis, and codebase-scale retrieval — tasks that demand long context — 256GB is a qualitative leap . You can drop an entire codebase or dozens of papers into context instead of forced chunking, retrieval, and stitching.

The Overlooked Variable: Memory Bandwidth

Many focus only on capacity, ignoring a harsher metric: memory bandwidth . In unified memory, token generation speed ≈ memory bandwidth ÷ model size. Higher bandwidth means more tokens per second and lower first-token latency.

M5 Ultra's memory bandwidth is nearly double M5 Max's (by Apple's generational scaling: Ultra ~1.0 TB/s, Max ~550 GB/s). This means buying the 256GB tier doesn't just let you fit larger models — it makes the same model run faster .

Concrete example: the same 70B Q4 might yield ~18 tokens/s on M5 Max but 35+ tokens/s on M5 Ultra. For someone interacting with the model dozens of times a day, that perceived speed gap hurts more than "can it run?"

The Hidden Killer: KV Cache

KV cache grows near-linearly with context length:

70B at 32k context: KV ≈ 8GB

70B at 128k context: KV ≈ 30GB

235B at 128k context: KV may hit 60–70GB

This is why some 128GB buyers regret: "70B is only 40GB, why does long-doc crash?" — KV cache is the culprit. The long-context headroom 256GB provides is something 128GB can never give.

Price Breakdown: What the ¥35K Difference Buys

128GB (M5 Max) ≈ ¥35K; 256GB (M5 Ultra) ≈ ¥70K. Difference ≈ ¥35K .

That ¥35K specifically buys:

Model ceiling jumps from 70B to 235B+ — from "usable" to "can play with the big ones";

Near-double inference speed on the same model — daily compounding benefit for heavy users;

Headroom for long context and parallel multi-model workloads — essential for RAG, agents, and privacy-sensitive workflows.

Rental perspective: if you currently spend ¥2,000/month on cloud APIs (¥24K/year), you break even in under two years — and data never leaves your machine, adding unpriced compliance and privacy value.

Decision Cheat Sheet: Which User Are You?

Students / hobbyists / only fine-tuning 7B–32B : 128GB is plenty; don't waste money.

Daily driver 70B + occasional short-context RAG : 128GB works, but you'll frequently hit the ceiling; if budget allows, go 256GB.

Serious users / want 100B+ / multi-agent parallelism / hard privacy requirements : must get 256GB; 128GB will constrain you at every turn.

Teams / research use : start at 256GB, even consider higher memory tiers.

My Conclusion

Back to the opening: don't just look at price, but don't blindly max out either.

My advice is clear: If you can accept 70B as your ceiling and mainly do short-to-medium context, 128GB saves money and is enough. But the moment you want a taste of 100B+ models, or treat AI as a core productivity tool, go straight to 256GB — don't hesitate over the ¥35K gap. Every future "instant reply" and "throw in a long doc freely" moment comes from that extra 128GB.

Buy once, then enjoy the freedom of local LLMs: zero latency, zero leakage, unlimited tinkering.