Developer Anemll demonstrates that the DS4 IQ2_Q2 version of DeepSeek V4 on an Apple M5 Max gains a 5‑tps throughput boost, using SSD‑streamed MoE sidecar loading to run large models without requiring high memory, and provides full build and execution instructions.
The ds4.c project, authored by Redis founder Salvatore Sanfilippo, is a Metal‑only C inference engine that uses asymmetric 2‑bit quantization, disk‑based KV caching, and OpenAI/Anthropic‑compatible APIs to achieve usable performance for DeepSeek V4 Flash on high‑end Apple Silicon Macs.
The DeepSeek V4 Flash quantized GGUF model and the dedicated ds4 inference engine, both released by antirez, offer dramatically reduced activation parameters, massive 1‑million‑token context windows, aggressive KV‑cache compression and hardware‑specific quantizations that enable smooth local inference on high‑memory Macs and CUDA machines, while sacrificing generality for performance.
