Architecture Digest

Oct 9, 2026 · Artificial Intelligence

How Strata Runs a 125B Model on a 12GB Gaming GPU

Strata is an open-source inference engine that enables running the 125B-parameter Qwen3.8-Flash-Next MoE model on consumer gaming PCs with 12GB VRAM by tiering experts across GPU, RAM, and SSD, using speculative decoding for 1.6-1.8x speedup, and providing OpenAI/Anthropic-compatible APIs for local coding agents.