Lao Guo's Learning Space

Nov 1, 2026 · Artificial Intelligence

Two RTX 3090s Hit 3145 tok/s on 180B Model: Strata Engine Brings Giant LLMs to Gaming PCs

The Strata inference engine enables Qwen3.8-Flash-Next (180B total params, 6B active) to achieve 3145 tokens/sec prefill and 120 tokens/sec decode at 400K context on dual RTX 3090s by offloading experts to CPU/SSD, using speculative decoding and chunked prefill, offering a low-cost alternative to Mac Studio for long-context workloads.