BestHub
Sign in
Tagged articles

disk streaming

4 articles · Page 1 of 1
Architecture Digest
Architecture Digest
Sep 21, 2026 · Artificial Intelligence

3 AI Open-Source Projects: DeepSeek Agent Runtime, Verified Diagrams, 744B MoE on 25GB RAM

This article reviews three cutting-edge AI open-source projects: DeepSeek's plugin-based Agent runtime framework (deepseek-harness), archify for generating verifiable architecture diagrams directly in coding agents, and colibri, a pure C inference engine enabling 744B MoE models to run on 25GB RAM via disk streaming.

DeepSeekMoEPure C
0 likes · 8 min read
3 AI Open-Source Projects: DeepSeek Agent Runtime, Verified Diagrams, 744B MoE on 25GB RAM
Architecture Digest
Architecture Digest
Sep 13, 2026 · Artificial Intelligence

Running 2.78T-Parameter Kimi K3 on 8GB RAM: Pure C Inference Engine Explained

A pure C inference engine runs the 2.78 trillion parameter Kimi K3 model on just 8GB RAM by streaming weights from disk, leveraging MoE sparsity (only 3.7% active per token) and computing directly on compressed formats, achieving correct output at 32.69 seconds per token on consumer hardware.

C languageKimi K3LLM compression
0 likes · 11 min read
Running 2.78T-Parameter Kimi K3 on 8GB RAM: Pure C Inference Engine Explained
Data STUDIO
Data STUDIO
Sep 11, 2026 · Artificial Intelligence

Running 744B MoE on 25GB RAM: Int4, Disk Streaming & MLA Compression

This article details how a custom C inference engine runs the 744B-parameter GLM 5.2 MoE model on 25GB RAM by quantizing routed experts to int4, streaming them from disk via pread, compressing KV cache 57x with Multi-head Latent Attention, and employing a five-tier memory hierarchy with speculative decoding.

AVX2 kernelsC inference engineGLM-5.2
0 likes · 48 min read
Running 744B MoE on 25GB RAM: Int4, Disk Streaming & MLA Compression