Machine Heart
Sep 27, 2026 · Artificial Intelligence
How DeepSeek mHC Uses Its Residual Streams: Selective Routing and Near-Identity Mixing
Analysis of DeepSeek-V4-Flash's mHC architecture shows its four residual streams are not uniformly utilized; read/write weights concentrate on ~2 streams per layer, and deep-layer residual mixing matrices approach identity. Interventions removing weakest paths or replacing deep mixing with identity cause minimal performance drops (≤0.38% avg score), suggesting simplification opportunities.
DeepSeek-V4-FlashHyper-Connectionsarchitecture simplification
0 likes · 12 min read