Machine Heart

Sep 27, 2026 · Artificial Intelligence

How DeepSeek mHC Uses Its Residual Streams: Selective Routing and Near-Identity Mixing

Analysis of DeepSeek-V4-Flash's mHC architecture shows its four residual streams are not uniformly utilized; read/write weights concentrate on ~2 streams per layer, and deep-layer residual mixing matrices approach identity. Interventions removing weakest paths or replacing deep mixing with identity cause minimal performance drops (≤0.38% avg score), suggesting simplification opportunities.