DeepSeek V4.1 Flash Architecture: The State Lifecycle Behind 1M Context

First Look at Long Tasks

Agent workloads differ from ordinary chat: a long task may read a web page, then a repository, then invoke tools that return command output and test logs. Each tool call appends new input to the model; if the previous state is not cached, a full prefill is required. With long contexts, cost comes not only from computation but also from KV writes, cache migration, and state recovery. KV may reside in HBM, host memory, or SSD, and movement between these layers is bounded by I/O and interconnect bandwidth. Therefore, 1M context is fundamentally a runtime problem: can the state pipeline sustain continuous operation?

Where Compute Is Saved

V4.1 Flash's language backbone has 40 layers: the first 20 are a Causal Encoder (CED), the last 20 are a Decoder. They share a single Transformer backbone split by global-attention responsibility. The key change is in the Decoder's global KV: instead of each layer projecting its own hidden state, the global KV for the last 20 layers is projected directly from the final hidden state of layer 20. This nearly halves prefill compute for long inputs because the model no longer needs to run the entire second half just to produce global KV. The technical report reflects this: prefill activates ~8B parameters per token, while decode activates ~16B per token.

However, the second half is not idle during prefill. Sliding Window Attention (SWA) local KV is still computed per layer. CED eliminates redundant global-KV processing in the upper layers, not all work. This resembles the YoCo paper's direction—lower network produces shareable global KV for upper layers—but V4.1 Flash's CED is a distinct implementation.

Cross-Layer Reuse

CSA2 splits the global attention objects into three parts: main KV: primary KV for global attention indexer K: keys used by the sparse indexer for scoring Top-K indices: positions the current query will actually read

Correspondingly, CSA2 defines three static modes per layer: Full: generate main KV and indexer K, and re-select Top-K Reindex: reuse previous layer's main KV and indexer K, but re-score and re-select Top-K with current query Reuse: reuse Top-K positions as well, skipping new index computation

An important boundary: sharing global KV does not skip the entire layer. Each layer still computes its own global Q and SWA KV. CSA2 reuses storage and index paths, not the Transformer layer itself.

Layer scheduling reflects this trade-off. Encoder layers 3–20 (18 layers) are grouped into 3 blocks of 6 layers each, using 1 Full + 5 Reuse with global KV sequence compression ratio 2. Decoder's 20 layers form 5 groups of 4 layers: first group uses 1 Full + 3 Reuse, the next four groups use 1 Reindex + 3 Reuse, with compression ratio 1. This reduces storage and index compute, but each layer loses some freedom to choose context positions. Reindex provides one re-selection opportunity; Reuse passes the already-chosen positions forward.

First Narrow the Search

Even with cross-layer index reuse, the first index pass faces the full long context. V4.1 Flash adds a candidate-pool narrowing step in the Decoder. The first Full layer scans all causally visible positions, then picks candidate blocks by maximum block score. The technical report example: up to 2,048 blocks of 8 positions each, yielding a candidate pool of at most 16,384 positions. Subsequent Reindex layers only re-score within this fixed pool, making their query cost nearly independent of context length. The initial Full scan remains, so the accurate view is: concentrate full-range search in a few layers, then let later layers work in a bounded range—not all index cost becomes context-length invariant.

Store in Low Precision

CSA2 reduces duplicate saves; FP4 further shrinks each KV entry. V4.1 Flash stores main KV in FP4 E2M1 with one E4M3 scale per 16 channels. Quantization-aware training adapts the network to this low-precision representation. SWA KV is more precision-sensitive and stays in FP8. "Low precision" applies to the storage path: FP4 mainly lowers KV cache storage and data-movement pressure; attention matmuls still dequantize or convert to compute-friendly precision before calculation. Thus ~890 bytes/token is the global KV storage account, not the full per-token runtime cost. 1M tokens ≈ 0.89 GB, excluding model weights, SWA local state, Engram, runtime metadata, and communication buffers.

What Happens When State Expires

Long tasks face state expiration. SWA covers only the recent window, acting like short-term working memory. V4.1 Flash does not persist all SWA KV to SSD; instead it places SWA state in a distributed memory pool of ~10% host DRAM per machine with a minute-level TTL. Global KV is guaranteed at least 72 hours. If SWA state expires, the system can replay recent tokens to recover a usable local state. Exact recovery would require replaying L × nwin tokens per layer; V4.1 Flash replays only the most recent nwin = 128 tokens—this is SWA Bounded Replay.

Bounded Replay is not a full forward-pass equivalent; it is a bounded approximate recovery. It trades a small prefill recomputation for avoiding long-term storage of all SWA KV. The technical report says quality impact is minimal in tested scenarios, but extreme long-context retrieval and session-recovery state changes need further validation. In an Agent Runtime, this makes sense: not all history deserves permanent storage; the key is a cost-controlled, observable recovery path after invalidation.

Engram Records Differently

V4.1 Flash integrates a 196B-parameter Engram. It records a different kind of memory than KV cache. KV cache saves intermediate states from the current context for continuous inference in this task. Engram targets relatively stable static knowledge: it uses tokenizer compression, multi-head hashing, and context-aware gating to map input n-grams into a conditional memory table, then feeds retrieved representations back into the Transformer. It acts like an internal conditional memory path, aiming to reduce repeated heavy computation for static knowledge—but it is not a simple lookup and not zero-cost.

The technical report notes Engram parameters are split evenly across two modules; inference uses host-memory prefetch with RDMA and compute overlap. The Engram paper reported that offloading a 100B table to host memory added <3% latency under its experimental conditions; this cannot be directly extrapolated to 196B on all hardware. Together, Engram and KV cache are two distinct memory paths: one saves current-task intermediate state, the other stores reusable knowledge. Both are called "memory" but differ in lifecycle, access pattern, and eviction policy.

The report also mentions Single-Pass mHC (reduces activation memory traffic) and DSpark (semi-autoregressive draft generation with confidence-based verification for decode efficiency). These affect deployment cost but do not change the core theme: how long-context state is computed, saved, and recovered.

What Environment the Model Runs In

Attention often focuses on model structure, but final behavior depends on the runtime environment. V4.1 Flash trained on 45T multimodal tokens, with long-sequence training first at 64K then extended to 1M; post-training heavily invested in Agent tasks, environments, and verification systems. The report compares the same model under different scaffolds: on DeepSWE v1.1, Claude Code 69.8, Codex 65.6, mini-SWE 74.2; on Terminal-Bench 2.1, Claude Code 88.0, Codex 84.1, mini-SWE 90.3, DeepSeek Harness Minimal 90.6. Changing tool definitions, system prompts, context management, turn control, or verification protocols changes results. Model capability cannot be discussed in isolation from its runtime.

This connects directly to Agent Runtime. A harness is not just attaching tools; the runtime must decide the current working set, fact provenance, when to compress, which states are recoverable, and where to resume after failure. Karpathy calls this "context engineering": task spec, retrieval, tools, state, history, compression, and verification jointly determine what the next call sees. He did not comment specifically on V4.1 Flash; viewed alongside this report, both address different layers of the same problem: model side manages KV state, runtime manages messages, tool results, and recoverable work scenes.

Five Accounts of 1M Context

V4.1 Flash's 1M context is not merely a larger input box. Following the state flow, at least five cost accounts emerge:

Compute account : CED reduces global compute depth for long inputs; decode FLOPs grow more gently with context.

Cache account : CSA2 shares global KV, indexer K, and Top-K across layers, cutting redundant state.

Precision account : FP4 main KV further reduces storage and movement size, though compute still requires precision adaptation.

Persistence account : SWA no longer fully writes to SSD; expiration triggers 128-token bounded replay for approximate recovery.

Retrieval account : Hierarchical Sparse Indexer confines later index layers to a candidate pool; Engram provides a separate conditional memory path for static knowledge.

These five accounts together approximate the complete deployability question for long context. Deriving total cost from 890 bytes/token alone, or extrapolating tok/s from a single test to all hardware, narrows a system problem too much. The design's notable aspect is that different states are not forced into a single storage scheme. Long-term global state, short-term local state, static conditional memory, and replayable recovery state each have their own placement and eviction rules. For Agents, this is closer to real deployment than simply doubling the context window. As context grows, the system must eventually answer: how long is this state kept, and how do we recover when it expires?

References

DeepSeek-V4.1-Flash Technical Report (https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf)

You Only Cache Once: Decoder-Decoder Architectures for Language Models (https://arxiv.org/abs/2405.05254)

Engram: Conditional Memory via Scalable Lookup (https://arxiv.org/abs/2601.07372)

Andrej Karpathy on context engineering (https://x.com/karpathy/status/1937902205765607626)

DeepSeek Harness Deep Dive (https://mp.weixin.qq.com/s/VEVDQBpPJhBt_15C-kjNlw)