How DeepSeek’s Lightning Indexer Enables Efficient Sparse Attention for Long Texts

When a language model reads an extremely long text such as the full novel Dream of the Red Chamber , the naive attention cost grows quadratically with the number of tokens, making a 128K‑token context computationally prohibitive.

Lightning Indexer as a Memory‑Filtering Expert

The Lightning Indexer (LI) solves this problem by quickly estimating an index score for every preceding token relative to the current query token. It uses a small number of indexer heads (a few attention heads) and a ReLU activation to keep the computation lightweight, aiming for high throughput.

Index Score Formula

The score is calculated with a compact formula (Formula 1) that can be expressed as: score = Σ_{head} ReLU(q·k) * q_s_frag[head] Here q and k are the query and key vectors, q_s_frag is the per‑head scaling factor, and the sum runs over the few indexer heads. Because the number of heads is tiny, the operation is far cheaper than the full multi‑head attention used in earlier DeepSeek‑V3.1‑Terminus.

Top‑k Selection to Reduce Core Attention

After scoring, the fine‑grained token selection mechanism keeps only the Top‑k entries. For example, if the novel contains 100 000 tokens, the LI selects just 2 048 key‑value pairs, and the main model’s attention is performed on these 2 048 tokens instead of the original 128 K, dramatically lowering the core attention complexity.

Mapping the Formula to the FP8 Kernel

The implementation lives in the fp8_index_kernel function of the DeepSeek‑V3.2‑Exp code base:

def fp8_index_kernel(h: int, d: int):
    # allocate shared memory in FP8
    q_smem = T.alloc_shared((h, d), FP8)
    k_smem = T.alloc_shared((blk_n2, d), FP8)
    # copy query and key tensors
    T.copy(q[i_b, i_m, 0, 0], q_smem)
    T.copy(k[i_b, i1_n * blk_n1 + i2_n * blk_n2, 0], k_smem)
    # matrix multiplication (dot product)
    T.gemm(k_smem, q_smem, logits, transpose_A=False, transpose_B=True, clear_accum=True)
    # ReLU activation and scaling
    for i_h, i3_n in T.Parallel(h, blk_n2):
        logits[i3_n, i_h] = T.max(logits[i3_n, i_h], 0) * q_s_frag[i_h]
    # sum across heads
    T.reduce_sum(logits, logits_sum, dim=1)
    # final key scaling
    logits_sum[i3_n] *= k_s_frag[i3_n]

Each line corresponds directly to a step of the formula: the gemm performs the dot product, T.max implements ReLU, multiplication by q_s_frag applies the query scaling, reduce_sum aggregates across heads, and the final multiplication by k_s_frag applies the key scaling.

Resulting Efficiency Gains

Because the LI uses only a few heads and FP8 arithmetic, its computational cost is orders of magnitude lower than the full attention used previously. The top‑k pruning reduces the effective token count from 128 K to 2 048, enabling DeepSeek‑V3.2‑Exp to handle very long contexts with manageable memory and latency.