DSpark Explained in 10 Essential Concepts: System‑Level Engineering Insights

GPU memory‑bandwidth bottleneck and continuous batching

Large‑model inference is limited by the bandwidth needed to move model weights from VRAM to compute cores for each token. Loading the weights once and decoding several tokens together (continuous batching) makes a ten‑token batch only slightly slower than a single‑token decode because the same memory transfer is amortised over many tokens.

Speculative decoding

LLM generation is autoregressive, so token N+1 depends on token N. Speculative decoding introduces a fast draft model that predicts a short sequence of future tokens. The target model then verifies the whole candidate sequence in a single batch using rejection sampling. The verification step preserves the exact output distribution, so there is no quality loss.

Draft model

A small model (e.g., Qwen 0.8B) can serve as the draft model for a huge model (e.g., Qwen 397B). The draft model quickly produces a candidate token block; the target model performs one forward pass to accept the longest correct prefix and falls back to resampling at the first divergence point.

Latency model

The paper defines token latency as

token latency = (draft time + verification time) / τ , where τ is the number of tokens accepted per draft. Acceleration can be achieved by (1) reducing draft time, (2) increasing τ (more accurate guesses), or (3) reducing verification waste.

Eagle and Multi‑Token Prediction (MTP)

Instead of training a separate small model, Eagle reuses the last‑layer hidden states of the target model and adds one or two lightweight transformer heads as the draft. This yields a draft that is both fast (few layers) and accurate because it inherits the target model’s internal representation.

DFlash: parallel one‑shot generation

DFlash adapts the diffusion‑model idea of generating all token positions in a single forward pass. While this provides a large speed boost, the independent generation causes “suffix decay”: later tokens become increasingly incoherent because each position is sampled without conditioning on previous positions.

DSpark: combining Eagle and DFlash

DSpark fuses the strengths of Eagle and DFlash in two stages. First, a parallel backbone (the DFlash component) produces logits for every position, ensuring raw speed. Second, a lightweight sequential head—by default a Markov head that looks only at the previous token and uses a rank‑256 low‑rank decomposition—injects a prefix‑dependency bias to correct suffix decay. This design keeps overall latency low while dramatically improving acceptance rates.

Variable‑length draft and hardware‑aware scheduling

DSpark predicts how many tokens to draft per request with a confidence head that scores each draft position. The system measures GPU load and, based on pre‑computed throughput curves, dynamically selects the optimal draft length entirely on‑GPU, avoiding CPU overhead.

Online draft confidence calibration

Neural networks tend to be over‑confident, making raw confidence scores unreliable. DSpark applies sequential temperature scaling as a post‑processing calibration step, reducing expected calibration error from 3‑8% to about 1%. The calibration runs online, continuously adapting thresholds to the current workload (e.g., more permissive for code‑generation, stricter for open‑ended chat).

Performance results

Average accepted token length exceeds Eagle 3 by 26%‑31% and DFlash by 16%‑18% in offline tests.

A two‑layer DSpark outperforms a five‑layer DFlash.

Extending draft length from 4 to 16 adds only 0.2%‑1.3% latency while boosting acceptance length by up to 30%.

Open‑source release

The DeepSpec library (GitHub) ships the full training stack for Eagle 3, DFlash, and DSpark, supporting external models such as Qwen 3 and Gemma. Repository URL: https://github.com/deepseek-ai/DeepSpec. The DSpark paper is available at https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf. Related tweets: https://x.com/dzhulgakov/status/2070922887595499930 and https://x.com/Hikari_07_jp/status/2070842526450479188.