Tencent's DSpine Accelerates Qwen3.8-27B 3.84x with 2B Draft Model
Tencent open-sources DSpine, a speculative decoding framework that uses a 2B draft model to accelerate Qwen3.8-27B, achieving 3.84x throughput on WorkBuddy Office via adjacent causal injection and transition cache, outperforming DFlash2 across math, coding, and agent benchmarks.
Background: Speculative Decoding and Parallel Draft Models
Traditional LLMs generate tokens autoregressively, causing serial latency. Speculative decoding introduces a lightweight draft model to predict multiple candidate tokens in parallel, which the target model verifies in a single forward pass. If candidates match, multiple tokens are accepted at once; on mismatch, verification corrects at the first rejection point.
Parallel draft models (like DFlash) generate a whole block of candidates in one forward pass, avoiding serial drafting steps. However, all positions compute simultaneously without knowing the final token chosen at preceding positions, leading to logical conflicts. Verification proceeds left-to-right, so a single mismatch discards all subsequent candidates, reducing average acceptance length.
DSpine: Adjacent Causal Injection and Unified Transfer Space
DSpine addresses this with adjacent causal injection : after each layer, each position receives the predicted feature from its immediate left neighbor via a gating mechanism, writing it into its residual stream. This builds causal dependencies progressively across layers.
To make feature transfer effective, DSpine introduces a Unified Transfer Space built from the target model's output embeddings, allowing both uncertain predicted features and confirmed token embeddings to share the same transfer mechanism.
Additionally, Layer-wise Output-Embedding Supervision aligns each layer's predicted features with the correct token embeddings.
Transition Cache for Parallel Precomputation
During token selection, a candidate's score depends on the final token chosen at the previous position. Computing these pairwise transition scores serially would reintroduce latency. DSpine precomputes all adjacent candidate-pair conditional scores in parallel, stores them in a Transition Cache , and then performs serial selection via simple lookups, achieving "parallel precomputation + serial selection."
Experimental Results
Qwen3-4B and Qwen3-8B Benchmarks
On seven math, code, and chat benchmarks, DSpine achieves leading average acceptance lengths. For Qwen3-8B at temperature 0, average acceptance length rises from 3.77 (DFlash) to 4.82 (+27.8%). In SGLang serving tests, throughput improves 23.3% over DFlash on average.
Scaling to Qwen3.8-27B
The companion draft model Qwen3.8-27B-DSpine-b16 (≈2B parameters, 4.05 GB) is released on Hugging Face. At temperature 0, DSpine achieves average acceptance lengths of 7.20 on GSM8K (vs. 5.69 for DFlash2, +26.5%), 6.49 on MATH-500 (vs. 5.49), 5.71 on HumanEval (vs. 4.87), and 6.14 on MBPP (vs. 5.04). All five tasks lead DFlash2.
Agent Task Throughput (WorkBuddy)
On WorkBuddy Code, Web, Office, SWE-bench Pro, and τ²-bench, DSpine's end-to-end throughput exceeds DFlash2. In WorkBuddy Office low-inference mode, throughput reaches 805.53 Token/s , 3.84× the original autoregressive decoding speed.
The DSpine framework and draft model are open-sourced, enabling faster LLM inference without modifying the target model.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
