Skip to content
Tencent's DSpine Accelerates Qwen3.8-27B 3.84x with 2B Draft Model

Tencent's DSpine Accelerates Qwen3.8-27B 3.84x with 2B Draft Model

Tencent open-sources DSpine, a speculative decoding framework that uses a 2B draft model to accelerate Qwen3.8-27B, achieving 3.84x throughput on WorkBuddy Office via adjacent causal injection and transition cache, outperforming DFlash2 across math, coding, and agent benchmarks.

SmartApe AI
SmartApe AI
SmartApe AI

Background: Speculative Decoding and Parallel Draft Models

Traditional LLMs generate tokens autoregressively, causing serial latency. Speculative decoding introduces a lightweight draft model to predict multiple candidate tokens in parallel, which the target model verifies in a single forward pass. If candidates match, multiple tokens are accepted at once; on mismatch, verification corrects at the first rejection point.

Parallel draft models (like DFlash) generate a whole block of candidates in one forward pass, avoiding serial drafting steps. However, all positions compute simultaneously without knowing the final token chosen at preceding positions, leading to logical conflicts. Verification proceeds left-to-right, so a single mismatch discards all subsequent candidates, reducing average acceptance length.

DSpine: Adjacent Causal Injection and Unified Transfer Space

DSpine addresses this with adjacent causal injection : after each layer, each position receives the predicted feature from its immediate left neighbor via a gating mechanism, writing it into its residual stream. This builds causal dependencies progressively across layers.

To make feature transfer effective, DSpine introduces a Unified Transfer Space built from the target model's output embeddings, allowing both uncertain predicted features and confirmed token embeddings to share the same transfer mechanism.

Additionally, Layer-wise Output-Embedding Supervision aligns each layer's predicted features with the correct token embeddings.

Transition Cache for Parallel Precomputation

During token selection, a candidate's score depends on the final token chosen at the previous position. Computing these pairwise transition scores serially would reintroduce latency. DSpine precomputes all adjacent candidate-pair conditional scores in parallel, stores them in a Transition Cache , and then performs serial selection via simple lookups, achieving "parallel precomputation + serial selection."

Experimental Results

Qwen3-4B and Qwen3-8B Benchmarks

On seven math, code, and chat benchmarks, DSpine achieves leading average acceptance lengths. For Qwen3-8B at temperature 0, average acceptance length rises from 3.77 (DFlash) to 4.82 (+27.8%). In SGLang serving tests, throughput improves 23.3% over DFlash on average.

Acceptance length comparison on Qwen3-4B and Qwen3-8B
Acceptance length comparison on Qwen3-4B and Qwen3-8B
SGLang throughput comparison
SGLang throughput comparison

Scaling to Qwen3.8-27B

The companion draft model Qwen3.8-27B-DSpine-b16 (≈2B parameters, 4.05 GB) is released on Hugging Face. At temperature 0, DSpine achieves average acceptance lengths of 7.20 on GSM8K (vs. 5.69 for DFlash2, +26.5%), 6.49 on MATH-500 (vs. 5.49), 5.71 on HumanEval (vs. 4.87), and 6.14 on MBPP (vs. 5.04). All five tasks lead DFlash2.

Acceptance length on Qwen3.8-27B
Acceptance length on Qwen3.8-27B

Agent Task Throughput (WorkBuddy)

On WorkBuddy Code, Web, Office, SWE-bench Pro, and τ²-bench, DSpine's end-to-end throughput exceeds DFlash2. In WorkBuddy Office low-inference mode, throughput reaches 805.53 Token/s , 3.84× the original autoregressive decoding speed.

WorkBuddy throughput comparison
WorkBuddy throughput comparison

The DSpine framework and draft model are open-sourced, enabling faster LLM inference without modifying the target model.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

Speculative DecodingBenchmarkQwen3.8-27BLLM Inference AccelerationDSpineDraft ModelAdjacent Causal InjectionTransition Cache
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.