Beyond DeepSeek: Open‑Source JetSpec and Other Projects Accelerate Large‑Model Decoding Up to 10×

Recent releases such as DeepSeek’s DSpark and the JetSpec project from the Jumpspec team target the growing demand for faster, more stable large‑model outputs when agents invoke models at high frequency.

JetSpec project site: https://jetspec-project.github.io/jetspec-web/

Paper: https://arxiv.org/abs/2606.18394

Open‑source code: https://github.com/hao-ai-lab/JetSpec

In brief, DSpark improves verification efficiency in inference services, while JetSpec enhances the draft generation process itself by using a causal parallel tree to increase the number of tokens accepted per verification step. The former reduces wasted computation at the system level; the latter raises the effective token generation rate at the algorithmic level.

Benchmark results show that DSpark still leaves 60‑85% (Flash model) and 57‑78% (Pro model) speed‑up potential in production systems. JetSpec delivers up to 9.64× end‑to‑end decoding acceleration on Qwen3‑8B compared with standard autoregressive decoding, and achieves average acceptance of 10.76 tokens per verification on the MATH‑500 benchmark. Similar gains are observed on HumanEval (7.12×), LiveCodeBench (7.67×), and MT‑Bench (4.58×).

On H100 GPUs, the figure below compares end‑to‑end speed‑up ratios of DFlash (original block‑parallel draft), DDTree (tree‑variant of DFlash), and JetSpec under a 256‑token tree budget.

The core bottleneck of speculative decoding is not the draft budget but the token‑acceptance rate. When draft cost becomes cheap, the limiting factor shifts to how many parallel candidates can pass the target model’s verification while preserving causal consistency.

The theoretical formula (shown in the next image) relates draft cost, acceptance rate, and draft length to the expected speed‑up.

Even with very low per‑token draft cost, increasing the acceptance rate from 0.85 to 0.95 can raise the theoretical maximum speed‑up beyond 5×, highlighting the “causality‑efficiency dilemma.”

Two draft families are discussed:

Autoregressive drafts (e.g., EAGLE series) : maintain strong causal consistency and high candidate quality, but deeper trees increase serial generation steps and cost.

Block‑parallel drafts (e.g., DFlash series) : use lightweight block‑parallel models to predict many future positions in a single forward pass, drastically lowering draft cost but often producing locally reasonable yet globally inconsistent tokens, which reduces acceptance rate.

Scenario‑driven design choices:

High‑concurrency, throughput‑oriented (DSpark) : keep the parallel draft backbone cheap, add a lightweight serial head and confidence estimator to select which candidates to verify, thereby improving overall throughput without raising per‑request verification cost.

Low‑concurrency, latency‑oriented (JetSpec) : with abundant FLOPs per request, allocate more budget to increase acceptance length, using a causal parallel draft tree to turn extra compute into lower per‑user latency.

DSpark’s budget‑aware correction works as follows: for each draft position i, a parallel draft model generates a token and hidden state; a confidence head estimates a score, and based on a budget‑aware threshold the longest prefix meeting the confidence requirement is sent to the target model for verification.

JetSpec’s approach converts a larger draft budget into higher acceptance length by generating a causal parallel draft tree where deeper nodes depend on earlier tokens in the same branch, leading to higher per‑position acceptance rates across code and math tasks.

Future work envisions a dynamic service framework that pushes both ends of the throughput‑latency Pareto frontier: boosting per‑user generation speed in low‑concurrency settings while maximizing overall throughput under strict verification budgets in high‑concurrency environments. The complementary nature of DSpark and JetSpec—DSpark’s budget‑aware confidence checks for high‑throughput services and JetSpec’s causal parallel drafts for ultra‑low latency—illustrates a promising direction for efficient agent‑driven large‑model deployment.

In the broader Flash model roadmap, JetSpec is not an isolated acceleration paper but part of a series (Step 3.5 Flash → Step 3.7 Flash) that emphasizes efficient inference for agent scenarios, where speed, cost, and tool‑calling capabilities become decisive for user experience and commercial viability.