Real-Time Multi-Shot Long Video Generation: Introducing ShotStream (ECCV 2026)

Generating a few seconds of a single video shot is now routine, but extending generation to multi‑shot long videos traditionally suffers from extreme latency and a lack of interactivity: users must feed hundreds of script lines to a large model, wait half an hour, and restart the whole process if any shot fails.

To break this bottleneck, researchers from The Chinese University of Hong Kong and Kuaishou Keling jointly present ShotStream, the first real‑time streaming framework for multi‑shot long video generation. ShotStream reframes multi‑shot synthesis as a next‑shot prediction task conditioned on historical context, allowing users to steer the narrative at runtime with dynamic streaming prompts. The model runs at 16 FPS on a single H200 GPU, a >25× inference speed improvement over prior bidirectional models.

Key Innovations

Streaming multi‑shot architecture: Enables dynamic adjustment of story direction during generation, overcoming the static, bidirectional pipelines of earlier work.

Dual‑cache memory mechanism: A Global Cache stores sparse condition frames to maintain cross‑shot consistency, while a Local Cache holds freshly generated frames for intra‑shot smoothness. To avoid temporal ambiguity when both caches are accessed, the authors modify the original rotary position encoding (RoPE) with a discontinuous phase shift at shot boundaries, cleanly separating global and local contexts without adding extra modules.

Two‑stage Self‑Forcing distillation: Stage 1 trains a bidirectional teacher model to predict the next shot using ground‑truth history. Stage 2 distills this teacher into a causal student model that uses its own generated history as condition, aligning training and inference distributions and mitigating error accumulation across shots.

Algorithm Details

Stage 1 – Bidirectional next‑shot teacher: The bidirectional video model is fine‑tuned to predict the subsequent shot from historical frames. To prevent memory explosion from hundreds of frames, a dynamic sampling strategy extracts sparse context frames, which are then concatenated with target noise and fed into a DiT model.

Stage 2 – Causal model distillation: Using distribution‑matching distillation (DMD), the teacher is distilled into a causal student that requires only four denoising steps, delivering >25× speedup and real‑time 16 FPS generation.

Dual‑cache and discontinuous RoPE: The Global Cache ensures cross‑shot consistency, while the Local Cache preserves intra‑shot flow. Discontinuous RoPE applies a discrete phase offset at each shot boundary, explicitly decoupling time‑dimension encodings for the two caches and stabilizing the distillation process.

Self‑Forcing training: In the first phase, the model learns next‑shot generation with true ground‑truth history. In the second phase, it trains on its own generated history, forcing the model to correct its imperfections and substantially improving visual stability and coherence across long videos.

Experimental Results

ShotStream outperforms or matches existing bidirectional models (Mask2DiT, CineTrans) and autoregressive models (LongLive) on core metrics such as cross‑shot consistency, shot transition control, and text‑alignment, while delivering real‑time 16 FPS generation.

Conclusion

As the first multi‑shot long video generation model, ShotStream demonstrates that a dual‑cache memory, discontinuous RoPE, and two‑stage self‑forcing distillation can achieve real‑time, interactive storytelling with high visual quality. The open‑source release of its code, models, and training data provides a strong foundation for future AIGC video research.