VDN-H3 Generates 14-Second 768p Video in 11 Seconds: 75x Speedup Over MiniMax H3

Introduction

Recent viral "infinite-stream" videos powered by MiniMax H3 demonstrate real-time generative video classrooms and interactive Twitch channels. OpenVDN's newly open-sourced Video Delta Net (VDN) builds on MiniMax H3 to deliver 75–90× faster generation: on 8 NVIDIA B200 GPUs, a 14.4-second 768p clip is produced in 11.23 seconds — faster than its playback duration.

Background: The Attention Bottleneck

In video generation models, softmax attention computes quadratically with sequence length. On MiniMax H3 this component consumes over 85% of total runtime. Linear attention is faster but degrades quality, struggling with subject consistency, scene layout, and long-range dependencies.

Hybrid Attention Architecture

VDN adopts a dual-branch design:

Sliding-window softmax branch — handles local detail fidelity.

Linear attention branch — processes long-range context.

Outputs are merged via a gating mechanism, preserving softmax quality while gaining linear-attention speed.

Video Delta Attention (VDA)

Traditional delta-rule updates state per token. Video, however, is naturally frame-structured. VDA lets all spatial tokens within a frame jointly solve a single state update, capturing intra-frame correlations. Mathematically, this update is non-expansive, preventing error accumulation.

Training Strategy

To avoid disturbing the pre-trained MiniMax H3 weights, VDN uses a three-stage regimen:

Layer-wise alignment of the linear branch.

End-to-end adaptation.

LoRA joint fine-tuning (most pre-trained parameters stay frozen).

Finally, DMD2 distillation reduces generation to 8 steps — a deliberate choice over aggressive 4-step distillation to balance quality and speed.

Performance Results

Single B200: per-layer latency drops from 332.5 ms to 125.3 ms (2.65× speedup).

With branch-specific Ulysses sequence parallelism and FP8, 8×B200 achieves 1.4 seconds per step.

8-step generation totals 11.23 seconds.

For comparison, dense H3 on a single B200 requires 13.95 minutes (50 steps) for the same clip. VDN-H3 on 8×B200 with 8-step distillation finishes in 11.23 seconds — a 74.5× overall acceleration.

Quality Comparison

Visual inspection shows the dense H3 and VDN-H3 outputs are nearly indistinguishable, confirming the hybrid design retains quality while drastically cutting latency.

Licensing and Availability

VDN-H3 is a derivative of MiniMax H3 and follows the MiniMax H3 Community License, which excludes the EU, UK, South Korea, and the US. Users should review the license before deployment.

All code, weights, training and inference pipelines are open-sourced:

GitHub repository: https://github.com/OpenVDN/vdn-minimax-h3 HuggingFace weights:

https://huggingface.co/OpenVDN/vdn-minimax-h3

Conclusion

VDN-H3 demonstrates that hybrid attention with frame-level delta updates can break the video generation speed barrier without sacrificing fidelity. The team plans to extend this approach to longer, faster, and truly real-time video synthesis, potentially transforming how generative video is produced and consumed.