Sand.ai Releases First 100B‑Parameter MoE Video Model – 10‑Sec 1080p for $0.05

Sand.ai announced the release of MAGI‑2‑preview , the world’s first open‑source video generation model with a total parameter count of 114 B and a per‑forward activation of roughly 6 B. Using eight NVIDIA H100 GPUs, generating a 10‑second 1080p video costs only about 0.5 CNY (≈$0.07), roughly one‑tenth the cost of mainstream models.

The article explains why scaling video models differs fundamentally from scaling large language models. Video generation must process dense spatial patches for every frame and maintain long‑range temporal dependencies, causing sequence lengths to explode when combined with audio and text. Dense models would make communication and training costs prohibitive, creating a “impossible triangle” of model size, video length, and affordable cost.

To break this triangle, the authors adopt a Mixture‑of‑Experts (MoE) approach that decouples total model capacity from the amount of computation per inference. Only a small subset of experts is activated, keeping the active parameter count manageable while preserving a massive overall capacity.

Key architectural innovations include:

A single‑stream architecture that feeds text, audio, and video into the same Transformer, allowing direct cross‑modal interaction at every layer.

A Multi‑Head Latent MoE design that splits the 3072‑dimensional hidden state into twelve 256‑dimensional heads; each head routes tokens to six independent experts, resulting in 72 active experts per token per layer.

Across 36 layers this yields a total of 3072 expert units, far exceeding the few‑hundred‑expert counts of other MoE models such as DeepSeek‑V4‑Pro.

A Head‑Parallel execution strategy that distributes computation by head before routing, keeping inter‑device communication constant regardless of the number of activated experts.

The custom MagiMoE kernel library that co‑locates routing, sorting, and expert computation to reduce memory movement.

A mixed optimizer setup: matrix parameters are updated with Muon, while expert parameters use AdamW, allowing different learning rates for different parameter groups.

The data pipeline is also redesigned: instead of aggressive filtering that yields a narrow dataset, the team expands the effective data scale, retains breadth, and then organizes it with precise annotations to teach the model fine‑grained audio‑visual correspondences. Pre‑training covers the full data distribution, while downstream fine‑tuning focuses on alignment, controllability, safety, and product‑specific adaptation.

Benchmark results show MAGI‑2‑preview ranking sixth on the AA video generation leaderboard, with visual quality approaching that of the top‑tier closed‑source models despite using only 6 B active parameters. The model’s cost efficiency—producing a 10‑second 1080p clip for roughly one‑tenth the price of comparable systems—demonstrates the practical impact of the scaling solutions.

Beyond technical performance, the open‑source release is positioned as a catalyst for the research community and smaller teams, providing a concrete object for studying expert routing stability, communication overhead, and scaling laws in video MoE models. For enterprises, especially those in data‑sensitive sectors like finance, healthcare, and industry, the availability of a privately deployable, fine‑tunable model opens new possibilities for secure, on‑premise video generation.

Overall, the article argues that while MAGI‑2‑preview is not the final answer for video generation, it demonstrates that a carefully co‑designed stack of model architecture, communication mechanisms, infrastructure, and data handling can make trillion‑parameter video MoE models both runnable and cost‑effective, reshaping the competitive landscape between open‑source and closed‑source solutions.