MoFu: Solving Scale & Order Challenges in Multi-Subject Video Generation

Multi-subject video generation aims to synthesize coherent videos from a text prompt and multiple reference images. Current diffusion-based methods struggle with two key issues: scale inconsistency (reference images captured at different distances or crops cause unrealistic relative sizes) and permutation sensitivity (sequential injection of references makes output depend on input order, leading to missing or distorted subjects).

Background and Problem Statement

As the number of subjects grows, these problems amplify. Existing frameworks lack explicit scale modeling and treat the unordered set of references as a sequence, introducing implicit order bias. The paper argues that a unified mechanism addressing both scale consistency and permutation invariance is missing.

MoFu Framework Overview

The proposed MoFu (Scale-Aware Modulation and Fourier Fusion) comprises three components:

Scale-Aware Modulation (SMO) extracts implicit scale cues from text and injects them as continuous modulation parameters into the generation network.

Fourier Fusion (FF) aggregates multi-reference features in the frequency domain, yielding a permutation-invariant unified representation.

Scale–Permutation Stability Loss (SPSL) jointly optimizes scale consistency and order invariance during training.

Scale-Aware Modulation (SMO)

Given a text prompt p, a frozen language model encodes it into a semantic representation h = LM(p). A lightweight Scale Control Adapter (SCA) maps h to modulation parameters γ, β, η: γ, β, η = SCA(h) Inside each DiT block, the intermediate feature F is modulated as: F' = γ ⊙ Layer(F) + β (with residual scaling η)

This injects scale constraints at multiple network depths, letting the model follow textual scale semantics rather than noisy visual scales.

Fourier Fusion (FF)

For N reference images {x₁,…,x_N}, each is segmented and encoded via a lightweight convolutional encoder ε(·) to obtain features f_i = ε(x_i). Features are transformed to the frequency domain: F_i = FFT(f_i) High- and low-frequency components are separated via a radial mask M: F_i^low = F_i ⊙ M, F_i^high = F_i ⊙ (1-M) All references are aggregated by simple summation in the frequency domain (commutative, thus order-agnostic): F_fused = Σ_i F_i The fused representation is transformed back: f_fused = IFFT(F_fused) This unified feature is then fed as a condition to the generator, eliminating order bias.

Scale–Permutation Stability Loss (SPSL)

Scale loss: Uses subject masks M_r and relative areas A_r to compute normalized weights w_r = A_r / Σ A_r. A scale weight map W = Σ_r w_r M_r is built. The scale loss is a weighted MSE between predicted and target noise: L_scale = || W ⊙ (ε_pred - ε_gt) ||² / (||W||₁ + ε) Permutation loss: For a random permutation π of the references, Fourier Fusion produces f_fused and f_fused^π. The loss penalizes their difference: L_perm = || f_fused - f_fused^π ||² Total loss: L_SPSL = L_scale + L_perm.

Experiments

Benchmark: MoFu-Bench

A dedicated benchmark with controlled scaling and random permutations evaluates robustness.

Quantitative Results (Table 1)

MoFu outperforms VACE, MAGREF, SkyReels-A2, and Phantom on ScaleScore and SubjectSim (scale consistency and subject fidelity), while matching Motion Score (dynamics preserved). It also leads on GmeScore and Aesthetics, showing no quality trade-off.

Qualitative Results

Figure 3: MoFu maintains subject consistency, scale correctness, and order robustness across Single-ID, Multi-ID, and mixed scale settings.

Figure 4: Visual comparison with SOTA methods (Phantom, SkyReels-A2, VACE, MAGREF) shows MoFu produces more natural relative sizes and stable layouts under permutation.

Figure 5: Ablation confirms SMO removal breaks scale consistency; FF removal degrades order robustness and overall quality.

Conclusion

MoFu provides a unified solution for multi-subject video generation by explicitly modeling scale from text, fusing references in a permutation-invariant frequency domain, and jointly optimizing both properties via SPSL. The method achieves state-of-the-art stability and controllability, enabling reliable content production for e-commerce advertising and beyond.

Paper: https://arxiv.org/abs/2512.22310 (AAAI 2026)