WorldCast: AI Generates Consistent 3D Camera Moves from Single Photos
Xiaomi's WorldCast framework enables generating 3D camera movement videos from a single photo by combining depth-trajectory scale alignment, geometry-aligned memory attention, and a new CLACT benchmark, achieving state-of-the-art camera control and identity consistency on REDMI K100.
Technical Core: Camera Moves, World Stays Still
Generating video along a specified camera trajectory is an existing research area, but large-angle and complex trajectories expose two coupled failure modes: trajectory deviation (the virtual camera fails to follow the intended path due to geometric inaccuracies) and identity drift (previously unseen regions such as side faces, backs, and hair are hallucinated inconsistently as the viewpoint changes). WorldCast addresses both by first building an explicit geometric scaffold and then supplementing details with multi-view memory.
Scale-Aligned Geometric Guidance: Building the Scaffold
The first step lifts a 2D photo into 3D space: a pre-trained depth estimator produces an initial depth map, which is back-projected into a colored 3D point cloud. This cloud is then rendered along the target camera trajectory to obtain RGB guidance images aligned with each target view. Two key designs ensure fidelity:
Depth-trajectory scale alignment : Monocular depth provides only relative scale, which is misaligned with the absolute translation of the camera. WorldCast enumerates candidate scales, renders each, and selects the scale whose induced image motion best matches a reference motion prior.
Coverage mask : Point-cloud rendering leaves holes in occluded or out-of-view regions. WorldCast outputs a coverage mask that explicitly tells the diffusion model which pixels have geometric support and which must be synthesized by the generative model. The target camera geometry is encoded as Plücker rays and injected into the diffusion model together with the geometric guidance and coverage mask, providing explicit spatial constraints for every frame.
Geometry-Aligned Memory Attention (GAMA): Bringing Evidence to the Right Moment
After the scaffold is built, the appearance of previously unseen views must be supplied. WorldCast uses multi-view anchor images of the same character as supplementary evidence and routes them via GAMA:
A segmentation model extracts the character foreground, removing background interference. A multi-view geometry model then jointly estimates camera poses for the initial frame and all anchor views.
Using the initial frame as a bridge, Umeyama alignment brings the anchor-view geometry and the target trajectory into a shared coordinate system.
In this shared space, the system retrieves the most relevant character features for each target view: when the camera rotates to the left cheek, left-cheek features are fetched; when it moves to the back, back features are fetched. Identity is thus grounded in geometric query rather than model hallucination.
An ablation figure shows that with geometric alignment, attention routing learns faster and converges to higher accuracy.
CLACT: 80,000 Training Samples and 42-Scene Benchmark
Data is a decisive factor. The team constructed CLACT (Character-centric Large-Angle Camera Trajectory dataset), comprising 80,000 training samples (50,000 rendered in Unreal Engine 5, 30,000 in Blender with multi-view character anchors) with precisely controllable camera geometry, and an evaluation set covering 42 synthetic and real character scenes across six trajectory families (orbit, tilt, dolly, lemniscate, pull-back, etc.). This unified benchmark measures trajectory accuracy, cross-view consistency, and visual quality on the same ruler.
Performance: Leading Across All Metrics
On the CLACT benchmark (42 scenes × 6 trajectory types, compared against 13 methods), WorldCast ranks first on every camera-control and identity-consistency metric:
Trajectory precision (ATE) : improved from the previous best 0.098 to 0.065, significantly reducing generation failure rate and eliminating the need for repeated retries.
Cross-view geometric consistency (MEt3R) : best overall, meaning the character maintains a stable 3D structure throughout the motion, appearing as if filmed by a real camera circling a real person.
Identity similarity : ~28% improvement over the second-best method, ensuring facial features, face shape, and hairstyle remain the same person from side profile to back view and back to front.
Visual quality (IQ) : maintained at 0.719, top-tier, with CLIP and DINO appearance similarities also highest, proving that strong geometric and identity constraints do not degrade image fidelity.
In practice on REDMI K100, this translates to a one-tap experience: select a photo, tap once, and obtain a 3D camera-move video with accurate trajectory, stable identity, and high visual quality.
From Paper to Product and Future Directions
Bridging the paper to a shipping feature required additional engineering: algorithm distillation for speed, edge-cloud collaborative deployment, and integration into a broader AI imaging matrix that includes one-click animation, Hitchcock zoom, and timelapse. WorldCast represents Xiaomi's latest breakthrough in controllable visual generation, establishing a foundation of method, dataset, and evaluation protocol. On this foundation, the team plans to explore more complex camera trajectories, longer continuous generation, and controllable 4D video generation.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
