VINCIE-NExT: Transferring Image Editing to Video via Keyframe Guidance

The paper VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling from ByteDance Seed (led by Zeng Yan, Seedance 2.0 pretraining lead) introduces a framework that leverages mature image editing capabilities for video editing. The core idea is a video→image→edited image→video pipeline: extract keyframes from the source video, edit them with an image editor guided by user instructions, then feed the edited keyframes as visual references to a unified image-video model that generates the full edited video.

Methodology

To propagate keyframe edits consistently across frames, the authors propose TDF3D-RoPE (Temporal Dual-Frequency 3D Rotary Positional Encoding). This encoding simultaneously captures sequential relationships between visual segments and spatial correspondences within segments, allowing the model to distinguish original vs. edited images and align corresponding regions.

Inference uses a Chain-of-Editing (CoE) strategy: Stage 1 generates the edited keyframe; Stage 2 uses that keyframe as a visual exemplar to generate the complete video. Both stages share the same model parameters.

Training Data Composition

The model is trained on three data mixtures:

Image editing data: 1.2M samples

Video editing data: 2.45M samples

Video–original keyframe–edited keyframe triplets: 1.63M samples constructed by prompting GPT-4o for edit instructions and applying Seedream 4.5 to sampled video frames.

This design reduces reliance on fully paired source video–instruction–target video data, though video editing data still participates in training.

Evaluation on OpenVE-Bench

OpenVE-Bench contains 431 videos across 8 task categories (global style transfer, background change, local edit, object removal, object addition, subtitle edit, creative edit, camera viewpoint edit). Scoring uses Gemini 2.5 Pro (1–5 scale).

Overall score: VINCIE-NExT 3.08 vs. OpenVE-Edit 2.49 (~24% relative improvement).

Per-task highlights:

Global style transfer: 4.17 (beats Runway Aleph 3.72)

Local object removal: 3.24 (vs. OpenVE-Edit 1.85)

Camera viewpoint edit: 2.06 (well below Runway Aleph 4.53)

The authors note that viewpoint changes require new geometry reasoning beyond appearance transfer, and tasks needing novel content generation (background change, object addition) still have room for improvement.

Qualitative Analysis & Modularity

Visualizations show successful global style conversions and combined local edits (fire, water, wind) while preserving temporal coherence. Ablations compare direct video-to-video editing, CoE with model-generated keyframes, and CoE with external image editor keyframes. Direct video-to-video produces flickering/ghosting; edited keyframes act as visual anchors that mitigate these artifacts. Plugging in a stronger external image editor improves results without retraining the video model, demonstrating the framework's modularity.

Limitations & Conclusion

VINCIE-NExT establishes a viable path for transferring image editing to video via keyframe guidance, unified image-video modeling, specialized positional encoding, and staged inference. It excels at style changes and local edits but struggles with camera viewpoint shifts, complex geometric transformations, and generating entirely new scene content. The paper was accepted at NeurIPS 2026 and posted on arXiv on October 8.