Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization

The traditional pipeline for 2D animation coloring requires artists to manually color each frame, which is time‑consuming and labor‑intensive. Recent advances in artificial intelligence enable automatic color transfer from a single reference frame to the remaining line‑art frames, dramatically reducing production cost.

This article introduces the research paper “Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization” (CMFT), accepted at WACV 2021. The method models the task as an image‑analogy problem: given a colored reference frame and its corresponding line‑art, the network predicts the colored version of other line‑art frames in the same shot.

The core component, the Correlation Matching Feature Transfer (CMFT) module, first extracts line‑art features from both the reference and target frames, computes a pixel‑wise similarity matrix, and then uses this matrix as attention weights to transfer color features from the reference to the target. The process is analogous to self‑attention but is explicitly designed for cross‑domain feature transformation.

The overall architecture consists of three encoders (line‑art encoder, reference‑color encoder, and a backbone encoder) and a single decoder. CMFT is embedded in a coarse‑to‑fine manner, progressively refining the transferred features. A line‑art semantic network is also incorporated to enrich the first‑stage feature matching.

For training data, the authors select pairs of frames that are far apart within the same shot, using a sliding window with stride 5. From three animated movies they generate about 60 k frame pairs. This strategy encourages the model to handle large motions and diverse color distributions.

Experimental results on seven real‑world animation movies show that the CMFT‑based model outperforms current state‑of‑the‑art methods (including TCVC, DeepAnalogy, and Pix2Pix) in both PSNR and SSIM metrics, while achieving a processing speed of roughly 0.7 seconds per frame.

The paper discusses future directions such as character‑aware coloring, application to comic coloring, and broader industrial deployment. The authors estimate that the AI‑driven coloring engine can reduce manual labor by about 30 % in dynamic‑animation pipelines, and they emphasize the need for close collaboration between researchers and artists to integrate such technology into existing workflows.