TTTIR: Instance-Specific State Evolution for Image Restoration via Test-Time Training

Background Challenge

Image restoration faces degradations that are inherently instance-specific: even within the same task (e.g., low-light enhancement or deraining), degradation severity, spatial distribution, and frequency composition vary widely across images. Yet most restoration models rely on a single set of shared parameters learned during training, limiting their ability to adapt to each test image's unique degradation.

Core Problem

Test-Time Training (TTT) makes optimization part of the forward pass, allowing the current input to generate internal learning signals that update lightweight fast weights. However, standard TTT focuses on modeling the current input or context, whereas image restoration has a clear directional goal: the degraded state is not the target but should evolve toward a more structurally complete and detail-rich state. Simply enabling the operator to change is insufficient; the model must also be guided on where to evolve next.

Method Overview: TTTIR

TTTIR reformulates image restoration as progressive instance-specific state evolution , decomposing the process into two coupled sub-problems:

Progressive State Sequence Generator (PSSG) — constructs a series of progressive target states from complementary spatial and frequency dimensions, explicitly organizing the restoration trajectory from global structure recovery to local detail reconstruction. This provides a clear restoration direction for the inner loop.

State Transition Evolver (STE) — uses the next target state as inner-loop guidance. It constructs an internal learning signal from the current state and the next target state, dynamically updating a lightweight transition operator. Because this update is driven by both the current image and its target state, different inputs induce distinct operator trajectories, achieving image-specific state transition.

The overall algorithm (Algorithm 1) alternates between generating the next target state via PSSG and performing the state transition via STE.

Experimental Results

TTTIR is evaluated on multiple benchmarks:

Low-light enhancement (LOL-v2-Real) : 30.78 dB PSNR / 0.9070 SSIM

Rain streak removal (five benchmarks average) : 32.35 dB PSNR / 0.918 SSIM

Dehazing (Haze4K) : 28.06 dB PSNR / 0.970 SSIM

The low-light configuration uses only 0.787M parameters .

Progressive State Verification

Figure 2(b) shows the DWT energy distribution of target states across evolution levels. Energy shifts from low-frequency structural information toward high-frequency details, confirming the designed progression from Structure Recovery → Detail Reconstruction. This demonstrates that restoration states are not redundant intermediate representations but play distinct roles in the evolution.

Instance-Specific Operator Verification

Figure 2(c) displays the relative operator updates induced by different test images across restoration stages. Distinct inputs produce different fast-weight update trajectories, confirming image-specific adaptation.

Scalability

Figure 3 compares inference speed and GPU memory across input resolutions. At comparable parameter counts, TTTIR maintains low inference overhead and memory usage, indicating good scalability to high-resolution images.

Ablation Study

Table 4 shows that both PSSG and STE contribute stable gains. Replacing the Next-State Target with a Current-State Target degrades performance, validating the necessity of restoration-oriented state transition.

Conclusion & Outlook

TTTIR is not a direct transplant of TTT to image restoration; it rethinks what the TTT inner loop should learn: shifting from reconstructing the current state to learning the next state transition. By combining explicit restoration direction with input-driven operator adaptation, each image follows its own state evolution path. The core of instance-specificity lies not in discarding shared knowledge but in redefining the boundary between shared recovery capability and instance-specific computation — the former provides stable restoration ability, while the latter dynamically adjusts state transitions to adapt shared knowledge to the concrete instance.

Resources

Code: https://github.com/Elysiaaaaaaaa/TTTIR.git

Paper: http://doi.org/10.13140/RG.2.2.32985.48485

Code example

本文
约1700字
，建议阅读
5分钟
本文介绍TTTIR框架，依托测试时训练实现实例专属的图像状态演化修复。