How Deep Convolutional Networks Boost Image Super-Resolution: A Paper Review

Image Super‑Resolution

Image super‑resolution (SR) reconstructs a high‑resolution (HR) image from a low‑resolution (LR) input. In deep learning the LR image is the network input and the model learns a mapping to the HR ground‑truth.

Contributions of the SRCNN Paper

SRCNN introduces a fully‑convolutional architecture that maps LR patches directly to HR patches with minimal pre‑ or post‑processing.

The work establishes a relationship between deep‑learning‑based SR and traditional sparse‑coding methods.

It demonstrates real‑time inference capability.

Method Overview

The authors define three operations:

Patch extraction and representation – overlapping LR patches are extracted and paired with corresponding HR patches.

Non‑linear mapping – the network learns to map each LR patch to its HR counterpart.

Reconstruction – predicted HR patches are assembled into a full‑size image.

Patch Extraction Details

Using the T91 dataset, 32×32 patches are extracted with a stride of 14, yielding 24,800 training patches. The same procedure applied to 395,909 ImageNet detection images produces over 5 million patches.

SRCNN Architecture

The network consists of three convolutional layers with the following parameters:

n₁ = 64 filters, kernel size f₁ = 9×9

n₂ = 32 filters, kernel size f₂ = 1×1

n₃ = 1 output channel, kernel size f₃ = 5×5

Training Pipeline and Hyper‑parameters

Loss function – mean‑squared error (MSE):

Peak signal‑to‑noise ratio (PSNR) in dB is the primary evaluation metric; SSIM and MSSIM are mentioned as alternatives.

Weight update – stochastic gradient descent with standard back‑propagation:

Training uses 8×10⁸ back‑propagation steps, batch size 128, and the same hyper‑parameters for all experiments.

Experimental Results

Training on T91 yields 32.39 dB PSNR; training on ImageNet yields 32.52 dB, both surpassing sparse‑coding baselines.

Filter‑number Experiments

Using 128 filters in the first layer and 64 in the second achieves 32.60 dB PSNR but increases inference time compared with the baseline (64/32).

Filter‑size Experiments

Configuration 9‑3‑5 (f₂=3) yields 32.66 dB PSNR; 9‑5‑5 (f₂=5) improves to 32.75 dB, with a corresponding runtime increase.

Depth Experiments

Adding a fourth convolutional layer does not improve PSNR; training time and model complexity increase.

Comparison with Other Methods

On Set5 and Set14, SRCNN outperforms bicubic interpolation and traditional sparse‑coding methods in PSNR.

Runtime

Measurements on an Intel CPU @ 3.10 GHz with 16 GB RAM show that the 9‑1‑5 variant is the fastest while still delivering top PSNR. Larger filter configurations (9‑5‑5, 9‑3‑5) provide higher PSNR with modest runtime overhead.

Conclusion

SRCNN demonstrates that a shallow fully‑convolutional network can achieve state‑of‑the‑art super‑resolution performance, establishing a strong baseline for subsequent GAN‑based methods.