How Deep Learning Unwarps Curved Document Images for Better OCR

Background

Rapid business growth and higher credit requirements have made document verification essential for services such as Alipay ID checks and 1688 business licenses. OCR is the first step for machines to read text, but image quality—especially distortion, blur, and skew—greatly affects recognition accuracy.

Related Work

Traditional Methods

Hardware‑based correction using 3D scanning.

3D model reconstruction to estimate and rectify deformation.

Content‑based correction that analyzes tilt, text lines, and character features without explicit geometry.

These methods work well in specific scenarios but lack generalization.

Deep Learning Methods

Recent advances treat document unwarping as a pixel‑wise regression problem using semantic segmentation networks such as U‑Net. Stacked U‑Net trained on synthetic warped images can perform end‑to‑end correction with better generalization.

Dataset Generation

Because public datasets for warped documents are scarce, a synthetic dataset was created by simulating folding and curling using graphics techniques. Labels were generated as 3‑D tensors containing grayscale values and displacement vectors for each pixel.

Issues such as empty pixels after integer rounding were solved with nearest‑neighbor interpolation.

Model Construction and Optimization

U‑Net Based Correction

The standard U‑Net encoder‑decoder architecture was used, but produced coarse predictions with artifacts like text distortion and tearing.

Stacked U‑Net

Two U‑Nets were stacked: the first produced a coarse prediction, which was concatenated with the original warped image and fed to the second U‑Net for refined output.

Loss Function Improvements

In addition to the standard RMSE loss, a scale‑invariant loss was added to reduce relative error of displacement vectors. An L1 loss formulation yielded finer details than L2.

Smoothing Post‑Processing

Simple smoothing reduced noise and tearing caused by large prediction differences between neighboring pixels.

Dilated U‑Net

Dilated convolutions increase receptive field without pooling, preserving resolution and reducing parameters. Parallel and serial multi‑scale dilated U‑Net designs were tested, with the serial version achieving the best performance.

Model Evaluation

Models were compared on parameter count, training/validation loss curves, and MS‑SSIM scores on a held‑out test set of 100 synthetic warped images. Dilated U‑Net showed the highest MS‑SSIM, lower parameter count, and faster training, confirming the “less is more” principle.

Future Work

Expand the dataset with natural‑scene images and explore GAN‑based data augmentation for better generalization.

Optimize the network for mobile deployment, reducing inference latency.

Investigate advanced architectures such as DeepLab and CRF‑based post‑processing to further improve accuracy.

References

Ma K, et al. DocUNet: Document Image Unwarping via a Stacked U‑Net. CVPR 2018.

Ronneberger O, et al. U‑Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI 2015.

Yu F, Koltun V. Multi‑scale Context Aggregation by Dilated Convolutions. arXiv 2015.

Wang Z, et al. Multi‑scale Structural Similarity for Image Quality Assessment. IEEE 2003.