How Deep Learning Unwarps Curved Document Images for Better OCR

Background

Rapid business growth and stricter credit requirements have made document verification essential for services such as Alipay ID checks and 1688 business license reviews. OCR accuracy directly impacts the effectiveness of intelligent verification, but scanned or mobile images often suffer from curling and folding.

Artificial intelligence can greatly enhance verification efficiency. To achieve high‑level intelligent verification, the OCR stage must first convert visual text into machine‑readable characters, and then NLP techniques can interpret the content.

Image quality—specifically tilt, clarity, and distortion—dominates OCR performance. This work focuses on correcting distorted document images to improve OCR accuracy.

Related Work

Traditional Methods

Hardware‑based correction using specialized scanners or structured light to capture 3‑D shape information.

3‑D model reconstruction that models document pose, lighting, and device characteristics to undo distortion.

Content‑segmentation methods that analyze tilt angles, text lines, and character features without explicit geometric modeling.

These approaches work well in constrained scenarios but lack generalization.

Deep Learning Methods

Recent advances treat dewarping as a pixel‑wise regression problem using semantic‑segmentation networks. A stacked U‑Net trained on synthetically generated warped documents (CVPR 2018) demonstrated end‑to‑end correction capability and better generalization to complex folds.

Dataset Generation

Because public datasets for warped documents are scarce, a synthetic dataset was created following the method in [1]. The process simulates both curling and folding using graphics equations, generates corresponding displacement labels for each pixel, and resolves empty‑pixel artifacts via nearest‑neighbor interpolation.

Model Construction and Optimization

U‑Net Based Correction

The classic U‑Net encoder‑decoder architecture extracts multi‑scale features and restores image resolution through transposed convolutions and skip connections. However, raw U‑Net predictions exhibit text distortion, line misalignment, and occasional tearing.

Stacked U‑Net

Two U‑Nets are stacked: the first provides a coarse prediction used as a prior, and the second refines the result by concatenating the prior with the original warped image, improving detail preservation.

Loss Function Improvements

Standard L2 loss leads to large errors in character shapes. Adding a scale‑invariant loss term reduces relative displacement errors, and an L1‑style loss further improves fine‑grained accuracy.

Post‑Processing Smoothing

Simple smoothing of the predicted displacement field mitigates isolated noisy pixels and discontinuities.

Dilated U‑Net

Replacing standard convolutions with dilated convolutions expands the receptive field without pooling, keeping resolution and parameter count low. Both parallel and serial multi‑scale dilated U‑Net designs were tested, with the serial version yielding the best results.

Model Evaluation

Model size, training/validation loss curves, and MS‑SSIM scores were compared across U‑Net, Stacked U‑Net, and Dilated U‑Net. Dilated U‑Net achieved the smallest parameter count, fastest training, and highest MS‑SSIM, confirming the “less is more” principle.

Key findings:

Dilated U‑Net outperforms the other two architectures.

L1‑based loss functions produce sharper corrections than L2.

Smoothing consistently improves all models.

Future Work

To further enhance performance:

Expand the dataset with natural‑scene images and explore GAN‑based data augmentation for better generalization.

Optimize the network for mobile deployment, reducing latency.

Investigate advanced segmentation backbones such as DeepLab and CRF‑based post‑processing.

References

[1] Ma K, Shu Z, Bai X, et al. DocUNet: Document Image Unwarping via a Stacked U‑Net. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, 4700‑4709.

[2] Ronneberger O, Fischer P, Brox T. U‑Net: Convolutional Networks for Biomedical Image Segmentation. International Conference on Medical Image Computing and Computer‑Assisted Intervention, 2015, 234‑241.

[3] Yu F, Koltun V. Multi‑scale Context Aggregation by Dilated Convolutions. arXiv preprint arXiv:1511.07122, 2015.

[4] Wang Z, Simoncelli E, Bovik A. Multi‑scale Structural Similarity for Image Quality Assessment. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003, 1398‑1402.