Recent Advances in Self‑Supervised Learning for Text Recognition

Introduction

Supervised training of deep neural networks for OCR is limited by the high cost of labeled data. Self‑supervised learning (SSL) leverages abundant unlabeled images to learn generic feature representations that can be transferred to downstream text‑recognition tasks. Recent SSL research in computer vision and NLP has shown strong performance, and two main paradigms—contrastive learning and masked image modeling—are now being adapted to OCR.

Typical OCR Model Pipeline

Modern OCR systems treat text recognition as a sequence‑to‑sequence problem and usually consist of optional geometric transformation, a CNN‑based feature extractor, a sequence‑modeling module (e.g., BiLSTM or Transformer encoder), and a decoder (CTC, attention, or Transformer). With the rise of Vision Transformers, the CNN+BiLSTM encoder can be replaced by a pure ViT.

Considerations for SSL on Text Images

While generic SSL tasks such as RotNet can be transferred directly (e.g., TextRotNet), OCR‑specific characteristics require tailored designs. Contrastive methods must respect the sequential nature of text lines, and mask‑based methods should exploit the uniform style and stroke width within a line. Moreover, the encoder used for pre‑training may include only the feature‑extraction part, adding the sequence‑modeling module only during downstream fine‑tuning.

Representative Works

1. SeqCLR

SeqCLR adapts contrastive learning to visual sequences by splitting feature maps into multiple instances, allowing each image to contribute several positive and negative pairs. Instance mapping strategies (all‑to‑instance, frame‑to‑instance, window‑to‑instance) balance negative‑sample quantity and robustness to augmentation. Experiments on handwritten and scene‑text datasets show that SeqCLR outperforms non‑sequential methods such as SimCLR, especially with the window‑to‑instance mapping.

2. SimAN

SimAN combines style‑aware instance normalization with a reconstruction task. Two adjacent patches are cropped from a text line; one is augmented, the other is kept unchanged. The encoder extracts features, which are split into content (instance‑norm) and style (global statistics). Scaled dot‑product attention aligns style, and the decoder reconstructs the patch. An adversarial loss together with L2 reconstruction encourages the encoder to learn representations that capture both content and style, leading to superior probe and semi‑supervised performance.

3. DiG

DiG fuses contrastive learning (MoCo‑v3‑style) and masked image modeling (SimMIM) in a dual‑branch ViT architecture. A randomly masked version and an augmented version of the same text line are processed by a shared encoder. The contrastive branch uses patch‑wise instance mapping and InfoNCE loss, while the mask branch predicts pixel values with an L2 loss. Weighted combination of the two losses yields a model that achieves the best feature‑representation and fine‑tuning accuracy across multiple scene‑text and handwritten benchmarks, surpassing prior SSL methods.

Conclusion

Self‑supervised learning has become a powerful paradigm for OCR, reducing reliance on synthetic data and improving performance on real‑world text images. The surveyed methods demonstrate that carefully designed SSL tasks—whether contrastive, generative, or hybrid—can substantially boost text‑recognition accuracy, and further exploration of this direction is encouraged.