Self‑Supervised Learning and Contrastive Methods for Computer Vision and OCR Applications

In machine learning, the scarcity of high‑quality labeled data is a major challenge; self‑supervised learning (SSL) addresses this by generating pseudo‑labels from the data itself and training on predefined pretext tasks.

The article, originating from Laiye Technology’s internal CV sharing, first defines SSL and places it among other learning paradigms: supervised, unsupervised, and reinforcement learning, highlighting its growing importance as emphasized by Yann LeCun and Yoshua Bengio.

It then enumerates common computer‑vision pretext tasks, including predicting image rotation, relative patch positions, jigsaw puzzles, image colorization, auto‑encoding, GAN‑based image restoration, and contrastive learning, each illustrated with example figures.

Contrastive learning’s core problems are identified as (1) how to construct positive and negative sample pairs and (2) how to design effective loss functions. The article reviews four typical losses—Contrast Loss, Triplet Loss, N‑Pair Loss, and InfoNCE Loss—providing their mathematical forms and intuition.

Typical contrastive frameworks are described: SimCLR (batch‑based positives/negatives with InfoNCE), MoCo (momentum encoder with a queue of negative keys), clustering‑based methods such as SwAV and SmAV, and asymmetric designs like SimSiam and BYOL. Key implementation details are shown, for example the MoCo momentum update:

@torch.no_grad()
def _momentum_update_key_encoder(self):
    """Momentum update of the key encoder"""
    for param_q, param_k in zip(self.encoder_q.parameters(), self.encoder_k.parameters()):
        param_k.data = param_k.data * self.m + param_q.data * (1. - self.m)

For clustering‑based SmAV, the article outlines the use of prototypes, optimal transport via Sinkhorn, and swapping prediction, accompanied by the relevant loss diagram.

In the OCR case study, Laiye applies SSL to captcha recognition. After pre‑training on 800 k unlabeled captcha images using SimCLR and SimSiam, a feature‑flattening function converts sequence features to a larger batch size:

def feature_flat(self, feature):
    dim = tf.shape(feature)[2]
    feature = tf.keras.layers.AvgPool1D(pool_size=5, padding="same")(feature)
    feature = tf.reshape(feature, [-1, dim])
    return feature

The augmentation pipeline includes random noise, Gaussian blur, jitter, color‑channel conversion, grayscale, contrast/brightness adjustment, and random scaling. Training proceeds with self‑supervised pre‑training (SGD, lr = 3e‑4, 10 epochs), followed by fine‑tuning on 40 k labeled samples (ADW optimizer, lr = 3e‑4, 10 epochs) and final fine‑tuning of the whole network (ADW, lr = 3e‑5, 40 epochs). Results show accuracy improvements from 0.9081 (supervised only) to 0.9541 (SimSiam + fine‑tune) and 0.957 (SimCLR + fine‑tune).

The article concludes that self‑supervised pre‑training can substantially boost OCR performance with limited labeled data, and provides an extensive bibliography of related works.