Captcha Recognition in Practice: Front‑End Engineers Skip UI to Train Models

Introduction

Front‑end developers who are used to Canvas, WebGL, and image‑upload components are already dealing with pixel matrices, which is conceptually similar to convolutional neural networks. With AI assistance, even TypeScript developers can train a CNN.

Background

Two types of 4‑character captchas were used: a simple set (light noise, normal distortion) and a complex set (severe character adhesion and rotation). Different training strategies were required.

Solution 1: DDDD low‑code trainer

Using the dddd_trainer tool, training is reduced to “configure + run command”. Data are organized as flat folders with filenames label_random.ext or via a labels.txt mapping. On an RTX 4070S the simple captcha model reaches >97 % validation accuracy after about 10 minutes of training.

Environment

conda create -n captcha python=3.10
conda activate captcha
pip install torch torchvision dddd_trainer

Configuration example

Model:
CharSet: []
ImageChannel: 1
ImageHeight: 64
ImageWidth: -1
System:
GPU: true
Val: 0.03
Train:
BATCH_SIZE: 64
CNN: {NAME: ddddocr}
LR: 0.01
TARGET:
Accuracy: 0.97
Epoch: 20

Solution 2: AI‑assisted PyTorch CNN

When the low‑code approach failed on the complex captchas, the team asked an LLM for a complete solution. The model skeleton (config, dataset, model, train) was generated, and after half an hour of data adaptation and hyper‑parameter tuning the training succeeded.

Project structure generated by AI

captcha_cnn/
├── config.py      # constants
├── dataset.py     # data loading and cleaning
├── model.py       # CNN definition
├── train.py       # training script
└── data/
    └── raw/       # raw images, e.g. a3b9_001.png

Key code snippets

Config defines image size 420×80, character set of 36, and training parameters (batch size 64, epochs 60, LR 1e‑3). The model uses four convolutional layers, adaptive average pooling to 2×4, and four independent classification heads (one per character).

class CaptchaCNN(nn.Module):
    """Input: (B,1,H,W) → Output: (B,MAX_LEN,NUM_CLASSES)"""
    def __init__(self):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(1,32,3,padding=1), nn.BatchNorm2d(32), nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(32,64,3,padding=1), nn.BatchNorm2d(64), nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(64,128,3,padding=1), nn.BatchNorm2d(128), nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(128,256,3,padding=1), nn.BatchNorm2d(256), nn.ReLU(),
            nn.AdaptiveAvgPool2d((2,4))
        )
        flat = 256*2*4
        self.heads = nn.ModuleList([
            nn.Sequential(
                nn.Linear(flat,256), nn.ReLU(), nn.Dropout(0.3),
                nn.Linear(256,NUM_CLASSES)
            ) for _ in range(MAX_LEN)
        ])

    def forward(self, x):
        feat = self.features(x).flatten(1)
        return torch.stack([h(feat) for h in self.heads], dim=1)

The training script reports character‑level and sequence‑level accuracy, uses Adam with cosine annealing, and selects the best model by sequence accuracy.

Practical tuning

Initial LR 1e‑3 was too high; lowering to 5e‑4 improved convergence.

Num_workers set to 2 on a Mac MPS; the AI‑suggested value 4 caused crashes.

RandomRotation limited to 5° after the AI‑suggested 15° produced out‑of‑bounds characters.

Results

For the complex captchas (≈10 k samples, 60 epochs) the final test sequence accuracy reached 99 %. The entire AI‑generated pipeline from first code to final model took about half an hour.

Takeaways for Front‑End Teams

AI can generate boilerplate model code, letting developers focus on data cleaning and hyper‑parameter tuning.

Pixel‑level intuition from Canvas/WebGL work translates to understanding convolutions.

Engineering practices such as centralized config, data validation, and performance monitoring map directly to model‑training workflows.

Front‑end engineers can move from consuming OCR APIs to producing their own models and even exporting to ONNX for browser inference.

Conclusion

AI is not a replacement but an amplifier of a front‑end developer’s capabilities, turning the “electric screwdriver” into a tool for model training as well as UI work.