Boosting Small Industrial Image Datasets with ModelArts Augmentation and Evaluation

Problem Background

Users often have very few labeled images for training defect detection models on industrial solar‑panel surfaces, sometimes only five per class, which is insufficient for reliable performance.

Dataset Description

The example uses an open‑source dataset of solar‑panel surface images: 754 normal samples and 358 defective samples for training, with a similar split for validation, totaling about 2000 images.

Initial training with a ResNet‑50 pretrained on ImageNet achieved an overall accuracy of ~86.06%, a recall of 97.3% for the normal class but only 62.9% for the defective class, falling short of user expectations.

Data Augmentation Strategy

Because the images are 300×300 grayscale top‑down views, simple horizontal and vertical flips were applied, doubling the dataset from 1100 to 2200 images.

ModelArts’ smart labeling with manual verification was also explored, and sensitivity analysis identified brightness and clarity as critical features for the defective class.

Targeted augmentations were then applied mainly to the defective class, resulting in a balanced set of roughly 1508 normal and 1432 defective images (≈3000 total).

Model Evaluation with ModelArts

The provided SDK function deep_moxing.model_analysis.analyse was used to generate a JSON analysis of model predictions, revealing that low‑brightness images caused the most errors for the defective class.

def validate(val_loader, model, criterion, args):
    batch_time = AverageMeter('Time', ':6.3f')
    losses = AverageMeter('Loss', ':.4e')
    top1 = AverageMeter('Acc@1', ':6.2f')
    top5 = AverageMeter('Acc@5', ':6.2f')
    progress = ProgressMeter(len(val_loader), [batch_time, losses, top1, top5], prefix='Test: ')
    pred_list = []
    target_list = []
    model.eval()
    with torch.no_grad():
        end = time.time()
        for i, (images, target) in enumerate(val_loader):
            if args.gpu is not None:
                images = images.cuda(args.gpu, non_blocking=True)
            target = target.cuda(args.gpu, non_blocking=True)
            output = model(images)
            loss = criterion(output, target)
            pred_list += output.cpu().numpy()[:, :2].tolist()
            target_list += target.cpu().numpy().tolist()
            acc1, acc5 = accuracy(output, target, topk=(1, 5), i=i)
            losses.update(loss.item(), images.size(0))
            top1.update(acc1[0], images.size(0))
            top5.update(acc5[0], images.size(0))
            batch_time.update(time.time() - end)
            end = time.time()
            if i % args.print_freq == 0:
                progress.display(i)
        print(' * Acc@1 {top1.avg:.3f} Acc@5 {top5.avg:.3f}'.format(top1=top1, top5=top5))
    name_list = val_loader.dataset.samples
    for idx in range(len(name_list)):
        name_list[idx] = name_list[idx][0]
    analyse(task_type='image_classification', save_path='/home/image_labeled/', pred_list=pred_list, label_list=target_list, name_list=name_list)
    return top1.avg

Results

After augmenting to 2940 images (including flips), accuracy improved slightly to 86.31% with similar recall values.

Original: Acc1 86.06%, recall normal 97.3%, recall defective 62.9%.

After augmentation to 2940 images: Acc1 86.31%, recall normal 97.6%, recall defective 62.5%.

Balancing the classes and focusing augmentations on the defective samples increased the dataset to ~3000 images, yielding Acc1 89.13%, recall normal 97.2%, recall defective 71.3%—an 8.4% gain in the critical recall metric.

The experiment shows that systematic data augmentation combined with model analysis can substantially improve defect detection performance, especially when addressing class imbalance and feature‑sensitive issues.