16-Model Segmentation Showdown: CNN, Transformer & Mamba on DeepGlobe Land Cover

Introduction

This project conducts a comprehensive benchmark of 16 semantic segmentation models for remote sensing land cover classification on the DeepGlobe Land Cover dataset. All models are trained under a unified protocol : identical data splits, loss function (Dice + CrossEntropy), learning rate scheduler (warmup + cosine), gradient clipping, early stopping, EMA, and evaluation via global confusion matrix mIoU (aggregated over the entire validation set, not batch-averaged). The 16 models comprise 15 mainline models (CNN, Transformer, Mamba, and hybrid architectures) plus one RTX 4090 experimental line (wangbq-net). Five models (U-Net, UNet++, DeepLabV3+, SegFormer-B0, SegFormer-B1) are implemented via segmentation_models_pytorch (SMP); the remaining 11 are custom implementations or paper reproductions. The repository also provides a full large-image inference pipeline: tiling → batched inference → merging → raster-to-vector (GeoJSON/Shapefile).

Dataset

The DeepGlobe Land Cover Classification Challenge (CVPR 2018 Workshop) dataset is used. Official sources: deepglobe.org (registration required) and a Kaggle mirror (recommended). The dataset contains 1,146 training images (2448×2448 RGB) with pixel-level color-coded masks for 7 classes: urban land, agriculture land, rangeland, forest land, water, barren land, unknown. Test set has 154 images without public labels. Masks are RGB color-coded; class definitions are in class_dict.csv.

In practice, after filtering via metadata.csv for existing mask files, 803 images remain. These are split internally with

seed=42</strong> into <strong>683 training / 120 validation</strong> images (official validation labels are missing). The directory structure follows the Kaggle layout:</p><pre><code>DeepGlobeLandCoverClassification/   # dataset root (--data_dir points here)
├── metadata.csv                # image index (sat_image_path / mask_path / split)
├── class_dict.csv              # class color definitions
├── train/
│   ├── xxx_sat.jpg             # satellite image
│   └── xxx_mask.png            # color mask
└── test/
    └── xxx_sat.jpg

Model Overview

Custom / Reproduced Models (9)

Each has a dedicated file imported by models.py :

1. Lite-DeepLabV3+ ( lite_deeplabv3.py) – custom lightweight ASPP decoder.

2. CFFormer-L ( cfformer_l.py) – cross-flying attention, CPU-friendly Transformer segmentation.

3. TerraSegNet ( terrasegnet.py) – EfficientNetV2-S backbone + custom decoder.

4. CM-UNet ( cm_unet.py) – Mamba-inspired lightweight UNet decoder (2024).

5. SegNeXt-S ( segnext.py) – ConvNeXt-style encoder + Hamburger decoder (requires torch.manual_seed(42) for determinism).

6. OffSeg-B ( offseg.py) – orthogonal attention efficient segmentation (ICCV 2025).

7. CGRSeg-T ( cgrseg.py) – content-guided real-time segmentation (ECCV 2024).

8. U-MixFormer-B1 ( umixformer.py) – MiT encoder + hybrid decoder (WACV 2025; encoder parameterizable).

9. wangbq-net ( wangbq_net.py) – RTX 4090 experimental line: MiT-B2/B3/B4/B5 + U-MixFormer decoder (reuses umixformer.py), supports 768×768 resolution.

Shared backbone file: efficientformer_v2_backbone.py used by OffSeg-B and CGRSeg-T.

SMP Library Models (7)

Created directly via segmentation_models_pytorch in models.py (5 architectures, 7 models due to two backbone-swap variants):

U-Net (MobileNetV2 encoder) – baseline.

UNet++ (MobileNetV2) – mainline.

UNet++ Eff-B0 (EfficientNet-B0) – backbone swap variant.

DeepLabV3+ (MobileNetV2) – mainline.

DeepLabV3+ Eff-B1 (EfficientNet-B1) – backbone swap variant.

SegFormer-B0 (MiT-B0 encoder).

SegFormer-B1 (MiT-B1 encoder) – current accuracy champion at 0.7438 mIoU .

Training & Evaluation Core Modules

run_training.py

– unified entry for training, evaluation ( --evaluate), visualization; all parameters configurable via YAML or CLI overrides. train.py – training loop: Dice+CE loss, warmup+cosine scheduler, gradient clipping, early stopping, EMA, global confusion matrix mIoU. dataset.py – DeepGlobe loader driven by metadata.csv, 512×512 random crops, four augmentation levels (none/light/medium/strong). models.py – model factory: model name + encoder → network; includes encoder whitelist validation. test_module.py – evaluation: TTA (4 transforms), per-class IoU, confusion matrix, prediction visualization. inference.py – single-image inference function used by large-image pipeline.

Large-Image Inference Pipeline

tile_large_image.py

– tiles large images with configurable overlap. process_large_image.py – batched inference on tiles. merge_tiles.py – stitches predictions back to full-size raster. raster_to_vector.py – converts classification raster to vector (GeoJSON/Shapefile).

Prediction Visualization

predict_single_samples.py produces quad-panel figures (original | ground truth | prediction | error map) and a summary CSV of mIoU scores.

Configuration Files

config_<model>_cpu.yaml

×15 – per-model CPU training configs (batch size, learning rate, epochs, augmentation level, etc.). config.yaml – common template. config_wangbq_net_4090.yaml – RTX 4090 config for wangbq-net (768² resolution, MiT-B2, EMA). requirements.txt – pinned dependencies.

Installation & Usage

1. Environment

git clone <repo-url>
cd DeepGlobeLandCoverClassification
# Python 3.10+ recommended, create virtual environment
pip install -r requirements.txt

2. Data Preparation

Download from Kaggle, extract, note root path (e.g., D:\datasets\DeepGlobeLandCoverClassification ).

3. Training Commands (16 models)

Each model has a dedicated YAML config. Example for SegFormer-B1:

python run_training.py --config config_segformer_b1_cpu.yaml --data_dir D:\datasets\DeepGlobeLandCoverClassification

CLI overrides supported: --batch_size 16 --lr 0.0001 etc. Checkpoints saved to checkpoints/<model>_<encoder>_<timestamp>/ with weights and training.log .

4. Evaluation & Prediction

# Test set evaluation with TTA and per-class IoU
python run_training.py --evaluate --checkpoint checkpoints/<model_dir>/<best>.pth --split test

# Single-sample visualization
python predict_single_samples.py --help

Best mIoU Comparison (15 models, unified protocol)

Validation set: 120 images, global confusion matrix mIoU, seed=42 split.

U-MixFormer-B1 (0.7385) – fastest convergence (within 40 epochs), best single-sample prediction quality.

SegFormer-B1 (0.7438) – highest overall mIoU (champion).

U-Net (0.6067) – classic baseline; all models should be compared against it.

Backbone-swap experiments: UNet++ Eff-B0 (0.5964) and DeepLabV3+ Eff-B1 (0.6822) show that blindly swapping to stronger ImageNet backbones can yield negative returns ; encoder-decoder compatibility is the decisive factor.

wangbq-net (MiT-B2 + 768 resolution) results pending RTX 4090 training completion.

License & Repository

Licensed under Apache License 2.0 (see LICENSE ). The complete open-source repository is hosted on Gitee; access keyword: DeepGlobeLandCoverClassify . The project is described as the most comprehensive open-source engineering effort for DeepGlobe land cover segmentation, covering CNN/Transformer/Mamba architectures, unified protocol, accuracy leaderboard, visualization, large-image inference, and raster-to-vector pipeline. Long-term maintenance and iterative improvements are planned.