Can One Model Master All Remote Sensing Tasks? Introducing the TSSUN Framework

Introduction

Rapid advances in remote‑sensing technology have produced massive multi‑source datasets that are essential for dense‑prediction tasks such as urban monitoring, land‑cover classification, and disaster assessment. However, these images vary widely in temporal length, spectral channels, and spatial resolution, making it difficult for conventional deep‑learning models, which are typically designed for a fixed input‑output configuration, to generalize across different tasks.

Problem Definition

Dense‑prediction tasks in remote sensing can be unified as a tensor‑mapping problem: given an input tensor of shape (T₁, C₁, H₁, W₁), the model must produce an output tensor of shape (T₂, C₂, H₂, W₂). Three core tasks are considered:

Semantic segmentation : single‑time‑point, multi‑class pixel labeling (T₂=1, C₂≥2).

Semantic change detection : per‑pixel semantic comparison between any two time steps (T₂=T₁, C₂≥2).

Binary change detection : binary decision of whether a change occurred between adjacent time steps (T₂=T₁‑1, C₂=2).

Proposed Method: Temporal‑Spectral‑Spatial Unified Network (TSSUN)

The authors introduce TSSUN, a unified network that decouples and then re‑integrates temporal, spectral, and spatial dimensions, allowing a single model to process arbitrary combinations of input and output configurations. TSSUN supports all three dense‑prediction tasks and flexible numbers of output classes.

Network Overview

Input stage : a Spectral‑Spatial Unified Module (SSUM) encodes heterogeneous spectral and spatial data into a common representation.

Feature extraction stage : a Local‑Global Window Attention (LGWA) mechanism captures multi‑scale local and global context efficiently.

Encoder‑decoder bridge : a Temporal Unification Module (TUM) merges temporal features and adapts the output temporal length according to the task.

Output stage : a second SSUM refines the decoded features, ensuring consistent spectral‑spatial reconstruction for any number of classes.

Temporal‑Spectral‑Spatial Unified Strategy (TSSUS)

TSSUS introduces a Dimension Unified Module (DUM) that generates adaptive linear weights and biases from metadata using a hyper‑network. The process consists of:

Metadata embedding with positional encoding and a learnable [CLS] token.

Cross‑variable relationship modeling via multiple Transformer blocks.

Adaptive parameter generation: the [CLS] token produces bias vectors, while remaining tokens generate weight matrices for a linear layer that maps input features to a unified space.

Local‑Global Window Attention (LGWA)

LGWA addresses the high computational cost of full‑image attention by employing three overlapping window sizes for local attention and a separate global attention block. This design balances efficiency and expressive power, enabling effective feature extraction for large‑scale remote‑sensing images.

Experiments

The authors evaluate TSSUN on six datasets covering two scenarios: building‑focused datasets (WHU, WHU‑CD, LEVIR‑CD, TSCD) and land‑use/land‑cover (LULC) datasets (LoveDA Urban, Dynamic EarthNet). For each scenario, a single TSSUN model is trained on the combined training sets and tested on each dataset’s test split.

Results on Building Scenario

TSSUN achieves the highest IoU and F1 scores on all four benchmarks, e.g., 91.00 % IoU and 95.29 % F1 on the WHU dataset, outperforming existing state‑of‑the‑art methods.

Results on LULC Scenario

On LoveDA, TSSUN records the best overall accuracy (71.82 %) and mean IoU (65.73 %). On Dynamic EarthNet, it leads all reported metrics, achieving SCS = 29.9, BC = 38.9, and mIoU = 54.7.

Contributions

Proposed TSSUN, a unified network that handles arbitrary temporal‑spectral‑spatial configurations and multiple dense‑prediction tasks without task‑specific retraining.

Designed the LGWA mechanism to efficiently capture both local and global context.

Conducted extensive experiments on diverse remote‑sensing datasets, demonstrating that a single TSSUN model matches or exceeds specialized state‑of‑the‑art methods.