SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observation
Yi Wang, Nassim Ait Ali Braham, Zhitong Xiong, Chenying Liu, Conrad M Albrecht, Xiao Xiang Zhu
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/zhu-xlab/ssl4eo-s12OfficialIn paperpytorch★ 288
- github.com/zhu-xlab/ssl4eo-reviewpytorch★ 38
- github.com/zhu-xlab/dino-mmpytorch★ 32
- github.com/zhu-xlab/softconpytorch★ 23
Abstract
Self-supervised pre-training bears potential to generate expressive representations without human annotation. Most pre-training in Earth observation (EO) are based on ImageNet or medium-size, labeled remote sensing (RS) datasets. We share an unlabeled RS dataset SSL4EO-S12 (Self-Supervised Learning for Earth Observation - Sentinel-1/2) to assemble a large-scale, global, multimodal, and multi-seasonal corpus of satellite imagery from the ESA Sentinel-1 \& -2 satellite missions. For EO applications we demonstrate SSL4EO-S12 to succeed in self-supervised pre-training for a set of methods: MoCo-v2, DINO, MAE, and data2vec. Resulting models yield downstream performance close to, or surpassing accuracy measures of supervised learning. In addition, pre-training on SSL4EO-S12 excels compared to existing datasets. We make openly available the dataset, related source code, and pre-trained models at https://github.com/zhu-xlab/SSL4EO-S12.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| BigEarthNet | MoCo-v2 (ResNet50, fine tune) | mAP (micro) | 91.8 | — | Unverified |
| BigEarthNet | MoCo-v3 (ViT-S/16, fine tune) | mAP (micro) | 89.9 | — | Unverified |
| BigEarthNet | MAE (ViT-S/16, fine tune) | mAP (micro) | 88.9 | — | Unverified |
| BigEarthNet (official test set) | MoCov3 (ViT-S/16) | mAP (micro) | 89.3 | — | Unverified |
| BigEarthNet (official test set) | MoCov2 (ResNet50) | mAP (micro) | 88.7 | — | Unverified |