SOTAVerified

Impact Assessment of Missing Data in Model Predictions for Earth Observation Applications

2024-03-21Code Available0· sign in to hype

Francisco Mena, Diego Arenas, Marcela Charfuelan, Marlon Nuske, Andreas Dengel

Code Available — Be the first to reproduce this paper.

Reproduce

Code

Abstract

Earth observation (EO) applications involving complex and heterogeneous data sources are commonly approached with machine learning models. However, there is a common assumption that data sources will be persistently available. Different situations could affect the availability of EO sources, like noise, clouds, or satellite mission failures. In this work, we assess the impact of missing temporal and static EO sources in trained models across four datasets with classification and regression tasks. We compare the predictive quality of different methods and find that some are naturally more robust to missing data. The Ensemble strategy, in particular, achieves a prediction robustness up to 100%. We evidence that missing scenarios are significantly more challenging in regression than classification tasks. Finally, we find that the optical view is the most critical view when it is missing individually.

Tasks

Benchmark Results

DatasetModelMetricClaimedVerifiedStatus
CropHarvest - GlobalFeature Gated FusionAverage Accuracy0.85Unverified
CropHarvest - GlobalInput FusionAverage Accuracy0.85Unverified
CropHarvest - GlobalEnsemble strategyAverage Accuracy0.83Unverified
CropHarvest multicrop - GlobalInput FusionAverage Accuracy0.74Unverified
CropHarvest multicrop - GlobalFeature Gated FusionAverage Accuracy0.73Unverified
CropHarvest multicrop - GlobalEnsemble strategyAverage Accuracy0.72Unverified

Reproductions