Towards Automatic Transformer-based Cloud Classification and Segmentation

Abstract

Clouds have been demonstrated to have a huge impact on the energy balance, temperature, and weather of the Earth. Classification and segmentation of clouds and coverage factors are crucial for climate modeling, meteorological studies, the solar energy industry, and satellite communication. For example, clouds have a tremendous impact on short-term predictions or 'nowcasts' of solar irradiance and can be used to optimize solar power plants and effectively exploit solar energy. However even today, cloud observation requires the intervention of highly-trained professionals to document their findings, which introduces bias. To overcome these issues and contribute to climate change technology, we propose the first two transformer-based models applied to cloud data tasks to the best of our knowledge. We use the CCSD Cloud classification dataset and achieve 90.06% accuracy, outperforming all other methods. To demonstrate the robustness of transformers in this domain, we perform Cloud segmentation on the SWIMSWG dataset and achieve 83.2% IoU, outperforming other methods. With this, we signal a potential shift away from pure CNN networks.

Tasks

Reproductions