A Self-Supervised Deep Denoiser for Hyperspectral and Multispectral Image Fusion

Code

• github.com/FanHSI/E2E-fusion
  Officialnone★ 4

Abstract

The plug-and-play (PnP) technique enables us to plug image priors into an alternating direction method of multipliers (ADMM) framework for solving a regularized optimization problem. Deep image priors have shown their flexibility and robustness in solving several image inverse problems. Hyperspectral image (HSI) super-resolution problem is an ill-posed inverse problem that aims to obtain a high-resolution HSI (HR-HSI) by combining the information of low-resolution HSI (LR-HSI) and HR multispectral image (HR-MSI) simultaneously. This article proposes an HSI and MSI fusion framework termed E2E-Fusion, plugged with a self-supervised deep learning prior called Eigenimage2Eigenimage. First, the spectral low-rank structure of HSIs is exploited via subspace representations of spectra vectors. Meanwhile, benefiting from the high quality of the first eigenimage (i.e., representation coefficients), we design a self-supervised deep eigenimage guidance network image prior, E2E. By using the PnP technique, we plugged the E2E prior into the ADMM fusion framework to update the optimal objective function iteratively. The numerical experimental results both on the simulated datasets and real datasets demonstrate that the proposed method performs better than state-of-the-art fusion methods.

Tasks

Reproductions