ArbRPN: A Bidirectional Recurrent Pansharpening Network for Multispectral Images With Arbitrary Numbers of Bands - Apprentissage de modèles visuels à partir de données massives Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Geoscience and Remote Sensing Année : 2022

ArbRPN: A Bidirectional Recurrent Pansharpening Network for Multispectral Images With Arbitrary Numbers of Bands

Résumé

Although the performance of pansharpening has been significantly improved by advanced deep-learning (DL) technologies in recent years, most DL-based methods fail to process multispectral (MS) images with arbitrary numbers of bands by a single model. Consequently, it is inevitable to train separate models for MS images with different numbers of bands, which is time- and storage-consuming as well as inefficient in practice. To tackle the above problem, we propose a bidirectional recurrent pansharpening network (named ArbRPN) for MS images with arbitrary numbers of bands. Our ArbRPN can dynamically reconstruct high-resolution (HR) MS images with different numbers of bands by adaptively changing the number of recurrence to the number of bands of the low-resolution (LR) MS images. Leveraging on the ability of the ArbRPN to process MS images with any number of bands, one can even customize the bands to be pansharpened. Moreover, to achieve superior performance, spectral discrepancy and dependence are considered in the ArbRPN. Details from the panchromatic (PAN) image are adaptively injected into the fused product according to the captured spectral dependence. Furthermore, training strategies of existing DL-based pansharpening methods can only group MS images with a constant number of bands into mini-batches. Therefore, we present a mask-based training method (called mask-training) to solve this problem. Benefiting from the mask-training, our ArbRPN can achieve superior performance and robustness during pansharpening. Extensive experiments show the superior performance of our ArbRPN with respect to the state-of-the-art (SOTA) methods applied to MS images with different numbers of bands. The code of our ArbRPN is available on https://github.com/Lihui-Chen/ArbRPN.git.
Fichier non déposé

Dates et versions

hal-03932775 , version 1 (10-01-2023)

Identifiants

Citer

Lihui Chen, Zhibing Lai, Gemine Vivone, Gwanggil Jeon, Jocelyn Chanussot, et al.. ArbRPN: A Bidirectional Recurrent Pansharpening Network for Multispectral Images With Arbitrary Numbers of Bands. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60, pp.5406418. ⟨10.1109/TGRS.2021.3131228⟩. ⟨hal-03932775⟩
29 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More