Pixel reflectance estimation with deep learning pansharpening methods

Pansharpening consists of fusing a multispectral (MS) image and a panchromatic (PAN) image to generate a high quality MS image. Several pansharpening techniques have been developed to enhance the spatial quality of MS data. The estimation of objects in the scene, such as cars, trees or buildings, require accurate pixels synthesis during fusion process. In this paper, we proposed to estimate the pixel reflectance of fused multispectral images using a generalized UIQI band-wise metric. This criteria is validated on pansharpened results at reduced-scale using Wald protocol. In this context, we presented two comparative studies. In the first case, we compared statistically the proposed criteria to the pixel correlation and the Euclidian distance. The proposed criteria presented promising results quantitatively. Concerning the second case study, we considered the assessment of deep learning-based fusion methods versus the state-of-the-art. Indeed, the pansharpening based on Neural Networks (PNN), Convolutional Neural Networks (CNN) and the adaptive PNN with fine tuning have been trained and tested. The state-of-the art pansharpening methods include the Generalized Laplacian Pyramids (GLP), Additive Wavelet Luminance Proportion (AWLP), Gram-Schmidt adaptive (GSA), Total variation (TV), Model-based fusion using principle component and wavelets (PWMBF) and Filter estimation based on a semi-blind deconvolution framework (FE). The experimental results have been performed on two remote sensing data sets captured by GeoEye-1 and Worldview-3 satellites. The comparative study allows a better understanding of the displacement of objects or the misregistration of PAN and MS images.