Mosaic convolution-attention network for demosaicing multispectral filter array images
 
Mosaic convolution-attention network for demosaicing multispectral filter array images ■
 
Kai Feng, Kai Feng, Kai Feng, Yongqiang Zhao, Yongqiang Zhao, Yongqiang Zhao, Jonathan C-W Chan, Jonathan C-W Chan, Jonathan C-W Chan, Seong Kong, Seong Kong, Seong Kong, Xun Zhang, Xun Zhang, Xun Zhang, Binglu Wang, Binglu Wang, Binglu Wang
 
Abstract ■

This paper presents a mosaic convolution-attention network (MCAN) for demosaicing spectral mosaic images captured using multispectral filter array (MSFA) imaging sensors. MSFA-based multispectral imaging systems acquire multispectral information of a scene in a single snap-shot operation. A complete multispectral image is reconstructed by demosaicing an MSFA-based spectral mosaic image. To avoid aliasing and artifacts in demosaicing, we utilize joint spatial-spectral correlation in a raw mosaic image. The proposed MCAN includes a mosaic convolution module (MCM) and a mosaic attention module (MAM). The MCM extracts features via a learning approach with a margin between splitting the periodic spectral mosaic and keeping the underlying spatial information of the raw image. Based on the strategy of position-sensitive weight sharing, MCM assigns the same weight to pixels with the same relative position in an MSFA. The MAM uses a position-sensitive feature aggregation strategy to describe the loading of mosaic patterns within the feature maps, which gradually reduces mosaic distortion through the attention mechanism. The experimental results on synthetic as well as real-world data show that the proposed scheme outperforms state-of-the-art methods in terms of spatial details and spectral fidelity.