Joint image super-resolution (SR) refers to the reconstruction of a high-resolution image from its low-resolution version with the aid of a high-resolution image from another modality. Inspired by the recent success of recurrent neural networks in single image SR, we propose a novel multimodal recurrent convolutional neural network with coupled sparse priors for joint image SR. Our network fuses representations of the two image modalities at input layers using a learned multimodal convolutional sparse coding network. Additional recurrent convolutional stages are performed to further learn the mapping between the input modalities and the desired high-resolution estimate. We apply the proposed network to the tasks of near-infrared image SR and multi-spectral image SR using RGB images as the guidance modality. Experimental results show the superior performance of the proposed multimodal recurrent convolutional network against several state-of-the-art single-modal and multimodal image SR methods.
Marivani, I, Tsiligianni, E, Cornelis, B & Deligiannis, N 2020, Joint Image Super-Resolution Via Recurrent Convolutional Neural Networks With Coupled Sparse Priors. in IEEE International Conference on Image Processing (ICIP)., 9190644, Proceedings - International Conference on Image Processing, ICIP, vol. 2020-October, IEEE, pp. 868-872, IEEE International Conference on Image Processing, Abu Dhabi, United Arab Emirates, 25/10/20. https://doi.org/10.1109/ICIP40778.2020.9190644
Marivani, I., Tsiligianni, E., Cornelis, B., & Deligiannis, N. (2020). Joint Image Super-Resolution Via Recurrent Convolutional Neural Networks With Coupled Sparse Priors. In IEEE International Conference on Image Processing (ICIP) (pp. 868-872). Article 9190644 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2020-October). IEEE. https://doi.org/10.1109/ICIP40778.2020.9190644
@inproceedings{61535b7a60364849b4230c8c06931e58,
title = "Joint Image Super-Resolution Via Recurrent Convolutional Neural Networks With Coupled Sparse Priors",
abstract = "Joint image super-resolution (SR) refers to the reconstruction of a high-resolution image from its low-resolution version with the aid of a high-resolution image from another modality. Inspired by the recent success of recurrent neural networks in single image SR, we propose a novel multimodal recurrent convolutional neural network with coupled sparse priors for joint image SR. Our network fuses representations of the two image modalities at input layers using a learned multimodal convolutional sparse coding network. Additional recurrent convolutional stages are performed to further learn the mapping between the input modalities and the desired high-resolution estimate. We apply the proposed network to the tasks of near-infrared image SR and multi-spectral image SR using RGB images as the guidance modality. Experimental results show the superior performance of the proposed multimodal recurrent convolutional network against several state-of-the-art single-modal and multimodal image SR methods.",
author = "Iman Marivani and Evangelia Tsiligianni and Bruno Cornelis and Nikolaos Deligiannis",
year = "2020",
month = oct,
doi = "10.1109/ICIP40778.2020.9190644",
language = "English",
series = "Proceedings - International Conference on Image Processing, ICIP",
publisher = "IEEE",
pages = "868--872",
booktitle = "IEEE International Conference on Image Processing (ICIP)",
note = "IEEE International Conference on Image Processing, ICIP ; Conference date: 25-10-2020 Through 28-10-2020",
url = "https://2020.ieeeicip.org/",
}