In linear inverse problems, the goal is to recover a target signal from undersampled, incomplete or noisy linear measurements. Typically, the recovery relies on complex numerical optimization methods; recent approaches perform an unfolding of a numerical algorithm into a neural network form, resulting in a substantial reduction of the computational complexity. In this letter, we consider the recovery of a target signal with the aid of a correlated signal, the so-called side information (SI), and propose a deep unfolding model that incorporates SI. The proposed model is used to learn coupled representations of correlated signals from different modalities, enabling the recovery of multi-modal data at a low computational cost. As such, our work introduces the first deep unfolding method with SI, which actually comes from a different modality. We apply our model to reconstruct near-infrared images from undersampled measurements given RGB images as SI. Experimental results demonstrate the superior performance of the proposed framework against single-modal deep learning methods that do not use SI, multi-modal deep learning designs, and optimization algorithms.
Tsiligianni, E & Deligiannis, N 2019, 'Deep coupled-representation learning for sparse linear inverse problems with side information', IEEE Signal Processing Letters, vol. 26, no. 12, 8844082, pp. 1768-1772. https://doi.org/10.1109/LSP.2019.2929869
Tsiligianni, E., & Deligiannis, N. (2019). Deep coupled-representation learning for sparse linear inverse problems with side information. IEEE Signal Processing Letters, 26(12), 1768-1772. Article 8844082. https://doi.org/10.1109/LSP.2019.2929869
@article{3fa467e03a71428a9bde3f4adeef5451,
title = "Deep coupled-representation learning for sparse linear inverse problems with side information",
abstract = "In linear inverse problems, the goal is to recover a target signal from undersampled, incomplete or noisy linear measurements. Typically, the recovery relies on complex numerical optimization methods; recent approaches perform an unfolding of a numerical algorithm into a neural network form, resulting in a substantial reduction of the computational complexity. In this letter, we consider the recovery of a target signal with the aid of a correlated signal, the so-called side information (SI), and propose a deep unfolding model that incorporates SI. The proposed model is used to learn coupled representations of correlated signals from different modalities, enabling the recovery of multi-modal data at a low computational cost. As such, our work introduces the first deep unfolding method with SI, which actually comes from a different modality. We apply our model to reconstruct near-infrared images from undersampled measurements given RGB images as SI. Experimental results demonstrate the superior performance of the proposed framework against single-modal deep learning methods that do not use SI, multi-modal deep learning designs, and optimization algorithms.",
keywords = "deep learning, Inverse Problem, Explainable AI",
author = "Evangelia Tsiligianni and Nikolaos Deligiannis",
year = "2019",
month = dec,
doi = "10.1109/LSP.2019.2929869",
language = "English",
volume = "26",
pages = "1768--1772",
journal = "IEEE Signal Processing Letters",
issn = "1070-9908",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "12",
}