A novel deep-learning-based method for semantic segmentation of RGB and Thermal images is introduced. The proposed method employs a novel neural network design for multi-modal fusion based on multi-resolution patch processing. A novel decoder module is introduced to fuse the RGB and Thermal features extracted by separate encoder streams. Experimental results on synthetic and real-world data demonstrate the efficiency of the proposed method compared with state-of-the-art methods.
Lyu, Y, Schiopu, I & Munteanu, A 2020, 'Multi-modal neural networks with multi-scale RGB-T fusion for semantic segmentation', Electronics Letters, vol. 56, no. 18, el.2020.1635, pp. 920-922. https://doi.org/10.1049/el.2020.1635
Lyu, Y., Schiopu, I., & Munteanu, A. (2020). Multi-modal neural networks with multi-scale RGB-T fusion for semantic segmentation. Electronics Letters, 56(18), 920-922. Article el.2020.1635. https://doi.org/10.1049/el.2020.1635
@article{323ffc4490564cd68e34706920219426,
title = "Multi-modal neural networks with multi-scale RGB-T fusion for semantic segmentation",
abstract = "A novel deep-learning-based method for semantic segmentation of RGB and Thermal images is introduced. The proposed method employs a novel neural network design for multi-modal fusion based on multi-resolution patch processing. A novel decoder module is introduced to fuse the RGB and Thermal features extracted by separate encoder streams. Experimental results on synthetic and real-world data demonstrate the efficiency of the proposed method compared with state-of-the-art methods.",
keywords = "Semantic segmentation, Deep-learning, multi-modal data fusion",
author = "Yangxintong Lyu and Ionut Schiopu and Adrian Munteanu",
year = "2020",
month = sep,
day = "3",
doi = "10.1049/el.2020.1635",
language = "English",
volume = "56",
pages = "920--922",
journal = "Electronics Letters",
issn = "0013-5194",
publisher = "Institution of Engineering and Technology",
number = "18",
}