This paper presents a feature descriptor using Histogram of Oriented Mosaic Gradient (HOMG) that extracts spatial-spectral features directly from mosaic spectral images. Spectral imaging utilizes unique spectral signatures to distinguish objects of interest in the scene more discriminatively. Snapshot spectral cameras equipped with spectral filter arrays (SFAs) capture spectral videos in real time, making it possible to detect/track fast moving targets based on spectral imaging. How to effectively extract the spatial-spectral feature directly from the mosaic spectral images acquired by snapshot spectral cameras is a core issue for detection/tracking. So far, there is a lack of comprehensive and in-depth research on this issue. To this end, this paper proposed a new spatial-spectral feature extractor for mosaic spectral images. The proposed scheme finds two forms of SFA neighborhood (SFAN) to construct a feature extractor suitable for any SFA structure. Exploiting the spatial-spectral correlation in two SFANs, we design six mosaic spatial-spectral gradient operators to compute spatial-spectral gradient maps (SGMs). HOMG descriptors are constructed using the magnitude and orientation of SGMs. The effectiveness and generalizability of the proposed method have been verified with object tracking experiments. Compared to the state-of-the-art feature descriptors, HOMG ranked first on two datasets captured with snapshot spectral camera with different SFAs, achieving a gain of 3.9% and 5.9% in average success rate over the second-ranked feature.
Chen, L, Zhao, Y, Chan, JC-W & Kong, S 2022, 'Histograms of oriented mosaic gradients for snapshot spectral image description', ISPRS Journal of Photogrammetry and Remote Sensing, vol. 183, pp. 79-93. https://doi.org/10.1016/j.isprsjprs.2021.10.018
Chen, L., Zhao, Y., Chan, J. C.-W., & Kong, S. (2022). Histograms of oriented mosaic gradients for snapshot spectral image description. ISPRS Journal of Photogrammetry and Remote Sensing, 183, 79-93. https://doi.org/10.1016/j.isprsjprs.2021.10.018
@article{a7fe7d618f764c27821f1f434ec698bf,
title = "Histograms of oriented mosaic gradients for snapshot spectral image description",
abstract = "This paper presents a feature descriptor using Histogram of Oriented Mosaic Gradient (HOMG) that extracts spatial-spectral features directly from mosaic spectral images. Spectral imaging utilizes unique spectral signatures to distinguish objects of interest in the scene more discriminatively. Snapshot spectral cameras equipped with spectral filter arrays (SFAs) capture spectral videos in real time, making it possible to detect/track fast moving targets based on spectral imaging. How to effectively extract the spatial-spectral feature directly from the mosaic spectral images acquired by snapshot spectral cameras is a core issue for detection/tracking. So far, there is a lack of comprehensive and in-depth research on this issue. To this end, this paper proposed a new spatial-spectral feature extractor for mosaic spectral images. The proposed scheme finds two forms of SFA neighborhood (SFAN) to construct a feature extractor suitable for any SFA structure. Exploiting the spatial-spectral correlation in two SFANs, we design six mosaic spatial-spectral gradient operators to compute spatial-spectral gradient maps (SGMs). HOMG descriptors are constructed using the magnitude and orientation of SGMs. The effectiveness and generalizability of the proposed method have been verified with object tracking experiments. Compared to the state-of-the-art feature descriptors, HOMG ranked first on two datasets captured with snapshot spectral camera with different SFAs, achieving a gain of 3.9% and 5.9% in average success rate over the second-ranked feature.",
author = "Lulu Chen and Yongqiang Zhao and Chan, {Jonathan Cheung-Wai} and Seong Kong",
note = "Funding Information: This work was supported by the National Natural Science Foundation of China (NSFC) (No. 61771391 ), the Key R & D plan of Shaanxi Province (No. 2020ZDLGY07-11 ), the Science, Technology and Innovation Commission of Shenzhen Municipality (No. JCYJ20170815162956949 , JCYJ20180306171146740 ), the Natural Science Basic Research Plan in Shaanxi Province of China (No. 2018JM6056 ), and National Research Foundation of Korea (No. NRF-2016R1D1A1B01008522 ). Publisher Copyright: {\textcopyright} 2021 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS)",
year = "2022",
month = jan,
doi = "10.1016/j.isprsjprs.2021.10.018",
language = "English",
volume = "183",
pages = "79--93",
journal = "ISPRS Journal of Photogrammetry and Remote Sensing",
issn = "0924-2716",
publisher = "Elsevier",
}