Estimating human posture is a key element of behavior analysis and human activity recognition (HAR) in many applications, such as public surveillance and gaming. Existing contactless human pose estimation (HPE) methods are mostly vision-based, which may violate privacy and lose functionality in harsh weather and poor light conditions. On the other hand, while being robust against these limitations, mm-wave radars provide high-resolution range data but suffer from no/poor angular resolution. In this article, we employ mm-wave radar along with the inverse synthetic aperture radar (ISAR) algorithm to obtain a high-resolution radar image of a moving person in both range and cross-range dimensions and use the binarized ISAR image as input to an HPE model. The HPE model is trained using labels generated by a vision-based HPE model (AlphaPose). We show that the proposed pipeline can estimate pose from afar (e.g., 4–12 m) using real-world data. We present the pipeline in a general case of a multiple-input-multiple-output (MIMO) radar; however, it can work using a single-input-single-output (SISO) radar as well, providing an extremely affordable solution for behavior analysis applications.
Javadi, SH, Bourdoux, A, Deligiannis, N & Sahli, H 2024, 'Human Pose Estimation Based on ISAR and Deep Learning', IEEE Sensors Journal, vol. 24, no. 17, pp. 28324-28337. https://doi.org/10.1109/JSEN.2024.3426030
Javadi, S. H., Bourdoux, A., Deligiannis, N., & Sahli, H. (2024). Human Pose Estimation Based on ISAR and Deep Learning. IEEE Sensors Journal, 24(17), 28324-28337. https://doi.org/10.1109/JSEN.2024.3426030
@article{6df12b4616024d0dae7427b7075a3784,
title = "Human Pose Estimation Based on ISAR and Deep Learning",
abstract = "Estimating human posture is a key element of behavior analysis and human activity recognition (HAR) in many applications, such as public surveillance and gaming. Existing contactless human pose estimation (HPE) methods are mostly vision-based, which may violate privacy and lose functionality in harsh weather and poor light conditions. On the other hand, while being robust against these limitations, mm-wave radars provide high-resolution range data but suffer from no/poor angular resolution. In this article, we employ mm-wave radar along with the inverse synthetic aperture radar (ISAR) algorithm to obtain a high-resolution radar image of a moving person in both range and cross-range dimensions and use the binarized ISAR image as input to an HPE model. The HPE model is trained using labels generated by a vision-based HPE model (AlphaPose). We show that the proposed pipeline can estimate pose from afar (e.g., 4–12 m) using real-world data. We present the pipeline in a general case of a multiple-input-multiple-output (MIMO) radar; however, it can work using a single-input-single-output (SISO) radar as well, providing an extremely affordable solution for behavior analysis applications.",
keywords = "Human Pose Estimation, Inverse Synthetic Aperture Radar, Convolutional Neural Network",
author = "Javadi, \{S. Hamed\} and Andr{\'e} Bourdoux and Nikos Deligiannis and Hichem Sahli",
note = "Funding Information: Manuscript received XYZ. The research leading to these results has received funding from IMEC.ICON and Flanders Innovation \& Entrepreneurship (nr HBC.2020.3106) \textbackslash{}u2013 Project Surv-AI-llance. Funding Information: The research leading to these results has received funding from IMEC.ICON and Flanders Innovation \& Entrepreneurship (nr HBC.2020.3106) \textbackslash{}u2013 Project Surv-AI-llance. Publisher Copyright: {\textcopyright} 2001-2012 IEEE.",
year = "2024",
month = jul,
day = "16",
doi = "10.1109/JSEN.2024.3426030",
language = "English",
volume = "24",
pages = "28324--28337",
journal = "IEEE Sensors Journal",
issn = "1530-437X",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "17",
}