Accurate 3-D models of human subjects are widely used in domains such as fashion design, non-contact body biometrics, computer animation, gaming, AR/VR, to cite a few. For these kinds of applications, a high-fidelity human body mesh in a canonical posture (e.g. pose or pose) is necessary. This paper proposes a deep learning approach to jointly reconstruct a clean, watertight body mesh and to normalize the posture of the human body model starting from an input set of impaired body point clouds. The proposed method, dubbed Impaired-to-High-fidelity human body network (I2H) is, to the best of our knowledge, the first deep learning approach in the literature that addresses these problems. The proposed method follows an Encoder-Decoder design. The Encoder directly takes the impaired point clouds (e.g. containing noise, occlusions and misalignments) as input without making any structural assumptions about the input. The Decoder interprets the latent feature and produces a high-fidelity T-pose body mesh. We compare the proposed approach against existing state-of-the-art methods through various experiments and show that our method achieves the best performance on both synthetic and scanned datasets for 3D human mesh reconstruction.
Hu, P, Zhao, R, Dai, X & Munteanu, A 2022, 'Predicting high-fidelity human body models from impaired point clouds', Signal Processing, vol. 192, 108375. https://doi.org/10.1016/j.sigpro.2021.108375
Hu, P., Zhao, R., Dai, X., & Munteanu, A. (2022). Predicting high-fidelity human body models from impaired point clouds. Signal Processing, 192, Article 108375. https://doi.org/10.1016/j.sigpro.2021.108375
@article{057adb26ff7045b79d7aeb199b630436,
title = "Predicting high-fidelity human body models from impaired point clouds",
abstract = "Accurate 3-D models of human subjects are widely used in domains such as fashion design, non-contact body biometrics, computer animation, gaming, AR/VR, to cite a few. For these kinds of applications, a high-fidelity human body mesh in a canonical posture (e.g. pose or pose) is necessary. This paper proposes a deep learning approach to jointly reconstruct a clean, watertight body mesh and to normalize the posture of the human body model starting from an input set of impaired body point clouds. The proposed method, dubbed Impaired-to-High-fidelity human body network (I2H) is, to the best of our knowledge, the first deep learning approach in the literature that addresses these problems. The proposed method follows an Encoder-Decoder design. The Encoder directly takes the impaired point clouds (e.g. containing noise, occlusions and misalignments) as input without making any structural assumptions about the input. The Decoder interprets the latent feature and produces a high-fidelity T-pose body mesh. We compare the proposed approach against existing state-of-the-art methods through various experiments and show that our method achieves the best performance on both synthetic and scanned datasets for 3D human mesh reconstruction.",
keywords = "Human body reconstruction, Posture normalization, 3D Scanning, Impaired point clouds, Deep learning on point clouds",
author = "Pengpeng Hu and Ran Zhao and Xinxin Dai and Adrian Munteanu",
note = "Funding Information: This work was supported in part by the Innoviris under Project AI43D in close collaboration with Spentys and in part by FWO under Project G084117. Publisher Copyright: {\textcopyright} 2021 Elsevier B.V. Copyright: Copyright 2022 Elsevier B.V., All rights reserved.",
year = "2022",
month = mar,
doi = "10.1016/j.sigpro.2021.108375",
language = "English",
volume = "192",
journal = "Signal Processing",
issn = "0165-1684",
publisher = "Elsevier",
}