Continuous affect recognition from facial images aims to estimate the values of multiple affective dimensions from a facial image sequence. To leverage relevant information between multiple affective dimensions, multitask learning has been used in the estimation of continuous affective states. Most of the existing multitask continuous affect recognition methods focus on designing elaborate multitask networks. Meanwhile, a few research works consider using multitask training strategies for continuous affect recognition. In general, existing multitask continuous affect recognition methods face the problem of unstable training effects. In this work, to improve the stability of multitask learning, we propose an ensemble learning-enhanced multitask network architecture for continuous affect recognition. In addition, we introduce a novel adaptive weighted loss-based multitask learning strategy to effectively train the proposed multitask continuous affect recognition model. Experimental results, on the RECOLA, SEMAINE and AFEW-VA datasets for continuous affect recognition, demonstrate the potential of the proposed method compared to state-of-the-art methods.
Pei, E, Hu, Z, LANG, HE, Ning, H & Berenguer, AD 2024, 'An ensemble learning-enhanced multitask learning method for continuous affect recognition from facial images', Expert Systems with Applications, vol. 236, no. February 2024, 121290, pp. 1-12. https://doi.org/10.1016/j.eswa.2023.121290
Pei, E., Hu, Z., LANG, HE., Ning, H., & Berenguer, A. D. (2024). An ensemble learning-enhanced multitask learning method for continuous affect recognition from facial images. Expert Systems with Applications, 236(February 2024), 1-12. Article 121290. https://doi.org/10.1016/j.eswa.2023.121290
@article{1d4e566eaa0d4ebead82754007510e40,
title = "An ensemble learning-enhanced multitask learning method for continuous affect recognition from facial images",
abstract = "Continuous affect recognition from facial images aims to estimate the values of multiple affective dimensions from a facial image sequence. To leverage relevant information between multiple affective dimensions, multitask learning has been used in the estimation of continuous affective states. Most of the existing multitask continuous affect recognition methods focus on designing elaborate multitask networks. Meanwhile, a few research works consider using multitask training strategies for continuous affect recognition. In general, existing multitask continuous affect recognition methods face the problem of unstable training effects. In this work, to improve the stability of multitask learning, we propose an ensemble learning-enhanced multitask network architecture for continuous affect recognition. In addition, we introduce a novel adaptive weighted loss-based multitask learning strategy to effectively train the proposed multitask continuous affect recognition model. Experimental results, on the RECOLA, SEMAINE and AFEW-VA datasets for continuous affect recognition, demonstrate the potential of the proposed method compared to state-of-the-art methods.",
author = "Ercheng Pei and Zhanxuan Hu and HE LANG and Hailong Ning and Berenguer, {Abel D{\'i}az}",
note = "Funding Information: This work is supported by the Natural Science Basic Research Program of Shaanxi Province (Program No. 2022JQ-662 ), the National Natural Science Foundation of China (Program No. 62206219 ), the Special Construction Fund for Key Disciplines of Shaanxi Provincial Higher Education , and the National Natural Science Foundation of China (Program No. 62376215 ). Publisher Copyright: {\textcopyright} 2023 Elsevier Ltd",
year = "2024",
month = feb,
doi = "10.1016/j.eswa.2023.121290",
language = "English",
volume = "236",
pages = "1--12",
journal = "Expert Systems with Applications",
issn = "0957-4174",
publisher = "Elsevier Limited",
number = "February 2024",
}