Artificial intelligence (AI) has incorporated various automatic systems and frameworks to diagnose the severity of depression using hand-crafted features. However, process of feature selection needs domain knowledge and is still time-consuming and subjective. Deep learning technology has been successfully adopted for depression recognition. Most previous works pre-train the deep models on large databases followed by fine-tuning with depression databases (i.e., AVEC2013, AVEC2014). In the present paper we propose an integrated framework – Deep Local Global Attention Convolutional Neural Network (DLGA-CNN) for depression recognition, which adopts CNN with attention mechanism as well as weighted spatial pyramid pooling (WSPP) to learn a deep and global representation. Two branches are introduced: Local Attention based CNN (LA-CNN) focuses on the local patches, while Global Attention based CNN (GA-CNN) learns the global patterns from the entire facial region. To capture the complementary information between the two branches, Local–Global Attention-based CNN (LGA-CNN) is proposed. After feature aggregation, WSPP is used to learn the depression patterns. Comprehensive experiments on AVEC2013 and AVEC2014 depression databases have demonstrated that the proposed method is capable of mining the underlying depression patterns of facial videos and outperforms the most of the state-of-the-art video-based depression recognition approaches.
He, L, Chan, JC-W & Wang, Z 2021, 'Automatic depression recognition using CNN with attention mechanism from videos', Neurocomputing, vol. 422, pp. 165-175. https://doi.org/10.1016/j.neucom.2020.10.015
He, L., Chan, J. C.-W., & Wang, Z. (2021). Automatic depression recognition using CNN with attention mechanism from videos. Neurocomputing, 422, 165-175. https://doi.org/10.1016/j.neucom.2020.10.015
@article{0f7c540d6e414506b7778f6453750813,
title = "Automatic depression recognition using CNN with attention mechanism from videos",
abstract = "Artificial intelligence (AI) has incorporated various automatic systems and frameworks to diagnose the severity of depression using hand-crafted features. However, process of feature selection needs domain knowledge and is still time-consuming and subjective. Deep learning technology has been successfully adopted for depression recognition. Most previous works pre-train the deep models on large databases followed by fine-tuning with depression databases (i.e., AVEC2013, AVEC2014). In the present paper we propose an integrated framework – Deep Local Global Attention Convolutional Neural Network (DLGA-CNN) for depression recognition, which adopts CNN with attention mechanism as well as weighted spatial pyramid pooling (WSPP) to learn a deep and global representation. Two branches are introduced: Local Attention based CNN (LA-CNN) focuses on the local patches, while Global Attention based CNN (GA-CNN) learns the global patterns from the entire facial region. To capture the complementary information between the two branches, Local–Global Attention-based CNN (LGA-CNN) is proposed. After feature aggregation, WSPP is used to learn the depression patterns. Comprehensive experiments on AVEC2013 and AVEC2014 depression databases have demonstrated that the proposed method is capable of mining the underlying depression patterns of facial videos and outperforms the most of the state-of-the-art video-based depression recognition approaches.",
author = "Lang He and Chan, {Jonathan Cheung-Wai} and Zhongmin Wang",
note = "Funding Information: This work is supported by the Shaanxi Provincial Office of Education Emergency Research Fund for Public Health Security (grant 20JG030), the Shaanxi Higher Education Association Fund for the Prevention and Control of Novel Coronavirus Pneumonia (grant XGH20201), the Shaanxi Provincial Public Scientific Quality Promotion Fund for Emergency Popularization of COVID-19 (grant 2020PSL(Y)040). Publisher Copyright: {\textcopyright} 2020 Elsevier B.V.",
year = "2021",
month = jan,
day = "21",
doi = "10.1016/j.neucom.2020.10.015",
language = "English",
volume = "422",
pages = "165--175",
journal = "Neurocomputing",
issn = "0925-2312",
publisher = "Elsevier",
}