摘要
Empathy ability is one of the most important social communication skills in early childhood development. To analyze the children's empathy ability, facial expression analysis (FEA) is an effective way due to its ability to understand children's emotional states. Previous works mainly focus on recognizing the facial expression categories yet fail to estimate expression intensity, the latter of which is more important for fine-grained emotion analysis. To this end, this article first proposes to analyze children's empathy ability with both the categories and the intensities of facial expressions. A novel FEA method based on intensity label distribution learning is presented, which aims to recognize expression categories and estimate their intensity levels in an end-to-end framework. First, the intensity label distribution is generated for each frame in the expression sequence using a linear interpolation estimation and a Gaussian function to address the lack of reasonable annotations for expression intensity. Then, the extended intensity label distribution is presented to automatically encode the expression intensity in a multidimensional expression space, which aims to integrate the expression recognition and intensity estimation into a unified framework as well as boost the expression recognition performance by suppressing the variations in appearance caused by intensity and by emphasizing those variations among weak expressions. Finally, a Siamese-like convolutional neural network is presented to learn the expression model from a pair of frames that includes an expressive frame and its corresponding neutral frame using the extended intensity label distribution as the supervised information, thus effectively eliminating the expression-unrelated information's influence on FEA. Numerous experiments validate that the proposed method is promising in analysis of the differences in empathy ability between typically developing children and children with autism spectrum disorder.
学科与关键词
- Ordinal information based facial expression intensity estimation for emotional interaction: a novel semi-supervised deep learning approachXu, Ruyi; Han, Jiaxu; Chen, Jingying*Computing (Vienna/New York), 2022.被引 1
- Design and application of facial expression analysis system in empathy ability of children with autism spectrum disorderGuo, Chen; Zhang, Kun*; Chen, Jingying; Xu, Ruyi; Gao, Lei16th Conference on Computer Science and Intelligence Systems (FedCSIS), ELECTR NETWORK, 2021-09-02 To 2021-09-05.被引 2
- Facial expression intensity estimation using label-distribution-learning-enhanced ordinal regressionXu, Ruyi; Wang, Zhun; Chen, Jingying*; Zhou, LongpuMultimedia Systems, 2024, 30(1): 13.
