ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Ordinal information based facial expression intensity estimation for emotional interaction: a novel semi-supervised deep learning approach

Xu, Ruyi; Han, Jiaxu; Chen, Jingying*cent china normal univ
Computing (Vienna/New York)2022-12-28DOI: 10.1007/s00607-022-01140-y
1被引2阅读
引用收藏
分享

摘要

Emotional understanding and expression plays a critical role in social interaction. To analyze children's emotional interaction automatically, this study focuses on developing a novel network architecture and a reliable algorithm for expression intensity estimation to measure children's facial expression responses to emotional stimuli. The facial expression intensity variation provides temporal dynamic information of facial behavior, which is critical to interpreting the meaning of expression. In order to avoid laborious manual annotations for expression intensity, existing unsupervised methods attempt to identify relative intensity using ordinal information within a facial expression sequence; however, they fail to estimate absolute intensity accurately. Moreover, appropriate features are needed to represent the continuous appearance changes caused by expression intensity to improve the model's ability to distinguish subtle differences in expression. This study therefore presents a novel semi-supervised method to estimate expression intensity using salient deep learning features. First, the facial expression is represented by the difference response of the convolutional neural network backbone between the target expression and its responding neutral expression, with the goal of suppressing the effects of expression-unrelated features on expression intensity estimation. Then, the pairwise data constructed with ordinal information is input into a Siamese network with a combined hinge loss that guides learning the relative intensity on unlabeled pairwise frames, the absolute intensity of a few labeled key frames, and the intensity range of most unlabeled frames. The average pearson correlation coefficient, intraclass correlation coefficient, and mean absolute error are 0.7683, 0.7405, and 0.1698 on the extended Cohn-Kanade dataset (CK+), and 0.7804, 0.6684, and 0.1864 on the Binghamton University 4D Facial Expression Dataset using the proposed method, results that are superior to the state of the art. The cross-dataset experiment indicates that the proposed method is promising for the analysis of children's emotional interactions.

学科与关键词

关键词
Expression intensity estimationSiamese networkSemi-supervised learningOrdinal regressionSocial interaction analysis
相关
加载中…

文献加载失败,请重试

重试

发表信息

论文状态
公开发表
期刊
Computing (Vienna/New York)
发表日期
2022-12-28
DOI
10.1007/s00607-022-01140-y
语言
ENGLISH
期刊等级
4区
成果收录
SCIE
资助标注
National Natural Science Foundation of China [61977027]; Science & Technology Major Project of Hubei Province Next-Generation AI Technologies [2021BEA159]; Research Funds of CCNU from the Colleges' Basic Research and Operation of MOE [30106220491]

作者

  • X

    Xu, Ruyi (第一作者)

    cent china normal univ

  • H

    Han, Jiaxu

    cent china normal univ

查看全部 3 位作者收起作者
  • C

    Chen, Jingying (通讯)

    cent china normal univ

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417