ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Understanding More About Human and Machine Attention in Deep Neural Networks

Lai, Qiuxia; Khan, Salman; Nie, Yongwei*; Sun, Hanqiu; Shen, Jianbing; Shao, Ling电子科技大学
IEEE TRANSACTIONS ON MULTIMEDIA2021DOI: 10.1109/TMM.2020.3007321
22被引4阅读
引用收藏
分享

摘要

Human visual system can selectively attend to parts of a scene for quick perception, a biological mechanism known as Human attention. Inspired by this, recent deep learning models encode attention mechanisms to focus on the most task-relevant parts of the input signal for further processing, which is called Machine/Neural/Artificial attention. Understanding the relation between human and machine attention is important for interpreting and designing neural networks. Many works claim that the attention mechanism offers an extra dimension of interpretability by explaining where the neural networks look. However, recent studies demonstrate that artificial attention maps do not always coincide with common intuition. In view of these conflicting evidence, here we make a systematic study on using artificial attention and human attention in neural network design. With three example computer vision tasks (i.e., salient object segmentation, video action recognition, and fine-grained image classification), diverse representative backbones (i.e., AlexNet, VGGNet, ResNet) and famous architectures (i.e., Two-stream, FCN), corresponding real human gaze data, and systematically conducted large-scale quantitative studies, we quantify the consistency between artificial attention and human visual attention and offer novel insights into existing artificial attention mechanisms by giving preliminary answers to several key questions related to human and artificial attention mechanisms. Overall results demonstrate that human attention can benchmark the meaningful 'ground-truth' in attention-driven tasks, where the more the artificial attention is close to human attention, the better the performance; for higher-level vision tasks, it is case-by-case. It would be advisable for attention-driven tasks to explicitly force a better alignment between artificial and human attention to boost the performance; such alignment would also improve the network explainability for higher-level computer vision tasks.

学科与关键词

关键词
Task analysisVisualizationComputer visionNeural networksObject segmentationImage recognitionReliabilityAttention mechanismhuman attentionartificial attentiondeep learning
相关
加载中…
  • Towards the quantitative evaluation of visual attention models
    Bylinskii, Z.*; DeGennaro, E. M.; Rajalingham, R.; Ruda, H.; Zhang, J.; Tsotsos, J. K.Vision Research, 2015, 116: 258-268.被引 39
  • Emotional Attention: From Eye Tracking to Computational Modeling
    Fan, Shaojing; Shen, Zhiqi; Jiang, Ming; Koenig, Bryan L. L.; Kankanhalli, Mohan S. S.; Zhao, Qi*IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 1682-1699.被引 5
  • Object-based visual attention for computer vision
    Sun, YR; Fisher, RArtificial Intelligence, 2003, 146(1): 77-123.被引 201
查看全部 →

发表信息

论文状态
公开发表
期刊
IEEE TRANSACTIONS ON MULTIMEDIA
发表日期
2021
卷期
第 23 卷
页码
2086-2099
DOI
10.1109/TMM.2020.3007321
语言
ENGLISH
期刊收录
NSL
期刊等级
1区
成果收录
SCIE
资助标注
Natural Science Foundation of Guangdong Province, China [2019A1515010860]; Fundamental Research Funds for the Central Universities [D2190670]; Science and Technology Project of Guangzhou City [201707010140]

作者

  • L

    Lai, Qiuxia (第一作者)

    chinese univ hong kong

  • K

    Khan, Salman

查看全部 6 位作者收起作者
  • N

    Nie, Yongwei (通讯)

  • S

    Sun, Hanqiu

    univ elect sci & technol china

  • S

    Shen, Jianbing

  • S

    Shao, Ling

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417