ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Comprehension is a double-edged sword: Over-interpreting unspecified information in intelligible machine learning explanations

Xuan, Yueqing*; Small, Edward; Sokol, Kacper; Hettiachchi, Danula; Sanderson, Mark
International Journal of Human Computer Studies2025-1DOI: 10.1016/j.ijhcs.2024.103376
1阅读
引用收藏
分享

摘要

Automated decision-making systems are becoming increasingly ubiquitous, which creates an immediate need for their interpretability and explainability. However, it remains unclear whether users know what insights an explanation offers and, more importantly, what information it lacks. To answer this question we conducted an online study with 200 participants, which allowed us to assess explainees' ability to realise explicated information - i.e., factual insights conveyed by an explanation - and unspecified information - i.e, insights that are not communicated by an explanation - across four representative explanation types: model architecture, decision surface visualisation, counterfactual explainability and feature importance. Our findings uncover that highly comprehensible explanations, e.g., feature importance and decision surface visualisation, are exceptionally susceptible to misinterpretation since users tend to infer spurious information that is outside of the scope of these explanations. Additionally, while the users gauge their confidence accurately with respect to the information explicated by these explanations, they tend to be overconfident when misinterpreting the explanations. Our work demonstrates that human comprehension can be a double-edged sword since highly accessible explanations may convince users of their truthfulness while possibly leading to various misinterpretations at the same time. Machine learning explanations should therefore carefully navigate the complex relation between their full scope and limitations to maximise understanding and curb misinterpretation.

学科与关键词

关键词
Human-centred computingComprehensionEvaluationUser studyExplainabilityInterpretabilityMachine learningArtificial intelligence
相关
加载中…
  • Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
    Rong, Yao*; Leemann, Tobias; Nguyen, Thai-Trang; Fiedler, Lisa; Qian, Peizhu; Unhelkar, Vaibhav; Seidel, Tina; Kasneci, Gjergji; Kasneci, EnkelejdaIEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(4): 2104-2122.被引 27
  • Who needs explanation and when? Juggling explainable AI and user epistemic uncertainty
    Jiang, Jinglu; Kahai, Surinder; Yang, Ming*International Journal of Human Computer Studies, 2022, 165: 102839.被引 26
  • The non-linear nature of the cost of comprehensibility
    Goethals, Sofie*; Martens, David; Evgeniou, TheodorosJOURNAL OF BIG DATA, 2022, 9(1): 30.被引 7
查看全部 →

发表信息

论文状态
公开发表
期刊
International Journal of Human Computer Studies
发表日期
2025-01
卷期
第 193 卷
页码
103376
DOI
10.1016/j.ijhcs.2024.103376
语言
ENGLISH
成果收录
SCIE
资助标注
ARC Centre of Excellence for Automated Decision-Making and Society [CE200100005]; Australian Government through the Australian Research Council; Hasler Foundation, Switzerland [23082]

作者

  • X

    Xuan, Yueqing (通讯) (第一作者)

  • S

    Small, Edward

查看全部 5 位作者收起作者
  • S

    Sokol, Kacper

  • H

    Hettiachchi, Danula

  • S

    Sanderson, Mark

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417