ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Association of Clinician Diagnostic Performance With Machine Learning-Based Decision Support Systems A Systematic Review

Vasey, Baptiste*; Ursprung, Stephan; Beddoe, Benjamin; Taylor, Elliott H.; Marlow, Neale; Bilbro, Nicole; Watkinson, Peter; McCulloch, Peter
JAMA Network Open2021-3-11DOI: 10.1001/jamanetworkopen.2021.1276
61被引4阅读
引用收藏
分享

摘要

IMPORTANCE An increasing number of machine learning (ML)-based clinical decision support systems (CDSSs) are described in the medical literature, but this research focuses almost entirely on comparing CDSS directly with clinicians (human vs computer). Little is known about the outcomes of these systems when used as adjuncts to human decision-making (human vs human with computer). OBJECTIVES To conduct a systematic review to investigate the association between the interactive use of ML-based diagnostic CDSSs and clinician performance and to examine the extent of the CDSSs' human factors evaluation. EVIDENCE REVIEW A search of MEDLINE, Embase, PsycINFO, and grey literature was conducted for the period between January 1, 2010, and May 31, 2019. Peer-reviewed studies published in English comparing human clinician performance with and without interactive use of an ML-based diagnostic CDSSs were included. All metrics used to assess human performance were considered as outcomes. The risk of biaswas assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Risk of Bias in Non-Randomised Studies-Intervention (ROBINS-I). Narrative summaries were produced for the main outcomes. Given the heterogeneity of medical conditions, outcomes of interest, and evaluation metrics, no meta-analysis was performed. FINDINGS A total of 8112 studies were initially retrieved and 5154 abstracts were screened; of these, 37 studiesmet the inclusion criteria. The median number of participating clinicians was 4 (interquartile range, 3-8). Of the 107 results that reported statistical significance, 54 (50%) were increased by the use of CDSSs, 4 (4%) were decreased, and 49 (46%) showed no change or an unclear change. In the subgroup of studies carried out in representative clinical settings, no association between the use of ML-based diagnostic CDSSs and improved clinician performance could be observed. Interobserver agreement was the commonly reported outcome whose change was the most strongly associated with CDSS use. Four studies (11%) reported on user feedback, and, in all but 1 case, clinicians decided to override at least some of the algorithms' recommendations. Twenty-eight studies (76%) were rated as having a high risk of bias in at least 1 of the 4 QUADAS-2 core domains, and 6 studies (16%) were considered to be at serious or critical risk of bias using ROBINS-I. CONCLUSIONS AND RELEVANCE This systematic review found only sparse evidence that the use of ML-based CDSSs is associated with improved clinician diagnostic performance. Most studies had a lownumber of participants, were at high or unclear risk of bias, and showed little or no consideration for human factors. Caution should be exercised when estimating the current potential of ML to improve human diagnostic performance, and more comprehensive evaluation should be conducted before deploying ML-based CDSSs in clinical settings. The results highlight the importance of considering supported human decisions as end points rather than merely the stand-alone CDSSs outputs.

学科与关键词

关键词
COMPUTER-AIDED DETECTIONFIELD DIGITAL MAMMOGRAPHYBREAST ULTRASOUNDPRACTITIONER PERFORMANCEARTIFICIAL-INTELLIGENCESCREENING MAMMOGRAPHYPATIENT OUTCOMESRADIOLOGISTSACCURACYIMPACT
相关
加载中…
  • Effects of machine learning-based clinical decision support systems on decision-making, care delivery, and patient outcomes: a scoping review
    Susanto, Anindya Pradipta*; Lyell, David; Widyantoro, Bambang; Berkovsky, Shlomo; Magrabi, FarahJournal of the American Medical Informatics Association : JAMIA , 2023, 30(12): 2050-2063.被引 6
  • When performance is not enough-A multidisciplinary view on clinical decision support
    Roller, Roland*; Burchardt, Aljoscha; Samhammer, David; Ronicke, Simon; Duettmann, Wiebke; Schmeier, Sven; Moeller, Sebastian; Dabrock, Peter; Budde, Klemens; Mayrdorfer, Manuel; Osmanodja, BilginPLoS One, 2023, 18(4): e0282619.被引 1
  • Bringing Machine Learning Systems into Clinical Practice: A Design Science Approach to Explainable Machine Learning-Based Clinical Decision Support Systems
    Pumplun, Luisa*; Peters, Felix; Gawlitza, Joshua F.; Buxmann, PeterJournal of the Association for Information Systems, 2023, 24(4): 953-979.被引 3
查看全部 →

发表信息

论文状态
公开发表
期刊
JAMA Network Open
发表日期
2021-03-11
卷期
第 4 卷, 第 3 期
页码
e211276
DOI
10.1001/jamanetworkopen.2021.1276
期刊收录
NSL
期刊等级
1区
成果收录
SCIE
资助标注
Berrow Foundation (Lincoln College, University of Oxford); Cambridge Commonwealth, European & International Trust; NIHR Biomedical Research Centre, Oxford

作者

  • V

    Vasey, Baptiste (通讯) (第一作者)

  • U

    Ursprung, Stephan

查看全部 8 位作者收起作者
  • B

    Beddoe, Benjamin

  • T

    Taylor, Elliott H.

  • M

    Marlow, Neale

  • B

    Bilbro, Nicole

  • W

    Watkinson, Peter

  • M

    McCulloch, Peter

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417