ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Multimodal Deep Learning for Integrating Chest Radiographs and Clinical Parameters: A Case for Transformers

Khader, Firas*; Mueller-Franzes, Gustav; Wang, Tianci; Han, Tianyu; Arasteh, Soroosh Tayebi; Haarburger, Christoph; Stegmaier, Johannes; Bressem, Keno; Kuhl, Christiane; Nebelung, Sven; Kather, Jakob Nikolas; Truhn, Daniel河北医科大学
RADIOLOGY2023-10DOI: 10.1148/radiol.230806
19被引1阅读
引用收藏
分享

摘要

Background: Clinicians consider both imaging and nonimaging data when diagnosing diseases; however, current machine learning approaches primarily consider data from a single modality.Purpose: To develop a neural network architecture capable of integrating multimodal patient data and compare its performance to models incorporating a single modality for diagnosing up to 25 pathologic conditions.Materials and Methods: In this retrospective study, imaging and nonimaging patient data were extracted from the Medical Information Mart for Intensive Care (MIMIC) database and an internal database comprised of chest radiographs and clinical parameters inpatients in the intensive care unit (ICU) (January 2008 to December 2020). The MIMIC and internal data sets were each split into training (n = 33 893, n = 28 809), validation (n = 740, n = 7203), and test (n = 1909, n = 9004) sets. A novel transformer-based neural network architecture was trained to diagnose up to 25 conditions using nonimaging data alone, imaging data alone, or multimodal data. Diagnostic performance was assessed using area under the receiver operating characteristic curve (AUC) analysis.Results: The MIMIC and internal data sets included 36 542 patients (mean age, 63 years +/- 17 [SD]; 20 567 male patients) and 45 016 patients (mean age, 66 years +/- 16; 27 577 male patients), respectively. The multimodal model showed improved diagnostic performance for all pathologic conditions. For the MIMIC data set, the mean AUC was 0.77 (95% CI: 0.77, 0.78) when both chest radiographs and clinical parameters were used, compared with 0.70 (95% CI: 0.69, 0.71; P < .001) for only chest radiographs and 0.72 (95% CI: 0.72, 0.73; P < .001) for only clinical parameters. These findings were confirmed on the internal data set.Conclusion: A model trained on imaging and nonimaging data outperformed models trained on only one type of data for diagnosing multiple diseases in patients in an ICU setting.

学科与关键词

关键词
ARTIFICIAL-INTELLIGENCEIMAGE
相关
加载中…

文献加载失败,请重试

重试

发表信息

论文状态
公开发表
期刊
RADIOLOGY
发表日期
2023-10
卷期
第 309 卷, 第 1 期
页码
e230806
DOI
10.1148/radiol.230806
语言
ENGLISH
成果收录
SCIE
资助标注
German Federal Ministry of Health (DEEP LIVER) [ZMVI1-2520DAT111]; Max-Eder-Programme of the German Cancer Aid; Deutsche Forschungsgemeinschaft (DFG) [NE 2136/3-1]

作者

  • K

    Khader, Firas (通讯) (第一作者)

  • M

    Mueller-Franzes, Gustav

查看全部 12 位作者收起作者
  • W

    Wang, Tianci

  • H

    Han, Tianyu

  • A

    Arasteh, Soroosh Tayebi

  • H

    Haarburger, Christoph

  • S

    Stegmaier, Johannes

  • B

    Bressem, Keno

  • K

    Kuhl, Christiane

  • N

    Nebelung, Sven

  • K

    Kather, Jakob Nikolas

    inst med

  • T

    Truhn, Daniel

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417