ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Multimodal Large Language Models in Health Care:Applications,Challenges, and Future Outlook

AlSaad, Rawan*; Abd-alrazaq, Alaa; Boughorbel, Sabri; Ahmed, Arfan; Renault, Max-Antoine; Damseh, Rafat; Sheikh, Javaid
JOURNAL OF MEDICAL INTERNET RESEARCH2024-9-25DOI: 10.2196/59505
3被引1阅读
引用收藏
分享

摘要

In the complex and multidimensional field of medicine, multimodal data are prevalent and crucial for informed clinical decisions.Multimodal data span a broad spectrum of data types, including medical images (eg, MRI and CT scans), time-series data (eg,sensor data from wearable devices and electronic health records), audio recordings (eg, heart and respiratory sounds and patientinterviews), text (eg, clinical notes and research articles), videos (eg, surgical procedures), and omics data (eg, genomics andproteomics). While advancements in large language models (LLMs) have enabled new applications for knowledge retrieval andprocessing in the medical field, most LLMs remain limited to processing unimodal data, typically text-based content, and oftenoverlook the importance of integrating the diverse data modalities encountered in clinical practice. This paper aims to present adetailed, practical, and solution-oriented perspective on the use of multimodal LLMs (M-LLMs) in the medical field. Ourinvestigation spanned M-LLM foundational principles, current and potential applications, technical and ethical challenges, andfuture research directions. By connecting these elements, we aimed to provide a comprehensive framework that links diverseaspects of M-LLMs, offering a unified vision for their future in health care. This approach aims to guide both future research andpractical implementations of M-LLMs in health care, positioning them as a paradigm shift toward integrated, multimodaldata-driven medical practice. We anticipate that this work will spark further discussion and inspire the development of innovativeapproaches in the next generation of medical M-LLM systems

学科与关键词

关键词
artificial intelligencelarge language modelsmultimodal large language modelsmultimodalitymultimodal generative artificialintelligencemultimodal generative AIgenerative artificial intelligencegenerative AIhealth care
相关
加载中…
  • Based on Medicine, The Now and Future of Large Language Models
    Su, Ziqing; Tang, Guozhang; Huang, Rui; Qiao, Yang; Zhang, Zheng; Dai, Xingliang*Cellular and Molecular Bioengineering, 2024, 17(4): 263-277.被引 1
  • Visual-language foundation models in medicine
    Liu, Chunyu; Jin, Yixiao; Guan, Zhouyu; Li, Tingyao; Qin, Yiming; Qian, Bo; Jiang, Zehua; Wu, Yilan; Wang, Xiangning; Zheng, Ying Feng; Zeng, Dian*VISUAL COMPUTER, 2024.
  • Artificial intelligence accelerates multi-modal biomedical process: A Survey
    Li, Jiajia; Han, Xue; Qin, Yiming; Tan, Feng; Chen, Yulong; Wang, Zikai; Song, Haitao; Zhou, Xi; Zhang, Yuan; Hu, Lun; Hu, Pengwei*NEUROCOMPUTING, 2023, 558: 126720.被引 9
查看全部 →

发表信息

论文状态
公开发表
期刊
JOURNAL OF MEDICAL INTERNET RESEARCH
发表日期
2024-09-25
卷期
第 26 卷
页码
e59505
DOI
10.2196/59505
语言
ENGLISH
成果收录
SCIE

作者

  • A

    AlSaad, Rawan (通讯) (第一作者)

  • A

    Abd-alrazaq, Alaa

查看全部 7 位作者收起作者
  • B

    Boughorbel, Sabri

  • A

    Ahmed, Arfan

  • R

    Renault, Max-Antoine

  • D

    Damseh, Rafat

  • S

    Sheikh, Javaid

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417