ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Visual-language foundation models in medicine

Liu, Chunyu; Jin, Yixiao; Guan, Zhouyu; Li, Tingyao; Qin, Yiming; Qian, Bo; Jiang, Zehua; Wu, Yilan; Wang, Xiangning; Zheng, Ying Feng; Zeng, Dian*清华大学; 上海交通大学
VISUAL COMPUTER2024-7-29DOI: 10.1007/s00371-024-03579-w
1阅读
引用收藏
分享

摘要

By integrating visual and linguistic understanding, visual-language foundation models (VLFMs) have the great potential to advance the interpretation of medical data, thereby enhancing diagnostic precision, treatment planning, and patient management. We reviewed the developmental strategies of VLFMs, detailing the pretraining strategies, and subsequent application across various healthcare facets. The challenges inherent to VLFMs are described, including safeguarding data privacy amidst sensitive medical data usage, ensuring algorithmic transparency, and fostering explainability for trust in clinical decision-making. We underscored the significance of VLFMs in addressing the complexity of multimodal medical data, from visual to textual, and their potential in tasks like image-based disease diagnosis, medicine report synthesis, and longitudinal patient monitoring. It also examines the progress in VLFMs like Med-Flamingo, LLaVA-Med, and their zero-shot learning capabilities, and the exploration of parameter-efficient fine-tuning methods for efficient adaptation. This review concludes by encouraging the community to pursue these emergent and promising directions to strengthen the impact of artificial intelligence and deep learning on healthcare delivery and research.

学科与关键词

关键词
Foundation modelMulti-modality dataDeep learningComputer visionNatural language processingHealthcare
相关
加载中…
  • Multimodal Large Language Models in Health Care:Applications,Challenges, and Future Outlook
    AlSaad, Rawan*; Abd-alrazaq, Alaa; Boughorbel, Sabri; Ahmed, Arfan; Renault, Max-Antoine; Damseh, Rafat; Sheikh, JavaidJOURNAL OF MEDICAL INTERNET RESEARCH, 2024, 26: e59505.被引 3
  • Understanding natural language: Potential application of large language models to ophthalmology
    Yang, Zefeng; Wang, Deming; Zhou, Fengqi; Song, Diping; Zhang, Yinhang; Jiang, Jiaxuan; Kong, Kangjie; Liu, Xiaoyi; Qiao, Yu; Chang, Robert T.; Han, Ying; Li, Fei*; Tham, Clement C.*; Zhang, Xiulan*Asia - Pacific Journal of Ophthalmology, 2024, 13(4): 100085.
  • A visual-language foundation model for computational pathology
    Lu, Ming Y.; Chen, Bowen; Williamson, Drew F. K.; Chen, Richard J.; Liang, Ivy; Ding, Tong; Jaume, Guillaume; Odintsov, Igor; Le, Long Phi; Gerber, Georg; Parwani, Anil V.; Zhang, Andrew; Mahmood, Faisal*Nature Medicine, 2024, 30(3).被引 39
查看全部 →

发表信息

论文状态
公开发表
期刊
VISUAL COMPUTER
发表日期
2024-07-29
DOI
10.1007/s00371-024-03579-w
语言
ENGLISH
成果收录
SCIE

作者

  • L

    Liu, Chunyu (第一作者)

    shanghai jiao tong univ · shanghai clin ctr diabet

  • J

    Jin, Yixiao

    tsinghua univ

查看全部 11 位作者收起作者
  • G

    Guan, Zhouyu

    shanghai jiao tong univ · shanghai clin ctr diabet

  • L

    Li, Tingyao

    shanghai jiao tong univ · shanghai clin ctr diabet

  • Q

    Qin, Yiming

    shanghai jiao tong univ · shanghai clin ctr diabet

  • Q

    Qian, Bo

    shanghai jiao tong univ · shanghai clin ctr diabet

  • J

    Jiang, Zehua

    tsinghua univ

  • W

    Wu, Yilan

    tsinghua univ

  • W

    Wang, Xiangning

    shanghai jiao tong univ

  • Z

    Zheng, Ying Feng

  • Z

    Zeng, Dian (通讯)

    tsinghua univ

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417