ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

A multimodal generative AI copilot for human pathology

Lu, Ming Y.; Chen, Bowen; Williamson, Drew F. K.; Chen, Richard J.; Zhao, Melissa; Chow, Aaron K.; Ikemura, Kenji; Kim, Ahrong; Pouli, Dimitra; Patel, Ankush; Soliman, Amr; Chen, Chengkuan; Ding, Tong; Wang, Judy J.; Gerber, Georg; Liang, Ivy; Le, Long Phi; Parwani, Anil V.; Weishaupt, Luca L.; Mahmood, Faisal*
Nature2024-10-10DOI: 10.1038/s41586-024-07618-3
26被引9阅读
引用收藏
分享

摘要

Computational pathology1,2 has witnessed considerable progress in the development of both task-specific predictive models and task-agnostic self-supervised vision encoders3,4. However, despite the explosive growth of generative artificial intelligence (AI), there have been few studies on building general-purpose multimodal AI assistants and copilots5 tailored to pathology. Here we present PathChat, a vision-language generalist AI assistant for human pathology. We built PathChat by adapting a foundational vision encoder for pathology, combining it with a pretrained large language model and fine-tuning the whole system on over 456,000 diverse visual-language instructions consisting of 999,202 question and answer turns. We compare PathChat with several multimodal vision-language AI assistants and GPT-4V, which powers the commercially available multimodal general-purpose AI assistant ChatGPT-4 (ref. 6). PathChat achieved state-of-the-art performance on multiple-choice diagnostic questions from cases with diverse tissue origins and disease models. Furthermore, using open-ended questions and human expert evaluation, we found that overall PathChat produced more accurate and pathologist-preferable responses to diverse queries related to pathology. As an interactive vision-language AI copilot that can flexibly handle both visual and natural language inputs, PathChat may potentially find impactful applications in pathology education, research and human-in-the-loop clinical decision-making. PathChat, a multimodal generative AI copilot for human pathology, has been trained on a large dataset of visual-language instructions to interactively assist users with diverse pathology tasks.

学科与关键词

关键词
ARTIFICIAL-INTELLIGENCEFOUNDATION MODELS
相关
加载中…
  • A visual-language foundation model for computational pathology
    Lu, Ming Y.; Chen, Bowen; Williamson, Drew F. K.; Chen, Richard J.; Liang, Ivy; Ding, Tong; Jaume, Guillaume; Odintsov, Igor; Le, Long Phi; Gerber, Georg; Parwani, Anil V.; Zhang, Andrew; Mahmood, Faisal*Nature Medicine, 2024, 30(3).被引 39
  • A vision-language foundation model for precision oncology
    Xiang, Jinxi; Wang, Xiyue; Zhang, Xiaoming; Xi, Yinghua; Eweje, Feyisope; Chen, Yijiang; Li, Yuchen; Bergstrom, Colin; Gopaulchan, Matthew; Kim, Ted; Yu, Kun-Hsing; Willens, Sierra; Olguin, Francesca Maria; Nirschl, Jeffrey J.; Neal, Joel; Diehn, Maximilian; Yang, Sen*; Li, Ruijiang*Nature, 2025.
  • Towards a general-purpose foundation model for computational pathology
    Chen, Richard J.; Ding, Tong; Lu, Ming Y.; Williamson, Drew F. K.; Jaume, Guillaume; Song, Andrew H.; Chen, Bowen; Zhang, Andrew; Shao, Daniel; Shaban, Muhammad; Williams, Mane; Oldenburg, Lukas; Weishaupt, Luca L.; Wang, Judy J.; Vaidya, Anurag; Le, Long Phi; Gerber, Georg; Sahai, Sharifa; Williams, Walt; Mahmood, Faisal*Nature Medicine, 2024, 30(3): 850-862.被引 84
查看全部 →

发表信息

论文状态
公开发表
期刊
Nature
发表日期
2024-10-10
卷期
第 634 卷, 第 8033 期
DOI
10.1038/s41586-024-07618-3
语言
ENGLISH
成果收录
SCIE
资助标注
Brigham & Women's Hospital president's fund; Brigham & Women's Hospital; National Science Foundation; Massachusetts Life Sciences Center infrastructure award

作者

  • L

    Lu, Ming Y. (第一作者)

  • C

    Chen, Bowen

查看全部 20 位作者收起作者
  • W

    Williamson, Drew F. K.

  • C

    Chen, Richard J.

  • Z

    Zhao, Melissa

  • C

    Chow, Aaron K.

  • I

    Ikemura, Kenji

  • K

    Kim, Ahrong

  • P

    Pouli, Dimitra

  • P

    Patel, Ankush

  • S

    Soliman, Amr

  • C

    Chen, Chengkuan

  • D

    Ding, Tong

  • W

    Wang, Judy J.

  • G

    Gerber, Georg

  • L

    Liang, Ivy

  • L

    Le, Long Phi

  • P

    Parwani, Anil V.

  • W

    Weishaupt, Luca L.

  • M

    Mahmood, Faisal (通讯)

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417