ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Development and evaluation of a large language model of ophthalmology in Chinese

Zheng, Ce; Ye, Hongfei; Guo, Jinming; Yang, Junrui; Fei, Ping; Yuan, Yuanzhi; Huang, Danqing; Huang, Yuqiang; Peng, Jie; Xie, Xiaoling; Xie, Meng; Zhao, Peiquan; Chen, Li*; Zhang, Mingzhi*复旦大学; 汕头大学; 上海交通大学
British Journal of Ophthalmology2024-10DOI: 10.1136/bjo-2023-324526
3被引2阅读
引用收藏
分享

摘要

Background Large language models (LLMs), such as ChatGPT, have considerable implications for various medical applications. However, ChatGPT's training primarily draws from English-centric internet data and is not tailored explicitly to the medical domain. Thus, an ophthalmic LLM in Chinese is clinically essential for both healthcare providers and patients in mainland China. Methods We developed an LLM of ophthalmology (MOPH) using Chinese corpora and evaluated its performance in three clinical scenarios: ophthalmic board exams in Chinese, answering evidence-based medicine-oriented ophthalmic questions and diagnostic accuracy for clinical vignettes. Additionally, we compared MOPH's performance to that of human doctors. Results In the ophthalmic exam, MOPH's average score closely aligned with the mean score of trainees (64.7 (range 62-68) vs 66.2 (range 50-92), p=0.817), but achieving a score above 60 in all seven mock exams. In answering ophthalmic questions, MOPH demonstrated an adherence of 83.3% (25/30) of responses following Chinese guidelines (Likert scale 4-5). Only 6.7% (2/30, Likert scale 1-2) and 10% (3/30, Likert scale 3) of responses were rated as 'poor or very poor' or 'potentially misinterpretable inaccuracies' by reviewers. In diagnostic accuracy, although the rate of correct diagnosis by ophthalmologists was superior to that by MOPH (96.1% vs 81.1%, p>0.05), the difference was not statistically significant. Conclusion This study demonstrated the promising performance of MOPH, a Chinese-specific ophthalmic LLM, in diverse clinical scenarios. MOPH has potential real-world applications in Chinese-language ophthalmology settings.

学科与关键词

关键词
Public healthMedical Education
相关
加载中…
  • ChatGPT and Beyond: An overview of the growing field of large language models and their use in ophthalmology
    Kedia, Nikita; Sanjeev, Suvansh; Ong, Joshua; Chhablani, Jay*Eye, 2024, 38(7): 1252-1261.被引 6
  • OphGLM: An ophthalmology large language-and-vision assistant
    Deng, Zhuo; Gao, Weihao; Chen, Chucheng; Niu, Zhiyuan; Gong, Zheng; Zhang, Ruiheng; Cao, Zhenjie; Li, Fang; Ma, Zhaoyi; Wei, Wenbin; Ma, Lan*Artificial Intelligence in Medicine, 2024, 157: 103001.
  • Assessing the possibility of using large language models in ocular surface diseases
    Ling, Qian; Xu, Zi-Song; Zeng, Yan-Mei; Hong, Qi; Qian, Xian-Zhe; Hu, Jin-Yu; Pei, Chong-Gang; Wei, Hong; Zou, Jie; Chen, Cheng; Wang, Xiao-Yu; Chen, Xu; Wu, Zhen-Kai*; Shao, Yi*INTERNATIONAL JOURNAL OF OPHTHALMOLOGY, 2025, 18(1): 1-8.
查看全部 →

发表信息

论文状态
公开发表
期刊
British Journal of Ophthalmology
发表日期
2024-10
卷期
第 108 卷, 第 10 期
页码
1390-1397
DOI
10.1136/bjo-2023-324526
语言
ENGLISH
成果收录
SCIE
资助标注
National Natural Science Foundation of China [82171044]; Special Strategic Project on Innovative Science and Technology of Guangdong Province [STKJ202209073]; Hospital Funded Clinical Research, Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine [21XJMR02]; Hospital Management Research Program of Institute of Hospital Development Strategy, China Hospital Development Institute, Shanghai Jiao Tong University [HDSI-2022-A-001]; Interdisciplinary Program of Shanghai Jiao Tong University [YG2021QN52]

作者

  • Z

    Zheng, Ce (第一作者)

    shanghai jiao tong univ

  • Y

    Ye, Hongfei

    shanghai jiao tong univ

查看全部 14 位作者收起作者
  • G

    Guo, Jinming

    shantou univ

  • Y

    Yang, Junrui

  • F

    Fei, Ping

    shanghai jiao tong univ

  • Y

    Yuan, Yuanzhi

    fudan univ

  • H

    Huang, Danqing

    shanghai jiao tong univ

  • H

    Huang, Yuqiang

    shantou univ

  • P

    Peng, Jie

    shanghai jiao tong univ

  • X

    Xie, Xiaoling

    shantou univ

  • X

    Xie, Meng

    shanghai jiao tong univ

  • Z

    Zhao, Peiquan

    shanghai jiao tong univ

  • C

    Chen, Li (通讯)

    shanghai jiao tong univ

  • Z

    Zhang, Mingzhi (通讯)

    shantou univ

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417