ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction

Rasmy, Laila; Xiang, Yang*; Xie, Ziqian; Tao, Cui; Zhi, Degui*
npj Digital Medicine2021-5-20DOI: 10.1038/s41746-021-00455-y
308被引19阅读
引用收藏
分享

摘要

Deep learning (DL)-based predictive models from electronic health records (EHRs) deliver impressive performance in many clinical tasks. Large training cohorts, however, are often required by these models to achieve high accuracy, hindering the adoption of DL-based models in scenarios with limited training data. Recently, bidirectional encoder representations from transformers (BERT) and related models have achieved tremendous successes in the natural language processing domain. The pretraining of BERT on a very large training corpus generates contextualized embeddings that can boost the performance of models trained on smaller datasets. Inspired by BERT, we propose Med-BERT, which adapts the BERT framework originally developed for the text domain to the structured EHR domain. Med-BERT is a contextualized embedding model pretrained on a structured EHR dataset of 28,490,650 patients. Fine-tuning experiments showed that Med-BERT substantially improves the prediction accuracy, boosting the area under the receiver operating characteristics curve (AUC) by 1.21-6.14% in two disease prediction tasks from two clinical databases. In particular, pretrained Med-BERT obtains promising performances on tasks with small fine-tuning training sets and can boost the AUC by more than 20% or obtain an AUC as high as a model trained on a training set ten times larger, compared with deep learning models without Med-BERT. We believe that Med-BERT will benefit disease prediction studies with small local training datasets, reduce data collection expenses, and accelerate the pace of artificial intelligence aided healthcare.

学科与关键词

关键词
ARTIFICIAL-INTELLIGENCEBIG DATACARE
相关
加载中…
  • Transformer models in biomedicine
    Madan, Sumit*; Lentzen, Manuel; Brandt, Johannes; Rueckert, Daniel; Hofmann-Apitius, Martin; Froehlich, HolgerBMC Medical Informatics and Decision Making, 2024, 24(1): 214.被引 1
  • Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT
    Chen, Peng; Zhang, Meng; Yu, Xiaosheng; Li, Songpu*BMC Medical Informatics and Decision Making, 2022, 22(1): 315.被引 5
  • FarSight: Long-Term Disease Prediction Using Unstructured Clinical Nursing Notes
    Gangavarapu, Tushaar*; Krishnan, Gokul S.; Kamath, Sowmya S.; Jeganathan, JayakumarIEEE Transactions on Emerging Topics in Computing, 2021, 9(3): 1151-1169.被引 23
查看全部 →

发表信息

论文状态
公开发表
期刊
npj Digital Medicine
发表日期
2021-05-20
卷期
第 4 卷, 第 1 期
页码
86
DOI
10.1038/s41746-021-00455-y
期刊收录
NSL
期刊等级
1区
成果收录
SCIE
资助标注
CPRIT Grant [RP170668]; Xiaoqian Jiang's UT Star Award; American Heart Association [19GPSGC35180031]; Cancer Prevention and Research Institute of Texas (CPRIT) [RP170668]; UTHealth Innovation for Cancer Prevention Research Training Program Pre-Doctoral Fellowship (CPRIT Grant) [RP160015]; NVIDIA Corporation

作者

  • R

    Rasmy, Laila (第一作者)

  • X

    Xiang, Yang (通讯)

查看全部 5 位作者收起作者
  • X

    Xie, Ziqian

  • T

    Tao, Cui

  • Z

    Zhi, Degui (通讯)

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417