ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Clinical concept extraction using transformers

Yang, Xi; Bian, Jiang; Hogan, William R.; Wu, Yonghui*
Journal of the American Medical Informatics Association : JAMIA2020-12DOI: 10.1093/jamia/ocaa189
82被引5阅读
引用收藏
分享

摘要

Objective: The goal of this study is to explore transformer-based models (eg, Bidirectional Encoder Representations from Transformers [BERT]) for clinical concept extraction and develop an open-source package with pretrained clinical models to facilitate concept extraction and other downstream natural language processing (NLP) tasks in the medical domain. Methods: We systematically explored 4 widely used transformer-based architectures, including BERT, RoBERTa, ALBERT, and ELECTRA, for extracting various types of clinical concepts using 3 public datasets from the 2010 and 2012 i2b2 challenges and the 2018 n2c2 challenge. We examined general transformer models pretrained using general English corpora as well as clinical transformer models pretrained using a clinical corpus and compared them with a long short-term memory conditional random fields (LSTM-CRFs) mode as a baseline. Furthermore, we integrated the 4 clinical transformer-based models into an open-source package. Results and Conclusion: The RoBERTa-MIMIC model achieved state-of-the-art performance on 3 public clinical concept extraction datasets with F1-scores of 0.8994, 0.8053, and 0.8907, respectively. Compared to the baseline LSTM-CRFs model, RoBERTa-MIMIC remarkably improved the F1-score by approximately 4% and 6% on the 2010 and 2012 i2b2 datasets. This study demonstrated the efficiency of transformer-based models for clinical concept extraction. Our methods and systems can be applied to other clinical tasks. The clinical transformer package with 4 pretrained clinical models is publicly available at https://github.com/uf-hobi-informatics-lab/ClinicalTransformerNER. We believe this package will improve current practice on clinical concept extraction and other tasks in the medical domain.

学科与关键词

关键词
named entity recognitiontransformer modelsdeep learningnatural language processing
相关
加载中…
  • Transformer models in biomedicine
    Madan, Sumit*; Lentzen, Manuel; Brandt, Johannes; Rueckert, Daniel; Hofmann-Apitius, Martin; Froehlich, HolgerBMC Medical Informatics and Decision Making, 2024, 24(1): 214.被引 1
  • Transformers and large language models in healthcare: A review
    Nerella, Subhash; Bandyopadhyay, Sabyasachi; Zhang, Jiaqing; Contreras, Miguel; Siegel, Scott; Bumin, Aysegul; Silva, Brandon; Sena, Jessica; Shickel, Benjamin; Bihorac, Azra; Khezeli, Kia; Rashidi, Parisa*Artificial Intelligence in Medicine, 2024, 154: 102900.被引 4
  • Clinical named entity recognition and relation extraction using natural language processing of medical free text: A systematic review
    Navarro, David Fraile*; Ijaz, Kiran; Rezazadegan, Dana; Rahimi-Ardabili, Hania; Dras, Mark; Coiera, Enrico; Berkovsky, ShlomoInternational Journal of Medical Informatics, 2023, 177: 105122.被引 12
查看全部 →

发表信息

论文状态
公开发表
期刊
Journal of the American Medical Informatics Association : JAMIA
发表日期
2020-12
卷期
第 27 卷, 第 12 期
页码
1935-1942
DOI
10.1093/jamia/ocaa189
语言
ENGLISH
期刊收录
NSL.SSCI.SCIE
期刊等级
1区
成果收录
SCIE
资助标注
Patient-Centered Outcomes Research Institute Award [ME-2018C3-14754]; National Cancer Institute [1R01CA246418 R01]; National Institute on Aging, NIA [R21AG062884]; University of Florida Informatics Institute Junior SEED Program [00129436]; University of Florida Health Cancer Center; University of Florida Clinical and Translational Science Institute

作者

  • Y

    Yang, Xi (第一作者)

  • B

    Bian, Jiang

查看全部 4 位作者收起作者
  • H

    Hogan, William R.

  • W

    Wu, Yonghui (通讯)

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417