ScholarMate
English简体中文繁體中文
Customer service: 400-1616-289
Sign inSign up
Journal article

Transformer models in biomedicine

Madan, Sumit*; Lentzen, Manuel; Brandt, Johannes; Rueckert, Daniel; Hofmann-Apitius, Martin; Froehlich, Holger
BMC Medical Informatics and Decision Making2024-7-29DOI: 10.1186/s12911-024-02600-5
1Cited by7Reads
CiteSave
Share

Abstract

Deep neural networks (DNN) have fundamentally revolutionized the artificial intelligence (AI) field. The transformer model is a type of DNN that was originally used for the natural language processing tasks and has since gained more and more attention for processing various kinds of sequential data, including biological sequences and structured electronic health records. Along with this development, transformer-based models such as BioBERT, MedBERT, and MassGenie have been trained and deployed by researchers to answer various scientific questions originating in the biomedical domain. In this paper, we review the development and application of transformer models for analyzing various biomedical-related datasets such as biomedical textual data, protein sequences, medical structured-longitudinal data, and biomedical images as well as graphs. Also, we look at explainable AI strategies that help to comprehend the predictions of transformer-based models. Finally, we discuss the limitations and challenges of current models, and point out emerging novel research directions.

Disciplines and keywords

Keywords
TransformerBiomedicineLife ScienceDeep learningNeural networksMachine learning
Related
Loading…
  • Transformers and large language models in healthcare: A review
    Nerella, Subhash; Bandyopadhyay, Sabyasachi; Zhang, Jiaqing; Contreras, Miguel; Siegel, Scott; Bumin, Aysegul; Silva, Brandon; Sena, Jessica; Shickel, Benjamin; Bihorac, Azra; Khezeli, Kia; Rashidi, Parisa*Artificial Intelligence in Medicine, 2024, 154: 102900.Cited by 4
  • Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction
    Rasmy, Laila; Xiang, Yang*; Xie, Ziqian; Tao, Cui; Zhi, Degui*npj Digital Medicine, 2021, 4(1): 86.Cited by 308
  • Clinical concept extraction using transformers
    Yang, Xi; Bian, Jiang; Hogan, William R.; Wu, Yonghui*Journal of the American Medical Informatics Association : JAMIA , 2020, 27(12): 1935-1942.Cited by 82
View all →

Publication info

Publication status
Published
Journal
BMC Medical Informatics and Decision Making
Publication date
2024-07-29
Volume / issue
Vol. 24, Issue 1
Pages
214
DOI
10.1186/s12911-024-02600-5
Language
ENGLISH
Output indexing
SCIE
Funding acknowledgment
Innovative Health Initiative (IHI) Joint Undertaking (JU)

Authors

  • M

    Madan, Sumit (Corresponding) (First author)

  • L

    Lentzen, Manuel

View all 6 authorsShow fewer authors
  • B

    Brandt, Johannes

  • R

    Rueckert, Daniel

  • H

    Hofmann-Apitius, Martin

  • F

    Froehlich, Holger

Sign in for more features

  • Save this paper in My collections
  • Claim your outputs and generate promotion materials
  • Follow authors for new outputs
  • Join groups and collaborate
Sign up / Sign in

Products

  • ScholarMate
  • InnoCity
  • IRISaaS

Support

  • Help Center
  • Privacy Policy
  • Terms of Service

Contact

Online support: []
Customer service: 400-1616-289
Email: support@scholarmate.com

Follow or download ScholarMate

WeChat QR code
WeChat official account
App download QR code
Download the app
OutputsResearchersResearch institutionsResearch newsIRISaaS

©2026 深圳市科研之友网络服务有限公司

Public security registration badge粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417