Alzheimer&'s Disease Prediction via Brain Structural-Functional Deep Fusing Network

作者：Zuo, Qiankun; Shen, Yanyan; Zhong, Ning; Chen, C. L. Philip; Lei, Baiying^*; Wang, Shuqiang^*

来源：IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 2023, 31: 4601-4612.

DOI：10.1109/TNSRE.2023.3333952

• Multimodal fusion
• brain network computing
• swapping bi-attention mechanism
• generative adversarial strategy

Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer's disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights into detecting AD-related abnormal neural circuits.

• 单位
  中国科学院