基于图学习的缺失脑网络生成及多模态融合诊断方法
作者:
作者单位:

1.南京航空航天大学数学学院,南京 211106;2.飞行器数学建模与高性能计算工信部重点实验室,南京 211106;3.南京航空航天大学人工智能学院,南京 211106;4.脑机智能技术教育部重点实验室,南京 211106

作者简介:

通讯作者:

基金项目:

国家自然科学基金(12071215,62076129, 62371234);江苏省自然科学基金(BK20231438)。


Graph Learning-Based Methods for Generating Missing Brain Networks and Multi-modal Fusion Diagnosis
Author:
Affiliation:

1.School of Mathematics, Nanjing University of Aeronautics & Astronautics, Nanjing 211106, China;2.Key Laboratory of Mathematical Modelling and High Performance Computing of Air Vehicles MIIT, Nanjing 211106, China;3.College of Artificial Intelligence, Nanjing University of Aeronautics & Astronautics, Nanjing 211106, China;4.Key Laboratory of Brain-Machine Intelligence Technology, Nanjing 211106, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    融合大脑结构和功能网络的多模态脑网络能够挖掘不同模态间的互补信息,有效提高癫痫等神经系统疾病的诊断准确率,在神经疾病诊断上具有优势。然而,由于多模态数据采集时间长、成本高,在实际应用中常面临模态缺失问题,导致可用数据量减少,模型的诊断精度和泛化能力下降。针对某一模态数据完全缺失问题,提出了基于图学习与循环一致生成对抗网络(Cycle-consistent generative adversarial networks, CycleGAN)的图CycleGAN方法。该方法通过引入图卷积神经网络与图注意力机制等图学习方法捕捉脑网络不同脑区间的特征信息,强化生成框架对图形式脑网络的特征提取能力,实现脑结构网络与功能网络的相互生成。此外,针对目前较少利用诊断结果评估生成数据质量的情况,提出了一种融合真实脑网络与生成脑网络的多模态融合分类模型,以进一步评估生成脑网络的有效性。在癫痫数据集上的实验结果表明,图CycleGAN方法能够有效利用已有的模态信息,实现缺失脑网络的生成。

    Abstract:

    The multi-modal brain network, which integrates the brain structural and functional networks, can effectively extract the complementary information from different modalities, significantly improving the diagnostic accuracy of neurological diseases such as epilepsy. However, due to the long acquisition time and high acquisition cost of multi-modal data collection, it often faces the problem of modality missingness in practical applications, leading to decreased diagnostic accuracy and generalization ability of the model. To address the issue of modality data completely missing, we propose a method based on graph learning methods and cycle-consistent generative adversarial networks, named Graph-CycleGAN method. This method captures feature information between different brain regions in the brain network by introducing graph neural networks, such as graph convolutional neural networks and graph attention mechanisms. Besides, it strengthens the feature extraction ability of the generative framework and realizes the mutual generation of brain structural network and functional network. In addition, to address the lack of diagnostic result-based evaluations for the quality of generated data, this paper proposes a classification model that integrates real and generated brain networks. Experimental results on the epilepsy dataset indicate that the proposed Graph-CycleGAN method can effectively realize the generation of missing brain network by utilizing the existing modality information.

    参考文献
    相似文献
    引证文献
引用本文

龚荣芳,黄麟雅,朱旗,李胜荣.基于图学习的缺失脑网络生成及多模态融合诊断方法[J].数据采集与处理,2024,(4):843-862

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2024-05-27
  • 最后修改日期:2024-07-04
  • 录用日期:
  • 在线发布日期: 2024-07-25