基于U-Net和Transformer结合的不完整多模态脑肿瘤分割方法
作者:
作者单位:

1.昆明理工大学信息工程与自动化学院, 昆明 650500;2.昆明理工大学电力工程学院, 昆明 650500

作者简介:

通讯作者:

基金项目:

国家自然科学基金(82160347);云南省基础研究专项——青年项目(202401AU070148)。


Incomplete Multimodal Brain Tumor Segmentation Method Based on the Combination of U-Net and Transformer
Author:
Affiliation:

1.School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China;2.School of Electric Power Engineering, Kunming University of Science and Technology, Kunming 650500, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    由于患者个体差异、采集协议多样性和数据损坏等因素,现有基于磁共振成像(Magnetic resonance imaging,MRI)的脑肿瘤分割方法存在模态数据丢失问题,导致分割精度不高。为此,本文提出了一种基于U-Net和Transformer结合的不完整多模态脑肿瘤分割(Incomplete multimodal brain tumor segmentation based on the combination of U-Net and Transformer,IM TransNet)方法。首先,针对脑肿瘤MRI的4个不同模态设计了单模态特定编码器,提升模型对各模态数据的表征能力。其次,在U-Net中嵌入双重注意力的Transformer模块,克服模态缺失引起的信息不完整问题,减少U-Net的长距离上下文交互和空间依赖性局限。在U-Net的跳跃连接中加入跳跃交叉注意力机制,动态关注不同层级和模态的特征,即使在模态缺失时,也能有效融合特征并进行重建。此外,针对模态缺失引起的训练不平衡问题,设计了辅助解码模块,确保模型在各种不完整模态子集上均能稳定高效地分割脑肿瘤。最后,基于公开数据集BRATS验证模型的性能。实验结果表明,本文提出的模型在增强型肿瘤、肿瘤核心和全肿瘤上的平均Dice评分分别为63.19%、76.42%和86.16%,证明了其在处理不完整多模态数据时的优越性和稳定性,为临床实践中脑肿瘤的准确、高效和可靠分割提供了一种可行的技术手段。

    Abstract:

    Given inherent variations among patients, discrepancies in imaging protocols, and potential data corruption, existing brain tumor segmentation methods based on magnetic resonance imaging (MRI) are often challenged by the issue of missing modality data, resulting in low segmentation accuracy. To address this, an innovative incomplete multimodal brain tumor segmentation method based on the combination of U-Net and Transformer (IM TransNet) is proposed. Firstly, a modality-specific encoder is developed for four distinct MRI modalities to enhance the model’s ability to capture unique characteristics of each modality. Secondly, a dual-attention Transformer module is embedded within the U-Net to mitigate the issue of incomplete information arising from missing modalities, thus alleviating the limitations imposed by long-range context interactions and spatial dependencies within the U-Net framework. Additionally, a skip-cross attention mechanism is incorporated into the U-Net’s skip connections to dynamically focus on features from various hierarchical levels and modalities, effectively facilitating feature fusion and reconstruction even in the presence of missing modalities. Furthermore, an auxiliary decoding module is devised to counteract the training imbalance induced by missing modalities, ensuring that the model can consistently and effectively segment brain tumors across diverse subsets of incomplete modalities. Finally, the model’s performance is validated on the publicly accessible BRATS dataset. Experimental results indicate that the proposed model attains average Dice scores of 63.19%, 76.42%, and 86.16% for enhancing tumor, tumor core, and whole tumor, respectively, highlighting its superiority and robustness in handling incomplete multimodal data. This approach offers a viable technical solution for accurate, efficient, and reliable brain tumor segmentation in clinical practice.

    参考文献
    相似文献
    引证文献
引用本文

汤占军,蹇洪,王健.基于U-Net和Transformer结合的不完整多模态脑肿瘤分割方法[J].数据采集与处理,2025,40(4):934-949

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2024-07-11
  • 最后修改日期:2024-11-14
  • 录用日期:
  • 在线发布日期: 2025-08-15