Incomplete Multimodal Brain Tumor Segmentation Method Based on the Combination of U-Net and Transformer
CSTR:
Author:
Affiliation:

1.School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China;2.School of Electric Power Engineering, Kunming University of Science and Technology, Kunming 650500, China

Clc Number:

R739.41

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Given inherent variations among patients, discrepancies in imaging protocols, and potential data corruption, existing brain tumor segmentation methods based on magnetic resonance imaging (MRI) are often challenged by the issue of missing modality data, resulting in low segmentation accuracy. To address this, an innovative incomplete multimodal brain tumor segmentation method based on the combination of U-Net and Transformer (IM TransNet) is proposed. Firstly, a modality-specific encoder is developed for four distinct MRI modalities to enhance the model’s ability to capture unique characteristics of each modality. Secondly, a dual-attention Transformer module is embedded within the U-Net to mitigate the issue of incomplete information arising from missing modalities, thus alleviating the limitations imposed by long-range context interactions and spatial dependencies within the U-Net framework. Additionally, a skip-cross attention mechanism is incorporated into the U-Net’s skip connections to dynamically focus on features from various hierarchical levels and modalities, effectively facilitating feature fusion and reconstruction even in the presence of missing modalities. Furthermore, an auxiliary decoding module is devised to counteract the training imbalance induced by missing modalities, ensuring that the model can consistently and effectively segment brain tumors across diverse subsets of incomplete modalities. Finally, the model’s performance is validated on the publicly accessible BRATS dataset. Experimental results indicate that the proposed model attains average Dice scores of 63.19%, 76.42%, and 86.16% for enhancing tumor, tumor core, and whole tumor, respectively, highlighting its superiority and robustness in handling incomplete multimodal data. This approach offers a viable technical solution for accurate, efficient, and reliable brain tumor segmentation in clinical practice.

    Reference
    Related
    Cited by
Get Citation

TANG Zhanjun, JIAN Hong, WANG Jian. Incomplete Multimodal Brain Tumor Segmentation Method Based on the Combination of U-Net and Transformer[J].,2025,40(4):934-949.

Copy
Related Videos

Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:July 11,2024
  • Revised:November 14,2024
  • Adopted:
  • Online: August 15,2025
  • Published:
Article QR Code