A Spatio-Temporal Multi-level Adaptive Interactive Attention Network for Open-Vocabulary EEG Decoding
CSTR:
Author:
Affiliation:

School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China

Clc Number:

TP391.41;TP183

Fund Project:

Natural Science Foundation of Shanghai (No.22ZR1443700).

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Decoding electroencephalography (EEG) signals into natural language text is a fundamental task in brain-computer interface (BCI) research. Existing approaches often suffer from insufficient modeling of spatiotemporal EEG characteristics, limited capability in capturing hierarchical semantic relationships, and poor generalization across subjects. To address these issues, this study proposes a spatio-temporal multi-level adaptive cross-subject interactive attention network (ST-MSIAN). First, the spatiotemporal feature enhancemant module (SFEM) employs parallel temporal and spatial branches to jointly model temporal dynamics and channel dependencies of EEG signals. Multi-scale dilated convolutions with dilation rates of 1, 2, and 4 are adopted to capture both local and long-range temporal dependencies, while spatial convolutions and attention mechanisms enhance inter-channel representations. Subsequently, subject-specific tokens are introduced to explicitly characterize individual neural patterns and reduce the influence of inter-subject variability. Based on these representations, the multi-level interactive attention mechanism (MIAM) progressively extracts hierarchical semantic information through channel-level, subject-level, and global-level attention interactions. The resulting features are further processed by a six-layer Transformer encoder to model contextual dependencies. Finally, a KAN-based projection head with learnable activation functions is employed to establish a more effective nonlinear mapping between high-dimensional EEG features and the language embedding space of a pretrained BART decoder, enabling accurate text generation. Experiments are conducted on the publicly available Zurich cognitive language processing corpus (ZuCo), including both ZuCo v1.0 and ZuCo v2.0 datasets. The proposed ST-MSIAN achieves a BLEU-1 score of 44.10%, a BLEU-3 score of 16.21%, a ROUGE-F score of 37.62%, and a BERTScore-F score of 60.08%. Compared with the strongest competing method, the proposed approach improves BLEU-1, ROUGE-F, and BERTScore-F by 2.23, 4.87, and 4.66 percentage points, respectively. The proposed ST-MSIAN provides an effective and robust solution for cross-subject neural language decoding and offers promising potential for future applications in assistive communication systems and intelligent brain-computer interfaces.

    Reference
    Related
    Cited by
Get Citation

CHANG Litao, WANG Yongxiong, HUANG Shuai, WANG Zhe. A Spatio-Temporal Multi-level Adaptive Interactive Attention Network for Open-Vocabulary EEG Decoding[J]. Journal of Data Acquisition and Processing,2026,(4):1010-1025.

Copy
Related Videos

Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:January 23,2026
  • Revised:July 15,2026
  • Adopted:
  • Online: August 13,2026
  • Published:
Article QR Code