Non-invasive Continuous Chinese Language Semantic Decoding and Reconstruction
CSTR:
Author:
Affiliation:

1.School of Information Science and Technology, Nantong University, Nantong 226019, China;2.Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London WC2R 2LS, UK

Clc Number:

TP181;Q189

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Language is an important tool for communication and cognition. Multiple functional areas of the brain, connected through complex neural networks, jointly participate in the perception, comprehension, and production of language. Exploring the neural mechanisms of Chinese semantic decoding is crucial for the development of Chinese brain-computer interface (BCI). This study aims to establish a long-sequence continuous semantic decoding method based on fMRI data, termed Chinese long-sequence continuous semantic decoder(CLCSD). Through signal processing workflows and algorithm optimization, it seeks to achieve efficient decoding of continuous Chinese semantics. The CLCSD framework is composed of four components: neural response dimensionality reduction, an encoding model, a word rate model, and a beam search decoding model. Neural response dimensionality reduction is performed through cortical reconstruction, image registration, and brain region parcellation to reduce four-dimensional brain response data into a two-dimensional matrix. The encoding model is constructed using L2-regularized regression (ridge regression) to establish the relationship between stimulus features and brain responses, with noise covariance estimated via bootstrapping to enhance generalization. The word rate model follows a similar approach to the encoding model, where brain response features are mapped to predicted word rate. The beam search decoding model uses the prior probability of the language model and likelihood probabilities of the encoding model to generate the most probable semantic sequence through beam search. On publicly available dataset SMN4Lang, CLCSD achieves a mean BERTScore of 0.674, outperforming other long-sequence Chinese continuous semantic decoding models. The proposed method provides an efficient long-sequence continuous Chinese semantic decoding approach, offering both theoretical foundations and methodological references for the advancement of Chinese BCI technologies.

    Reference
    Related
    Cited by
Get Citation

MA Lei, CUI Wenhao, YANG Wenwen, WANG Zhaoxin. Non-invasive Continuous Chinese Language Semantic Decoding and Reconstruction[J].,2025,40(3):616-636.

Copy
Related Videos

Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:December 30,2024
  • Revised:May 08,2025
  • Adopted:
  • Online: June 13,2025
  • Published:
Article QR Code