非侵入性连续中文语言语义解码与重建
作者:
作者单位:

1.南通大学信息科学技术学院,南通 226019;2.伦敦国王学院精神病学、心理学和神经科学研究所,伦敦 WC2R 2LS

作者简介:

通讯作者:

基金项目:


Non-invasive Continuous Chinese Language Semantic Decoding and Reconstruction
Author:
Affiliation:

1.School of Information Science and Technology, Nantong University, Nantong 226019, China;2.Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London WC2R 2LS, UK

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    语言是沟通和认知的基础,大脑多功能区域通过复杂神经网络共同参与语言的感知、理解与生成,深入探索中文语义解码的神经机制对于中文脑机接口(Brain-computer interface, BCI)的研究意义重大。本研究旨在构建一种基于功能性磁共振成像(Functional magnetic resonance imaging, fMRI)的长序列中文连续语义解码方法,称为中文长序列连续语义解码器(Chinese long-sequence continuous semantic decoder, CLCSD),通过信号处理流程和算法优化,实现连续中文语义的高效解码。CLCSD包含神经响应降维、编码模型、语速模型和束搜索解码模型4个部分。神经响应降维通过皮层重建、图像配准和脑区划定等方法,将4维脑响应数据降为2维矩阵。编码模型采用L2正则化回归(岭回归)建立刺激特征与脑响应之间的关系,通过自举法估计噪声协方差以增强泛化。语速模型采用与编码模型类似的思路,将脑响应特征映射到预测的语速。束搜索解码模型利用语言模型的先验概率和编码模型的似然概率,通过束搜索生成最可能的语义序列。CLCSD在公开数据集SMN4Lang上取得了0.674的BERTScore,高于其他长序列中文连续语义解码模型。本研究提出一种高效的长序列中文连续语义解码方法,为中文脑机接口技术的发展提供理论基础和方法参考。

    Abstract:

    Language is an important tool for communication and cognition. Multiple functional areas of the brain, connected through complex neural networks, jointly participate in the perception, comprehension, and production of language. Exploring the neural mechanisms of Chinese semantic decoding is crucial for the development of Chinese brain-computer interface (BCI). This study aims to establish a long-sequence continuous semantic decoding method based on fMRI data, termed Chinese long-sequence continuous semantic decoder(CLCSD). Through signal processing workflows and algorithm optimization, it seeks to achieve efficient decoding of continuous Chinese semantics. The CLCSD framework is composed of four components: neural response dimensionality reduction, an encoding model, a word rate model, and a beam search decoding model. Neural response dimensionality reduction is performed through cortical reconstruction, image registration, and brain region parcellation to reduce four-dimensional brain response data into a two-dimensional matrix. The encoding model is constructed using L2-regularized regression (ridge regression) to establish the relationship between stimulus features and brain responses, with noise covariance estimated via bootstrapping to enhance generalization. The word rate model follows a similar approach to the encoding model, where brain response features are mapped to predicted word rate. The beam search decoding model uses the prior probability of the language model and likelihood probabilities of the encoding model to generate the most probable semantic sequence through beam search. On publicly available dataset SMN4Lang, CLCSD achieves a mean BERTScore of 0.674, outperforming other long-sequence Chinese continuous semantic decoding models. The proposed method provides an efficient long-sequence continuous Chinese semantic decoding approach, offering both theoretical foundations and methodological references for the advancement of Chinese BCI technologies.

    参考文献
    相似文献
    引证文献
引用本文

马磊,崔文浩,杨汶汶,王朝欣.非侵入性连续中文语言语义解码与重建[J].数据采集与处理,2025,40(3):616-636

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2024-12-30
  • 最后修改日期:2025-05-08
  • 录用日期:
  • 在线发布日期: 2025-06-13