一种基于特征融合的声音事件检测方法
DOI:
作者:
作者单位:

上海海事大学信息工程学院

作者简介:

通讯作者:

基金项目:

国家自然科学基金


A Sound Event Detection Method Based On Feature Fusion
Author:
Affiliation:

College of Information Engineering, Shanghai Maritime University

Fund Project:

The National Natural Science Foundation of China

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    现有的基于深度学习的声音事件检测方法多使用传统的二维卷积,然而其平移不变性的特点并不适用于声音信号,这使得模型难以检测复杂的声音事件。为此,提出一种基于特征融合的混合卷积神经网络模型,通过计算频谱图的分布来自适应地生成卷积核,动态地提取与声音信号保持物理一致性的局部特征;同时并行地使用自注意力算法提取全局特征,捕获频谱图的长距离特征联系;为消除局部特征与全局特征的语义差异从而将两种不同的特征表示有效结合,提出一种特征融合模块。此外,为进一步提升模型对声音事件的检测性能,提出一种基于多尺度注意力机制的双向门控单元,对融合后的特征信息进行充分整合,突出事件帧并抑制背景帧。在DCASE2020数据集上的实验结果表明,本方法的F1分数达到52.57%并优于现有的其他方法。

    Abstract:

    Most of the existing sound event detection methods based on deep learning utilize normal 2D convolution, while the characteristics of its translation invariance do not apply to audio clips, which makes it difficult to detect complex sound events. To address the issue, a hybrid convolutional neural network based on feature fusion is proposed. By calculating the distribution of audio spectrum and generating adaptive convolutional kernels, the proposed model extracts feature maps with local details and physical consistency to audio spectrum dynamically. Moreover, the proposed model captures long-distance feature relationships by applying self-attention mechanism in parallel. In order to fill the semantic gap between local details together with global relationships, a feature fusion module is proposed to concatenate them validly. Besides, to improve the detection performance of neural network, an enhanced bidirectional gated recurrent unit based on multi-resolution attention module is proposed to refine the fused feature representations. It emphasizes the frames where sound events tend to be active and suppress those frames tend to be background. The experiment results on the DCASE2020 dataset indicate that the proposed model has achieved an F1-score of 52.57%, which outperforms other existing methods.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2024-12-25
  • 最后修改日期:2025-04-01
  • 录用日期:2025-04-03
  • 在线发布日期: