基于卷积神经网络的光流估计模型
作者:
作者单位:

青岛科技大学信息科学技术学院,青岛 266100

作者简介:

通讯作者:

基金项目:

国家自然科学基金(61672305)资助项目;国家青年科学基金(61702295)资助项目。


Optical Flow Estimation Model Based on Convolutional Neural Network
Author:
Affiliation:

School of Information Science and Technology, Qingdao University of Science and Technology, Qingdao 266100, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    光流信息是图像像素的运动表示,现有光流估计方法在应对图像遮挡、大位移和细节呈现等复杂情况时难以保证高精度。为了克服这些难点问题,本文建立一种新型的卷积神经网络模型,通过改进卷积形式和特征融合的方式来提高估计精度。首先,加入调整优化能力更强的可形变卷积,以便于提取相邻帧图像的大位移和细节等空间特征;然后利用基于注意力机制生成特征关联层,将相邻两帧的特征进行融合,以其作为由反卷积和上采样构成的解码部分的输入,旨在克服基于特征匹配等估计光流传统方法精度低的缺点;最后将得到的估计光流通过多网络堆栈的循环优化模型实现最终的光流估计。实验表明,本文网络模型在处理遮挡、大位移和细节呈现等方面的表现优于现有方法。

    Abstract:

    The optical flow information is the motion representation of the image pixels. The existing optical flow estimation methods are difficult to ensure high precision in dealing with complex situations, such as occlusion, large displacement and detailed presentation. In order to overcome these difficult problems, a new convolutional neural network is proposed. The model improves the estimation accuracy by improving the convolution form and feature fusion. Firstly, the deformable convolution with stronger adjustment and optimization ability is added to extract the spatial features such as large displacement and details of adjacent frame images. Then, the feature correlation layer is generated by using the attention-based mechanism to carry out the feature fusion of the two adjacent frames, which is used as the input of the decoding part composed of deconvolution and upsampling and aims to overcome the disadvantage of low accuracy for the traditional methods of estimating optical flow based on feature matching. And finally the above estimated optical flow is optimized with a set of network stack. Experiments show that the proposed network model performs better than existing methods in dealing with occlusion, large displacement and detail presentation.

    表 2 不同方法在Mpi Sintel的表现Table 2 Performance of different methods on Mpi Sintel dataset
    表 1 不同的深度学习网络在Flying Chairs上的表现Table 1 Performance of different deep learning networks on Flying Chairs dataset
    图1 DANet-S结构Fig.1 DANet-S structure
    图2 卷积结构图Fig.2 Convolution structure
    图3 可形变卷积操作Fig.3 Deformable convolution operation
    图4 可形变池化操作Fig.4 Deformable pooling operation
    图5 关联层操作Fig.5 Association layer operation
    图6 DANet-C结构Fig.6 DANet-C structure diagram
    图7 图像光流估计结果对比Fig.7 Comparison of image optical flow estimation results
    图8 光流图像局部放大对比Fig.8 Magnification contrast of optical flow image local
    参考文献
    相似文献
    引证文献
引用本文

丰艳,刘帅,王传旭.基于卷积神经网络的光流估计模型[J].数据采集与处理,2021,36(1):63-75

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2019-08-08
  • 最后修改日期:2020-03-21
  • 录用日期:
  • 在线发布日期: 2021-01-25