The optical flow information is the motion representation of the image pixels. The existing optical flow estimation methods are difficult to ensure high precision in dealing with complex situations, such as occlusion, large displacement and detailed presentation. In order to overcome these difficult problems, a new convolutional neural network is proposed. The model improves the estimation accuracy by improving the convolution form and feature fusion. Firstly, the deformable convolution with stronger adjustment and optimization ability is added to extract the spatial features such as large displacement and details of adjacent frame images. Then, the feature correlation layer is generated by using the attention-based mechanism to carry out the feature fusion of the two adjacent frames, which is used as the input of the decoding part composed of deconvolution and upsampling and aims to overcome the disadvantage of low accuracy for the traditional methods of estimating optical flow based on feature matching. And finally the above estimated optical flow is optimized with a set of network stack. Experiments show that the proposed network model performs better than existing methods in dealing with occlusion, large displacement and detail presentation.
表 2 不同方法在Mpi Sintel的表现Table 2 Performance of different methods on Mpi Sintel dataset
表 1 不同的深度学习网络在Flying Chairs上的表现Table 1 Performance of different deep learning networks on Flying Chairs dataset
图1 DANet-S结构Fig.1 DANet-S structure
图2 卷积结构图Fig.2 Convolution structure
图3 可形变卷积操作Fig.3 Deformable convolution operation
图4 可形变池化操作Fig.4 Deformable pooling operation
图5 关联层操作Fig.5 Association layer operation
图6 DANet-C结构Fig.6 DANet-C structure diagram
图7 图像光流估计结果对比Fig.7 Comparison of image optical flow estimation results
图8 光流图像局部放大对比Fig.8 Magnification contrast of optical flow image local