基于感知推理和外部空间先验特征的图像修复
DOI:
作者:
作者单位:

上海理工大学 光电信息与计算机工程学院上海 200093

作者简介:

通讯作者:

基金项目:

上海市晨光学者基金,上海市自然科学基金


Image inpainting based on perceptual inference and external spatial prior features
Author:
Affiliation:

School of Optical-Electrical and Computer Engineering,University of Shanghai for Science and Technology,Shanghai,200093. P. R. China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    近些年来,基于深度学习的图像修复算法研究取得了很多令人瞩目的进展。然而当存在大面积掩码时,由于缺乏合理的先验信息指导,修复结果往往会出现伪影和模糊纹理等现象。针对此问题,我们提出将先验特征与图像预测滤波相结合的图像修复算法。它包含两个分支:图像滤波核预测分支、特征推理与图像滤波分支。从图像滤波核预测分支的解码器部分提取特征,利用多尺度外部空间特征融合对掩码区域特征进行重建,并传递给另一分支的解码阶段作为先验特征,为图像修复提供更为丰富的语义信息。然后,在特征推理和图像滤波分支部分引入空间特征感知推理块,它能够过滤掉分散注意力的特征,同时捕捉信息丰富的远距离图像上下文进行推理。最后阶段,使用图像预测滤波核进行过滤消除伪影。在CelebA和Places2数据集上与其他修复网络进行对比实验,证明了该方法在修复质量上的优越性。

    Abstract:

    In recent years, the research of image inpainting based on deep learning has made a lot of remarkable progress. However, when there is a large area mask, due to the lack of reasonable prior information guidance, the repair results often appear artifacts and blurred textures. Therefore, based on this problem, we propose an image inpainting algorithm that combines prior features with image predictive filtering. It consists of two branches: image filtering kernel prediction branch, feature inference and image filtering branch. The features are extracted from the decoder part of the image filter kernel prediction branch. The multi-scale external spatial feature fusion is used to reconstruct the mask region features, and the decoding stage is passed to another branch as a prior feature to provide richer semantic information for image inpainting. Then, a spatial feature-aware inference block is introduced in the feature inference and image filtering branches, which can filter out the distracting features and capture the informative long-distance image context for inference. In the final stage, the image prediction filter kernel is used to filter and eliminate artifacts. Compared with other repair networks on CelebA and Places2 datasets, the superiority of this method in repair quality is proved.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2023-04-25
  • 最后修改日期:2023-06-29
  • 录用日期:2023-10-11
  • 在线发布日期: