基于融合语义信息的上下文感知图像修复
作者:
作者单位:

上海理工大学光电信息与计算机工程学院,上海 200093

作者简介:

通讯作者:

基金项目:

国家自然科学基金(61603255);上海市晨光计划项目(18CG52)。


Context-Aware Image Restoration Based on Fused Semantic Information
Author:
Affiliation:

School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    近年来,生成对抗网络广泛应用于图像修复领域并取得了不错的效果。但目前的方法并没有考虑在高分辨率图像(512×512)中会产生模糊的结构以及纹理的问题,这些问题主要来源于缺乏有效特征信息。针对此问题,提出一种将图像特征与语义信息相结合的生成对抗网络。主要基于语义信息,提出一种上下文感知的图像修复模型,该模型自适应地将语义信息与图像特征融合,并且提出自适应卷积替代传统卷积,以及在解码器后增添一个多尺度上下文聚合模块捕捉远距离信息来进行上下文推理。在Places2、CelebA-HQ、Paris Street View和Openlogo数据集上进行实验,实验结果表明,在L1损失、峰值信噪比(PSNR)和结构相似度(SSIM)上所提方法与现有方法对比均有所提升。

    Abstract:

    In recent years, generative adversarial networks have been widely used in the field of image restoration and have achieved good results. However, current methods do not consider problems of blurred structures and textures in high-resolution images (512×512), which mainly come from the lack of effective feature information. To address this problem, this paper proposes a generative adversarial network that combines image features with semantic information. Based mainly on semantic information, a context-aware image restoration model is proposed, which adaptively fuses semantic information with image features, and adaptive convolution is proposed to replace the traditional convolution, as well as a multi-scale context aggregation module is added after the decoder to capture long-distance information for contextual inference. Experiments are conducted on Places2, CelebA-HQ, Paris Street View, and Openlogo datasets, whose results show that the proposed method improves in terms of L1 loss, peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) in comparison with the existing methods.

    参考文献
    相似文献
    引证文献
引用本文

祖奕,张孙杰,吴鹏,马悦恒.基于融合语义信息的上下文感知图像修复[J].数据采集与处理,2025,40(2):401-416

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2024-05-11
  • 最后修改日期:2024-08-01
  • 录用日期:
  • 在线发布日期: 2025-04-11