A Survey on Risks and Governance of Content Generated by Visual Generation Models
CSTR:
Author:
Affiliation:

1School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China;2School of New Media and Communication, Tianjin University, Tianjin 300072, China

Clc Number:

TP3

Fund Project:

National Natural Science Foundation of China (Nos.62425307, 62572346, U21B2024).

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    With breakthroughs in deep generative technologies such as diffusion models, visual generation models have achieved significant leaps in generation quality and semantic consistency, finding extensive applications in fields like artistic creation and industrial design. However, the powerful generative capability has also triggered severe content safety risks. Malicious users can induce models to generate pornographic, violent, or copyright-infringing images, posing an urgent need for the safety governance of generative AI. This paper provides a systematic review that focuses on two core adversarial tasks of T2I models: (1) Jailbreak attacks, which aim to induce models to breach safety guardrails; (2) Concept erasure, which aims to eliminate internal risk knowledge from the models. First, we establish a taxonomy of jailbreak attacks. By analyzing them across four dimensions: Technical category, perturbation strategy, query type, and adversary knowledge, we reveal the evolutionary trend of attack methods shifting from feature-space perturbations to semantic-space reasoning. Second, regarding risk governance, this paper delves into concept erasure technologies, comparatively analyzing three mainstream technical routes: Model fine-tuning, model editing, and inference guidance. We elucidate the trade-offs among erasure effectiveness, computational efficiency, and the preservation of general generation capabilities. Finally, we summarize the commonly used benchmark datasets in this field and identify the current challenges and future directions regarding adversarial robustness and multi-concept joint governance, aiming to provide theoretical references and technical guidance for building safe and controllable T2I systems.

    Reference
    Related
    Cited by
Get Citation

LIU An’an, ZHANG Chenyu, WANG Lanjun, LI Wenhui. A Survey on Risks and Governance of Content Generated by Visual Generation Models[J]. Journal of Data Acquisition and Processing,2026,(2):620-640.

Copy
Related Videos

Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:January 10,2026
  • Revised:February 25,2026
  • Adopted:
  • Online: April 15,2026
  • Published:
Article QR Code