With the rapid development of the animation industry, the face generation with animation characters becomes a key technology. The existing style transfer technology of painting style cannot obtain satisfactory animation results due to the following characteristics of animation: (1) Animation has a highly simplified and abstract unique style, and (2) animation tends to have clear edges and smooth shadows and relatively simple textures, which poses great challenges to the loss function in existing methods. This paper proposes a novel loss function suitable for animation. In the loss function, the semantic loss is expressed as a regularized form in the high-level feature map of the VGG network to deal with the different styles between real and animation images, and the edge sharpness loss with edge enhancement can preserve the edge sharpness of animation images. Experiments on the four public data sets show that through the proposed loss function, clear and vivid animation images can be generated. Moreover, in the CK+ data set, the recognition rate of the proposed method is increased by 0.43% (Miyazaki Hayao style) and 3.29% (Makoto Shinkai style) compared with the existing method, increased by 0.85% (Miyazaki Hayao style) and 2.42% (Makoto Shinkai style) in the RAF data set, and increased by 0.71% (Miyazaki Hayao style) and 3.14% (Makoto Shinkai style) in the SFEW data set, respectively. The generation effect in the Celeba data set is also demonstrated. The above results show that the proposed method combines the advantages of the deep learning model to make the detection result more accurate.
表 1 CK+数据集上动漫图像不同表情的识别结果Table 1 Expression recognition results of anime images with different expressions on CK+ dataset
表 2 RAF数据集上动漫图像不同表情的识别结果Table 2 Expression recognition results of anime images with different expressions on RAF dataset
图1 生成对抗网络框架Fig.1 Generative adversarial net framework
图3 有无边缘清晰损失的生成动漫图像(宫崎骏风格)Fig.3 Generate animation images with and without loss of edge sharpness (Miyazaki Hayao style)
图4 有无边缘清晰损失的生成动漫图像(新海诚风格)Fig.4 Generate animation images with and without loss of sharp edges (Makoto Shinkai style)
图5 无语义损失生成的图像Fig.5 Generated images without semantic loss
图6 本文模型(新海诚风格)在CK+数据集中的表情识别结果混淆矩阵Fig.6 Confusion matrix of facial expression recognition results of CK+ dataset by the proposed model (Makoto Shinkai style)
图7 本文模型(新海诚风格)在RAF数据集中的表情识别结果混淆矩阵Fig.7 Confusion matrix of facial expression recognition results of RAF dataset by the proposed model (Makoto Shinkai style)
图8 本文模型(新海诚风格)在SFEW数据集中的表情识别结果混淆矩阵Fig.8 Confusion matrix of facial expression recognition results of SFEW dataset by the proposed model (Makoto Shinkai style)
图9 风格图像与真实动漫域图像Fig.9 Style images and real anime domain images
图10 Celeba数据集中生成的老人图像效果Fig.10 Generated effect of elderly images in the Celeba dataset
图11 Celeba数据集中生成的成年人图像效果Fig.11 Generated effect of adult images in the Celeba dataset
图12 RAF数据集中生成的小孩图像效果Fig.12 Generated effect of child images in the RAF dataset
表 3 SFEW数据集上动漫图像不同表情的识别结果Table 3 Expression recognition results of anime images with different expressions on SFEW dataset