不同纹理复杂度图像生成中CycleGAN循环一致损失系数优化选择方法

doi:10.11896／j.issn.1002-137X.2019.01.015

摘要/Abstract

摘要： 高质量的图像生成一直是计算机视觉等领域探索的难点和热点。通过使用循环一致损失,CycleGAN在无监督图像生成任务中取得了良好效果。但是面对不同纹理复杂度的图像生成任务,CycleGAN的循环一致损失系数是默认不变的,使得生成图像存在纹理变形甚至消失等弱点,不能很好地保证生成图像的质量。文中融合图像的空间维度和时间维度来度量图像的纹理复杂性,阐明循环一致损失函数在优化目标函数中的重要性,发现并解释循环一致损失系数的大小与不同纹理复杂度图像生成质量的关联性:纹理复杂度越高,应选择越大的循环一致损失系数;反之,应取越小的循环一致损失系数。文中使用基准和自采集的图像数据集,引入了基于迁移学习的分类准确性等生成图像质量评估指标。实验结果表明,优化选择大小合适的循环一致损失系数,可有效提高生成图像的质量。

关键词: CycleGAN, 图像生成, 纹理复杂度, 循环一致损失, 优化选择系数

Abstract: High-quality image generation has always been a difficult and hot topic in the field of computer vision and other exploration.CycleGAN achieves good results in unsupervised image generation tasks by using cycle-consistent losses.However,in face of image generation tasks with different texture complexity,CycleGAN’s cycle-consistent loss coefficient is unchanged by default,and its generated images have weak points such as texture distortion or even disappear,which can not guarantee the quality of generated images.In this paper,the complexity of image texture was mea-sured by integrating the spatial dimension and time dimension of images,the importance of cycle-consistent loss function in optimizing objective function was clarified,the correlation between the size of the cycle-consistent loss coefficient and the quality of image with different texture complexity was discovered and explained.The higher the texture complexity,the larger the cycle-consistent loss coefficient should be selected.Otherwise,the smaller coefficient should be taken.Using benchmarks and self-acquired image data sets,the classification accuracy based on migration learning was introduced to generate image quality assessment indicators.The experimental results show that the optimal choice of the appropriate cycle-consistent loss factor can effectively improve the quality of generated images.

Key words: Cycle-consistent loss, CycleGAN, Image generation, Optimization of selection coefficient, Texture complexity

中图分类号:

TP183

徐强, 钟尚平, 陈开志, 张春阳. 不同纹理复杂度图像生成中CycleGAN循环一致损失系数优化选择方法[J]. 计算机科学, 2019, 46(1): 100-106. https://doi.org/10.11896／j.issn.1002-137X.2019.01.015

XU Qiang, ZHONG Shang-ping, CHEN Kai-zhi, ZHANG Chun-yang. Optimized Selection Method of Cycle-consistent Loss Coefficient of CycleGAN in Image Generation with Different Texture Complexity[J]. Computer Science, 2019, 46(1): 100-106. https://doi.org/10.11896／j.issn.1002-137X.2019.01.015

参考文献

[1]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:3431-3440. [2]GATYS L,ECKER A S,BETHGE M.Texture synthesis using convolutional neural networks[C]//Advances in Neural Information Processing Systems.Cambridge,Massachusetts:MIT Press,2015:262-270. [3]GATYS L A,ECKER A S,BETHGE M.Image style transfer using convolutional neural networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,NJ:IEEE,2016:2414-2423. [4]JOHNSON J,ALAHI A,FEI-FEI L.Perceptual losses for real-time style transfer and super-resolution[C]//European Con-ference on Computer Vision.Berlin,German:Springer,2016:694-711. [5]NASH J.Non-Cooperative Games.Annals of Mathematics,1951,54(2):286-295. [6]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//International Conference on Neural Information Processing Systems.Cambridge,Massachusetts:MIT Press,2014:2672-2680. [7]DENTON E L,CHINTALA S,FERGUS R.Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks[C]//Advances in Neural Information Processing Systems.Cambridge,Massachusetts:MIT Press,2015:1486-1494. [8]LI C,WAND M.Precomputed real-time texture synthesis with markovian generative adversarial networks[C]//European Conference on Computer Vision.Berlin,German:Springer,2016:702-716. [9]LI C,ZHAO X Y,XIAO L M,et al.Multi-layer perceptual defogging algorithm for image under generative adversarial networks[J].Journal of Computer-Aided Design & Computer Graphics,2017,29(10):1835-1843.(in Chinese) 李策,赵新宇,肖利梅,等.生成对抗映射网络下的图像多层感知去雾算法[J].计算机辅助设计与图形学学报,2017,29(10):1835-1843. [10]LEDIG C,WANG Z,SHI W,et al.Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network//Computer Vision and Pattern Recognition.IEEE,2017:105-114. [11]MIRZA M,OSINDERO S.Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784,2014. [12]ZHU J Y,KRÄHENBÜHL P,SHECHTMAN E,et al.Generative visual manipulation on the natural image manifold[C]//European Conference on Computer Vision.Berlin,German:Sprin-ger,2016:597-613. [13]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-Image Translation with Conditional Adversarial Networks[C]//IEEE Con-ference on Computer Vision and Pattern Recognition.Pisca-taway,NJ:IEEE,2017:5967-5976. [14]RADFORD A,METZ L,CHINTALA S.Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[J].arXiv:1511.06434.2016. [15]FUKUSHIMA K.Neural network model for a mechanism of pattern recognition unaffected by shift in position-Neocognitron[J].IEICE Technical Report A,1979,62(10):658-665. [16]YI Z,ZHANG H,TAN P,et al.DualGAN:Unsupervised Dual Learning for Image-to-Image Translation[C]//IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE,2017:2868-2876. [17]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[J].arXiv preprint arXiv:1701.07875,2017. [18]GOLDSTEIN T,OSHER S.The split Bregman method for L1-regularized problems[J].SIAM Journal on Imaging Sciences,2009,2(2):323-343. [19]KIM T,CHA M,KIM H,et al.Learning to Discover Cross-Domain Relations with Generative Adversarial Networks[C]//Proceedings of the 34th International Conference on Machine Learning.New York:ACM,2017:1857-1865. [20]ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks[C]//IEEE International Conference on Computer Vision.Piscata-way,NJ:IEEE,2017:2242-2251. [21]HE D,XIA Y,QIN T,et al.Dual learning for machine translation[C]//Advances in Neural Information Processing Systems.Cambridge,Massachusetts:MIT Press,2016:820-828. [22]ZHOU T,KRAHENBUHL P,AUBRY M,et al.Learning dense correspondence via 3d-guided cycle consistency[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2016:117-126. [23]CARDACI M,DI GESÙ V,PETROU M,et al.A fuzzy approach to the evaluation of image complexity[J].Fuzzy Sets and Systems,2009,160(10):1474-1484. [24]RUSSAKOVSKY O,DENG J,SU H,et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252. [25]SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2016:2818-2826. [26]WANG X,GUPTA A.Generative image modeling using style and structure adversarial networks[C]//European Conference on Computer Vision.Berlin,German:Springer,2016:318-335. [27]MAO X,LI Q,XIE H,et al.Least squares generative adversarial networks//2017 IEEE International Conference on ComputerVision (ICCV).IEEE,2017:2813-2821. [28]SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems.Berlin,German:MIT Press,2017:3859-3869. [29]LEEUWENBERG E,BUFFART H.An outline of coding theory[J].Advances in psychology,1983,11:25-47. [30]SU H,BOURIDANE A,CROOKES D.Scale Adaptive Complexity Measure of 2D Shapes//International Conference on Pattern Recognition.Piscataway,NJ:IEEE,2006:134-137. [31]PAN S J,YANG Q.A survey on transfer learning[J].IEEE Transactions on knowledge and data engineering,2010,22(10):1345-1359. [32]ZHOU B,LAPEDRIZA A,XIAO J,et al.Learning deep features for scene recognition using places database[C]//Advances in neural information processing systems.Cambridge,Massachusetts:MIT Press,2014:487-495. [33]GARCIA B, BRUNET P.3D reconstruction with projective octrees and epipolar geometry//International Conference on Computer Vision.Piscataway,NJ: IEEE,2008:1067-1072.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed