计算机科学 ›› 2019, Vol. 46 ›› Issue (1): 100-106.doi: 10.11896/j.issn.1002-137X.2019.01.015

• 2018 年第七届中国数据挖掘会议 • 上一篇    下一篇


徐强, 钟尚平, 陈开志, 张春阳   

  1. (福州大学数学与计算机科学学院 福州350116)
    (福州大学网络系统信息安全福建省高校重点实验室 福州350116)
  • 收稿日期:2018-05-11 出版日期:2019-01-15 发布日期:2019-02-25
  • 作者简介:徐 强(1993-),男,硕士生,主要研究方向为深度学习、图像处理,E-mail:happyxuwork@163.com;钟尚平(1969-),男,教授,硕士生导师,主要研究方向为机器学习、模式识别、大规模核学习,E-mail:spzhong@fzu.edu.cn(通信作者);陈开志(1983-),男,讲师,硕士生导师,主要研究方向为生物特征识别、图像安全;张春阳(1987-),男,副教授,硕士生导师,主要研究方向为深度学习、大数据与云计算。

Optimized Selection Method of Cycle-consistent Loss Coefficient of CycleGAN in Image Generation with Different Texture Complexity

XU Qiang, ZHONG Shang-ping, CHEN Kai-zhi, ZHANG Chun-yang   

  1. (College of Mathematics and Computer Science,Fuzhou University,Fuzhou 350116,China)
    (Network System Information Security Fujian Provincial University Key Laboratory,Fuzhou University,Fuzhou 350116,China)
  • Received:2018-05-11 Online:2019-01-15 Published:2019-02-25

摘要: 高质量的图像生成一直是计算机视觉等领域探索的难点和热点。通过使用循环一致损失,CycleGAN在无监督图像生成任务中取得了良好效果。但是面对不同纹理复杂度的图像生成任务,CycleGAN的循环一致损失系数是默认不变的,使得生成图像存在纹理变形甚至消失等弱点,不能很好地保证生成图像的质量。文中融合图像的空间维度和时间维度来度量图像的纹理复杂性,阐明循环一致损失函数在优化目标函数中的重要性,发现并解释循环一致损失系数的大小与不同纹理复杂度图像生成质量的关联性:纹理复杂度越高,应选择越大的循环一致损失系数;反之,应取越小的循环一致损失系数。文中使用基准和自采集的图像数据集,引入了基于迁移学习的分类准确性等生成图像质量评估指标。实验结果表明,优化选择大小合适的循环一致损失系数,可有效提高生成图像的质量。

关键词: CycleGAN, 图像生成, 纹理复杂度, 循环一致损失, 优化选择系数

Abstract: High-quality image generation has always been a difficult and hot topic in the field of computer vision and other exploration.CycleGAN achieves good results in unsupervised image generation tasks by using cycle-consistent losses.However,in face of image generation tasks with different texture complexity,CycleGAN’s cycle-consistent loss coefficient is unchanged by default,and its generated images have weak points such as texture distortion or even disappear,which can not guarantee the quality of generated images.In this paper,the complexity of image texture was mea-sured by integrating the spatial dimension and time dimension of images,the importance of cycle-consistent loss function in optimizing objective function was clarified,the correlation between the size of the cycle-consistent loss coefficient and the quality of image with different texture complexity was discovered and explained.The higher the texture complexity,the larger the cycle-consistent loss coefficient should be selected.Otherwise,the smaller coefficient should be taken.Using benchmarks and self-acquired image data sets,the classification accuracy based on migration learning was introduced to generate image quality assessment indicators.The experimental results show that the optimal choice of the appropriate cycle-consistent loss factor can effectively improve the quality of generated images.

Key words: Cycle-consistent loss, CycleGAN, Image generation, Optimization of selection coefficient, Texture complexity


  • TP183
[1]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:3431-3440.<br /> [2]GATYS L,ECKER A S,BETHGE M.Texture synthesis using convolutional neural networks[C]//Advances in Neural Information Processing Systems.Cambridge,Massachusetts:MIT Press,2015:262-270.<br /> [3]GATYS L A,ECKER A S,BETHGE M.Image style transfer using convolutional neural networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,NJ:IEEE,2016:2414-2423.<br /> [4]JOHNSON J,ALAHI A,FEI-FEI L.Perceptual losses for real-time style transfer and super-resolution[C]//European Con-ference on Computer Vision.Berlin,German:Springer,2016:694-711.<br /> [5]NASH J.Non-Cooperative Games.Annals of Mathematics,1951,54(2):286-295.<br /> [6]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//International Conference on Neural Information Processing Systems.Cambridge,Massachusetts:MIT Press,2014:2672-2680.<br /> [7]DENTON E L,CHINTALA S,FERGUS R.Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks[C]//Advances in Neural Information Processing Systems.Cambridge,Massachusetts:MIT Press,2015:1486-1494.<br /> [8]LI C,WAND M.Precomputed real-time texture synthesis with markovian generative adversarial networks[C]//European Conference on Computer Vision.Berlin,German:Springer,2016:702-716.<br /> [9]LI C,ZHAO X Y,XIAO L M,et al.Multi-layer perceptual defogging algorithm for image under generative adversarial networks[J].Journal of Computer-Aided Design & Computer Graphics,2017,29(10):1835-1843.(in Chinese)<br /> 李策,赵新宇,肖利梅,等.生成对抗映射网络下的图像多层感知去雾算法[J].计算机辅助设计与图形学学报,2017,29(10):1835-1843.<br /> [10]LEDIG C,WANG Z,SHI W,et al.Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network//Computer Vision and Pattern Recognition.IEEE,2017:105-114.<br /> [11]MIRZA M,OSINDERO S.Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784,2014.<br /> [12]ZHU J Y,KRÄHENBÜHL P,SHECHTMAN E,et al.Generative visual manipulation on the natural image manifold[C]//European Conference on Computer Vision.Berlin,German:Sprin-ger,2016:597-613.<br /> [13]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-Image Translation with Conditional Adversarial Networks[C]//IEEE Con-ference on Computer Vision and Pattern Recognition.Pisca-taway,NJ:IEEE,2017:5967-5976.<br /> [14]RADFORD A,METZ L,CHINTALA S.Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[J].arXiv:1511.06434.2016.<br /> [15]FUKUSHIMA K.Neural network model for a mechanism of pattern recognition unaffected by shift in position-Neocognitron[J].IEICE Technical Report A,1979,62(10):658-665.<br /> [16]YI Z,ZHANG H,TAN P,et al.DualGAN:Unsupervised Dual Learning for Image-to-Image Translation[C]//IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE,2017:2868-2876.<br /> [17]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[J].arXiv preprint arXiv:1701.07875,2017.<br /> [18]GOLDSTEIN T,OSHER S.The split Bregman method for L1-regularized problems[J].SIAM Journal on Imaging Sciences,2009,2(2):323-343.<br /> [19]KIM T,CHA M,KIM H,et al.Learning to Discover Cross-Domain Relations with Generative Adversarial Networks[C]//Proceedings of the 34th International Conference on Machine Learning.New York:ACM,2017:1857-1865.<br /> [20]ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks[C]//IEEE International Conference on Computer Vision.Piscata-way,NJ:IEEE,2017:2242-2251.<br /> [21]HE D,XIA Y,QIN T,et al.Dual learning for machine translation[C]//Advances in Neural Information Processing Systems.Cambridge,Massachusetts:MIT Press,2016:820-828.<br /> [22]ZHOU T,KRAHENBUHL P,AUBRY M,et al.Learning dense correspondence via 3d-guided cycle consistency[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2016:117-126.<br /> [23]CARDACI M,DI GESÙ V,PETROU M,et al.A fuzzy approach to the evaluation of image complexity[J].Fuzzy Sets and Systems,2009,160(10):1474-1484.<br /> [24]RUSSAKOVSKY O,DENG J,SU H,et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.<br /> [25]SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2016:2818-2826.<br /> [26]WANG X,GUPTA A.Generative image modeling using style and structure adversarial networks[C]//European Conference on Computer Vision.Berlin,German:Springer,2016:318-335.<br /> [27]MAO X,LI Q,XIE H,et al.Least squares generative adversarial networks//2017 IEEE International Conference on ComputerVision (ICCV).IEEE,2017:2813-2821.<br /> [28]SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems.Berlin,German:MIT Press,2017:3859-3869.<br /> [29]LEEUWENBERG E,BUFFART H.An outline of coding theory[J].Advances in psychology,1983,11:25-47.<br /> [30]SU H,BOURIDANE A,CROOKES D.Scale Adaptive Complexity Measure of 2D Shapes//International Conference on Pattern Recognition.Piscataway,NJ:IEEE,2006:134-137.<br /> [31]PAN S J,YANG Q.A survey on transfer learning[J].IEEE Transactions on knowledge and data engineering,2010,22(10):1345-1359.<br /> [32]ZHOU B,LAPEDRIZA A,XIAO J,et al.Learning deep features for scene recognition using places database[C]//Advances in neural information processing systems.Cambridge,Massachusetts:MIT Press,2014:487-495.<br /> [33]GARCIA B, BRUNET P.3D reconstruction with projective octrees and epipolar geometry//International Conference on Computer Vision.Piscataway,NJ: IEEE,2008:1067-1072.
[1] 石达, 芦天亮, 杜彦辉, 张建岭, 暴雨轩.
Generation Model of Gender-forged Face Image Based on Improved CycleGAN
计算机科学, 2022, 49(2): 31-39. https://doi.org/10.11896/jsjkx.210600012
[2] 谈馨悦, 何小海, 王正勇, 罗晓东, 卿粼波.
Text-to-Image Generation Technology Based on Transformer Cross Attention
计算机科学, 2022, 49(2): 107-115. https://doi.org/10.11896/jsjkx.210600085
[3] 崔雯昊, 蒋慕蓉, 杨磊, 傅鹏铭, 朱凌霄.
Combining MCycleGAN and RFCNN to Realize High Resolution Reconstruction of Solar Speckle Image
计算机科学, 2021, 48(6A): 38-42. https://doi.org/10.11896/jsjkx.201000160
[4] 胡妤婕, 常建慧, 张健.
Image Synthesis with Semantic Region Style Constraint
计算机科学, 2021, 48(2): 134-141. https://doi.org/10.11896/jsjkx.200800201
[5] 张扬, 马小虎.
Anime Character Portrait Generation Algorithm Based on Improved Generative Adversarial Networks
计算机科学, 2021, 48(1): 182-189. https://doi.org/10.11896/jsjkx.191100092
[6] 叶亚男, 迟静, 于志平, 战玉丽, 张彩明.
Expression Animation Synthesis Based on Improved CycleGan Model and Region Segmentation
计算机科学, 2020, 47(9): 142-149. https://doi.org/10.11896/jsjkx.190900203
[7] 周兵, 刘玉霞, 杨欣欣, 刘扬.
Review of Research on Image Complexity
计算机科学, 2018, 45(9): 30-37. https://doi.org/10.11896/j.issn.1002-137X.2018.09.004
[8] 李清,李东晖.
Lossless Color Image Compression Method Based on Fuzzy Logic
计算机科学, 2014, 41(Z11): 103-106.
[9] 张芹 吴慧中 张健.

计算机科学, 2003, 30(8): 144-146.
[10] 王建华 解凯.

计算机科学, 2002, 29(12): 160-161.
Full text



No Suggested Reading articles found!