Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (11A): 625-629.doi: 10.11896/jsjkx.210300114

• Interdiscipline & Application • Previous Articles     Next Articles

Research on Application of Improved GAN Network in Generating Short Video

YU Xiao-ming, HUANG Hua   

  1. College of Electronic Information and Artificial Intelligence,Shaanxi University of Science and Technology,Xi'an 710021,China
  • Online:2021-11-10 Published:2021-11-12
  • About author:YU Xiao-ming,born in 1965,associate professor.Her main research interests include intelligent information proces-sing,graphics and image processing
    HUANG Hua,born in 1995,postgra-duate.His main research interests include graphics and image processing.
  • Supported by:
    Science and Technology Department of Shaanxi Province,China(2014KRM80) and Project of Xianyang Science and Technology Bureau,China(2013K15-07).

PDF (PC)

718

Abstract: In the study of the dynamic image generated by GAN,there are many problems,such as inconsistent colors of some objects and unnatural details of generated images.In order to solve the problem of unsatisfactory video generation,the main schemes adopted are to improve the generator and discriminator of GAN network respectively,which are shown in two aspects.On the one hand,the foreground and background of the videos are modeled separately in the generator and the Multi Spatial-Adaptive Normalization (M-Spade) algorithm is used.The other aspect is the use of dual video discriminator (DVD-GAN) on discriminator selection,which trained on Kinetics 600 dataset.The experimental results are compared with F-VID2VID,WC-VID2VID and other generation methods.The results show that the method of combining the two methods has a great effect on the problem of color inconsistency before and after the short video and the details processing,and the generated images are relatively clearer.

Key words: Dual video discriminator, Foreground-background image modeling, Generative adversarial networks, Multispace adaptive normalization, Synthesis short video

CLC Number: 

  • TP391
[1]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//International Conference on Neural Information Processing Systems.MIT Press,2014:2672-2680.
[2]JIA Y F,MA L.Conditional self-attention generates adversarial networks[J].Journal of Xidian University,2019,46(6):163-170.
[3]BROCK A,DONAHUE J,SIMONYAN K.Large scale GANtraining for high fidelity natural image synthesis[C]//ICLR.2019.
[4]NITISH S,ELMAN M,RUSLAN S.Unsupervised learning of video representations using LSTMs[R].In ICML,2015.
[5]YAMAGUCHI A,CABATUAN M.Generative model basedframe generation of volcanic flow video[C]//IEEE International Conference on Humanoid.IEEE,2017:1-5.
[6]WANG T C,LIU M Y,ZHU J Y,et al.Video-to-Video Synthesis[J].arXiv:1808.06601.2018.
[7]WANG T C,LIU M Y,TAO R,et al.Few-shot Video-to-Video Synthesis[J].arXiv:1910.12713.2019.
[8]MALLYA A,WANG T C,SAPRA K,et al.World-ConsistentVideo-to-Video Synthesis[J].arXiv:2007.08509.2020.
[9]CLARK A,JEFF D,KAREN S.Efficient Video Generation on Complex Datasets[J].arXiv:1907.06571,2019.
[10]AAYUSH B,SHUGAO M,YASER S.Recycle-GAN:Unsupervised video retargeting[C]//ECCV.2018.
[11]ZHOU Y P,WANG Z W,CHEN F,et al.Dance Dance Generation:Motion Transfer for Internet Videos[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).IEEE,2020.
[12]VONDRICK C,PIRSIAVASH H,TORRALBA A.Generatingvideos with scene dynamics[C]//NeurIPS.2016.
[13]TAESUNG P,LIU M Y.Semantic Image Synthesis with Spatially-Adaptive Normalization[J].arXiv:1903.07291.
[14]ZHAO W Z,CHEN X,CHEN J G,et al.Sample Generation with Self-Attention Generative Adversarial Adaptation Network (SaGAAN) for Hyperspectral Image Classification[J].Remote Sensing,2020,12(5):843.
[15]AIDAN C,JEFF D,KAREN S,et al.Efficient Video Generation on Complex Datasets[J].arXiv:1907.06571.2019.
[16]TULYAKOV S,LIU M Y,YANG X D.MoCoGAN:Decomposing otion and content for video generation[C]//CVPR.2018.
[17]LENA G,MOSHE B,ELI S,et al.Actions as space-time shapes[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2008,29(12):2247-2253.
[18]ZHANG H,GOODFELLOW I J,METAXAS D,et al.Self-attention generative adversarial networks[J].arXiv:1805.08318,2018.
[19]ANDREW B,JEFF D,KAREN S.Large scale GAN training for high fidelity natural image synthesis[C]//ICLR.2019.
[20]SALIMANS T,ZAREMBA W,CHEUNG V.Improved tech-niques for training GANs[C]//NeurIPS.2016.
[21]HEUSEL M,RAMSAUER H,UNTERTHINER T,et al.GANs trained by a twotime-scale update rule converge to a local nash equilibrium[C]//NeurIPS.2017.
[22]ZHENG S B,ZHOU G X,ZHANG B H,et.al.A map matching algorithm based on discrete Frechet distance[J].Journal of Hefei University of Technology (Natural Science),2017,40(1):42-46.
[1] XU Guo-ning, CHEN Yi-peng, CHEN Yi-ming, CHEN Jin-yin, WEN Hao. Data Debiasing Method Based on Constrained Optimized Generative Adversarial Networks [J]. Computer Science, 2022, 49(6A): 184-190.
[2] XU Hui, KANG Jin-meng, ZHANG Jia-wan. Digital Mural Inpainting Method Based on Feature Perception [J]. Computer Science, 2022, 49(6): 217-223.
[3] DOU Zhi, WANG Ning, WANG Shi-jie, WANG Zhi-hui, LI Hao-jie. Sketch Colorization Method with Drawing Prior [J]. Computer Science, 2022, 49(4): 195-202.
[4] GAO Zhi-yu, WANG Tian-jing, WANG Yue, SHEN Hang, BAI Guang-wei. Traffic Prediction Method for 5G Network Based on Generative Adversarial Network [J]. Computer Science, 2022, 49(4): 321-328.
[5] LI Si-quan, WAN Yong-jing, JIANG Cui-ling. Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal [J]. Computer Science, 2022, 49(3): 179-184.
[6] TAN Xin-yue, HE Xiao-hai, WANG Zheng-yong, LUO Xiao-dong, QING Lin-bo. Text-to-Image Generation Technology Based on Transformer Cross Attention [J]. Computer Science, 2022, 49(2): 107-115.
[7] ZHANG Wei-qi, TANG Yi-feng, LI Lin-yan, HU Fu-yuan. Image Stream From Paragraph Method Based on Scene Graph [J]. Computer Science, 2022, 49(1): 233-240.
[8] XU Tao, TIAN Chong-yang, LIU Cai-hua. Deep Learning for Abnormal Crowd Behavior Detection:A Review [J]. Computer Science, 2021, 48(9): 125-134.
[9] LIN Zhen-xian, ZHANG Meng-kai, WU Cheng-mao, ZHENG Xing-ning. Face Image Inpainting with Generative Adversarial Network [J]. Computer Science, 2021, 48(9): 174-180.
[10] PAN Xiao-qin, LU Tian-liang, DU Yan-hui, TONG Xin. Overview of Speech Synthesis and Voice Conversion Technology Based on Deep Learning [J]. Computer Science, 2021, 48(8): 200-208.
[11] WANG Jian-ming, LI Xiang-feng, YE Lei, ZUO Dun-wen, ZHANG Li-ping. Medical Image Deblur Using Generative Adversarial Networks with Channel Attention [J]. Computer Science, 2021, 48(6A): 101-106.
[12] YE Hong-liang, ZHU Wan-ning, HONG Lei. Music Style Transfer Method with Human Voice Based on CQT and Mel-spectrum [J]. Computer Science, 2021, 48(6A): 326-330.
[13] HU Yu-jie, CHANG Jian-hui, ZHANG Jian. Image Synthesis with Semantic Region Style Constraint [J]. Computer Science, 2021, 48(2): 134-141.
[14] WEI Li-qi, ZHAO Zhi-hong, BAI Guang-wei, SHEN Hang. Location Privacy Game Mechanism Based on Generative Adversarial Networks [J]. Computer Science, 2021, 48(10): 266-271.
[15] ZHANG Yang, MA Xiao-hu. Anime Character Portrait Generation Algorithm Based on Improved Generative Adversarial Networks [J]. Computer Science, 2021, 48(1): 182-189.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.