计算机科学 ›› 2021, Vol. 48 ›› Issue (11A): 625-629.doi: 10.11896/jsjkx.210300114
于晓明, 黄铧
YU Xiao-ming, HUANG Hua
摘要: 在研究生成对抗网络(GAN)生成动态图像时,经常出现前后帧图像内容中的部分物体颜色不一致和生成的细节不自然等问题。针对当前生成视频的不理想问题,采用的主要方案是分别对GAN网络中的生成器和判别器进行改进,具体表现在两个方面:一方面是在生成器中对视频的前景和背景分别建模,并且使用多重空间自适应归一化(Multi Spatially-Adaptive Normalization,M-SPADE)算法;另一方面是在判别器的选取上使用双视频判别器(DVD-GAN),然后在Kinetics-600数据集进行训练,实验后的结果分别比对F-Vid2Vid,WC-Vid2Vid等生成方法。实验结果证明了对GAN网络改进的方法在处理生成短视频的前后帧颜色不一致的问题和细节上有着不错的效果,生成的图像相对的更加清晰。
中图分类号:
[1]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//International Conference on Neural Information Processing Systems.MIT Press,2014:2672-2680. [2]JIA Y F,MA L.Conditional self-attention generates adversarial networks[J].Journal of Xidian University,2019,46(6):163-170. [3]BROCK A,DONAHUE J,SIMONYAN K.Large scale GANtraining for high fidelity natural image synthesis[C]//ICLR.2019. [4]NITISH S,ELMAN M,RUSLAN S.Unsupervised learning of video representations using LSTMs[R].In ICML,2015. [5]YAMAGUCHI A,CABATUAN M.Generative model basedframe generation of volcanic flow video[C]//IEEE International Conference on Humanoid.IEEE,2017:1-5. [6]WANG T C,LIU M Y,ZHU J Y,et al.Video-to-Video Synthesis[J].arXiv:1808.06601.2018. [7]WANG T C,LIU M Y,TAO R,et al.Few-shot Video-to-Video Synthesis[J].arXiv:1910.12713.2019. [8]MALLYA A,WANG T C,SAPRA K,et al.World-ConsistentVideo-to-Video Synthesis[J].arXiv:2007.08509.2020. [9]CLARK A,JEFF D,KAREN S.Efficient Video Generation on Complex Datasets[J].arXiv:1907.06571,2019. [10]AAYUSH B,SHUGAO M,YASER S.Recycle-GAN:Unsupervised video retargeting[C]//ECCV.2018. [11]ZHOU Y P,WANG Z W,CHEN F,et al.Dance Dance Generation:Motion Transfer for Internet Videos[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).IEEE,2020. [12]VONDRICK C,PIRSIAVASH H,TORRALBA A.Generatingvideos with scene dynamics[C]//NeurIPS.2016. [13]TAESUNG P,LIU M Y.Semantic Image Synthesis with Spatially-Adaptive Normalization[J].arXiv:1903.07291. [14]ZHAO W Z,CHEN X,CHEN J G,et al.Sample Generation with Self-Attention Generative Adversarial Adaptation Network (SaGAAN) for Hyperspectral Image Classification[J].Remote Sensing,2020,12(5):843. [15]AIDAN C,JEFF D,KAREN S,et al.Efficient Video Generation on Complex Datasets[J].arXiv:1907.06571.2019. [16]TULYAKOV S,LIU M Y,YANG X D.MoCoGAN:Decomposing otion and content for video generation[C]//CVPR.2018. [17]LENA G,MOSHE B,ELI S,et al.Actions as space-time shapes[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2008,29(12):2247-2253. [18]ZHANG H,GOODFELLOW I J,METAXAS D,et al.Self-attention generative adversarial networks[J].arXiv:1805.08318,2018. [19]ANDREW B,JEFF D,KAREN S.Large scale GAN training for high fidelity natural image synthesis[C]//ICLR.2019. [20]SALIMANS T,ZAREMBA W,CHEUNG V.Improved tech-niques for training GANs[C]//NeurIPS.2016. [21]HEUSEL M,RAMSAUER H,UNTERTHINER T,et al.GANs trained by a twotime-scale update rule converge to a local nash equilibrium[C]//NeurIPS.2017. [22]ZHENG S B,ZHOU G X,ZHANG B H,et.al.A map matching algorithm based on discrete Frechet distance[J].Journal of Hefei University of Technology (Natural Science),2017,40(1):42-46. |
[1] | 张佳, 董守斌. 基于评论方面级用户偏好迁移的跨领域推荐算法 Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer 计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131 |
[2] | 孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061 |
[3] | 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105 |
[4] | 尹文兵, 高戈, 曾邦, 王霄, 陈怡. 基于时频域生成对抗网络的语音增强算法 Speech Enhancement Based on Time-Frequency Domain GAN 计算机科学, 2022, 49(6): 187-192. https://doi.org/10.11896/jsjkx.210500114 |
[5] | 徐辉, 康金梦, 张加万. 基于特征感知的数字壁画复原方法 Digital Mural Inpainting Method Based on Feature Perception 计算机科学, 2022, 49(6): 217-223. https://doi.org/10.11896/jsjkx.210500105 |
[6] | 高志宇, 王天荆, 汪悦, 沈航, 白光伟. 基于生成对抗网络的5G网络流量预测方法 Traffic Prediction Method for 5G Network Based on Generative Adversarial Network 计算机科学, 2022, 49(4): 321-328. https://doi.org/10.11896/jsjkx.210300240 |
[7] | 黎思泉, 万永菁, 蒋翠玲. 基于生成对抗网络去影像的多基频估计算法 Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal 计算机科学, 2022, 49(3): 179-184. https://doi.org/10.11896/jsjkx.201200081 |
[8] | 石达, 芦天亮, 杜彦辉, 张建岭, 暴雨轩. 基于改进CycleGAN的人脸性别伪造图像生成模型 Generation Model of Gender-forged Face Image Based on Improved CycleGAN 计算机科学, 2022, 49(2): 31-39. https://doi.org/10.11896/jsjkx.210600012 |
[9] | 唐雨潇, 王斌君. 基于深度生成模型的人脸编辑研究进展 Research Progress of Face Editing Based on Deep Generative Model 计算机科学, 2022, 49(2): 51-61. https://doi.org/10.11896/jsjkx.210400108 |
[10] | 李建, 郭延明, 于天元, 武与伦, 王翔汉, 老松杨. 基于生成对抗网络的多目标类别对抗样本生成算法 Multi-target Category Adversarial Example Generating Algorithm Based on GAN 计算机科学, 2022, 49(2): 83-91. https://doi.org/10.11896/jsjkx.210800130 |
[11] | 谈馨悦, 何小海, 王正勇, 罗晓东, 卿粼波. 基于Transformer交叉注意力的文本生成图像技术 Text-to-Image Generation Technology Based on Transformer Cross Attention 计算机科学, 2022, 49(2): 107-115. https://doi.org/10.11896/jsjkx.210600085 |
[12] | 陈贵强, 何军. 自然场景下遥感图像超分辨率重建算法研究 Study on Super-resolution Reconstruction Algorithm of Remote Sensing Images in Natural Scene 计算机科学, 2022, 49(2): 116-122. https://doi.org/10.11896/jsjkx.210700095 |
[13] | 蒋宗礼, 樊珂, 张津丽. 基于生成对抗网络和元路径的异质网络表示学习 Generative Adversarial Network and Meta-path Based Heterogeneous Network Representation Learning 计算机科学, 2022, 49(1): 133-139. https://doi.org/10.11896/jsjkx.201000179 |
[14] | 张玮琪, 汤轶丰, 李林燕, 胡伏原. 基于场景图的段落生成序列图像方法 Image Stream From Paragraph Method Based on Scene Graph 计算机科学, 2022, 49(1): 233-240. https://doi.org/10.11896/jsjkx.201100207 |
[15] | 徐涛, 田崇阳, 刘才华. 基于深度学习的人群异常行为检测综述 Deep Learning for Abnormal Crowd Behavior Detection:A Review 计算机科学, 2021, 48(9): 125-134. https://doi.org/10.11896/jsjkx.201100015 |
|