Computer Science ›› 2019, Vol. 46 ›› Issue (3): 283-286.doi: 10.11896/j.issn.1002-137X.2019.03.042

• Graphics ,Image & Pattern Recognition • Previous Articles     Next Articles

Method of Fast Neural Style Transfer with Spatial Constraint

LIU Hong-lin, SHUAI Ren-jun   

  1. College of Computer Science and Technology,Nanjing Tech University,Nanjing 211816,China
  • Received:2018-06-01 Revised:2018-08-05 Online:2019-03-15 Published:2019-03-22

Abstract: Gram matrix,a method to get the inner product in simple terms,was commonly used for image style extraction in the style-transfer techniques.The Gram matrix can only extract the static features,but it is completely unconstrained to the spatial sequence of objects in the picture.This paper proposed a fast neural style transfer method with space constraints.First,the residuals are used to redesign the transform network of fast neural style transfer.Then,the method of spatial offset is used to transform the Feature map.Feature map T(al) are used for Gram matrix computation to get the cross-correlation,which contains the spatial information.That is to say,it can constrain the object’s spatial sequence in the picture.Finally,experiments show that the method’s ability of space constraint is better than traditional method,and the stylized image with better effect can be quickly obtained.

Key words: Cross-correlation, Fast neural style transfer, Gram matrix, Residual, Spatial offset

CLC Number: 

  • TP391
[1]DENG Y Y,TANG F,DONG W M.The status research of styli-
zation of art image[J].Journal of Nanjing University of Information Technology(Natural Science Edition),2017,9(6):593-598.(in Chinese)
邓盈盈,唐帆,董未名.图像艺术风格化的研究现状[J].南京信息工程大学学报(自然科学版),2017,9(6):593-598.
[2]GATYS L A,ECKER A S,BETHGE M.A neural algorithm of artistic style[J].Computer Science,2015,111(1):98-136.
[3]GATYS L A,ECKER A S,BETHGE M.Image style transfer
using convolutional neural networks[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2016:2414-2423.
[4]WILMOT P,RISSER E,BARNES C.Stable and controllable
neural texture synthesis and style transfer using histogram losses[J].Computer Science,2017,15(1):96-140.
[5]JOHNSON J,ALAHI A,FEI-FEI L.Perceptual losses for real-time style transfer and super-resolution[C]∥EuropeanConfe-rence on Computer Vision.Springer,Cham,2016:694-711.
[6]ULYANOV D,VEDALDI A,LEMPITSKY V.Improved tex-
ture networks:Maximizing quality and diversity in feed-forward stylization and texture synthesis[C]∥The IEEE Conference on Computer Vision and Pattern Recognition.2017.
[7]CAO J F.Research and Implementation of Stylization of Image and Video Based on Deep Learning[D].Beijing:Chinese Academy of Sciences University,2017.(in Chinese)
操江峰.基于深度学习的图像与视频风格化研究与实现[D].北京:中国科学院大学,2017.
[8]MAHENDRAN A,VEDALDI A.Understanding deep image
representations by inverting them[C]∥Proceedings of the IEEE Conference on Computer Vision and PatternRecognition (CVPR).2015.
[9]CHIKONTWE P,LEE H J.Towards Robust Face Sketch Synthesis with Style Transfer Algorithms[M]∥IT Convergence and Security 2017.Singapore:Springer,2018:172-179.
[10]ZHU J Y,KRHENBHL P,SHECHTMAN E,et al.Generative visual manipulation on the natural image manifold[C]∥European Conference on Computer Vision.Springer,Cham,2016:597-613.
[11]LV L Z,QIANG Y.Medical CT Image Enhancement Algorithm Based on Laplacian Pyramid and Wavelet Transform[J].Computer Science,2016,43(11):300-303.(in Chinese)
吕鲤志,强彦.基于Laplacian金字塔和小波变换的医学CT图像增强算法[J].计算机科学,2016,43(11):300-303.
[12]IGNATOV A,KOBYSHEV N,VANHOEY K,et al.DSLR-
quality photos on mobile devices with deep convolutional networks[C]∥The IEEE International Conference on Computer Vision (ICCV).2017.
[1] WANG Xin-tong, WANG Xuan, SUN Zhi-xin. Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network [J]. Computer Science, 2022, 49(8): 314-322.
[2] WU Su-jie, ZHOU Jie, WANG Xue-ying, LYU Zhi-kang, SHAO Gen-fu. Study on Characteristics of Millimeter-wave MIMO Channel in Rainfall Environment [J]. Computer Science, 2022, 49(7): 297-303.
[3] GAO Rong-hua, BAI Qiang, WANG Rong, WU Hua-rui, SUN Xiang. Multi-tree Network Multi-crop Early Disease Recognition Method Based on Improved Attention Mechanism [J]. Computer Science, 2022, 49(6A): 363-369.
[4] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[5] HAN Hong-qi, RAN Ya-xin, ZHANG Yun-liang, GUI Jie, GAO Xiong, YI Meng-lin. Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning [J]. Computer Science, 2022, 49(5): 33-42.
[6] GAO Yuan-hao, LUO Xiao-qing, ZHANG Zhan-cheng. Infrared and Visible Image Fusion Based on Feature Separation [J]. Computer Science, 2022, 49(5): 58-63.
[7] ZHAO Ren-xing, XU Pin-jie, LIU Yao. ECG-based Atrial Fibrillation Detection Based on Deep Convolutional Residual Neural Network [J]. Computer Science, 2022, 49(5): 186-193.
[8] QU Zhong, CHEN Wen. Concrete Pavement Crack Detection Based on Dilated Convolution and Multi-features Fusion [J]. Computer Science, 2022, 49(3): 192-196.
[9] ZHANG Lyu, ZHOU Bo-wen, WU Liang-hong. SSD Network Based on Improved Convolutional Attention Module and Residual Structure [J]. Computer Science, 2022, 49(3): 211-217.
[10] ZHOU Wen-hui, SHI Min, ZHU Deng-ming, ZHOU Jun. Seismic Data Super-resolution Method Based on Residual Attention Network [J]. Computer Science, 2021, 48(8): 24-31.
[11] GUO Lin, LI Chen, CHEN Chen, ZHAO Rui, FAN Shi-lin, XU Xing-yu. Image Super-resolution Reconstruction Using Recursive ResidualNetwork Based on ChannelAttention [J]. Computer Science, 2021, 48(8): 139-144.
[12] XU Hua-jie, ZHANG Chen-qiang, SU Guo-shao. Accurate Segmentation Method of Aerial Photography Buildings Based on Deep Convolutional Residual Network [J]. Computer Science, 2021, 48(8): 169-174.
[13] BAO Yu-xuan, LU Tian-liang, DU Yan-hui, SHI Da. Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation [J]. Computer Science, 2021, 48(7): 77-85.
[14] WANG Jian-ming, LI Xiang-feng, YE Lei, ZUO Dun-wen, ZHANG Li-ping. Medical Image Deblur Using Generative Adversarial Networks with Channel Attention [J]. Computer Science, 2021, 48(6A): 101-106.
[15] FENG Jiao, LU Chang-yu. Cross Media Retrieval Method Based on Residual Attention Network [J]. Computer Science, 2021, 48(6A): 122-126.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!