Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (6A): 345-352.doi: 10.11896/jsjkx.210700236

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Image Arbitrary Style Transfer via Criss-cross Attention

YANG Yue1, FENG Tao2, LIANG Hong1, YANG Yang1   

  1. 1 School of Information Science and Engineering,Yunnan University,Kunming 650500,China
    2 School of Information Science and Engineering,Yunnan University of Finance and Economics,Kunming 650221,China
  • Online:2022-06-10 Published:2022-06-08
  • About author:YANG Yue,born in 1996,postgraduate.Her main research interests include deep learning and style transfer.
    FENG Tao,born in 1981,postgraduate,associate professor,postgraduate supervisor,is a member of China Computer Federation.His main research interests include machine learning,intelligence information processing and cyber security.

PDF (PC)

678

Abstract: Arbitrary style transfer is a technique for transferring an ordinary photo to an image with another artistic style.With the development of deep learning,some image arbitrary style transfer algorithms have emerged to generate stylized images with arbitrary styles.To solve the problems in adapting to both global and local styles,maintaining spatial consistency,this paper proposes an arbitrary style transfer via criss-cross attention network,which can efficiently generate stylized images with coordinated global and local styles by capturing long-range dependencies.To address the problem of the distorted content structure of stylized images,a group of the parallel channel and spatial attention networks are added before style transfer,which can further emphasize key features and retain key information.In addition,a new loss function is proposed to eliminate artifacts while preserving the structural information of the content images.This algorithm can match the closest semantic style feature to the content feature,and adjust the local style efficiently and flexibly according to the semantic spatial distribution of the content image.Moreover,it can retain more original information about the structure.The experimental results show that the proposed method can transfer the image into different styles with higher quality and better visual effects.

Key words: Arbitrary style transfer, Channel and spatial attention, Convolutional neural network, Criss-cross attention, Feature fusion, Long-range dependencies

CLC Number: 

  • TP391.41
[1] GATYS L A,ECKER A S,BETHGE M.Image style transferusing convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2016:2414-2423.
[2] JOHNSON J,ALAHI A,LI F F.Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision.Berlin:Springer,2016:694-711.
[3] LUAN F,PARIS S,SHECHTMAN E,et al.Deep photo style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2017:4990-4998.
[4] GU S,CHEN C,LIAO J,et al.Arbitrary style transfer withdeep feature reshuffle[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2018:8222-8231.
[5] JING Y,LIU X,DING Y,et al.Dynamic instance normalization for arbitrary style transfer[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:IAAA Press,2020:4369-4376.
[6] LI X,LIU S,KAUTZ J,et al.Learning linear transformations for fast image and video style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2019:3809-3817.
[7] HUANG X,BELONGIE S.Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2017:1501-1510.
[8] LI Y,FANG C,YANG J,et al.Universal style transfer via feature transforms[J].arXiv:1705.08086,2017.
[9] PARK D Y,LEE K H.Arbitrary style transfer with style-atten-tional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2019:5880-5888.
[10] KYPRIANIDIS J E,COLLOMOSSE J,WANG T,et al.State of the “art”:A taxonomy of artistic stylization techniques for images and video[J].IEEE Transactions on Visualization and Computer Graphics,Institute of Electrical and Electronics Engineers,2013,19(5):866-885.
[11] EFROS A A,LEUNG T K.Texture synthesis by non-parametric sampling[C]//IEEE International Conference on Computer Vision.Piscataway:IEEE Press,1999:1033-1038.
[12] ALEXEI A,EFROS W T.Image quilting for texture synthesis and transfer[C]//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM,2001:341-346.
[13] ELAD M,MILANFAR P.Style-transfer via texture-synthesis[J].arXiv:1609.03057,2016.
[14] ULYANOV D,LEBEDEV V,VEDALDI A,et al.Texture networks:feed-forward synthesis of textures and stylized images[J].arXiv:1603.03417,2016.
[15] SHENG L,LIN Z,SHAO J,et al.Avatar-Net:Multiscale zero-shot style transfer by feature decoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2018:8242-8250.
[16] ULYANOV D,VEDALDI A,LEMPITSKY V.Improved texture networks:Maximizing quality and diversity in feed-forward stylization and texture synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2017:4105-4113.
[17] LI C,WANG M.Combining markov random fields and convolutional neural networks for image synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2016:2479-2486.
[18] WANG X,ZHANG D,WANG Y.Multimodal transfer:A hierarchical deep convolutional neural network for fast artistic style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2017:7178-7186.
[19] GATYS L A,ECKER A S,BETHGE M,et al.Controlling perceptual factors in neural style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2017:3730-3738.
[20] DENG Y,TANG F,DONG W,et al.Arbitrary style transfer via multi-adaptation network[C]//Proceedings of the 28th ACM International Conference on Multimedia.New York:ACM,2020:2719-2727.
[21] LI Y,FANG C,YANG J,et al.Diversified texture synthesiswith feed-forward networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Pisca-taway:IEEE Press,2017:3266-3274.
[22] LI X,LIU S,YANG M.Learning linear transformations for fast image and video style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Pisca-taway:IEEE Press,2019:3804-3812.
[23] ALEX J,CHAMPANDAR D.Semantic style transfer and tur-ning two-bit doodles into fine artworks[J].arXiv:1603.01768,2016.
[24] DUMOULIN V,SHLENS J,KUDLUR M.A learned representation for artistic style[J].arXiv:1610.07629,2016.
[25] YAO X,PUY G,PÉREZ P.Photo style transfer with consistency losses[C]//International Conference on Image Processing.Piscataway:IEEE Press,2019:2314-2318.
[26] LI Y,LIU M Y,LI X,et al.A Closed-form Solution to Photo-realistic Image Stylization[J].arXiv:1802.06474,2018.
[27] MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems.Massachusetts:MIT Press,2014:2204-2212.
[28] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Advances in Neural Information Processing Systems.Massachusetts:MIT Press,2017:5998-6008.
[29] WANG X,GIRSHICK R,GUPTA A,et al.Non-local neuralnetworks[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.Piscataway:IEEE Press,2018:7794-7803.
[30] HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern recognition.Piscataway:IEEE Press,2020:2011-2023.
[31] WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2018:3-19.
[32] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[33] LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft coco:Common objects in context[C]//European Conference on Computer Vision.Berlin:Springer,2014:740-755.
[34] PHILLIPS F,MACKINTOSH B.Wiki Art Gallery,Inc:A case for critical thinking[J].Issues in Accounting Education,2011,26(3):593-608.
[35] DENG J,DONG W,SOCHER R,et al.Imagenet:A large-scale hierarchical image database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Pisca-taway:IEEE Press,2009:248-255.
[1] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[2] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[3] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[4] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[5] DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[6] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[7] LIU Yue-hong, NIU Shao-hua, SHEN Xian-hao. Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(7): 127-131.
[8] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[9] WU Zi-bin, YAN Qiao. Projected Gradient Descent Algorithm with Momentum [J]. Computer Science, 2022, 49(6A): 178-183.
[10] YU Shu-hao, ZHOU Hui, YE Chun-yang, WANG Tai-zheng. SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion [J]. Computer Science, 2022, 49(6A): 256-260.
[11] YANG Jian-nan, ZHANG Fan. Classification Method for Small Crops Combining Dual Attention Mechanisms and Hierarchical Network Structure [J]. Computer Science, 2022, 49(6A): 353-357.
[12] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[13] WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[14] CHEN Yong-ping, ZHU Jian-qing, XIE Yi, WU Han-xiao, ZENG Huan-qiang. Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss [J]. Computer Science, 2022, 49(6A): 424-428.
[15] SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.