Computer Science ›› 2019, Vol. 46 ›› Issue (9): 259-264.doi: 10.11896/j.issn.1002-137X.2019.09.039

• Graphics,Image & Pattern Recognition • Previous Articles     Next Articles

Image Localized Style Transfer Based on Convolutional Neural Network

MIAO Yong-wei1,2, LI Gao-yi1, BAO Chen1, ZHANG Xu-dong1, PENG Si-long3   

  1. (College of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China)1;
    (College of Information Science and Technology,Zhejiang Sci-Tech University,Hangzhou 310018,China)2;
    (Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China)3
  • Received:2018-07-20 Online:2019-09-15 Published:2019-09-02

Abstract: Image style transfer is a research hot topic in computer graphics and computer vision.Aiming at the difficulty in the style transfer of the local area of the content image in the existing image style transfer method,this paper proposed a localized image transfer framework based on convolutional neural network.First,according to the input content image and style image,the image style transfer network is used to generate the whole style transferred image.Then,the image foreground and the background area are determined by the mask generated by automatic semantic segmentation.Finally,according to style transfer result of the foreground or the background region,an image fusion algorithm based on Manhattan distance is proposed to optimize the convergence and smooth transition between the stylized object and the original area.The framework comprehensively considers the pixel values and positions of the target area and the boundary band,and experiments on three public image datasets demonstrate that the method can efficiently,quickly and naturally implement local style transfer of input content maps,and produce visual effects that are both artistic and authentic.

Key words: Automatic semantic segmentation, Convolutional neural network (CNN), Deep learning, Localized image style transfer, Manhattan distance

CLC Number: 

  • TP391
[1]GOOCH B,GOOCH A.Non-photorealistic rendering [M].New York:AK Peters/CRC Press,2001.
[2]DECARLO D,SANTELL A.Stylization and abstraction of photographs [J].ACM Transactions on Graphics,2002,21(3):769-776.
[3]WANG J,XU Y,SHUM H Y,et al.Video toning [J].ACM Transactions on Graphics,2004,23(3):574-583.
[4]LI P,SUN H Q,SHENG B,et al.Image stylization with enhanced structure on GPU [J].Science China Information Scie-nces,2012,55(5):1093-1105.
[5]ZANG Y,HUANG H,LI C F.Stroke style analysis for painterly rendering [J].Journal of Computer Science and Technology,2013,28(5):762-775.
[6]GATYS L A,Ecker A S,BETHGE M.Image style transfer using convolutional neural networks [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE Press,2016:2414-2423.
[7]GATYS L A,BETHGE M,HERTZMANN A,et al.Preserving color in neural artistic style transfer[J].arXiv:1606.05897,2016.
[8]JOHNSON J,ALAHI A,LI F F.Perceptual losses for real-time style transfer and super-resolution [C]//Proceedings of Euro-pean Conference on Computer Vision.Cham:Springer,2016:694-711.
[9]DUMOULIN V,SHLENS J,KUDLUR M,et al.A learned representation for artistic style [J].arXiv:1610.07629,2016.
[10]SHELHAMER E,LONG J,DARRELL T.Fully convolutional networks for semantic segmentation [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(4):640-651.
[11]LAFFERTY J D,MCCALLUM A,PEREIRA F C N.Condi-tional random fields:probabilistic models for segmenting and labeling sequence data [C]//Proceedings of Eighteenth International Confe-rence on Machine Learning.Morgan Kaufmann Publishers Inc.,2001:282-289.
[12]DENG J,DONG W,SOCHER R,et al.ImageNet:A large-scale hierarchical image database [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE Press,2009:710-719.
[13]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition [J].arXiv:1409.1556,2014.
[14]HE K,ZHANG X,REN S,et al.Deep residual learning for ima-ge recognition [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE Press,2016:770-778.
[15]MALKAUTHEKAR M D.Analysis of euclidean distance andManhattan Distance measure in face recognition [C]//International Conference on Computational Intelligence and Information Technology.IET,2013:503-507.
[16]ABADI M,AGARWAL A,BARHAM P,et al.Tensorflow:large-scale machine learning on heterogeneous distributed systems [J].arXiv:1603.04467,2016.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[5] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[6] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[9] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[10] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[11] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[12] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[13] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[14] SUN Fu-quan, CUI Zhi-qing, ZOU Peng, ZHANG Kun. Brain Tumor Segmentation Algorithm Based on Multi-scale Features [J]. Computer Science, 2022, 49(6A): 12-16.
[15] KANG Yan, XU Yu-long, KOU Yong-qi, XIE Si-yu, YANG Xue-kun, LI Hao. Drug-Drug Interaction Prediction Based on Transformer and LSTM [J]. Computer Science, 2022, 49(6A): 17-21.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!