Computer Science ›› 2020, Vol. 47 ›› Issue (11): 174-178.doi: 10.11896/jsjkx.191100014

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Semantic Segmentation Transfer Algorithm Based on Atrous Convolution Discriminator

YANG Pei-jian1, WU Xiao-fu1, ZHANG Suo-fei2, ZHOU Quan1   

  1. 1 School of Telecommunication and Information Engineering,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    2 School of Internet of Things,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
  • Received:2019-11-03 Revised:2020-06-06 Online:2020-11-15 Published:2020-11-05
  • About author:YANG Pei-jian,born in 1995,postgra-duate.His main research interests include semantic segmentation and transfer learning.
    WU Xiao-fu,born in 1975,Ph.D,professor.His main research interests include computer vision,face recognition and transfer learning.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61372123,61701252).

Abstract: Supervised semantic segmentation with convolutional neural networks has made great progress in recent years.Since the pix-level labeling required by supervised sematic segmentation is tedious and labor intensive,one way that becomes recently prevalent is to collect photo-realistic synthetic data from video games,where pixel-level annotation can be automatically generated.Despite this,the intrinsic domain difference between synthetic and real images usually causes a significant performance drop when applying the learned model to real world scenarios.To solve this problem,we propose a novel domain adaptive semantic segmentation method.It firstly performs image style conversion over the source domain for reducing the domain difference.Then,the generative adversarial network is employed for feature alignment between source and target domains.In particular,we propose to use the atrous convolution for constructing the powerful discriminator network with the enlarged field of view.Extensive experiments show that the proposed algorithm can achieve 4.5% mIoU improvement on the GTA5 dataset and 2.6% on the SYNTHIA dataset,compared with the classic AdaptSegNet algorithm.

Key words: Atrous convolution, Deep learning, Domain adaptation, Generative adversarial network, Semantic segmentation, Transfer learning

CLC Number: 

  • TP391
[1] CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Deeplab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(4):834-848.
[2] LONG J,SHELHAMER E,DARRELL T.Fully convolutionalnetworks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440.
[3] LIU Z,LI X,LUO P,et al.Semantic image segmentation viadeep parsing network[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1377-1385.
[4] LIN G,SHEN C,VAN DEN HENGEL A,et al.Efficient piecewise training of deep structured models for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:3194-3203.
[5] ZHENG S,JAYASUMANA S,ROMERA-PAREDES B,et al.Conditional random fields as recurrent neural networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1529-1537.
[6] ROS G,SELLART L,MATERZYNSKA J,et al.The synthiadataset:A large collection of synthetic images for semantic segmentation of urban scenes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:3234-3243.
[7] RICHTER S R,VINEET V,ROTH S,et al.Playing for data:Ground truth from computer games[C]//European Conference on Computer Vision.Cham:Springer,2016:102-118.
[8] CARIUCCI F M,PORZI L,CAPUTO B,et al.Autodial:Automatic domain alignment layers[C]//2017 IEEE International Conference on Computer Vision (ICCV).IEEE,2017:5077-5085.
[9] MANCINI M,PORZI L,ROTA BULÒ S,et al.Boosting domainadaptation by discovering latent domains[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3771-3780.
[10] HOFFMAN J,WANG D,YU F,et al.Fcns in the wild:Pixel-level adversarial and constraint-based adaptation[J].arXiv:1612.02649,2016.
[11] SANKARANARAYANAN S,BALAJI Y,JAIN A,et al.Learning from synthetic data:Addressing domain shift for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3752-3761.
[12] TSAI Y H,HUNG W C,SCHULTER S,et al.Learning toadapt structured output space for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7472-7481.
[13] ZHU J Y,PARK T,ISOLA P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2223-2232.
[14] HOFFMAN J,TZENG E,PARK T,et al.Cycada:Cycle-consistent adversarial domain adaptation[J].arXiv:1711.03213,2017.
[15] WU Z,HAN X,LIN Y L,et al.Dcan:Dual channel-wise alignment networks for unsupervised scene adaptation[C]//Proceedings of the European Conference on Computer Vision (ECCV).2018:518-534.
[16] CORDTS M,OMRAN M,RAMOS S,et al.The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:3213-3223.
[17] DENG J,DONG W,SOCHER R,et al.Imagenet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2009:248-255.
[18] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[1] ZHANG Jia, DONG Shou-bin. Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer [J]. Computer Science, 2022, 49(9): 41-47.
[2] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[3] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[4] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[5] FANG Yi-qiu, ZHANG Zhen-kun, GE Jun-wei. Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning [J]. Computer Science, 2022, 49(8): 70-77.
[6] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[7] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[8] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[9] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[10] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[11] DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[12] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[13] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[14] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[15] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
Full text



No Suggested Reading articles found!