Computer Science ›› 2021, Vol. 48 ›› Issue (2): 142-147.doi: 10.11896/jsjkx.200500158

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Street Scene Change Detection Based on Multiple Difference Features Network

ZHAN Rui, LEI Yin-jie, CHEN Xun-min, YE Shu-han   

  1. College of Electronics and Information Engineering,Sichuan University,Chengdu 610065,China
  • Received:2020-05-29 Revised:2020-08-03 Online:2021-02-15 Published:2021-02-04
  • About author:ZHAN Rui,born in 1996,postgraduate.His main research interests include deep learning and computer vision.
    LEI Yin-jie,born in 1983,Ph.D,asso-ciate professor.His main research in-terests include machine learning,multimedia communication,pattern recognition and image processing.
  • Supported by:
    The National Natural Science Foundation of China (61972435).

Abstract: Street scene change detection plays an important role in the study of natural disaster damage and urban development.Its main goal is to mark out the changing areas in the pair of input images,which is essentially a semantic segmentation problem of binary classification.There may be many interference factors such as light,weather,background noise,viewpoints error and so on when taking street view pictures at different times,which challenges traditional change detection methods.To solve this problem,a new neural network model (Multiple Difference Features Network,MDFNet) is proposed.First,siamese networks are used to extract the different depth features of pairs of input images,and the difference modules are used to calculate the difference of the same depth features to effectively obtain the change information of different depth.Then,by using JPU module to fuse multiple difference features,the deep semantic information can be extracted without losing detail information.Finally,the pyramid pooling module is used to generate the change detection image of the binary classification combined with the global and local information.MDFNet has obtained 0.787 and 0.862 F-scores in the GSV and TSUNAMI part on PCD dataset with 5 fold cross-validation,which are 11.9% and 2.9% higher than the second ranked DOF-CDNet,and can segment the change details more accurately.Therefore,the proposed model can effectively deal with interferences and has an excellent detection ability for complex scenes.

Key words: Change detection, Convolution neural network, Feature fusion, Image processing, Multiple difference features, Semantic segmentation

CLC Number: 

  • TP391
[1] SAKURADA K,OKATANI T.Change detection from a street image pair using cnn features and superpixel segmentation[C]//Proceedings of the British Machive Vision Conference.Swansea,UK:BMVA Press,2015,61:1-12.
[2] LOWE D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.
[3] TANEJA A,BALLAN L,POLLEFEYS M.City-scale changedetection in cadastral 3d models using images[C]// IEEE Conference on Computer Vision & Pattern Recognition.IEEE,2013:113-120.
[4] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[5] ALACANTARILLA P F,STENT S,ROS G,et al.Street-view change detection with deconvolutional networks[J].Autonomous Robots,2018,42(7):1301-1322.
[6] GUO E,FU X,ZHU J,et al.Learning to measure change:fully convolutional Siamese metric networks for scene change detection[J].arXiv:1810.09111,2018.
[7] SAKURADA K,WANG W,KAWAGUCHI N,et al.Dense optical flow based change detection network robust to difference of camera viewpoints[J].arXiv:1712.02941,2017.
[8] DAUDT R C,LE SAUX B,BOULCH A.Fully convolutional sia-mese networks for change detection[C]//2018 25th IEEE International Conference on Image Processing (ICIP).IEEE,2018:4063-4067.
[9] CHEN H,WU C,DU B,et al.Deep siamese multi-scale convolutional network for change detection in multi-temporal vhr images[C]//2019 10th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp).IEEE,2019:1-4.
[10] JIANG H,HU X,LI K,et al.Pga-siamnet:pyramid feature-based attention-guided siamese network for remote sensing orthoimagery building change detection[J].Remote Sensing,2020,12(3):484.
[11] WANG M,TAN K,JIA X,et al.A deep siamese network with hybrid convolutional feature extraction module for change detection based on multi-sensor remote sensing images[J].Remote Sensing,2020,12(2):205.
[12] LV Z,LIU T,BENEDIKTSSON J A,et al.Object-oriented key point vector distance for binary land cover change detection using vhr remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2020,58(9):6524-6533.
[13] LIN H,SHI Z,ZOU Z.Fully convolutional network with task partitioning for inshore ship detection in optical remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2017,14(10):1665-1669.
[14] WU H,ZHANG J,HUANG K,et al.Fastfcn:rethinking dilated convolution in the backbone for semantic segmentation[J].ar-Xiv:1903.11816,2019.
[15] ZHAO H,SHI J,QI X,et al.Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2881-2890.
[16] HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
[17] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems.2012:1097-1105.
[18] HE K,ZHANG X,REN S,et al.Deep residual learning for ima-ge recognition[C]//Computer Vision and Pattern Recognition.2016:770-778.
[1] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[2] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[3] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[4] YU Shu-hao, ZHOU Hui, YE Chun-yang, WANG Tai-zheng. SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion [J]. Computer Science, 2022, 49(6A): 256-260.
[5] LAI Teng-fei, ZHOU Hai-yang, YU Fei-hong. Real-time Extend Depth of Field Algorithm for Video Processing [J]. Computer Science, 2022, 49(6A): 314-318.
[6] YANG Yue, FENG Tao, LIANG Hong, YANG Yang. Image Arbitrary Style Transfer via Criss-cross Attention [J]. Computer Science, 2022, 49(6A): 345-352.
[7] WANG Shan, XU Chu-yi, SHI Chun-xiang, ZHANG Ying. Study on Cloud Classification Method of Satellite Cloud Images Based on CNN-LSTM [J]. Computer Science, 2022, 49(6A): 675-679.
[8] CHEN Yong-ping, ZHU Jian-qing, XIE Yi, WU Han-xiao, ZENG Huan-qiang. Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss [J]. Computer Science, 2022, 49(6A): 424-428.
[9] SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
[10] LAN Ling-xiang, CHI Ming-min. Remote Sensing Change Detection Based on Feature Fusion and Attention Network [J]. Computer Science, 2022, 49(6): 193-198.
[11] LI Fa-guang, YILIHAMU·Yaermaimaiti. Real-time Detection Model of Insulator Defect Based on Improved CenterNet [J]. Computer Science, 2022, 49(5): 84-91.
[12] DONG Qi-da, WANG Zhe, WU Song-yang. Feature Fusion Framework Combining Attention Mechanism and Geometric Information [J]. Computer Science, 2022, 49(5): 129-134.
[13] LI Peng-zu, LI Yao, Ibegbu Nnamdi JULIAN, SUN Chao, GUO Hao, CHEN Jun-jie. Construction and Classification of Brain Function Hypernetwork Based on Overlapping Group Lasso with Multi-feature Fusion [J]. Computer Science, 2022, 49(5): 206-211.
[14] HU Fu-yuan, WAN Xin-jun, SHEN Ming-fei, XU Jiang-lang, YAO Rui, TAO Zhong-ben. Survey Progress on Image Instance Segmentation Methods of Deep Convolutional Neural Network [J]. Computer Science, 2022, 49(5): 10-24.
[15] FAN Xin-nan, ZHAO Zhong-xin, YAN Wei, YAN Xi-jun, SHI Peng-fei. Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism [J]. Computer Science, 2022, 49(5): 50-57.
Full text



No Suggested Reading articles found!