计算机科学 ›› 2021, Vol. 48 ›› Issue (2): 142-147.doi: 10.11896/jsjkx.200500158

• 计算机图形学&多媒体 • 上一篇    下一篇

基于多重差异特征网络的街景变化检测

詹瑞, 雷印杰, 陈训敏, 叶书函   

  1. 四川大学电子信息学院 成都610065
  • 收稿日期:2020-05-29 修回日期:2020-08-03 出版日期:2021-02-15 发布日期:2021-02-04
  • 通讯作者: 雷印杰(yinjie@scu.edu.cn)
  • 作者简介:zhanrray@163.com
  • 基金资助:
    国家自然科学基金(61972435)

Street Scene Change Detection Based on Multiple Difference Features Network

ZHAN Rui, LEI Yin-jie, CHEN Xun-min, YE Shu-han   

  1. College of Electronics and Information Engineering,Sichuan University,Chengdu 610065,China
  • Received:2020-05-29 Revised:2020-08-03 Online:2021-02-15 Published:2021-02-04
  • About author:ZHAN Rui,born in 1996,postgraduate.His main research interests include deep learning and computer vision.
    LEI Yin-jie,born in 1983,Ph.D,asso-ciate professor.His main research in-terests include machine learning,multimedia communication,pattern recognition and image processing.
  • Supported by:
    The National Natural Science Foundation of China (61972435).

摘要: 街景变化检测对于自然灾害破坏和城市发展变化的研究起着重要作用。其主要目标是将成对的输入图片中变化的区域标注出来,其实质是二分类的语义分割问题。不同时间拍摄的街景图片可能受到如光线、天气、背景噪声、视角误差等诸多干扰因素的影响,这给传统的变化检测方法带来挑战。针对该问题,提出了一种新的神经网络模型(Multiple Difference Features Network,MDFNet)。该模型首先使用孪生网络提取成对输入图片的不同深度特征,并使用差异模块对相同深度特征计算差异,以此有效获得不同尺度的变化信息;然后通过JPU模块融合多重差异特征,在不损失细节信息的情况下提取其深层语义信息;最后使用金字塔池化模块结合全局和局部信息生成二分类的变化检测图像。在PCD数据集上的GSV和TSUNAMI部分分别采用5折交叉验证法对模型进行实验,实验结果表明,MDFNet获得了0.787和0.862的F-score,相比排名第二的DOF-CDNet方法,其值提高了约11.9%和2.9%,同时其能够更精准地分割变化细节。因此,所提模型可以有效应对干扰,对于复杂场景也具备优秀的检测能力。

关键词: 变化检测, 多重差异特征, 卷积神经网络, 特征融合, 图像处理, 语义分割

Abstract: Street scene change detection plays an important role in the study of natural disaster damage and urban development.Its main goal is to mark out the changing areas in the pair of input images,which is essentially a semantic segmentation problem of binary classification.There may be many interference factors such as light,weather,background noise,viewpoints error and so on when taking street view pictures at different times,which challenges traditional change detection methods.To solve this problem,a new neural network model (Multiple Difference Features Network,MDFNet) is proposed.First,siamese networks are used to extract the different depth features of pairs of input images,and the difference modules are used to calculate the difference of the same depth features to effectively obtain the change information of different depth.Then,by using JPU module to fuse multiple difference features,the deep semantic information can be extracted without losing detail information.Finally,the pyramid pooling module is used to generate the change detection image of the binary classification combined with the global and local information.MDFNet has obtained 0.787 and 0.862 F-scores in the GSV and TSUNAMI part on PCD dataset with 5 fold cross-validation,which are 11.9% and 2.9% higher than the second ranked DOF-CDNet,and can segment the change details more accurately.Therefore,the proposed model can effectively deal with interferences and has an excellent detection ability for complex scenes.

Key words: Change detection, Convolution neural network, Feature fusion, Image processing, Multiple difference features, Semantic segmentation

中图分类号: 

  • TP391
[1] SAKURADA K,OKATANI T.Change detection from a street image pair using cnn features and superpixel segmentation[C]//Proceedings of the British Machive Vision Conference.Swansea,UK:BMVA Press,2015,61:1-12.
[2] LOWE D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.
[3] TANEJA A,BALLAN L,POLLEFEYS M.City-scale changedetection in cadastral 3d models using images[C]// IEEE Conference on Computer Vision & Pattern Recognition.IEEE,2013:113-120.
[4] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[5] ALACANTARILLA P F,STENT S,ROS G,et al.Street-view change detection with deconvolutional networks[J].Autonomous Robots,2018,42(7):1301-1322.
[6] GUO E,FU X,ZHU J,et al.Learning to measure change:fully convolutional Siamese metric networks for scene change detection[J].arXiv:1810.09111,2018.
[7] SAKURADA K,WANG W,KAWAGUCHI N,et al.Dense optical flow based change detection network robust to difference of camera viewpoints[J].arXiv:1712.02941,2017.
[8] DAUDT R C,LE SAUX B,BOULCH A.Fully convolutional sia-mese networks for change detection[C]//2018 25th IEEE International Conference on Image Processing (ICIP).IEEE,2018:4063-4067.
[9] CHEN H,WU C,DU B,et al.Deep siamese multi-scale convolutional network for change detection in multi-temporal vhr images[C]//2019 10th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp).IEEE,2019:1-4.
[10] JIANG H,HU X,LI K,et al.Pga-siamnet:pyramid feature-based attention-guided siamese network for remote sensing orthoimagery building change detection[J].Remote Sensing,2020,12(3):484.
[11] WANG M,TAN K,JIA X,et al.A deep siamese network with hybrid convolutional feature extraction module for change detection based on multi-sensor remote sensing images[J].Remote Sensing,2020,12(2):205.
[12] LV Z,LIU T,BENEDIKTSSON J A,et al.Object-oriented key point vector distance for binary land cover change detection using vhr remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2020,58(9):6524-6533.
[13] LIN H,SHI Z,ZOU Z.Fully convolutional network with task partitioning for inshore ship detection in optical remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2017,14(10):1665-1669.
[14] WU H,ZHANG J,HUANG K,et al.Fastfcn:rethinking dilated convolution in the backbone for semantic segmentation[J].ar-Xiv:1903.11816,2019.
[15] ZHAO H,SHI J,QI X,et al.Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2881-2890.
[16] HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
[17] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems.2012:1097-1105.
[18] HE K,ZHANG X,REN S,et al.Deep residual learning for ima-ge recognition[C]//Computer Vision and Pattern Recognition.2016:770-778.
[1] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[2] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[3] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[4] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[5] 郭拯危, 付泽文, 李宁, 白澜.
高分辨率斜视聚束SAR回波仿真加速算法研究
Study on Acceleration Algorithm for Raw Data Simulation of High Resolution Squint Spotlight SAR
计算机科学, 2022, 49(8): 178-183. https://doi.org/10.11896/jsjkx.210600066
[6] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[7] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[8] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[9] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[10] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[11] 刘月红, 牛少华, 神显豪.
基于卷积神经网络的虚拟现实视频帧内预测编码
Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network
计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179
[12] 徐鸣珂, 张帆.
Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法
Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition
计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085
[13] 刘伟业, 鲁慧民, 李玉鹏, 马宁.
指静脉识别技术研究综述
Survey on Finger Vein Recognition Research
计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056
[14] 孙福权, 崔志清, 邹彭, 张琨.
基于多尺度特征的脑肿瘤分割算法
Brain Tumor Segmentation Algorithm Based on Multi-scale Features
计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217
[15] 吴子斌, 闫巧.
基于动量的映射式梯度下降算法
Projected Gradient Descent Algorithm with Momentum
计算机科学, 2022, 49(6A): 178-183. https://doi.org/10.11896/jsjkx.210500039
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!