Computer Science ›› 2021, Vol. 48 ›› Issue (9): 50-58.doi: 10.11896/jsjkx.210500220

Special Issue: Intelligent Data Governance Technologies and Systems

• Intelligent Data Governance Technologies and Systems • Previous Articles     Next Articles

Research on Urban Function Recognition Based on Multi-modal and Multi-level Data Fusion Method

ZHOU Xin-min1,2, HU Yi-gui2, LIU Wen-jie2, SUN Rong-jun2   

  1. 1 Key Laboratory of Hunan Province for New Retail Virtual Reality Technology,Hunan University of Technology and Business,Changsha 410205,China
    2 School of Computer and Information Engineering,Hunan University of Technology and Business,Changsha 410205,China
  • Received:2021-05-30 Revised:2021-07-27 Online:2021-09-15 Published:2021-09-10
  • About author:ZHOU Xin-min,born in 1977,Ph.D,professor,is a member of China Computer Federation.His main research interests include New Smart City and business intelligence and Big Data.
  • Supported by:
    Major Program of the National Natural Science Foundation of China(72091515)

Abstract: The division and identification of urban functional areas is of great significance for analyzing the distribution status of urban functional areas and understanding the internal spatial structure of cities.This has stimulated the demand for multi-source geospatial data fusion,especially the fusion of urban remote sensing data and social sensing data.However,how to realize the fusion of urban remote sensing and social sensing data is a technical problem effectively.In order to realize the fusion of urban remote sensing and social sensing data and improve the accuracy of urban function recognition,taking remote sensing images and social sensing data as examples,introducing a multi-modal data fusion mechanism,and proposing a joint deep learning and ensemble learning model to infer urban regional functions.The model uses DenseNet and DPN network to extract urban remote sensing image features and social sensing features from multi-source geospatial data,and carries out multi-level data fusion of feature fusion,decision fusion and hybrid fusion to identify urban functions.The proposed model is verified on the URFC dataset,and these three evaluation index values of hybrid fusion overall classification accuracy,Kappa coefficient and average F1 are 74.29%,0.67,71.92%,respectively.Compared with the best classification method of single modal data,the three evaluation indexes of the proposed fusion model are increased by 18.83%,0.24,35.46% respectively.The experimental results show that the data fusion model has better classification performance,so that it can effectively fuse remote sensing image data and social sensing data,and realize the accurate identification of urban regional functions.

Key words: Deep learning, Ensemble learning, Multi-modal data fusion, Social sensing, Urban function recognition

CLC Number: 

  • TP391
[1]EAGLE N,PENTLAND A S.Reality Mining:Sensing Complex Social Systems[J].Personal and Ubiquitous Computing,2006,10(4):255-268.
[2]CAO Y G,WANG Z P,YANG L.Research progress on road extraction methods from high-resolution remote sensing images[J].Remote Sensing Technology and Application,2017,32(1):20-26.
[3]NÚEZ J M,MEDINA S,VILA G,et al.High-Resolution Satellite Imagery Classification for Urban Form Detection[M]//Urban Form and Productivity in Mexico.NewYork:IntechOpen,2019:1-9.
[4]RASHEED S,ASGHAR M A,RAZZAQ S,et al.High-Resolution Remote Sensing Image Classification through Deep Neural Network[C]//2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2).IEEE,2021:1-6.
[5]YU L,XI L,SONG G,et al.Social Sensing:A New Approach to Understanding Our Socioeconomic Environments[J].Annals of the Association of American Geographers,2015,105(3):512-530.
[6]GAO Q,FU J,YU Y,et al.Identification of urban regions'functions in Chengdu,China,based on vehicle trajectory data[J].PLoS ONE,2019,14(4):e0215656.
[7]XIAO F,WANG Y,MEI Y N,et al.Urban functional area discovery method based on travel pattern subgraph[J].Computer Science,2018,45(12):268-278.
[8]YAO Y,LI X,LIU X,et al.Sensing spatial distribution of urban land use by integrating points-of-interest and Google Word2Vec model[J].International Journal of Geographical Information Science,2016(4):1-24.
[9]KANG X,PAN J J,ZHU Y X,et al.An urban core area identification method based on POI big data[J].Remote Sensing Technology and Application,2021,36(1):237-246.
[10]JIANG G L,HU F Y,SHI L X.Urban functional area identification based on call detailed record data[J].Computer Applications,2016,36(7):2046-2050.
[11]JIN P,CHEN M,SUN Z H.Research on the Recognition Me-thod of Urban Land Function Area Based on Mobile Phone Signaling Data[J].Information and Communication,2018(1):268-270.
[12]HOFFMANN E J,WANG Y,WERNER M,et al.Model fusion for building type classification from aerial and street view images[J].Remote Sensing,2019,11(11):1259.
[13]DU X,ZHENG X,LU X,et al.Multisource Remote Sensing Data Classification With Graph Fusion Network[J].IEEE Tran-sactions on Geoscience and Remote Sensing,2021(99):1-11.
[14]XING H,YUAN M.Integrating landscape metrics and socioeconomic features for urban functional region classification[J].Computers Environment and Urban Systems,2018,72:S0198971518300462.
[15]TU W,HU Z,LI L,et al.Portraying urban functional zones by coupling remote sensing imagery and human sensing data[J].Remote Sensing,2018,10(1):141.
[16]QI L,LI J,WANG Y,et al.Urban observation:Integration of remote sensing and social media data[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sen-sing,2019,12(11):4252-4264.
[17]XU N,LUO J,WU T,et al.Identification and portrait of urban functional zones based on multisource heterogeneous data and ensemble learning[J].Remote Sensing,2021,13(3):373.
[18]ZHAO W,BO Y,CHEN J,et al.Exploring semantic elementsfor urban scene recognition:Deep integration of high-resolutionimagery and OpenStreetMap (OSM)[J].ISPRS Journal of Pho-togrammetry and Remote Sensing,2019,151:237-250.
[19]BAO H,MING D,GUO Y,et al.DFCNN-Based Semantic Re-cognition of Urban Functional Zones by Integrating Remote Sensing Data and POI Data[J].Remote Sensing,2020,12(7):1088.
[20]WANG J Y,HE X,WANG Z,et al.CD-CNN:a partially supervised cross-domain deep learning model for urban resident recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018:192-199.
[21]2019 The 5th Baidu & XJTU Big Data Contest The FirstIKCEST “The Belt and Road” International Big Data Contest[EB/OL].(2019-04-25)[2021-07-14].https://dianshi.bce.baidu.com/competition/30/data.
[22]TZIRAKIS P,CHEN J,ZAFEIRIOU S,et al.End-to-end multimodal affect recognition in real-world environments[J].Information Fusion,2021,68:46-53.
[23]HUANG G,LIU Z,VAN D,et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4700-4708.
[24]CHEN B,ZHAO T,LIU J,et al.Multipath feature recalibration DenseNet for image classification[J].International Journal of Machine Learning and Cybernetics,2021,12(3):651-660.
[25]WANG L F,WANG R F,LIN S Z,et al.Multimodal medical image fusion based on dual residual super-dense networks[J].Computer Science,2021,48(2):160-166.
[26]HUANG Z,LI W,LI J,et al.Dual-path attention network for single image super-resolution[J].Expert Systems With Applications,2021,169(1):114450.
[27]LIU X,WANG Z,WANG L.Multimodal Fusion for Image and Text Classification with Feature Selection and Dimension Reduction[C]//Journal of Physics:Conference Series.IOP Publishing,2021:012064.
[28]CHATZIMPARMPAS A,MARTINS R M,KUCHER K,et al.StackGenVis:Alignment of Data,Algorithms,and Models for Stacking Ensemble Learning Using Performance Metrics[J].arXiv:2005.01575,2020.
[29]CAO R,TU W,YANG C,et al.Deep learning-based remote and social sensing data fusion for urban region function recognition[J].ISPRS Journal of Photogrammetry and Remote Sensing,2020,163:82-97.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[9] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[10] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[11] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[12] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[13] ZHU Wen-tao, LAN Xian-chao, LUO Huan-lin, YUE Bing, WANG Yang. Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN [J]. Computer Science, 2022, 49(6A): 378-383.
[14] WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[15] MAO Dian-hui, HUANG Hui-yu, ZHAO Shuang. Study on Automatic Synthetic News Detection Method Complying with Regulatory Compliance [J]. Computer Science, 2022, 49(6A): 523-530.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!