基于多模态多层级数据融合方法的城市功能识别研究

doi:10.11896/jsjkx.210500220

计算机科学 ›› 2021, Vol. 48 ›› Issue (9): 50-58.doi: 10.11896/jsjkx.210500220

所属专题：智能数据治理技术与系统

• 智能数据治理技术与系统^* • 上一篇下一篇

基于多模态多层级数据融合方法的城市功能识别研究

周新民^1,2, 胡宜桂², 刘文洁², 孙荣俊²

1 湖南工商大学新零售虚拟现实技术湖南省重点实验室长沙410205
2 湖南工商大学计算机与信息工程学院长沙410205

收稿日期:2021-05-30 修回日期:2021-07-27 出版日期:2021-09-15 发布日期:2021-09-10
通讯作者: 周新民(zhouxinmin2699@163.com)
基金资助:
国家自然科学基金重大项目(72091515)

Research on Urban Function Recognition Based on Multi-modal and Multi-level Data Fusion Method

ZHOU Xin-min^1,2, HU Yi-gui², LIU Wen-jie², SUN Rong-jun²

1 Key Laboratory of Hunan Province for New Retail Virtual Reality Technology,Hunan University of Technology and Business,Changsha 410205,China
2 School of Computer and Information Engineering,Hunan University of Technology and Business,Changsha 410205,China

Received:2021-05-30 Revised:2021-07-27 Online:2021-09-15 Published:2021-09-10
About author:ZHOU Xin-min,born in 1977,Ph.D,professor,is a member of China Computer Federation.His main research interests include New Smart City and business intelligence and Big Data.
Supported by:
Major Program of the National Natural Science Foundation of China(72091515)

摘要/Abstract

摘要： 城市功能区的划分与识别对分析城市功能区的分布现状和了解城市内部空间结构具有重要意义。这激发了多源地理空间数据融合的需求,特别是城市遥感数据与社会感知数据的融合。然而,如何有效实现城市遥感数据与社会感知数据的融合是一个技术难题。为了实现城市遥感数据与社会感知数据的融合,提高城市功能识别精度,以遥感图像和社会感知数据为例,引入多模态数据融合机制,提出了一种联合深度学习与集成学习的模型来推断城市区域功能。该模型分别利用DenseNet和DPN网络,从多源地理空间数据中提取城市遥感图像特征和社会感知特征,并进行特征级融合、决策级融合以及混合融合的多层级数据融合,对城市功能进行识别。所提模型在URFC数据集上得到了验证,其混合融合总体分类准确度、Kappa系数和平均F1值3个评价指标值分别为74.29%,0.67,71.92%。相比单模态数据的最佳分类方法,所提融合模型的3个评价指标值分别提高了18.83%,0.24,35.46%。实验结果表明,该数据融合模型具有更好的分类性能,能有效融合遥感图像数据和社会感知数据,实现城市区域功能的精准识别。

关键词: 城市功能区识别, 多模态数据融合, 集成学习, 社会感知, 深度学习

Abstract: The division and identification of urban functional areas is of great significance for analyzing the distribution status of urban functional areas and understanding the internal spatial structure of cities.This has stimulated the demand for multi-source geospatial data fusion,especially the fusion of urban remote sensing data and social sensing data.However,how to realize the fusion of urban remote sensing and social sensing data is a technical problem effectively.In order to realize the fusion of urban remote sensing and social sensing data and improve the accuracy of urban function recognition,taking remote sensing images and social sensing data as examples,introducing a multi-modal data fusion mechanism,and proposing a joint deep learning and ensemble learning model to infer urban regional functions.The model uses DenseNet and DPN network to extract urban remote sensing image features and social sensing features from multi-source geospatial data,and carries out multi-level data fusion of feature fusion,decision fusion and hybrid fusion to identify urban functions.The proposed model is verified on the URFC dataset,and these three evaluation index values of hybrid fusion overall classification accuracy,Kappa coefficient and average F1 are 74.29%,0.67,71.92%,respectively.Compared with the best classification method of single modal data,the three evaluation indexes of the proposed fusion model are increased by 18.83%,0.24,35.46% respectively.The experimental results show that the data fusion model has better classification performance,so that it can effectively fuse remote sensing image data and social sensing data,and realize the accurate identification of urban regional functions.

Key words: Deep learning, Ensemble learning, Multi-modal data fusion, Social sensing, Urban function recognition

中图分类号:

TP391

周新民, 胡宜桂, 刘文洁, 孙荣俊. 基于多模态多层级数据融合方法的城市功能识别研究[J]. 计算机科学, 2021, 48(9): 50-58. https://doi.org/10.11896/jsjkx.210500220

ZHOU Xin-min, HU Yi-gui, LIU Wen-jie, SUN Rong-jun. Research on Urban Function Recognition Based on Multi-modal and Multi-level Data Fusion Method[J]. Computer Science, 2021, 48(9): 50-58. https://doi.org/10.11896/jsjkx.210500220

参考文献

[1]EAGLE N,PENTLAND A S.Reality Mining:Sensing Complex Social Systems[J].Personal and Ubiquitous Computing,2006,10(4):255-268.
[2]CAO Y G,WANG Z P,YANG L.Research progress on road extraction methods from high-resolution remote sensing images[J].Remote Sensing Technology and Application,2017,32(1):20-26.
[3]NÚEZ J M,MEDINA S,VILA G,et al.High-Resolution Satellite Imagery Classification for Urban Form Detection[M]//Urban Form and Productivity in Mexico.NewYork:IntechOpen,2019:1-9.
[4]RASHEED S,ASGHAR M A,RAZZAQ S,et al.High-Resolution Remote Sensing Image Classification through Deep Neural Network[C]//2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2).IEEE,2021:1-6.
[5]YU L,XI L,SONG G,et al.Social Sensing:A New Approach to Understanding Our Socioeconomic Environments[J].Annals of the Association of American Geographers,2015,105(3):512-530.
[6]GAO Q,FU J,YU Y,et al.Identification of urban regions'functions in Chengdu,China,based on vehicle trajectory data[J].PLoS ONE,2019,14(4):e0215656.
[7]XIAO F,WANG Y,MEI Y N,et al.Urban functional area discovery method based on travel pattern subgraph[J].Computer Science,2018,45(12):268-278.
[8]YAO Y,LI X,LIU X,et al.Sensing spatial distribution of urban land use by integrating points-of-interest and Google Word2Vec model[J].International Journal of Geographical Information Science,2016(4):1-24.
[9]KANG X,PAN J J,ZHU Y X,et al.An urban core area identification method based on POI big data[J].Remote Sensing Technology and Application,2021,36(1):237-246.
[10]JIANG G L,HU F Y,SHI L X.Urban functional area identification based on call detailed record data[J].Computer Applications,2016,36(7):2046-2050.
[11]JIN P,CHEN M,SUN Z H.Research on the Recognition Me-thod of Urban Land Function Area Based on Mobile Phone Signaling Data[J].Information and Communication,2018(1):268-270.
[12]HOFFMANN E J,WANG Y,WERNER M,et al.Model fusion for building type classification from aerial and street view images[J].Remote Sensing,2019,11(11):1259.
[13]DU X,ZHENG X,LU X,et al.Multisource Remote Sensing Data Classification With Graph Fusion Network[J].IEEE Tran-sactions on Geoscience and Remote Sensing,2021(99):1-11.
[14]XING H,YUAN M.Integrating landscape metrics and socioeconomic features for urban functional region classification[J].Computers Environment and Urban Systems,2018,72:S0198971518300462.
[15]TU W,HU Z,LI L,et al.Portraying urban functional zones by coupling remote sensing imagery and human sensing data[J].Remote Sensing,2018,10(1):141.
[16]QI L,LI J,WANG Y,et al.Urban observation:Integration of remote sensing and social media data[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sen-sing,2019,12(11):4252-4264.
[17]XU N,LUO J,WU T,et al.Identification and portrait of urban functional zones based on multisource heterogeneous data and ensemble learning[J].Remote Sensing,2021,13(3):373.
[18]ZHAO W,BO Y,CHEN J,et al.Exploring semantic elementsfor urban scene recognition:Deep integration of high-resolutionimagery and OpenStreetMap (OSM)[J].ISPRS Journal of Pho-togrammetry and Remote Sensing,2019,151:237-250.
[19]BAO H,MING D,GUO Y,et al.DFCNN-Based Semantic Re-cognition of Urban Functional Zones by Integrating Remote Sensing Data and POI Data[J].Remote Sensing,2020,12(7):1088.
[20]WANG J Y,HE X,WANG Z,et al.CD-CNN:a partially supervised cross-domain deep learning model for urban resident recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018:192-199.
[21]2019 The 5th Baidu & XJTU Big Data Contest The FirstIKCEST “The Belt and Road” International Big Data Contest[EB/OL].(2019-04-25)[2021-07-14].https://dianshi.bce.baidu.com/competition/30/data.
[22]TZIRAKIS P,CHEN J,ZAFEIRIOU S,et al.End-to-end multimodal affect recognition in real-world environments[J].Information Fusion,2021,68:46-53.
[23]HUANG G,LIU Z,VAN D,et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4700-4708.
[24]CHEN B,ZHAO T,LIU J,et al.Multipath feature recalibration DenseNet for image classification[J].International Journal of Machine Learning and Cybernetics,2021,12(3):651-660.
[25]WANG L F,WANG R F,LIN S Z,et al.Multimodal medical image fusion based on dual residual super-dense networks[J].Computer Science,2021,48(2):160-166.
[26]HUANG Z,LI W,LI J,et al.Dual-path attention network for single image super-resolution[J].Expert Systems With Applications,2021,169(1):114450.
[27]LIU X,WANG Z,WANG L.Multimodal Fusion for Image and Text Classification with Feature Selection and Dimension Reduction[C]//Journal of Physics:Conference Series.IOP Publishing,2021:012064.
[28]CHATZIMPARMPAS A,MARTINS R M,KUCHER K,et al.StackGenVis:Alignment of Data,Algorithms,and Models for Stacking Ensemble Learning Using Performance Metrics[J].arXiv:2005.01575,2020.
[29]CAO R,TU W,YANG C,et al.Deep learning-based remote and social sensing data fusion for urban region function recognition[J].ISPRS Journal of Photogrammetry and Remote Sensing,2020,163:82-97.

相关文章 15

[1]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[2]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[3]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[4]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8]	胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[9]	程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[10]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[11]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[12]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[13]	祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋. 改进Faster R-CNN的光学遥感飞机目标检测 Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN 计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
[14]	王建明, 陈响育, 杨自忠, 史晨阳, 张宇航, 钱正坤. 不同数据增强方法对模型识别精度的影响 Influence of Different Data Augmentation Methods on Model Recognition Accuracy 计算机科学, 2022, 49(6A): 418-423. https://doi.org/10.11896/jsjkx.210700210
[15]	毛典辉, 黄晖煜, 赵爽. 符合监管合规性的自动合成新闻检测方法研究 Study on Automatic Synthetic News Detection Method Complying with Regulatory Compliance 计算机科学, 2022, 49(6A): 523-530. https://doi.org/10.11896/jsjkx.210300083

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于多模态多层级数据融合方法的城市功能识别研究

Research on Urban Function Recognition Based on Multi-modal and Multi-level Data Fusion Method

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0