计算机科学 ›› 2021, Vol. 48 ›› Issue (9): 50-58.doi: 10.11896/jsjkx.210500220

• 智能数据治理技术与系统* 上一篇    下一篇

基于多模态多层级数据融合方法的城市功能识别研究

周新民1,2, 胡宜桂2, 刘文洁2, 孙荣俊2   

  1. 1 湖南工商大学新零售虚拟现实技术湖南省重点实验室 长沙410205
    2 湖南工商大学计算机与信息工程学院 长沙410205
  • 收稿日期:2021-05-30 修回日期:2021-07-27 出版日期:2021-09-15 发布日期:2021-09-10
  • 通讯作者: 周新民(zhouxinmin2699@163.com)
  • 基金资助:
    国家自然科学基金重大项目(72091515)

Research on Urban Function Recognition Based on Multi-modal and Multi-level Data Fusion Method

ZHOU Xin-min1,2, HU Yi-gui2, LIU Wen-jie2, SUN Rong-jun2   

  1. 1 Key Laboratory of Hunan Province for New Retail Virtual Reality Technology,Hunan University of Technology and Business,Changsha 410205,China
    2 School of Computer and Information Engineering,Hunan University of Technology and Business,Changsha 410205,China
  • Received:2021-05-30 Revised:2021-07-27 Online:2021-09-15 Published:2021-09-10
  • About author:ZHOU Xin-min,born in 1977,Ph.D,professor,is a member of China Computer Federation.His main research interests include New Smart City and business intelligence and Big Data.
  • Supported by:
    Major Program of the National Natural Science Foundation of China(72091515)

摘要: 城市功能区的划分与识别对分析城市功能区的分布现状和了解城市内部空间结构具有重要意义。这激发了多源地理空间数据融合的需求,特别是城市遥感数据与社会感知数据的融合。然而,如何有效实现城市遥感数据与社会感知数据的融合是一个技术难题。为了实现城市遥感数据与社会感知数据的融合,提高城市功能识别精度,以遥感图像和社会感知数据为例,引入多模态数据融合机制,提出了一种联合深度学习与集成学习的模型来推断城市区域功能。该模型分别利用DenseNet和DPN网络,从多源地理空间数据中提取城市遥感图像特征和社会感知特征,并进行特征级融合、决策级融合以及混合融合的多层级数据融合,对城市功能进行识别。所提模型在URFC数据集上得到了验证,其混合融合总体分类准确度、Kappa系数和平均F1值3个评价指标值分别为74.29%,0.67,71.92%。相比单模态数据的最佳分类方法,所提融合模型的3个评价指标值分别提高了18.83%,0.24,35.46%。实验结果表明,该数据融合模型具有更好的分类性能,能有效融合遥感图像数据和社会感知数据,实现城市区域功能的精准识别。

关键词: 城市功能区识别, 多模态数据融合, 深度学习, 集成学习, 社会感知

Abstract: The division and identification of urban functional areas is of great significance for analyzing the distribution status of urban functional areas and understanding the internal spatial structure of cities.This has stimulated the demand for multi-source geospatial data fusion,especially the fusion of urban remote sensing data and social sensing data.However,how to realize the fusion of urban remote sensing and social sensing data is a technical problem effectively.In order to realize the fusion of urban remote sensing and social sensing data and improve the accuracy of urban function recognition,taking remote sensing images and social sensing data as examples,introducing a multi-modal data fusion mechanism,and proposing a joint deep learning and ensemble learning model to infer urban regional functions.The model uses DenseNet and DPN network to extract urban remote sensing image features and social sensing features from multi-source geospatial data,and carries out multi-level data fusion of feature fusion,decision fusion and hybrid fusion to identify urban functions.The proposed model is verified on the URFC dataset,and these three evaluation index values of hybrid fusion overall classification accuracy,Kappa coefficient and average F1 are 74.29%,0.67,71.92%,respectively.Compared with the best classification method of single modal data,the three evaluation indexes of the proposed fusion model are increased by 18.83%,0.24,35.46% respectively.The experimental results show that the data fusion model has better classification performance,so that it can effectively fuse remote sensing image data and social sensing data,and realize the accurate identification of urban regional functions.

Key words: Urban function recognition, Multi-modal data fusion, Deep learning, Ensemble learning, Social sensing

中图分类号: 

  • TP391
[1]EAGLE N,PENTLAND A S.Reality Mining:Sensing Complex Social Systems[J].Personal and Ubiquitous Computing,2006,10(4):255-268.
[2]CAO Y G,WANG Z P,YANG L.Research progress on road extraction methods from high-resolution remote sensing images[J].Remote Sensing Technology and Application,2017,32(1):20-26.
[3]NÚEZ J M,MEDINA S,VILA G,et al.High-Resolution Satellite Imagery Classification for Urban Form Detection[M]//Urban Form and Productivity in Mexico.NewYork:IntechOpen,2019:1-9.
[4]RASHEED S,ASGHAR M A,RAZZAQ S,et al.High-Resolution Remote Sensing Image Classification through Deep Neural Network[C]//2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2).IEEE,2021:1-6.
[5]YU L,XI L,SONG G,et al.Social Sensing:A New Approach to Understanding Our Socioeconomic Environments[J].Annals of the Association of American Geographers,2015,105(3):512-530.
[6]GAO Q,FU J,YU Y,et al.Identification of urban regions'functions in Chengdu,China,based on vehicle trajectory data[J].PLoS ONE,2019,14(4):e0215656.
[7]XIAO F,WANG Y,MEI Y N,et al.Urban functional area discovery method based on travel pattern subgraph[J].Computer Science,2018,45(12):268-278.
[8]YAO Y,LI X,LIU X,et al.Sensing spatial distribution of urban land use by integrating points-of-interest and Google Word2Vec model[J].International Journal of Geographical Information Science,2016(4):1-24.
[9]KANG X,PAN J J,ZHU Y X,et al.An urban core area identification method based on POI big data[J].Remote Sensing Technology and Application,2021,36(1):237-246.
[10]JIANG G L,HU F Y,SHI L X.Urban functional area identification based on call detailed record data[J].Computer Applications,2016,36(7):2046-2050.
[11]JIN P,CHEN M,SUN Z H.Research on the Recognition Me-thod of Urban Land Function Area Based on Mobile Phone Signaling Data[J].Information and Communication,2018(1):268-270.
[12]HOFFMANN E J,WANG Y,WERNER M,et al.Model fusion for building type classification from aerial and street view images[J].Remote Sensing,2019,11(11):1259.
[13]DU X,ZHENG X,LU X,et al.Multisource Remote Sensing Data Classification With Graph Fusion Network[J].IEEE Tran-sactions on Geoscience and Remote Sensing,2021(99):1-11.
[14]XING H,YUAN M.Integrating landscape metrics and socioeconomic features for urban functional region classification[J].Computers Environment and Urban Systems,2018,72:S0198971518300462.
[15]TU W,HU Z,LI L,et al.Portraying urban functional zones by coupling remote sensing imagery and human sensing data[J].Remote Sensing,2018,10(1):141.
[16]QI L,LI J,WANG Y,et al.Urban observation:Integration of remote sensing and social media data[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sen-sing,2019,12(11):4252-4264.
[17]XU N,LUO J,WU T,et al.Identification and portrait of urban functional zones based on multisource heterogeneous data and ensemble learning[J].Remote Sensing,2021,13(3):373.
[18]ZHAO W,BO Y,CHEN J,et al.Exploring semantic elementsfor urban scene recognition:Deep integration of high-resolutionimagery and OpenStreetMap (OSM)[J].ISPRS Journal of Pho-togrammetry and Remote Sensing,2019,151:237-250.
[19]BAO H,MING D,GUO Y,et al.DFCNN-Based Semantic Re-cognition of Urban Functional Zones by Integrating Remote Sensing Data and POI Data[J].Remote Sensing,2020,12(7):1088.
[20]WANG J Y,HE X,WANG Z,et al.CD-CNN:a partially supervised cross-domain deep learning model for urban resident recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018:192-199.
[21]2019 The 5th Baidu & XJTU Big Data Contest The FirstIKCEST “The Belt and Road” International Big Data Contest[EB/OL].(2019-04-25)[2021-07-14].https://dianshi.bce.baidu.com/competition/30/data.
[22]TZIRAKIS P,CHEN J,ZAFEIRIOU S,et al.End-to-end multimodal affect recognition in real-world environments[J].Information Fusion,2021,68:46-53.
[23]HUANG G,LIU Z,VAN D,et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4700-4708.
[24]CHEN B,ZHAO T,LIU J,et al.Multipath feature recalibration DenseNet for image classification[J].International Journal of Machine Learning and Cybernetics,2021,12(3):651-660.
[25]WANG L F,WANG R F,LIN S Z,et al.Multimodal medical image fusion based on dual residual super-dense networks[J].Computer Science,2021,48(2):160-166.
[26]HUANG Z,LI W,LI J,et al.Dual-path attention network for single image super-resolution[J].Expert Systems With Applications,2021,169(1):114450.
[27]LIU X,WANG Z,WANG L.Multimodal Fusion for Image and Text Classification with Feature Selection and Dimension Reduction[C]//Journal of Physics:Conference Series.IOP Publishing,2021:012064.
[28]CHATZIMPARMPAS A,MARTINS R M,KUCHER K,et al.StackGenVis:Alignment of Data,Algorithms,and Models for Stacking Ensemble Learning Using Performance Metrics[J].arXiv:2005.01575,2020.
[29]CAO R,TU W,YANG C,et al.Deep learning-based remote and social sensing data fusion for urban region function recognition[J].ISPRS Journal of Photogrammetry and Remote Sensing,2020,163:82-97.
[1] 董晓梅, 王蕊, 邹欣开. 面向推荐应用的差分隐私方案综述[J]. 计算机科学, 2021, 48(9): 21-35.
[2] 钱梦薇, 过弋. 融合偏置深度学习的距离分解Top-N推荐算法[J]. 计算机科学, 2021, 48(9): 103-109.
[3] 徐涛, 田崇阳, 刘才华. 基于深度学习的人群异常行为检测综述[J]. 计算机科学, 2021, 48(9): 125-134.
[4] 张新峰, 宋博. 一种基于改进三元组损失和特征融合的行人重识别方法[J]. 计算机科学, 2021, 48(9): 146-152.
[5] 林椹尠, 张梦凯, 吴成茂, 郑兴宁. 利用生成对抗网络的人脸图像分步补全法[J]. 计算机科学, 2021, 48(9): 174-180.
[6] 黄晓生, 徐静. 基于PCANet的非下采样剪切波域多聚焦图像融合[J]. 计算机科学, 2021, 48(9): 181-186.
[7] 田野, 陈宏巍, 王法胜, 陈兴文. 室内移动机器人的SLAM算法综述[J]. 计算机科学, 2021, 48(9): 223-234.
[8] 谢良旭, 李峰, 谢建平, 许晓军. 基于融合神经网络模型的药物分子性质预测[J]. 计算机科学, 2021, 48(9): 251-256.
[9] 冯霞, 胡志毅, 刘才华. 跨模态检索研究进展综述[J]. 计算机科学, 2021, 48(8): 13-23.
[10] 王立梅, 朱旭光, 汪德嘉, 张勇, 邢春晓. 基于深度学习的民事案件判决结果分类方法研究[J]. 计算机科学, 2021, 48(8): 80-85.
[11] 郭琳, 李晨, 陈晨, 赵睿, 范仕霖, 徐星雨. 基于通道注意递归残差网络的图像超分辨率重建[J]. 计算机科学, 2021, 48(8): 139-144.
[12] 刘帅, 芮挺, 胡育成, 杨成松, 王东. 基于深度学习SuperGlue算法的单目视觉里程计[J]. 计算机科学, 2021, 48(8): 157-161.
[13] 王施云, 杨帆. 基于U-Net特征融合优化策略的遥感影像语义分割方法[J]. 计算机科学, 2021, 48(8): 162-168.
[14] 田嵩旺, 蔺素珍, 杨博. 基于多判别器的多波段图像自监督融合方法[J]. 计算机科学, 2021, 48(8): 185-190.
[15] 潘孝勤, 芦天亮, 杜彦辉, 仝鑫. 基于深度学习的语音合成与转换技术综述[J]. 计算机科学, 2021, 48(8): 200-208.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 郑秀林,宋海燕,付伊鹏. MORUS-1280-128算法的区分分析[J]. 计算机科学, 2018, 45(4): 152 -156 .
[2] 戴文静, 袁家斌. 隐含子群问题的研究现状[J]. 计算机科学, 2018, 45(6): 1 -8 .
[3] 崔一辉, 宋伟, 彭智勇, 杨先娣. 基于差分隐私的多源数据关联规则挖掘方法[J]. 计算机科学, 2018, 45(6): 36 -40 .
[4] 王占兵, 宋伟, 彭智勇, 杨先娣, 崔一辉, 申远. 一种面向密文基因数据的子序列外包查询方法[J]. 计算机科学, 2018, 45(6): 51 -56 .
[5] 刘景玮, 刘京菊, 陆余良, 杨斌, 朱凯龙. 基于网络攻防博弈模型的最优防御策略选取方法[J]. 计算机科学, 2018, 45(6): 117 -123 .
[6] 郭莹莹, 张丽平, 李松. 障碍环境中线段组最近邻查询方法研究[J]. 计算机科学, 2018, 45(6): 172 -175 .
[7] 黄一龙, 李培峰, 朱巧明. 事件因果与时序关系识别的联合推理模型[J]. 计算机科学, 2018, 45(6): 204 -207 .
[8] 徐晓玲, 金忠, 贲圣兰. 基于标签敏感最大间隔准则的人脸年龄两步估计算法[J]. 计算机科学, 2018, 45(6): 284 -290 .
[9] 李宗鑫, 秦勃, 王梦倩. 基于时空关系模型的交通信号灯的实时检测与识别[J]. 计算机科学, 2018, 45(6): 314 -319 .
[10] 徐丽丽,董一鸿,潘剑飞,陈华辉. 面向复杂网络的图稀疏算法综述[J]. 计算机科学, 2018, 45(5): 24 -30 .