计算机科学 ›› 2022, Vol. 49 ›› Issue (11A): 211000176-11.doi: 10.11896/jsjkx.211000176
任飞1, 常青玲1, 刘兴林1, 杨鑫1, 李敏华1, 崔岩1,2
REN Fei1, CHANG Qing-ling1, LIU Xing-lin1, YANG Xin1, LI Ming-hua1, CUI Yan1,2
摘要： 室内结构三维重建本质上是一个还原室内布局的多任务问题,可以进一步对墙体细节和家具进行重建和语义分割。主要介绍基于点云数据的室内结构三维重建。首先概述了室内结构三维重建常用的数据集;然后对基于点云的室内结构3维重建的主要方法展开叙述和讨论,并分析总结了3种类型重建方法的优缺点;最后对当前室内结构三维重建研究所面临的困难和挑战进行阐述,并对未来的研究趋势做出展望。可以得出,目前大部分重建模型所针对的场景和完成任务的多样性较为贫乏,利用不同角度的冗余信息共同优化的多任务协调方案在室内结构重建中具有较大潜力。此外,模型对于室内外环境的无缝融合以及实现内外建筑的充分表现仍需要进行改善。
中图分类号:
| [1]NIKOOHEMAT S,DIAKITÉ A,ZLATANOVA S,et al.Indoor 3d reconstruction from point clouds for optimal routing in complex buildings to support disaster management [J].Automation in Construction,2020,113:103109. [2]LIM G,OH Y,KIM D,et al.Modeling of architectural components for large-scale indoor spaces from point cloud measurements [J].IEEE Robotics and Automation Letters,2020,5(3):3830-3837. [3]CHANG K H,CHENG C Y,LUO J,et al.Building-gan:Graph-conditioned architectural volumetric design generation[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision.2021. [4]NAUATA N,CHANG K H,CHENG C Y,et al.House-gan:Relational generative adversarial networks for graph-constrained house layout generation[C]//Proceedings of the European Conference on Computer Vision.Springer,2020. [5]PREVITALI M,BARAZZETTI L,BRUMANA R,et al.To-wards automatic indoor reconstruction of cluttered building rooms from point clouds [C]//ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences.2014. [6]TURNER E,ZAKHOR A.Watertight as-built architecturalfloor plans generated from laser range data[C]//Proceedings of the 2012 Second International Conference on 3D Imaging,Mo-deling,Processing,Visualization & Transmission.IEEE,2012. [7]STOJANOVIC V,TRAPP M,RICHTER R,et al.Generation of approximate 2d and 3d floor plans from 3d point clouds[C]//Proceedings of the 14th International Joint Conference on Computer Vision,Imaging and Computer Graphics Theory and Applications(VISIGRAPP 2019).2018. [8]WU Y,SHANG J,XUE F.Regard:Symmetry-based coarse re-gistration of smartphone's colorful point clouds with cad drawings for low-cost digital twin buildings [J].Remote Sensing,2021,13(10):1882. [9]CAI Y,FAN L.An efficient approach to automatic construction of 3d watertight geometry of buildings using point clouds [J].Remote Sensing,2021,13(10):1947. [10]LI M,NAN L.Feature-preserving 3d mesh simplification for urban buildings [J].ISPRS Journal of Photogrammetry and Remote Sensing,2021,173:135-150. [11]DAI A,RUIZHONGTAI Q C,NIEßNER M.Shape completion using 3d-encoder-predictor cnns and shape synthesis [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017. [12]HAN X,LI Z,HUANG H,et al.High-resolution shape comple-tion using deep neural networks for global structure and local geometry inference[C]//Proceedings of the IEEE International Conference on Computer Vision.2017. [13]RIEGLER G,OSMAN ULUSOY A,GEIGER A.Octnet:Lear-ning deep 3d representations at high resolutions[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2017. [14]HÄHNEL D,BURGARD W,THRUN S.Learning compact 3d models of indoor and outdoor environments with a mobile robot [J].Robotics and Autonomous Systems,2003,44(1):15-27. [15]COUGHLAN J M,YUILLE A L.Manhattan world:Orientation and outlier detection by bayesian inference [J].Neural Computation,2003,15(5):1063-1088. [16]LI M,WONKA P,NAN L.Manhattan-world urban reconstruction from point clouds[C]//Proceedings of the ECCV.2016. [17]NAN L,WONKA P.Polyfit:Polygonal surface reconstructionfrom point clouds[C]//Proceedings of the ICCV 2017.2017. [18]COUDRON I,PUTTEMANS S,GOEDEMÉ T.Polygonalreconstruction of building interiors from cluttered pointclouds[C]//Proceedings of the European Conference on Computer Vision.2018. [19]ZHU L,SHEN S,GAO X,et al.Urban scene vectorized mode-ling based on contour deformation [J].ISPRS International Journal of Geo-Information,2020,9(3):162. [20]IKEHATA S,YANG H,FURUKAWA Y.Structured indoor modeling[C]// Proceedings of the IEEE International Confe-rence on Computer Vision.2015. [21]Piecewise-planar reconstruction of multi-room interiors with arbitrary wall arrangements [J].Computer Graphics Forum,2016,35(7):179-88. [22]BASSIER M,VERGAUWEN M.Clustering of wall geometryfrom unstructured point clouds using conditional random fields [J].Remote Sensing,2019,11(13):1586. [23]NIKOOHEMAT S,DIAKITÉ A,LEHTOLA V,et al.Consistency grammar for 3d indoor model checking [J].Transactions in GIS,2020,25(1):189-212. [24]CUI Y,LI Q,YANG B,et al.Automatic 3-d reconstruction of indoor environment with mobile laser scanning point clouds [J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2019,PP(99):1-14. [25]COUDRON I,PUTTEMANS S,GOEDEMÉ T,et al.Semantic extraction of permanent structures for the reconstruction of building interiors from point clouds [J].Sensors,2020,20(23):6916. [26]LIM G,DOH N.Automatic reconstruction of multi-level indoor spaces from point cloud and trajectory [J].Sensors,2021,21(10):3493. [27]LIU Y,D′ARONCO S,SCHINDLER K,et al.Pc2wf:3d wireframe reconstruction from raw point clouds [J].arXiv:2103.02766,2021. [28]WU Y,XUE F.Floorpp-net:Reconstructing floor plans using point pillars for scan-to-bim [J].arXiv:2106.10635,2021. [29]WANG C,DAI Y,EL-SHEIMY N,et al.Progress on isprsbenchmark on multisensory indoor mapping and positioning [J].Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci.,2019,42:1709-1713. [30]KHOSHELHAM K,VILARIÑO L D,PETER M,et al.The isprs benchmark on indoor modelling [J].Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci.,2017,42(2):W7. [31]SONG S,YU F,ZENG A,et al.Semantic scene completion from a single depth image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017. [32]SONG S,LICHTENBERG S P,XIAO J.Sun rgb-d:A rgb-dscene understanding benchmark suite[C]//IEEE Conference on Computer Vision & Pattern Recognition.2015. [33]DAI A,CHANG A X,SAVVA M,et al.Scannet:Richly-annotated 3d reconstructions of indoor scenes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017. [34]SILBERMAN N,FERGUS R.Indoor scene segmentation using a structured light sensor[C]//2011 IEEE International Confe-rence on Computer Vision Workshops(ICCV Workshops).2015. [35]ARMENI I,SAX S,ZAMIR A R,et al.Joint 2d-3d-semantic data for indoor scene understanding [J].arXiv:1702.01105,2017. [36]CHANG A,DAI A,FUNKHOUSER T,et al.Matterport3d:Learning from rgb-d data in indoor environments [C]//2017 International Conference on 3D Vision(3DV).2017. [37]OCHMANN S,VOCK R,WESSEL R,et al.Automatic reconstruction of parametric building models from indoor point clouds [J].Computers & Graphics,2016,54:94-103. [38]BALLARD D H.Generalizing the hough transform to detect arbitrary shapes [J].Pattern Recognition,1981,13(2):111-122. [39]TURNER E,ZAKHOR A.Floor plan generation and room labeling of indoor environments from laser range data[C]//Procee-dings of the International Conference on Computer Graphics Theory & Applications.2015. [40]OESAU S,LAFARGE F,ALLIEZ P.Indoor scene reconstruction using feature sensitive primitive extraction and graph-cut [J].Isprs Journal of Photogrammetry & Remote Sensing,2014,90(APR.):68-82. [41]MONSZPART A,MELLADO N,BROSTOW G J,et al.Rapter:Rebuilding man-made scenes with regular arrangements of planes [J].ACM Trans Graph,2015,34(4):1-12. [42]LIN C,LI C,WANG W.Floorplan-jigsaw:Jointly estimatingscene layout and aligning partial scans[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019. [43]PREVITALI M,DÍAZ-VILARIO L,SCAIONI M.Indoor buil-ding reconstruction from occluded point clouds using graph-cut and ray-tracing [J].Applied Sciences,2018,8(9):1529. [44]WANG R,XIE L,CHEN D.Modeling indoor spaces using decomposition and reconstruction of structural elements [J].Photogrammetric Engineering & Remote Sensing,2017,83(12):827-841. [45]BASSIER M,VERGAUWEN M.Topology reconstruction ofbim wall objects from point cloud data [J].Remote Sensing,2020,12(11):1800. [46]SCHINDLER G,DELLAERT F.Atlanta world:An expectation maximization framework for simultaneous low-level edge grou-ping and camera calibration in complex man-made environments [C]//Proceedings of the 2004 IEEE Computer Society Confe-rence on Computer Vision and Pattern Recognition(CVPR 2004).IEEE,2004. [47]NIKOOHEMAT S,DIAKITÉ A,ZLATANOVA S,et al.Indoor 3d modeling and flexible space subdivision from point clouds [J].ISPRS Annals of Photogrammetry,Remote Sensing & Spatial Information Sciences,2019,4:285-292. [48]MURALI S,SPECIALE P,OSWALD M R,et al.Indoorscan2bim:Building information models of house interiors[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).2017. [49]OCHMANN S,VOCK R,KLEIN R.Automatic reconstruction of fully volumetric 3d building models from point clouds [J].Isprs Journal of Photogrammetry & Remote Sensing,2019,151:251-262. [50]AGGARWAL C C,WANG H.A survey of clustering algo-rithms for graph data [M]//Managing and Mining Graph Data.Springer,2010:275-301. [51]TRAN H,KHOSHELHAM K,KEALY A,et al.Extracting topological relations between indoor spaces from point clouds [J].ISPRS Annals of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2017,4:401-406. [52]GUO L,FAN G,SHENG W.Dual graphical models for relational modeling of indoor object categories[C]//2019. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).2019. [53]YU L,LI X,FU C W,et al.Ec-net:An edge-aware point set consolidation network [C]//European Conference on Computer Vision.2018. [54]WANG X,XU Y,XU K,et al.Pie-net:Parametric inference of point cloud edges [J].arXiv:2007.04883,2020. [55]HÜBNER P,WEINMANN M,WURSTHORN S,et al.Posenormalization of indoor mapping datasets partially compliant to the manhattan world assumption [J].arXiv:2107.07778,2021. [56]DAI A,RITCHIE D,BOKELOH M,et al.Scancomplete:Large-scale scene completion and semantic segmentation for 3d scans[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018. [57]LAWIN F J,DANELLJAN M,TOSTEBERG P,et al.Deep projective 3d semantic segmentation[C]//Proceedings of the International Conference on Computer Analysis of Images and Patterns.Springer,2017. [58]TCHAPMI L,CHOY C,ARMENI I,et al.Segcloud:Semantic segmentation of 3d point clouds[C]//Proceedings of the 2017 International Conference on 3D Vision(3DV).2017. [59]QI C R,SU H,MO K,et al.Pointnet:Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017. [60]YU L,LI X,FU C W,et al.Pu-net:Point cloud upsampling network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018. [61]VOLK R,STENGEL J,SCHULTMANN F.Building information modeling(bim) for existing buildings—literature review and future needs [J].Automation in Construction,2014,38:109-127. [62]FURUKAWA Y,CURLESS B,SEITZ S M,et al.Manhattan-world stereo [C]//Proceedings/CVPR.2009. [63]STRAUB J,ROSMAN G,FREIFELD O,et al.A mixture ofmanhattan frames:Beyond the manhattan world[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2014. [64]PINTORE G,PINTUS R,GANOVELLI F,et al.Recovering 3d existing-conditions of indoor structures from spherical images [J].Computers & Graphics,2018(10):77. [65]PINTORE G,GANOVELLI F,VILLANUEVA A J,et al.Automatic modeling of cluttered multi-room floor plans from panoramic images[C]//Proceedings of the Computer Graphics Forum.2019. [66]COUGHLAN J M,YUILLE A L.Manhattan world:Compass direction from a single image by bayesian inference[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision.IEEE,1999. [67]LIU Y,CHEN G,KNOLL A.Globally optimal vertical direction estimation in atlanta world [J].arXiv:1904.12717,2019. [68]JOO K,OH T H,KWEON I S,et al.Globally optimal vertical direction estimation in atlanta world [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(4):1949-1962. [69]ZOU C,COLBURN A,SHAN Q,et al.Layoutnet:Reconstructing the 3d room layout from a single rgb image[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision & Pattern Recognition.2018. [70]MURA C,MATTAUSCH O,PAJAROLA R.Reconstructingcomplex indoor environments with arbitrary wall orientations[C]//Proceedings of the Proc Eurographics Posters.2014. [71]SCHWING A G,FIDLER S,POLLEFEYS M,et al.Box in the box:Joint 3d layout and object reasoning from single images[C]//Proceedings of the IEEE International Conference on Computer Vision.2013. [72]YANG Y,SHI J,LIU R,et al.Automatic 3d indoor scene mo-deling from single panorama[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2018. [73]ZHANG W,ZHANG W,GU J.Edge-semantic learning strategy for layout estimation in indoor environment [J].IEEE Trans. Cybern.,2020,50(6):2730-2739. [74]TANG L,LI L,YING S,et al.A full level-of-detail specification for 3d building models combining indoor and outdoor scenes [J].International Journal of Geo-Information,2018,7(11):419. [75]LANG A H,VORA S,CAESAR H,et al.Pointpillars:Fast encoders for object detection from point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019. [76]ZHANG C,XU S,JIANG T,et al.Integrating normal vector features into an atrous convolution residual network for lidar point cloud classification [J].Remote Sensing,2021,13(17):3427. [77]VANCLOOSTER A,NICO V,MAEYER P D.Integrating in-door and outdoor spaces for pedestrian navigation guidance:A review [J].Transactions in GIS,2016,20 (4):491-525. [78]CHEN S,TIAN D,FENG C,et al.Fast resampling of three-dimensional point clouds via graphs [J].IEEE Transactions on Signal Processing,2017,66(3):666-681. [79]KOCH T.Automatic alignment of indoor and outdoor building models using 3d line segments [C]//Proceedings of the Computer Vision & Pattern Recognition Workshops.2016. [80]WANG C,HOU S,WEN C,et al.Semantic line framework-based indoor building modeling using backpacked laser scanning point cloud [J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,143(SEP.):150-166. [81]HONG S,JUNG J,KIM S,et al.Semi-automated approach to indoor mapping for 3d as-built building information modeling [J].Computers Environment and Urban Systems,2015,51:34-46. [82]GALLIANI S,LASINGER K,SCHINDLER K.Massively parallel multiview stereopsis by surface normal diffusion [C]//Proceedings of the IEEE International Conference on Computer Vision.2015. [83]KONTARINIS A,ZEITOUNI K,MARINICA C,et al.Towards a semantic indoor trajectory model:Application to museum visits [J].Geoinformatica,2021,25(2):311-352. [84]ZHU Q,LI Y,XIONG Q,et al.Indoor multi-dimensional location gml and its application for ubiquitous indoor location ser-vices [J].ISPRS International Journal of Geo-Information,2016,5(12):220. |
| [1] | 杨文坤, 原晓佩, 陈小锋, 郭睿. 三维激光雷达点云空间多特征分割 Spatial Multi-feature Segmentation of 3D Lidar Point Cloud 计算机科学, 2022, 49(8): 143-149. https://doi.org/10.11896/jsjkx.210300275 |
| [2] | 李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023 |
| [3] | 唐枫, 冯翔, 虞慧群. 基于自适应知识迁移与资源分配的多任务协同优化算法 Multi-task Cooperative Optimization Algorithm Based on Adaptive Knowledge Transfer andResource Allocation 计算机科学, 2022, 49(7): 254-262. https://doi.org/10.11896/jsjkx.210600184 |
| [4] | 杜丽君, 唐玺璐, 周娇, 陈玉兰, 程建. 基于注意力机制和多任务学习的阿尔茨海默症分类 Alzheimer's Disease Classification Method Based on Attention Mechanism and Multi-task Learning 计算机科学, 2022, 49(6A): 60-65. https://doi.org/10.11896/jsjkx.201200072 |
| [5] | 封雷, 朱登明, 李兆歆, 王兆其. 一种基于遮罩的稀疏点云滤波算法 Sparse Point Cloud Filtering Algorithm Based on Mask 计算机科学, 2022, 49(5): 25-32. https://doi.org/10.11896/jsjkx.210600129 |
| [6] | 赵凯, 安卫超, 张晓宇, 王彬, 张杉, 相洁. 共享浅层参数多任务学习的脑出血图像分割与分类 Intracerebral Hemorrhage Image Segmentation and Classification Based on Multi-taskLearning of Shared Shallow Parameters 计算机科学, 2022, 49(4): 203-208. https://doi.org/10.11896/jsjkx.201000153 |
| [7] | 杨晓宇, 殷康宁, 候少麒, 杜文仪, 殷光强. 基于特征定位与融合的行人重识别算法 Person Re-identification Based on Feature Location and Fusion 计算机科学, 2022, 49(3): 170-178. https://doi.org/10.11896/jsjkx.210100132 |
| [8] | 喻孟娟, 聂建辉. 基于PCPNET的点云特征线提取算法 Point Cloud Feature Line Extraction Algorithm Based on PCPNET 计算机科学, 2022, 49(11A): 210800017-6. https://doi.org/10.11896/jsjkx.210800017 |
| [9] | 车爱博, 张辉, 李晨, 王耀南. 基于点云数据的交通环境下单阶段三维目标检测方法 Single-stage 3D Object Detector in Traffic Environment Based on Point Cloud Data 计算机科学, 2022, 49(11A): 210900079-6. https://doi.org/10.11896/jsjkx.210900079 |
| [10] | 郑顺源, 胡良校, 吕晓倩, 孙鑫, 张盛平. 基于边缘引导的自校正皮肤检测 Edge Guided Self-correction Skin Detection 计算机科学, 2022, 49(11): 141-147. https://doi.org/10.11896/jsjkx.220600012 |
| [11] | 宋龙泽, 万怀宇, 郭晟楠, 林友芳. 面向出租车空载时间预测的多任务时空图卷积网络 Multi-task Spatial-Temporal Graph Convolutional Network for Taxi Idle Time Prediction 计算机科学, 2021, 48(7): 112-117. https://doi.org/10.11896/jsjkx.201000089 |
| [12] | 郭文, 尹童灵, 张天柱, 徐常胜. 时间一致性保持的多任务稀疏深度表达视觉跟踪 Temporal Consistency Preserving Multi-Mask Sparse Deep Representation for Visual Tracking 计算机科学, 2021, 48(6): 110-117. https://doi.org/10.11896/jsjkx.200800212 |
| [13] | 刘小龙, 韩芳, 王直杰. 基于知识表示的联合问答模型 Joint Question Answering Model Based on Knowledge Representation 计算机科学, 2021, 48(6): 241-245. https://doi.org/10.11896/jsjkx.200600011 |
| [14] | 赵新灿, 常寒星, 金仁标. 3D点云形状补全GAN 3D Point Cloud Shape Completion GAN 计算机科学, 2021, 48(4): 192-196. https://doi.org/10.11896/jsjkx.200100048 |
| [15] | 周晓进, 徐陈铭, 阮彤. 面向中文电子病历的多粒度医疗实体识别 Multi-granularity Medical Entity Recognition for Chinese Electronic Medical Records 计算机科学, 2021, 48(4): 237-242. https://doi.org/10.11896/jsjkx.200100036 |
|
||
首页