Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (11A): 211000176-11.doi: 10.11896/jsjkx.211000176

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Overview of 3D Reconstruction of Indoor Structures Based on Point Clouds

REN Fei1, CHANG Qing-ling1, LIU Xing-lin1, YANG Xin1, LI Ming-hua1, CUI Yan1,2   

  1. 1 Faculty of Intelligent Manufacturing,Wuyi University,Jiangmen,Guangdong 529000,China
    2 4Dage,Zhuhai,Guangdong 519000,China
  • Online:2022-11-10 Published:2022-11-21
  • About author:REN Fei,born in 1998,postgraduate.His main research interests include 3D reconstruction and deep learning.
    CUI Yan,born in 1984,professor.His main research interests include computervision and computer graphic.

PDF (PC)

481

Abstract: The 3D reconstruction of indoor structure is essentially a multi-task problem of restoring the indoor layout,which can further reconstruct and semantically segment wall details and furniture.This paper mainly introduces the 3D reconstruction of indoor structure based on point cloud data.Firstly,the data set commonly used for 3D reconstruction of indoor structure is summarized,and then the main methods of 3D reconstruction of indoor structure based on point cloud are described and discussed,and the advantages and disadvantages of the three types of reconstruction methods are analyzed and summarized.Finally,the difficulties and challenges faced by the current 3D reconstruction research of indoor structures are explained,and the future research trends are prospected.It can be concluded that the diversity of scenes and tasks completed by most reconstruction models at pre-sent is relatively poor,and the multi-task coordination scheme that uses redundant information from different angles to jointly optimize has great potential in the reconstruction of indoor structures.In addition,the model still needs to be improved for the seamless integration of the indoor and outdoor environments and the full performance of the interior and exterior buildings.

Key words: Indoor structure, 3D reconstruction, Layout estimation, Multi-task, Point cloud

CLC Number: 

  • TP391
[1]NIKOOHEMAT S,DIAKITÉ A,ZLATANOVA S,et al.Indoor 3d reconstruction from point clouds for optimal routing in complex buildings to support disaster management [J].Automation in Construction,2020,113:103109.
[2]LIM G,OH Y,KIM D,et al.Modeling of architectural components for large-scale indoor spaces from point cloud measurements [J].IEEE Robotics and Automation Letters,2020,5(3):3830-3837.
[3]CHANG K H,CHENG C Y,LUO J,et al.Building-gan:Graph-conditioned architectural volumetric design generation[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision.2021.
[4]NAUATA N,CHANG K H,CHENG C Y,et al.House-gan:Relational generative adversarial networks for graph-constrained house layout generation[C]//Proceedings of the European Conference on Computer Vision.Springer,2020.
[5]PREVITALI M,BARAZZETTI L,BRUMANA R,et al.To-wards automatic indoor reconstruction of cluttered building rooms from point clouds [C]//ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences.2014.
[6]TURNER E,ZAKHOR A.Watertight as-built architecturalfloor plans generated from laser range data[C]//Proceedings of the 2012 Second International Conference on 3D Imaging,Mo-deling,Processing,Visualization & Transmission.IEEE,2012.
[7]STOJANOVIC V,TRAPP M,RICHTER R,et al.Generation of approximate 2d and 3d floor plans from 3d point clouds[C]//Proceedings of the 14th International Joint Conference on Computer Vision,Imaging and Computer Graphics Theory and Applications(VISIGRAPP 2019).2018.
[8]WU Y,SHANG J,XUE F.Regard:Symmetry-based coarse re-gistration of smartphone's colorful point clouds with cad drawings for low-cost digital twin buildings [J].Remote Sensing,2021,13(10):1882.
[9]CAI Y,FAN L.An efficient approach to automatic construction of 3d watertight geometry of buildings using point clouds [J].Remote Sensing,2021,13(10):1947.
[10]LI M,NAN L.Feature-preserving 3d mesh simplification for urban buildings [J].ISPRS Journal of Photogrammetry and Remote Sensing,2021,173:135-150.
[11]DAI A,RUIZHONGTAI Q C,NIEßNER M.Shape completion using 3d-encoder-predictor cnns and shape synthesis [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017.
[12]HAN X,LI Z,HUANG H,et al.High-resolution shape comple-tion using deep neural networks for global structure and local geometry inference[C]//Proceedings of the IEEE International Conference on Computer Vision.2017.
[13]RIEGLER G,OSMAN ULUSOY A,GEIGER A.Octnet:Lear-ning deep 3d representations at high resolutions[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2017.
[14]HÄHNEL D,BURGARD W,THRUN S.Learning compact 3d models of indoor and outdoor environments with a mobile robot [J].Robotics and Autonomous Systems,2003,44(1):15-27.
[15]COUGHLAN J M,YUILLE A L.Manhattan world:Orientation and outlier detection by bayesian inference [J].Neural Computation,2003,15(5):1063-1088.
[16]LI M,WONKA P,NAN L.Manhattan-world urban reconstruction from point clouds[C]//Proceedings of the ECCV.2016.
[17]NAN L,WONKA P.Polyfit:Polygonal surface reconstructionfrom point clouds[C]//Proceedings of the ICCV 2017.2017.
[18]COUDRON I,PUTTEMANS S,GOEDEMÉ T.Polygonalreconstruction of building interiors from cluttered pointclouds[C]//Proceedings of the European Conference on Computer Vision.2018.
[19]ZHU L,SHEN S,GAO X,et al.Urban scene vectorized mode-ling based on contour deformation [J].ISPRS International Journal of Geo-Information,2020,9(3):162.
[20]IKEHATA S,YANG H,FURUKAWA Y.Structured indoor modeling[C]// Proceedings of the IEEE International Confe-rence on Computer Vision.2015.
[21]Piecewise-planar reconstruction of multi-room interiors with arbitrary wall arrangements [J].Computer Graphics Forum,2016,35(7):179-88.
[22]BASSIER M,VERGAUWEN M.Clustering of wall geometryfrom unstructured point clouds using conditional random fields [J].Remote Sensing,2019,11(13):1586.
[23]NIKOOHEMAT S,DIAKITÉ A,LEHTOLA V,et al.Consistency grammar for 3d indoor model checking [J].Transactions in GIS,2020,25(1):189-212.
[24]CUI Y,LI Q,YANG B,et al.Automatic 3-d reconstruction of indoor environment with mobile laser scanning point clouds [J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2019,PP(99):1-14.
[25]COUDRON I,PUTTEMANS S,GOEDEMÉ T,et al.Semantic extraction of permanent structures for the reconstruction of building interiors from point clouds [J].Sensors,2020,20(23):6916.
[26]LIM G,DOH N.Automatic reconstruction of multi-level indoor spaces from point cloud and trajectory [J].Sensors,2021,21(10):3493.
[27]LIU Y,D′ARONCO S,SCHINDLER K,et al.Pc2wf:3d wireframe reconstruction from raw point clouds [J].arXiv:2103.02766,2021.
[28]WU Y,XUE F.Floorpp-net:Reconstructing floor plans using point pillars for scan-to-bim [J].arXiv:2106.10635,2021.
[29]WANG C,DAI Y,EL-SHEIMY N,et al.Progress on isprsbenchmark on multisensory indoor mapping and positioning [J].Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci.,2019,42:1709-1713.
[30]KHOSHELHAM K,VILARIÑO L D,PETER M,et al.The isprs benchmark on indoor modelling [J].Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci.,2017,42(2):W7.
[31]SONG S,YU F,ZENG A,et al.Semantic scene completion from a single depth image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017.
[32]SONG S,LICHTENBERG S P,XIAO J.Sun rgb-d:A rgb-dscene understanding benchmark suite[C]//IEEE Conference on Computer Vision & Pattern Recognition.2015.
[33]DAI A,CHANG A X,SAVVA M,et al.Scannet:Richly-annotated 3d reconstructions of indoor scenes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017.
[34]SILBERMAN N,FERGUS R.Indoor scene segmentation using a structured light sensor[C]//2011 IEEE International Confe-rence on Computer Vision Workshops(ICCV Workshops).2015.
[35]ARMENI I,SAX S,ZAMIR A R,et al.Joint 2d-3d-semantic data for indoor scene understanding [J].arXiv:1702.01105,2017.
[36]CHANG A,DAI A,FUNKHOUSER T,et al.Matterport3d:Learning from rgb-d data in indoor environments [C]//2017 International Conference on 3D Vision(3DV).2017.
[37]OCHMANN S,VOCK R,WESSEL R,et al.Automatic reconstruction of parametric building models from indoor point clouds [J].Computers & Graphics,2016,54:94-103.
[38]BALLARD D H.Generalizing the hough transform to detect arbitrary shapes [J].Pattern Recognition,1981,13(2):111-122.
[39]TURNER E,ZAKHOR A.Floor plan generation and room labeling of indoor environments from laser range data[C]//Procee-dings of the International Conference on Computer Graphics Theory & Applications.2015.
[40]OESAU S,LAFARGE F,ALLIEZ P.Indoor scene reconstruction using feature sensitive primitive extraction and graph-cut [J].Isprs Journal of Photogrammetry & Remote Sensing,2014,90(APR.):68-82.
[41]MONSZPART A,MELLADO N,BROSTOW G J,et al.Rapter:Rebuilding man-made scenes with regular arrangements of planes [J].ACM Trans Graph,2015,34(4):1-12.
[42]LIN C,LI C,WANG W.Floorplan-jigsaw:Jointly estimatingscene layout and aligning partial scans[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019.
[43]PREVITALI M,DÍAZ-VILARIO L,SCAIONI M.Indoor buil-ding reconstruction from occluded point clouds using graph-cut and ray-tracing [J].Applied Sciences,2018,8(9):1529.
[44]WANG R,XIE L,CHEN D.Modeling indoor spaces using decomposition and reconstruction of structural elements [J].Photogrammetric Engineering & Remote Sensing,2017,83(12):827-841.
[45]BASSIER M,VERGAUWEN M.Topology reconstruction ofbim wall objects from point cloud data [J].Remote Sensing,2020,12(11):1800.
[46]SCHINDLER G,DELLAERT F.Atlanta world:An expectation maximization framework for simultaneous low-level edge grou-ping and camera calibration in complex man-made environments [C]//Proceedings of the 2004 IEEE Computer Society Confe-rence on Computer Vision and Pattern Recognition(CVPR 2004).IEEE,2004.
[47]NIKOOHEMAT S,DIAKITÉ A,ZLATANOVA S,et al.Indoor 3d modeling and flexible space subdivision from point clouds [J].ISPRS Annals of Photogrammetry,Remote Sensing & Spatial Information Sciences,2019,4:285-292.
[48]MURALI S,SPECIALE P,OSWALD M R,et al.Indoorscan2bim:Building information models of house interiors[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).2017.
[49]OCHMANN S,VOCK R,KLEIN R.Automatic reconstruction of fully volumetric 3d building models from point clouds [J].Isprs Journal of Photogrammetry & Remote Sensing,2019,151:251-262.
[50]AGGARWAL C C,WANG H.A survey of clustering algo-rithms for graph data [M]//Managing and Mining Graph Data.Springer,2010:275-301.
[51]TRAN H,KHOSHELHAM K,KEALY A,et al.Extracting topological relations between indoor spaces from point clouds [J].ISPRS Annals of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2017,4:401-406.
[52]GUO L,FAN G,SHENG W.Dual graphical models for relational modeling of indoor object categories[C]//2019. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).2019.
[53]YU L,LI X,FU C W,et al.Ec-net:An edge-aware point set consolidation network [C]//European Conference on Computer Vision.2018.
[54]WANG X,XU Y,XU K,et al.Pie-net:Parametric inference of point cloud edges [J].arXiv:2007.04883,2020.
[55]HÜBNER P,WEINMANN M,WURSTHORN S,et al.Posenormalization of indoor mapping datasets partially compliant to the manhattan world assumption [J].arXiv:2107.07778,2021.
[56]DAI A,RITCHIE D,BOKELOH M,et al.Scancomplete:Large-scale scene completion and semantic segmentation for 3d scans[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018.
[57]LAWIN F J,DANELLJAN M,TOSTEBERG P,et al.Deep projective 3d semantic segmentation[C]//Proceedings of the International Conference on Computer Analysis of Images and Patterns.Springer,2017.
[58]TCHAPMI L,CHOY C,ARMENI I,et al.Segcloud:Semantic segmentation of 3d point clouds[C]//Proceedings of the 2017 International Conference on 3D Vision(3DV).2017.
[59]QI C R,SU H,MO K,et al.Pointnet:Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017.
[60]YU L,LI X,FU C W,et al.Pu-net:Point cloud upsampling network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018.
[61]VOLK R,STENGEL J,SCHULTMANN F.Building information modeling(bim) for existing buildings—literature review and future needs [J].Automation in Construction,2014,38:109-127.
[62]FURUKAWA Y,CURLESS B,SEITZ S M,et al.Manhattan-world stereo [C]//Proceedings/CVPR.2009.
[63]STRAUB J,ROSMAN G,FREIFELD O,et al.A mixture ofmanhattan frames:Beyond the manhattan world[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2014.
[64]PINTORE G,PINTUS R,GANOVELLI F,et al.Recovering 3d existing-conditions of indoor structures from spherical images [J].Computers & Graphics,2018(10):77.
[65]PINTORE G,GANOVELLI F,VILLANUEVA A J,et al.Automatic modeling of cluttered multi-room floor plans from panoramic images[C]//Proceedings of the Computer Graphics Forum.2019.
[66]COUGHLAN J M,YUILLE A L.Manhattan world:Compass direction from a single image by bayesian inference[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision.IEEE,1999.
[67]LIU Y,CHEN G,KNOLL A.Globally optimal vertical direction estimation in atlanta world [J].arXiv:1904.12717,2019.
[68]JOO K,OH T H,KWEON I S,et al.Globally optimal vertical direction estimation in atlanta world [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(4):1949-1962.
[69]ZOU C,COLBURN A,SHAN Q,et al.Layoutnet:Reconstructing the 3d room layout from a single rgb image[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision & Pattern Recognition.2018.
[70]MURA C,MATTAUSCH O,PAJAROLA R.Reconstructingcomplex indoor environments with arbitrary wall orientations[C]//Proceedings of the Proc Eurographics Posters.2014.
[71]SCHWING A G,FIDLER S,POLLEFEYS M,et al.Box in the box:Joint 3d layout and object reasoning from single images[C]//Proceedings of the IEEE International Conference on Computer Vision.2013.
[72]YANG Y,SHI J,LIU R,et al.Automatic 3d indoor scene mo-deling from single panorama[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2018.
[73]ZHANG W,ZHANG W,GU J.Edge-semantic learning strategy for layout estimation in indoor environment [J].IEEE Trans. Cybern.,2020,50(6):2730-2739.
[74]TANG L,LI L,YING S,et al.A full level-of-detail specification for 3d building models combining indoor and outdoor scenes [J].International Journal of Geo-Information,2018,7(11):419.
[75]LANG A H,VORA S,CAESAR H,et al.Pointpillars:Fast encoders for object detection from point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019.
[76]ZHANG C,XU S,JIANG T,et al.Integrating normal vector features into an atrous convolution residual network for lidar point cloud classification [J].Remote Sensing,2021,13(17):3427.
[77]VANCLOOSTER A,NICO V,MAEYER P D.Integrating in-door and outdoor spaces for pedestrian navigation guidance:A review [J].Transactions in GIS,2016,20 (4):491-525.
[78]CHEN S,TIAN D,FENG C,et al.Fast resampling of three-dimensional point clouds via graphs [J].IEEE Transactions on Signal Processing,2017,66(3):666-681.
[79]KOCH T.Automatic alignment of indoor and outdoor building models using 3d line segments [C]//Proceedings of the Computer Vision & Pattern Recognition Workshops.2016.
[80]WANG C,HOU S,WEN C,et al.Semantic line framework-based indoor building modeling using backpacked laser scanning point cloud [J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,143(SEP.):150-166.
[81]HONG S,JUNG J,KIM S,et al.Semi-automated approach to indoor mapping for 3d as-built building information modeling [J].Computers Environment and Urban Systems,2015,51:34-46.
[82]GALLIANI S,LASINGER K,SCHINDLER K.Massively parallel multiview stereopsis by surface normal diffusion [C]//Proceedings of the IEEE International Conference on Computer Vision.2015.
[83]KONTARINIS A,ZEITOUNI K,MARINICA C,et al.Towards a semantic indoor trajectory model:Application to museum visits [J].Geoinformatica,2021,25(2):311-352.
[84]ZHU Q,LI Y,XIONG Q,et al.Indoor multi-dimensional location gml and its application for ubiquitous indoor location ser-vices [J].ISPRS International Journal of Geo-Information,2016,5(12):220.
[1] LI Zong-min, ZHANG Yu-peng, LIU Yu-jie, LI Hua. Deformable Graph Convolutional Networks Based Point Cloud Representation Learning [J]. Computer Science, 2022, 49(8): 273-278.
[2] YANG Wen-kun, YUAN Xiao-pei, CHEN Xiao-feng, GUO Rui. Spatial Multi-feature Segmentation of 3D Lidar Point Cloud [J]. Computer Science, 2022, 49(8): 143-149.
[3] TANG Feng, FENG Xiang, YU Hui-qun. Multi-task Cooperative Optimization Algorithm Based on Adaptive Knowledge Transfer andResource Allocation [J]. Computer Science, 2022, 49(7): 254-262.
[4] DU Li-jun, TANG Xi-lu, ZHOU Jiao, CHEN Yu-lan, CHENG Jian. Alzheimer's Disease Classification Method Based on Attention Mechanism and Multi-task Learning [J]. Computer Science, 2022, 49(6A): 60-65.
[5] FENG Lei, ZHU Deng-ming, LI Zhao-xin, WANG Zhao-qi. Sparse Point Cloud Filtering Algorithm Based on Mask [J]. Computer Science, 2022, 49(5): 25-32.
[6] ZHAO Kai, AN Wei-chao, ZHANG Xiao-yu, WANG Bin, ZHANG Shan, XIANG Jie. Intracerebral Hemorrhage Image Segmentation and Classification Based on Multi-taskLearning of Shared Shallow Parameters [J]. Computer Science, 2022, 49(4): 203-208.
[7] YANG Xiao-yu, YIN Kang-ning, HOU Shao-qi, DU Wen-yi, YIN Guang-qiang. Person Re-identification Based on Feature Location and Fusion [J]. Computer Science, 2022, 49(3): 170-178.
[8] CHE Ai-bo, ZHANG Hui, LI Chen, WANG Yao-nan. Single-stage 3D Object Detector in Traffic Environment Based on Point Cloud Data [J]. Computer Science, 2022, 49(11A): 210900079-6.
[9] YU Meng-juan, NIE Jian-hui. Point Cloud Feature Line Extraction Algorithm Based on PCPNET [J]. Computer Science, 2022, 49(11A): 210800017-6.
[10] ZHENG Shun-yuan, HU Liang-xiao, LYU Xiao-qian, SUN Xin, ZHANG Sheng-ping. Edge Guided Self-correction Skin Detection [J]. Computer Science, 2022, 49(11): 141-147.
[11] SONG Long-ze, WAN Huai-yu, GUO Sheng-nan, LIN You-fang. Multi-task Spatial-Temporal Graph Convolutional Network for Taxi Idle Time Prediction [J]. Computer Science, 2021, 48(7): 112-117.
[12] LIU Xiao-long, HAN Fang, WANG Zhi-jie. Joint Question Answering Model Based on Knowledge Representation [J]. Computer Science, 2021, 48(6): 241-245.
[13] ZHAO Xin-can, CHANG Han-xing, JIN Ren-biao. 3D Point Cloud Shape Completion GAN [J]. Computer Science, 2021, 48(4): 192-196.
[14] ZHOU Xiao-jin, XU Chen-ming, RUAN Tong. Multi-granularity Medical Entity Recognition for Chinese Electronic Medical Records [J]. Computer Science, 2021, 48(4): 237-242.
[15] ZHANG Chun-yun, QU Hao, CUI Chao-ran, SUN Hao-liang, YIN Yi-long. Process Supervision Based Sequence Multi-task Method for Legal Judgement Prediction [J]. Computer Science, 2021, 48(3): 227-232.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.