计算机科学 ›› 2024, Vol. 51 ›› Issue (1): 184-189.doi: 10.11896/jsjkx.230600161

• 计算机图形学&多媒体 • 上一篇    下一篇

基于加权损失的点云占用图视频上采样

陈航, 李礼, 刘东, 李厚强   

  1. 中国科学技术大学信息科学技术学院 合肥230026
  • 收稿日期:2023-06-20 修回日期:2023-10-27 出版日期:2024-01-15 发布日期:2024-01-12
  • 通讯作者: 李礼(lil1@ustc.edu.cn)
  • 作者简介:(ashlynchen@mail.ustc.edu.cn)
  • 基金资助:
    国家自然科学基金(62171429)

Weighted-loss-based Up-sampling for Point Cloud Occupancy Map Video

CHEN Hang, LI Li, LIU Dong, LI Houqiang   

  1. School of Information and Technology,University of Science and Technology of China,Hefei 230026,China
  • Received:2023-06-20 Revised:2023-10-27 Online:2024-01-15 Published:2024-01-12
  • About author:CHEN Hang,born in 2000,postgra-duate,is a student member of CCF(No.P3178G).Her main research interests include point cloud compression and video sampling.
    LI Li,born in 1990,Ph.D.His main research interests include image/video coding and processing.
  • Supported by:
    National Natural Science Foundation of China(62171429).

摘要: 基于视频的点云压缩标准(Video-based Point Cloud Compression,V-PCC)中,3D点云会被分成数百个块并投影到2D平面中,形成记录点云纹理信息的纹理视频和记录点云空间信息的几何视频。同时,还需要生成一个占用图视频(Occupancy Map Video),以记录纹理视频和几何视频中每一个像素点是否对应重建点云中的某个点。因此,占用图视频质量与重建点云质量直接相关。为了节约编码比特数,占用图视频在编码端会先被下采样,然后在解码端通过简单的上采样恢复到原分辨率。文中的基本思路是引入深度学习来代替V-PCC中的简单上采样方法,使得上采样后的占用图视频质量更高,从而提高点云的重建质量。在网络训练阶段提出使用加权损失函数,使得在重建点云时能尽可能少地移除正常点并尽可能多地移除噪声点。实验结果证明,所提方法可以大幅提升V-PCC的主客观性能。

关键词: 点云压缩, 基于视频的点云压缩标准, 占用图视频, 视频上采样, 加权失真损失

Abstract: In video-based point cloud compression(V-PCC),a 3D point cloud is divided into hundreds of patches and then mapped onto a 2D grid,generating a texture video that captures texture information and a geometry video that captures geometry information.Meanwhile,an occupancy map video is also generated to record whether each pixel in the former two videos corresponds to a point in the reconstructed point cloud.Therefore,the quality of the occupancy map video is directly linked to the quality of the reconstructed point cloud.To save bit cost,the occupancy map video is down-sampled at the encoder and up-sampled with a simplistic method at the decoder.This paper aims to use a deep learning-based up-sampling method to replace the simple up-sampling method in the original V-PCC to improve the quality of the up-sampled occupancy map videos as well as that of the reconstructed point cloud.A weighted distortion loss function in the network training process is introduced to remove the normal points as few as possible while removing the noisy points as many as possible when reconstructing a point cloud.Experimental results show that the proposed method can significantly improve the subjective and objective performances of the V-PCC.

Key words: Point cloud compression, Video-based point cloud compression standard, Occupancy map video, Video up-sampling, Weighted loss

中图分类号: 

  • TP391
[1]SUI L C,ZHANG B Y.Principle and Trend of Airborne Laser Scanning Remote Sensing [J].Journal of Zhengzhou Institute of Surveying and Mapping,2006,23(2):127-129.
[2]BRUDER G,STEINICKE F,NÜCHTER A.Poster:Immersive point cloud virtual environments[C]//IEEE Symposium on 3D User Interfaces(3DUI).Minneapolis,MN,USA.IEEE,2014:161-162.
[3]SCHWARZ S,PREDA M,BARONCINI V,et al.EmergingMPEG standards for point cloud compression[J].IEEE Journal on Emerging and Selected Topics in Circuits and Systems,2019,9(1):133-148.
[4]YANG J H,ZHAO X,FANG Y Y,et al.3D Laser Point Cloud Based Vehicle Target Recognition Algorithm[C]//International Conference on Mechanical and Electronics Engineering(IC-MEE).Xi’an,China,2022:286-291.
[5]DE QUEIROZ R L,CHOU P A.Motion-compensated compression of dynamic voxelized point clouds[J].IEEE Transactions on Image Processing,2017,26(8):3886-3895.
[6]LI L,LI Z,LIU S.et al.Motion estimation and coding structure for inter-prediction of LiDAR point cloud geometry[J].IEEE Transactions on Multimedia,2022,24:4504-4513.
[7]PERRY S,CONG H P,DA SILVA CRUZ L A,et al.Qualityevaluation of static point clouds encoded using MPEG codecs[C]//IEEE International Conference on Image Processing.Abu Dhabi,United Arab Emirates.IEEE,2020:3428-3432.
[8]SULLIVAN G J,OHM J R,HAN W J,et al.Overview of the high efficiency video coding(HEVC) standard[J].IEEE Transactions on Circuits and Systems for Video Technology,2012,22(12):1649-1668.
[9]SCHWARZ S,SHEIKHIPOUR N,FAKOUR SEVOM V,et al.Video coding of dynamic 3D point cloud data[J].APSIPA Transactions on Signal and Information Processing,2019, 8:e31.
[10]HE L Y,ZHU W J,XU Y L.Best-effort projection based attri-bute compression for 3D point cloud[C]//23rd Asia-Pacific Conference on Communications(APCC).Perth,WA,Australia.IEEE,2017:1-6.
[11]JIA W,LI L,AKHTAR A,et al.Convolutional neural network-based occupancy map accuracy improvement for video-based point cloud compression[J].IEEE Transactions on Multimedia,2022,24:2352-2365.
[12]AKHTAR A,GAO W,LI L,et al.Video-Based Point CloudCompression Artifact Removal[J].IEEE Transactions on Multimedia,2022,24:2866-2876.
[13]HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA,IEEE,2016:770-778.
[14]LI L,LI Z,LIU S,et al.Efficient projected frame padding for video-based point cloud compression[J].IEEE Transactions on Multimedia,2021,23:2806-2819.
[15]CHEN P L,WANG S Q,LI Z.Occupancy Map Guided Attri-butes Deblocking for Video-based Point Cloud Compression[C]//Data Compression Conference(DCC).Snowbird,UT,USA,2023:332-332.
[16]LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops.Honolulu,HI,USA.IEEE,2017:1132-1140.
[17]CHANG A X,FUNKHOUSER T,GUIBAS L,et al.Shapenet:An information-rich 3d model repository[J].arXiv:1512.03012,2015.
[18]D’EON E,HARRISON B,MYERS T,et al.8i voxelized full bodies-a voxelized point cloud datase.ISO/IEC JTC1/SC29 Joint WG11/WG1(MPEG/JPEG) input document WG11M40059/WG1M74006[S].2017,7:8.
[19]Point cloud compression category 2 reference software,TMC2-4.0[OL].http://mpegx.intevry.fr/software/MPEG/PCC/TM/mpeg-pcc-tmc2.git,accessed:2021.
[20]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[21]SCHWARZ,MARTIN-COCHER G, FLYNN D,et al.Budagavi,“Common Test Conditions for Point Cloud Compression,” Document ISO/IEC JTC1/SC29/WG11 w17766[S].Ljubljana,Slovenia,2018.
[22]BJØNTEGAARD G.Calculation of Average PSNR Differences Between RD Curves”,ITU-T SG16/Q6,13th VCEG Meeting[S].Austin,Texas,USA,April 2001,Doc.VCEG-M33.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!