Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (6): 66-72.doi: 10.11896/jsjkx.220400034

• Smart IoT Technologies and Applications Empowered by 6G • Previous Articles     Next Articles

Tile Partition Optimized Omnidirectional Video Coding for 6G Network

YANG Tao-yu, XU Yuan-yuan, TAN Zeng-jie   

  1. School of Computer and Information,Hohai University,Nanjing 211100,China
  • Received:2022-04-02 Revised:2022-04-24 Online:2022-06-15 Published:2022-06-08
  • About author:YANG Tao-yu,born in 1998,master,is a member of China Computer Federation.Her main research interests include immersive videos and video coding.
    XU Yuan-yuan,born in 1983,Ph.D,associate professor,postgraduate supervisor,is a member of China Computer Federation.Her main research interests include image/video coding,image/video communication,immersive videos,visual perception and applications.
  • Supported by:
    National Natural Science Foundation of China (61801125).

PDF (PC)

1049

Abstract: The rise of the 6G wireless communication provides a broader prospect for the development of virtual reality panoramic video.The tile-based panoramic video coding scheme can improve the viewing experience of 360° video under the same condition of network bandwidth.Tile partition affects the video transmission performance.Compared with small tiles,using large tiles can effectively improve coding efficiency,but it will cause more pixel overhead by transmitting a larger area to cover the viewport.The existing tile partition work is mainly designed for the rectangular viewport,but the projection of spherical video to a two-dimensional plane will stretch different areas of the viewport to varying degrees in practice.Deriving the pixel overhead associated with covering irregular viewport areas with rectangular tiles is more complicated.To address this challenge,a tile partitioning algorithm for the user's real viewport has been proposed in this paper.Firstly,the stretching distortion of the viewport caused by the projection format is analyzed,and the pixel overhead of the irregular viewport with different tile partition sizes is derived.Secondly,by trading off the pixel overhead and the coding efficiency of tiles with different granularity,an optimal tile partition scheme has been proposed for the panoramic video sequence.Finally,the proposed scheme is compared with the exhaustive search method for tile partition in the experiment,and the results show that the proposed algorithm can achieve almost the same transmission efficiency as the exhaustive search method with less computational complexity.

Key words: 6G networks, Omnidirectional video, Sensory interconnection, Tile partition, Video coding

CLC Number: 

  • TP751.1
[1] AHMAD A,MANSOOR A,BARAKABITZE A,et al.Supervised-learning-Based QoE Prediction of Video Streaming in Future Networks:A Tutorial with Comparative Study[J].IEEE Communications Magazine,2021,59(11):88-94.
[2] ALWIS C D,ANSHUMAN K,PHAM Q V,et al.Survey on 6G Frontiers:Trends,Applications,Requirements,Technologies and Future Research[J].IEEE Open Journal of the Communications Society,2021,2:836-886.
[3] NGUYEN D C,MING D,PATHIRANA P N,et al.6G Internet of Things:A Comprehensive Survey[J].IEEE Internet of Things Journal,2022,9(1):359-383.
[4] KAUL A,GUPTA J.Revolutionary 6G:Technologies,Archi-tecture,Coverage,and Performance[C]//Proceedings of International Conference on Computing Communication and Net-working Technologies.New York:IEEE Press,2021:1-6.
[5] SKUPIN R,SANCHEZ Y,HELLGE C,et al.Tile Based HEVC Video for Head Mounted Displays[C]//Proceedings of IEEE International Symposium on Multimedia.California:IEEE Press,2016:399-400.
[6] CHIARIOTTI F.A Survey on 360-Degree Video:Coding,Qua-lity of Experience and Streaming[J].Computer Communications,2021,177(1):133-155.
[7] SREEDHAR K K,AMINLOU A,HANNUKSELA M M,et al.Viewport-Adaptive Encoding and Streaming of 360-Degree Video for Virtual Reality Applications[C]//IEEE International Symposium on Multimedia.New York:IEEE Press,2016:583-586.
[8] NASRABADI A T,MAHZARI A,BESHAY J D,et al.Adaptive 360-Degree Video Streaming Using Layered Video Coding[C]//IEEE Virtual Reality.California:IEEE Press,2017:347-348.
[9] VISHWANATH B,NANJUNDASWAM T,ROSE K.Rota-tional Motion Model for Temporal Prediction in 360 video coding[C]//IEEE 19th International Workshop on Multimedia Signal Processing.United Kingdom:IEEE Press,2017:1-6.
[10] FREMEREY S,SINGLA A,MESEBERG K.Avtrack360:An Open Dataset and Software Recording People’s Head Rotations Watching 360° Videos on an HMD[C]//ACM Multimedia System Conference.New York:ACM Press,2018:403-408.
[11] XU M,LI C,ZHANG S,et al.State-of-the-Art in 360° Video/Image Processing:Perception,Assessment and Compression[J].IEEE Journal of Selected Topics in Signal Processing,2020,14(1):5-26.
[12] SULLIVAN G J,OHM J R,HAN W J,et al.Overview of The High Efficiency Video Coding (HEVC) Standard[J].IEEE Transactions on circuits and systems for video technology,2012,22(12):1649-1668.
[13] LI L,YAN N,LI Z,et al.λ-Domain Perceptual Rate Control for 360-Degree Video Compression[J].IEEE Journal of Selected Topics in Signal Processing,2020,14(1):130-145.
[14] ZHOU Y,TIAN L,ZHU C,et al.Video Coding Optimization for Virtual Reality 360-Degree Source[J].IEEE Journal of Selected Topics in Signal Processing,2020,14(1):118-129.
[15] XU Y,YANG T,TAN Z,et al.FoV-based Coding Optimization for 360-degree Virtual Reality Videos[C] //Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing.Singapore:IEEE Press,2022.
[16] LI Y,XU J,CHEN Z.Spherical Domain Rate-Distortion Optimization for 360-Degree Video Coding[C] //Proceedings of IEEE International Conference on Multimedia and Expo.Hong Kong:IEEE Press,2017:709-714.
[17] LIU Y,GUO H,ZHU C,et al.Spherical Position DependentRate-Distortion Optimization for 360-degree Video Coding[C]//Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.Lanzhou:IEEE Press,2019:992-996.
[18] CHIANG J,YANG C,DEDHIA B,et al.Saliency-Driven Rate-Distortion Optimization for 360-Degree Image Coding[J].Multimedia Tools and Applications,2021,80(6):8309-8329.
[19] YANG C,CHIANG J,LIE W.Rate-Distortion Optimization for 360-degree Image Considering Visual Attention[C]//Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.Los Alamitos:IEEE Computer Society,2020:1128-1131.
[20] SUN Y,YU L.Coding Optimization Based on Weighted-to-Spherically-Uniform Quality Metric for 360 Video[C]//Proceedings of IEEE Visual Communications and Image Processing.USA:IEEE Press,2017:1-4.
[21] LUZ G,ASCENSO J,BRITES C,et al.Saliency-Driven Omnidirectional Imaging Adaptive Coding:Modeling and Assessment[C]//Proceedings of IEEE 19th International Workshop on Multimedia Signal Processing.United Kingdom:IEEE Press,2017:1-6.
[22] XIAO M,ZHOU C,LIU Y,et al.Optile:Toward Optimal Tiling in 360-Degree Video Streaming[C]//Proceedings of the 25th ACM International Conference on Multimedia.New York:ACM Press,2017:708-716.
[23] YU M,LAKSHMAN H,GIROD B.Content Adaptive Repre-sentations of Omnidirectional Videos for Cinematic Virtual Reality[C]//Proceedings of the 3rd International Workshop on Immersive Media Experiences.New York:ACM Press,2015:1-6.
[24] MAVLANKAR A,GIROD B.Spatial-Random-Access-Enabled Video Coding for Interactive Virtual Pan/Tilt/Zoom Functionality[J].IEEE Transactions on Circuits and Systems for Video Technology,2011,21(5):577-588.
[25] GUAN Y,ZHENG C,ZHANG X,et al.Pano:Optimizing 360 Video Streaming with A Betterunderstanding of Quality Perception[C]//Proceedings of the ACM Special Interest Group on Data Communication.New York:ACM Press,2019:394-407.
[26] SANCHEZ Y,SKUPIN R,HELLGE C,et al.Spatio-Temporal Activity based Tiling for Panorama Streaming[C]//Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video.New York:ACM Press,2017:61-66.
[27] OZCINAR C,CABRERA J,SMOLIC A.Omnidirectional Video Streaming Using Visual Attention-Driven Dynamic Tiling for VR[C]//IEEE Visual Communications and Image Processing.New York:IEEE Press,2018:1-4.
[28] KAN N,ZOU J,LI C,et al.RAPT360:Reinforcement Learning-Based Rate Adaptation For 360-Degree Video Streaming with Adaptive Prediction and Tiling[J].IEEE Transactions on Circuits and Systems for Video Technology,2021,67(2):409-423.
[29] GUO H W,LUO H J,LIU S,et al.Improved R-λ Model Based Rate Control Algorithm[J].Computer Science,2019,46(3):142-147.
[30] CHEN L,GAO L,REN J,et al.Adaptive Bitrate Streaming for Energy-Efficiency Mobile Augmented Reality[J].Computer Science,2022,49(1):194-203.
[31] OZCINAR C,CABRERA J,SMOLIC A.Visual Attention-Aware Omnidirectional Video Streaming Using Optimal Tiles for Virtual Reality[J].IEEE Journal on Emerging and Selected Topics in Circuits and Systems,2019,9(1):217-230.
[32] NGUYEN D V,TRAN H,THANG T.Adaptive Tiling Selection for Viewport Adaptive Streaming of 360-degree Video[J].IEICE Transactions on Information and Systems,2019,102(1):48-51.
[33] KATTADIGE C,THILAKARATHNA K.VAD360:Viewport Aware Dynamic 360-Degree Video Frame Tiling[J].arXiv:2105.11563,2021.
[34] JEAN L,CONCOLATO C.Tiled-Based Adaptive StreamingUsing MPEG-DASH[J].Multimedia Systems,2016,41(3):1-3.
[1] JIN Hua, ZHU Jing-yu, WANG Chang-da. Review on Video Privacy Protection [J]. Computer Science, 2022, 49(1): 306-313.
[2] CHENG Zhao-wei, SHEN Hang, WANG Yue, WANG Min, BAI Guang-wei. Deep Reinforcement Learning Based UAV Assisted SVC Video Multicast [J]. Computer Science, 2021, 48(9): 271-277.
[3] JI Xiao-xiang, SHEN Hang, BAI Guang-wei. Non-orthogonal Multiple Access Enabled Scalable Video Multicast in HetNets [J]. Computer Science, 2021, 48(11): 356-362.
[4] CAI Yu-han,XIONG Shu-hua,SUN Wei-heng,Karn PRADEEP,HE Xiao-hai. Video Compression Algorithm Combining Frame Rate Up-conversion with HEVC Standard Based on Motion Vector Refinement [J]. Computer Science, 2020, 47(2): 76-82.
[5] HE Xiao-yi,DUAN Ling-yu,LIN Wei-yao. Deep Residual Network Based HEVC Compressed Videos Enhancement [J]. Computer Science, 2019, 46(3): 88-91.
[6] GUO Hong-wei, LUO Hong-jun, LIU Shuai, NIU Lin, YANG Bo. Improved R-λ Model Based Rate Control Algorithm [J]. Computer Science, 2019, 46(3): 142-147.
[7] DU Xiu-li, HU Xing, CHEN Bo, QIU Shao-ming. Multi-hypothesis Reconstruction Algorithm of DCVS Based on Weighted Non-local Similarity [J]. Computer Science, 2019, 46(1): 291-296.
[8] REN Yun, CHENG Fu-lin and LI Hong-song. Disparity Estimation Algorithm Based on Frequency Sensitive Three-dimensional Self-organizing Map [J]. Computer Science, 2017, 44(Z11): 225-227.
[9] SHAN Na-na, ZHOU Wei and DUAN Zhe-min. Improved Entropy Coding Algorithm for Transform Coefficients in HEVC [J]. Computer Science, 2017, 44(6): 290-293.
[10] . Overlay Construction Policy in Layered P2P Streaming System [J]. Computer Science, 2012, 39(5): 69-74.
[11] . Improved Side Information Generation Algorithm in Distributed Video Coding [J]. Computer Science, 2011, 38(11): 275-277.
[12] LU Ji-yuan,SUN Shao-hui,CHAO Hong-yang. Fast Algorithm for Multi-frame Selection with Jointly Optimization of Fractional Pixel Motion Estimation [J]. Computer Science, 2010, 37(6): 283-285.
[13] CAO Lei,SHEN Hang,LUO Bin,BAI Guam-wei. Research on the Key Technologies of Media Streaming for 3G Networks [J]. Computer Science, 2010, 37(5): 57-61.
[14] LU Ji-yuan,CHAO Hong-yang. Simple and High Precision Fast Motion Estimation Algorithm [J]. Computer Science, 2010, 37(2): 282-285.
[15] SU Shu-guang, SU Yue-ming, LIU Yun-sheng. Design and Implement of Multichannel Video Coding Terminal Based on DSP [J]. Computer Science, 2009, 36(9): 252-254.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.