计算机科学 ›› 2022, Vol. 49 ›› Issue (6): 66-72.doi: 10.11896/jsjkx.220400034
杨桃雨, 徐媛媛, 谭增洁
YANG Tao-yu, XU Yuan-yuan, TAN Zeng-jie
摘要: 第6代无线通信的兴起为虚拟现实全景视频的发展提供了更广阔的前景。基于片的全景视频编码方案能够在相同网络带宽条件下提升全景视频的观看体验,片划分大小会影响视频传输的性能,与较小的片相比,采用较大的片能够降低编码冗余,但需传送更大区域来覆盖相同视口,这会造成更多的像素开销。目前已有片划分工作主要是针对矩形视口而设计的,而实际中,球型视频到二维平面的投影会对视口不同区域进行不同程度的拉伸,用矩形片覆盖不规则视口区域时像素开销计算更为复杂。针对这一挑战,文中提出了一种适用于用户真实视口的片划分算法。首先,分析了投影格式对视口区域造成的拉伸失真,并对在不同片划分下不规则视口区域的额外像素开销进行了推导;然后,通过权衡不同划片粒度编码单元的像素开销与编码效率的关系,提出了一种全景视频序列最优片划分编码方案;最后,通过与片划分穷举搜索法进行对比,验证了所提算法在较少的计算复杂度下能取得与穷举搜索法相当的传输效率。
中图分类号:
[1] AHMAD A,MANSOOR A,BARAKABITZE A,et al.Supervised-learning-Based QoE Prediction of Video Streaming in Future Networks:A Tutorial with Comparative Study[J].IEEE Communications Magazine,2021,59(11):88-94. [2] ALWIS C D,ANSHUMAN K,PHAM Q V,et al.Survey on 6G Frontiers:Trends,Applications,Requirements,Technologies and Future Research[J].IEEE Open Journal of the Communications Society,2021,2:836-886. [3] NGUYEN D C,MING D,PATHIRANA P N,et al.6G Internet of Things:A Comprehensive Survey[J].IEEE Internet of Things Journal,2022,9(1):359-383. [4] KAUL A,GUPTA J.Revolutionary 6G:Technologies,Archi-tecture,Coverage,and Performance[C]//Proceedings of International Conference on Computing Communication and Net-working Technologies.New York:IEEE Press,2021:1-6. [5] SKUPIN R,SANCHEZ Y,HELLGE C,et al.Tile Based HEVC Video for Head Mounted Displays[C]//Proceedings of IEEE International Symposium on Multimedia.California:IEEE Press,2016:399-400. [6] CHIARIOTTI F.A Survey on 360-Degree Video:Coding,Qua-lity of Experience and Streaming[J].Computer Communications,2021,177(1):133-155. [7] SREEDHAR K K,AMINLOU A,HANNUKSELA M M,et al.Viewport-Adaptive Encoding and Streaming of 360-Degree Video for Virtual Reality Applications[C]//IEEE International Symposium on Multimedia.New York:IEEE Press,2016:583-586. [8] NASRABADI A T,MAHZARI A,BESHAY J D,et al.Adaptive 360-Degree Video Streaming Using Layered Video Coding[C]//IEEE Virtual Reality.California:IEEE Press,2017:347-348. [9] VISHWANATH B,NANJUNDASWAM T,ROSE K.Rota-tional Motion Model for Temporal Prediction in 360 video coding[C]//IEEE 19th International Workshop on Multimedia Signal Processing.United Kingdom:IEEE Press,2017:1-6. [10] FREMEREY S,SINGLA A,MESEBERG K.Avtrack360:An Open Dataset and Software Recording People’s Head Rotations Watching 360° Videos on an HMD[C]//ACM Multimedia System Conference.New York:ACM Press,2018:403-408. [11] XU M,LI C,ZHANG S,et al.State-of-the-Art in 360° Video/Image Processing:Perception,Assessment and Compression[J].IEEE Journal of Selected Topics in Signal Processing,2020,14(1):5-26. [12] SULLIVAN G J,OHM J R,HAN W J,et al.Overview of The High Efficiency Video Coding (HEVC) Standard[J].IEEE Transactions on circuits and systems for video technology,2012,22(12):1649-1668. [13] LI L,YAN N,LI Z,et al.λ-Domain Perceptual Rate Control for 360-Degree Video Compression[J].IEEE Journal of Selected Topics in Signal Processing,2020,14(1):130-145. [14] ZHOU Y,TIAN L,ZHU C,et al.Video Coding Optimization for Virtual Reality 360-Degree Source[J].IEEE Journal of Selected Topics in Signal Processing,2020,14(1):118-129. [15] XU Y,YANG T,TAN Z,et al.FoV-based Coding Optimization for 360-degree Virtual Reality Videos[C] //Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing.Singapore:IEEE Press,2022. [16] LI Y,XU J,CHEN Z.Spherical Domain Rate-Distortion Optimization for 360-Degree Video Coding[C] //Proceedings of IEEE International Conference on Multimedia and Expo.Hong Kong:IEEE Press,2017:709-714. [17] LIU Y,GUO H,ZHU C,et al.Spherical Position DependentRate-Distortion Optimization for 360-degree Video Coding[C]//Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.Lanzhou:IEEE Press,2019:992-996. [18] CHIANG J,YANG C,DEDHIA B,et al.Saliency-Driven Rate-Distortion Optimization for 360-Degree Image Coding[J].Multimedia Tools and Applications,2021,80(6):8309-8329. [19] YANG C,CHIANG J,LIE W.Rate-Distortion Optimization for 360-degree Image Considering Visual Attention[C]//Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.Los Alamitos:IEEE Computer Society,2020:1128-1131. [20] SUN Y,YU L.Coding Optimization Based on Weighted-to-Spherically-Uniform Quality Metric for 360 Video[C]//Proceedings of IEEE Visual Communications and Image Processing.USA:IEEE Press,2017:1-4. [21] LUZ G,ASCENSO J,BRITES C,et al.Saliency-Driven Omnidirectional Imaging Adaptive Coding:Modeling and Assessment[C]//Proceedings of IEEE 19th International Workshop on Multimedia Signal Processing.United Kingdom:IEEE Press,2017:1-6. [22] XIAO M,ZHOU C,LIU Y,et al.Optile:Toward Optimal Tiling in 360-Degree Video Streaming[C]//Proceedings of the 25th ACM International Conference on Multimedia.New York:ACM Press,2017:708-716. [23] YU M,LAKSHMAN H,GIROD B.Content Adaptive Repre-sentations of Omnidirectional Videos for Cinematic Virtual Reality[C]//Proceedings of the 3rd International Workshop on Immersive Media Experiences.New York:ACM Press,2015:1-6. [24] MAVLANKAR A,GIROD B.Spatial-Random-Access-Enabled Video Coding for Interactive Virtual Pan/Tilt/Zoom Functionality[J].IEEE Transactions on Circuits and Systems for Video Technology,2011,21(5):577-588. [25] GUAN Y,ZHENG C,ZHANG X,et al.Pano:Optimizing 360 Video Streaming with A Betterunderstanding of Quality Perception[C]//Proceedings of the ACM Special Interest Group on Data Communication.New York:ACM Press,2019:394-407. [26] SANCHEZ Y,SKUPIN R,HELLGE C,et al.Spatio-Temporal Activity based Tiling for Panorama Streaming[C]//Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video.New York:ACM Press,2017:61-66. [27] OZCINAR C,CABRERA J,SMOLIC A.Omnidirectional Video Streaming Using Visual Attention-Driven Dynamic Tiling for VR[C]//IEEE Visual Communications and Image Processing.New York:IEEE Press,2018:1-4. [28] KAN N,ZOU J,LI C,et al.RAPT360:Reinforcement Learning-Based Rate Adaptation For 360-Degree Video Streaming with Adaptive Prediction and Tiling[J].IEEE Transactions on Circuits and Systems for Video Technology,2021,67(2):409-423. [29] GUO H W,LUO H J,LIU S,et al.Improved R-λ Model Based Rate Control Algorithm[J].Computer Science,2019,46(3):142-147. [30] CHEN L,GAO L,REN J,et al.Adaptive Bitrate Streaming for Energy-Efficiency Mobile Augmented Reality[J].Computer Science,2022,49(1):194-203. [31] OZCINAR C,CABRERA J,SMOLIC A.Visual Attention-Aware Omnidirectional Video Streaming Using Optimal Tiles for Virtual Reality[J].IEEE Journal on Emerging and Selected Topics in Circuits and Systems,2019,9(1):217-230. [32] NGUYEN D V,TRAN H,THANG T.Adaptive Tiling Selection for Viewport Adaptive Streaming of 360-degree Video[J].IEICE Transactions on Information and Systems,2019,102(1):48-51. [33] KATTADIGE C,THILAKARATHNA K.VAD360:Viewport Aware Dynamic 360-Degree Video Frame Tiling[J].arXiv:2105.11563,2021. [34] JEAN L,CONCOLATO C.Tiled-Based Adaptive StreamingUsing MPEG-DASH[J].Multimedia Systems,2016,41(3):1-3. |
[1] | 曲倩文, 车啸平, 曲晨鑫, 李瑾如. 基于信息感知的虚拟现实用户临场感研究 Study on Information Perception Based User Presence in Virtual Reality 计算机科学, 2022, 49(9): 146-154. https://doi.org/10.11896/jsjkx.220500200 |
[2] | Ran WANG, Jiang-tian NIE, Yang ZHANG, Kun ZHU. Clustering-based Demand Response for Intelligent Energy Management in 6G-enabled Smart Grids Clustering-based Demand Response for Intelligent Energy Management in 6G-enabled Smart Grids 计算机科学, 2022, 49(6): 44-54. https://doi.org/10.11896/jsjkx.220400002 |
[3] | 金华, 朱靖宇, 王昌达. 视频隐私保护技术综述 Review on Video Privacy Protection 计算机科学, 2022, 49(1): 306-313. https://doi.org/10.11896/jsjkx.201200047 |
[4] | 成昭炜, 沈航, 汪悦, 王敏, 白光伟. 基于深度强化学习的无人机辅助弹性视频多播机制 Deep Reinforcement Learning Based UAV Assisted SVC Video Multicast 计算机科学, 2021, 48(9): 271-277. https://doi.org/10.11896/jsjkx.201000078 |
[5] | 吉晓祥, 沈航, 白光伟. 异构无线网络中基于非正交多址的可伸缩视频多播机制 Non-orthogonal Multiple Access Enabled Scalable Video Multicast in HetNets 计算机科学, 2021, 48(11): 356-362. https://doi.org/10.11896/jsjkx.200900080 |
[6] | 蔡于涵,熊淑华,孙伟恒,Karn Pradeep,何小海. 基于运动矢量细化的帧率上变换与HEVC结合的视频压缩算法 Video Compression Algorithm Combining Frame Rate Up-conversion with HEVC Standard Based on Motion Vector Refinement 计算机科学, 2020, 47(2): 76-82. https://doi.org/10.11896/jsjkx.190500092 |
[7] | 何晓艺,段凌宇,林巍峣. 基于深度残差网络的HEVC压缩视频增强 Deep Residual Network Based HEVC Compressed Videos Enhancement 计算机科学, 2019, 46(3): 88-91. https://doi.org/10.11896/j.issn.1002-137X.2019.03.011 |
[8] | 郭红伟, 骆洪军, 刘帅, 牛林, 杨波. 一种改进的R-λ模型码率控制算法 Improved R-λ Model Based Rate Control Algorithm 计算机科学, 2019, 46(3): 142-147. https://doi.org/10.11896/j.issn.1002-137X.2019.03.021 |
[9] | 杜秀丽, 胡兴, 陈波, 邱少明. 基于加权非局部相似性的视频压缩感知多假设重构算法 Multi-hypothesis Reconstruction Algorithm of DCVS Based on Weighted Non-local Similarity 计算机科学, 2019, 46(1): 291-296. https://doi.org/10.11896/j.issn.1002-137X.2019.01.045 |
[10] | 任云,程福林,黎洪松. 基于频率敏感三维自组织映射的视差估计算法 Disparity Estimation Algorithm Based on Frequency Sensitive Three-dimensional Self-organizing Map 计算机科学, 2017, 44(Z11): 225-227. https://doi.org/10.11896/j.issn.1002-137X.2017.11A.047 |
[11] | 单娜娜,周巍,段哲民. HEVC中的变换系数熵编码优化算法 Improved Entropy Coding Algorithm for Transform Coefficients in HEVC 计算机科学, 2017, 44(6): 290-293. https://doi.org/10.11896/j.issn.1002-137X.2017.06.051 |
[12] | 张培君,金小娟,王淑慧,周开伦,林涛. 结合全色度HEVC和有损字典算法的屏幕图像编码 Screen Content Coding of Combining Full-chroma HEVC and Lossy Matching Dictionary Coder 计算机科学, 2014, 41(3): 286-292. |
[13] | 陈卓,冯钢,陆毅. 一种分层P2 P流媒体系统重叠网的构建策略 Overlay Construction Policy in Layered P2P Streaming System 计算机科学, 2012, 39(5): 69-74. |
[14] | 张晓星,刘冀伟,胡广大,崔朝辉. 分布式视频编码中的边信息改进算法 Improved Side Information Generation Algorithm in Distributed Video Coding 计算机科学, 2011, 38(11): 275-277. |
[15] | 陆寄远,孙少晖,朝红阳. 联合分数运动估计优化的多参考帧快速选择算法 Fast Algorithm for Multi-frame Selection with Jointly Optimization of Fractional Pixel Motion Estimation 计算机科学, 2010, 37(6): 283-285. |
|