计算机科学 ›› 2022, Vol. 49 ›› Issue (1): 194-203.doi: 10.11896/jsjkx.201100107

• 计算机图形学&多媒体 • 上一篇    下一篇

基于自适应码率移动增强现实应用的能效优化研究

陈乐1, 高岭1,2, 任杰3, 党鑫1, 王祎昊1, 曹瑞1, 郑杰1, 王海1   

  1. 1 西北大学信息科学与技术学院/新型网络智能信息服务国家地方联合工程研究中心 西安710127
    2 西安工程大学计算机科学学院/新型网络智能信息服务国家地方联合工程研究中心 西安710600
    3 陕西师范大学计算机科学学院 西安710119
  • 收稿日期:2020-11-16 修回日期:2021-04-12 出版日期:2022-01-15 发布日期:2022-01-18
  • 通讯作者: 高岭(gl@nwu.edu.cn)
  • 作者简介:chenle@stumail.nwu.edu.cn
  • 基金资助:
    国家重点研发计划(2019YFC1521400);国家自然科学基金(61902229,61872294);陕西省国际科技合作计划项目(2020KW-006);中央高校基本科研业务费专项资金资助项目(GK202103084)

Adaptive Bitrate Streaming for Energy-Efficiency Mobile Augmented Reality

CHEN Le1, GAO Ling1,2, REN Jie3, DANG Xin1, WANG Yi-hao1, CAO Rui1, ZHENG Jie1, WANG Hai1   

  1. 1 School of Information Science and Technology,Northwest University,State-Province Joint Engineering and Research Center of Advanced Networking and Intelligent Information Services,Xi'an 710127,China
    2 College of Computer Science State-Province,Xi'an Polytechnic University,Joint Engineering and Research Center of Advanced Networking and Intelligent Information Services,Xi'an 710600,China
    3 School of Computer Science,Shaanxi Normal University,Xi'an 710119,China
  • Received:2020-11-16 Revised:2021-04-12 Online:2022-01-15 Published:2022-01-18
  • About author:CHEN Le,born in 1996,postgraduate.Her main research interests include mobile computing and deep learning.
    GAO Ling,born in 1964,Ph.D,professor,is a member of China Computer Federation.His main research interests include cyber security,network ma-nagement and embedded system.
  • Supported by:
    National Natural Science Foundation of China(61672552) and Science and Technology Program of Guangzhou,China(202002020045).

摘要: 随着移动增强现实(Mobile Augmented Reality,MAR)技术的飞速发展,MAR应用的种类及功能也越来越丰富多样,与此同时用户对MAR应用的视频质量及响应时间也提出了更高的要求。通常来说,MAR应用会将计算密集型任务(目标识别及渲染)卸载到云端或边缘服务器进行处理,并将渲染后的图像下载到移动端。但由于移动网络状态的不稳定性及网络带宽的限制,海量数据的传输将延长MAR应用响应时间,进而增加移动设备的传输能耗开销,严重影响用户使用体验。由此,文中提出了一种基于梯度提升回归(Gradient Boosting Regression,GBR)的自适应码率控制模型。该模型通过感知当前网络环境及拍摄内容,预测用户观感需求并对非关注点部分进行低码率压缩,从而在不影响用户体验的情况下尽可能地降低传输数据量,缩短响应时间。具体来说,通过分析200个热门视频的视频特征,构建视频特征同用户观感需求的内在联系,从而针对不同的用户需求提供合适的视频码率配置,由此达到维持体验、减少时延、节约能耗的目标。实验结果显示,同直接下载渲染后的1080p视频相比,提出的自适应码率控制模型在尽可能维持用户观感体验的前提下,每帧的下载时间平均减少了58%(19.13 ms)。

关键词: 码率自适应控制, 能效优化, 梯度提升回归, 移动增强现实

Abstract: With the development of the mobile augmented reality (MAR),users have higher requirements on video quality and response time on it.MAR applications offload computation-intensive tasks to the cloud or edge servers for processing.In order to provide users with high-quality rendering services,MAR needs to download massive amounts of data from cloud or edge servers.Due to the instability of network condition and the limitation of network bandwidth,data transmission will extend MAR application response time,which increases the energy consumption,and seriously affects the user experience.This paper proposes a bit-rate adaptive model based on gradient boosting regression (GBR).The model considers the different needs of users in different network conditions,analyzes the features of the 200 popular videos,finds the connection between the video features and the user requirements,and provides appropriate video bitrate configuration according to different needs,thus to achieve the goal of maintaining experience,reducing latency and saving energy.The results show that compared with the original rendered 1080p video,the proposed bitrate adaptive model can save 58% downloading time latency(19.13 ms) per frame while maintaining the user's viewing experience as much as possible.

Key words: Bitrate adaptive control, Gradient boosting regression, Mobile augmented reality, Performance optimization

中图分类号: 

  • TP391
[1]CHEN H X,DAI Y T,MENG H,et al.Understanding the cha-racteristics of mobile augmented reality applications[C]//2018 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).Piscataway NJ:IEEE,2018:128-138.
[2]ZHOU Y Q,SUN B L,QI Y L,et al.Mobile AR/VR in 5Gbased on csonvergence of communication and computing[J].Telecommunications Science,2018,34(8):19-33.
[3]LIU G Q.Research on offloading strategy based on mobile edge computing[D].Harbin:Harbin Institute of Technology,2018.
[4]SHAUKAT N.Minimization of energy consumption for mobile edge computing-based augmented reality application[D].Beijing:Beijing University of Post and Telecommunication,2019.
[5]GALIH S,IANSYAH S,YUDISTIRA H.Location and orientation based augmented reality mobile application for enhancing heritage landmark history and information[J].Universal Journal of Electrical and Electronic Engineering,2019,6(2A):39-45.
[6]KICO I,LIAROKAPIS F.Investigating the learning process of folk dances using mobile augmented reality[J].Applied Sciences,2020,10(2):599.
[7]CHEN Y H,RAVINDRANATH L,DENG S,et al.Glimpse:Continuous,real-time object recognition on mobile devices[C]//Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems(Sensys).New York NY:ACM,2015:155-168.
[8]REN J,GAO L,YU J L,et al.Energy-efficient Deep Learning Task Scheduling Strategy for Edge Device[J].Chinese Journal of Computers,2020,43(3):440-452.
[9]LIU L Y,LI H Y,GRUTESER M.Edge assisted real-time object detection for mobile augmented reality[C]//Proc of the the 25th Annual International Conference on Mobile Computing and Networking(MobiCom'19).New York NY:ACM,2019:1-16.
[10]ZHANG W X,LIN S K,BIJARBOONEH F H.CloudAR:A Cloud-based Framework for Mobile Augmented Reality[C]//Proceedings of the on Thematic Workshops of ACM Multimedia(Workshops '17).2017:194-200.
[11]QIAN F,HAN B,XIAO Q Y.Flare:practical viewport-adaptive 360-degree video streaming for mobile devices[C]//:Proc of the 24th Annual International Conference on Mobile Computing and Networking(MobiCom'18).New York NY:ACM,India,2018:99-114.
[12]LENG Y,CHEN C C,SUN Q Y,et al.Energy-efficient videoprocessing for virtual reality[C]//Proceedings of the 46th International Symposium on Computer Architecture (ISCA).New York NY:ACM,2019:91-103.
[13]RUAN X K,CHEN H L,ZHU X D,et al.Deep decision:a mobile deep learning framework for edge video analytics[C]//INFOCOM,IEEE Conference on Computer Communications (INFOCOM).Piscataway NJ:IEEE,2018:1421-1429.
[14]ZHAO H,SHI J,QI X,et al.Pyramid scene parsing network[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway NJ:IEEE,2017:2881-2890.
[15]LI Y,SHI J,LIN D.Low-latency video semantic segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway NJ:IEEE,2018:5997-6005.
[16]GADDE R,JAMPANI V,GEHLER P V.Semantic video cnns through representation warping[C]//The IEEE Interational Conference on Computer Vision (ICCV).Piscataway NJ:IEEE,2017:4453-4462.
[17]ZHU X,XIONG Y,DAI J,et al.Deep feature flow for video re-cognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway NJ:IEEE,2017:2349-2358.
[18]LIU N,HAN J.Deep hierarchical saliency network for salient object detection[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway NJ:IEEE,2016:678-686.
[19]WANG L,LU H,RUAN X,et al.Deep networks for saliencydetection via local estimation and global search[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway NJ:IEEE,2015:3183-3192.
[20]ZHAO R,OUYANG W,LI H,et al.Saliency detection by multi-context deep learning[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway NJ:IEEE,2015:1265-1274.
[21]DONOSER M,URSCHLER M,HIRZER M,et al.Saliencydriven total variation segmentation[C]//The IEEE Interational Conference on Computer Vision(ICCV).Piscataway NJ:IEEE,2009:817-824.
[22]BAI X,WANG W,LIANG J.An active contour model based on region saliency for image segmentation[J].Journal of Computer Research and Development,2012:49(12):2686-2695.
[23]BORJI A,FRINTROP S,SIHITE D L,et al.Adaptive objecttracking by learning background context[C]//2012 IEEE Computer Society Conference on Computer Vision and Pattern Re-cognition Workshops.Piscataway NJ:IEEE,2012:23-30.
[24]GAO Y,WANG M,TAO D,et al.3-D object retrieval and re-cognition with hypergraph analysis[J].IEEE Transactions on Image Processing,2012,21(9):4290-4303.
[25]REN J,WANG X,FANG J.Proteus:network-aware web browsing on heterogeneous mobile systems[C]//Proc of the 14th International Conference on Emerging Networking Experiments and Technologies.New York NY:ACM,2018:379-392.
[26]REN J,YUAN L,NURMI P,et al.Camel:smart,adaptive energy optimization for mobile web interactions[C]//IEEE Confe-rence on Computer Communications (INFOCOM).Piscataway NJ:IEEE,2020:119-128.
[27]QIN Q,REN J,GAO L,et al.To compress,or not to compress:Characterizing deep learning model compression for embedded inference[C]//2018 IEEE Intl Symp on Parallel and Distributed Processing with Applications(ISPA).Piscataway NJ:IEEE,2019:729-736
[28]MARK B,ADRIAN C,GUN L.A survey of augmented reality[M].Boston,Amercia:Now Publishers Foundations and Trends,2015:126-146
[29]HOU Q I,CHEN M M,HU X,et al.Deeply supervised salient object detection with short connections [C]//The IEEEConfe-rence on Computer Vision and Pattern Recognition (CVPR).Piscataway NJ:IEEE,2017:3203-3212
[30]WAN S,YANG Z F.A new generation of efficient video coding H.265/HEVC:principle,standard and implementation[M].Beijing Publishing House of Electronics Industry,2014:2-15.
[31]BT R I.Methodology for the subjective assessment of videoquality in multimedia applications[S].Geneva:The InternationalTelecommunication Union (ITU),2007.
[32]SEVCIK L,BEHAN L,FRNDA J,et al.Prediction of subjective video quality based on objective assessment[C]//2018 26th Te-lecommunications Forum (TELFOR).Piscataway NJ:IEEE,2018:1-4.
[33]JEROME H F.Greedy Function approximation:a gradient boosting machine[J].Annals of Statistics,2001,29(5):1189-1232.
[34]LEO B.Bagging predictors[J].Machine Learning,1996(24):123-140
[35]FREUND Y,SCHAPIRE R E.A decision-theoretic generalization of on-line learning and an application to boosting[J].Journal of Computer and System Sciences,1997,55(1):119-139.
[36]BREIMAN L,FRIEDMAN J H,OLSHEN R A,et al.Classification and regression trees[J].International Biometric Society,1984,40(3):874-874.
[37]BLAU Y,MICHAELI T.The perception-distortion Tradeoff[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway NJ:IEEE,2018:6228-6237.
[38]ANISH M,RAJIV S,ALAN B.Making a “Completely Blind” Image Quality Analyzer[J].IEEE Signal Processing Letters,2013,20(3):209-212.
[39]MA C,YANG C,YANG X.Learning a No-Reference Quality Metric for Single-Image Super-Resolution[J].Computer Vision and Image Understanding,2016,158:1-16.
[1] 赵磊, 周金和.
基于复杂网络内容场的ICN能效优化策略
ICN Energy Efficiency Optimization Strategy Based on Content Field of Complex Networks
计算机科学, 2019, 46(9): 137-142. https://doi.org/10.11896/j.issn.1002-137X.2019.09.019
[2] 朱维军, 王鑫, 钟英辉, 樊永文, 陈永华.
一种基于梯度提升回归树的系外行星宜居性预测方法
Habitability Prediction of Exoplanets Based on GBRT Algorithm
计算机科学, 2019, 46(6A): 71-73.
[3] 沈夏炯, 张俊涛, 韩道军.
基于梯度提升回归树的短时交通流预测模型
Short-term Traffic Flow Prediction Model Based on Gradient Boosting Regression Tree
计算机科学, 2018, 45(6): 222-227. https://doi.org/10.11896/j.issn.1002-137X.2018.06.040
[4] 邢文凯,高雪霞,侯小毛,翟萍.
云计算环境下的模糊解耦能效优化算法研究
Research on Fuzzy Decoupling Energy Efficiency Optimization Algorithm in Cloud Computing Environment
计算机科学, 2017, 44(12): 75-79. https://doi.org/10.11896/j.issn.1002-137X.2017.12.015
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!