计算机科学 ›› 2019, Vol. 46 ›› Issue (3): 88-91.doi: 10.11896/j.issn.1002-137X.2019.03.011

• 2018 中国多媒体大会 • 上一篇    下一篇

基于深度残差网络的HEVC压缩视频增强

何晓艺1,段凌宇2,林巍峣1   

  1. (上海交通大学电子信息与电气工程学院 上海 200240)1
    (北京大学数字视频编解码技术国家工程实验室 北京 100871)2
  • 收稿日期:2018-07-05 修回日期:2018-09-21 出版日期:2019-03-15 发布日期:2019-03-22
  • 通讯作者: 林巍峣(1980-),男,博士,教授,主要研究方向为视频压缩编码、计算机视觉,E-mail:wylin@sjtu.edu.cn
  • 作者简介:何晓艺(1995-),男,硕士生,主要研究方向为视频压缩编码、计算机视觉,E-mail:xiaoyi.he@outlook.com;段凌宇(1975-),男,教授,主要研究方向为图像识别与多媒体大数据分析
  • 基金资助:
    国家自然科学基金项目(61471235),上海市“一带一路”青年科学家交流国际合作项目(17510740100),Ng Teng Fong慈善基金PKU-NTU联合研究中心项目(JRI)资助

Deep Residual Network Based HEVC Compressed Videos Enhancement

HE Xiao-yi1,DUAN Ling-yu2,LIN Wei-yao1   

  1. (School of Electronic Information and Electrical Engineering,Shanghai Jiao Tong University,Shanghai 200240,China)1
    (National Engineering Laboratory for Video Technology,Peking University,Beijing 100871,China)2
  • Received:2018-07-05 Revised:2018-09-21 Online:2019-03-15 Published:2019-03-22

摘要: 文中提出了一种基于深度残差网络的HEVC压缩视频增强方法。该方法利用一系列级联的残差模块来完成特征提取,然后基于这些特征进行视频的质量增强。与现有的方法相比,所提方法能够捕捉到压缩视频帧更清晰和泛化的特征。实验结果表明,所提方法在20个通用的测试视频序列上能够实现平均6.92%的BD-rate增益,是所有参与比较的方法中效果最好的。

关键词: 残差网络, 高效率视频编码, 卷积神经网络, 压缩视频增强

Abstract: This paper proposed a HEVC-compressed videos enhancement method based on deep residual network.This method utilizes several stacked residual blocks to achieve feature extraction,followed by feature enhancement and reconstruction.Compared with the existing methods which only use a few convolutional layers,the proposed method can capture the feature of input compressed frames in a more distinctive and stable way.Experimental results show that the proposed method leads to over 6.92% BD-rate saving on 20 benchmark sequences and achieves the best performance among the compared methods.

Key words: Compressed videos enhancement, Convolutional neural network, High efficiency video coding, Residual network

中图分类号: 

  • TP751
[1]SULLIVAN G J,OHM J,HAN W J,et al.Overview of the high efficiency video coding (HEVC) standard[J].IEEE Transactions on Circuits and Systems for Video Technology,2012,22(12):1649-1668.
[2]OHM J R,SULLIVAN G J,SCHWARZ H,et al.Comparison of the coding efficiency of video coding standards-including high efficiency video coding (HEVC)[J].IEEE Transactions on circuits and systems for video technology,2012,22(12):1669-1684.
[3]DONG C,DENG Y,CHANGE LOY C,et al.Compression artifacts reduction by a deep convolutional network[C]∥Procee-dings of the IEEE International Conference on Computer Vision.2015:576-584.
[4]PARK W S,KIM M.CNN-based in-loop filtering for coding efficiency improvement[C]∥Image,Video,and Multidimensional Signal Processing Workshop (IVMSP).IEEE,2016:1-5.
[5]DONG C,LOY C C,HE K,et al.Learning a deep convolutional network for image super-resolution[C]∥European Conference on Computer Vision.Cham:Springer,2014:184-199.
[6]DAI Y,LIU D,WU F.A convolutional neural network approach for post-processing in HEVC intra coding[C]∥International Conference on Multimedia Modeling.Cham:Springer,2017:28-39.
[7]YANG R,XU M,WANG Z,et al.Enhancing Quality for HEVC Compressed Videos[J].arXiv preprint arXiv:1709.06734,2017.
[8]WANG T,CHEN M,CHAO H.A novel deep learning-based
method of improving coding efficiency from the decoder-end for hevc[C]∥Data Compression Conference (DCC),2017.IEEE,2017:410-419.
[9]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv preprint arXiv:1409.1556,2014.
[10]HE K,SUN J.Convolutional neural networks at constrained
time cost[C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2015:5353-5360.
[11]HE K,ZHANG X,REN S,et al.Deep residual learning for ima-
ge recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[12]GROSS S,WILBER M.Training and investigating residual nets[EB/OL].http://torch.ch/blog/2016/02/04/resnets.html 2016.
[13]SCHWARZ H,MARPE D,WIEGAND T.Analysis of hierarchi-
cal B pictures and MCTF[C]∥2006 IEEE International Confe-rence on Multimedia and Expo.IEEE,2006:1929-1932.
[14]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv preprint arXiv:1412.6980,2014.
[15]IOFFE S,SZEGEDY C.Batch normalization:Accelerating deep network training by reducing internal covariate shift[J].arXiv preprint arXiv:1502.03167,2015.
[16]ABADI M,BARHAM P,CHEN J,et al.TensorFlow:A System for Large-Scale Machine Learning[C]∥Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation.2016:265-283.
[17]BOSSEN F.Common test conditions and software reference con-
figurations[C]∥12th Joint Collaborative Team on Video Coding Meeting.2011.
[18]BJONTEGARRD G.Calculation of average PSNR differences
between RD-curves:ITU-T SG16/Q6[R].VCEG-M33,Austin,US,2001.
[1] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[2] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[3] 王馨彤, 王璇, 孙知信.
基于多尺度记忆残差网络的网络流量异常检测模型
Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network
计算机科学, 2022, 49(8): 314-322. https://doi.org/10.11896/jsjkx.220200011
[4] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[5] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[6] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[7] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[8] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[9] 刘月红, 牛少华, 神显豪.
基于卷积神经网络的虚拟现实视频帧内预测编码
Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network
计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179
[10] 徐鸣珂, 张帆.
Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法
Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition
计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085
[11] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[12] 高荣华, 白强, 王荣, 吴华瑞, 孙想.
改进注意力机制的多叉树网络多作物早期病害识别方法
Multi-tree Network Multi-crop Early Disease Recognition Method Based on Improved Attention Mechanism
计算机科学, 2022, 49(6A): 363-369. https://doi.org/10.11896/jsjkx.210500044
[13] 张嘉淏, 刘峰, 齐佳音.
一种基于Bottleneck Transformer的轻量级微表情识别架构
Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer
计算机科学, 2022, 49(6A): 370-377. https://doi.org/10.11896/jsjkx.210500023
[14] 王建明, 陈响育, 杨自忠, 史晨阳, 张宇航, 钱正坤.
不同数据增强方法对模型识别精度的影响
Influence of Different Data Augmentation Methods on Model Recognition Accuracy
计算机科学, 2022, 49(6A): 418-423. https://doi.org/10.11896/jsjkx.210700210
[15] 孙洁琪, 李亚峰, 张文博, 刘鹏辉.
基于离散小波变换的双域特征融合深度卷积神经网络
Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation
计算机科学, 2022, 49(6A): 434-440. https://doi.org/10.11896/jsjkx.210900199
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!