基于残差的端对端图像超分辨率

doi:10.11896/j.issn.1002-137X.2019.06.037

Abstract

Abstract: Image super-resolution reconstruction technology is widely used in real life.An end-to-end deep convolutional neural network (CNN) based on residual learning wasproposed to solve the problems of simple network structure,slow convergence rate and reconstructed texture blur in the network super-resolution CNN to further improve the quality of image reconstruction.The network is jointly trained by the local residual network and the global residual network,which increases the width of the network and learns different effective features.The local residual network includes three stages:feature extraction,upsampling and multi-scale reconstruction.The effective local features are extracted by densely concatenated blocks and the rich context information is obtained by multi-scale reconstruction,which is beneficial to the recovery of high-frequency information.In the global residual network,progressive upsampling is used to achieve multi-scale image reconstruction,and the convergence speed is improved by residual learning.Quantitative and qualitative evaluations are performed on the benchmark datasets Set5,Set14,B100,and Urban100 for scale factor of 2,3,and 4.The proposed algorithm shows improved performances by 34.70dB/0.9295,30.54dB/0.8490,29.27dB/0.8096,and 28.81 dB/0.8653 on scale factor of 3.In terms of qualitative comparison,the proposed method reconstructs a clearer image,and preserves the edge details in the image better.The experimental results show that the proposed me-thod has been greatly improved in subjective vision and objective quantization,which can improve the quality of image reconstruction effectively.

Key words: Convolutional neural network, End-to-end, Joint training, Residual learning, Super resolution

CLC Number:

TP391

HUA Zhen, ZHANG Hai-cheng, LI Jin-jiang. End-to-end Image Super Resolution Based on Residuals[J].Computer Science, 2019, 46(6): 246-255.

References

[1]BAKER S,KANADE T.Limits on super-resolution and how tobreak them[J].IEEE Transactions on Pattern Analysis andMachine Intelligence,2002,24(9):1167-1183.
[2]XIAO J S,XU Z M,LI S,et al.Interpolation algorithm based on Improved adaptive shock filter image super-resolution[J].Chinese Journal of Computers,2015,38(6):1131-1139.(in Chinese)
肖进胜,饶天宇,贾茜,等.改进的自适应冲击滤波图像超分辨率插值算法[J].计算机学报,2015,38(6):1131-1139.
[3]QIN X J,SHAN Y Y,XIAO J J,et al.Self-learning single image super-resolution reconstruction based on compressive sensing and SVR[J].Computer Science,2017,44(S2):169-174.(in Chinese)
秦绪佳,单扬洋,肖佳吉,等.基于压缩感知和SVR的自学习单幅图像超分辨率重建[J].计算机科学,2017,44(S2):169-174.
[4]HUANG J B,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.LosAlamitos:IEEE Computer Society Press,2015:5197-5206.
[5]XIAO J S,GAO W,PENG H,et al.Detail enhancement image super-resolution algorithm based on SVDand local self-similarity[J].Chinese Journal of Computers,2016,39(7):1393-1406.(in Chinese)
肖进胜,高威,彭红,等.基于局部自相似性和奇异值分解的超采样图像细节增强[J].计算机学报,2016,39(7):1393-1406.
[6]FREEMAN W T,PASZTOR E C,CARMICHAEL O T.Lear-ning low-levelvision[J].International Journal of Computer Vision,2000,40(1):25-47 [7]CHANG H,XIONG Y M,YEUNG D Y.Super-resolution through neighbor embedding[C]∥Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2004:275-282 [8]YANG J H,WRIGHT J,HUANG T S,et al.Image super-resolutionvia sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873 [9]LI J H,WU Y R,LV J J.Online single image super-resolution algorithm based on group sparse representation[J].Computer Science,2018,45(4):312-318.(in Chinese)
李键红,吴亚榕,吕巨建.基于组稀疏表示的在线单帧图像超分辨率算法[J].计算机科学,2018,45(4):312-318.
[10]YANG C Y,YANG M H.Fast Direct Super-Resolution by Simple Functions[C]∥Proceedings of the IEEE International Conference on Computer Vision.IEEE,2014:561-568.
[11]TIMOFTE R,DE V,VAN GOOL L.Anchored neighborhood regression for fast example-based super-resolution[C]∥Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2013:1920-1927 [12]TIMOFTE R,SMET V D,GOOL L V.A+:Adjusted Anchored Neighborhood Regression for Fast Super-Resolution[C]∥Proceedings of the Asian Conference on Computer Vision.Cham:Springer,2014:111-126.
[13]DONG C,CHEN C L,HE K,et al.Learning a Deep Convolutional Network for Image Super-Resolution[C]∥Proceedings of the European Conference on Computer Vision.Cham:Springer,2014:184-199.
[14]BENGIO Y,SIMARD P,FRASCONI P.Learning long-term dependencies with gradient descent is difficult[J].IEEE Transactions on Neural Networks,2002,5(2):157-166.
[15]HE K,ZHANG X,REN S,et al.Deep Residual Learning for Ima-ge Recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamites:IEEE Computer Society,2015:770-778.
[16]KIM J,LEE J K,LEE K M.Accurate Image Super-Resolution Using Very Deep Convolutional Networks[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:1646-1654.
[17]LAI W S,HUANG J B,AHUJA N,et al.Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamites:IEEE Computer Society,2017:5835-5843.
[18]LIM B,SON S,KIM H,et al.Enhanced Deep Residual Networks for Single Image Super-Resolution[C]∥Proceedings of the Computer Vision and Pattern Recognition Workshops.IEEE,2017:1132-1140.
[19]HARIS M,SHAKHNAROVICH G,UKITA N.Deep Back-Projection Networks For Super-Resolution[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018.
[20]HU Y,GAO X,LI J,et al.Single Image Super-Resolution via Cascaded Multi-Scale Cross Network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018.
[21]ZAGORUYKO S,KOMODAKIS N.DiracNets:Training Very Deep Neural Networks Without Skip-Connections[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017.
[22]WANG Y,WANG L,WANG H,et al.End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks[C]∥Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.IEEE,2016.
[23]REN H,EL-KHAMY M,LEE J.Image Super Resolution Based on Fusing Multiple Convolution Neural Networks[C]∥Proceedings of the Computer Vision & Pattern Recognition.IEEE,2017.
[24]ZHANG Y,TIAN Y,KONG Y,et al.Residual Dense Network for Image Super-Resolution[C]∥Proceedings of the Computer Vision and Pattern Recognition,2018.
[25]KIM J,LEE J K,LEE K M.Deeply-Recursive Convolutional Network for Image Super-Resolution[C]∥Proceedings of the Computer Vision and Pattern Recognition.IEEE,2016:1637-1645.
[26]SHI W,CABALLERO J,HUSZR F,et al.Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2016:1874-1883.
[27]DONG C,CHEN C L,TANG X.Accelerating the Super-Resolution Convolutional Neural Network[C]∥Proceedings of the IEEE International Conference on Computer Vision.IEEE,2016:391-407.
[28]XU B,WANG N Y,CHEN T Q,et al.Empirical evaluationof rectifled activations in convolutional network[C]∥Proceedings of the 32th International Conference on Machine Learning:Deep Learning Workshop.Lille,France:ICML,2015.
[29]LEDIG C,THEIS L,HUSZáR F,et al.Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2017:105-114.
[30]HUANG G,LIU Z,MAATEN L V D,et al.Densely Connected Convolutional Networks[C]∥Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.IEEE Computer Society,2017:2261-2269.
[31]TONG T,LI G,LIU X,et al.Image Super-Resolution Using Dense Skip Connections[C]∥IEEE International Conference on Computer Vision.IEEE Computer Society,2017:4809-4817.
[32]NAIR V,HINTON G E.Rectified linear units improve restric-ted boltzmann machines[C]∥Proceedings of the International Conference on Machine Learning.Omnipress,2010:807-814.
[33]SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the Inception Architecture for Computer Vision[C]∥Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.IEEE,2016:2818-2826.
[34]SZEGEDY C,LIU W,JIA Y,et al.Going Deeper with Convolutions[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015.
[35]BENGIO Y,SIMARD P,FRASCONI P.Learning long-term dependencies with gradient descent is difficult[J].IEEE Transactions on Neural Networks,2002,5(2):157-166.
[36]BRUHN A,WEICKERT J,SCHN,et al.Lucas/Kanade meets Horn/Schunck:combining local and global optic flow methods[J].International Journal of Computer Vision,2005,61(3):211-231.
[37]PASCANU R,MIKOLOV T,BENGIO Y.On the difficulty of training recurrent neural networks[C]∥Proceedings of the International Conference on International Conference on Machine Learning.JMLR.org,2013:1310-1318.
[38]TIMOFTE R,AGUSTSSON E,GOOL L V,et al.NTIRE 2017 Challenge on Single Image Super-Resolution:Methods and Results[C]∥Proceedings of the Computer Vision and Pattern Re-cognition Workshops.IEEE,2017:1110-1121.
[39]BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Low-Complexity Single Image Super-Resolution Based on Nonnegative Neighbor Embedding[C]∥Proceedings of the British Machine Vision Conference.BMVA Press,2012:135.1-135.10.
[40]ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]∥Proceedings of the International Conference on Curves and Surfaces.Berlin Heidelberg:Springer,2010:711-730.
[41]MARTIN D R,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]∥Proceedings of the IEEE International Conference on Computer Vision.IEEE,2002:1110-1121.
[42]HUANG J B,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]∥Proceedings of theComputer Vision and Pattern Recognition.IEEE,2015:5197-5206.
[43]JIA Y Q,SHELHAMER,et al.Caffe:Convolutional Architec-ture for Fast Feature Embedding[C]∥Proceedings of the ACM International Conference Multimedia.ACM,2014:675-678.
[44]HE K,ZHANG X,REN S,et al.Delving Deep into Rectifiers:Surpassing Human-Level Performance on ImageNet Classification[C]∥Proceedings of the IEEE International Conference on Computer Vision.IEEE Computer Society,2015:1026-1034.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

End-to-end Image Super Resolution Based on Residuals

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[2]	CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[3]	ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[4]	DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[5]	LIU Yue-hong, NIU Shao-hua, SHEN Xian-hao. Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(7): 127-131.
[6]	XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[7]	LI Sun, CAO Feng. Analysis and Trend Research of End-to-End Framework Model of Intelligent Speech Technology [J]. Computer Science, 2022, 49(6A): 331-336.
[8]	YANG Yue, FENG Tao, LIANG Hong, YANG Yang. Image Arbitrary Style Transfer via Criss-cross Attention [J]. Computer Science, 2022, 49(6A): 345-352.
[9]	YANG Jian-nan, ZHANG Fan. Classification Method for Small Crops Combining Dual Attention Mechanisms and Hierarchical Network Structure [J]. Computer Science, 2022, 49(6A): 353-357.
[10]	WU Zi-bin, YAN Qiao. Projected Gradient Descent Algorithm with Momentum [J]. Computer Science, 2022, 49(6A): 178-183.
[11]	ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[12]	WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[13]	SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
[14]	ZHAO Zheng-peng, LI Jun-gang, PU Yuan-yuan. Low-light Image Enhancement Based on Retinex Theory by Convolutional Neural Network [J]. Computer Science, 2022, 49(6): 199-209.
[15]	LIU Lin-yun, CHEN Kai-yan, LI Xiong-wei, ZHANG Yang, XIE Fang-fang. Overview of Side Channel Analysis Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(5): 296-302.