Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (11): 258-267.doi: 10.11896/jsjkx.201000033

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Image Super-resolution by Residual Attention Network with Multi-skip Connection

LIU Zun-xiong, ZHU Cheng-jia, HUANG Ji, CAI Ti-jian   

  1. School of Information Engineering,East China Jiaotong University,Nanchang 330013,China
  • Received:2020-10-08 Revised:2021-01-31 Online:2021-11-15 Published:2021-11-10
  • About author:LIU Zun-xiong,born in 1967,Ph.D,professor.His main research interests include deep learning,computer vision,image processing,financial data analysis and machine learning theory algorithm and application.
  • Supported by:
    National Natural Science Foundation of China(61861017) and Science Foundation for Young Scientist of Jiangxi Province,China(20181BAB211013).

PDF (PC)

690

Abstract: Deep convolutional neural networks (Deep CNNs) are difficult to train as they become deeper.Moreover,in image super-resolution,channel-wise features and inputs of the low-resolution (LR) image are treated equally between different channels,resulting in the deficiency of the representational ability of the CNNs.To resolve these issues,residual attention network with multi-skip Connection (RANMC) is proposed for single-image super resolution (SISR),which employs residual in multi-skip connection (RIMC) structure,then a very deep network is formulated with serval residual groups.Each residual group (RG) contains a certain number of short skip connections (SSC) and multi-skip connections (MC).Based on RIMC,rich low-frequency (LF) information is allowed to be bypassed through multi-skip connection,and high-frequency (HF) information is focused on learning by the principal network.Furthermore,considering interdependencies in channel and spatial dimension,attention mechanism block(AMBlock) is proposed to focus on the location of the information and adaptively readjust channel-wise features,where the spatial attention (SA) mechanism and channel attention (CA) mechanism are taken in the approach.Experiments indicate that RANMC can not only recover image details better,but also obtain higher image quality and network performance.

Key words: Attention mechanism block, Image super-resolution, Residual in multi-skip connection, Residual network, Skip connection

CLC Number: 

  • TP391.41
[1]IRANI M,PELEG S.Improving resolution by image registration[J].GVGIP:Graphical Models and Image Processing,1991,53(3):231-239.
[2]BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding[C]//BMVC.2012.
[3]DONG C,LOY C C,HE K,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(2):295-307.
[4]WANG Y,LI A,WANG S Q.Research on the influence oftraining set automatic target recognition under super-resolution of remote sensing images[J].Journal of Chongqing University of Technology(Natural Science),2021,35(2):136-143.
[5]WANG W,HU T,LI X W,et al.Study on Super-resolutionImage Reconstruction of Leukocytes[J].Computer Science,2021,48(4):164-168.
[6]ZOU W W W,YUEN P C.Very Low Resolution Face Recognition Problem[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2012,21(1):327-340.
[7] KARRAS T,AILA T,LAINE S,et al. Progressive Growing of GANs for Improved Quality,Stability,and Variation[J].arXiv:1710.10196,2018.
[8]CABALLERO J.Cardiac image super-resolution with global correspondence using multi-atlas patchmatch[J].Medical Image Computing and Computer-assisted Intervention,2013,16(Pt3):9-16.
[9]TIMOFTE R,DE V,GOOL L V.Anchored Neighborhood Regression for Fast Example-Based Super-Resolution[C]//IEEE International Conference on Computer Vision.IEEE,2014.
[10]TIMOFTE R,VINCENT D S,LUC V G.A+:Adjusted Anchored Neighborhood Regression for Fast Super-Resolution[C]//Asian Conference on Computer Vision.Cham:Springer,2014.
[11]PELEG T,ELAD M.A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution[J].IEEE Transactions on Image Processing,2014,23(6):2569-2582.
[12]LAI W S,HUANG J B,AHUJA N,et al.Fast and AccurateImage Super-Resolution with Deep Laplacian Pyramid Networks[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence.2017.
[13]HUANG J B,SINGH A,AHUJA N.Single Image Super-resolution from Transformed Self-Exemplars[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015.
[14]LEE C Y,XIE S,GALLAGHER P,et al.Deeply-supervised nets[C]//Artificial Intelligence and Statistics.2015:562-570.
[15]LI K,WU Z,PENG K C,et al.Tell me where to look:Guided attention inference network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:9215-9223.
[16]ZHANG K,GAO X,TAO D,et al.Single image super-resolution with non-local means and steering kernel regression[J].IEEE Transactions on Image Processing,2012,21(11):4544-4556.
[17]ZHANG L,WU X.An edge-guided image interpolation algo-rithm via directional filtering and data fusion[J].IEEE transactions on Image Processing,2006,15(8):2226-2238.
[18]CHAO D,CHEN C L,HE K,et al.Learning a Deep Convolutional Network for Image Super-Resolution[C]//ECCV.Sprin-ger International Publishing.2014.
[19]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[20]LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2017:136-144.
[21]TONG T,LI G,LIU X,et al.Image super-resolution usingdense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4799-4807.
[22]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[23]WANG F,JIANG M,QIAN C,et al.Residual attention network for image classification[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2017:3156-3164.
[24]KIM J,KWON L J,MU L K.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1646-1654.
[25]KIM J,KWON L J,MU L K.Deeply-recursive convolutionalnetwork for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1637-1645.
[26]DONG C,LOY C C,TANG X.Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision.Cham:Springer,2016:391-407.
[27]NAIR V,HINTON G E.Rectified linear units improve restric-ted boltzmann machines[C]//ICML.2010.
[28]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time singleimage and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[29]YANG C Y,YANG M H.Fast direct super-resolution by simple functions[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.2013:561-568.
[30]WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[31]LIM B,SON S,KIM H,et al.Enhanced Deep Residual Networks for Single Image Super-Resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).IEEE,2017.
[32]CHATTERJEE S,CHU W T.Thermal Face Recognition Based on Transformation by Residual U-Net and Pixel Shuffle Upsampling[M]//Multimedia Modeling,2020:679-689.
[33]TIMOFTE R,AGUSTSSON E,VAN G L,et al.Ntire 2017challenge on single image super-resolution:Methods and results[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2017:114-125.
[34]BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Low-Complexity Single Image Super-Resolution Based on Nonnegative Neighbor Embedding[C]//BMVC.2012.
[35]ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//International Conference on Curves and Surfaces.Berlin:Springer,2010:711-730.
[36]MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings Eighth IEEE International Conference on Computer Vision.2001:416-423.
[37]HUANG J B,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5197-5206.
[38]MATSUI Y,ITO K,ARAMAKI Y,et al.Sketch-based mangaretrieval using manga109 dataset[J].Multimedia Tools and Applications,2017,76(20):21811-21838.
[39]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[40]WANG Z,LIU D,YANG J,et al.Deep networks for image super-resolution with sparse prior[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:370-378.
[41]TAI Y,YANG J,LIU X.Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3147-3155.
[42]ZHANG K,ZUO W,ZHANG L.Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3262-3271.
[43]HARIS M,SHAKHNAROVICH G,UKITA N.Deep back-projection networks for super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:1664-1673.
[44]TAI Y,YANG J,LIU X,et al.Memnet:A persistent memory network for image restoration[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4539-4547.
[45]LAI W S,HUANG J B,AHUJA N,et al.Fast and accurateimage super-resolution with deep laplacian pyramid networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,41(11):2599-2613.
[46]LAI W S,HUANG J B,AHUJA N,et al.Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:624-632.
[47]LIU D,WEN B,FAN Y,et al.Non-local recurrent network for image restoration[C]//Advances in Neural Information Processing Systems.2018:1673-1682.
[48]ZHANG Y,TIAN Y,KONG Y,et al.Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:2472-2481.
[1] WANG Xin-tong, WANG Xuan, SUN Zhi-xin. Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network [J]. Computer Science, 2022, 49(8): 314-322.
[2] GAO Rong-hua, BAI Qiang, WANG Rong, WU Hua-rui, SUN Xiang. Multi-tree Network Multi-crop Early Disease Recognition Method Based on Improved Attention Mechanism [J]. Computer Science, 2022, 49(6A): 363-369.
[3] HAN Hong-qi, RAN Ya-xin, ZHANG Yun-liang, GUI Jie, GAO Xiong, YI Meng-lin. Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning [J]. Computer Science, 2022, 49(5): 33-42.
[4] ZHAO Ren-xing, XU Pin-jie, LIU Yao. ECG-based Atrial Fibrillation Detection Based on Deep Convolutional Residual Neural Network [J]. Computer Science, 2022, 49(5): 186-193.
[5] QU Zhong, CHEN Wen. Concrete Pavement Crack Detection Based on Dilated Convolution and Multi-features Fusion [J]. Computer Science, 2022, 49(3): 192-196.
[6] LIN Zhen-xian, ZHANG Meng-kai, WU Cheng-mao, ZHENG Xing-ning. Face Image Inpainting with Generative Adversarial Network [J]. Computer Science, 2021, 48(9): 174-180.
[7] GUO Lin, LI Chen, CHEN Chen, ZHAO Rui, FAN Shi-lin, XU Xing-yu. Image Super-resolution Reconstruction Using Recursive ResidualNetwork Based on ChannelAttention [J]. Computer Science, 2021, 48(8): 139-144.
[8] XU Hua-jie, ZHANG Chen-qiang, SU Guo-shao. Accurate Segmentation Method of Aerial Photography Buildings Based on Deep Convolutional Residual Network [J]. Computer Science, 2021, 48(8): 169-174.
[9] BAO Yu-xuan, LU Tian-liang, DU Yan-hui, SHI Da. Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation [J]. Computer Science, 2021, 48(7): 77-85.
[10] WANG Jian-ming, LI Xiang-feng, YE Lei, ZUO Dun-wen, ZHANG Li-ping. Medical Image Deblur Using Generative Adversarial Networks with Channel Attention [J]. Computer Science, 2021, 48(6A): 101-106.
[11] NIU Kang-li, CHEN Yu-zhang, ZHANG Gong-ping, TAN Qian-cheng, WANG Yi-chong, LUO Mei-qi. Vehicle Flow Measuring of UVA Based on Deep Learning [J]. Computer Science, 2021, 48(6A): 275-280.
[12] CHAI Bing, LI Dong-dong, WANG Zhe, GAO Da-qi. EEG Emotion Recognition Based on Frequency and Channel Convolutional Attention [J]. Computer Science, 2021, 48(12): 312-318.
[13] WANG Run-zheng, GAO Jian, HUANG Shu-hua, TONG Xin. Malicious Code Family Detection Method Based on Knowledge Distillation [J]. Computer Science, 2021, 48(1): 280-286.
[14] MENG Li-sha, REN Kun, FAN Chun-qi, HUANG Long. Dense Convolution Generative Adversarial Networks Based Image Inpainting [J]. Computer Science, 2020, 47(8): 202-207.
[15] SONG Ya-fei, CHEN Yu-zhang, SHEN Jun-feng and ZENG Zhang-fan. Underwater Image Reconstruction Based on Improved Residual Network [J]. Computer Science, 2020, 47(6A): 500-504.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.