Computer Science ›› 2024, Vol. 51 ›› Issue (4): 243-253.doi: 10.11896/jsjkx.230100140

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Progressive Multi-stage Image Denoising Algorithm Combining Convolutional Neural Network and
Multi-layer Perceptron

XUE Jinqiang1, WU Qin1,2   

  1. 1 School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi,Jiangsu 214122,China
    2 Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence,Wuxi,Jiangsu 214122,China
  • Received:2023-01-31 Revised:2023-05-19 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    National Natural Science Foundation of China(61972180).

Abstract: Among the existing image denoising methods based on deep learning,there are problems at the network architecture dimension that single-stage network is hard to represents feature dependency and it is difficult to reconstruct clear images in complex scenarios.The internal features of multi-stage networks are not tightly connected and the original image details are easily lost.At the basic building block dimension,there are problems that the convolutional layer is difficult to handle cross-level features at large noise levels,and the fully connected layer is difficult to capture the spatial details of the image locality.To solve the above problems,this paper proposes solutions from two aspects.On the one hand,a novel cross-stage gating feature fusion is proposed at the architecture dimension,so as to better connect the shallow features of the first-stage network with the deep features of the second-stage network,promote the interaction of information flow and make the internal correlation of the denoising network closer,while avoiding the loss of original spatial details.On the other hand,a dual-axis shifted block combining convolu-tional neural network(CNN) and multi-layer perceptron(MLP) is proposed,which is applied to low-resolution and multi-channel number feature maps to alleviate the problem of insufficient learning ability of CNN on cross-level feature dependencies in complex noise scenarios.And CNN is used to focus on high-resolution feature maps with low channel number to fully extract the spatial local dependencies of noisy images.Many quantitative and qualitative experiments prove that the proposed algorithm achieves the best peak signal-to-noise ratio(PSNR) and structural similarity(SSIM)denoising indicators with a small number of parameters and computational costs in real-world image denoising and Gaussian noise removal tasks.

Key words: Image processing, Image denoising, Deep learning, Convolutional neural network, Multi-layer perceptron, Feature fusion

CLC Number: 

  • TP391.413
[1]DABOV K,FOI A,KATKOVNIK V,et al.Image denoising by sparse 3-D transform-domain collaborative filtering[J].IEEE Transactions on Image Processing,2007,16(8):2080-2095.
[2]LI G,LI,J,FAN H.Image Denoising Algorithm Based on Adaptive Matching Pursuit[J].Computer Science,2020,47(1):176-185.
[3]ZHAO J,MA Y,LIU S.Image Denoising Optimization Algorithm Combined with Visual Saliency [J].Computer Science,2018,45(2):312-317.
[4]CHANG M,LI Q,FENG H,et al.Spatial-Adaptive Network for Single Image Denoising[C]//European Conference on Computer Vision.2020,Online:Springer:2020:171-187.
[5]CHENG S,WANG Y,HUANG H,et al.NBNet:Noise BasisLearning for Image Denoising with Subspace Projection[C]//IEEE/CVFF Conference on Computer Vision and Pattern Re-cognition.IEEE,2021:4896-4906.
[6]KIM Y,SOH J W,PARK G Y,et al.Transfer learning from synthetic to real-noise denoising with adaptive instance normalization[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,USA:IEEE,2020:3482-3492.
[7]GUO S,YAN Z,ZHANG K,et al.Toward Convolutional Blind Denoising of Real Photo-graphs[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach,USA:IEEE,2019:1712-1722.
[8]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//Medical image Computing and Computer-assisted Intervention.Cham:Springer,2015:234-241.
[9]ISOLA P,ZHU J,ZHOU T,et al.Image-to-Image Translation with Conditional Adversarial Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.USA:IEEE,2017:5967-5976.
[10]ZAMIR S W,ARORA A,KHAN S,et al.Learning enriched features for real image restoration and enhancement[C]//European Conference on Computer Vision.Cham:Springer,2020:492-511.
[11]ANWAR S,BARNERS N.Real Image Denoising With Feature Attention[C]//IEEE/CVF International Conference on Computer Vision.KOR:IEEE,2019:3155-3164.
[12]ZAMIR S W,ARORA A,KHAN S,et al.CycleISP:Real Image Restoration via Improved Data Synthesis[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.USA:IEEE,2020:2693-2702.
[13]ZAMIR S W,ARORA A,KHAN S,et al.Multi-Stage Progressive Image Restoration[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.USA:IEEE,2021:14816-14826.
[14]TU Z,TALEBI H,ZHANG H,et al.MAXIM:Multi-Axis MLP for Image Processing[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New Or-leans,LA,USA,2022:5759-5770.
[15]ILYA O T,NEIL H,ALEXANDER K,et al.MLP-Mixer:An all-MLP Architecture for Vision[J].Advances in Neural Information Processing Systems,2021,34:24261-24272.
[16]LIU H,DAI Z,SO D,et al.Pay Attention to MLPs[J].Advances in Neural Information Processing Systems,2021,34:9204-9215.
[17]HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.USA:IEEE,2016:770-778.
[18]ABDELHAMED A,LIN S,BROWN M S.A high-quality de-noising dataset for smartphone cameras[C]//IEEE Conference on Computer Vision and Pattern Recognition.USA:IEEE,2018:1692-1700.
[19]PLOTZ T,ROTH S.Benchmarking denoising algorithms with real photographs[C]//IEEE Conference on Computer Vision and Pattern Recognition.USA:IEEE,2017:1586-1595.
[20]ZHANG K,ZUO W,CHEN Y,et al.Beyond a Gaussian Denoi-ser:Residual Learning of Deep CNN for Image Denoising[J].IEEE Transactions on Image Processing,2017,26:3142-3155.
[21]MARTIN D R,FOWLKES C C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings Eighth IEEE International Conference on Computer Vision.2001:416-423.
[22]HUANG J,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.CVPR,2015:5197-5206
[23]FRANZEN R.Kodak lossless true color image suite[EB/OL].http://r0k.us/graphics/kodak/.
[24]WU X.Color demosaicking by local directional interpolation and nonlocal adaptive thresholding[J].Journal of Electronic Imaging,2011,20:23016.
[25]REN D,ZUO W,HU Q,et al.Progressive image deraining networks:A better and simpler baseline[C]//CVPR.2019.
[26]LI X,WU J,LIN Z,et al.Recurrent Squeeze-and-ExcitationContext Aggregation Net for Single Image Deraining[C]//European Conference on Computer Vision.Germany:Springer,2018:262-277.
[27]YUE Z,YONG H,ZHAO Q,et al.Variational denoising network:toward blind noise modeling and removal[C]//International Conference on Neural Information Processing Systems.USA:Curran Associates Inc.2019:1690-1701.
[28]YUE Z,ZHAO Q,ZHANG L,et al.Dual Adver-sarial Net-work:Toward Real-world Noise Removal and Noise Generation[C]//Proceedings of the European Conference on Computer Vision.Cham:Springer,2020:41-58.
[29]LIU Z,LIN Y,CAO Y,et al.Swin Transformer:Hierarchical Vision Transformer using Shifted Windows[C]//IEEE/CVF International Con-ference on Computer Vision.Canada:IEEE,2021:9992-10002.
[30]CHO K,MERRIENBOER B,GULCEHRE C,et al.LearningPhrase Representations using RNN Encoder-Decoder for Statistical Machine Translation[C]//Proceedings of the 2014 Confe-rence on Empirical Methods in Natural Language Processing.Qatar:Association for Computational Linguistics,2014:1724-1734.
[31]JOSUE A,ADRIAN B.RENOIR-A dataset for real low-light image noise reduction[J].Journal of Visual Communication and Image Representation,2018,51:144-154.
[32]AGUSTSSON E,TIMOFTE R.NTIRE 2017 Challenge on Single Image Super-Resolution:Dataset and Study[C]//Procee-dings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).2017:1122-1131.
[33]WANG Z.Waterloo Exploration Database[EB/OL].http://ivc.uwaterloo.ca/database/WaterlooExploration/exploration_database_and_code.rar.
[34]CHEN Y L,HUANG T W,CHANG K H,et al.Quantitative Analysis of Automatic Image Cropping Algorithms:A Dataset and Comparative Study[C]//Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision(WACV).2017:226-234.
[35]FAN C M,LIU T J,LIU K H,et al.Selective Residual M-Net for Real Image Denoising[C]//30th European Signal Processing Conference.IEEE,2022.
[36]WANG Z,CUN X,BAO J,et al.Uformer:A General U-ShapedTransformer for Image Restoration[J].arXiv:2106.03106v1,2021.
[37]BAE W,YOO J,YE J C.Beyond Deep Residual Learning for Image Restoration:Persistent Homology-Guided Manifold Simplification[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).2017:1141-1149.
[38]ZHANG K,ZUO W,ZHANG L.FFDNet:Toward a Fast andFlexible Solution for CNN-Based Image Denoising[J].IEEE Transactions on Image Processing,2018,27:4608-4622.
[39]ZHANG K,ZUO W,GU S,et al.Learning Deep CNN Denoiser Prior for Image Restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2017:3929-3938.
[40]JIA X,LIU S,FENG X,et al.FOCNet:A Fractional OptimalControl Network for Image Denoising[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019.
[41]LIU P,ZHANG H,LIAN W,et al.Multi-Level Wavelet Convolutional Neural Networks[J].IEEE Access,2019,7:74973-74985.
[42]REN C,HE X,WANG C,et al.Adaptive Consistency Priorbased Deep Network for Image Denoising[C]//Proceedings of the CVPR.2021.
[43]PENG Y,ZHANG L,LIU S,et al.Dilated Residual Networks with Symmetric Skip Connection for image denoising[J].Neurocomputing 2019,345:67-76.
[44]XIA Z,CHAKRABARTI A.Identifying Recurring Patternswith Deep Neural Networks for Natural Image Denoising[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV).2020.
[45]TIAN C,XU Y,ZUO W.Image denoising using deep CNN with batch renormalization.[J] Neural Networks 2020,121:461-473.
[1] ZHANG Liying, SUN Haihang, SUN Yufa , SHI Bingbo. Review of Node Classification Methods Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(4): 95-105.
[2] CHEN Runhuan, DAI Hua, ZHENG Guineng, LI Hui , YANG Geng. Urban Electricity Load Forecasting Method Based on Discrepancy Compensation and Short-termSampling Contrastive Loss [J]. Computer Science, 2024, 51(4): 158-164.
[3] LIN Binwei, YU Zhiyong, HUANG Fangwan, GUO Xianwei. Data Completion and Prediction of Street Parking Spaces Based on Transformer [J]. Computer Science, 2024, 51(4): 165-173.
[4] WANG Ruiping, WU Shihong, ZHANG Meihang, WANG Xiaoping. Review of Vision-based Neural Network 3D Dynamic Gesture Recognition Methods [J]. Computer Science, 2024, 51(4): 193-208.
[5] SONG Hao, MAO Kuanmin, ZHU Zhou. Algorithm of Stereo Matching Based on GAANET [J]. Computer Science, 2024, 51(4): 229-235.
[6] YAN Wenjie, YIN Yiying. Human Action Recognition Algorithm Based on Adaptive Shifted Graph Convolutional Neural
Network with 3D Skeleton Similarity
[J]. Computer Science, 2024, 51(4): 236-242.
[7] CHEN Jinyin, LI Xiao, JIN Haibo, CHEN Ruoxi, ZHENG Haibin, LI Hu. CheatKD:Knowledge Distillation Backdoor Attack Method Based on Poisoned Neuronal Assimilation [J]. Computer Science, 2024, 51(3): 351-359.
[8] HUANG Kun, SUN Weiwei. Traffic Speed Forecasting Algorithm Based on Missing Data [J]. Computer Science, 2024, 51(3): 72-80.
[9] ZHANG Yang, XIA Ying. Object Detection Method with Multi-scale Feature Fusion for Remote Sensing Images [J]. Computer Science, 2024, 51(3): 165-173.
[10] ZHENG Cheng, SHI Jingwei, WEI Suhua, CHENG Jiaming. Dual Feature Adaptive Fusion Network Based on Dependency Type Pruning for Aspect-basedSentiment Analysis [J]. Computer Science, 2024, 51(3): 205-213.
[11] ZHAO Jiangfeng, HE Hongjie, CHEN Fan, YANG Shubin. Two-stage Visible Watermark Removal Model Based on Global and Local Features for Document Images [J]. Computer Science, 2024, 51(2): 172-181.
[12] ZHANG Feng, HUANG Shixin, HUA Qiang, DONG Chunru. Novel Image Classification Model Based on Depth-wise Convolution Neural Network andVisual Transformer [J]. Computer Science, 2024, 51(2): 196-204.
[13] WANG Yangmin, HU Chengyu, YAN Xuesong, ZENG Deze. Study on Deep Reinforcement Learning for Energy-aware Virtual Machine Scheduling [J]. Computer Science, 2024, 51(2): 293-299.
[14] HUANG Changxi, ZHAO Chengxin, JIANG Xiaoteng, LING Hefei, LIU Hui. Screen-shooting Resilient DCT Domain Watermarking Method Based on Deep Learning [J]. Computer Science, 2024, 51(2): 343-351.
[15] QIAO Fan, WANG Peng, WANG Wei. Multivariate Time Series Classification Algorithm Based on Heterogeneous Feature Fusion [J]. Computer Science, 2024, 51(2): 36-46.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!