计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (4): 243-253.doi: 10.11896/jsjkx.230100140

• 计算机图形学&多媒体 • 上一篇    下一篇

结合卷积神经网络与多层感知机的渐进式多阶段图像去噪算法

薛金强1, 吴秦1,2   

  1. 1 江南大学人工智能与计算机学院 江苏 无锡214122
    2 江苏省模式识别与计算智能工程实验室 江苏 无锡214122
  • 收稿日期:2023-01-31 修回日期:2023-05-19 出版日期:2024-04-15 发布日期:2024-04-10
  • 通讯作者: 吴秦(qinwu@jiangnan.edu.cn)
  • 作者简介:(6201910035@stu.jiangnan.edu.cn)
  • 基金资助:
    国家自然科学基金(61972180)

Progressive Multi-stage Image Denoising Algorithm Combining Convolutional Neural Network and
Multi-layer Perceptron

XUE Jinqiang1, WU Qin1,2   

  1. 1 School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi,Jiangsu 214122,China
    2 Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence,Wuxi,Jiangsu 214122,China
  • Received:2023-01-31 Revised:2023-05-19 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    National Natural Science Foundation of China(61972180).

PDF (PC)

90

摘要： 现有基于深度学习的图像去噪方法中,在网络架构层面存在单阶段网络特征表达能力不足而难以在复杂场景下重构清晰图像,以及多阶段网络内部特征连接不紧密而容易丢失原始图像细节的问题。在基础构建块层面,存在卷积层难以处理较大噪声级别下的跨层次特征,以及全连接层难以捕获图像邻域空间细节的问题。为解决以上问题,从两方面提出解决方法:一方面,在架构层面提出新颖的跨阶段门控特征融合,从而更好地连接一阶段网络的浅层特征与二阶段的深层特征,促进信息流的交互并使得去噪网络内部关联更为紧密,同时避免丢失原始像素细节;另一方面,在基础构建块层面提出结合卷积神经网络和多层感知机特性的双轴特征偏移块,作用于低分辨率多通道数的特征图,从而缓解卷积网络在复杂噪声场景下难以捕获跨层次特征依赖关系的问题,对于高分辨率、少通道数的特征图,使用卷积网络以充分提取噪声图像的空间邻域依赖关系。大量定量与定性实验表明,所提算法在真实世界图像去噪和高斯噪声去除任务中,都以较小的参数量和计算代价取得了最佳的PSNR和SSIM。

关键词: 图像处理, 图像去噪, 深度学习, 卷积神经网络, 多层感知机, 特征融合

Abstract: Among the existing image denoising methods based on deep learning,there are problems at the network architecture dimension that single-stage network is hard to represents feature dependency and it is difficult to reconstruct clear images in complex scenarios.The internal features of multi-stage networks are not tightly connected and the original image details are easily lost.At the basic building block dimension,there are problems that the convolutional layer is difficult to handle cross-level features at large noise levels,and the fully connected layer is difficult to capture the spatial details of the image locality.To solve the above problems,this paper proposes solutions from two aspects.On the one hand,a novel cross-stage gating feature fusion is proposed at the architecture dimension,so as to better connect the shallow features of the first-stage network with the deep features of the second-stage network,promote the interaction of information flow and make the internal correlation of the denoising network closer,while avoiding the loss of original spatial details.On the other hand,a dual-axis shifted block combining convolu-tional neural network(CNN) and multi-layer perceptron(MLP) is proposed,which is applied to low-resolution and multi-channel number feature maps to alleviate the problem of insufficient learning ability of CNN on cross-level feature dependencies in complex noise scenarios.And CNN is used to focus on high-resolution feature maps with low channel number to fully extract the spatial local dependencies of noisy images.Many quantitative and qualitative experiments prove that the proposed algorithm achieves the best peak signal-to-noise ratio(PSNR) and structural similarity(SSIM)denoising indicators with a small number of parameters and computational costs in real-world image denoising and Gaussian noise removal tasks.

Key words: Image processing, Image denoising, Deep learning, Convolutional neural network, Multi-layer perceptron, Feature fusion

中图分类号: 

  • TP391.413
[1]DABOV K,FOI A,KATKOVNIK V,et al.Image denoising by sparse 3-D transform-domain collaborative filtering[J].IEEE Transactions on Image Processing,2007,16(8):2080-2095.
[2]LI G,LI,J,FAN H.Image Denoising Algorithm Based on Adaptive Matching Pursuit[J].Computer Science,2020,47(1):176-185.
[3]ZHAO J,MA Y,LIU S.Image Denoising Optimization Algorithm Combined with Visual Saliency [J].Computer Science,2018,45(2):312-317.
[4]CHANG M,LI Q,FENG H,et al.Spatial-Adaptive Network for Single Image Denoising[C]//European Conference on Computer Vision.2020,Online:Springer:2020:171-187.
[5]CHENG S,WANG Y,HUANG H,et al.NBNet:Noise BasisLearning for Image Denoising with Subspace Projection[C]//IEEE/CVFF Conference on Computer Vision and Pattern Re-cognition.IEEE,2021:4896-4906.
[6]KIM Y,SOH J W,PARK G Y,et al.Transfer learning from synthetic to real-noise denoising with adaptive instance normalization[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,USA:IEEE,2020:3482-3492.
[7]GUO S,YAN Z,ZHANG K,et al.Toward Convolutional Blind Denoising of Real Photo-graphs[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach,USA:IEEE,2019:1712-1722.
[8]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//Medical image Computing and Computer-assisted Intervention.Cham:Springer,2015:234-241.
[9]ISOLA P,ZHU J,ZHOU T,et al.Image-to-Image Translation with Conditional Adversarial Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.USA:IEEE,2017:5967-5976.
[10]ZAMIR S W,ARORA A,KHAN S,et al.Learning enriched features for real image restoration and enhancement[C]//European Conference on Computer Vision.Cham:Springer,2020:492-511.
[11]ANWAR S,BARNERS N.Real Image Denoising With Feature Attention[C]//IEEE/CVF International Conference on Computer Vision.KOR:IEEE,2019:3155-3164.
[12]ZAMIR S W,ARORA A,KHAN S,et al.CycleISP:Real Image Restoration via Improved Data Synthesis[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.USA:IEEE,2020:2693-2702.
[13]ZAMIR S W,ARORA A,KHAN S,et al.Multi-Stage Progressive Image Restoration[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.USA:IEEE,2021:14816-14826.
[14]TU Z,TALEBI H,ZHANG H,et al.MAXIM:Multi-Axis MLP for Image Processing[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New Or-leans,LA,USA,2022:5759-5770.
[15]ILYA O T,NEIL H,ALEXANDER K,et al.MLP-Mixer:An all-MLP Architecture for Vision[J].Advances in Neural Information Processing Systems,2021,34:24261-24272.
[16]LIU H,DAI Z,SO D,et al.Pay Attention to MLPs[J].Advances in Neural Information Processing Systems,2021,34:9204-9215.
[17]HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.USA:IEEE,2016:770-778.
[18]ABDELHAMED A,LIN S,BROWN M S.A high-quality de-noising dataset for smartphone cameras[C]//IEEE Conference on Computer Vision and Pattern Recognition.USA:IEEE,2018:1692-1700.
[19]PLOTZ T,ROTH S.Benchmarking denoising algorithms with real photographs[C]//IEEE Conference on Computer Vision and Pattern Recognition.USA:IEEE,2017:1586-1595.
[20]ZHANG K,ZUO W,CHEN Y,et al.Beyond a Gaussian Denoi-ser:Residual Learning of Deep CNN for Image Denoising[J].IEEE Transactions on Image Processing,2017,26:3142-3155.
[21]MARTIN D R,FOWLKES C C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings Eighth IEEE International Conference on Computer Vision.2001:416-423.
[22]HUANG J,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.CVPR,2015:5197-5206
[23]FRANZEN R.Kodak lossless true color image suite[EB/OL].http://r0k.us/graphics/kodak/.
[24]WU X.Color demosaicking by local directional interpolation and nonlocal adaptive thresholding[J].Journal of Electronic Imaging,2011,20:23016.
[25]REN D,ZUO W,HU Q,et al.Progressive image deraining networks:A better and simpler baseline[C]//CVPR.2019.
[26]LI X,WU J,LIN Z,et al.Recurrent Squeeze-and-ExcitationContext Aggregation Net for Single Image Deraining[C]//European Conference on Computer Vision.Germany:Springer,2018:262-277.
[27]YUE Z,YONG H,ZHAO Q,et al.Variational denoising network:toward blind noise modeling and removal[C]//International Conference on Neural Information Processing Systems.USA:Curran Associates Inc.2019:1690-1701.
[28]YUE Z,ZHAO Q,ZHANG L,et al.Dual Adver-sarial Net-work:Toward Real-world Noise Removal and Noise Generation[C]//Proceedings of the European Conference on Computer Vision.Cham:Springer,2020:41-58.
[29]LIU Z,LIN Y,CAO Y,et al.Swin Transformer:Hierarchical Vision Transformer using Shifted Windows[C]//IEEE/CVF International Con-ference on Computer Vision.Canada:IEEE,2021:9992-10002.
[30]CHO K,MERRIENBOER B,GULCEHRE C,et al.LearningPhrase Representations using RNN Encoder-Decoder for Statistical Machine Translation[C]//Proceedings of the 2014 Confe-rence on Empirical Methods in Natural Language Processing.Qatar:Association for Computational Linguistics,2014:1724-1734.
[31]JOSUE A,ADRIAN B.RENOIR－A dataset for real low-light image noise reduction[J].Journal of Visual Communication and Image Representation,2018,51:144-154.
[32]AGUSTSSON E,TIMOFTE R.NTIRE 2017 Challenge on Single Image Super-Resolution:Dataset and Study[C]//Procee-dings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).2017:1122-1131.
[33]WANG Z.Waterloo Exploration Database[EB/OL].http://ivc.uwaterloo.ca/database/WaterlooExploration/exploration_database_and_code.rar.
[34]CHEN Y L,HUANG T W,CHANG K H,et al.Quantitative Analysis of Automatic Image Cropping Algorithms:A Dataset and Comparative Study[C]//Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision(WACV).2017:226-234.
[35]FAN C M,LIU T J,LIU K H,et al.Selective Residual M-Net for Real Image Denoising[C]//30th European Signal Processing Conference.IEEE,2022.
[36]WANG Z,CUN X,BAO J,et al.Uformer:A General U-ShapedTransformer for Image Restoration[J].arXiv:2106.03106v1,2021.
[37]BAE W,YOO J,YE J C.Beyond Deep Residual Learning for Image Restoration:Persistent Homology-Guided Manifold Simplification[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).2017:1141-1149.
[38]ZHANG K,ZUO W,ZHANG L.FFDNet:Toward a Fast andFlexible Solution for CNN-Based Image Denoising[J].IEEE Transactions on Image Processing,2018,27:4608-4622.
[39]ZHANG K,ZUO W,GU S,et al.Learning Deep CNN Denoiser Prior for Image Restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2017:3929-3938.
[40]JIA X,LIU S,FENG X,et al.FOCNet:A Fractional OptimalControl Network for Image Denoising[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019.
[41]LIU P,ZHANG H,LIAN W,et al.Multi-Level Wavelet Convolutional Neural Networks[J].IEEE Access,2019,7:74973-74985.
[42]REN C,HE X,WANG C,et al.Adaptive Consistency Priorbased Deep Network for Image Denoising[C]//Proceedings of the CVPR.2021.
[43]PENG Y,ZHANG L,LIU S,et al.Dilated Residual Networks with Symmetric Skip Connection for image denoising[J].Neurocomputing 2019,345:67-76.
[44]XIA Z,CHAKRABARTI A.Identifying Recurring Patternswith Deep Neural Networks for Natural Image Denoising[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV).2020.
[45]TIAN C,XU Y,ZUO W.Image denoising using deep CNN with batch renormalization.[J] Neural Networks 2020,121:461-473.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发