计算机科学 ›› 2023, Vol. 50 ›› Issue (1): 147-155.doi: 10.11896/jsjkx.211100161
赵倩, 周冬明, 杨浩, 王长城
ZHAO Qian, ZHOU Dongming, YANG Hao, WANG Changchen
摘要: 动态场景下的非均匀盲去模糊是一个极具挑战性的计算机视觉问题。虽然基于深度学习的去模糊算法已经取得很大进展,但仍存在去模糊不彻底和细节丢失等问题。针对这些问题,提出了一种基于残差注意力和多特征融合的去模糊网络。与现有的单分支网络结构不同,所提网络由两个独立的特征提取子网组成。主干网络采用基于U-Net结构的编码器-解码器网络来获取不同层级的图像特征,并使用残差注意力模块对特征进行筛选,从而自适应地学习图像的轮廓特征和空间结构特征。另外,为了补偿主干网络中下采样操作和上采样操作造成的信息损失,进一步利用具有大感受野的深层次加权残差密集子网来提取特征图的细节信息。最后,使用多特征融合模块逐步融合原分辨率模糊图像以及主干网络和加权残差密集子网生成的特征信息,使得网络能够以整体的方式自适应地学习更有效的特征来复原模糊图像。为了评估网络的去模糊效果,在基准数据集GoPro数据集和HIDE数据集上进行了测试,结果表明所提方法能够有效复原模糊图像。与现有方法相比,提出的去模糊算法在视觉效果上和客观评价指标上均取得了很好的去模糊效果。
中图分类号:
[1]WANG J,SONG L,LI Z,et al.End-to-end object detection with fully convolutional network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:15849-15858. [2]LI X,HE H,LI X,et al.PointFlow:Flowing semantics through points for aerial image segmentation[C]//Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition.2021:4217-4226. [3]DAI P,WENG R,CHOI W,et al.Learning a proposal classifier for multiple object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:2443-2452. [4]PAN J S.Research progress on deep learning-based image deblurring[J].Computer Science,2021,48(3):9-13. [5]KRISHNAN D,TAY T,FERGUS R.Blind deconvolution using a normalized sparsity measure[C]//CVPR 2011.IEEE,2011:233-240. [6]XU L,ZHENG S,JIA J.Unnatural l0 sparse representation for natural image deblurring[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2013:1107-1114. [7]PAN J,SUN D,PFISTER H,et al.Blind image deblurring using dark channel prior[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1628-1636. [8]NAH S,HYUN KIM T,MU LEE K.Deep multi-scale convolu-tional neural network for dynamic scene deblurring[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3883-3891. [9]TAO X,GAO H,SHEN X,et al.Scale-recurrent network fordeep image deblurring[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2018:8174-8182. [10]KUPYN O,BUDZAN V,MYKHAILYCH M,et al.Deblurgan:Blind motion deblurring using conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8183-8192. [11]WANG J M,LI X F,YE L,et al.Medical Image Deblur Using Generative Adversarial Networks with Channel Attention[J].Computer Science,2021,48(6A):101-106. [12]MIAO S,ZHU Y X.Differentiable Neural Architecture Search Method for Blind Image Deblurring[J].Computer Engineering,2021,47(9):313-320. [13]MATSUSHITA Y,OFEK E,GE W,et al.Full-frame video stabilization with motion inpainting[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(7):1150-1163. [14]CHO S,WANG J,LEE S.Video deblurring for hand-held cameras using patch-based synthesis[J].ACM Transactions on Graphics(TOG),2012,31(4):1-9. [15]HYUN KIM T,MU LEE K.Generalized video deblurring for dynamic scenes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5426-5434. [16]PAN J,BAI H,TANG J.Cascaded deep video deblurring using temporal sharpness prior[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3043-3051. [17]FERGUS R,SINGH B,HERTZMANN A,et al.Removing ca-mera shake from a single photograph[M]//ACM SIGGRAPH 2006 Papers.2006:787-794. [18]SHAN Q,JIA J,AGARWALA A.High-quality motion deblurring from a single image[J].ACM Transactions on Graphics(tog),2008,27(3):1-10. [19]HYUN KIM T,AHN B,MU LEE K.Dynamic scene deblurring[C]//Proceedings of the IEEE International Conference on Computer Vision.2013:3160-3167. [20]HYUN KIM T,MU LEE K.Segmentation-free dynamic scene deblurring[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:2766-2773. [21]BAI Y,CHEUNG G,LIU X,et al.Graph-based blind image deblurring from a single photograph[J].IEEE Transactions on Image Processing,2018,28(3):1404-1418. [22]XU L,REN J S,LIU C,et al.Deep convolutional neural network for image deconvolution[J].Advances in Neural Information Processing Systems,2014,27:1790-1798. [23]SUN J,CAO W,XU Z,et al.Learning a convolutional neuralnetwork for non-uniform motion blur removal[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:769-777. [24]SCHULER C J,HIRSCH M,HARMELING S,et al.Learning to deblur[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(7):1439-1451. [25]KUPYN O,MARTYNIUK T,WU J,et al.Deblurgan-v2:Deblurring(orders-of-magnitude) faster and better[C]//Procee-dings of the IEEE/CVF International Conference on Computer Vision.2019:8878-8887. [26]LIU X,SUGANUMA M,SUN Z,et al.Dual residual networks leveraging the potential of paired operations for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:7007-7016. [27]ZHANG H,DAI Y,LI H,et al.Deep stacked hierarchical multi-patch network for image deblurring[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:5978-5986. [28]PARK D,KANG D U,KIM J,et al.Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training[C]//European Conference on Computer Vision.Cham:Springer,2020:327-343. [29]ZHANG K,LUO W,ZHONG Y,et al.Deblurring by realistic blurring[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2737-2746. [30]CHEN L,LU X,ZHANG J,et al.HINet:Half instance normalization network for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:182-192. [31]TSAI F J,PENG Y T,LIN Y Y,et al.BANet:Blur-aware attention networks for dynamic scene deblurring[J].arXiv:2101.07518,2021. [32]LI F,CONG R,BAI H,et al.Learning deep interleaved networks with asymmetric co-attention for image restoration[J].arXiv:2010.15689,2020. [33]HUYNH-THU Q,GHANBARI M.Scope of validity of PSNR in image/video quality assessment[J].Electronics Letters,2008,44(13):800-801. [34]SHEN Z,WANG W,LU X,et al.Human-aware motion deblurring[C]//Proceedings of the IEEE/CVF International Confe-rence on Computer Vision.2019:5572-5581. [35]WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612. |
[1] | 蔡肖, 陈志华, 盛斌. 基于移位窗口金字塔Transformer的遥感图像目标检测 SPT:Swin Pyramid Transformer for Object Detection of Remote Sensing 计算机科学, 2023, 50(1): 105-113. https://doi.org/10.11896/jsjkx.211100208 |
[2] | 张婧媛, 王宏霞, 何沛松. 基于Transformer的多任务图像拼接篡改检测算法 Multitask Transformer-based Network for Image Splicing Manipulation Detection 计算机科学, 2023, 50(1): 114-122. https://doi.org/10.11896/jsjkx.211100269 |
[3] | 李雪辉, 张拥军, 史殿习, 徐化池, 史燕燕. 融合注意力特征的无锚框视觉目标跟踪方法 AFTM:Anchor-free Object Tracking Method with Attention Features 计算机科学, 2023, 50(1): 138-146. https://doi.org/10.11896/jsjkx.211000083 |
[4] | 郑诚, 梅亮, 赵伊研, 张苏航. 基于双向注意力机制和门控图卷积网络的文本分类方法 Text Classification Method Based on Bidirectional Attention and Gated Graph Convolutional Networks 计算机科学, 2023, 50(1): 221-228. https://doi.org/10.11896/jsjkx.211100095 |
[5] | 周芳泉, 成卫青. 基于全局增强图神经网络的序列推荐 Sequence Recommendation Based on Global Enhanced Graph Neural Network 计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085 |
[6] | 戴禹, 许林峰. 基于文本行匹配的跨图文本阅读方法 Cross-image Text Reading Method Based on Text Line Matching 计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032 |
[7] | 周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026 |
[8] | 熊丽琴, 曹雷, 赖俊, 陈希亮. 基于值分解的多智能体深度强化学习综述 Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization 计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112 |
[9] | 饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277 |
[10] | 汪鸣, 彭舰, 黄飞虎. 基于多时间尺度时空图网络的交通流量预测模型 Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction 计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188 |
[11] | 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153 |
[12] | 孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061 |
[13] | 闫佳丹, 贾彩燕. 基于双图神经网络信息融合的文本分类方法 Text Classification Method Based on Information Fusion of Dual-graph Neural Network 计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042 |
[14] | 姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046 |
[15] | 张颖涛, 张杰, 张睿, 张文强. 全局信息引导的真实图像风格迁移 Photorealistic Style Transfer Guided by Global Information 计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036 |
|