基于深度学习的图像去模糊方法研究进展

doi:10.11896/jsjkx.201200043

Abstract

Abstract: With the increasing development of portable and smart digital imaging devices,the way to capture photos is more convenient and flexible.Digital images play an important role in video surveillance,medical diagnosis,space exploration,and so on.However,the captured images usually contain significant blur and noise due to the limited quality of the camera sensors,the skill of the photographers,and the imaging environments.How to restore the clear images from blurry ones so that they can facilitate the following intelligent analysis tasks is important but challenging.Image deburring is a classical ill-posed problem.Represented methods for this problem include the statistical prior-based methods and data-driven methods.However,conventional statistical prior-based methods have limited ability for modeling the inherent properties of the clear images.The data-driven methods,especially the deep learning methods,provide an effective way to solve image deblurring.This paper focuses on the deep learning-based image deblurring methods.It first introduces the research progress of the image deblurring problem,and then analyzes the challenges of the image deblurring problem.Finally,it discusses the research trends of the image deblurring problem.

Key words: Deep learning, Ill-posed problem, Image deblurring, Motion blur, Prior modeling

CLC Number:

TP391

PAN Jin-shan. Research Progress on Deep Learning-based Image Deblurring[J].Computer Science, 2021, 48(3): 9-13.

References

[1]FERGUS R,SINGH B,HERTZMANN A,et al.Freeman,Re-moving camera shake from a single photograph [J].ACM Transactions on Graphics,2006,25(3):787-794.
[2]KRISHNAN D,TAY T,FERGUS R.Blind deconvolution using a normalized sparsity measure [C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2011:233-240.
[3]LEVIN A,WEISS Y,DURAND F,et al.Efficient marginal likelihood optimization in blind deconvolution[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).2011:2657-2664.
[4]XU L,ZHENG S C,JIA J Y.Unnatural L0 sparse representation for natural image deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2013:1107-1114.
[5]SHAN Q,JIA J Y,AGARWALA A.High-quality motion de-blurring from a single image[J].ACM Transactions on Gra-phics,2008,27(3):73.
[6]MICHAELI T,IRANI M.Blind DEBLURRING Using Internal Patch Recurrence[C]//European Conference on Computer Vision.2014:783-798.
[7]CHEN X G,HE X J,YANG J,et al.An effective document ima-ge deblurring algorithm [C]//IEEE Conference on Computer Vision and Pattern Recognition.2011:369-376.
[8]CHO H,WANG J,LEE S.Text image deblurring using text-specific properties[C]//European Conference on Computer Vision.2012:524-537.
[9]PAN J S,HU Z,SU Z X,et al.Deblurring text images via L0-regularized intensity and gradient prior[C]//IEEE Conference on Computer Vision and Pattern Recognition.2014:2901-2908.
[10]PAN J S,HU Z,SU Z X,et al.Deblurring Face Images with Exemplars[C]//European Conference on Computer Vision.2014:47-62.
[11]PAN J S,SUN D Q,PFISTER H,et al.Deblurring images via dark channel prior [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(10):2315-2328.
[12]ZHANG J W,PAN J S,LAI W S,et al.Learning fully convolutional networks for iterative non-blind deconvolution[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:6969-6977.
[13]ZHANG K,ZUO W M,GU S H,et al.Learning deep CNN denoiser prior for image restoration[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:2808-2817.
[14]LI L,PAN JS,LAI W S,et al.Learning a Discriminative Prior for Blind Image Deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition.2018:6616-6625.
[15]LEVIN A,WEISS Y,DURAND F,et al.Understanding andevaluating blind deconvolution algorithms[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition.2009:1964-1971.
[16]KOHLER R,HIRSCH M,MOHLER B,et al.Stefan Harme-ling:Recording and playback of camera shake:benchmarking blind deconvolution with a real-world database[C]//European Conference on Computer Vision.2012:27-40.
[17]SUN J,CAO W F,XU Z B,et al.Learning a convolutional neural network for non-uniform motion blur removal[C]//IEEE Conference on Computer Vision and Pattern Recognition.2015:769-777.
[18]ZORAN D,WEISS Y.From learning models of natural image patches to whole image restoration[C]//IEEE International Conference on Computer Vision.2011:479-486.
[19]GONG D,YANG J,LIU L Q,et al.From motion blur to motion flow:a deep learning solution for removing heterogeneous motion blur[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:3806-3815.
[20]SCHULER C J,HIRSCH M,HARMELING S,et al.Learning to deblur [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(7):1439-1451.
[21]PAN J S,REN W Q,HU Z,et al.Learning to Deblur Images with Exemplars [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,41(6):1412-1425.
[22]XU L,LU C W,XU Y,et al.Image smoothing via L0 gradient minimization.photograph [J].ACM Transactions on Graphics,2011,30(6):174.
[23]NAH S J,KIM T H,LEE K M.Deep multi-scale convolutional neural network for dynamic scene deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:257-265.
[24]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[25]TAO X,GAO H Y,SHEN X Y,et al.Scale-recurrent network for deep image deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition.2018:8174-8182.
[26]ZHANG J W,PAN J S,REN J S J,et al.Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.2018:2521-2529.
[27]GAO H Y,TAO X,SHEN X Y,et al.Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections[C]//IEEE Conference on Computer Vision and Pattern Recognition.2019:3848-3856.
[28]ZHANG H G,DAI Y C,LI H D,et al.Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition.2019:5978-5986.
[29]ALJADAANY R,PAL D K,SAVVIDES M.Douglas-Rachford Networks:Learning Both the Image Prior and Data Fidelity Terms for Blind Image Deconvolution[C]//IEEE Conference on Computer Vision and Pattern Recognition.2019:10235-10244.
[30]SUIN M,PUROHIT K,RAJAGOPALAN A N.Spatially-At-tentive Patch-Hierarchical Network for Adaptive Motion Deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition.2020:3603-3612.
[31]YUAN Y,SU W,MA D D.Efficient Dynamic Scene Deblurring Using Spatially Variant Deconvolution Network With Optical Flow Guided Training[C]//IEEE Conference on Computer Vision and Pattern Recognition.2020:3552-3561.
[32]KUPYN O,BUDZAN V,MYKHAILYCH M,et al.Deblur-GAN:Blind motion deblurring using conditional adversarial networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.2018:8183-8192.
[33]KUPYN O,MARTYNIUK T,WU J R,et al.DeblurGAN-v2:Deblurring (orders-of-magnitude) faster and better [C]//IEEE International Conference on Computer Vision.2019:8877-8886.
[34]NIMISHA T M,KUMAR S,RAJAGOPALAN A N.Unsupervised Class-Specific Deblurring[C]//European Conference on Computer Vision.2018:358-374.
[35]LU B Y,CHEN J C,CHELLAPPA R.Unsupervised Domain-Specific Deblurring via Disentangled Representations[C]//IEEE Conference on Computer Vision and Pattern Recognition.2019:10225-10234.
[36]ZHANG K H,LUO W H,ZHONG Y R,et al.Deblurring by Realistic Blurring[C]//IEEE Conference on Computer Vision and Pattern Recognition.2020:2734-2743.
[37]PAN J S,LIU Y,DONG J X,et al.Physics-Based Generative Adversarial Models for Image Restoration and Beyond.deblur [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020.
[38]LIN S N,ZHANG J W,PAN J S,et al.Learning to Deblur Face Images via Sketch Synthesis[C]//AAAI Conference on Artificial Intelligence.2020:11523-11530.
[39]SUN L B,CHO S,WANG J,et al.Edge-based blur kernel estimation using patch priors[C]//IEEE International Conference on Computational Photography.2013:1-8.
[40]LAI W S,HUANG J B,HU Z,et al.A comparative study for single image blind deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition.2016:1701-1709.
[41]SU S C,DELBRACIO M,WANG J,et al.Deep video deblurring for hand-held cameras[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:237-246.
[42]NAH S J,BAIK S Y,HONG S,et al.NTIRE 2019 challenge on video deblurring and super-resolution:Dataset and study[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops.2019:1996-2005.
[43]RIM J,LEE H,WON J,et al.Real-World Blur Dataset forLearning and Benchmarking Deblurring Algorithms[C]//European Conference on Computer Vision.2020:184-201.
[44]ASIM M,SHAMSHAD F,AHMED A.Blind Image Deconvolution using Pretrained Generative Priors[C]//British Machine Vision Conference.2019:205.
[45]REN D W,ZHANG K,WANG Q L,et al.Neural Blind Deconvolution Using Deep Priors [C]//IEEE Conference on Compu-ter Vision and Pattern Recognition.2020:3338-3347.
[46]PAN J S,BAI H R,TANG J H.Cascaded Deep Video Deblurring Using Temporal Sharpness Prior[C]//IEEE Conference on Computer Vision and Pattern Recognition.2020:3040-3048.
[47]ZHOU S C,ZHANG J W,PAN J S,et al.Spatio-temporal filter adaptive network for video deblurring[C]//IEEE International Conference on Computer Vision.2019:2482-2491.
[48]NAH S J,SON S Y,LEE K M.Recurrent neural networks with intra-frame iterations for video deblurring[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition.2019:8102-8111.
[49]KIM T H,LEE K M,SCHOLKOPF B,et al.Online video deblurring via dynamic temporal blending network[C]//IEEE International Conference on Computer Vision.2017:4058-4067.
[50]KIM T H,SAJJADI M S M,HIRSCH M,et al.Spatio-temporal transformer network for video restoration[C]//European Conference on Computer Vision.2018:111-127.
[51]JIANG Z,ZHANG Y,ZOU D Q,et al.Learning Event-BasedMotion Deblurring[C]//IEEE Conference on Computer Vision and Pattern Recognition.2020:3317-3326.
[52]LIN S N,ZHANG J W,PAN J S,et al.Learning Event-Driven Video Deblurring and Interpolation[C]//European Conference on Computer Vision.2020:695-710.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Research Progress on Deep Learning-based Image Deblurring

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2]	TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3]	XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4]	WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5]	HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6]	JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7]	SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8]	HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[9]	CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[10]	HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[11]	ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[12]	SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[13]	WANG Jun-feng, LIU Fan, YANG Sai, LYU Tan-yue, CHEN Zhi-yu, XU Feng. Dam Crack Detection Based on Multi-source Transfer Learning [J]. Computer Science, 2022, 49(6A): 319-324.
[14]	CHU Yu-chun, GONG Hang, Wang Xue-fang, LIU Pei-shun. Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4 [J]. Computer Science, 2022, 49(6A): 337-344.
[15]	LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.