深度学习驱动的水下图像处理研究进展

doi:10.11896/jsjkx.230400107

Abstract

Abstract: With the development of artificial intelligence and underwater equipments,autonomous underwater vehicles can conveniently obtain underwater images.Underwater images are essential for exploring and developing the ocean.However,due to the complex underwater imaging environment,the acquired underwater images have low image quality,such as low contrast,blurring,and color distortion,making it difficult to meet the requirements of underwater production activities.In recent years,the development of deep learning-based underwater image processing methods and quality evaluation metrics has received much attention from scholars.Although there have been some reviews on deep learning-based underwater image processing methods,there are still issues such as incomplete summarization and a lack of the latest research results.Therefore,this paper first analyzes the causes of underwater image degradation and proposes the necessary processing issues,and classifies underwater image processing methods based on the principles and characteristics of various algorithms.Secondly,the latest research results on deep learning-based underwater image processing are analyzed and summarized,and the main features of various algorithms are summarized.Then,existing publicly available underwater image datasets and current mainstream and latest learning-based underwater image quality evaluation metrics are detailed,and traditional algorithms and deep learning-based underwater image processing methods are compared and analyzed through experimental design.Finally,some unresolved issues in the field of underwater image proces-sing are analyzed and summarized,and future development directions are discussed.

Key words: Deep learning, Autonomous underwater vehicle, Underwater image, Image processing, Image quality evaluation

CLC Number:

TP391

ZHANG Tianchi, LIU Yuxuan. Research Progress of Underwater Image Processing Based on Deep Learning[J].Computer Science, 2024, 51(6A): 230400107-12.

References

[1]YANG B,LIU Y Y,LIAO J W.Manned Submersible:A “National Treasure” for Deep-Sea Scientific Exploration and Ocean Resource Development[J].Bulletin of Chinese Academy of Sciences,2021,36(5):622-631.
[2]LI S,WU Y T,LI C,et al.Applications and Prospects of Underwater Robots[J].Bulletin of Chinese Academy of Sciences,2022,37(7):910-920.
[3]FENG X S,LI Y P,XU H X,et al.Development of Deep-Sea Autonomous Underwater Robots and Their Applications in Resource Investigation[J].The Chinese Journal of Nonferrous Metals,2021,31(10):2746-2756.
[4]XU Y L,DU J H,LEI Z Y,et al.Current status and key technologies review of underwater robots in fishery applications[J].Robot,2023,45(1),110-128.
[5]HU K,WENG C,ZHANG Y,et al.An overview of underwater vision enhancement:from traditional methods to recent deep learning[J].Journal of Marine Science and Engineering,2022,10(2):241.
[6]LUO H L,AO Y,YUAN P.A Method of Image RestorationBased on Generative Adversarial Network[J].Acta Electronica Sinica,2020,48(10):1891-1898.
[7]FU X,CAO X.Underwater image enhancement with global-local networks and compressed-histogram equalization[J].Signal Processing:Image Communication,2020,86:115892.
[8]HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[9]JAFFE J S.Computer modeling and the design of optimal underwater imaging systems[J].IEEE Journal of Oceanic Engineering,1990,15(2):101-111.
[10]SCHECHNER Y Y,KARPEL N.Recovery of underwater visibility and structure by polarization analysis[J].IEEE Journal of Oceanic Cngineering,2005,30(3):570-587.
[11]CONG R M,ZHANG Y M,ZHANG C,et al.Research Progress on Deep Learning-Driven Underwater Image Enhancement and Restoration[J].Signal Processing,2020,36(9):1377-1389.
[12]ZHOU J,YANG T,ZHANG W.Underwater vision enhance-ment technologies:A comprehensive review,challenges,and recent trends[J].Applied Intelligence,2023,53(3):3594-3621.
[13]FENG F,WU G J,WU Y F,et al.Underwater Polarization Imaging Algorithm Based on Global Estimation[J].Acta Optica Sinica,2020,40(21):75-83.
[14]SHPILMAN B,ABOOKASIS D.Experimental results of imaging objects in turbid liquid integrating multiview circularly polarized speckle images and deconvolution method[J].Optics & Laser Technology,2020,121:105774.
[15]XIANG Y,YANG X,REN Q,et al.Underwater PolarizationImaging Recovery Based on Polarimetric Residual Dense Network[J].IEEE Photonics Journal,2022,14(6):1-6.
[16]HE K,SUN J,TANG X.Single image haze removal using dark channel prior[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(12):2341-2353.
[17]DREWS P,NASCIMENTO E,MORAES F,et al.Transmission estimation in underwater single images[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2013:825-830.
[18]BERMAN D,LEVY D,AVIDAN S,et al.Underwater singleimage color restoration using haze-lines and a new quantitative dataset[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(8):2822-2837.
[19]ZHOU Y,GU X T,LI Q W.Underwater Image Restoration Based on Background Light Correction Imaging Model[J].Journal of Electronics & Information Technology,2022,44(10):3363-3371.
[20]QI P,LI X,HAN Y,et al.U2R-pGAN:Unpaired underwater-image recovery with polarimetric generative adversarial network[J].Optics and Lasers in Engineering,2022,157:107112.
[21]HUMMEL R.Image enhancement by histogram transformation[J].Computer Graphics and Image Processing,1977,6(2):184-195.
[22]PIZER S M,AMBURN E P,AUSTIN J D,et al.Adaptive histogram equalization and its variations[J].Computer Vision,Graphics,and Image Processing,1987,39(3):355-368.
[23]REZA A M.Realization of the contrast limited adaptive histogram equalization(CLAHE) for real-time image enhancement[J].Journal of VLSI Signal Processing Systems for Signal,Image and Video Technology,2004,38:35-44.
[24]LI X,HOU G,TAN L,et al.A hybrid framework for underwater image enhancement[J].IEEE Access,2020,8:197448-197462.
[25]LEI X Y,ZHANG W D,PAN X P.Underwater Image Color Correction Method Based on Histogram Distribution Characteristics[J].Computer Engineering and Design,2022,43(8):2284-2293.
[26]LAND E H,MCCANN J J.Lightness and retinex theory[J].Josa,1971,61(1):1-11.
[27]FU X,ZHUANG P,HUANG Y,et al.A retinex-based enhancing approach for single underwater image[C]//2014 IEEE International Conference on Image Processing(ICIP).IEEE,2014:4572-4576.
[28]SHI L,XI M L,SUN J.Underwater Image Enhancement Algorithm Based on Controllable Kernel Bilateral Filtering Retinex[J].Chinese Journal of Quantum Electronics,2018,35(1):7-12.
[29]ZOU L,LU J Y,HU Y,et al.Underwater Image Enhancement Algorithm under Non-uniform Illumination Conditions[J].Journal of Shandong University of Science and Technology(Natural Science Edition),2020,39(2):118-125.
[30]TANG Z,JIANG L,LUO Z.A new underwater image enhancement algorithm based on adaptive feedback and Retinex algorithm[J].Multimedia Tools and Applications,2021,80(18):28487-28499.
[31]ANCUTI C,ANCUTI C O,HABER T,et al.Enhancing underwater images and videos by fusion[C]//2012 IEEE conference on computer vision and pattern recognition.IEEE,2012:81-88.
[32]GAO F,WANG K,YANG Z,et al.Underwater image enhancement based on local contrast correction and multi-scale fusion[J].Journal of Marine Science and Engineering,2021,9(2):225.
[33]SONG H,WANG R.Underwater image enhancement based on multi-scale fusion and global stretching of dual-model[J].Mathematics,2021,9(6):595.
[34]FU Z,LIN H,YANG Y,et al.Unsupervised Underwater Image Restoration:From a Homology Perspective[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:643-651.
[35]WU S,LUO T,JIANG G,et al.A two-stage underwater en-hancement network based on structure decomposition and characteristics of underwater imaging[J].IEEE Journal of Oceanic Engineering,2021,46(4):1213-1227.
[36]SHIN Y-S,CHO Y,PANDEY G,et al.Estimation of ambient light and transmission map with common convolutional architecture[C]//OCEANS 2016 MTS/IEEE Monterey.IEEE,2016:1-7.
[37]CAO K,PENG Y-T,COSMAN P C.Underwater image restoration using deep networks to estimate background light and scene depth[C]//2018 IEEE Southwest Symposium on Image Analysis and Interpretation(SSIAI).IEEE,2018:1-4.
[38]WANG K,HU Y,CHEN J,et al.Underwater image restoration based on a parallel convolutional neural network[J].Remote sensing,2019,11(13):1591.
[39]WANG Y,SONG W,FORTINO G,et al.An experimental-based review of image enhancement and image restorationme-thods for underwater imaging[J].IEEE Access,2019,7:140233-140251.
[40]KAR A,DHARA S K,SEN D,et al.Zero-shot single image restoration through controlled perturbation of koschmieder’s mo-del[C]//Proceedings of the IEEE/CVF Conference on Compu-ter Vision and Pattern Recognition.2021:16205-16215.
[41]CHAI S,FU Z,HUANG Y,et al.Unsupervised and Untrained Underwater Image Restoration Based on Physical Image Formation Model[C]//ICASSP 2022-2022 IEEE International Confe-rence on Acoustics,Speech and Signal Processing(ICASSP).IEEE,2022:2774-2778.
[42]LI C,ANWAR S,HOU J,et al.Underwater image enhancement via medium transmission-guided multi-color space embedding[J].IEEE Transactions on Image Processing,2021,30:4985-5000.
[43]WANG Y,ZHANG J,CAO Y,et al.A deep CNN method for underwater image enhancement[C]//2017 IEEE International Conference on Image Processing(ICIP).IEEE,2017:1382-1386.
[44]SUN X,LIU L,LI Q,et al.Deep pixel-to-pixel network for underwater image enhancement and restoration[J].IET Image Processing,2019,13(3):469-474.
[45]LI C,GUO C,REN W,et al.An underwater image enhancement benchmark dataset and beyond[J].IEEE Transactions on Image Processing,2019,29:4376-4389.
[46]DOU Z,WANG N,LI B,et al.Dual color space guided sketch colorization[J].IEEE Transactions on Image Processing,2021,30:7292-7304.
[47]WANG Y,GUO J,GAO H,et al.UIEC^∧ 2-Net:CNN-based underwater image enhancement using two color space[J].Signal Processing:Image Communication,2021,96:116250.
[48]CHEN Q J,XIE Y L.Underwater Image Enhancement Based on Dense Cascaded Convolutional Neural Network[J].Laser & Optoelectronics Progress,2022,59(22):235-244.
[49]ZHANG B,GU S,ZHANG B,et al.Styleswin:Transformer-based gan for high-resolution image generation[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:11304-11314.
[50]MARATHE A,JAIN P,WALAMBE R,et al.RestoreX-AI:A Contrastive Approach towards Guiding Image Restoration via Explainable AI Systems[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:3030-3039.
[51]TORBUNOV D,HUANG Y,YU H,et al.Uvcgan:Unet vision transformer cycle-consistent gan for unpaired image-to-image translation[C]//Proceedings of the IEEE/CVF Winter Confe-rence on Applications of Computer Vision.2023:702-712.
[52]GUI J,SUN Z,WEN Y,et al.A review on generative adversarial networks:Algorithms,theory,and applications[J].IEEE Transactions on Knowledge and Data Engineering,2021,35(4):3313-3332.
[53]MIRZA M,OSINDERO S.Conditional generative adversarialnets[J].arXiv:1411.1784,2014.
[54]ISLAM M J,XIA Y,SATTAR J.Fast underwater image enhancement for improved visual perception[J].IEEE Robotics and Automation Letters,2020,5(2):3227-3234.
[55]YANG M,HU K,DU Y,et al.Underwater image enhancement based on conditional generative adversarial network[J].Signal Processing:Image Communication,2020,81:115723.
[56]LI Y,YANG D Y,LIU L Y,et al.Underwater Image Enhancement Using Generative Adversarial Networks[J].Journal of Shanghai Jiao Tong University,2022,56(2):134-142.
[57]PENG Y F,LI J,GU L R,et al.Underwater Image Enhancement Method Based on Improved Conditional Generative Adversarial Network[J].Chinese Journal of Liquid Crystals and Displays,2022,37(6):768-776.
[58]TANG P,LI L,XUE Y,et al.Real-World Underwater ImageEnhancement Based on Attention U-Net[J].Journal of Marine Science and Engineering,2023,11(3):662.
[59]LIU X,LIN S,TAO Z.Learning multiscale pipeline gated fusion for underwater image enhancement[J].Multimedia Tools and Applications,2023,82(21):32281-32304.
[60]ZHU J Y,PARK T,ISOLA P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2223-2232.
[61]FABBRI C,ISLAM M J,SATTAR J.Enhancing underwater imagery using generative adversarial networks[C]//2018 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2018:7159-7165.
[62]LU J,LI N,ZHANG S,et al.Multi-scale adversarial network for underwater image restoration[J].Optics & Laser Technology,2019,110:105-113.
[63]ZONG X,CHEN Z,WANG D.Local-CycleGAN:a general end-to-end network for visual enhancement in complex deep-water environment[J].Applied Intelligence,2021,51:1947-1958.
[64]HU K,ZHANG Y,WENG C,et al.An underwater image enhancement algorithm based on generative adversarial network and natural image quality evaluation index[J].Journal of Marine Science and Engineering,2021,9(7):691.
[65]MITTAL A,SOUNDARARAJAN R,BOVIK A C.Making a“completely blind” image quality analyzer[J].IEEE Signal Processing Letters,2012,20(3):209-212.
[66]ZHANG H,SUN L,WU L,et al.DuGAN:An effective framework for underwater image enhancement[J].IET Image Processing,2021,15(9):2010-2019.
[67]LIU C,WANG H R.An Improved CycleGAN-Based Method for Underwater Color Image Enhancement[J].Mechanical Science and Technology for Aerospace Engineering,2023,42(12):2093-2099.
[68]JIAN M,QI Q,DONG J,et al.The OUC-vision large-scale underwater image database[C]//2017 IEEE International Confe-rence on Multimedia and Expo(ICME).IEEE,2017:1297-1302.
[69]AKKAYNAK D,TREIBITZ T.Sea-thru:A method for remo-ving water from underwater images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:1682-1691.
[70]LIU R,FAN X,ZHU M,et al.Real-world underwater enhancement:Challenges,benchmarks,and solutions under natural light[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,30(12):4861-4875.
[71]ISLAM M J,LUO P,SATTAR J.Simultaneous enhancement and super-resolution of underwater imagery for improved visual perception[J].arXiv:2002.01155,2020.
[72]SONG W,WANG Y,HUANG D,et al.Enhancement of underwater images with statistical model of background light and optimization of transmission map[J].IEEE Transactions on Broadcasting,2020,66(1):153-169.
[73]PENG L,ZHU C,BIAN L.U-shape transformer for underwater image enhancement[J].IEEE Transactions on Image Proces-sing,2023,32:3066-3079.
[74]HAN J,SHOEIBY M,MALTHUS T,et al.Underwater image restoration via contrastive learning and a real-world dataset[J].Remote Sensing,2022,14(17):4297.
[75]CHANG L,SONG H,LI M,et al.UIDEF:A real-world underwater image dataset and a color-contrast complementary image enhancement framework[J].ISPRS Journal of Photogrammetry and Remote Sensing,2023,196:415-428.
[76]ZHAI G,MIN X.Perceptual image quality assessment:a survey[J].Science China Information Sciences,2020,63:1-52.
[77]MÜNSSON L.Methods for the evaluation of image quality:a review[J].Radiation Protection Dosimetry,2000,90(1/2):89-99.
[78]BOSSE S,MANIRY D,MÜLLER K R,et al.Deep neural networks for no-reference and full-reference image quality assessment[J].IEEE Transactions on Image Processing,2017,27(1):206-219.
[79]WANG Z,BOVIK A C,SHEIKH H R,et al.Image qualityassessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[80]YANG M,SOWMYA A.An underwater color image quality evaluation metric[J].IEEE Transactions on Image Processing,2015,24(12):6062-6071.
[81]PANETTA K,GAO C,AGAIAN S.Human-visual-system-inspired underwater image quality measures[J].IEEE Journal of Oceanic Engineering,2015,41(3):541-551.
[82]ZHENG Y,CHEN W,LIN R,et al.UIF:An objective qualityassessment for underwater image enhancement[J].IEEE Transactions on Image Processing,2022,31:5456-5468.
[83]KHAN Z A,DARDOURI T,KAANICHE M,et al.NNCD-IQA:A new neural networks based compressed database for image quality assessment[J].Multimedia Tools and Applications,2023,82(9):13951-13971.
[84]GAO R,HUANG Z,LIU S.QL-IQA:Learning distance distribution from quality levels for blind image quality assessment[J].Signal Processing:Image Communication,2022,101:116576.
[85]CAO Y D,LIU H Y,JIA X,et al.A Survey on Image QualityEvaluation Methods Based on Deep Learning[J].Computer Engineering and Applications,2021,57(23):27-36.
[86]FU Z,FU X,HUANG Y,et al.Twice mixing:a rank learning based quality assessment approach for underwater image enhancement[J].Signal Processing:Image Communication,2022,102:116622.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Research Progress of Underwater Image Processing Based on Deep Learning

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	LYU Yiming, WANG Jiyang. Iron Ore Image Classification Method Based on Improved Efficientnetv2 [J]. Computer Science, 2024, 51(6A): 230600212-6.
[2]	YANG Xiuzhang, WU Shuai, REN Tianshu, LIAO Wenjing, XIANG Meiyu, YU Xiaomin, LIU Jianyi, CHEN Dengjian. Complex Environment License Plate Recognition Algorithm Based on Improved Image Enhancement and CNN [J]. Computer Science, 2024, 51(6A): 220200162-7.
[3]	SONG Zhen, WANG Jiqiang, HOU Moyu, ZHAO Lin. Conveyor Belt Defect Detection Network Combining Attention Mechanism with Line Laser Assistance [J]. Computer Science, 2024, 51(6A): 230800115-6.
[4]	WU Chunming, LIU Yali. Method for Lung Nodule Detection on CT Images Using Improved YOLOv5 [J]. Computer Science, 2024, 51(6A): 230500019-6.
[5]	YIN Xudong, CHEN Junyang, ZHOU Bo. Study on Industrial Defect Augmentation Data Filtering Based on OOD Scores [J]. Computer Science, 2024, 51(6A): 230700111-7.
[6]	QIAO Hong, XING Hongjie. Attention-based Multi-scale Distillation Anomaly Detection [J]. Computer Science, 2024, 51(6A): 230300223-11.
[7]	SI Jia, LIANG Jianfeng, XIE Shuo, DENG Yingjun. Research Progress of Anomaly Detection in IaaS Cloud Operation Driven by Deep Learning [J]. Computer Science, 2024, 51(6A): 230400016-8.
[8]	DUAN Pengsong, DIAO Xianguang, ZHANG Dalong, CAO Yangjie, LIU Guangyi, KONG Jinsheng. WiCare:Non-contact Fall Monitoring Model for Elderly in Toilet [J]. Computer Science, 2024, 51(6A): 230700044-8.
[9]	WANG Yingjie, ZHANG Chengye, BAI Fengbo, WANG Zumin. Named Entity Recognition Approach of Judicial Documents Based on Transformer [J]. Computer Science, 2024, 51(6A): 230500164-9.
[10]	LIANG Fang, XU Xuyao, ZHAO Kailong, ZHAO Xuanfeng, ZHANG Guijun. Remote Template Detection Algorithm and Its Application in Protein Structure Prediction [J]. Computer Science, 2024, 51(6A): 230600225-7.
[11]	PENG Bo, LI Yaodong, GONG Xianfu, LI Hao. Method for Entity Relation Extraction Based on Heterogeneous Graph Neural Networks and TextSemantic Enhancement [J]. Computer Science, 2024, 51(6A): 230700071-5.
[12]	WANG Guogang, DONG Zhihao. Lightweight Image Semantic Segmentation Based on Attention Mechanism and Densely AdjacentPrediction [J]. Computer Science, 2024, 51(6A): 230300204-8.
[13]	LU Dongsheng, LONG Hua. Method for Homologous Spectrum Monitoring Data Identification Based on Spectrum SIFT [J]. Computer Science, 2024, 51(6A): 230300177-7.
[14]	WANG Li, CHEN Gang, XIA Mingshan, HU Hao. DUWe:Dynamic Unknown Word Embedding Approach for Web Anomaly Detection [J]. Computer Science, 2024, 51(6A): 230300191-5.
[15]	HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.