Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (5): 171-178.doi: 10.11896/jsjkx.240200020

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Restoration of Atmospheric Turbulence-degraded Images Based on Contrastive Learning

MIAO Zhuang, CUI Haoran, ZHANG Qiyang, WANG Jiabao, LI Yang   

  1. Command and Control Engineering College,Army Engineering University of PLA,Nanjing 210007,China
  • Received:2024-02-04 Revised:2024-06-21 Online:2025-05-15 Published:2025-05-12
  • About author:MIAO Zhuang,born in 1976,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,pattern recognition and computer vision.
    WANG Jiabao,born in 1985,Ph.D,associate professor.His main research interests include computer vision and image processing.
  • Supported by:
    Natural Science Foundation of Jiangsu Province,China(BK20200581).

PDF (PC)

116

Abstract: Image degradation caused by atmospheric turbulence seriously affects the performance of downstream computer vision tasks such as object detection and image recognition.Existing deep learning-based image restoration models for atmospheric turbulence degradation have achieved good performance,but have not fully utilized the feature information of the turbulence effect.To improve restoration results,a method for restoring of atmospheric turbulence-degraded images based on contrastive learning is proposed.Aiming at the blurring and distortion caused by atmospheric turbulence degradation,a turbulence mitigation block is designed,which integrates a Transformer-based channel information interaction module and a CNN-based spatial information interaction module to suppress the turbulence interference to the image at both global and local levels.At the same time,contrastive learning is introduced to consider the clear image and the degraded image of atmospheric turbulence as positive and negative samples,to pull the output of the atmospheric turbulence restoration network closer to the positive samples and push it farther away from the negative samples in the feature space,so that feature extraction and image restoration can be performed more efficiently.The proposed method achieves 26.78 dB and 22.42 dB PSNR and 0.790 9 and 0.682 0 SSIM on the synthetic Helen dataset and synthetic Places dataset,respectively,which is the best result compared with the existing five methods,and it is suitable for improving the quality of atmospheric turbulence degradation images.

Key words: Deep learning, Image restoration, Atmospheric turbulence, Contrastive learning, Feature extraction

CLC Number: 

  • TP391
[1]ROGGEMANN M C,WELSH B M,HUNT B R.Imagingthrough turbulence[M].CRC press,1996.
[2]ANANTRASIRICHAI N,ACHIM A,KINGSBURY N G,et al.Atmospheric turbulence mitigation using complex wavelet-based fusion[J].IEEE Transactions on Image Processing,2013,22(6):2398-2408.
[3]MAO Z,CHIMITT N,CHAN S H.Image reconstruction ofstatic and dynamic scenes through anisoplanatic turbulence[J].IEEE Transactions on Computational Imaging,2020,6:1415-1428.
[4]ZHU X,MILANFAR P.Removing atmospheric turbulence via space-invariant deconvolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,35(1):157-170.
[5]HIRSCH M,SRA S,SCHÖLKOPF B,et al.Efficient filter flow for space-variant multiframe blind deconvolution[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:607-614.
[6]FRIED D L.Probability of getting a lucky short-exposure image through turbulence[J].JOSA,1978,68(12):1651-1658.
[7]YASARLA R,PATEL V M.Learning to restore a single face image degraded by atmospheric turbulence using cnns[J].ar-Xiv:2007.08404,2020.
[8]MAO Z,JAISWAL A,WANG Z,et al.Single frame atmosphericturbulence mitigation:A benchmark study and a new physics-inspired transformer model[C]//Computer Vision-ECCV 2022:17th European Conference,Tel Aviv,Israel,October 23-27,2022,Proceedings,Part XIX.Cham:Springer Nature Switzerland,2022:430-446.
[9]ZHANG X,MAO Z,CHIMITT N,et al.Imaging through the atmosphere using turbulence mitigation transformer[J].arXiv:2207.06465,2022.
[10]JAISWAL A,ZHANG X,CHAN S H,et al.Physics-DrivenTurbulence Image Restoration with Stochastic Refinement[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:12170-12181.
[11]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[12]WU H,QU Y,LIN S,et al.Contrastive learning for compactsingle image dehazing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10551-10560.
[13]ZHANG Y,LI K,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:286-301.
[14]YANG W,TAN R T,FENG J,et al.Deep joint rain detection and removal from a single image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1357-1366.
[15]CAI B,XU X,JIA K,et al.Dehazenet:An end-to-end system for single image haze removal[J].IEEE transactions on image processing,2016,25(11):5187-5198.
[16]YASARLA R,PATEL V M.Learning to restore images de-graded by atmospheric turbulence using uncertainty[C]//2021 IEEE International Conference on Image Processing(ICIP).IEEE,2021.
[17]ZAMIR S W,ARORA A,KHAN S,et al.Multi-stage progressive image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14821-14831.
[18]WANG Z,CUN X,BAO J,et al.Uformer:A general u-shaped transformer for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17683-17693.
[19]CHEN H,WANG Y,GUO T,et al.Pre-trained image proces-sing transformer[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2021:12299-12310.
[20]ZAMIR S W,ARORA A,KHAN S,et al.Restormer:Efficient transformer for high-resolution image restoration[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5728-5739.
[21]CHEN L,CHU X,ZHANG X,et al.Simple baselines for image restoration[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:17-33.
[22]SERMANET P,LYNCH C,HSU J,et al.Time-contrastive networks:Self-supervised learning from multi-view observation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2017:486-487.
[23]OORD A,LI Y,VINYALS O.Representation learning with contrastive predictive coding[J].arXiv:1807.03748,2018.
[24]CHEN T,KORNBLITH S,NOROUZI M,et al.A simpleframework for contrastive learning of visual representations[C]//International Conference on Machine Learning.PMLR,2020:1597-1607.
[25]LIANG D,LI L,WEI M,et al.Semantically contrastive learning for low-light image enhancement[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:1555-1563.
[26]WANG Y,XIONG J,YAN X,et al.USCFormer:Unified Transformer With Semantically Contrastive Learning for Image Dehazing[J].IEEE Transactions on Intelligent Transportation Systems,2023,24(10):11321-11333.
[27]WU G,JIANG J,LIU X.A practical contrastive learning framework for single-image super-resolution[J].arXiv:2111.13924,2023.
[28]YE Y,YU C,CHANG Y,et al.Unsupervised deraining:Where contrastive learning meets self-similarity[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5821-5830.
[29]HIRSCH M,SRA S,SCHÖLKOPF B,et al.Efficient filter flow for space-variant multiframe blind deconvolution[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:607-614.
[30]CHAN S H.Tilt-then-blur or blur-then-tilt? clarifying the at-mospheric turbulence model[J].IEEE Signal Processing Letters,2022,29:1833-1837.
[31]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time singleimage and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[32]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[33]WANG Z,SIMONCELLI E P,BOVIK A C.Multiscale structuralsimilarity for image quality assessment[C]//The Thrity-Se-venth Asilomar Conference on Signals,Systems & Computers.IEEE,2003:1398-1402.
[34]ZHAO H,GALLO O,FROSIO I,et al.Loss functions for image restoration with neural networks[J].IEEE Transactions on Computational Imaging,2016,3(1):47-57.
[35]LIU Z,LUO P,WANG X,et al.Deep learning face attributes in the wild[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:3730-3738.
[36]MAO Z,CHIMITT N,CHAN S H.Accelerating atmospheric turbulence simulation via learned phase-to-space transform[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:14759-14768.
[37]LE V,BRANDT J,LIN Z,et al.Interactive facial feature localization[C]//Computer Vision-ECCV 2012:12th European Conference on Computer Vision,Florence,Italy,Part III 12.SpringerBerlin Heidelberg,2012:679-692.
[38]ZHOU B,LAPEDRIZA A,KHOSLA A,et al.Places:A 10 million image database for scene recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(6):1452-1464.
[39]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[40]LIANG J,CAO J,SUN G,et al.Swinir:Image restoration using swin transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:1833-1844.
[41]CHEN F,WANG H.Design of fuzzy image quality restoration algorithm based on multi-channel visual attention[J].Journal of Jilin University(Engineering and Technology Edition),2023,53(9):2626-2631.
[1] LIU Wei, XU Yong, FANG Juan, LI Cheng, ZHU Yujun, FANG Qun, HE Xin. Multimodal Air-writing Gesture Recognition Based on Radar-Vision Fusion [J]. Computer Science, 2025, 52(9): 259-268.
[2] YIN Shi, SHI Zhenyang, WU Menglin, CAI Jinyan, YU De. Deep Learning-based Kidney Segmentation in Ultrasound Imaging:Current Trends and Challenges [J]. Computer Science, 2025, 52(9): 16-24.
[3] ZENG Lili, XIA Jianan, LI Shaowen, JING Maike, ZHAO Huihui, ZHOU Xuezhong. M2T-Net:Cross-task Transfer Learning Tongue Diagnosis Method Based on Multi-source Data [J]. Computer Science, 2025, 52(9): 47-53.
[4] LI Yaru, WANG Qianqian, CHE Chao, ZHU Deheng. Graph-based Compound-Protein Interaction Prediction with Drug Substructures and Protein 3D Information [J]. Computer Science, 2025, 52(9): 71-79.
[5] LUO Chi, LU Lingyun, LIU Fei. Partial Differential Equation Solving Method Based on Locally Enhanced Fourier NeuralOperators [J]. Computer Science, 2025, 52(9): 144-151.
[6] LIU Leyuan, CHEN Gege, WU Wei, WANG Yong, ZHOU Fan. Survey of Data Classification and Grading Studies [J]. Computer Science, 2025, 52(9): 195-211.
[7] HUANG Chao, CHENG Chunling, WANG Youkang. Source-free Domain Adaptation Method Based on Pseudo Label Uncertainty Estimation [J]. Computer Science, 2025, 52(9): 212-219.
[8] TANG Boyuan, LI Qi. Review on Application of Spatial-Temporal Graph Neural Network in PM2.5 ConcentrationForecasting [J]. Computer Science, 2025, 52(8): 71-85.
[9] ZHANG Shiju, GUO Chaoyang, WU Chengliang, WU Lingjun, YANG Fengyu. Text Clustering Approach Based on Key Semantic Driven and Contrastive Learning [J]. Computer Science, 2025, 52(8): 171-179.
[10] LIU Zhengyu, ZHANG Fan, QI Xiaofeng, GAO Yanzhao, SONG Yijing, FAN Wang. Review of Research on Deep Learning Compiler [J]. Computer Science, 2025, 52(8): 29-44.
[11] ZHENG Cheng, YANG Nan. Aspect-based Sentiment Analysis Based on Syntax,Semantics and Affective Knowledge [J]. Computer Science, 2025, 52(7): 218-225.
[12] ZHANG Taotao, XIE Jun, QIAO Pingjuan. Specific Emitter Identification Based on Progressive Self-training Open Set Domain Adaptation [J]. Computer Science, 2025, 52(7): 279-286.
[13] LI Mengxi, GAO Xindan, LI Xue. Two-way Feature Augmentation Graph Convolution Networks Algorithm [J]. Computer Science, 2025, 52(7): 127-134.
[14] ZHOU Lei, SHI Huaifeng, YANG Kai, WANG Rui, LIU Chaofan. Intelligent Prediction of Network Traffic Based on Large Language Model [J]. Computer Science, 2025, 52(6A): 241100058-7.
[15] GUAN Xin, YANG Xueyong, YANG Xiaolin, MENG Xiangfu. Tumor Mutation Prediction Model of Lung Adenocarcinoma Based on Pathological [J]. Computer Science, 2025, 52(6A): 240700010-8.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.