Computer Science ›› 2025, Vol. 52 ›› Issue (5): 171-178.doi: 10.11896/jsjkx.240200020

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Restoration of Atmospheric Turbulence-degraded Images Based on Contrastive Learning

MIAO Zhuang, CUI Haoran, ZHANG Qiyang, WANG Jiabao, LI Yang   

  1. Command and Control Engineering College,Army Engineering University of PLA,Nanjing 210007,China
  • Received:2024-02-04 Revised:2024-06-21 Online:2025-05-15 Published:2025-05-12
  • About author:MIAO Zhuang,born in 1976,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,pattern recognition and computer vision.
    WANG Jiabao,born in 1985,Ph.D,associate professor.His main research interests include computer vision and image processing.
  • Supported by:
    Natural Science Foundation of Jiangsu Province,China(BK20200581).

Abstract: Image degradation caused by atmospheric turbulence seriously affects the performance of downstream computer vision tasks such as object detection and image recognition.Existing deep learning-based image restoration models for atmospheric turbulence degradation have achieved good performance,but have not fully utilized the feature information of the turbulence effect.To improve restoration results,a method for restoring of atmospheric turbulence-degraded images based on contrastive learning is proposed.Aiming at the blurring and distortion caused by atmospheric turbulence degradation,a turbulence mitigation block is designed,which integrates a Transformer-based channel information interaction module and a CNN-based spatial information interaction module to suppress the turbulence interference to the image at both global and local levels.At the same time,contrastive learning is introduced to consider the clear image and the degraded image of atmospheric turbulence as positive and negative samples,to pull the output of the atmospheric turbulence restoration network closer to the positive samples and push it farther away from the negative samples in the feature space,so that feature extraction and image restoration can be performed more efficiently.The proposed method achieves 26.78 dB and 22.42 dB PSNR and 0.790 9 and 0.682 0 SSIM on the synthetic Helen dataset and synthetic Places dataset,respectively,which is the best result compared with the existing five methods,and it is suitable for improving the quality of atmospheric turbulence degradation images.

Key words: Deep learning, Image restoration, Atmospheric turbulence, Contrastive learning, Feature extraction

CLC Number: 

  • TP391
[1]ROGGEMANN M C,WELSH B M,HUNT B R.Imagingthrough turbulence[M].CRC press,1996.
[2]ANANTRASIRICHAI N,ACHIM A,KINGSBURY N G,et al.Atmospheric turbulence mitigation using complex wavelet-based fusion[J].IEEE Transactions on Image Processing,2013,22(6):2398-2408.
[3]MAO Z,CHIMITT N,CHAN S H.Image reconstruction ofstatic and dynamic scenes through anisoplanatic turbulence[J].IEEE Transactions on Computational Imaging,2020,6:1415-1428.
[4]ZHU X,MILANFAR P.Removing atmospheric turbulence via space-invariant deconvolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,35(1):157-170.
[5]HIRSCH M,SRA S,SCHÖLKOPF B,et al.Efficient filter flow for space-variant multiframe blind deconvolution[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:607-614.
[6]FRIED D L.Probability of getting a lucky short-exposure image through turbulence[J].JOSA,1978,68(12):1651-1658.
[7]YASARLA R,PATEL V M.Learning to restore a single face image degraded by atmospheric turbulence using cnns[J].ar-Xiv:2007.08404,2020.
[8]MAO Z,JAISWAL A,WANG Z,et al.Single frame atmosphericturbulence mitigation:A benchmark study and a new physics-inspired transformer model[C]//Computer Vision-ECCV 2022:17th European Conference,Tel Aviv,Israel,October 23-27,2022,Proceedings,Part XIX.Cham:Springer Nature Switzerland,2022:430-446.
[9]ZHANG X,MAO Z,CHIMITT N,et al.Imaging through the atmosphere using turbulence mitigation transformer[J].arXiv:2207.06465,2022.
[10]JAISWAL A,ZHANG X,CHAN S H,et al.Physics-DrivenTurbulence Image Restoration with Stochastic Refinement[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:12170-12181.
[11]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[12]WU H,QU Y,LIN S,et al.Contrastive learning for compactsingle image dehazing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10551-10560.
[13]ZHANG Y,LI K,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:286-301.
[14]YANG W,TAN R T,FENG J,et al.Deep joint rain detection and removal from a single image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1357-1366.
[15]CAI B,XU X,JIA K,et al.Dehazenet:An end-to-end system for single image haze removal[J].IEEE transactions on image processing,2016,25(11):5187-5198.
[16]YASARLA R,PATEL V M.Learning to restore images de-graded by atmospheric turbulence using uncertainty[C]//2021 IEEE International Conference on Image Processing(ICIP).IEEE,2021.
[17]ZAMIR S W,ARORA A,KHAN S,et al.Multi-stage progressive image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14821-14831.
[18]WANG Z,CUN X,BAO J,et al.Uformer:A general u-shaped transformer for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17683-17693.
[19]CHEN H,WANG Y,GUO T,et al.Pre-trained image proces-sing transformer[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2021:12299-12310.
[20]ZAMIR S W,ARORA A,KHAN S,et al.Restormer:Efficient transformer for high-resolution image restoration[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5728-5739.
[21]CHEN L,CHU X,ZHANG X,et al.Simple baselines for image restoration[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:17-33.
[22]SERMANET P,LYNCH C,HSU J,et al.Time-contrastive networks:Self-supervised learning from multi-view observation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2017:486-487.
[23]OORD A,LI Y,VINYALS O.Representation learning with contrastive predictive coding[J].arXiv:1807.03748,2018.
[24]CHEN T,KORNBLITH S,NOROUZI M,et al.A simpleframework for contrastive learning of visual representations[C]//International Conference on Machine Learning.PMLR,2020:1597-1607.
[25]LIANG D,LI L,WEI M,et al.Semantically contrastive learning for low-light image enhancement[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:1555-1563.
[26]WANG Y,XIONG J,YAN X,et al.USCFormer:Unified Transformer With Semantically Contrastive Learning for Image Dehazing[J].IEEE Transactions on Intelligent Transportation Systems,2023,24(10):11321-11333.
[27]WU G,JIANG J,LIU X.A practical contrastive learning framework for single-image super-resolution[J].arXiv:2111.13924,2023.
[28]YE Y,YU C,CHANG Y,et al.Unsupervised deraining:Where contrastive learning meets self-similarity[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5821-5830.
[29]HIRSCH M,SRA S,SCHÖLKOPF B,et al.Efficient filter flow for space-variant multiframe blind deconvolution[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:607-614.
[30]CHAN S H.Tilt-then-blur or blur-then-tilt? clarifying the at-mospheric turbulence model[J].IEEE Signal Processing Letters,2022,29:1833-1837.
[31]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time singleimage and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[32]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[33]WANG Z,SIMONCELLI E P,BOVIK A C.Multiscale structuralsimilarity for image quality assessment[C]//The Thrity-Se-venth Asilomar Conference on Signals,Systems & Computers.IEEE,2003:1398-1402.
[34]ZHAO H,GALLO O,FROSIO I,et al.Loss functions for image restoration with neural networks[J].IEEE Transactions on Computational Imaging,2016,3(1):47-57.
[35]LIU Z,LUO P,WANG X,et al.Deep learning face attributes in the wild[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:3730-3738.
[36]MAO Z,CHIMITT N,CHAN S H.Accelerating atmospheric turbulence simulation via learned phase-to-space transform[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:14759-14768.
[37]LE V,BRANDT J,LIN Z,et al.Interactive facial feature localization[C]//Computer Vision-ECCV 2012:12th European Conference on Computer Vision,Florence,Italy,Part III 12.SpringerBerlin Heidelberg,2012:679-692.
[38]ZHOU B,LAPEDRIZA A,KHOSLA A,et al.Places:A 10 million image database for scene recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(6):1452-1464.
[39]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[40]LIANG J,CAO J,SUN G,et al.Swinir:Image restoration using swin transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:1833-1844.
[41]CHEN F,WANG H.Design of fuzzy image quality restoration algorithm based on multi-channel visual attention[J].Journal of Jilin University(Engineering and Technology Edition),2023,53(9):2626-2631.
[1] GAO Wei, WANG Lei, LI Jianan, LI Shuailong, HAN Lin. Operator Fusion Optimization for Deep Learning Compiler TVM [J]. Computer Science, 2025, 52(5): 58-66.
[2] WU Pengyuan, FANG Wei. Study on Graph Collaborative Filtering Model Based on FeatureNet Contrastive Learning [J]. Computer Science, 2025, 52(5): 139-148.
[3] KONG Yu, XIONG Fengguang, ZHANG Zhiqiang, SHEN Chaofan, HU Mingyue. Low Overlap Point Cloud Registration Method Based on Deep Position-aware Transformer [J]. Computer Science, 2025, 52(5): 199-211.
[4] ZHENG Xu, HUANG Xiangjie, YANG Yang. Reversible Facial Privacy Protection Method Based on “Invisible Masks” [J]. Computer Science, 2025, 52(5): 384-391.
[5] SUN Tanghui, ZHAO Gang, GUO Meiqian. Long-tail Distributed Medical Image Classification Based on Large Selective Nuclear Bilateral-branch Networks [J]. Computer Science, 2025, 52(4): 231-239.
[6] MENG Sijiang, WANG Hongxia, ZENG Qiang, ZHOU Yang. Multi-view and Multi-scale Fusion Attention Network for Document Image Forgery Localization [J]. Computer Science, 2025, 52(4): 327-335.
[7] LI Xiaolan, MA Yong. Study on Lightweight Flame Detection Algorithm with Progressive Adaptive Feature Fusion [J]. Computer Science, 2025, 52(4): 64-73.
[8] DENG Ceyu, LI Duantengchuan, HU Yiren, WANG Xiaoguang, LI Zhifei. Joint Inter-word and Inter-sentence Multi-relationship Modeling for Review-basedRecommendation Algorithm [J]. Computer Science, 2025, 52(4): 119-128.
[9] PENG Linna, ZHANG Hongyun, MIAO Duoqian. Complex Organ Segmentation Based on Edge Constraints and Enhanced Swin Unetr [J]. Computer Science, 2025, 52(4): 177-184.
[10] KONG Jialin, ZHANG Qi, WEI Jianze, LI Qi. Adaptive Contextual Learning Method Based on Iris Texture Perception [J]. Computer Science, 2025, 52(4): 185-193.
[11] ZHOU Yi, MAO Kuanmin. Research on Individual Identification of Cattle Based on YOLO-Unet Combined Network [J]. Computer Science, 2025, 52(4): 194-201.
[12] WANG Tao, BAI Xuefei, WANG Wenjian. Selective Feature Fusion for 3D CT Image Segmentation of Renal Cancer Based on Edge Enhancement [J]. Computer Science, 2025, 52(3): 41-49.
[13] WANG Jie, WANG Chuangye, XIE Jiucheng, GAO Hao. Animatable Head Avatar Reconstruction Algorithm Based on Region Encoding [J]. Computer Science, 2025, 52(3): 50-57.
[14] ZHONG Yue, GU Jieming. 3D Reconstruction of Single-view Sketches Based on Attention Mechanism and Contrastive Loss [J]. Computer Science, 2025, 52(3): 77-85.
[15] WANG Yuan, HUO Peng, HAN Yi, CHEN Tun, WANG Xiang, WEN Hui. Survey on Deep Learning-based Meteorological Forecasting Models [J]. Computer Science, 2025, 52(3): 112-126.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!