Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (5): 171-178.doi: 10.11896/jsjkx.240200020

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Restoration of Atmospheric Turbulence-degraded Images Based on Contrastive Learning

MIAO Zhuang, CUI Haoran, ZHANG Qiyang, WANG Jiabao, LI Yang   

  1. Command and Control Engineering College,Army Engineering University of PLA,Nanjing 210007,China
  • Received:2024-02-04 Revised:2024-06-21 Online:2025-05-15 Published:2025-05-12
  • About author:MIAO Zhuang,born in 1976,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,pattern recognition and computer vision.
    WANG Jiabao,born in 1985,Ph.D,associate professor.His main research interests include computer vision and image processing.
  • Supported by:
    Natural Science Foundation of Jiangsu Province,China(BK20200581).

PDF (PC)

316

Abstract: Image degradation caused by atmospheric turbulence seriously affects the performance of downstream computer vision tasks such as object detection and image recognition.Existing deep learning-based image restoration models for atmospheric turbulence degradation have achieved good performance,but have not fully utilized the feature information of the turbulence effect.To improve restoration results,a method for restoring of atmospheric turbulence-degraded images based on contrastive learning is proposed.Aiming at the blurring and distortion caused by atmospheric turbulence degradation,a turbulence mitigation block is designed,which integrates a Transformer-based channel information interaction module and a CNN-based spatial information interaction module to suppress the turbulence interference to the image at both global and local levels.At the same time,contrastive learning is introduced to consider the clear image and the degraded image of atmospheric turbulence as positive and negative samples,to pull the output of the atmospheric turbulence restoration network closer to the positive samples and push it farther away from the negative samples in the feature space,so that feature extraction and image restoration can be performed more efficiently.The proposed method achieves 26.78 dB and 22.42 dB PSNR and 0.790 9 and 0.682 0 SSIM on the synthetic Helen dataset and synthetic Places dataset,respectively,which is the best result compared with the existing five methods,and it is suitable for improving the quality of atmospheric turbulence degradation images.

Key words: Deep learning, Image restoration, Atmospheric turbulence, Contrastive learning, Feature extraction

CLC Number: 

  • TP391
[1]ROGGEMANN M C,WELSH B M,HUNT B R.Imagingthrough turbulence[M].CRC press,1996.
[2]ANANTRASIRICHAI N,ACHIM A,KINGSBURY N G,et al.Atmospheric turbulence mitigation using complex wavelet-based fusion[J].IEEE Transactions on Image Processing,2013,22(6):2398-2408.
[3]MAO Z,CHIMITT N,CHAN S H.Image reconstruction ofstatic and dynamic scenes through anisoplanatic turbulence[J].IEEE Transactions on Computational Imaging,2020,6:1415-1428.
[4]ZHU X,MILANFAR P.Removing atmospheric turbulence via space-invariant deconvolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,35(1):157-170.
[5]HIRSCH M,SRA S,SCHÖLKOPF B,et al.Efficient filter flow for space-variant multiframe blind deconvolution[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:607-614.
[6]FRIED D L.Probability of getting a lucky short-exposure image through turbulence[J].JOSA,1978,68(12):1651-1658.
[7]YASARLA R,PATEL V M.Learning to restore a single face image degraded by atmospheric turbulence using cnns[J].ar-Xiv:2007.08404,2020.
[8]MAO Z,JAISWAL A,WANG Z,et al.Single frame atmosphericturbulence mitigation:A benchmark study and a new physics-inspired transformer model[C]//Computer Vision-ECCV 2022:17th European Conference,Tel Aviv,Israel,October 23-27,2022,Proceedings,Part XIX.Cham:Springer Nature Switzerland,2022:430-446.
[9]ZHANG X,MAO Z,CHIMITT N,et al.Imaging through the atmosphere using turbulence mitigation transformer[J].arXiv:2207.06465,2022.
[10]JAISWAL A,ZHANG X,CHAN S H,et al.Physics-DrivenTurbulence Image Restoration with Stochastic Refinement[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:12170-12181.
[11]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[12]WU H,QU Y,LIN S,et al.Contrastive learning for compactsingle image dehazing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10551-10560.
[13]ZHANG Y,LI K,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:286-301.
[14]YANG W,TAN R T,FENG J,et al.Deep joint rain detection and removal from a single image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1357-1366.
[15]CAI B,XU X,JIA K,et al.Dehazenet:An end-to-end system for single image haze removal[J].IEEE transactions on image processing,2016,25(11):5187-5198.
[16]YASARLA R,PATEL V M.Learning to restore images de-graded by atmospheric turbulence using uncertainty[C]//2021 IEEE International Conference on Image Processing(ICIP).IEEE,2021.
[17]ZAMIR S W,ARORA A,KHAN S,et al.Multi-stage progressive image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14821-14831.
[18]WANG Z,CUN X,BAO J,et al.Uformer:A general u-shaped transformer for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17683-17693.
[19]CHEN H,WANG Y,GUO T,et al.Pre-trained image proces-sing transformer[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2021:12299-12310.
[20]ZAMIR S W,ARORA A,KHAN S,et al.Restormer:Efficient transformer for high-resolution image restoration[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5728-5739.
[21]CHEN L,CHU X,ZHANG X,et al.Simple baselines for image restoration[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:17-33.
[22]SERMANET P,LYNCH C,HSU J,et al.Time-contrastive networks:Self-supervised learning from multi-view observation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2017:486-487.
[23]OORD A,LI Y,VINYALS O.Representation learning with contrastive predictive coding[J].arXiv:1807.03748,2018.
[24]CHEN T,KORNBLITH S,NOROUZI M,et al.A simpleframework for contrastive learning of visual representations[C]//International Conference on Machine Learning.PMLR,2020:1597-1607.
[25]LIANG D,LI L,WEI M,et al.Semantically contrastive learning for low-light image enhancement[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:1555-1563.
[26]WANG Y,XIONG J,YAN X,et al.USCFormer:Unified Transformer With Semantically Contrastive Learning for Image Dehazing[J].IEEE Transactions on Intelligent Transportation Systems,2023,24(10):11321-11333.
[27]WU G,JIANG J,LIU X.A practical contrastive learning framework for single-image super-resolution[J].arXiv:2111.13924,2023.
[28]YE Y,YU C,CHANG Y,et al.Unsupervised deraining:Where contrastive learning meets self-similarity[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5821-5830.
[29]HIRSCH M,SRA S,SCHÖLKOPF B,et al.Efficient filter flow for space-variant multiframe blind deconvolution[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:607-614.
[30]CHAN S H.Tilt-then-blur or blur-then-tilt? clarifying the at-mospheric turbulence model[J].IEEE Signal Processing Letters,2022,29:1833-1837.
[31]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time singleimage and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[32]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[33]WANG Z,SIMONCELLI E P,BOVIK A C.Multiscale structuralsimilarity for image quality assessment[C]//The Thrity-Se-venth Asilomar Conference on Signals,Systems & Computers.IEEE,2003:1398-1402.
[34]ZHAO H,GALLO O,FROSIO I,et al.Loss functions for image restoration with neural networks[J].IEEE Transactions on Computational Imaging,2016,3(1):47-57.
[35]LIU Z,LUO P,WANG X,et al.Deep learning face attributes in the wild[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:3730-3738.
[36]MAO Z,CHIMITT N,CHAN S H.Accelerating atmospheric turbulence simulation via learned phase-to-space transform[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:14759-14768.
[37]LE V,BRANDT J,LIN Z,et al.Interactive facial feature localization[C]//Computer Vision-ECCV 2012:12th European Conference on Computer Vision,Florence,Italy,Part III 12.SpringerBerlin Heidelberg,2012:679-692.
[38]ZHOU B,LAPEDRIZA A,KHOSLA A,et al.Places:A 10 million image database for scene recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(6):1452-1464.
[39]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[40]LIANG J,CAO J,SUN G,et al.Swinir:Image restoration using swin transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:1833-1844.
[41]CHEN F,WANG H.Design of fuzzy image quality restoration algorithm based on multi-channel visual attention[J].Journal of Jilin University(Engineering and Technology Edition),2023,53(9):2626-2631.
[1] GAO Tai, REN Yanzhang, WANG Huiqing, LI Ying, WANG Bin. KGMamba:Gene Regulatory Network Prediction Model Based on Kolmogorov-Arnold Network Optimizing Graph Convolutional Network and Mamba [J]. Computer Science, 2026, 53(4): 101-111.
[2] LIU Yichen, LIN Yan, ZHOU Zeyu, GUO Shengnan, LIN Youfang, WAN Huaiyu. Efficient Semantic-aware Trajectory Representation Learning Method via State Space Model [J]. Computer Science, 2026, 53(4): 134-142.
[3] ZHANG Xueqin, WANG Zhineng, LI Jinsheng, LU Yisong, LUO Fei. Key Node Identification in Temporal Social Networks Based on Deep Learning and Multi-feature Fusion [J]. Computer Science, 2026, 53(4): 143-154.
[4] GU Bokai, LIU Dun, SUN Yang. STWD-DLFRD:Multi-granularity Fake Review Detection via Sequential Three-way Decisions and Deep Learning [J]. Computer Science, 2026, 53(4): 188-196.
[5] CHENG Zimeng, YANG Xinyue, AI Haojun, WANG Zhongyuan. Unsupervised Infrared Image Generation Method Based on Dual Semantic Contrastive Learning [J]. Computer Science, 2026, 53(4): 260-268.
[6] ZHENG Cheng, BAN Qingqing. Knowledge-assisted and Reinforced Syntax-driven for Aspect-based Sentiment Analysis [J]. Computer Science, 2026, 53(4): 406-414.
[7] YIN Chuang, LIU Jianyi, ZHANG Ru. Cross-modal Fusion Few-sample Ransomware Classifier:Multimodal Encoding Based on Pre-trained Models [J]. Computer Science, 2026, 53(4): 435-444.
[8] MENG Siyu, NIU Chunxiang, TAN Quange, WANG Rong. Deepfake Detection Method Based on Positional Enhancement and Frequency Domain ComponentInteraction [J]. Computer Science, 2026, 53(4): 445-453.
[9] LI Zequn, DING Fei. Fatigue Driving Detection Based on Dual-branch Fusion and Segmented Domain AdaptationTransfer Learning [J]. Computer Science, 2026, 53(3): 78-87.
[10] YU Chengcheng, JIANG Yongfa, CHEN Fangshu, WANG Jiahui, MENG Xiankai. Multi-view Exercise Representation and Forgetting Mechanism for Deep KnowledgeTracing [J]. Computer Science, 2026, 53(3): 107-114.
[11] ZHAO Binbei, ZHU Li, ZHAO Hongli, LI Yutong. Computer Vision Applications in Rail Transit Systems [J]. Computer Science, 2026, 53(3): 214-224.
[12] FU Yukai, LI Qingzhen, DONG Zhixue, SHI Dongli, ZHAO Peng. Pedestrian Re-identification Methods Based on Limited Target Data and Deep Learning [J]. Computer Science, 2026, 53(3): 287-294.
[13] LI Wenli, FENG Xiaonian, QIAN Tieyun. Few-shot Continuous Toxicity Detection Based on Large Language Model Augmentation [J]. Computer Science, 2026, 53(3): 321-330.
[14] YU Ding, LI Zhangwei. Prediction Method of RNA Secondary Structure Based on Transformer Architecture [J]. Computer Science, 2026, 53(3): 375-382.
[15] DU Jiantong, GUAN Zeli, XUE Zhe. Multi-task Learning-based Ophthalmic Video Feature Fusion and Multi-dimensional Profiling [J]. Computer Science, 2026, 53(3): 383-391.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.