Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (11A): 220800229-7.doi: 10.11896/jsjkx.220800229

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Reconstruction of Solar Speckle Image Combined with Gated Fusion Network and Residual Fourier Transform

HUANG Yaqun1, ZHENG Peiyu1, JIANG Murong1, YANG Lei2, LUO Jun1   

  1. 1 School of Information Science and Engineering,Yunnan University,Kunming 650500,China
    2 Yunnan Observatories,Chinese Academy of Sciences,Kunming 650011,China
  • Published:2023-11-09
  • About author:HUANG Yaqun,born in 1971,associate professor,is a member of China Computer Federation.Her main research interests include image reconstruction and intelligent data analysis.
    ZHENG Peiyu,born in 1997,postgra-duate.His main research interests include deep learning and image reconstruction.
  • Supported by:
    National Natural Science Foundation of China(11773073),Science and Technology Innovation Team Sopport Project of Yunnan Province(IRTSTYN) and Construction Project of Teaching Case Base for Professional Degree Graduates of Yunnan University(2022XJALK02).

PDF (PC)

404

Abstract: When using the existing deep learning algorithm to reconstruct the highly blurred solar speckle image taken by the Yunnan Observatory,there are problems such as loss of high-frequency information,blurred edges,and difficulty in reconstruction.This paper proposesd a solar speckle image reconstruction algorithm combining gated fusion network and residual Fourier transform.The gated fusion network consists of a generator and two discriminators.The generator contains a deblurring module,a high-dimensional feature extraction module,a gating module and a reconstruction module.The deblurring module adopts the U-shaped network framework based on the double attention mechanism to obtain the deblurred features of the low-resolution image;the high-dimensional feature extraction module uses the convolution block of the residual Fourier transform to extract high-dimensional features containing image spatial details;the gating module fuses the above two features to obtain a weight map,the weight map weighted with the deblurred features,and then fused with high-dimensional features to obtain fused features;the reconstruction module uses the residual Fourier transform convolution block and pixel shuffling layer to reconstruct the fusion feature map obtained by the gating module to obtain a high-resolution image.Two discriminators are used to identify the authenticity of the deblurred image produced by the deblurring module and the high-resolution image produced by the reconstruction module,respectively.Finally,a combined training loss function including pixel content loss,perceptual loss and adversarial loss is designed to guide model training.Experimental results show that compared with existing deep learning reconstruction methods,the proposed method has stronger recovery ability of high-frequency information,clearer edge contours,higher structural similarity and peak signal-to-noise ratio indicators.

Key words: Solar speckle image reconstruction, Gated fusion network, Residual Fourier transform, Dual attention mechanism

CLC Number: 

  • TP391.41
[1]XIANG Y Y.Research on high speed high resolution solar reconstruction algorithm [D].Beijing:University of Chinese Academy of Science,2016:19-22.
[2]SCHULZ T J.Multiframe blind deconvolution of astronomical images[J].Journal of the Optical Society of America A,1993,10(5):1064-1073.
[3]LIU Z.Research on high-resolution image reconstruction andspatial characteristics of astronomical images[D].Beijing:University of Chinese Academy of Science,2003:25-39.
[4]GONSALVES R A,CHIDLAW R.Wavefront sensing by phase retrieval[C]//Applications of Digital Image Processing III.SPIE,1979:32-39.
[5]KUPYN O,MARTYNIUK T,WU J,et al.Deblurgan-v2:De-blurring(orders-of-magnitude) faster and better[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:8878-8887.
[6]CHENG S,WANG Y,HUANG H,et al.Nbnet:Noise basislearning for image denoising with subspace projection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:4896-4906.
[7]GUO Y,CHEN J,WANG J,et al.Closed-loop matters:Dual regression networks for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5407-5416.
[8]REN Y,JIANG M,FU Z,et al.Reconstruction of single-framesolar speckle image with cycle consistency loss and perceptual loss[C]//2019 6th International Conference on Information Science and Control Engineering(ICISCE).IEEE,2019:439-443.
[9]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[10]WANG H,WU X,HUANG Z,et al.High-frequency component helps explain the generalization of convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:8684-8694.
[11]MAO X,LIU Y,SHEN W,et al.Deep Residual Fourier Transformation for Single Image Deblurring[J].Computer Science-Computer Vision and Pattern Recognition.arXiv:2111.11745,2021.
[12]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[13]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[14]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block at-tention module[C]//Proceedings of the European Conference on Computer Vision.2018:3-19.
[15]REN W,MA L,ZHANG J,et al.Gated fusion network for single image dehazing[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3253-3261.
[16]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time singleimage and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[17]JOHNSON J,ALAHI A,LI F F.Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision.Cham:Springer,2016:694-711.
[18]MAO X,LI Q,XIE H,et al.Least squares generative adversarialnetworks[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.2017:2794-2802.
[19]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].Computer Science-Computer Vision and Pattern Recognition.arXiv:1409.1556,2014.
[20]CRESWELL A,WHITE T,DUMOULIN V,et al.Generativeadversarial networks:An overview[J].IEEE Signal Processing Magazine,2018,35(1):53-65.
[21]JIA P,HUANG Y,CAI B,et al.Solar image restoration withthe CycleGAN based on multi-fractal properties of texture features[J].The Astrophysical Journal Letters,2019,881(2):L30.
[22]LI F H,J M R,Y L,et al.Solar speckle image deblurring methodwith gradient guidance based on generative adversarial network [J].Computer Applications,2021,41(11):3345-3352.
[23]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].Computer Science-Machine Learning.arXiv:1412.6980,2014.
[24]TAO X,GAO H,SHEN X,et al.Scale-recurrent network fordeep image deblurring[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8174-8182.
[25]GAO H,TAO X,SHEN X,et al.Dynamic scene deblurring with parameter selective sharing and nested skip connections[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3848-3856.
[26]ZHANG K,LUO W,ZHONG Y,et al.Deblurring by realistic blurring[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2737-2746.
[27]NAH S,HYUN KIM T,MU LEE K.Deep multi-scale convolutional neural network for dynamic scene deblurring[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3883-3891.
[1] HUANG Yang-lin, HU Kai, GUO Jian-qiang, Peng ChengKey Laboratory of Intelligent Computing & Information Processing of Ministry of Education, Xiangtan University, Xiangtan, Hunan 411105,China. Liver CT Images Segmentation Based on Multi-scale Feature Fusion and Dual AttentionMechanism [J]. Computer Science, 2022, 49(11A): 210800162-9.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.