计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (11A): 220800229-7.doi: 10.11896/jsjkx.220800229

• 图像处理&多媒体技术 • 上一篇    下一篇

结合门控融合网络和残差傅里叶变换重建太阳斑点图

黄亚群1, 郑培煜1, 蒋慕蓉1, 杨磊2, 罗俊1   

  1. 1 云南大学信息学院 昆明 650500
    2 中国科学院云南天文台 昆明 650011
  • 发布日期:2023-11-09
  • 通讯作者: 郑培煜(zhengpy@mail.ynu.edu.cn)
  • 作者简介:(huangyq@ynu.edu.cn)
  • 基金资助:
    国家自然科学基金(11773073);云南省高校科技创新团队支持项目(IRTSTYN);云南大学专业学位研究生教学案例库建设项目(2022XJALK02)

Reconstruction of Solar Speckle Image Combined with Gated Fusion Network and Residual Fourier Transform

HUANG Yaqun1, ZHENG Peiyu1, JIANG Murong1, YANG Lei2, LUO Jun1   

  1. 1 School of Information Science and Engineering,Yunnan University,Kunming 650500,China
    2 Yunnan Observatories,Chinese Academy of Sciences,Kunming 650011,China
  • Published:2023-11-09
  • About author:HUANG Yaqun,born in 1971,associate professor,is a member of China Computer Federation.Her main research interests include image reconstruction and intelligent data analysis.
    ZHENG Peiyu,born in 1997,postgra-duate.His main research interests include deep learning and image reconstruction.
  • Supported by:
    National Natural Science Foundation of China(11773073),Science and Technology Innovation Team Sopport Project of Yunnan Province(IRTSTYN) and Construction Project of Teaching Case Base for Professional Degree Graduates of Yunnan University(2022XJALK02).

PDF (PC)

146

摘要： 使用现有深度学习算法重建云南天文台拍摄的高度模糊太阳斑点图像时,出现高频信息丢失、边缘模糊、重建难度大等问题。对此,提出一种结合门控融合网络与残差傅里叶变换的太阳斑点图重建算法,其中门控融合网络由一个生成器和两个鉴别器组成,生成器包含去模糊模块、高维特征提取模块、门控模块和重建模块。去模糊模块采用基于双注意力机制的U形网络框架,获取低分辨率图像去模糊后的特征;高维特征提取模块使用残差傅里叶变换的卷积块,提取包含图像空间细节的高维特征;门控模块将上述两个特征进行融合,得到权重图,与去模糊后的特征进行加权后,再与高维特征融合,得到融合特征;重建模块采用残差傅里叶变换的卷积块和像素混洗层,将门控模块得到的融合特征图进行重建,得到高分辨率图像。利用两个鉴别器分别鉴别去模糊模块产生的去模糊图像和重建模块产生的高分辨率图像的真实性。最后,设计包含像素内容损失、感知损失和对抗损失的组合训练损失函数,指导模型训练。实验结果显示,所提方法与现有深度学习重建方法相比,高频信息的恢复能力更强,边缘轮廓更清晰,结构相似性和峰值信噪比指标更高。

关键词: 太阳斑点图重建, 门控融合网络, 残差傅里叶变换, 双注意力机制

Abstract: When using the existing deep learning algorithm to reconstruct the highly blurred solar speckle image taken by the Yunnan Observatory,there are problems such as loss of high-frequency information,blurred edges,and difficulty in reconstruction.This paper proposesd a solar speckle image reconstruction algorithm combining gated fusion network and residual Fourier transform.The gated fusion network consists of a generator and two discriminators.The generator contains a deblurring module,a high-dimensional feature extraction module,a gating module and a reconstruction module.The deblurring module adopts the U-shaped network framework based on the double attention mechanism to obtain the deblurred features of the low-resolution image;the high-dimensional feature extraction module uses the convolution block of the residual Fourier transform to extract high-dimensional features containing image spatial details;the gating module fuses the above two features to obtain a weight map,the weight map weighted with the deblurred features,and then fused with high-dimensional features to obtain fused features;the reconstruction module uses the residual Fourier transform convolution block and pixel shuffling layer to reconstruct the fusion feature map obtained by the gating module to obtain a high-resolution image.Two discriminators are used to identify the authenticity of the deblurred image produced by the deblurring module and the high-resolution image produced by the reconstruction module,respectively.Finally,a combined training loss function including pixel content loss,perceptual loss and adversarial loss is designed to guide model training.Experimental results show that compared with existing deep learning reconstruction methods,the proposed method has stronger recovery ability of high-frequency information,clearer edge contours,higher structural similarity and peak signal-to-noise ratio indicators.

Key words: Solar speckle image reconstruction, Gated fusion network, Residual Fourier transform, Dual attention mechanism

中图分类号: 

  • TP391.41
[1]XIANG Y Y.Research on high speed high resolution solar reconstruction algorithm [D].Beijing:University of Chinese Academy of Science,2016:19-22.
[2]SCHULZ T J.Multiframe blind deconvolution of astronomical images[J].Journal of the Optical Society of America A,1993,10(5):1064-1073.
[3]LIU Z.Research on high-resolution image reconstruction andspatial characteristics of astronomical images[D].Beijing:University of Chinese Academy of Science,2003:25-39.
[4]GONSALVES R A,CHIDLAW R.Wavefront sensing by phase retrieval[C]//Applications of Digital Image Processing III.SPIE,1979:32-39.
[5]KUPYN O,MARTYNIUK T,WU J,et al.Deblurgan-v2:De-blurring(orders-of-magnitude) faster and better[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:8878-8887.
[6]CHENG S,WANG Y,HUANG H,et al.Nbnet:Noise basislearning for image denoising with subspace projection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:4896-4906.
[7]GUO Y,CHEN J,WANG J,et al.Closed-loop matters:Dual regression networks for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5407-5416.
[8]REN Y,JIANG M,FU Z,et al.Reconstruction of single-framesolar speckle image with cycle consistency loss and perceptual loss[C]//2019 6th International Conference on Information Science and Control Engineering(ICISCE).IEEE,2019:439-443.
[9]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[10]WANG H,WU X,HUANG Z,et al.High-frequency component helps explain the generalization of convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:8684-8694.
[11]MAO X,LIU Y,SHEN W,et al.Deep Residual Fourier Transformation for Single Image Deblurring[J].Computer Science-Computer Vision and Pattern Recognition.arXiv:2111.11745,2021.
[12]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[13]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[14]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block at-tention module[C]//Proceedings of the European Conference on Computer Vision.2018:3-19.
[15]REN W,MA L,ZHANG J,et al.Gated fusion network for single image dehazing[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3253-3261.
[16]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time singleimage and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[17]JOHNSON J,ALAHI A,LI F F.Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision.Cham:Springer,2016:694-711.
[18]MAO X,LI Q,XIE H,et al.Least squares generative adversarialnetworks[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.2017:2794-2802.
[19]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].Computer Science-Computer Vision and Pattern Recognition.arXiv:1409.1556,2014.
[20]CRESWELL A,WHITE T,DUMOULIN V,et al.Generativeadversarial networks:An overview[J].IEEE Signal Processing Magazine,2018,35(1):53-65.
[21]JIA P,HUANG Y,CAI B,et al.Solar image restoration withthe CycleGAN based on multi-fractal properties of texture features[J].The Astrophysical Journal Letters,2019,881(2):L30.
[22]LI F H,J M R,Y L,et al.Solar speckle image deblurring methodwith gradient guidance based on generative adversarial network [J].Computer Applications,2021,41(11):3345-3352.
[23]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].Computer Science-Machine Learning.arXiv:1412.6980,2014.
[24]TAO X,GAO H,SHEN X,et al.Scale-recurrent network fordeep image deblurring[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8174-8182.
[25]GAO H,TAO X,SHEN X,et al.Dynamic scene deblurring with parameter selective sharing and nested skip connections[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3848-3856.
[26]ZHANG K,LUO W,ZHONG Y,et al.Deblurring by realistic blurring[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2737-2746.
[27]NAH S,HYUN KIM T,MU LEE K.Deep multi-scale convolutional neural network for dynamic scene deblurring[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3883-3891.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发