计算机科学 ›› 2021, Vol. 48 ›› Issue (6A): 38-42.doi: 10.11896/jsjkx.201000160

• 图像处理&多媒体技术 • 上一篇    下一篇

结合MCycleGAN与RFCNN实现太阳斑点图高分辨重建

崔雯昊1, 蒋慕蓉1, 杨磊2, 傅鹏铭1, 朱凌霄1   

  1. 1 云南大学信息学院 昆明650500
    2 中国科学院云南天文台 昆明650011
  • 出版日期:2021-06-10 发布日期:2021-06-17
  • 通讯作者: 蒋慕蓉(jiangmr@ynu.edu.cn)
  • 作者简介:1207042764@qq.com
  • 基金资助:
    云南省高校科技创新团队支持项目(IRTSTYN);国家自然科学基金(11773073)

Combining MCycleGAN and RFCNN to Realize High Resolution Reconstruction of Solar Speckle Image

CUI Wen-hao1, JIANG Mu-rong1, YANG Lei2, FU Peng-ming1, ZHU Ling-xiao1   

  1. 1 School of Information Science and Technology,Yunnan University,Kunming 650500,China
    2 Yunnan Observatories,Chinese Academy of Sciences,Kunming 650011,China
  • Online:2021-06-10 Published:2021-06-17
  • About author:CUI Wen-hao,born in 1996,postgra-duate.His main research directions include deep learning and image reconstruction.
    JIANG Mu-rong,born in 1963,professor,Ph.D.Her main research directions include mathematical methods of image processing and intelligent calculation.
  • Supported by:
    Science and Technology Innovation Team Support Project of Yunnan Province (IRTSTYN) and National Natural Science Foundation of China (11773073).

摘要: 太阳斑点图高分辨率重建是天文图像处理的重要研究内容之一。基于深度学习的图像高分辨率重建,通过神经网络模型学习获得低分辨率图像到高分辨率图像的端到端映射函数,能够恢复图像高频信息,但在用于特征单一、噪音较多、局部细节模糊的太阳斑点图重建时,存在边缘过于平滑、高频信息易丢失等不足。将输入图像与重建图像结构特征加入CycleGAN网络中得到MCycleGAN,利用生成器网络从结构特征中获取高频信息,计算特征差来增强网络重建高频信息的能力;将残差块和融合层加入DeepFuse网络中构建RFCNN,利用图像帧间相似信息互补进行多帧重建,使重建图像边缘更加清晰。利用所提方法的重建结果与云南天文台使用的斑点掩膜法Level1+的结果对比表明,所提算法具有误差小、重建图像清晰度高等优点。

关键词: MCycleGAN, RFCNN, 高分辨率重建, 深度学习, 太阳斑点图像

Abstract: High resolution reconstruction of solar speckle image is one of the important research contents in astronomical image processing.High resolution image reconstruction based on deep learning can obtain the end-to-end mapping function from low-resolution image to high-resolution image through neural network model learning,which can recover the high-frequency information of the image.However,when reconstructing the sun speckle image with single feature,more noise and fuzzy local details,there are some shortcomings such as too smooth edge and easy loss of high-frequency information.In this paper,the structure features of input image and reconstructed image are added to CycleGAN network to get MCycleGAN.High frequency information is obtained from structural features by generator network,and the feature difference is calculated to enhance the ability of network to reconstruct high-frequency information.Residual block and fusion layer are added to DeepFuse network to construct RFCNN,and multi frame reconstruction is carried out by using similar information between image frames.The edge of the reconstructed image is clearer.The reconstruction result is compared with the speckle mask method Level1+ used by Yunnan Observatory,which shows that the proposed algorithm has the advantages of small error and high definition of reconstructed image.

Key words: Deep learning, High resolution reconstruction, MCycleGAN, RFCNN, Solar speckle image

中图分类号: 

  • TP391.41
[1] XIANG Y Y,LIU Z,JIN Z Y.High resolution solar image reconstruction method [J].Progress in astronomy,2016,1:94-110.
[2] HUO Z X,ZHOU J F.Method of astronomical image recon-struction from speckle pattern [J].Progress in Astronomy,2010(1):74-94.
[3] WU R G.Image super resolution reconstruction based on deep learning [D].Chengdu:University of Chinese Academy of Sciences (Institute of Optoelectronic Technology,Chinese Academy of Sciences),2020.
[4] GOODFELLOW I J,POUGET-ANADIE J.Genera-tive Adversarial Networks [J].Advances in Neural Information Processing Systems,2014(3):2672-2680.
[5] TANG C,LI J.Unpaired Low-Dose CT Denoising NetworkBased on Cycle-Consistent Generative Adversarial Network with Prior Image Information[J].Computational and Mathematical Methods in Medicine,2019(12):1-11.
[6] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[C]//Computer Vision and Pattern Recognition.2014.
[7] LEDIG C,THEIS L.Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network [C]//CVPR.2017:574-591.
[8] PRABHAKAR K R,SRIKAR V S,BABU R V.DeepFuse:A Deep Unsupervised Approach for Exposure Fusion with Ex-treme Exposure Image Pairs [C]//ICCV.2017:472.
[9] TAI Y,GUO Y,CHEN Q,et al.Auto-embedding generativeadversarialnetworks for high resolution image synthesis[J].IEEE Transactions on Multimedia,2019(6):112.
[10] ZENG R H ,XU H M,HUANG W B,et al.Dense regression networkfor video grounding[C]//IEEE Conference on Compu-ter Vision and Pattern Recognition.2020:618.
[11] LEDIG C,THEIS L,HUSZAR F,et al.Photo-Realistic SingleImage Super-Resolution Using a Generative Adversarial Network [C]//IEEE Conference on Computer Vision & Pattern Recognition(CVPR).2016:137.
[12] LEDIG C,THEIS L.Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network [C]//CVPR.2017:574-591.
[13] ZHANG S L,YI B S,LI W Z,et al.Multi focus image fusion method based on image matting technology [J].Computer Applications,2016,36(7):1949-1953.
[14] LIU Y,LIU S,WANG Z.Multi-focus image fusion with dense SIFT[J].Information Fusion,2015,23:139-155.
[15] BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//The British Machine VisionConfe-rence(BMVC).2012.
[16] ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//Curves and Surfaces.Sprin-ger,2012:711-730.
[17] MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//IEEE International Conference on Computer Vision (ICCV).2001:416-423.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[9] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[10] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[11] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[12] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[13] 刘伟业, 鲁慧民, 李玉鹏, 马宁.
指静脉识别技术研究综述
Survey on Finger Vein Recognition Research
计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056
[14] 孙福权, 崔志清, 邹彭, 张琨.
基于多尺度特征的脑肿瘤分割算法
Brain Tumor Segmentation Algorithm Based on Multi-scale Features
计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217
[15] 康雁, 徐玉龙, 寇勇奇, 谢思宇, 杨学昆, 李浩.
基于Transformer和LSTM的药物相互作用预测
Drug-Drug Interaction Prediction Based on Transformer and LSTM
计算机科学, 2022, 49(6A): 17-21. https://doi.org/10.11896/jsjkx.210400150
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!