计算机科学 ›› 2021, Vol. 48 ›› Issue (4): 169-173.doi: 10.11896/jsjkx.200600047

• 计算机图形学&多媒体 • 上一篇    下一篇

基于语义对比生成对抗网络的高倍欠采MRI重建

马凤飞1, 蔺素珍1, 刘峰2, 王丽芳1, 李大威1   

  1. 1 中北大学大数据学院 太原030051
    2 昆士兰大学信息技术与电子工程学院 布里斯班QLD 4072
  • 收稿日期:2020-06-24 修回日期:2020-09-15 出版日期:2021-04-15 发布日期:2021-04-09
  • 通讯作者: 蔺素珍(lsz@nuc.edu.cn)
  • 基金资助:
    山西省应用基础研究项目(201701D121062);山西省自然科学基金(201901D111152);中北大学第十六届研究生科技立项资助项目(20191635)

Semantic-contrast Generative Adversarial Network Based Highly Undersampled MRI Reconstruction

MA Feng-fei1, LIN Su-zhen1, LIU Feng2, WANG Li-fang1, LI Da-wei1   

  1. 1 School of Data Science and Technology,North University of China,Taiyuan 030051,China
    2 School of Information Technology and Electrical Engineering,The University of Queensland,Brisbane QLD 4072,Australia
  • Received:2020-06-24 Revised:2020-09-15 Online:2021-04-15 Published:2021-04-09
  • About author:MA Feng-fei,born in 1991,postgra-duate,is a member of China Computer Federation.Her main research interests include image processing and MRI reconstruction.(751812591@qq.com)
    LIN Su-zhen,born in 1966,Ph.D,professor,master supervisor,is a member of China Computer Federation.Her main research interests include image processing and information fusion.
  • Supported by:
    Applied Basic Research Project of Shanxi Province,China(201701D121062), Natural Science Foundation of Shanxi Province,China(201901D111152) and 16th Graduate Science and Technology Project of North University of China(20191635).

摘要: 利用数据的稀疏性从随机欠采样的K空间重建图像,是解决磁共振成像(Magnetic Resonance Imaging,MRI)因采集时间过长而应用受限问题的主要手段。然而,现有的方法重建高倍欠采图像时纹理细节丢失严重。针对这一问题,借鉴生成对抗网络的对抗学习思想,文中提出一种基于语义对比生成对抗网络的高倍欠采MRI重建方法(Semantic-Contrast Generative Adversarial Network,SC-GAN)。该方法由连续的两部分组成。第一部分将笛卡尔高倍随机欠采样MRI图像输入基于U-NET的生成器,与鉴别器不断博弈对抗生成初步重建图像,以此构建重建子网;另一部分是语义对比子网,通过VGG-16比较初步重建图像与全采样图像的语义信息,比较结果反馈给第一部分进行参数调优,直到生成最佳的重建图像。实验结果表明,在加速因子高达7(14%)时,获得了主客观评价结果均较好的重建图像。与先进的重建方法相比,所提方法的内存损耗更低、收敛速度更快且纹理细节更丰富,可为下一代MRI机器的研发提供算法支持。

关键词: MRI重建, 高倍欠采图像, 深度学习, 生成对抗网络, 稀疏性, 语义对比

Abstract: Exploiting the sparse nature of data to reconstruct image from randomly undersampled K-space is the main method to solve the problem of limited application of magnetic resonance imaging(MRI) due to long acquisition time.However,the loss of textures is serious when the existing methods are used to reconstruct the highly undersampled MRI.Aiming at this problem,referring to the adversarial learning idea of generative adversarial networks(GAN),a novel method for highly undersampled MRI reconstruction based on semantic-contrast generative adversarial network(SC-GAN) is proposed.This method consists of two successive parts.In the first part,the MRI image with Cartesian highly random undersampled is input into the U-NET-based ge-nerator to compete with the discriminator for generating the initial reconstructed image,so as to construct the reconstruction subnet.The other part is the semantic contrast subnet,which compares the semantic information between the initial reconstructed image and its fully-sampled image with VGG-16 network.Comparison results are fed back to the first part for parameter adjustment until the best reconstructed image is generated .Experimental results show that the excellent reconstructed results are verified by subjective and objective evaluation when the acceleration factor is up to 7(14%).Compared with the state-of-art methods,the proposed SC-GAN has lower memory consumption,faster convergence speed and more textures,and can provide algorithm support for the development of next-generation MRI machines.

Key words: Deep learning, Generative adversarial network, Highly undersampled images, MRI reconstruction, Semantic contrast, Sparsity

中图分类号: 

  • TP391
[1]SCHLEMPER J,CABALLERO J,HAJNAL J V,et al.A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction[J].IEEE transactions on medical imaging,2018,37(2):491-503.
[2]DESHMANE A,GULANI V,GRISWOLD M A,et al.Parallel MR imaging[J].Journal of Magnetic Resonance Imaging,2012,36(1):55-72.
[3]UECKER M,LAI P,MURPHY M J,et al.ESPIRiT—an eigenvalue approach to autocalibrating parallel MRIwhere SENSE meets GRAPPA[J].Magnetic Resonance in Medicine,2014,71(3):990-1001.
[4]HAMMERNIK K,KLATZER T,KOBLER E,et al.Learning a variational network for reconstruction of accelerated MRI data[J].Magnetic Resonance in Medicine,2018,79(6):3055-3071.
[5]WU Y,MA Y,LIU J,et al.Self-attention convolutional neural network for improved MR image reconstruction[J].Information Sciences,2019,490:317-328.
[6]SENOUF O,VEDULA S,WEISS T,et al.Self-supervised lear-ning of inverse problem solvers in medical imaging[J].arXiv:1905.09325,2019.
[7]YANG G,YU S,DONG H,et al.DAGAN:deep de-aliasing ge-nerative adversarial networks for fast compressed sensing MRI re-construction[J].IEEE Transactions on Medical Imaging,2018,37(6):1310-1321.
[8]HUANG J,ZHANG S,METAXAS D.Efficient MR image re-construction for compressed MR imaging[J].Medical Image Analysis,2011,15(5):670-679.
[9]WANG Y,YING L.Undersampled dynamic magnetic resonance imaging using kernel principal component analysis[C]//2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.IEEE,2014:1533-1536.
[10]YANG J,ZHANG Y,YIN W.A fast alternating direction me-thod for TVL1-L2 signal reconstruction from partial Fourier data[J].IEEE Journal of Selected Topics in Signal Processing,2010,4(2):288-297.
[11]YAO H,DAI F,ZHANG S,et al.Dr2-net:Deep residual reconstruction network for image compressive sensing[J].Neurocomputing,2019,359:483-493.
[12]LIU Y H,LIU S Y,FU F X.Optimization of Compressed Sen-sing Reconstruction Algorithms Based on Convolutional Neural Network[J].Computer Science,2020,47(3):143-148.
[13]WANG S,SU Z,YING L,et al.Accelerating magnetic resonance imaging via deep learning[C]//2016 IEEE 13th International Symposium on Biomedical Imaging(ISBI).IEEE,2016:514-517.
[14]LEE D,YOO J,YE J C.Deep artifact learning for compressed sensing and parallel MRI[J].arXiv:1703.01120,2017.
[15]QIN C,SCHLEMPER J,CABALLERO J,et al.Convolutionalrecurrent neural networks for dynamic MR image reconstruction[J].IEEE Transactions on Medical Imaging,2018,38(1):280-290.
[16]KE Z,WANG S,CHENG H,et al.CRDN:Cascaded Residual Dense Networks for Dynamic MR Imaging with Edge-enhanced Loss Constraint[J].arXiv:1901.06111,2019.
[17]QUAN T M,NGUYEN-DUC T,JEONG W K.Compressedsensing MRI reconstruction using a generative adversarial network with a cyclic loss[J].IEEE Transactions on Medical Imaging,2018,37(6):1488-1497.
[18]YANG X L,LIN S Z.Method for multi-band image feature-level fusion based on the attention mechanism[J].Journal of Xidian University,2020,47(1):120-127.
[19]ZHENG Z,HU Q H,LIU Q S,et al.Quantizing Weights and Activations in Generative Adversarial Networks[J].Computer Science,2020,47(5):144-148.
[20]TONG G,LI Y,CHEN H,et al.Improved U-NET network for pulmonary nodules segmentation[J].Optik,2018,174:460-469.
[21]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[22]DAI Y,ZHUANG P.Compressed Sensing MRI via a Multi-scale Dilated Residual Convolution Network[J].arXiv:1906.05251,2019.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 张佳, 董守斌.
基于评论方面级用户偏好迁移的跨领域推荐算法
Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer
计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131
[4] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[5] 李霞, 马茜, 白梅, 王习特, 李冠宇, 宁博.
RIIM:基于独立模型的在线缺失值填补
RIIM:Real-Time Imputation Based on Individual Models
计算机科学, 2022, 49(8): 56-63. https://doi.org/10.11896/jsjkx.210600180
[6] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[7] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[8] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[9] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[10] 孙晓寒, 张莉.
基于评分区域子空间的协同过滤推荐算法
Collaborative Filtering Recommendation Algorithm Based on Rating Region Subspace
计算机科学, 2022, 49(7): 50-56. https://doi.org/10.11896/jsjkx.210600062
[11] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[12] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[13] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[14] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[15] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!