计算机科学 ›› 2019, Vol. 46 ›› Issue (5): 221-227.doi: 10.11896/j.issn.1002-137X.2019.05.034

所属专题: 医学图像

• 图形图像与模式识别 • 上一篇    下一篇

改进型循环生成对抗网络的血管内超声图像增强

姚哲维, 杨丰, 黄靖, 刘娅琴   

  1. (南方医科大学生物医学工程学院 广州510515)
    (南方医科大学广东省医学图像处理重点实验室 广州510515)
  • 发布日期:2019-05-15
  • 作者简介:姚哲维(1994-),男,硕士,主要研究方向为机器学习、医学图像处理;杨 丰(1965-),男,教授,博士生导师,主要研究方向为模式识别、机器学习、医学图像处理、生物医学信号处理等,E-mail:yangf@smu.edu.cn(通信作者);黄 靖(1981-),男,博士,副教授,主要研究方向为生物医学信号处理;刘娅琴(1965-),女,硕士,教授,主要研究方向为电子信息应用、生物特征识别。
  • 基金资助:
    国家自然科学基金(61771233)资助。

Improved CycleGANs for Intravascular Ultrasound Image Enhancement

YAO Zhe-wei, YANG Feng, HUANG Jing, LIU Ya-qin   

  1. (School of Biomedical Engineering,Southern Medical University,Guangzhou 510515,China)
    (Guangdong Provincial Key Laboratory of Medical Image Processing,Southern Medical University,Guangzhou 510515,China)
  • Published:2019-05-15

摘要: 血管内超声(Intravascular Ultrasound,IVUS)图像采集过程中使用的低频或高频超声探头各有其特点,医生在诊断冠脉粥样硬化等疾病过程中需根据临床需求选择频率不同的超声探头。因此,文中提出一种基于Wasserstein距离的循环生成对抗网络(Cycle Generative Adversarial Networks,CycleGANs)用于增强血管内超声图像,其目的是融合高频超声细节信号,克服低频超声图像边缘模糊且分辨率较低等问题,以辅助医生诊断心血管疾病。首先基于心血管的形状特点,使用旋转、缩放和Gamma变换等方法将图像数据训练集增加30倍,降低网络训练过拟合的风险;然后利用对抗网络训练的思想,构建基于对抗损失和循环一致损失的联合函数,在损失函数中引入Wasserstein距离作为正则项,加快网络训练的收敛速度,解决训练不稳定的问题;最后以低频IVUS图像为输入对象,输出含有高频图像细节信息的IVUS增强图像。在实验过程中以国际标准IVUS图像数据库为基础进行算法验证比较;以清晰度、对比度和边缘能量为评价标准进行定量分析。实验结果:所提算法的收敛速度是原始CycleGANs模型的两倍,且3个评价标准数值分别提升了15.8%,11.4%和46.6%。实验结果表明:W-CycleGANs模型能够有效地学习高频图像域的特征信息,并且进一步丰富低频图像边缘细节,增强图像的诊断信息,提高医生判断心血管疾病的敏感性。此外,文中采用100幅临床IVUS图像数据进行推广验证,也获得了较好的增强效果。

关键词: Wasserstein距离, 深度学习, 生成对抗网络, 图像增强, 血管内超声

Abstract: Low-frequency and high-frequency ultrasound probes used in intravascular ultrasound (IVUS) image acquisition have their own characteristics.Doctors have to choose ultrasound probes with different frequencies according to clinical needs during the diagnosis of Coronary atherosclerosis and other diseases.Therefore,a Cycle Generative Adversarial Networks (CycleGANs) based on the Wasserstein distance for intravascular ultrasound images enhancement was presented to combine high-frequency ultrasonic details and overcome the problems of edge blur and low resolution of low-frequency ultrasound image,assisting doctors in the diagnosis of cardiovascular disease.Firstly, according to the shape characteristics of coronary artery,several approaches used for data augmentationsuch as rotating,scaling up or down and implementing gamma transformation,are applied to increase the number of IVUS samples in training set,in order to reduce the risk of over-fitting during the training stage.Then,in the spirit of adversarial training,a joint loss function based on adversarial loss and cycle-consistent loss is constructed.Finally,the Wasserstein distance is added to the loss function as a regular term to stabilize the training and accelerated the convergence process.The input of this model is a low-frequency IVUS image and the output is an enhanced IVUS image containing high frequency detail information.An international standard IVUS image database was used for verification in the experiment.Clarity,contrast and edge energy were used as evaluation criteria to quantify.It is verified that the convergence speed of this model is twice of the original CycleGANs model.Three evaluation criteria are increased by 15.8%,11.4% and 46.6%,respectively.The experimental results show that the W-CycleGANs model can learn the feature information of the image domain effectively.Based on the original CycleGANs algorithm,it can further enrich the details of image edges and enhance the diagnostic information,also improve the sensitivity of doctors to diagnosis cardiovascular disease.In addition,100 pieces of clinical IVUS images are used for verification and well enhancement results are gotten.

Key words: Deep lear-ning, Generation of anti-network, Image enhancement, Intravascular ultrasound, Wasserstein distance

中图分类号: 

  • TP391.4
[1]ZENG Y J,YANG X,XU H W,et al.Research on Ultrasound Carotid Image Segmentation Methods Based on Local Chan-Vese Model Using Level Set Method [J].Computer Science,2013,40(S2):304-308.(in Chinese)曾雅洁,杨鑫,徐红卫,等.基于局部Chan-Vese模型的超声颈动脉图像水平集分割方法研究[J].计算机科学,2013,40(S2):304-308.
[2]ZHAO R P,MENG X D.Application of intravascular ultrasound in coronary artery [J].Chinese Journal of Cardiovascular Research,2011,9(7):548-550.(in Chinese)赵瑞平,孟显达.冠脉血管内超声的应用研究[J].中国心血管病研究,2011,9(7):548-550.
[3]HUANG Z J,WANG Y N,WANG Q.Automatic Characterization Study of Atherothrombotic Plaques Based on Intravascular Ultrasound Images [J].Computer Science,2018,45(5):260-265.(in Chinese)黄志杰,王伊侬,王青.基于血管内超声图像的心血管动脉粥样硬化斑块组织自动定征的研究[J].计算机科学,2018,45(5):260-265.
[4]YUAN S F,YANG F,LIU S J,et al.Media-Adventitia Border Detection Based on Local Shape Structure Classification for Intravascular Ultrasound Images [J].Acta Electronica Sinica,2018,46(7):1601-1608.(in Chinese)袁绍锋,杨丰,刘树杰,等.基于局部形状结构分类的心血管内超声图像中外膜边界检测[J].电子学报,2018,46(7):1601-1608.
[5]HE F J,LI F Y,GONG W B,et al.An intracavitary convex array probe for detecting internal carotid artery disease [J].Journal of Southern Medical University,2009,29(8):1670-1672,1674.(in Chinese)何锋坚,李发友,龚渭冰,等.腔内凸阵超声探头检测颈动脉的初步研究[J].南方医科大学学报,2009,29(8):1670-1672,1674.
[6]ORTIZ S H C,CHIU T,FOX M D.Ultrasound image enhancement:A review[J].Biomedical Signal Processing & Control,2012,7(5):419-428.
[7]QIU X,HUANG J,YANG F,et al.Image enhancement based media-adventitia border detection in intravascular ultrasound ima-ges[J].Journal of Image and Graphics,2012(4):537-545.(in Chinese)邱璇,黄靖,杨丰,等.结合图像增强的心血管内超声中-外膜边缘检测[J].中国图象图形学报,2012(4):537-545.
[8]SHEN X Z,YE Q Z.Implementation of Ultrasonic Enhancement Based on Wiener Filtering [J].Journal of Data Acquisition and Processing,2018(3):455-460.(in Chinese)沈希忠,叶秋泽.基于维纳滤波的超声增强实现方法[J].数据采集与处理,2018(3):455-460.
[9]KER J,WANG L,RAO J,et al.Deep Learning Applications in Medical Image Analysis[J].IEEE Access,2017,PP(99):1.
[10]SHEN D,WU G,SUK H I.Deep Learning in Medical Image Analysis[J].Annual Review of Biomedical Engineering,2017,19(1):221-248.
[11]SUZUKI K.Overview of deep learning in medical imaging[J].Radiological Physics & Technology,2017,10(3):1-17.
[12]ZHANG L,ZHAO D,CHI X B.Review for Deep LearningBased on Medical Imaging Diagnosis [J].Computer Science,2017,44(S2):1-7.(in Chinese)张巧丽,赵地,迟学斌.基于深度学习的医学影像诊断综述[J].计算机科学,2017,44(S2):1-7.
[13]KINGMA D P,WELLING M.Auto-Encoding Variational Bayes [J].Preprint ArXiv:1312.6114 2013.
[14]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.
Generative adversarial nets[C]∥International Conference on Neural Information Processing Systems.2014,3:2672-2680.
[15]HU Y,GIBSON E,VERCAUTEREN T,et al.IntraoperativeOrgan Motion Models with an Ensemble of Conditional Generative Adversarial Networks[C]∥International Conference on Medical Image Computing and Computer-Assisted Intervention.2017:368-376.
[16]KITCHEN A,SEAH J.Deep Generative Adversarial NeuralNetworks for Realistic Prostate Lesion MRI Synthesis [J].Preprint ArXiv:1708.00129 2017.
[17]CHENNAMSETTY S S.Generative adversarial networks forbrain lesion detection[C]∥Medical Imaging 2017:Image Processing.2017,10133:101330G.
[18]SHIN H C,TENENHOLTZ N A,ROGERS J K,et al.Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks[C]∥International Workshop on Simulation and Synthesis in Medical Imaging.Springer,Cham,2018:1-11.
[19]QUAN T M,NGUYEN-DUC T,JEONG W K.CompressedSensing MRI Reconstruction Using a Generative Adversarial Network with a Cyclic Loss [J].IEEE Trans Med Imaging,2018,37(6):1488-1497.
[20]HIASA Y,OTAKE Y,TAKAO M,et al.Cross-modality image synthesis from unpaired data using CycleGAN:Effects of gra-dient consistency loss and training data size [J].Preprint Ar-Xiv:1803.06629 2018.
[21]ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks [C]∥2017 IEEE International Conference on Computer Vision.2017:2242-2251.
[22]HOFFMAN J,TZENG E,PARK T,et al.CyCADA:Cycle-Consistent Adversarial Domain Adaptation [J].Preprint ArXiv:1711.03213 2017.
[23]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-Image Translation with Conditional Adversarial Networks[C]∥Conference on Vision and Pattern Recognition.2016:5967.
[24]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN [J].Preprint ArXiv:1701.07875 2017.
[25]LU Y,TAI Y W,TANG C K.Conditional CycleGAN for At-tribute Guided Face Image Generation [J].Preprint ArXiv:1705.09966 2017.
[26]GULRAJANI I,AHMED F,ARJOVSKY M,et al.ImprovedTraining of Wasserstein GANs [J].Preprint ArXiv:1704.00028 2017.
[27]SHEN X,HERTZMANN A,JIA J,et al.Automatic PortraitSegmentation for Image Stylization [J].Computer Graphics Forum,2016,35(2):93-102.
[28]ABADI M,AGARWAL A,BARHAM P,et al.TensorFlow:Large-Scale Machine Learning on Heterogeneous Distributed Systems [J].Preprint ArXiv:1603.04467 2016.
[29]KINGMA D,BA J.Adam:A Method for Stochastic Optimization [J].Computer Science.Preprint ArXiv:1412.6980 2014.
[30]BALOCCO S,GATTA C,CIOMPI F,et al.Standardized evaluation methodology and reference database for evaluating IVUS image segmentation.Computerized Medical Imaging and Graphics,2014,38(2):70-90.
[31]GATYS L A,ECKER A S,BETHGE M.A Neural Algorithm ofArtistic Style [J].Computer Science.Preprint ArXiv:1508.06576 2015.
[1] 张佳, 董守斌.
基于评论方面级用户偏好迁移的跨领域推荐算法
Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer
计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131
[2] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[3] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[4] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[5] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[6] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[7] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[8] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[9] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[10] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[11] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[12] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[13] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[14] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[15] 祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋.
改进Faster R-CNN的光学遥感飞机目标检测
Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN
计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!