Computer Science

Computer Science ›› 2019, Vol. 46 ›› Issue (5): 221-227.doi: 10.11896/j.issn.1002-137X.2019.05.034

Special Issue: Medical Imaging

Previous Articles     Next Articles

Improved CycleGANs for Intravascular Ultrasound Image Enhancement

YAO Zhe-wei, YANG Feng, HUANG Jing, LIU Ya-qin   

  1. (School of Biomedical Engineering,Southern Medical University,Guangzhou 510515,China)
    (Guangdong Provincial Key Laboratory of Medical Image Processing,Southern Medical University,Guangzhou 510515,China)
  • Published:2019-05-15

PDF (PC)

1282

Abstract: Low-frequency and high-frequency ultrasound probes used in intravascular ultrasound (IVUS) image acquisition have their own characteristics.Doctors have to choose ultrasound probes with different frequencies according to clinical needs during the diagnosis of Coronary atherosclerosis and other diseases.Therefore,a Cycle Generative Adversarial Networks (CycleGANs) based on the Wasserstein distance for intravascular ultrasound images enhancement was presented to combine high-frequency ultrasonic details and overcome the problems of edge blur and low resolution of low-frequency ultrasound image,assisting doctors in the diagnosis of cardiovascular disease.Firstly, according to the shape characteristics of coronary artery,several approaches used for data augmentationsuch as rotating,scaling up or down and implementing gamma transformation,are applied to increase the number of IVUS samples in training set,in order to reduce the risk of over-fitting during the training stage.Then,in the spirit of adversarial training,a joint loss function based on adversarial loss and cycle-consistent loss is constructed.Finally,the Wasserstein distance is added to the loss function as a regular term to stabilize the training and accelerated the convergence process.The input of this model is a low-frequency IVUS image and the output is an enhanced IVUS image containing high frequency detail information.An international standard IVUS image database was used for verification in the experiment.Clarity,contrast and edge energy were used as evaluation criteria to quantify.It is verified that the convergence speed of this model is twice of the original CycleGANs model.Three evaluation criteria are increased by 15.8%,11.4% and 46.6%,respectively.The experimental results show that the W-CycleGANs model can learn the feature information of the image domain effectively.Based on the original CycleGANs algorithm,it can further enrich the details of image edges and enhance the diagnostic information,also improve the sensitivity of doctors to diagnosis cardiovascular disease.In addition,100 pieces of clinical IVUS images are used for verification and well enhancement results are gotten.

Key words: Deep lear-ning, Generation of anti-network, Image enhancement, Intravascular ultrasound, Wasserstein distance

CLC Number: 

  • TP391.4
[1]ZENG Y J,YANG X,XU H W,et al.Research on Ultrasound Carotid Image Segmentation Methods Based on Local Chan-Vese Model Using Level Set Method [J].Computer Science,2013,40(S2):304-308.(in Chinese)曾雅洁,杨鑫,徐红卫,等.基于局部Chan-Vese模型的超声颈动脉图像水平集分割方法研究[J].计算机科学,2013,40(S2):304-308.
[2]ZHAO R P,MENG X D.Application of intravascular ultrasound in coronary artery [J].Chinese Journal of Cardiovascular Research,2011,9(7):548-550.(in Chinese)赵瑞平,孟显达.冠脉血管内超声的应用研究[J].中国心血管病研究,2011,9(7):548-550.
[3]HUANG Z J,WANG Y N,WANG Q.Automatic Characterization Study of Atherothrombotic Plaques Based on Intravascular Ultrasound Images [J].Computer Science,2018,45(5):260-265.(in Chinese)黄志杰,王伊侬,王青.基于血管内超声图像的心血管动脉粥样硬化斑块组织自动定征的研究[J].计算机科学,2018,45(5):260-265.
[4]YUAN S F,YANG F,LIU S J,et al.Media-Adventitia Border Detection Based on Local Shape Structure Classification for Intravascular Ultrasound Images [J].Acta Electronica Sinica,2018,46(7):1601-1608.(in Chinese)袁绍锋,杨丰,刘树杰,等.基于局部形状结构分类的心血管内超声图像中外膜边界检测[J].电子学报,2018,46(7):1601-1608.
[5]HE F J,LI F Y,GONG W B,et al.An intracavitary convex array probe for detecting internal carotid artery disease [J].Journal of Southern Medical University,2009,29(8):1670-1672,1674.(in Chinese)何锋坚,李发友,龚渭冰,等.腔内凸阵超声探头检测颈动脉的初步研究[J].南方医科大学学报,2009,29(8):1670-1672,1674.
[6]ORTIZ S H C,CHIU T,FOX M D.Ultrasound image enhancement:A review[J].Biomedical Signal Processing & Control,2012,7(5):419-428.
[7]QIU X,HUANG J,YANG F,et al.Image enhancement based media-adventitia border detection in intravascular ultrasound ima-ges[J].Journal of Image and Graphics,2012(4):537-545.(in Chinese)邱璇,黄靖,杨丰,等.结合图像增强的心血管内超声中-外膜边缘检测[J].中国图象图形学报,2012(4):537-545.
[8]SHEN X Z,YE Q Z.Implementation of Ultrasonic Enhancement Based on Wiener Filtering [J].Journal of Data Acquisition and Processing,2018(3):455-460.(in Chinese)沈希忠,叶秋泽.基于维纳滤波的超声增强实现方法[J].数据采集与处理,2018(3):455-460.
[9]KER J,WANG L,RAO J,et al.Deep Learning Applications in Medical Image Analysis[J].IEEE Access,2017,PP(99):1.
[10]SHEN D,WU G,SUK H I.Deep Learning in Medical Image Analysis[J].Annual Review of Biomedical Engineering,2017,19(1):221-248.
[11]SUZUKI K.Overview of deep learning in medical imaging[J].Radiological Physics & Technology,2017,10(3):1-17.
[12]ZHANG L,ZHAO D,CHI X B.Review for Deep LearningBased on Medical Imaging Diagnosis [J].Computer Science,2017,44(S2):1-7.(in Chinese)张巧丽,赵地,迟学斌.基于深度学习的医学影像诊断综述[J].计算机科学,2017,44(S2):1-7.
[13]KINGMA D P,WELLING M.Auto-Encoding Variational Bayes [J].Preprint ArXiv:1312.6114 2013.
[14]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.
Generative adversarial nets[C]∥International Conference on Neural Information Processing Systems.2014,3:2672-2680.
[15]HU Y,GIBSON E,VERCAUTEREN T,et al.IntraoperativeOrgan Motion Models with an Ensemble of Conditional Generative Adversarial Networks[C]∥International Conference on Medical Image Computing and Computer-Assisted Intervention.2017:368-376.
[16]KITCHEN A,SEAH J.Deep Generative Adversarial NeuralNetworks for Realistic Prostate Lesion MRI Synthesis [J].Preprint ArXiv:1708.00129 2017.
[17]CHENNAMSETTY S S.Generative adversarial networks forbrain lesion detection[C]∥Medical Imaging 2017:Image Processing.2017,10133:101330G.
[18]SHIN H C,TENENHOLTZ N A,ROGERS J K,et al.Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks[C]∥International Workshop on Simulation and Synthesis in Medical Imaging.Springer,Cham,2018:1-11.
[19]QUAN T M,NGUYEN-DUC T,JEONG W K.CompressedSensing MRI Reconstruction Using a Generative Adversarial Network with a Cyclic Loss [J].IEEE Trans Med Imaging,2018,37(6):1488-1497.
[20]HIASA Y,OTAKE Y,TAKAO M,et al.Cross-modality image synthesis from unpaired data using CycleGAN:Effects of gra-dient consistency loss and training data size [J].Preprint Ar-Xiv:1803.06629 2018.
[21]ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks [C]∥2017 IEEE International Conference on Computer Vision.2017:2242-2251.
[22]HOFFMAN J,TZENG E,PARK T,et al.CyCADA:Cycle-Consistent Adversarial Domain Adaptation [J].Preprint ArXiv:1711.03213 2017.
[23]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-Image Translation with Conditional Adversarial Networks[C]∥Conference on Vision and Pattern Recognition.2016:5967.
[24]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN [J].Preprint ArXiv:1701.07875 2017.
[25]LU Y,TAI Y W,TANG C K.Conditional CycleGAN for At-tribute Guided Face Image Generation [J].Preprint ArXiv:1705.09966 2017.
[26]GULRAJANI I,AHMED F,ARJOVSKY M,et al.ImprovedTraining of Wasserstein GANs [J].Preprint ArXiv:1704.00028 2017.
[27]SHEN X,HERTZMANN A,JIA J,et al.Automatic PortraitSegmentation for Image Stylization [J].Computer Graphics Forum,2016,35(2):93-102.
[28]ABADI M,AGARWAL A,BARHAM P,et al.TensorFlow:Large-Scale Machine Learning on Heterogeneous Distributed Systems [J].Preprint ArXiv:1603.04467 2016.
[29]KINGMA D,BA J.Adam:A Method for Stochastic Optimization [J].Computer Science.Preprint ArXiv:1412.6980 2014.
[30]BALOCCO S,GATTA C,CIOMPI F,et al.Standardized evaluation methodology and reference database for evaluating IVUS image segmentation.Computerized Medical Imaging and Graphics,2014,38(2):70-90.
[31]GATYS L A,ECKER A S,BETHGE M.A Neural Algorithm ofArtistic Style [J].Computer Science.Preprint ArXiv:1508.06576 2015.
[1] ZHAO Ming-hua, ZHOU Tong-tong, DU Shuang-li, SHI Zheng-hao. Single Backlit Image Enhancement Based on Virtual Exposure Method [J]. Computer Science, 2022, 49(6A): 384-389.
[2] ZHAO Zheng-peng, LI Jun-gang, PU Yuan-yuan. Low-light Image Enhancement Based on Retinex Theory by Convolutional Neural Network [J]. Computer Science, 2022, 49(6): 199-209.
[3] JIANG Zong-li, FAN Ke, ZHANG Jin-li. Generative Adversarial Network and Meta-path Based Heterogeneous Network Representation Learning [J]. Computer Science, 2022, 49(1): 133-139.
[4] HUANG Xue-bing, WEI Jia-yi, SHEN Wen-yu, LING Li. MR Image Enhancement Based on Adaptive Weighted Duplicate Filtering and Homomorphic Filtering [J]. Computer Science, 2021, 48(6A): 21-27.
[5] YANG Xiu-zhang, WU Shuai, XIA Huan, YU Xiao-min. Research on Shui Characters Extraction and Recognition Based on Adaptive Image Enhancement Technology [J]. Computer Science, 2021, 48(6A): 74-79.
[6] ZHENG Chun-jun, WANG Chun-li, JIA Ning. Survey of Acoustic Feature Extraction in Speech Tasks [J]. Computer Science, 2020, 47(5): 110-119.
[7] LI Chang-xing, LEI Liu, ZHANG Xiao-lu. Brain CT and MRI Image Fusion Based on Morphological Image Enhancement and PCNN [J]. Computer Science, 2020, 47(10): 194-199.
[8] XU Min-min, KOU Guang-jie, MA Yun-yan, YUE Jun, JIA Shi-xiang, ZHANG Zhi-wang. Color Image Enhancement Algorithm Based on PCNN Internal Activities [J]. Computer Science, 2019, 46(6A): 259-262.
[9] YANG Xiu-zhang, XIA Huan, YU Xiao-min. Image Enhancement and Recognition Method Based on Shui-characters [J]. Computer Science, 2019, 46(11A): 324-328.
[10] LIU Yang, ZHANG Jie, ZHANG Hui. Study and Application of Improved Retinex Algorithm in Image Defogging [J]. Computer Science, 2018, 45(6A): 242-243.
[11] ZHOU Li-jun. Low-contrast Crack Extraction Method Based on Image Enhancement and Watershed Segmentation [J]. Computer Science, 2018, 45(6A): 259-261.
[12] HUANG Zhi-jie, WANG Yi-nong and WANG Qing. Automatic Characterization Study of Atherothrombotic Plaques Based on Intravascular Ultrasound Images [J]. Computer Science, 2018, 45(5): 260-265.
[13] ZHAO Jun-hui, WU Yu-feng, HU Kun-rong and PU Bin. Color Image Enhancement Algorithm Based on Lab Color Space and Tone Mapping [J]. Computer Science, 2018, 45(2): 297-300.
[14] SUN Quan, ZENG Xiao-qin. Image Inpainting Based on Generative Adversarial Networks [J]. Computer Science, 2018, 45(12): 229-234.
[15] ZHANG Xiang, WANG Wei, XIAO Di. Improved Image Enhancemengt Algorithm Based on Multi-scale Retinex with Chromaticity Preservation [J]. Computer Science, 2018, 45(10): 246-249.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.