Computer Science ›› 2021, Vol. 48 ›› Issue (8): 118-124.doi: 10.11896/jsjkx.200600150

Special Issue: Medical Imaging

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Choroidal Neovascularization Segmentation Combining Temporal Supervision and Attention Mechanism

YE Zhong-yu, WU Meng-lin   

  1. College of Computer Science and Technology,Nanjing Tech University,Nanjing 211816,China
  • Received:2020-06-23 Revised:2020-09-09 Published:2021-08-10
  • About author:YE Zhong-yu,born in 1995,master.His main research interests include medical image segmentation,computer vision,and deep learning.( Meng-lin,born in 1982,associate professor,Ph.D,master's supervisor.His main research interests include medical image processing,medical imaging lesion analysis,and medical information retrieval and mining.
  • Supported by:
    National Natural Science Foundation of China(61701222).

Abstract: Choroidal neovascularization (CNV) generally occurs at the late stage of senile macular degeneration (AMD),and accurate segmentation of CNV in optical coherence tomography (SD-OCT) is of great significance for the diagnosis and treatment of AMD.This paper proposes a CNV multi-task segmentation network that combines time series model and attention mechanism.The continuous SD-OCT image is input into the segmentation network,and the multi-scale information of the picture is extracted in the encoder part.In order to better extract the local features of the picture,the attention gate is added in the skip connection part.In order to solve the problem of discontinuous scanning segmentation,after the segmentation network is pooled,the timing constraint network is passed to generate the continuity constraint of adjacent frames and gradient constraints are added to the loss function to better preserve the lesion boundary.The spatial pyramid is used to fuse the two parts of the network feature map to produce segmentation loss,which improves the final segmentation accuracy.Based on patient independence,effective cross-validation is performed on 200 eyes of 12 patients.The Dice coefficient reaches 76.3% and the overlap reaches 60.7%.CNV can be reliably segmented in SD-OCT images.

Key words: Attention mechanism, Choroidal neovascularization, Feature fusion, Medical image segmentation, Sequential network

CLC Number: 

  • TP391.41
[1]MIYATA M,OOTO S,HATA M,et al.Detection of myopicchoroidal neovascularization using optical coherence tomography angiography[J].American Journal of Ophthalmology,2016,165:108-114.
[2]WU J Y,YOU G D,YAN Y,et al.Analysis of blood vessels of diabetic retinopathy based on image segmentation[J].Chinese Medical Equipment Journal,2017,38(6):27-29,40.
[3]ZHU S,SHI F,XIANG D,et al.Choroid neovascularizationgrowth prediction with treatment based on reaction-diffusion model in 3-D OCT images[J].IEEE Journal of Biomedical and Health Informatics,2017,21(6):1667-1674.
[4]XIANG D,TIAN H,YANG X,et al.Automatic segmentation of retinal layer in OCT images with choroidal neovascularization[J].IEEE Transactions on Image Processing,2018,27(12):5880-5891.
[5]LI Y,NIU S,JI Z,et al.Automated choroidal neovascularization detection for time series SD-OCT images[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham:Springer,2018:381-388.
[6]BRANKIN E,MCCUILAGH P,BLACK N,et al.The optimisation of thresholding techniques for the identification of choroidal neovascular membranes in exudative age-related macular de-generation[C]//19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06).IEEE,2006:430-435.
[7]BRANKIN E,MCCULLAGH P,PATTON W,et al.Identification of choroidal neovascularisation on fluorescein angiograms using gradient vector flow active contours[C]//2008 Internatio-nal Machine Vision and Image Processing Conference.IEEE,2008:165-169.
[8]LIANG L M,HUANG C L,SHI F,et al.Vascular Segmentation of Fundus Image of Level Set Based on Shape Prior[J].Compu-ter Science,2018,41(7):1678-1692.
[9]ROY A G,CONJETI S,KARRI S P K,et al.ReLayNet:retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks[J].Biomedical Optics Express,2017,8(8):3627-3642.
[10]CHEN L C,ZHU Y,PAPANDREOU G,et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV).2018:801-818.
[11]XUE J,YAN S,WANG Y,et al.Unsupervised Segmentation of Choroidal Neovascularization for Optical Coherence Tomography Angiography by Grid Tissue-Like Membrane Systems[J].IEEE Access,2019,7:143058-143066.
[12]PERDOMO O,OTÁLORA S,GONZÁLEZ F A,et al.Oct-net:A convolutional network for automatic classification of normal and diabetic macular edema using sd-oct volumes[C]//2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).IEEE,2018:1423-1426.
[13]WANG G,LUO P,LIN L,et al.Learning object interactions and descriptions for semantic image segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5859-5867.
[14]QIN X,WANG Z,BAI Y,et al.FFA-Net:Feature Fusion Attention Network for Single Image Dehazing[C]//AAAI.2020:11908-11915.
[15]SINHA A,DOLZ J.Multi-scale self-guided attention for medical image segmentation[J].IEEE Journal of Biomedical and Health Informatics,2020,25(1):121-130.
[16]JETLEY S,LORD N A,LEE N,et al.Learn to pay attention[J].arXiv:1804.02391,2018.
[17]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical ImageComputing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[18]OKTAY O,SCHLEMPER J,FOLGOC L L,et al.Attention u-net:Learning where to look for the pancreas[J].arXiv:1804.03999,2018.
[19]SHI X J,CHEN Z R,WANG H,et al.Convolutional LSTM network:A machine learning approach for precipitation nowcasting[J].arXiv:1506.04214,2105.
[20]TAKIKAWA T,ACUNA D,JAMPANI V,et al.Gated-scnn:Gated shape cnns for semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision.2019:5229-5238.
[21]REN M,ZEMEL R S.End-to-end instance segmentation withrecurrent attention[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:6656-6664.
[22]ROMERA-PAREDES B,TORR P H S.Recurrent instance segmentation[C]//European Conference on Computer Vision.Cham:Springer,2016:312-329.
[23]SALVADOR A,BELLVER M,CAMPOS V,et al.Recurrentneural networks for semantic instance segmentation[J].arXiv:1712.00617,2017.
[24]ZHANG Y,JI Z,WANG Y,et al.Mpb-cnn:a multi-scale parallel branch cnn for choroidal neovascularization segmentation in sd-oct images[J].OSA Continuum,2019,2(3):1011-1027.
[25]HE K,SUN J.Convolutional Neural Networks at Constrained Time Cost[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5353-5360.
[1] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[3] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[5] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[6] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[9] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[10] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[11] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[12] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[15] MENG Yue-bo, MU Si-rong, LIU Guang-hui, XU Sheng-jun, HAN Jiu-qiang. Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism [J]. Computer Science, 2022, 49(7): 142-147.
Full text



No Suggested Reading articles found!