计算机科学 ›› 2021, Vol. 48 ›› Issue (8): 118-124.doi: 10.11896/jsjkx.200600150

• 计算机图形学& 多媒体 • 上一篇    下一篇

融合时序监督和注意力机制的脉络膜新生血管分割

叶中玉, 吴梦麟   

  1. 南京工业大学计算机科学与技术学院 南京211816
  • 收稿日期:2020-06-23 修回日期:2020-09-09 发布日期:2021-08-10
  • 通讯作者: 吴梦麟(wumenglin@njtech.edu.cn)
  • 基金资助:
    国家自然科学基金(61701222)

Choroidal Neovascularization Segmentation Combining Temporal Supervision and Attention Mechanism

YE Zhong-yu, WU Meng-lin   

  1. College of Computer Science and Technology,Nanjing Tech University,Nanjing 211816,China
  • Received:2020-06-23 Revised:2020-09-09 Published:2021-08-10
  • About author:YE Zhong-yu,born in 1995,master.His main research interests include medical image segmentation,computer vision,and deep learning.(1355523372@qq.com)WU Meng-lin,born in 1982,associate professor,Ph.D,master's supervisor.His main research interests include medical image processing,medical imaging lesion analysis,and medical information retrieval and mining.
  • Supported by:
    National Natural Science Foundation of China(61701222).

摘要: 脉络膜新生血管(Choroidal Neovascularization,CNV)一般出现在老年性黄斑变性(Age-related macular degeneration,AMD)晚期,在光学相干断层成像(SD-OCT)中对CNV进行准确分割对AMD的诊疗具有重要意义。文中提出了一种融合时序模型与注意力机制的CNV分割网络。该方法将连续的SD-OCT图像输入分割网络,在编码器部分提取图片多尺度信息,为了更好地提取图片局部特征,又在跳跃连接部分加入注意力门;同时,为了解决分割不连续的问题,在分割网络池化后加入了时序约束网络以构建相邻帧连续性约束,并在损失函数中加入梯度约束以更好地保留病变边界;采用空间金字塔将两部分网络特征图融合以产生分割损失,提高了最终的分割精度。基于患者独立性对12名患者的200组眼睛数据进行实验,该方法的Dice系数为76.3%,overlap达到60.7%,能够在SD-OCT图像中对CNV进行可靠的分割。

关键词: 脉络膜新生血管, 医学图像分割, 注意力机制, 时序网络, 特征融合

Abstract: Choroidal neovascularization (CNV) generally occurs at the late stage of senile macular degeneration (AMD),and accurate segmentation of CNV in optical coherence tomography (SD-OCT) is of great significance for the diagnosis and treatment of AMD.This paper proposes a CNV multi-task segmentation network that combines time series model and attention mechanism.The continuous SD-OCT image is input into the segmentation network,and the multi-scale information of the picture is extracted in the encoder part.In order to better extract the local features of the picture,the attention gate is added in the skip connection part.In order to solve the problem of discontinuous scanning segmentation,after the segmentation network is pooled,the timing constraint network is passed to generate the continuity constraint of adjacent frames and gradient constraints are added to the loss function to better preserve the lesion boundary.The spatial pyramid is used to fuse the two parts of the network feature map to produce segmentation loss,which improves the final segmentation accuracy.Based on patient independence,effective cross-validation is performed on 200 eyes of 12 patients.The Dice coefficient reaches 76.3% and the overlap reaches 60.7%.CNV can be reliably segmented in SD-OCT images.

Key words: Choroidal neovascularization, Medical image segmentation, Attention mechanism, Sequential network, Feature fusion

中图分类号: 

  • TP391.41
[1]MIYATA M,OOTO S,HATA M,et al.Detection of myopicchoroidal neovascularization using optical coherence tomography angiography[J].American Journal of Ophthalmology,2016,165:108-114.
[2]WU J Y,YOU G D,YAN Y,et al.Analysis of blood vessels of diabetic retinopathy based on image segmentation[J].Chinese Medical Equipment Journal,2017,38(6):27-29,40.
[3]ZHU S,SHI F,XIANG D,et al.Choroid neovascularizationgrowth prediction with treatment based on reaction-diffusion model in 3-D OCT images[J].IEEE Journal of Biomedical and Health Informatics,2017,21(6):1667-1674.
[4]XIANG D,TIAN H,YANG X,et al.Automatic segmentation of retinal layer in OCT images with choroidal neovascularization[J].IEEE Transactions on Image Processing,2018,27(12):5880-5891.
[5]LI Y,NIU S,JI Z,et al.Automated choroidal neovascularization detection for time series SD-OCT images[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham:Springer,2018:381-388.
[6]BRANKIN E,MCCUILAGH P,BLACK N,et al.The optimisation of thresholding techniques for the identification of choroidal neovascular membranes in exudative age-related macular de-generation[C]//19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06).IEEE,2006:430-435.
[7]BRANKIN E,MCCULLAGH P,PATTON W,et al.Identification of choroidal neovascularisation on fluorescein angiograms using gradient vector flow active contours[C]//2008 Internatio-nal Machine Vision and Image Processing Conference.IEEE,2008:165-169.
[8]LIANG L M,HUANG C L,SHI F,et al.Vascular Segmentation of Fundus Image of Level Set Based on Shape Prior[J].Compu-ter Science,2018,41(7):1678-1692.
[9]ROY A G,CONJETI S,KARRI S P K,et al.ReLayNet:retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks[J].Biomedical Optics Express,2017,8(8):3627-3642.
[10]CHEN L C,ZHU Y,PAPANDREOU G,et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV).2018:801-818.
[11]XUE J,YAN S,WANG Y,et al.Unsupervised Segmentation of Choroidal Neovascularization for Optical Coherence Tomography Angiography by Grid Tissue-Like Membrane Systems[J].IEEE Access,2019,7:143058-143066.
[12]PERDOMO O,OTÁLORA S,GONZÁLEZ F A,et al.Oct-net:A convolutional network for automatic classification of normal and diabetic macular edema using sd-oct volumes[C]//2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).IEEE,2018:1423-1426.
[13]WANG G,LUO P,LIN L,et al.Learning object interactions and descriptions for semantic image segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5859-5867.
[14]QIN X,WANG Z,BAI Y,et al.FFA-Net:Feature Fusion Attention Network for Single Image Dehazing[C]//AAAI.2020:11908-11915.
[15]SINHA A,DOLZ J.Multi-scale self-guided attention for medical image segmentation[J].IEEE Journal of Biomedical and Health Informatics,2020,25(1):121-130.
[16]JETLEY S,LORD N A,LEE N,et al.Learn to pay attention[J].arXiv:1804.02391,2018.
[17]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical ImageComputing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[18]OKTAY O,SCHLEMPER J,FOLGOC L L,et al.Attention u-net:Learning where to look for the pancreas[J].arXiv:1804.03999,2018.
[19]SHI X J,CHEN Z R,WANG H,et al.Convolutional LSTM network:A machine learning approach for precipitation nowcasting[J].arXiv:1506.04214,2105.
[20]TAKIKAWA T,ACUNA D,JAMPANI V,et al.Gated-scnn:Gated shape cnns for semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision.2019:5229-5238.
[21]REN M,ZEMEL R S.End-to-end instance segmentation withrecurrent attention[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:6656-6664.
[22]ROMERA-PAREDES B,TORR P H S.Recurrent instance segmentation[C]//European Conference on Computer Vision.Cham:Springer,2016:312-329.
[23]SALVADOR A,BELLVER M,CAMPOS V,et al.Recurrentneural networks for semantic instance segmentation[J].arXiv:1712.00617,2017.
[24]ZHANG Y,JI Z,WANG Y,et al.Mpb-cnn:a multi-scale parallel branch cnn for choroidal neovascularization segmentation in sd-oct images[J].OSA Continuum,2019,2(3):1011-1027.
[25]HE K,SUN J.Convolutional Neural Networks at Constrained Time Cost[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5353-5360.
[1] 王雷全, 候文艳, 袁韶祖, 赵欣, 林瑶, 吴春雷. 利用全局与局部帧级特征进行基于共享注意力的视频问答[J]. 计算机科学, 2021, 48(8): 145-149.
[2] 王施云, 杨帆. 基于U-Net特征融合优化策略的遥感影像语义分割方法[J]. 计算机科学, 2021, 48(8): 162-168.
[3] 张瑾, 段利国, 李爱萍, 郝晓燕. 基于注意力与门控机制相结合的细粒度情感分析[J]. 计算机科学, 2021, 48(8): 226-233.
[4] 宋龙泽, 万怀宇, 郭晟楠, 林友芳. 面向出租车空载时间预测的多任务时空图卷积网络[J]. 计算机科学, 2021, 48(7): 112-117.
[5] 桑春艳, 胥文, 贾朝龙, 文俊浩. 社交网络中基于注意力机制的网络舆情事件演化趋势预测[J]. 计算机科学, 2021, 48(7): 118-123.
[6] 卿来云, 张建功, 苗军. 在线异常事件检测的时序建模[J]. 计算机科学, 2021, 48(7): 206-212.
[7] 张曼, 李杰, 朱新忠, 沈霁, 成昊天. 基于改进DCGAN算法的遥感数据集增广方法[J]. 计算机科学, 2021, 48(6A): 80-84.
[8] 徐少伟, 秦品乐, 曾建朝, 赵致楷, 高媛, 王丽芳. 基于多级特征和全局上下文的纵膈淋巴结分割算法[J]. 计算机科学, 2021, 48(6A): 95-100.
[9] 刘翔宇, 蹇木伟, 鲁祥伟, 何为凯, 李晓峰, 尹义龙. 基于眼动点视觉先验与边缘优化的显著性检测[J]. 计算机科学, 2021, 48(6A): 107-112.
[10] 冯姣, 陆昶谕. 基于残差注意力网络的跨媒体检索方法[J]. 计算机科学, 2021, 48(6A): 122-126.
[11] 刘梦炀, 武利娟, 梁慧, 段旭磊, 刘尚卿, 高一波. 一种高精度LSTM-FC大气污染物浓度预测模型[J]. 计算机科学, 2021, 48(6A): 184-189.
[12] 潘芳, 张会兵, 董俊超, 首照宇. 基于高效Transformer的中文在线课程评论方面情感分析[J]. 计算机科学, 2021, 48(6A): 264-269.
[13] 潘明远, 宋慧慧, 张开华, 刘青山. 学习全局引导渐进特征聚合轻量级网络的显著性目标检测[J]. 计算机科学, 2021, 48(6): 103-109.
[14] 李佳倩, 严华. 基于跨列特征融合的人群计数方法[J]. 计算机科学, 2021, 48(6): 118-124.
[15] 徐泽, 帅仁俊, 刘开凯, 马力, 吴梦麟. 基于特征融合的文本到图像的生成[J]. 计算机科学, 2021, 48(6): 125-130.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 周燕萍,业巧林. 基于L1-范数距离的最小二乘对支持向量机[J]. 计算机科学, 2018, 45(4): 100 -105 .
[2] 张文华,张明慧,郭义昊,卢振泰,刘颖. 拟合精度引导的扩散加权图像配准[J]. 计算机科学, 2018, 45(5): 243 -249 .
[3] 徐涛,杜昱萱,吕宗磊. 基于线性规划的传感器节点布局模型[J]. 计算机科学, 2018, 45(7): 110 -115 .
[4] 王慧研,张腾飞,马福民. 基于空间距离自适应权重度量的粗糙K-means算法[J]. 计算机科学, 2018, 45(7): 190 -196 .
[5] 赵楠, 张小芳, 张利军. 不平衡数据分类研究综述[J]. 计算机科学, 2018, 45(6A): 22 -27 .
[6] 邱先标, 陈笑蓉. 一种基于SA_LDA模型的文本相似度计算方法[J]. 计算机科学, 2018, 45(6A): 106 -109 .
[7] 马晓迪, 吴茜茵, 金忠. 基于字典和加权低秩恢复的显著目标检测[J]. 计算机科学, 2018, 45(6A): 146 -150 .
[8] 王磊, 梁燕, 孙尚勇, 王光宇. 多载波时分多址及其快速卷积实现[J]. 计算机科学, 2018, 45(8): 88 -93 .
[9] 汪琳娜, 杨新, 杨习贝. 监督邻域粗糙集[J]. 计算机科学, 2018, 45(8): 186 -190 .
[10] 周雯, 史雪菲, 吴毅坚, 赵文耘. 数据需求驱动的Storm应用辅助开发框架[J]. 计算机科学, 2018, 45(9): 81 -88 .