计算机科学 ›› 2021, Vol. 48 ›› Issue (8): 118-124.doi: 10.11896/jsjkx.200600150

所属专题: 医学图像

• 计算机图形学& 多媒体 • 上一篇    下一篇

融合时序监督和注意力机制的脉络膜新生血管分割

叶中玉, 吴梦麟   

  1. 南京工业大学计算机科学与技术学院 南京211816
  • 收稿日期:2020-06-23 修回日期:2020-09-09 发布日期:2021-08-10
  • 通讯作者: 吴梦麟(wumenglin@njtech.edu.cn)
  • 基金资助:
    国家自然科学基金(61701222)

Choroidal Neovascularization Segmentation Combining Temporal Supervision and Attention Mechanism

YE Zhong-yu, WU Meng-lin   

  1. College of Computer Science and Technology,Nanjing Tech University,Nanjing 211816,China
  • Received:2020-06-23 Revised:2020-09-09 Published:2021-08-10
  • About author:YE Zhong-yu,born in 1995,master.His main research interests include medical image segmentation,computer vision,and deep learning.(1355523372@qq.com)WU Meng-lin,born in 1982,associate professor,Ph.D,master's supervisor.His main research interests include medical image processing,medical imaging lesion analysis,and medical information retrieval and mining.
  • Supported by:
    National Natural Science Foundation of China(61701222).

摘要: 脉络膜新生血管(Choroidal Neovascularization,CNV)一般出现在老年性黄斑变性(Age-related macular degeneration,AMD)晚期,在光学相干断层成像(SD-OCT)中对CNV进行准确分割对AMD的诊疗具有重要意义。文中提出了一种融合时序模型与注意力机制的CNV分割网络。该方法将连续的SD-OCT图像输入分割网络,在编码器部分提取图片多尺度信息,为了更好地提取图片局部特征,又在跳跃连接部分加入注意力门;同时,为了解决分割不连续的问题,在分割网络池化后加入了时序约束网络以构建相邻帧连续性约束,并在损失函数中加入梯度约束以更好地保留病变边界;采用空间金字塔将两部分网络特征图融合以产生分割损失,提高了最终的分割精度。基于患者独立性对12名患者的200组眼睛数据进行实验,该方法的Dice系数为76.3%,overlap达到60.7%,能够在SD-OCT图像中对CNV进行可靠的分割。

关键词: 脉络膜新生血管, 时序网络, 特征融合, 医学图像分割, 注意力机制

Abstract: Choroidal neovascularization (CNV) generally occurs at the late stage of senile macular degeneration (AMD),and accurate segmentation of CNV in optical coherence tomography (SD-OCT) is of great significance for the diagnosis and treatment of AMD.This paper proposes a CNV multi-task segmentation network that combines time series model and attention mechanism.The continuous SD-OCT image is input into the segmentation network,and the multi-scale information of the picture is extracted in the encoder part.In order to better extract the local features of the picture,the attention gate is added in the skip connection part.In order to solve the problem of discontinuous scanning segmentation,after the segmentation network is pooled,the timing constraint network is passed to generate the continuity constraint of adjacent frames and gradient constraints are added to the loss function to better preserve the lesion boundary.The spatial pyramid is used to fuse the two parts of the network feature map to produce segmentation loss,which improves the final segmentation accuracy.Based on patient independence,effective cross-validation is performed on 200 eyes of 12 patients.The Dice coefficient reaches 76.3% and the overlap reaches 60.7%.CNV can be reliably segmented in SD-OCT images.

Key words: Attention mechanism, Choroidal neovascularization, Feature fusion, Medical image segmentation, Sequential network

中图分类号: 

  • TP391.41
[1]MIYATA M,OOTO S,HATA M,et al.Detection of myopicchoroidal neovascularization using optical coherence tomography angiography[J].American Journal of Ophthalmology,2016,165:108-114.
[2]WU J Y,YOU G D,YAN Y,et al.Analysis of blood vessels of diabetic retinopathy based on image segmentation[J].Chinese Medical Equipment Journal,2017,38(6):27-29,40.
[3]ZHU S,SHI F,XIANG D,et al.Choroid neovascularizationgrowth prediction with treatment based on reaction-diffusion model in 3-D OCT images[J].IEEE Journal of Biomedical and Health Informatics,2017,21(6):1667-1674.
[4]XIANG D,TIAN H,YANG X,et al.Automatic segmentation of retinal layer in OCT images with choroidal neovascularization[J].IEEE Transactions on Image Processing,2018,27(12):5880-5891.
[5]LI Y,NIU S,JI Z,et al.Automated choroidal neovascularization detection for time series SD-OCT images[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham:Springer,2018:381-388.
[6]BRANKIN E,MCCUILAGH P,BLACK N,et al.The optimisation of thresholding techniques for the identification of choroidal neovascular membranes in exudative age-related macular de-generation[C]//19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06).IEEE,2006:430-435.
[7]BRANKIN E,MCCULLAGH P,PATTON W,et al.Identification of choroidal neovascularisation on fluorescein angiograms using gradient vector flow active contours[C]//2008 Internatio-nal Machine Vision and Image Processing Conference.IEEE,2008:165-169.
[8]LIANG L M,HUANG C L,SHI F,et al.Vascular Segmentation of Fundus Image of Level Set Based on Shape Prior[J].Compu-ter Science,2018,41(7):1678-1692.
[9]ROY A G,CONJETI S,KARRI S P K,et al.ReLayNet:retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks[J].Biomedical Optics Express,2017,8(8):3627-3642.
[10]CHEN L C,ZHU Y,PAPANDREOU G,et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV).2018:801-818.
[11]XUE J,YAN S,WANG Y,et al.Unsupervised Segmentation of Choroidal Neovascularization for Optical Coherence Tomography Angiography by Grid Tissue-Like Membrane Systems[J].IEEE Access,2019,7:143058-143066.
[12]PERDOMO O,OTÁLORA S,GONZÁLEZ F A,et al.Oct-net:A convolutional network for automatic classification of normal and diabetic macular edema using sd-oct volumes[C]//2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).IEEE,2018:1423-1426.
[13]WANG G,LUO P,LIN L,et al.Learning object interactions and descriptions for semantic image segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5859-5867.
[14]QIN X,WANG Z,BAI Y,et al.FFA-Net:Feature Fusion Attention Network for Single Image Dehazing[C]//AAAI.2020:11908-11915.
[15]SINHA A,DOLZ J.Multi-scale self-guided attention for medical image segmentation[J].IEEE Journal of Biomedical and Health Informatics,2020,25(1):121-130.
[16]JETLEY S,LORD N A,LEE N,et al.Learn to pay attention[J].arXiv:1804.02391,2018.
[17]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical ImageComputing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[18]OKTAY O,SCHLEMPER J,FOLGOC L L,et al.Attention u-net:Learning where to look for the pancreas[J].arXiv:1804.03999,2018.
[19]SHI X J,CHEN Z R,WANG H,et al.Convolutional LSTM network:A machine learning approach for precipitation nowcasting[J].arXiv:1506.04214,2105.
[20]TAKIKAWA T,ACUNA D,JAMPANI V,et al.Gated-scnn:Gated shape cnns for semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision.2019:5229-5238.
[21]REN M,ZEMEL R S.End-to-end instance segmentation withrecurrent attention[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:6656-6664.
[22]ROMERA-PAREDES B,TORR P H S.Recurrent instance segmentation[C]//European Conference on Computer Vision.Cham:Springer,2016:312-329.
[23]SALVADOR A,BELLVER M,CAMPOS V,et al.Recurrentneural networks for semantic instance segmentation[J].arXiv:1712.00617,2017.
[24]ZHANG Y,JI Z,WANG Y,et al.Mpb-cnn:a multi-scale parallel branch cnn for choroidal neovascularization segmentation in sd-oct images[J].OSA Continuum,2019,2(3):1011-1027.
[25]HE K,SUN J.Convolutional Neural Networks at Constrained Time Cost[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5353-5360.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[3] 戴禹, 许林峰.
基于文本行匹配的跨图文本阅读方法
Cross-image Text Reading Method Based on Text Line Matching
计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[4] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[5] 熊丽琴, 曹雷, 赖俊, 陈希亮.
基于值分解的多智能体深度强化学习综述
Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization
计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[6] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7] 汪鸣, 彭舰, 黄飞虎.
基于多时间尺度时空图网络的交通流量预测模型
Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction
计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[8] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[9] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[10] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[11] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[12] 曾志贤, 曹建军, 翁年凤, 蒋国权, 徐滨.
基于注意力机制的细粒度语义关联视频-文本跨模态实体分辨
Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism
计算机科学, 2022, 49(7): 106-112. https://doi.org/10.11896/jsjkx.210500224
[13] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[14] 徐鸣珂, 张帆.
Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法
Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition
计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085
[15] 孟月波, 穆思蓉, 刘光辉, 徐胜军, 韩九强.
基于向量注意力机制GoogLeNet-GMP的行人重识别方法
Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism
计算机科学, 2022, 49(7): 142-147. https://doi.org/10.11896/jsjkx.210600198
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!