计算机科学 ›› 2022, Vol. 49 ›› Issue (11A): 210800162-9.doi: 10.11896/jsjkx.210800162

• 图像处理&多媒体技术 • 上一篇    下一篇

基于多尺度特征融合和双重注意力机制的肝脏CT图像分割

黄扬林, 胡凯, 郭建强, 彭诚   

  1. 湘潭大学智能计算与信息处理教育部重点实验室 湖南 湘潭 411105
    湘潭大学计算机学院·网络空间安全学院 湖南 湘潭 411105
  • 出版日期:2022-11-10 发布日期:2022-11-21
  • 通讯作者: 胡凯(kaihu@xtu.edu.cn)
  • 作者简介:(yanglinhuang@yeah.net)
  • 基金资助:
    国家自然科学基金(62272404);湖南省大学生创新创业训练计划项目(S202010530031)

Liver CT Images Segmentation Based on Multi-scale Feature Fusion and Dual AttentionMechanism

HUANG Yang-lin, HU Kai, GUO Jian-qiang, Peng ChengKey Laboratory of Intelligent Computing & Information Processing of Ministry of Education, Xiangtan University, Xiangtan, Hunan 411105,China   

  1. School of Computer Science & School of Cyberspace Science,Xiangtan University,Xiangtan,Hunan 411105,China
  • Online:2022-11-10 Published:2022-11-21
  • About author:HUANG Yang-lin,born in 2000,undergraduate.His main research interests include deep learning and medical image processing.
    HU Kai,born in 1984,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include machine learning,pattern recognition,bioinformatics,and medical image processing.
  • Supported by:
    National Natural Science Foundation of China(62272404) and Innovation and Entrepreneurship Training Program for Hunan University Students(S202010530031).

摘要: 肝脏疾病是医学上最常见的疾病之一,对其进行精确的分割是辅助肝脏疾病诊断及手术规划的必要步骤。然而,由于肝脏CT图像的复杂性,肝脏分割仍然是一个极具挑战性的问题。以往的研究大多简单地使用拼接或求和操作来融合不同语义,导致无法充分利用其互补性。针对这一问题,提出了一种基于多尺度特征融合和双重注意力机制的网络模型MD-AUNet。首先利用分层多尺度注意力下采样模块中分层级的双重注意力机制有效地融合不同尺度特征信息,提取富含空间信息的特征表示。然后通过全局注意力上采样模块获取高层特征的全局上下文用于对低层特征信息加权,从而选择更为精确的空间信息。同时在网络训练时采用深层监督策略,以学习不同解码层的层次表示。最后提出了一种简洁有效的后处理方法,用于进一步细化MD-AUNet粗分割结果。在医院采集的肝脏数据集(经专家手动标注)上的实验结果表明,所提算法在主观视觉感受和客观评价指标方面均优于其他现有肝脏分割算法,其平均像素精度、平均交并比和Dice相似系数分别为97.6%,95.4%和95.5%。

关键词: CT图像, 肝脏分割, 多尺度特征融合, 双重注意力机制, MD-AUNet

Abstract: Liver disease is one of the most common diseases in medicine,and accurate segmentation of liver disease is a necessary step to assist liver disease diagnosis and surgical planning.However,liver segmentation is still a challenging task due to the complexity of liver CT images.With the deepening of research,people begin to consider combining high-level semantics with low-level semantics to further enhance the segmentation effect.However,most of previous studies simply use splicing or summation operation to fuse different semantics,resulting in failure to make full use of its complementarity.To solve the above problems,a network(MD-AUNet) based on multi-scale feature fusion and dual attention mechanism is proposed in this paper.Firstly,the hierarchical dual attention mechanism in the hierarchical multi-scale attention down-sampling module(HAM) is used to effectively fuse feature information of different scales and extract feature representations rich in spatial information.Then,the global context of high-level features is obtained through the global attention up-sampling module(GAM) for weighting the low-level feature information,so as to select more accurate spatial information.At the same time,deep supervision strategy is used in network training to learn the hierarchical representation of different decoding layers.Moreover,a concise and effective post-processing method is proposed to refine the coarse segmentation result of MD-AUNet.Experimental results on the liver datasets collected by the hospital(manually annotated by experts) demonstrate that the proposed algorithm is superior to other existing liver segmentation algorithms in subjective visual perception and objective evaluation indicators,and its mean pixel accuracy,mean IoU and Dice are 97.6%,95.4%,and 95.5% respectively.

Key words: CT image, Liver segmentation, Multi-scale feature fusion, Dual attention mechanism, MD-AUNet

中图分类号: 

  • TP391
[1]HE L,WU Q.Automatic liver segmentation based on three-dimensional convolutional neural network[J].Chinese Journal of Medical Physics,2018,35(6):581-589.
[2]WANG L D,WANG J,HU C J,et al.Liver Segmentation byUsing an Optimal Framework for CT Images[J].Chinese Journal of Computers,2016,39(7):1477-1489.
[3]CASTRO P,HUERGA C,LA GLARÍA,et al.Target volume segmentation of PET images by an iterative method based on threshold value[J].Rev Esp Med Nucl Imagen Mol,2014,33(6):331-339.
[4]RUSKÓ L,BEKES G,FIDRICH M,Automatic segmentation of the liver from multi- and single-phase contrast-enhanced CT images[J].Medical Image Analysis,2009,13(6):871-882.
[5]WU Y,HE C.Indirectly regularized variational level set model for image segmentation[J].Neurocomputing,2016,171:194-208.
[6]LIM S J,JEONG Y Y,LEE C W,et al.Automatic segmentation of liver in CT images using the watershed algorithm base on morphological filtering[C]//Proceedings of SPIE.2004:1658-1666.
[7]VORONTSOV E,ABI-JAOUDEH N,KADOURY S.Metastatic liver tumor segmentation using texture-based omni-directional deformable surface models[C]//International MICCAI Workshop on Computational and Clinical Challenges in Abdominal Imaging.Springer,2014:74-83.
[8]RACHANTA,SHAJI A.SLIC Superpixels Compared to State-of-the-Art Superpixel Methods [J].IEEE Press,2012(11):2274.
[9]LI Y,HARA S,SHIMURA K.A machine learning approach for locating boundaries of liver tumors in ct images[C]//18th International Conference on Pattern Recognition(ICPR’06).IEEE,2006,1:400-403.
[10]SHELHAMER E,LONG J,DARRELL T.Fully Convolutional Networks for Semantic Segmentation[J].IEEE Transaction on PatternAnalysis and Machine Intelligence,2017,39(4):640-651.
[11]BI L,KIM J,KUMAR A,et al.Automatic Liver Lesion Detection using Cascaded Deep Residual Networks[J].arXiv:abs/1704.02703,2017.
[12]HE K,ZHANG X,REN S,et al.Deep Residual Learning for Image Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2016:770-778.
[13]SUN C,GUO S,ZHANG H,et al.Automatic segmentation of liver tumors from multiphase contrast-enhanced CT images based on FCNs[J].Artificial Intelligence in Medicine,2017,83:58-66.
[14]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional Networks for Biomedical Image Segmentation[J].arXiv:abs/1505.04597,2015.
[15]ZHOU Z,SIDDIQUEE M M R,TAJBAKHSH N,et al.UNet++:Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation[J].IEEE Transactions on Medical Imaging,2020,39(6):1856-1867.
[16]HUANG H l.UNet 3+:A Full-Scale Connected UNet for Me-dical Image Segmentation[C]//2020 IEEE International Confe-rence on Acoustics,Speech and Signal Processing(ICASSP 2020).2020:1055-1059.
[17]YU W,YANG K Y,YAO H X,et al.Exploiting the complementary strengths of multi-layer CNN features for image retrieval[J].Neurocomputing,2017,237:235-241.
[18]WANG X L,LIU F,JIAO L C.Multiscale Bayesian Image Segmentation Fusing Context Information[J].Chinese Journal of Computers,2005,28(3):386-391.
[19]BEN-COHEN A,DIAMANI T,KLANGE D,et al.Fully Convolutional Network for Liver Segmentation and Lesions Detection[C]//International Workshop on Deep Learning in Medical Image Analysis.2016:77-85.
[20]CHRIST P F.Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields[C]//Medical Image Computing and Computer-Assisted Intervention(MICCAI 2016).2016:415-423.
[21]SCHNURR A K,SCHAD L R,ZÖLLNER F G.Sparsely Connected Convolutional Layers in CNNs for Liver Segmentation in CT[C]//Bildverarbeitung für die Medizin 2019.Springer Vieweg,Wiesbaden,2019:80-85.
[22]OKTAY O.Attention U-Net:Learning Where to Look for the Pancreas[J].arXiv abs/1804.03999,2018.
[23]HU J,SHEN L,ALBANIE S,et al.Squeeze-and-Excitation Networks[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence.2020:2011-2023.
[24]ROY A G,NAVAB N.Concurrent Spatial and Channel ‘Sque-eze & Excitation’ [C]//Fully Convolutional Networks.Medical Image Computing and Computer Assisted Intervention(MICCAI 2018).2018:421-429.
[25]WOO S,PARK J.CBAM:Convolutional Block Attention Module[C]//European Conference on Computer Vision.2018:3-19.
[26]FU J,LIU J,TIAN H,et al.Dual Attention Network for Scene Segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:3141-3149.
[27]LI H,XIONG P.Pyramid Attention Network for Semantic Segmentation[J].arXiv,abs/1805.10180,2018.
[28]SINHA A,DOLZ J.Multi-Scale Self-Guided Attention for Medical Image Segmentation[C]//IEEE Journal of Biomedical and Health Informatics.2021:121-130.
[29]TAO A,SAPRA K,CATANZARO B.Hierarchical Multi-Scale Attention for Semantic Segmentation[J].arXiv:abs/2005.10821,2020.
[30]DUAN L J,SUN Q C,QIAO Y H,et al.Attention-Aware and Semantic-Aware Network for RGB-D Indoor Semantic Segmentation[J].Chinese Journal of Computers,2021,44(2):275-291.
[31]XIONG L H.Pyramid Attention Network for Semantic Segmentation[J].arXiv:abs/1805.10180,2018.
[32]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[J].IEEE Trans PatternAnalMach Intell,2020,42(2):318-327.
[33]SHU H H,XIA H X,GA L H,et al.Water Body Extraction of High Resolution Remote Sensing Image based on Improved U-Net Network[J].Journal of Geo-information Science,2020,22(10):2010-2022.
[34]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Semantic im-age segmentation with deep convolutional nets and fully connected crfs[J].arXiv:1412.7062,2014:97-105.
[35]CHEN J X,JIA X M.Real-time Color Object Recognition Method Based on Flood Fill Algorithm[J].Chinese Journal of Computer Simulation,2012,29(3):4-9.
[36]DIEDERIK P K,JIMMY B,YOSHUA B,et al.A Method for Stochastic Optimization[C]//ICLR(Poster).2015.
[37]SZEGEDY C.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2015:1-9.
[38]TAN M X,LE Q V.EfficientNet:Rethinking Model Scaling for Convolutional Neural Networks[J].arXiv:abs/1905.11946,2019.
[1] 范新南, 赵忠鑫, 严炜, 严锡君, 史朋飞.
结合注意力机制的多尺度特征融合图像去雾算法
Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism
计算机科学, 2022, 49(5): 50-57. https://doi.org/10.11896/jsjkx.210400093
[2] 张瑛, 聂仁灿, 马朝振, 余仕双.
基于子空间特征相互学习的MRI与PET/SPECT图像融合
MRI and PET/SPECT Image Fusion Based on Subspace Feature Mutual Learning
计算机科学, 2022, 49(11A): 211000171-6. https://doi.org/10.11896/jsjkx.211000171
[3] 孙荣荣, 单飞, 叶雯.
基于DenseNet和混合域注意力的COVID-19低剂量CT图像质量评价
Image Quality Assessment for Low-dose-CT Images of COVID-19 Based on DenseNet and Mixed Domain Attention
计算机科学, 2021, 48(11A): 620-624. https://doi.org/10.11896/jsjkx.201200252
[4] 张天柱, 邹承明.
使用模糊聚类的胶囊网络在图像分类上的研究
Study on Image Classification of Capsule Network Using Fuzzy Clustering
计算机科学, 2019, 46(12): 279-285. https://doi.org/10.11896/jsjkx.190200315
[5] 骆岩红.
CT图像重建滤波反投影算法中指数滤波器的研究
Study on Exponential Filter of Filter Back Projection Algorithm for CT Image Reconstruction
计算机科学, 2014, 41(Z6): 220-223.
[6] 陈刚,吕煊,王志成,陈宇飞.
肺CT图像的血管骨架化方法
Vessel Skeletonization Method for Lung CT Images
计算机科学, 2013, 40(5): 274-278.
[7] 刘技,康晓东,贾富仓,刘玲玲,王昊.
基于图割与概率图谱的肝脏自动分割研究
Automatic Liver Segmentation Using Graph Cuts and Probabilistic Atlas
计算机科学, 2012, 39(2): 288-290.
[8] .
面向RP的工业CT切片数据格式转换软件开发

计算机科学, 2008, 35(1): 263-265.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!