Computer Science ›› 2022, Vol. 49 ›› Issue (11A): 210800162-9.doi: 10.11896/jsjkx.210800162

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Liver CT Images Segmentation Based on Multi-scale Feature Fusion and Dual AttentionMechanism

HUANG Yang-lin, HU Kai, GUO Jian-qiang, Peng ChengKey Laboratory of Intelligent Computing & Information Processing of Ministry of Education, Xiangtan University, Xiangtan, Hunan 411105,China   

  1. School of Computer Science & School of Cyberspace Science,Xiangtan University,Xiangtan,Hunan 411105,China
  • Online:2022-11-10 Published:2022-11-21
  • About author:HUANG Yang-lin,born in 2000,undergraduate.His main research interests include deep learning and medical image processing.
    HU Kai,born in 1984,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include machine learning,pattern recognition,bioinformatics,and medical image processing.
  • Supported by:
    National Natural Science Foundation of China(62272404) and Innovation and Entrepreneurship Training Program for Hunan University Students(S202010530031).

Abstract: Liver disease is one of the most common diseases in medicine,and accurate segmentation of liver disease is a necessary step to assist liver disease diagnosis and surgical planning.However,liver segmentation is still a challenging task due to the complexity of liver CT images.With the deepening of research,people begin to consider combining high-level semantics with low-level semantics to further enhance the segmentation effect.However,most of previous studies simply use splicing or summation operation to fuse different semantics,resulting in failure to make full use of its complementarity.To solve the above problems,a network(MD-AUNet) based on multi-scale feature fusion and dual attention mechanism is proposed in this paper.Firstly,the hierarchical dual attention mechanism in the hierarchical multi-scale attention down-sampling module(HAM) is used to effectively fuse feature information of different scales and extract feature representations rich in spatial information.Then,the global context of high-level features is obtained through the global attention up-sampling module(GAM) for weighting the low-level feature information,so as to select more accurate spatial information.At the same time,deep supervision strategy is used in network training to learn the hierarchical representation of different decoding layers.Moreover,a concise and effective post-processing method is proposed to refine the coarse segmentation result of MD-AUNet.Experimental results on the liver datasets collected by the hospital(manually annotated by experts) demonstrate that the proposed algorithm is superior to other existing liver segmentation algorithms in subjective visual perception and objective evaluation indicators,and its mean pixel accuracy,mean IoU and Dice are 97.6%,95.4%,and 95.5% respectively.

Key words: CT image, Liver segmentation, Multi-scale feature fusion, Dual attention mechanism, MD-AUNet

CLC Number: 

  • TP391
[1]HE L,WU Q.Automatic liver segmentation based on three-dimensional convolutional neural network[J].Chinese Journal of Medical Physics,2018,35(6):581-589.
[2]WANG L D,WANG J,HU C J,et al.Liver Segmentation byUsing an Optimal Framework for CT Images[J].Chinese Journal of Computers,2016,39(7):1477-1489.
[3]CASTRO P,HUERGA C,LA GLARÍA,et al.Target volume segmentation of PET images by an iterative method based on threshold value[J].Rev Esp Med Nucl Imagen Mol,2014,33(6):331-339.
[4]RUSKÓ L,BEKES G,FIDRICH M,Automatic segmentation of the liver from multi- and single-phase contrast-enhanced CT images[J].Medical Image Analysis,2009,13(6):871-882.
[5]WU Y,HE C.Indirectly regularized variational level set model for image segmentation[J].Neurocomputing,2016,171:194-208.
[6]LIM S J,JEONG Y Y,LEE C W,et al.Automatic segmentation of liver in CT images using the watershed algorithm base on morphological filtering[C]//Proceedings of SPIE.2004:1658-1666.
[7]VORONTSOV E,ABI-JAOUDEH N,KADOURY S.Metastatic liver tumor segmentation using texture-based omni-directional deformable surface models[C]//International MICCAI Workshop on Computational and Clinical Challenges in Abdominal Imaging.Springer,2014:74-83.
[8]RACHANTA,SHAJI A.SLIC Superpixels Compared to State-of-the-Art Superpixel Methods [J].IEEE Press,2012(11):2274.
[9]LI Y,HARA S,SHIMURA K.A machine learning approach for locating boundaries of liver tumors in ct images[C]//18th International Conference on Pattern Recognition(ICPR’06).IEEE,2006,1:400-403.
[10]SHELHAMER E,LONG J,DARRELL T.Fully Convolutional Networks for Semantic Segmentation[J].IEEE Transaction on PatternAnalysis and Machine Intelligence,2017,39(4):640-651.
[11]BI L,KIM J,KUMAR A,et al.Automatic Liver Lesion Detection using Cascaded Deep Residual Networks[J].arXiv:abs/1704.02703,2017.
[12]HE K,ZHANG X,REN S,et al.Deep Residual Learning for Image Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2016:770-778.
[13]SUN C,GUO S,ZHANG H,et al.Automatic segmentation of liver tumors from multiphase contrast-enhanced CT images based on FCNs[J].Artificial Intelligence in Medicine,2017,83:58-66.
[14]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional Networks for Biomedical Image Segmentation[J].arXiv:abs/1505.04597,2015.
[15]ZHOU Z,SIDDIQUEE M M R,TAJBAKHSH N,et al.UNet++:Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation[J].IEEE Transactions on Medical Imaging,2020,39(6):1856-1867.
[16]HUANG H l.UNet 3+:A Full-Scale Connected UNet for Me-dical Image Segmentation[C]//2020 IEEE International Confe-rence on Acoustics,Speech and Signal Processing(ICASSP 2020).2020:1055-1059.
[17]YU W,YANG K Y,YAO H X,et al.Exploiting the complementary strengths of multi-layer CNN features for image retrieval[J].Neurocomputing,2017,237:235-241.
[18]WANG X L,LIU F,JIAO L C.Multiscale Bayesian Image Segmentation Fusing Context Information[J].Chinese Journal of Computers,2005,28(3):386-391.
[19]BEN-COHEN A,DIAMANI T,KLANGE D,et al.Fully Convolutional Network for Liver Segmentation and Lesions Detection[C]//International Workshop on Deep Learning in Medical Image Analysis.2016:77-85.
[20]CHRIST P F.Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields[C]//Medical Image Computing and Computer-Assisted Intervention(MICCAI 2016).2016:415-423.
[21]SCHNURR A K,SCHAD L R,ZÖLLNER F G.Sparsely Connected Convolutional Layers in CNNs for Liver Segmentation in CT[C]//Bildverarbeitung für die Medizin 2019.Springer Vieweg,Wiesbaden,2019:80-85.
[22]OKTAY O.Attention U-Net:Learning Where to Look for the Pancreas[J].arXiv abs/1804.03999,2018.
[23]HU J,SHEN L,ALBANIE S,et al.Squeeze-and-Excitation Networks[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence.2020:2011-2023.
[24]ROY A G,NAVAB N.Concurrent Spatial and Channel ‘Sque-eze & Excitation’ [C]//Fully Convolutional Networks.Medical Image Computing and Computer Assisted Intervention(MICCAI 2018).2018:421-429.
[25]WOO S,PARK J.CBAM:Convolutional Block Attention Module[C]//European Conference on Computer Vision.2018:3-19.
[26]FU J,LIU J,TIAN H,et al.Dual Attention Network for Scene Segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:3141-3149.
[27]LI H,XIONG P.Pyramid Attention Network for Semantic Segmentation[J].arXiv,abs/1805.10180,2018.
[28]SINHA A,DOLZ J.Multi-Scale Self-Guided Attention for Medical Image Segmentation[C]//IEEE Journal of Biomedical and Health Informatics.2021:121-130.
[29]TAO A,SAPRA K,CATANZARO B.Hierarchical Multi-Scale Attention for Semantic Segmentation[J].arXiv:abs/2005.10821,2020.
[30]DUAN L J,SUN Q C,QIAO Y H,et al.Attention-Aware and Semantic-Aware Network for RGB-D Indoor Semantic Segmentation[J].Chinese Journal of Computers,2021,44(2):275-291.
[31]XIONG L H.Pyramid Attention Network for Semantic Segmentation[J].arXiv:abs/1805.10180,2018.
[32]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[J].IEEE Trans PatternAnalMach Intell,2020,42(2):318-327.
[33]SHU H H,XIA H X,GA L H,et al.Water Body Extraction of High Resolution Remote Sensing Image based on Improved U-Net Network[J].Journal of Geo-information Science,2020,22(10):2010-2022.
[34]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Semantic im-age segmentation with deep convolutional nets and fully connected crfs[J].arXiv:1412.7062,2014:97-105.
[35]CHEN J X,JIA X M.Real-time Color Object Recognition Method Based on Flood Fill Algorithm[J].Chinese Journal of Computer Simulation,2012,29(3):4-9.
[36]DIEDERIK P K,JIMMY B,YOSHUA B,et al.A Method for Stochastic Optimization[C]//ICLR(Poster).2015.
[37]SZEGEDY C.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2015:1-9.
[38]TAN M X,LE Q V.EfficientNet:Rethinking Model Scaling for Convolutional Neural Networks[J].arXiv:abs/1905.11946,2019.
[1] FAN Xin-nan, ZHAO Zhong-xin, YAN Wei, YAN Xi-jun, SHI Peng-fei. Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism [J]. Computer Science, 2022, 49(5): 50-57.
[2] ZHANG Ying, NIE Ren-can, MA Chao-zhen, YU Shi-shuang. MRI and PET/SPECT Image Fusion Based on Subspace Feature Mutual Learning [J]. Computer Science, 2022, 49(11A): 211000171-6.
[3] SUN Rong-rong, SHAN Fei, YE Wen. Image Quality Assessment for Low-dose-CT Images of COVID-19 Based on DenseNet and Mixed Domain Attention [J]. Computer Science, 2021, 48(11A): 620-624.
[4] CAO Yi-qin, WU Dan, HUANG Xiao-sheng. Track Defect Image Classification Based on Improved Ant Colony Algorithm [J]. Computer Science, 2019, 46(8): 292-297.
[5] MAO Ying-chi,WANG Jing,CHEN Xiao-li,XU Shu-fang,CHEN Hao. Dam Defect Recognition and Classification Based on Feature Combination and CNN [J]. Computer Science, 2019, 46(3): 267-276.
[6] ZHANG Tian-zhu, ZOU Cheng-ming. Study on Image Classification of Capsule Network Using Fuzzy Clustering [J]. Computer Science, 2019, 46(12): 279-285.
[7] ZHANG Hong-bin, JI Dong-hong, YIN Lan and REN Ya-feng. Product Image Sentence Annotation Based on Gradient Kernel Feature and N-gram Model [J]. Computer Science, 2016, 43(5): 269-273.
[8] LUO Yan-hong. Study on Exponential Filter of Filter Back Projection Algorithm for CT Image Reconstruction [J]. Computer Science, 2014, 41(Z6): 220-223.
[9] LAN Hong and WANG Xuan. Insect Image Segmentation Algorithm Based on Multiple Linear Regression and Transition Region [J]. Computer Science, 2014, 41(5): 315-319.
[10] CHEN Gang,LV Xuan,WANG Zhi-cheng and CHEN Yu-fei. Vessel Skeletonization Method for Lung CT Images [J]. Computer Science, 2013, 40(5): 274-278.
[11] . Automatic Liver Segmentation Using Graph Cuts and Probabilistic Atlas [J]. Computer Science, 2012, 39(2): 288-290.
[12] LIU Lu,CHU Chun-yu,MA Jian-wei,LIU Wan-yu. CT Image Segmentation Based on Support Vector Machine and Regional Growth [J]. Computer Science, 2010, 37(5): 237-239.
[13] . [J]. Computer Science, 2008, 35(1): 263-265.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!