Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 220600142-9.doi: 10.11896/jsjkx.220600142

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

COVID-19 Instance Segmentation and Classification Network Based on CT Image Semantics

BAI Zhengyao, FAN Shenglan, LU Qianjie, ZHOU Xue   

  1. School of Information Science and Engineering,Yunnan University,Kunming 650500,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:BAI Zhengyao,born in 1967,Ph.D,professor,master supervisor.His main research interests include signal proces-sing,image processing,pattern re-cognition and machine learning.
  • Supported by:
    Major Science and Technology Project of Yunnan Province(202002AD080001).

PDF (PC)

270

Abstract: To assist clinicians in the diagnosis and treatment of COVID-19 patients,a computer-aided diagnosis network AIS-Net is proposed to classify,detect and segment COVID-19 lesions in CT images.First,the network integrates semantic and instance segmentation to improve the accuracy.Then,the two modules are designed,the information enhanced attention module(IEAM) for weighing input features and the instance segmentation monitoring module focusing on the lesions at different scales.Furthermore,the classification module with the main header and the auxiliary header discerns COVID-19 pneumonia,common pneumonia,and non-pneumonia.Finally,the Swin Transformer is introduced into the auxiliary classification to distinguish the lesions of common pneumonia and COVID-19.On the CC-CCII dataset,the mean average precision(mAP) of instance segmentation is 56.53%,which is 11.77% higher than the state-of-the art(SOTA).Dice coefficient,sensitivity and specificity is 80%,85.1% and 99.3% respectively,which is 4.7%,3.7% and 1.2% higher than the SOTA.The overall classification accuracy is 99.07% on the COVIDX-CT dataset,0.92% higher than the SOTA.AIS-Net can effectively diagnose COVID-19 patients through CT images,and segment and detect the lesion sites.

Key words: COVID-19 classification, Instance segmentation, Semantic segmentation, Swin Transformer, CT images

CLC Number: 

  • TP391
[1]LU R,ZHAO X,LI J,et al.Genomic characterisation and epidemiology of 2019 novel coronavirus:implications for virus origins and receptor binding[J].The lancet,2020,395(10224):565-574.
[2]WANG W,XU Y,GAO R,et al.Detection of SARS-CoV-2 in different types of clinical specimens[J].Jama,2020,323(18):1843-1844.
[3]HUANG Z,ZHAO S,LI Z,et al.The battle against coronavirusdisease 2019(COVID-19):emergency management and infection control in a radiology department[J].Journal of the American College of Radiology,2020,17(6):710-716.
[4]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[5]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2015:1-9.
[6]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:770-778.
[7]HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely con-nected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2017:4700-4708.
[8]SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems.2017:3859-3869.
[9]TAN M,LE Q.Efficientnet:Rethinking model scaling for con-volutional neural networks[C]//International Conference on Machine Learning.PMLR,2019:6105-6114.
[10]DAS A K,KALAM S,KUMAR C,et al.TLCoV-An automated Covid-19 screening model using Transfer Learning from chest X-ray images[J].Chaos,Solitons & Fractals,2021,144:110713.
[11]WANG S H,FERNANDES S,ZHU Z,et al.AVNC:attention-based VGG-style network for COVID-19 diagnosis by CBAM[J].IEEE Sensors Journal,2021,22(18):17431-17438.
[12]FU J,ZHENG H,MEI T.Look closer to see better:Recurrent attention convolutional neural network for fine-grained image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4438-4446.
[13]WANG S H,GOVINDARAJ V V,GÓRRIZ J M,et al.Covid-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network[J].Information Fusion,2021,67:208-229.
[14]ELHARROUSS O,SUBRAMANIAN N,AL-MAADEED S.An encoder-decoder-based method for segmentation of COVID-19 lung infection in CT images[J].SN Computer Science,2022,3(1):1-12.
[15]XIE S,GIRSHICK R,DOLLÁR P,et al.Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1492-1500.
[16]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Deeplab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(4):834-848.
[17]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2117-2125.
[18]HE K,GKIOXARI G,DOLLÁR P,et al.Mask r-cnn[C]//Proceedings of the IEEE international conference on computer vision.2017:2961-2969.
[19]LIU Z,LIN Y,CAO Y,et al.Swin transformer:Hierarchical vi-sion transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:10012-10022.
[20]PANG J,CHEN K,SHI J,et al.Libra r-cnn:Towards balanced learning for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:821-830.
[21]GHIASI G,LIN T Y,LE Q V.Nas-fpn:Learning scalable feature pyramid architecture for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:7036-7045.
[22]XIANG W,MAO H,ATHITSOS V.ThunderNet:A turbo unified network for real-time semantic segmentation[C]//2019 IEEE winter Conference on Applications of Computer Vision(WACV).IEEE,2019:1789-1796.
[23]CHEN K,PANG J,WANG J,et al.Hybrid task cascade for instance segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:4974-4983.
[24]ZHANG G,LU X,TAN J,et al.Refinemask:Towards high-quality instance segmentation with fine-grained features[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:6861-6869.
[25]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[26]GUNRAJ H,WANG L,WONG A.COVIDNet-CT:A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases From Chest CT Images[J].Front Med(Lausanne),2020,7:608525-608525.
[27]DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.AnImage is Worth 16x16 Words:Transformers for Image Recognition at Scale[C]//International Conference on Learning Representations.2020.
[28]TER-SARKISOV A.Detection and Segmentation of Lesion Areas in Chest CT Scans For The Prediction of COVID-19[J].Science in Information and Technology Letters,2021,1(2):92-99.
[29]TER-SARKISOV A.Lightweight Model for the Prediction ofCOVID-19 Through the Detection and Segmentation of Lesions in Chest CT Scans[J].International Journal of Automation,Artificial Intelligence and Machine Learning,2021,2(1):1-15.
[30]FAN D P,JI G P,ZHOU T,et al.Pranet:Parallel reverse attention network for polyp segmentation[C]//International Confe-rence on Medical Image Computing and Computer-assisted Intervention.Cham:Springer,2020:263-273.
[31]MEI H,JI G P,WEI Z,et al.Camouflaged object segmentation with distraction mining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8772-8781.
[32]HUANG C H,WU H Y,LIN Y L.Hardnet-mseg:a simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps[J].arXiv:2101.07172,2021.
[33]WEI J,WANG S,HUANG Q.F3Net:fusion,feedback and focus for salient object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:12321-12328.
[34]TER-SARKISOV A.Covid-ct-mask-net:Prediction of covid-19 from ct scans using regional features[J].Applied Intelligence,2022,52(9):9664-9675.
[35]TER-SARKISOV A.One Shot Model for the Prediction ofCOVID-19 And Lesions Segmentation In Chest CT Scans Through The Affinity Among Lesion Mask Features[J].Applied Soft Computing,2021,116:108261-108261.
[36]TER-SARKISOV A.One Shot Model For COVID-19 Classification and Lesions Segmentation In Chest CT Scans Using LSTM With Attention Mechanism[J].IEEE Intelligent Systems,2022,37(3):54-64.
[37]FAN D P,ZHOU T,JI G P,et al.Inf-net:Automatic covid-19 lung infection segmentation from ct images[J].IEEE Transactions on Medical Imaging,2020,39(8):2626-2637.
[1] CHEN Qiaosong, ZHANG Yu, PU Liu, TAN Chongchong, DENG Xin, WANG Jin, SUN Kaiwei, OUYANG Weihua. Multi-path Semantic Segmentation Based on Edge Optimization and Global Modeling [J]. Computer Science, 2023, 50(6A): 220700137-7.
[2] LUO Huilan, YE Ju. Study of Multi-task Learning with Joint Semantic Segmentation and Depth Estimation [J]. Computer Science, 2023, 50(6A): 220100111-10.
[3] SUN Kaiwei, LIU Hu, RAN Xue, GUO Hao. Few-shot Segmentation Based on Multi-scale Prototype Hierarchical Matching [J]. Computer Science, 2023, 50(6A): 220300275-7.
[4] GU Yuhang, HAO Jie, CHEN Bing. Semi-supervised Semantic Segmentation for High-resolution Remote Sensing Images Based on DataFusion [J]. Computer Science, 2023, 50(6A): 220500001-6.
[5] LI Yang, HAN Ping. Human Parsing Model Combined with Regional Sampling and Inter-class Loss [J]. Computer Science, 2023, 50(4): 103-109.
[6] MA Weiqi, YUAN Jiabin, ZHA Keke, FAN Lili. Onboard Rock Detection Algorithm Based on Spiking Neural Network [J]. Computer Science, 2023, 50(1): 98-104.
[7] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[8] HU Fu-yuan, WAN Xin-jun, SHEN Ming-fei, XU Jiang-lang, YAO Rui, TAO Zhong-ben. Survey Progress on Image Instance Segmentation Methods of Deep Convolutional Neural Network [J]. Computer Science, 2022, 49(5): 10-24.
[9] JIN Yu-jie, CHU Xu, WANG Ya-sha, ZHAO Jun-feng. Variational Domain Adaptation Driven Semantic Segmentation of Urban Scenes [J]. Computer Science, 2022, 49(11): 126-133.
[10] WANG Shi-yun, YANG Fan. Remote Sensing Image Semantic Segmentation Method Based on U-Net Feature Fusion Optimization Strategy [J]. Computer Science, 2021, 48(8): 162-168.
[11] ZHAN Rui, LEI Yin-jie, CHEN Xun-min, YE Shu-han. Street Scene Change Detection Based on Multiple Difference Features Network [J]. Computer Science, 2021, 48(2): 142-147.
[12] WANG Xin, ZHANG Hao-yu, LING Cheng. Semantic Segmentation of SAR Remote Sensing Image Based on U-Net Optimization [J]. Computer Science, 2021, 48(11A): 376-381.
[13] ZHU Rong, YE Kuan, YANG Bo, XIE Huan, ZHAO Lei. Feature Classification Method Based on Improved DeeplabV3+ [J]. Computer Science, 2021, 48(11A): 382-385.
[14] SUN Rong-rong, SHAN Fei, YE Wen. Image Quality Assessment for Low-dose-CT Images of COVID-19 Based on DenseNet and Mixed Domain Attention [J]. Computer Science, 2021, 48(11A): 620-624.
[15] REN Tian-ci, HUANG Xiang-sheng, DING Wei-li, AN Chong-yang and ZHAI Peng-bo. Global Bilateral Segmentation Network for Segmantic Segmentation [J]. Computer Science, 2020, 47(6A): 161-165.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.