基于CT图像语义的COVID-19实例分割与分类网络

doi:10.11896/jsjkx.220600142

摘要/Abstract

摘要： 为了辅助临床医生进行COVID-19患者的诊断及治疗,提出了一个从患者肺部CT图像中分类、检测和分割COVID-19病变的辅助诊断网络AIS-Net。首先,该网络将语义分割与实例分割融合,提升了实例分割精度,提出了信息增强注意力模块(IEAM),用于提升输入特征关键信息的权重。为了提高网络对假阴性的关注度,提出了一个实例分割监督方法,用于不同尺度的病变进行监控。其次,设计了一个包含主分类头与辅助分类头的模块,对新冠肺炎、普通肺炎和非肺炎进行分类。在辅助分类中引入了Swin Transformer,提出了区分普通肺炎与新冠肺炎病变的方法。在CC-CCII分割数据集上实例分割的平均精度均值(mAP)为56.53%,比目前最好的方法提升了11.77%;Dice系数、灵敏度、特异度分别为80%,85.1%,99.3%,比目前最好的方法分别提升了4.7%,3.7%,1.2%。在COVIDX-CT分类数据集上实现了99.07%的总体准确度,比目前最好的方法提升了0.92%。AIS-Net可通过CT图像对COVID-19患者进行有效诊断,并对病变部位进行分割及检测。

关键词: COVID-19分类, 实例分割, 语义分割, Swin Transformer, CT图像

Abstract: To assist clinicians in the diagnosis and treatment of COVID-19 patients,a computer-aided diagnosis network AIS-Net is proposed to classify,detect and segment COVID-19 lesions in CT images.First,the network integrates semantic and instance segmentation to improve the accuracy.Then,the two modules are designed,the information enhanced attention module(IEAM) for weighing input features and the instance segmentation monitoring module focusing on the lesions at different scales.Furthermore,the classification module with the main header and the auxiliary header discerns COVID-19 pneumonia,common pneumonia,and non-pneumonia.Finally,the Swin Transformer is introduced into the auxiliary classification to distinguish the lesions of common pneumonia and COVID-19.On the CC-CCII dataset,the mean average precision(mAP) of instance segmentation is 56.53%,which is 11.77% higher than the state-of-the art(SOTA).Dice coefficient,sensitivity and specificity is 80%,85.1% and 99.3% respectively,which is 4.7%,3.7% and 1.2% higher than the SOTA.The overall classification accuracy is 99.07% on the COVIDX-CT dataset,0.92% higher than the SOTA.AIS-Net can effectively diagnose COVID-19 patients through CT images,and segment and detect the lesion sites.

Key words: COVID-19 classification, Instance segmentation, Semantic segmentation, Swin Transformer, CT images

中图分类号:

TP391

柏正尧, 樊圣澜, 陆倩杰, 周雪. 基于CT图像语义的COVID-19实例分割与分类网络[J]. 计算机科学, 2023, 50(6A): 220600142-9. https://doi.org/10.11896/jsjkx.220600142

BAI Zhengyao, FAN Shenglan, LU Qianjie, ZHOU Xue. COVID-19 Instance Segmentation and Classification Network Based on CT Image Semantics[J]. Computer Science, 2023, 50(6A): 220600142-9. https://doi.org/10.11896/jsjkx.220600142

参考文献

[1]LU R,ZHAO X,LI J,et al.Genomic characterisation and epidemiology of 2019 novel coronavirus:implications for virus origins and receptor binding[J].The lancet,2020,395(10224):565-574.
[2]WANG W,XU Y,GAO R,et al.Detection of SARS-CoV-2 in different types of clinical specimens[J].Jama,2020,323(18):1843-1844.
[3]HUANG Z,ZHAO S,LI Z,et al.The battle against coronavirusdisease 2019(COVID-19):emergency management and infection control in a radiology department[J].Journal of the American College of Radiology,2020,17(6):710-716.
[4]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[5]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2015:1-9.
[6]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:770-778.
[7]HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely con-nected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2017:4700-4708.
[8]SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems.2017:3859-3869.
[9]TAN M,LE Q.Efficientnet:Rethinking model scaling for con-volutional neural networks[C]//International Conference on Machine Learning.PMLR,2019:6105-6114.
[10]DAS A K,KALAM S,KUMAR C,et al.TLCoV-An automated Covid-19 screening model using Transfer Learning from chest X-ray images[J].Chaos,Solitons & Fractals,2021,144:110713.
[11]WANG S H,FERNANDES S,ZHU Z,et al.AVNC:attention-based VGG-style network for COVID-19 diagnosis by CBAM[J].IEEE Sensors Journal,2021,22(18):17431-17438.
[12]FU J,ZHENG H,MEI T.Look closer to see better:Recurrent attention convolutional neural network for fine-grained image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4438-4446.
[13]WANG S H,GOVINDARAJ V V,GÓRRIZ J M,et al.Covid-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network[J].Information Fusion,2021,67:208-229.
[14]ELHARROUSS O,SUBRAMANIAN N,AL-MAADEED S.An encoder-decoder-based method for segmentation of COVID-19 lung infection in CT images[J].SN Computer Science,2022,3(1):1-12.
[15]XIE S,GIRSHICK R,DOLLÁR P,et al.Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1492-1500.
[16]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Deeplab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(4):834-848.
[17]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2117-2125.
[18]HE K,GKIOXARI G,DOLLÁR P,et al.Mask r-cnn[C]//Proceedings of the IEEE international conference on computer vision.2017:2961-2969.
[19]LIU Z,LIN Y,CAO Y,et al.Swin transformer:Hierarchical vi-sion transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:10012-10022.
[20]PANG J,CHEN K,SHI J,et al.Libra r-cnn:Towards balanced learning for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:821-830.
[21]GHIASI G,LIN T Y,LE Q V.Nas-fpn:Learning scalable feature pyramid architecture for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:7036-7045.
[22]XIANG W,MAO H,ATHITSOS V.ThunderNet:A turbo unified network for real-time semantic segmentation[C]//2019 IEEE winter Conference on Applications of Computer Vision(WACV).IEEE,2019:1789-1796.
[23]CHEN K,PANG J,WANG J,et al.Hybrid task cascade for instance segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:4974-4983.
[24]ZHANG G,LU X,TAN J,et al.Refinemask:Towards high-quality instance segmentation with fine-grained features[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:6861-6869.
[25]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[26]GUNRAJ H,WANG L,WONG A.COVIDNet-CT:A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases From Chest CT Images[J].Front Med(Lausanne),2020,7:608525-608525.
[27]DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.AnImage is Worth 16x16 Words:Transformers for Image Recognition at Scale[C]//International Conference on Learning Representations.2020.
[28]TER-SARKISOV A.Detection and Segmentation of Lesion Areas in Chest CT Scans For The Prediction of COVID-19[J].Science in Information and Technology Letters,2021,1(2):92-99.
[29]TER-SARKISOV A.Lightweight Model for the Prediction ofCOVID-19 Through the Detection and Segmentation of Lesions in Chest CT Scans[J].International Journal of Automation,Artificial Intelligence and Machine Learning,2021,2(1):1-15.
[30]FAN D P,JI G P,ZHOU T,et al.Pranet:Parallel reverse attention network for polyp segmentation[C]//International Confe-rence on Medical Image Computing and Computer-assisted Intervention.Cham:Springer,2020:263-273.
[31]MEI H,JI G P,WEI Z,et al.Camouflaged object segmentation with distraction mining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8772-8781.
[32]HUANG C H,WU H Y,LIN Y L.Hardnet-mseg:a simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps[J].arXiv:2101.07172,2021.
[33]WEI J,WANG S,HUANG Q.F³Net:fusion,feedback and focus for salient object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:12321-12328.
[34]TER-SARKISOV A.Covid-ct-mask-net:Prediction of covid-19 from ct scans using regional features[J].Applied Intelligence,2022,52(9):9664-9675.
[35]TER-SARKISOV A.One Shot Model for the Prediction ofCOVID-19 And Lesions Segmentation In Chest CT Scans Through The Affinity Among Lesion Mask Features[J].Applied Soft Computing,2021,116:108261-108261.
[36]TER-SARKISOV A.One Shot Model For COVID-19 Classification and Lesions Segmentation In Chest CT Scans Using LSTM With Attention Mechanism[J].IEEE Intelligent Systems,2022,37(3):54-64.
[37]FAN D P,ZHOU T,JI G P,et al.Inf-net:Automatic covid-19 lung infection segmentation from ct images[J].IEEE Transactions on Medical Imaging,2020,39(8):2626-2637.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed