基于深度学习的病理切片质量控制算法综述

doi:10.11896/jsjkx.231000167

摘要/Abstract

摘要： 病理切片是病理医生对肿瘤患者的病程及预后进行诊断和分析的重要依据。然而,由于切片的制备过程自动化程度较低,人为操作的失误和设备噪声均会降低切片的质量,进而影响诊断。当前病理切片的质量控制主要采用人工抽样检查的方式,该方式工作强度大,工作时间长,易因视觉疲劳造成评估偏差。借助深度学习技术对病理切片进行质量控制受到了医学界和工业界的关注,并取得了一定的进展。文中对该领域的研究现状进行了回顾:首先简要介绍了病理切片的制作和数字化流程,分析了质量控制工作中存在的困难和挑战;然后对现有的关于病理切片的质量控制相关工作进行了分析和总结,从染色标准化、聚焦质量评估、伪影检测、图像的修复与重建等方面,对现有工作的方法理论和应用现状进行了概述;最后对该领域未来可能的研究热点进行了展望。

关键词: 病理切片, 数字病理图像, 质量控制, 人工智能, 深度学习

Abstract: Pathological sections are an important basis for pathologists to diagnose and analyze the course and prognosis of tumor patients.However,due to the low degree of automation in the preparation process of slides,both human operation and equipment noise will reduce the quality of the slides,thereby affecting diagnosis.Currently,the quality control of pathological slides mainly uses manual sampling inspection,which has the characteristics of high work intensity and long working hours,and can easily lead to evaluation bias due to visual fatigue.Quality control of pathological slides using deep learning technology attracts attention from the medical and engineering communities and makes certain progress.This paper reviews the research status in this field.First,the production and digitization process of pathological slides is briefly introduced,and the difficulties and challenges in qua-lity control work are analyzed.Then,the existing work related to the quality control of pathological slides is analyzed and summarized,and the method theory and application status of the existing work are reviewed from aspects such as staining standardization,focus quality assessment,artifact detection,image repair and reconstruction.Finally,the possible future research hotspots in this field are prospected.

Key words: Pathological slide, Digital pathological images, Quality control, Artificial intelligence, Deep learning

中图分类号:

TP391

丁维龙, 刘津龙, 朱伟, 廖婉茵. 基于深度学习的病理切片质量控制算法综述[J]. 计算机科学, 2024, 51(10): 276-286. https://doi.org/10.11896/jsjkx.231000167

DING Weilong, LIU Jinlong, ZHU Wei, LIAO Wanyin. Review of Quality Control Algorithms for Pathological Slides Based on Deep Learning[J]. Computer Science, 2024, 51(10): 276-286. https://doi.org/10.11896/jsjkx.231000167

参考文献

[1]PAN X,LIN X,CAO D,et al.Deep learning for drug repurpo-sing:Methods,databases,and applications[J].Wiley Online Library,2022,12(4):1597-1618.
[2]WEI D H,STEVEN S,LOUIS R,et al.From telemedicine to internet＋artificial intelligence(AI) healthcare to see the evolution of healthcare digitization[J].Chinese Research Hospital,2022,9(5):64-76.
[3]SUNG H,FERLAY J,SIEGEL R,et al.Global cancer statistics 2020:globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J].CA:A Cancer Journal for Clinicians,2021,71(3):209-249.
[4]KOMURA D,ISHIKAWA S.Machine learning methods for his-topathological image analysis[J].Computational and Structural Biotechnology Journal,2018,16(1):34-42.
[5]JIN X,WEN K,LYU G F,et al.Survey on the applications of deep learning in histopathology[J].Journal of Image and Gra-phics,2020,25(10):1982-1993.
[6]SONG J,XIAO L,LIAN Z C,et al.Deep learning algorithms for image segmentation in digital pathology:overview and prospect[J].Journal of Software,2021,32(5):1427-1460.
[7]QU L H,LIU S Y,LIU X Y,et al.Towards label-efficient automatic diagnosis and analysis:a comprehensive survey of advanced deep learning-based weakly-supervised,semi-supervised and self-supervised techniques in histopathological image analysis[J].Physics in Medicine & Biology,2022,67(20):1-34.
[8]ZHU X L,LUO X L,WU H M,et al.Quality control for pathological specimen processing[J].Journal of Clinical and Patholo-gical Research,2019,35(10):1249-1250.
[9]KANWAL N,PÉREZ-BUENO F,SCHMIDT A,et al.The devil is in the details:whole slide image acquisition and processing for artifacts detection,color variation,and data augmentation:a review[J].IEEE Access,2022,10(1):58821-58844.
[10]MALEKI D,AFSHARI M,BABAIE M,et al.Ink marker segmentation in histopathology images using deep learning[C]//Advances in Visual Computing:15th International Symposium.Springer International Publishing,2020:359-368.
[11]SHAKARAMI A,NICOLÈ L,TERRERAN M,et al.TCNN:a transformer convolutional neural network for artifact classification in whole slide images[J].Biomedical Signal Processing and Control,2023,84(1):104812.
[12]JANOWCZYK A,ZUO R,GILMORE H,et al.HistoQC:an open-source quality control tool for digital pathology slides[J].JCO Clinical Cancer Informatics,2019,3(1):1-7.
[13]HOSSEINI M S,ZHANG Y Y,PLATANIOTIS K N.Encoding visual sensitivity by maxpol convolution filters for image sharpness assessment[J].IEEE Transactions on Image Processing,2019,28(9):4510-4525.
[14]SENARAS C,NIAZI M K K,LOZANSKI G.DeepFocus:detection of out-of-focus regions in whole slide digital images using deep learning[J].PLoS One,2018,13(10):1-13.
[15]WANG Z,HOSSEINI M S,MILES A,et al.FocusLiteNN:high efficiency focus quality assessment for digital pathology[C]//Medical Image Computing and Computer Assisted Intervention.Springer,2020,12265:403-413.
[16]JIANG S,LIAO J,BIAN Z,et al.Transform-and multi-domain deep learning for single-frame rapid autofocusing in whole slide imaging[J].Biomedical Optics Express,2018,9(4):1601-1612.
[17]HOMEYER,A,GEIBLER C,SCHWEN L O,et al.Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology[J].Modern Pathology,2022,35(1):1759-1769.
[18]CAMPANELLA G,HANNA M G,GENESLAW L,et al.Clinical-grade computational pathology using weakly supervised deep learning on whole slide images[J].Nature Medicine,2019,25(8):1301-1309.
[19]MOBADERSANY P,YOUSEFI S,AMGAD M,et al.Predicting cancer outcomes from histology and genomics using convolu-tional networks[J].Biological Sciences,2018,115(13):e2970-e2979.
[20]CHEN R J,LU M Y,WANG J W,et al.Pathomic fusion:an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis[J].IEEE Transactions on Medical Imaging,2022,41(4):757-770.
[21]DIMITRIOU N,ARANDJELOVIC,CAIE P D.Deep learningfor whole slide image analysis:an overview[J].Frontiers in Medicine,2019,6(1):1-7.
[22]KALRA S,TIZHOOSH H R,CHOI C,et al.Yottixel－animage search engine for large archives of histopathology whole slide images[J].Medical Image Analysis,2020,65(1):1-12.
[23]ADNAN M,KALRA S,TIZHOOSH H R.Representationlearning of histopathology images using graph neural networks[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.IEEE,2020:4254-4261.
[24]SUVARNA K S,LAYTON C,BANCROFT J D.Bancroft'stheory and practice of histological techniques[M].Elsevier Health Sciences,2018.
[25]ZHENG X,WANG R,ZHANG X K,et al.A deep learning modeland human-machine fusion for prediction of EBV-associated gastric cancer from histopathology[J].Nature Communications,2022,13(1):1-12.
[26]RETAMERO J A,ANERIOS F J,RAIMUNDO G D M.Complete digital pathology for routine histopathology diagnosis in a multicenter hospital network[J].Arch Pathol Lab Med,2020,144(2):221-228.
[27]SRINIDHI C L,CIGA O,MARTEL A L.Deep neural network models for computational histopathology:A survey[J].Medical Image Analysis,2021,67(1):1-26.
[28]BEL T D,BOKHORST J M,LAAK J,et al.Residual CycleGAN for robust domain transformation of histopathological tissue slides[J].Medical Image Analysis,2021,70(2):1-12.
[29]BÁNDI P,LOO R,INTEZAR M,et al.Comparison of different methods for tissue segmentation in histopathological whole-slide images[C]//IEEE 14th International Symposium on Biomedical Imaging.IEEE,2017:591-595.
[30]LUO S H,LIU Z M,YANG B W,et al.Advances in staining processing of histological pathology images in deep learning[J].Acta Laser Biology Sinica,2022,31(6):481-487.
[31]TELLEZ D,LITJENS G,BÁNDI P,et al.Quantifying theeffects of data augmentation andstain color normalization in convolutional neural networks for computational pathology[J].Medical Image Analysis,2019,58(1):1-9.
[32]LIU Y,GADEPALLI K,NOROUZI M,et al.Detecting cancer metastases on gigapixel pathology images[EB/OL].[2023-94-13].https://arxiv.org/pdf/1703.02442.pdf.
[33]ABADI M,AGARWAL A,BARHAM P,et al.Tensorflow:large-scale machine learning on heterogeneous distributed systems[EB/OL].[2023-04-15].https://arxiv.org/pdf/1603.04467.pdf.
[34]TELLEZ D,BALKENHOL M,OTTE-HOLLER I,et al.Whole-slide mitosis detection in h&e breast histology using phh3 as a reference to train distilled stain-invariant convolu-tional networks[J].IEEE Transactions on Medical Imaging,2018,37(9):2126-2136.
[35]WAGNER S J,KHALILI N,SHARMA R,et al.Structure-preserving multi-domain stain color augmentation using style-transfer with disentangled representations[C]//Medical Image Computing and Computer Assisted Intervention 24th International Conference.Springer International Publishing,2021:257-266.
[36]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial networks[J].Communications of the ACM,2014,63(11):139-144.
[37]BUG D,STEFFEN S,GROTE A,et al.Context-based normalization of histological stains using deep convolutional features[C]//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support:Third International Workshop,DLMIA 2017,and 7th International Workshop,ML-CDS 2017.Springer International Publishing,2017:135-142.
[38]GLOROT X,BORDES A,BENGIO Y.Deep sparse rectifierneural networks[C]//Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics Workshop and Conference Proceedings.2011:315-323.
[39]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2023-04-25].https://arxiv.org/abs/1409.1556v1.
[40]ZANJANI F G,ZINGER S,BEJNORDI B E,et al.Histopatho-logy stain-color normalization using deep generative models[C]//Medical Imaging with Deep Learning.2018:1-11.
[41]KINGMA D P,WELLING M.Auto-encoding variational bayes[EB/OL].[2023-03-29].https://arxiv.org/pdf/1312.6114.pdf.
[42]JANOWCZYK A,BASAVANHALLY A,MADABHUSHI A.Stain Normalization using Sparse AutoEncoders(StaNoSA):Application to digital pathology[J].Computerized Medical Imaging and Graphics,2016,57(1):50-61.
[43]SHABAN M,CHRISTOPH B,NASSIR N.Staingan:stain style transfer for digital histological images[C]//IEEE 16th International Symposium on Biomedical Imaging.IEEE,2019:953-956.
[44]ZHU J Y,PARK T,ISOLA P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:2223-2232.
[45]CONG C,LIU S,IEVA A D,et al.Colouradaptive generative networks for stain normalisation of histopathology images[J].Medical Image Analysis,2022,82(1):102580.
[46]SALEHI P,CHALECHALE A.Pix2pix-based stain-to-staintranslation:a solution for robust stain normalization in histopathology images analysis[C]//International Conference on Machine Vision and Image Processing.2020:1-7.
[47]RONNEBERGER O,FISCHER P,BROX T.U-net:convolu-tional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015:18th International Conference.Springer International Publishing,2015:234-241.
[48]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks[C]//2017 IEEE Confe-rence on Computer Vision and Pattern Recognition.IEEE,2017:5967-5976.
[49]BENTAIEB A,HAMARNEH G.Adversarial stain transfer for histopathology image analysis[J].IEEE Transactions on Medical Imaging,2018,37(3):792-802.
[50]GATYS L A,ECKER A S,BETHG E.Image style transferusing convolutional neural networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:2414-2423.
[51]HOSSEINI M S,ZHANG Y Y,CHANG L D,et al.Focus quality assessment of high-throughput whole slide imaging in digital pathology[J].IEEE Transaction on Medical Imaging,2018,39(1):62-74.
[52]LI Q,LIU X M,HAN K,et al.Learning to autofocus in whole slide imaging via physics-guided deep cascade networks[J].Optics Express,2022,30(9):14319-14340.
[53]DASTIDAR T R.Automated focus distance estimation for digi-tal microscopy using deep convolutional neural networks[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.IEEE,2019:1049-1056.
[54]DENG Y,FENG M,JIANG Y,et al.Development of pathological super-resolution images using artificial intelligence based on whole slide image[J].MedComm,2020,1(3):410-417.
[55]KOTHARI S,PHAN J H,WANG M D.Eliminating tissue-fold artifacts in histopathological whole-slide images for improved image-based prediction of cancer grade[J].Journal Pathology Informatics,2013,4(1)1-11.
[56]PALOKANGAS S,SELINUMMI J,YLI-HARJA O.Segmentation of folds in tissue section images[C]//Annual International Conference of the IEEE Engineering in Medicine and Biology Society.IEEE,2007:5642-5645.
[57]BABAIE M,TIZHOOSH H R.Deep features for tissue-fold detection in histopathology images[C]//Digital Pathology:15th European Congress.Springer International Publishing,2019:125-132.
[58]CORTES C,VAPNI K.Support-vector networks[J].Machine Learning,1995,20(1):273-297.
[59]HEMMATIRAD K,BABAIE M,AFSHARI M,et al.Qualitycontrol of whole slide images using the yolo concept[C]//2022 IEEE 10th International Conference on Healthcare Informatics.IEEE,2000:282-287.
[60]BOCHKOVSKIY A,WANG C Y,LIAO H Y M.Yolov4:optimal speed and accuracy of object detection[EB/OL].[2023-04-23].https://arxiv.org/pdf/2004.10934.pdf.
[61]SMIT G,CIOMPI F,CIGÉHN M,et al.Quality control ofwhole-slide images through multi-class semantic segmentation of artifacts[C]//Medical Imaging with Deep Learning.2021:1-3.
[62]CHEN L C,ZHU Y,PAPANDREOU G,et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision.Springer,2018:801-818.
[63]TAN M,LE Q.Efficientnet:rethinking model scaling for convolutional neural networks[C]//International Conference on Machine Learning.2019:6105-6114.
[64]HOSSAIN M S,SHAHRIAR G M,SYEED M M M,et al.Tissue artifact segmentation and severity assessment for automatic analysis using wsi[J].IEEE Access,2023,11(1):21977-21991.
[65]JHA D,RIEGLER M,JOHANSEN D,et al.DoubleU-Net:adeep convolutional neural network for medical image segmentation[C]//IEEE 33rd International Symposium on Computer-Based Medical Systems.IEEE,2020:558-564.
[66]SHAIKH N N,WASAG K,NIE Y.Artifact identification indigital histopathology images using few-shot learning[C]//IEEE 19th International Symposium on Biomedical Imaging.IEEE,2022:1-4.
[67]FOUCART A,DEBEIR O,DECAESTECKER C.Artifact identification in digital pathology fromweak and noisy supervision with deep residual networks[C]//4th International Conference on Cloud Computing Technologies and Applications Brussels.Belgium,2018:1-6.
[68]TIAN X C,TANG H J.Software collection of the optical mea-suring len's distortion in the CCD-assembly scanner[J].Optics and Precision Engineering,2001,9(3):234-237.
[69]XU W,GU S,CHU C Z,et al.Real-time correction of imagedrift in scanning electron microscope[J].Opto-Electronic Engineering,2018,45(12):40-47.
[70]LIANG M Y,ZHANG Q N.Pathological image reconstruction based on deep transfer learning[J].Journal of Shanxi University(Nat.Sci.Ed),2022,45(6):1504-1512.
[71]BROCK A,DONAHUE J,SIMONYAN K.Large scale gantraining for high fidelity natural image synthesis[C]//7th International Conference on Learning Representations.ICLR,2019:1-35.
[72]喻罡,高燕华,孙凯.基于稀疏采样的病理切片多分辨率图像重建方法及系统:中国CN115908283A[P].2023.04.04.
[73]LIANG M Y,ZHANG Y,LIANG J N,et al.Pathological Image Super-resolution Reconstruction Based on Sparse Coding Non-local AttentionDual Network[J].Computer Science,2023,50(S1):220700016-8.
[74]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:770-778.
[75]ALI S,ALHAM N K,VERRILL C,et al.Ink removal from histopathology whole slide images by combining classification,detection and image generation models[C]//IEEE 16th International Symposium on Biomedical Imaging.IEEE,2019:928-932.
[76]VENKATESH B,SHAHT T,CHEN A,et al.Restoration of marker occluded hematoxylin and eosin stained whole slide histology images using generative adversarial networks[C]//IEEE 17th International Symposium on Biomedical Imaging.IEEE,2020:591-595.
[77]GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adversarial training of neural networks[J].The Journal of Machine Learning Research,2015,17(1):2096-2030.
[78]WU Y Z,KANG Y,LUO J H,et al.FedCG:leverage conditional gan for protecting privacy and maintaining competitive perfor-mance in federated learning[EB/OL].[2023-04-16].https://arXiv.org/abs/2111.08211.
[79]MITTAL A,MOORTHY A K,BOVIK A C.No-referenceimage quality assessment in the spatial domain[J].IEEE Transactions on Image Processing,2012,21(12):4695-4708.
[80]SCHÖMIG-MARKIEFKA B,PRYALUKHIN A,HULLA W,et al.Quality control stress test for deep learning-based diagnostic model in digital pathology[J].Modern Pathology,2021,34(1):2098-2108.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed