Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (6): 172-185.doi: 10.11896/jsjkx.230400106

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Survey of Breast Cancer Pathological Image Analysis Methods Based on Graph Neural Networks

CHEN Sishuo, WANG Xiaodong, LIU Xiyang   

  1. School of Computer Science and Technology,Xidian University,Xi’an 710126,China
  • Received:2023-04-14 Revised:2023-08-09 Online:2024-06-15 Published:2024-06-05
  • About author:CHEN Sishuo,born in 1999,postgraduate.His main research interests include computational pathology and graph neural network.
    LIU Xiyang,born in 1970,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.12354S).His main research interests include clinical application of high-perfor-mance medical image analysis,computational pathology,and clinical decision support in complex and uncertain conditions.
  • Supported by:
    National Natural Science Foundation of China(82172860) and Young Scientists Fund of the National Natural Science Foundation of China(8210101340).

PDF (PC)

619

Abstract: Pathological diagnosis is the gold standard for cancer diagnosis and treatment,the use of artificial intelligence(AI) models for analyzing pathological images has the potential to not only reduce the workload of pathologists but also improve the accuracy of cancer diagnosis and treatment.However,these methods face challenges due to the large scale of pathological images and the difficulty in interpreting the predicted results.In recent studies,graph neural networks have shown their strong abilities in modeling spatial context and interpretability of entities in images,which provides a new idea for the study of digital pathology.In this survey,we review recent related works in computer vision,analyze the advantages of graph neural networks for breast cancer pathology,classify and compare existing graph construction methods,and analyze and compare graph neural network models proposed in recent years.We also summarize the challenges that exist in using graph neural networks for analyzing pathological images of breast cancer and prospect the future research directions.

Key words: Breast cancer pathological image, Graph neural network, Graph classification, Digital pathology

CLC Number: 

  • TP391.4
[1]SUNG H,FERLAY J,SIEGEL R L,et al.Global Cancer Statistics 2020:GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries[J].CA:A Cancer Journal for Clinicians,2021,71(3):209-249.
[2]LOIBL S,POORTMANS P,MORROW M,et al.Breast cancer[J].The Lancet,2021,397(10286):1750-1769.
[3]HOU L,SAMARAS D,KURC T M,et al.Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Las Vegas,NV,USA:IEEE,2016:2424-2433.
[4]YAN R,REN F,WANG Z,et al.Breast cancer histopathological image classification using a hybrid deep neural network[J].Methods,2020,173:52-60.
[5]ZHAN Y,BIAN H,CHEN Y,et al.Breast Tumor Image Classification in Bright Challenge VIA Multiple Instance Learning and Deep Transformers[C]//2022 IEEE International Symposium on Biomedical Imaging Challenges(ISBIC).Kolkata,India:IEEE,2022.
[6]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isAll you Need[C]//Advances in Neural Information Processing Systems.Long Beach:Curran Associates,2017.
[7]LI F,YANG Y,WEI Y,et al.Deep learning-based predictive bio-marker of pathological complete response to neoadjuvant chemotherapy from histological images in breast cancer[J].Journal of Translational Medicine,2021,19(1):348.
[8]LOU X,ZHOU N,FENG L,et al.Deep Learning Model for Predicting the Pathological Complete Response to Neoadjuvant Chemoradiotherapy of Locally Advanced Rectal Cancer[J].Frontiers in Oncology,2022,12:807264.
[9]HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[10]CHEN R J,CHEN C,LI Y,et al.Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New Orleans,LA,USA:IEEE,2022:16123-16134.
[11]DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.AnImage is Worth 16×16 Words:Transformers for Image Recognition at Scale[C]//International Conference on Learning Representations.2021.
[12]WAN S,GONG C,ZHONG P,et al.Hyperspectral Image Classification With Context-Aware Dynamic Graph Convolutional Network[J].IEEE Transactions on Geoscience and Remote Sensing,2021,59(1):597-612.
[13]JIAO L,CHEN J,LIU F,et al.Graph Representation Learning Meets Computer Vision:A Survey[J].IEEE Transactions on Artificial Intelligence,2022.4(1):1-22.
[14]PATI P,JAUME G,FONCUBIERTA-RODRÍGUEZ A,et al.Hierarchical graph representations in digital pathology[J].Me-dical Image Analysis,2022,75:102264.
[15]ZHANG J,HUA Z,YAN K,et al.Joint fully convolutional and graph convolutional networks for weakly-supervised segmentation of pathology images[J].Medical Image Analysis,2021,73:102183.
[16]ZHENG Y,JIANG Z,SHI J,et al.Encoding histopathologywhole slide images with location-aware graphs for diagnostically relevant regions retrieval[J].Medical Image Analysis,2022,76:102308.
[17]GAO Z,LU Z,WANG J,et al.A Convolutional Neural Network and Graph Convolutional Network Based Framework for Classification of Breast Histopathological Images[J].IEEE Journal of Biomedical and Health Informatics,2022,26(7):3163-3173.
[18]CHEN R J,LU M Y,WANG J,et al.Pathomic Fusion:An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis[J].IEEE Transactions on Medical Imaging,2022,41(4):757-770.
[19]CHEN R J,LU M Y,SHABAN M,et al.Whole Slide Images are 2D Point Clouds:Context-Aware Survival Prediction Using Patch-Based Graph Convolutional Networks[C]//MedicalImage Computing and Computer Assisted Intervention-MICCAI 2021.Cham:Springer International Publishing,2021:339-349.
[20]VELIČKOVIĆ P,CUCURULL G,CASANOVA A,et al.Graph Attention Networks[C]//International Conference on Learning Representations.Vancouver,2018.
[21]XU B B,CEN K Y,HUANG J J,et al.A Survey on Graph Convolutional Neural Network[J].Chinese Journal of Computers,2020,43(5):755-780.
[22]YANG C,LIU Z,ZHAO D,et al.Network RepresentationLearning with Rich Text Information[C]//Proceedings of the 24th International Conference on Artificial Intelligence.2015:2111-2117.
[23]PEROZZI B,AL-RFOU R,SKIENA S.DeepWalk:OnlineLearning of Social Representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Disco-very and Data Mining.2014:701-710.
[24]CHAWLA N,WANG W.Accelerated Attributed Network Embedding[C]//Proceedings of the 2017 SIAM International Conference on Data Mining.Society for Industrial and Applied Mathematics,2017:633-641.
[25]GROVER A,LESKOVEC J.node2vec:Scalable Feature Lear-ning for Networks.[C]//KDD:Proceedings.International Conference on Knowledge Discovery & Data Mining.New York:Association for Computing Machinery,2016:855-864.
[26]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient Estimation of Word Representations in Vector Space[C]//Interna-tional Conference on Learning Representations.2013.
[27]HAMILTON W,YING Z,LESKOVEC J.Inductive Representation Learning on Large Graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:1025-1035.
[28]ACHANTA R,SHAJI A,SMITH K,et al.SLIC SuperpixelsCompared to State-of-the-Art Superpixel Methods[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(11):2274-2282.
[29]BILGIN C,DEMIR C,NAGI C,et al.Cell-Graph Mining forBreast Tissue Modeling and Classification[C]//2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.Lyon,France:IEEE,2007:5311-5314.
[30]GRAHAM S,VU Q D,RAZA S E A,et al.Hover-Net:Simultaneous segmentation and classification of nuclei in multi-tissue histology images[J].Medical Image Analysis,2019,58:101563.
[31]LU C,KOYUNCU C,CORREDOR G,et al.Feature-driven local cell graph(FLocK):New computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers[J].Medical Image Analysis,2021,68:101903.
[32]LU W,GRAHAM S,BILAL M,et al.Capturing Cellular Topo-logy in Multi-Gigapixel Pathology Images[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).Seattle,WA,USA:IEEE,2020:1049-1058.
[33]MÜLLNER D.Modern hierarchical,agglomerative clustering algorithms[J].arXiv,1109.2378,2011.
[34]COMANICIU D,MEER P.Mean shift:a robust approach toward feature space analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(5):603-619.
[35]LEE Y,PARK J H,OH S,et al.Derivation of prognostic contextual histopathological features from whole-slide images of tumours via graph deep learning[J].Nature Biomedical Engineering,2022:1-15.
[36]ZHANG M,DONG B,LI Q.Ms-Gwnn:Multi-Scale GraphWavelet Neural Network for Breast Cancer Diagnosis[C]//2022 IEEE 19th International Symposium on Biomedical Imaging(ISBI).Kolkata,India:IEEE,2022:1-5.
[37]OZEN Y,AKSOY S,KOSEMEHMETOGLU K,et al.Self-Supervised Learning with Graph Neural Networks for Region of Interest Retrieval in Histopathology[C]//2020 25th International Conference on Pattern Recognition(ICPR).Milan,Italy:IEEE,2021:6329-6334.
[38]CHEN Z,ZHANG J,CHE S,et al.Diagnose Like A Patholo-gist:Weakly-Supervised Pathologist-Tree Network for Slide-Level Immunohistochemical Scoring[J].Proceedings of the AAAI Conference on Artificial Intelligence,2021,35(1):47-54.
[39]HOU W,HUANG H,PENG Q,et al.Spatial-HierarchicalGraph Neural Network with Dynamic Structure Learning for Histological Image Classification[C]//Medical Image Computing and Computer Assisted Intervention(MICCAI 2022).Cham:Springer Nature Switzerland,2022:181-191.
[40]KIPF T N,WELLING M.Semi-Supervised Classification with Graph Convolutional Networks[C]//International Conference on Learning Representations.Toulon,2017.
[41]XU K,HU W,LESKOVEC J,et al.How Powerful are Graph Neural Networks?[C]//International Conference on Learning Representations.2019.
[42]ZHOU P,WU Z,WEN G,et al.Multi-scale graph classification with shared graph neural network[J].World Wide Web,2023:26(3):949-966.
[43]CAI L,JI S.A Multi-Scale Approach for Graph Link Prediction[C]//34TH AAAI Conference on Artificial Intelligence.2020:3308-3315.
[44]STENZINGER A,ALBER M,ALLGÄUER M,et al.Artificial intelligence and pathology:From principles to practice and future applications in histomorphology and molecular profiling[J].Seminars in Cancer Biology,2022,84:129-143.
[45]KATHER J N,PEARSON A T,HALAMA N,et al.Deeplearning can predict microsatellite instability directly from histology in gastrointestinal cancer[J].Nature Medicine,2019,25(7):1054-1056.
[46]SHAO D,DAI Y,LI N,et al.Artificial intelligence in clinical research of cancers[J].Briefings in Bioinformatics,2022,23(1):bbab523.
[47]ROY A G,REN J,AZIZI S,et al.Does Your Dermatology Classifier Know What It Doesn’t Know? Detecting the Long-Tail of Unseen Conditions[J].Medical Image Analysis,2022,75:102274-102292.
[48]CHEN T,KORNBLITH S,NOROUZI M,et al.A SimpleFramework for Contrastive Learning of Visual Representations[C]//International Conference on Machine Learning.2020.
[49]BACCIU D,ERRICA F,MICHELI A,et al.A Gentle Introduction to Deep Learning for Graphs[J].Neural Networks,2020,129:203-221.
[50]YING Z,YOU J,MORRIS C,et al.Hierarchical Graph Representation Learning with Differentiable Pooling[C]//Interna-tional Conference on Neural Information Processing Systems.New York:Curran Associates,2018:4805-4815.
[51]MORRIS C,RITZERT M,FEY M,et al.Weisfeiler and Leman Go Neural:Higher-Order Graph Neural Networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Hawaii,2019:4602-4609.
[52]ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning DeepFeatures for Discriminative Localization[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Las Vegas,NV,USA:IEEE,2016:2921-2929.
[53]LIU Z,LI J,SHEN Z,et al.Learning Efficient ConvolutionalNetworks through Network Slimming[C]//2017 IEEE International Conference on Computer Vision(ICCV).Venice:IEEE,2017:2755-2763.
[54]ACS B,RANTALAINEN M,HARTMAN J.Artificial intelligence as the next step towards precision pathology[J].Journal of Internal Medicine,2020,288(1):62-81.
[55]SINGH A,SENGUPTA S,LAKSHMINARAYANAN V.Ex-plainable Deep Learning Models in Medical Image Analysis[J].Journal of Imaging,2020,6(6):52-70.
[56]JAUME G,PATI P,BOZORGTABAR B,et al.Quantifying Explainers of Graph Neural Networks in Computational Pathology[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville,2021:8106-8116.
[57]JAUME G,PATI P,FONCUBIERTA-RODRIGUEZ A,et al.Towards Explainable Graph Representations in Digital Pathology[J].arXiv:2007.00311,2020.
[58]YING Z,BOURGEOIS D,YOU J,et al.GNNExplainer:Generating Explanations for Graph Neural Networks[C]//Advances in Neural Information Processing Systems.2019:9240-9251.
[59]SCHWARZENBERG R,HÜBNER M,HARBECKE D,et al.Layerwise Relevance Visualization in Convolutional Text Graph Classifiers[C]//Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing.2019:58-62.
[60]POPE P E,KOLOURI S,ROSTAMI M,et al.ExplainabilityMethods for Graph Convolutional Neural Networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach,CA,USA:IEEE,2019:10764-10773.
[61]CHATTOPADHAY A,SARKAR A,HOWLADER P,et al.Grad-CAM++:Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks[C]//2018 IEEE Winter Conference on Applications of Computer Vision(WACV).Lake Tahoe,NV:IEEE,2018:839-847.
[62]BRANCATI N,ANNICIELLO A M,PATI P,et al.BRACS:A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images[J].Database,2021,2022(2022):baac093.
[63]ARESTA G.BACH:Grand challenge on breast cancer histology images[J].Medical Image Analysis,2019,56:122-139.
[64]ARAÚJO T,ARESTA G,CASTRO E,et al.Classification of breast cancer histology images using Convolutional Neural Networks[J].PLOS ONE,2017,12(6)::e0177544.
[65]BOLHASANI H,AMJADI E,TABATABAEIAN M,et al.A histopathological image dataset for grading breast invasive ductal carcinomas[J].Informatics in Medicine Unlocked,2020,19:100341.
[66]SPANHOL F A,OLIVEIRA L S,PETITJEAN C,et al.A Dataset for Breast Cancer Histopathological Image Classification[J].IEEE Transactions on Biomedical Engineering,2016,63(7):1455-1462.
[67]CLARK K,VENDT B,SMITH K,et al.The Cancer ImagingArchive(TCIA):Maintaining and Operating a Public Information Repository[J].Journal of Digital Imaging,2013,26(6):1045-1057.
[68]THE CANCER GENOME ATLAS NETWORK.Comprehen-sive molecular portraits of human breast tumours[J].Nature,2012,490(7418):61-70.
[69]WANG M,ZHENG D,YE Z,et al.Deep Graph Library:AGraph-Centric,Highly-Performant Package for Graph Neural Networks[J].arXiv:1909.01315,2019.
[70]FEY M,LENSSEN J E.Fast Graph Representation Learning with PyTorch Geometric[J].arXiv:1903.02428,2019.
[71]JAUME G,PATI P,ANKLIN V,et al.HistoCartography:AToolkit for Graph Analytics in Digital Pathology[C]//Procee-dings of the MICCAI Workshop on Computational Pathology.PMLR,2021:117-128.
[72]WU Z,PAN S,CHEN F,et al.A Comprehensive Survey onGraph Neural Networks[J].IEEE Transactions on Neural Networks and Learning Systems,2021,32(1):4-24.
[73]WANG Z H,SHEN H W,CAO Q,et al.Survey on Graph Classification[J].Journal of Software,2022,33(1):171-192.
[1] CHU Xiaoxi, ZHANG Jianhui, ZHANG Desheng, SU Hui. Browser Fingerprint Tracking Based on Improved GraphSAGE Algorithm [J]. Computer Science, 2024, 51(6): 409-415.
[2] LU Min, YUAN Ziting. Graph Contrast Learning Based Multi-graph Neural Network for Session-based RecommendationMethod [J]. Computer Science, 2024, 51(5): 54-61.
[3] LAN Yongqi, HE Xingxing, LI Yingfang, LI Tianrui. New Graph Reduction Representation and Graph Neural Network Model for Premise Selection [J]. Computer Science, 2024, 51(5): 193-199.
[4] ZHANG Liying, SUN Haihang, SUN Yufa , SHI Bingbo. Review of Node Classification Methods Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(4): 95-105.
[5] ZHANG Tao, LIAO Bin, YU Jiong, LI Ming, SUN Ruina. Benchmarking and Analysis for Graph Neural Network Node Classification Task [J]. Computer Science, 2024, 51(4): 132-150.
[6] LUO Zeyang, TIAN Hua, DOU Yingtong, LI Manwen, ZHANG Zehua. Fake Review Detection Based on Residual Networks Fusion of Multi-relationship Review Features [J]. Computer Science, 2024, 51(4): 314-323.
[7] CHEN Yufeng , HUANG Zengfeng. Framework and Algorithms for Accelerating Training of Semi-supervised Graph Neural Network Based on Heuristic Coarsening Algorithms [J]. Computer Science, 2024, 51(3): 48-55.
[8] HUANG Kun, SUN Weiwei. Traffic Speed Forecasting Algorithm Based on Missing Data [J]. Computer Science, 2024, 51(3): 72-80.
[9] CHEN Wei, ZHOU Lihua, WANG Yafeng, WANG Lizhen, CHEN Hongmei. Community Search Based on Disentangled Graph Neural Network in Heterogeneous Information Networks [J]. Computer Science, 2024, 51(3): 90-101.
[10] ZHANG Guohao, WANG Yi, ZHOU Xi, WANG Baoquan. Deep Collaborative Truth Discovery Based on Variational Multi-hop Graph Attention Encoder [J]. Computer Science, 2024, 51(3): 109-117.
[11] ZHENG Cheng, SHI Jingwei, WEI Suhua, CHENG Jiaming. Dual Feature Adaptive Fusion Network Based on Dependency Type Pruning for Aspect-basedSentiment Analysis [J]. Computer Science, 2024, 51(3): 205-213.
[12] XU Tianyue, LIU Xianhui, ZHAO Weidong. Knowledge Graph and User Interest Based Recommendation Algorithm [J]. Computer Science, 2024, 51(2): 55-62.
[13] JIN Yu, CHEN Hongmei, LUO Chuan. Interest Capturing Recommendation Based on Knowledge Graph [J]. Computer Science, 2024, 51(1): 133-142.
[14] WU Jiawei, FANG Quan, HU Jun, QIAN Shengsheng. Pre-training of Heterogeneous Graph Neural Networks for Multi-label Document Classification [J]. Computer Science, 2024, 51(1): 143-149.
[15] GUO Yuxing, YAO Kaixuan, WANG Zhiqiang, WEN Liangliang, LIANG Jiye. Black-box Graph Adversarial Attacks Based on Topology and Feature Fusion [J]. Computer Science, 2024, 51(1): 355-362.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.