Computer Science ›› 2025, Vol. 52 ›› Issue (2): 202-211.doi: 10.11896/jsjkx.240400048

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Unsupervised Multi-class Anomaly Detection Based on Prototype Reverse Distillation

HE Liren1, PENG Bo2, CHI Mingmin1   

  1. 1 School of Computer Science,Fudan University,Shanghai 200438,China
    2 College of Information Technology,Shanghai Ocean University,Shanghai 201306,China
  • Received:2024-04-08 Revised:2024-07-01 Online:2025-02-15 Published:2025-02-17
  • About author:HE Liren,born in 2000,postgraduate.His main research interests include deep learning and anomaly detection.
    CHI Mingmin,born in 1976,Ph.D,associate professor,Ph.D supervisor.Her main research interests include machine learning and deep learning.
  • Supported by:
    National Natural Science Foundation of China(62171139).

Abstract: Unsupervised anomaly detection is widely used in industrial quality inspection and other domains due to its requirement of only normal samples for training.Existing single-class anomaly detection methods exhibit a significant performance decrease when directly applied to multi-class anomaly detection.Among them,knowledge distillation-based anomaly detection methods distill the feature knowledge of pre-trained teacher models on normal samples into student models.However,they have the problem that they can't guarantee thestudent models learn only normal sample knowledge in multi-class anomaly detection.This paper proposes an unsupervised multi-class anomaly detection method,PRD(Prototype based reverse distillation),based on a reverse knowledge distillation framework.It utilizes the Multi-class Normal Prototype module and Sparse Prototype Recall training stra-tegy to learn prototypes of multiple-class normal sample features from the teacher model.These prototypes are then used to filter the input features of the student model,ensuring that the student model only learns the feature knowledge of normal samples from the teacher model.PRD surpasses existing state-of-the-art methods on various industrial anomaly detection datasets.Qualitative,quantitative,and ablation experiments validate the effectiveness of the PRD framework and its internal modules.

Key words: Anomaly detection, Unsupervised learning, Prototype learning, Knowledge distillation, Pre-trained features

CLC Number: 

  • TP181
[1]BERGMANN P,FAUSER M,SATTLEGGER D,et al.Mvte-cad-a comprehensive real-world dataset for unsupervised ano-maly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:9592-9600.
[2]ZOU Y,JEONG J,PEMULA L,et al.Spot-the-difference self-supervised pre-training for anomaly detection and segmentation[C]//European Conference on Computer Vision.Springer,2022:392-408.
[3]FERNANDO T,GAMMULLE H,DENMAN S,et al.Deeplearning for medical anomaly detection-a survey[J].ACM Computing Surveys(CSUR),2021,54(7):1-37.
[4]DEFARD T,SETKOV A,LOESCH A,et al.Padim:a patch distribution modeling framework for anomaly detection and localization[C]//International Conference on Pattern Recognition.Springer,2021:475-489.
[5]GUDOVSKIY D,ISHIZAKA S,KOZUKA K.Cflow-ad:Real-time unsupervised anomaly detection with localization via conditional normalizing flows[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2022:98-107.
[6]LEI J,HU X,WANG Y,et al.Pyramidflow:High-resolution defect contrastive localization using pyramid normalizing flow[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:14143-14152.
[7]RIPPEL O,MERTENS P,MERHOF D.Modeling the distribution of normal data in pre-trained deep features for anomaly detection[C]//2020 25th International Conference on Pattern Recognition(ICPR).IEEE,2021:6726-6733.
[8]ROTH K,PEMULA L,ZEPEDA J,et al.Towards total recall in industrial anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:14318-14328.
[9]YAN X,ZHANG H,XU X,et al.Learning semantic contextfrom normal samples for unsupervised anomaly detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence:Vol.35.2021:3110-3118.
[10]YAO X,LI R,QIAN Z,et al.Focus the discrepancy:Intra-and inter-correlation learning for image anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:6803-6813.
[11]YOU Z,YANG K,LUO W,et al.Adtr:Anomaly detectiontransformer with feature reconstruction[C]//International Conference on Neural Information Processing.Springer,2022:298-310.
[12]ZAVRTANIK V,KRISTAN M,SKOCˇAJ D.Draem-a discriminatively trained reconstruction embedding for surface anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:8330-8339.
[13]ZHANG H,WANG Z,WU Z,et al.Diffusionad:Denoising diffusion for anomaly detection[J].arXiv:2303.08730,2023.
[14]BERGMANN P,FAUSER M,SATTLEGGER D,et al.Unin-formed students:Student-teacher anomaly detection with discriminative latent embeddings[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:4183-4192.
[15]DENG H,LI X.Anomaly detection via reverse distillation from one-class embedding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:9737-9746.
[16]SALEHI M,SADJADI N,BASELIZADEH S,et al.Multiresolution knowledge distillation for anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14902-14912.
[17]TIEN T D,NGUYEN A T,TRAN N H,et al.Revisiting reverse distillation for anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:24511-24520.
[18]YOU Z,CUI L,SHEN Y,et al.A unified model for multi-class anomaly detection[J].Advances in Neural Information Proces-sing Systems,2022,35:4571-4584.
[19]ZHAO Y.Omnial:A unified cnn framework for unsupervisedanomaly localization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:3924-3933.
[20]MASOUDNIA S,EBRAHIMPOUR R.Mixture of experts:a li-terature survey[J].Artificial Intelligence Review,2014,42:275-293.
[21]RUDOLPH M,WANDT B,ROSENHAHN B.Same same butdiffernet:Semi-supervised defect detection with normalizing flows[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2021:1907-1916.
[22]RUDOLPH M,WEHRBEIN T,ROSENHAHN B,et al.Fully convolutional cross-scale-flows for image-based defect detection[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2022:1088-1097.
[23]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial networks[J].Communications of the ACM,2020,63(11):139-144.
[24]HO J,JAIN A,ABBEEL P.Denoising diffusion probabilisticmodels[J].Advances in Neural Information Processing Systems,2020,33:6840-6851.
[25]GU Z,LIU L,CHEN X,et al.Remembering normality:Memory-guided knowledge distillation for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:16401-16409.
[26]DENG J,DONG W,SOCHER R,et al.Imagenet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2009:248-255.
[27]ZAGORUYKO S,KOMODAKIS N.Wide residual networks[J].arXiv:1605.07146,2016.
[28]TAN M,LE Q.Efficientnet:Rethinking model scaling for con-volutional neural networks[C]//International Conference on Machine Learning.PMLR,2019:6105-6114.
[29]CAI Z,VASCONCELOS N.Cascade r-cnn:Delving into highquality object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2018:6154-6162.
[30]LIU Z,ZHOU Y,XU Y,et al.Simplenet:A simple network for image anomaly detection and localization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:20402-20411.
[31]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[1] YIN Wencui, XIE Ping, YE Chengxu, HAN Jiaxin, XIA Xing. Anomaly Detection of Multi-variable Time Series Data Based on Variational Graph Auto-encoders [J]. Computer Science, 2025, 52(6A): 240700124-8.
[2] ZHANG Hang, WEI Shoulin, YIN Jibin. TalentDepth:A Monocular Depth Estimation Model for Complex Weather Scenarios Based onMultiscale Attention Mechanism [J]. Computer Science, 2025, 52(6A): 240900126-7.
[3] LIAO Sirui, HUANG Feihu, ZHAN Pengxiang, PENG Jian, ZHANG Linghao. DCDAD:Differentiated Context Dependency for Time Series Anomaly Detection Method [J]. Computer Science, 2025, 52(6): 106-117.
[4] SI Yuehang, CHENG Qing, HUANG Jincai. Multi-assistant Dynamic Setting Method for Knowledge Distillation [J]. Computer Science, 2025, 52(5): 241-247.
[5] AN Rui, LU Jin, YANG Jingjing. Deep Clustering Method Based on Dual-branch Wavelet Convolutional Autoencoder and DataAugmentation [J]. Computer Science, 2025, 52(4): 129-137.
[6] YAN Tingju, CAO Yan, WANG Yijing. Low-power Bluetooth Spoofing Attack Detection Technology Based on RFFAD_DeepSVDD [J]. Computer Science, 2025, 52(2): 380-387.
[7] YUAN Ye, CHEN Ming, WU Anbiao, WANG Yishu. Graph Anomaly Detection Model Based on Personalized PageRank and Contrastive Learning [J]. Computer Science, 2025, 52(2): 80-90.
[8] DING Xinyu, KONG Bing, CHEN Hongmei, BAO Chongming, ZHOU Lihua. Path-masked Autoencoder Guiding Unsupervised Attribute Graph Node Clustering [J]. Computer Science, 2025, 52(1): 160-169.
[9] KONG Lingchao, LIU Guozhu. Review of Outlier Detection Algorithms [J]. Computer Science, 2024, 51(8): 20-33.
[10] GUO Fangyuan, JI Genlin. Video Anomaly Detection Method Based on Dual Discriminators and Pseudo Video Generation [J]. Computer Science, 2024, 51(8): 217-223.
[11] FAN Yi, HU Tao, YI Peng. Host Anomaly Detection Framework Based on Multifaceted Information Fusion of SemanticFeatures for System Calls [J]. Computer Science, 2024, 51(7): 380-388.
[12] ZENG Zihui, LI Chaoyang, LIAO Qing. Multivariate Time Series Anomaly Detection Algorithm in Missing Value Scenario [J]. Computer Science, 2024, 51(7): 108-115.
[13] LI Dongyang, NIE Rencan, PAN Linna, LI He. UMGN:An Infrared and Visible Image Fusion Network Based on Unsupervised Significance MaskGuidance [J]. Computer Science, 2024, 51(6A): 230600170-5.
[14] SHI Songhao, WANG Xiaodan, YANG Chunxiao, WANG Yifei. SAR Image Target Recognition Based on Cross Domain Few Shot Learning [J]. Computer Science, 2024, 51(6A): 230800136-7.
[15] LOU Ren, HE Renqiang, ZHAO Sanyuan, HAO Xin, ZHOU Yueqi, WANG Xinyuan, LI Fangfang. Single Stage Unsupervised Visible-infrared Person Re-identification [J]. Computer Science, 2024, 51(6A): 230600138-7.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!