计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (2): 202-211.doi: 10.11896/jsjkx.240400048

• 计算机图形学&多媒体 • 上一篇    下一篇

基于Prototype反向蒸馏的无监督多类别异常检测

何立仁1, 彭博2, 池明旻1   

  1. 1 复旦大学计算机科学技术学院 上海 200438
    2 上海海洋大学信息学院 上海 201306
  • 收稿日期:2024-04-08 修回日期:2024-07-01 出版日期:2025-02-15 发布日期:2025-02-17
  • 通讯作者: 池明旻(mmchi@fudan.edu.cn)
  • 作者简介:(lrhe21@m.fudan.edu.cn)
  • 基金资助:
    国家自然科学基金(62171139)

Unsupervised Multi-class Anomaly Detection Based on Prototype Reverse Distillation

HE Liren1, PENG Bo2, CHI Mingmin1   

  1. 1 School of Computer Science,Fudan University,Shanghai 200438,China
    2 College of Information Technology,Shanghai Ocean University,Shanghai 201306,China
  • Received:2024-04-08 Revised:2024-07-01 Online:2025-02-15 Published:2025-02-17
  • About author:HE Liren,born in 2000,postgraduate.His main research interests include deep learning and anomaly detection.
    CHI Mingmin,born in 1976,Ph.D,associate professor,Ph.D supervisor.Her main research interests include machine learning and deep learning.
  • Supported by:
    National Natural Science Foundation of China(62171139).

PDF (PC)

113

摘要： 无监督异常检测因只需要正常样本进行训练而被广泛应用于工业质检等领域。直接将现有的单类别异常检测方法应用到多类别异常检测中会导致性能显著下降,其中基于知识蒸馏的异常检测方法将预训练的教师模型关于正常样本的特征知识蒸馏到学生模型中,然而它们在多类别异常检测中存在无法保证学生模型只学习到正常样本知识的问题。文中提出一种基于反向知识蒸馏框架的无监督多类别异常检测方法(Prototype based Reverse Distillation,PRD ),其通过Multi-class Normal Prototype模块和Sparse Prototype Recall训练策略来学习教师模型关于多类别正常样本特征的 Prototype,并以此来过滤学生模型的输入特征,从而确保学生模型只学习到教师模型关于正常样本的特征知识。PRD在多种工业异常检测数据集上性能均超越了现有的SOTA方法,定性、定量和消融实验验证了PRD整体框架和内部模块的有效性。

关键词: 异常检测, 无监督学习, Prototype 学习, 知识蒸馏, 预训练特征

Abstract: Unsupervised anomaly detection is widely used in industrial quality inspection and other domains due to its requirement of only normal samples for training.Existing single-class anomaly detection methods exhibit a significant performance decrease when directly applied to multi-class anomaly detection.Among them,knowledge distillation-based anomaly detection methods distill the feature knowledge of pre-trained teacher models on normal samples into student models.However,they have the problem that they can't guarantee thestudent models learn only normal sample knowledge in multi-class anomaly detection.This paper proposes an unsupervised multi-class anomaly detection method,PRD(Prototype based reverse distillation),based on a reverse knowledge distillation framework.It utilizes the Multi-class Normal Prototype module and Sparse Prototype Recall training stra-tegy to learn prototypes of multiple-class normal sample features from the teacher model.These prototypes are then used to filter the input features of the student model,ensuring that the student model only learns the feature knowledge of normal samples from the teacher model.PRD surpasses existing state-of-the-art methods on various industrial anomaly detection datasets.Qualitative,quantitative,and ablation experiments validate the effectiveness of the PRD framework and its internal modules.

Key words: Anomaly detection, Unsupervised learning, Prototype learning, Knowledge distillation, Pre-trained features

中图分类号: 

  • TP181
[1]BERGMANN P,FAUSER M,SATTLEGGER D,et al.Mvte-cad－a comprehensive real-world dataset for unsupervised ano-maly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:9592-9600.
[2]ZOU Y,JEONG J,PEMULA L,et al.Spot-the-difference self-supervised pre-training for anomaly detection and segmentation[C]//European Conference on Computer Vision.Springer,2022:392-408.
[3]FERNANDO T,GAMMULLE H,DENMAN S,et al.Deeplearning for medical anomaly detection－a survey[J].ACM Computing Surveys(CSUR),2021,54(7):1-37.
[4]DEFARD T,SETKOV A,LOESCH A,et al.Padim:a patch distribution modeling framework for anomaly detection and localization[C]//International Conference on Pattern Recognition.Springer,2021:475-489.
[5]GUDOVSKIY D,ISHIZAKA S,KOZUKA K.Cflow-ad:Real-time unsupervised anomaly detection with localization via conditional normalizing flows[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2022:98-107.
[6]LEI J,HU X,WANG Y,et al.Pyramidflow:High-resolution defect contrastive localization using pyramid normalizing flow[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:14143-14152.
[7]RIPPEL O,MERTENS P,MERHOF D.Modeling the distribution of normal data in pre-trained deep features for anomaly detection[C]//2020 25th International Conference on Pattern Recognition(ICPR).IEEE,2021:6726-6733.
[8]ROTH K,PEMULA L,ZEPEDA J,et al.Towards total recall in industrial anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:14318-14328.
[9]YAN X,ZHANG H,XU X,et al.Learning semantic contextfrom normal samples for unsupervised anomaly detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence:Vol.35.2021:3110-3118.
[10]YAO X,LI R,QIAN Z,et al.Focus the discrepancy:Intra-and inter-correlation learning for image anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:6803-6813.
[11]YOU Z,YANG K,LUO W,et al.Adtr:Anomaly detectiontransformer with feature reconstruction[C]//International Conference on Neural Information Processing.Springer,2022:298-310.
[12]ZAVRTANIK V,KRISTAN M,SKOCˇAJ D.Draem－a discriminatively trained reconstruction embedding for surface anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:8330-8339.
[13]ZHANG H,WANG Z,WU Z,et al.Diffusionad:Denoising diffusion for anomaly detection[J].arXiv:2303.08730,2023.
[14]BERGMANN P,FAUSER M,SATTLEGGER D,et al.Unin-formed students:Student-teacher anomaly detection with discriminative latent embeddings[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:4183-4192.
[15]DENG H,LI X.Anomaly detection via reverse distillation from one-class embedding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:9737-9746.
[16]SALEHI M,SADJADI N,BASELIZADEH S,et al.Multiresolution knowledge distillation for anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14902-14912.
[17]TIEN T D,NGUYEN A T,TRAN N H,et al.Revisiting reverse distillation for anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:24511-24520.
[18]YOU Z,CUI L,SHEN Y,et al.A unified model for multi-class anomaly detection[J].Advances in Neural Information Proces-sing Systems,2022,35:4571-4584.
[19]ZHAO Y.Omnial:A unified cnn framework for unsupervisedanomaly localization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:3924-3933.
[20]MASOUDNIA S,EBRAHIMPOUR R.Mixture of experts:a li-terature survey[J].Artificial Intelligence Review,2014,42:275-293.
[21]RUDOLPH M,WANDT B,ROSENHAHN B.Same same butdiffernet:Semi-supervised defect detection with normalizing flows[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2021:1907-1916.
[22]RUDOLPH M,WEHRBEIN T,ROSENHAHN B,et al.Fully convolutional cross-scale-flows for image-based defect detection[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2022:1088-1097.
[23]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial networks[J].Communications of the ACM,2020,63(11):139-144.
[24]HO J,JAIN A,ABBEEL P.Denoising diffusion probabilisticmodels[J].Advances in Neural Information Processing Systems,2020,33:6840-6851.
[25]GU Z,LIU L,CHEN X,et al.Remembering normality:Memory-guided knowledge distillation for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:16401-16409.
[26]DENG J,DONG W,SOCHER R,et al.Imagenet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2009:248-255.
[27]ZAGORUYKO S,KOMODAKIS N.Wide residual networks[J].arXiv:1605.07146,2016.
[28]TAN M,LE Q.Efficientnet:Rethinking model scaling for con-volutional neural networks[C]//International Conference on Machine Learning.PMLR,2019:6105-6114.
[29]CAI Z,VASCONCELOS N.Cascade r-cnn:Delving into highquality object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2018:6154-6162.
[30]LIU Z,ZHOU Y,XU Y,et al.Simplenet:A simple network for image anomaly detection and localization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:20402-20411.
[31]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发