Computer Science ›› 2023, Vol. 50 ›› Issue (11): 151-159.doi: 10.11896/jsjkx.221100023

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Surface Anomaly Detection Based on Image Reconstruction and Semantic Difference Discrimination

WANG Shangshang, JIN Cheng   

  1. School of Computer Science,Fudan University,Shanghai 200438,China
  • Received:2022-11-03 Revised:2023-03-16 Online:2023-11-15 Published:2023-11-06
  • About author:WANG Shangshang,born in 1997,postgraduate.His main research interests include unsupervised anomaly detection and localization.JIN Cheng,born in 1978,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include computer vision and multimedia information retrieval.
  • Supported by:
    National Key Research and Development Program of China(2019YFB2102800).

Abstract: Reconstruction-based methods are widely used for surface anomaly detection.These methods are expected to only reconstruct normal patterns well and detect and localize anomalies by the larger reconstruction error in anomalous areas.Previous methods either tend to “generalize” too well,resulting in high fidelity reconstruction of anomalies,or measure reconstruction differences in image space,which doesn’t really capture the semantic differences.To tackle these problems,this paper proposes a model consisting of a reconstruction network and a discrimination network.In the reconstruction network,we design a multiscale location-augmented dynamic prototype unit to reinforce the learning of normal patterns.In the discrimination network,we fuse the multiscale deep features of the input image and its anomaly-free reconstruction to utilize the multiscale semantic difference information before and after reconstruction,which reinforces the discrimination of semantic differences.On the MVTec dataset,our method reaches 99.5% AUROC in the detection task,and 98.5% AUROC,95.0% PRO in the location task,outperforms pre-vious reconstruction-based methods by a large margin.

Key words: Image reconstruction, Surface anomaly detection, Multiscale location-augmented dynamic prototype unit, Semantic difference discrimination

CLC Number: 

  • TP391
[1]BERGMANN P,FAUSER M,SATTLEGGER D,et al.MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2020.
[2]SCHLEGL T,SEEBCK P,WALDSTEIN S M,et al.Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery [C]//International Conference on Information Processing in Medical Imaging.Cham:Springer,2017:146-157.
[3]ZAVRTANIK V,KRISTAN M,SKOAJ D.DR{\AEM-A discriminatively trained reconstruction embedding for surface anomaly detection[C]//IEEE/CVF International Conference on Computer Vision.2021:8330- 8339.
[4]GONG D,LIU L,LE V,et al.Memorizing Normality to Detect Anomaly:Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).IEEE,2020.
[5]PARK H,NOH J,HAM B.Learning Memory-guided Normality for Anomaly Detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2020.
[6]ZAVRTANIK V,KRISTAN M,SKOAJ D.Reconstruction by inpainting for visual anomaly detection[J].Pattern Recognition,2021,112:107706
[7]MEI S,YANG H,YIN Z.An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces[J].IEEE Transactions on Instrumentation and Measurement,2018,67(6):1266-1277.
[8]PARK C,CHO M A,LEE M,et al.FastAno:Fast Anomaly Detection via Spatio-temporal Patch Transformation[C]//IEEE/CVF Winter Conference on Applications of Computer Vision.2022:2249-2259.
[9]SABOKROU M,KHALOOEI M,FATHY M,et al.Adversa-rially Learned One-Class Classifier for Novelty Detection[C]//IEEE/CVF Conference on Computer Vision & Pattern Recognition.IEEE,2018.
[10]BERGMANN P,LWE S,FAUSER M,et al.Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders[J].arXiv:1807.02011,2018.
[11]YAN X,ZHANG H,XU X,et al.Learning Semantic Context from Normal Samples for Unsupervised Anomaly Detection[C]//National Conference on Artificial Intelligence.2021.
[12]LI Z,LI N,JIANG K,et al.Superpixel Masking and Inpainting for Self-Supervised Anomaly Detection[C]//British Machine Vision Conference.2020.
[13]AKCAY S,ATAPOUR-ABARGHOUEI A,BRECKON T P.GANomaly:semi-supervised anomaly detection via adversarial training [C]//Asian Conference on Computer Vision.Cham:Springer,2018:622-637.
[14]AKAY S,ATAPOUR-ABARGHOUEI A,BRECKON T P.Skip-GANomaly:Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection[C]//2019 International Joint Conference on Neural Networks(IJCNN).IEEE,2019.
[15]POURREZA M,MOHAMMADI B,KHAKI M,et al.G2D:Generate to Detect Anomaly[C]//Workshop on Applications of Computer Vision.IEEE,2021.
[16]SCHLEGL T,SEEBCK P,WALDSTEIN S M,et al.f-Ano-GAN:Fast Unsupervised Anomaly Detection with Generative Adversarial Networks[J].Medical Image Analysis,2019,54:30-44.
[17]COLLIN A S,VLEESCHOUWER C D.Improved anomaly detection by training an autoencoder with skip connections on images corrupted with Stain-shaped noise[C]//25th International Conference on Pattern Recognition(ICPR).IEEE,2021:7915-7922.
[18]ZHAO Z,LI B,DONG R,et al.A surface defect detection me-thod based on positive samples[C]//Pacific RIM International Conference on Artificial Intelligence.2018:473-481.
[19]HOU J,ZHANG Y,ZHONG Q,et al.Divide-and-Assemble:Learning Block-wise Memory for Unsupervised Anomaly Detection[C]//IEEE/CVF International Conference on Computer Vision.2021:8791-8800.
[20]LV H,CHEN C,CUI Z,et al.Learning Normal Dynamics inVideos with Meta Prototype Network[C]//Computer Vision and Pattern Recognition.2021.
[21]DENG J,DONG W,SOCHER R,et al.ImageNet:a Large-Scale Hierarchical Image Database[C]//2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR 2009).IEEE,2009:248-255.
[22]RIPPEL O,MERTENS P,MERHOF D.Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection[C]//2020 25th International Conference on Pattern Recognition(ICPR).IEEE,2021:6726-6733.
[23]DEFARD T,SETKOV A,LOESCH A,et al.PaDiM:A Patch Distribution Modeling Framework for Anomaly Detection and Localization[C]//International Conference on Pattern Recognition.Springer,2021:475-489.
[24]RUDOLPH M,WANDT B,ROSENHAHN B.Same Same But DifferNet:Semi-Supervised Defect Detection with Normalizing Flows[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2021:1907-1916.
[25]GUDOVSKIY D,ISHIZAKA S,KOZUKA K.CFLOW-AD:Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows[C]//IEEE/CVF Winter Confe-rence on Applications of Computer Vision.2022:98-107.
[26]REZENDE D J,MOHAMED S.Variational Inference with Normalizing Flows[C]//International Conference on Machine Learning.JMLR.org,2015.
[27]BERGMANN P,FAUSER M,SATTLEGGER D,et al.Unin-formed Students:Student-Teacher Anomaly Detection with Discriminative Latent Embeddings[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:4183-4192.
[28]SALEHI M,SADJADI N,BASELIZADEH S,et al.Multiresolution Knowledge Distillation for Anomaly Detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14902-14912.
[29]DENG H,LI X.Anomaly Detection via Reverse Distillationfrom One-Class Embedding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:9737-9746.
[30]ROTH K,PEMULA L,ZEPEDA J,et al.Towards Total Recall in Industrial Anomaly Detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:14318-14328.
[31]COHEN N,HOSHEN Y.Sub-Image Anomaly Detection withDeep Pyramid Correspondences[J].arXiv:2005.02357,2020.
[32]REISS T,COHEN N,BERGMAN L,et al.PANDA:Adapting Pretrained Features for Anomaly Detection and Segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:2806-2814.
[33]TSAI C,WU T,LAI S,et al.Multi-scale patch-based representation learning for image anomaly detection and segmentation[C]//IEEE/CVF Winter Conference on Applications of Computer Vision.2022:3065-3073.
[34]RACKI D,TOMAZEVIC D,SKOCAJ D.A Compact Convolu-tional Neural Network for Textured Surface Anomaly Detection[C]//2018 IEEE Winter Conference on Applications of Computer Vision(WACV).IEEE,2018:1331-1339.
[35]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional Networks for Biomedical Image Segmentation[C]//International Conference on Medical Image Computing and Compu-ter-Assisted Intervention.Springer International Publishing,2015.
[36]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal Loss for Dense Object Detection[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017:2980-2988.
[37]WIELER M,HAHN T.Weakly supervised learning for industrial optical inspection [C]//DAGM Symposium.2007.
[38]CIMPOI M,MAJI S,KOKKINOS I,et al.Describing Textures in the Wild[C]//IEEE Conference on Computer Vision and Pattern Recognition.2014:3606-3613.
[39]SHI Y,YANG J,QI Z.Unsupervised anomaly segmentation via deep feature reconstruction[J].Neurocomputing,2020,424(11):9-22.
[1] JIN Yahui, JIANG Murong, LI Fuhai, YANG Lei, CHEN Junyi. Combining Multi-focus Fusion and DSGEF Two-stage Network to Reconstruct Solar Speckle Image [J]. Computer Science, 2023, 50(6A): 220600182-6.
[2] HUANG Yaqun, ZHENG Peiyu, JIANG Murong, YANG Lei, LUO Jun. Reconstruction of Solar Speckle Image Combined with Gated Fusion Network and Residual Fourier Transform [J]. Computer Science, 2023, 50(11A): 220800229-7.
[3] SUN Zheng, ZHANG Xiao-xue. Review on Methods of Reducing Acoustic Reflection Artifact in Biological Photoacoustic Imaging [J]. Computer Science, 2021, 48(6A): 10-14.
[4] SUN Zheng and WANG Xin-yu. Application of Deep Learning in Photoacoustic Imaging [J]. Computer Science, 2020, 47(6A): 148-152.
[5] LAN Zhang-li, SHEN De-xing, CAO Juan and ZHANG Yu-xin. Content-independent Method for Basis Image Extraction and Image Reconstruction [J]. Computer Science, 2020, 47(6A): 226-229.
[6] ZHANG Sai, RUI Ting, REN Tong-wei, YANG Cheng-song, ZOU Jun-hua. Image Reconstruction Based on Supervised Learning Deep Auto-encoder [J]. Computer Science, 2018, 45(11): 267-271.
[7] LIU Ya-nan, YANG Xiao-mei and CHEN Chao-nan. Super-resolution Image Reconstruction Based on Fractional Order Total Variation Regularization [J]. Computer Science, 2016, 43(5): 274-278.
[8] LV Li-zhi and QIANG Yan. Medical CT Image Enhancement Algorithm Based on Laplacian Pyramid and Wavelet Transform [J]. Computer Science, 2016, 43(11): 300-303.
[9] CHEN Xiu-mei, WANG Jing-shi, WANG Wei, ZHAO Yang and TANG Min. Compressed Sensing Reconstruction of MRI Images Based on Nonsubsampled Contourlet Transform [J]. Computer Science, 2015, 42(11): 299-304.
[10] LUO Yan-hong. Study on Exponential Filter of Filter Back Projection Algorithm for CT Image Reconstruction [J]. Computer Science, 2014, 41(Z6): 220-223.
[11] CHENG Li-jun,ZHANG Yu-bo and XU Cong-fu. Image Reconstruction Algorithm Based on Graph Cuts and Gradient-based Algorithms [J]. Computer Science, 2014, 41(4): 314-318.
[12] . Image Reconstruction Method Using Smoothed-to and Dual-tree Wavelet Transform [J]. Computer Science, 2013, 40(3): 263-265.
[13] . Fast Calculation of Simulation Projection for Cone-beam CT Based on Shepp-Logan Head Phantom [J]. Computer Science, 2012, 39(5): 257-260.
[14] ZHANG Shun-li,ZHANG Ding-hua,CHENG Yun-yong,ZHANG Xiao-bo. Fast Calculation of CT Simulation Projection Based on Pixel Phantom [J]. Computer Science, 2011, 38(7): 290-293.
[15] ZHANG Yan-jun,CHEN Yu,CHEN De-yun,YU Xiao-yang. Analyze of Sensitivity Field Characteristics and Image Reconstruction for Electrical Resistance Tomography System [J]. Computer Science, 2010, 37(8): 257-261.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!