Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (6): 135-143.doi: 10.11896/jsjkx.230300194

• Database & Big Data & Data Science • Previous Articles     Next Articles

Deep Multiple-sphere Support Vector Data Description Based on Variational Autoencoder with Mixture-of-Gaussians Prior

WU Huinan1, XING Hongjie1, LI Gang2,3   

  1. 1 Hebei Key Laboratory of Machine Learning and Computational Intelligence,College of Mathematics and Information Science,Hebei University,Baoding,Hebei 071002,China
    2 Department of Computer,North China Electric Power University,Baoding,Hebei 071003,China
    3 Engineering Research Center of Intelligent Computing for Complex Energy Systems,Baoding,Hebei 071003,China
  • Received:2023-03-24 Revised:2023-08-12 Online:2024-06-15 Published:2024-06-05
  • About author:WU Huinan,born in 1997,postgra-duate.Her main research interests include anomaly detection,variational autoencoder and deep learning.
    XING Hongjie,Ph.D,professor,Ph.D supervisor.His main research interests include kernel methods,neural networks,novelty detection,and ensemble learning.
  • Supported by:
    National Natural Science Foundation of China(61672205),Natural Science Foundation of Hebei Province,China(F2017201020),High-Level Talents Research Start-Up Project of Hebei University(521100222002) and Open Foundation of Engineering Research Center of Intelligent Computing for Complex Energy Systems(ESIC202101).

PDF (PC)

487

Abstract: With the continuous increase of data dimension and scale,anomaly detection methods based on deep learning have achieved excellent detection performance,among which deep support vector data description(Deep SVDD) has been widely used.However,it is necessary to impose constraints on various parameters of the mapping network in Deep SVDD to alleviate the hypersphere collapse problem.In order to further improve the feature learning ability of the mapping network in Deep SVDD and solve the hypersphere collapse problem,deep multiple-sphere support vector data description based on variational autoencoder with mixture-of-gaussians prior(DMSVDD-VAE-MoG) is proposed.First,the network parameters and multiple hypersphere centers are initialized by pre-training.Second,the latent features of the training data are obtained by mapping network.The VAE loss,the average radius of multiple hyperspheres together with the average distance between the latent features and their corres-ponding hypersphere centers are jointly optimized to obtain the optimal network connection weights and multiple minimum hyperspheres.In comparison with the other eight related methods,the experimental results show that the proposed DMSVDD-VAE-MoG achieves better detection performance upon MNIST,Fashion-MNIST and CIFAR-10.

Key words: Deep support vector data description, Mixture-of-Gaussians prior, Variational autoencoder, Anomaly detection, Hypersphere collapse

CLC Number: 

  • TP391.4
[1]CHANDOLA V,BANERJEE A,KUMAR V.Anomaly Detection:A Survey[J].ACM Computing Surveys,2009,41(3):1-58.
[2]AGGARWAL C.An Introduction to Outlier Analysis[M]//Outlier Analysis.Cham:Springer,2017:1-34.
[3]NEUSCHMIED H,WINTER M,HOFER-SCHMITZ K,et al.Two Stage Anomaly Detection for Network Intrusion Detection[C]//International Conference on Information Systems Security and Privacy.Online Streaming:SCITEPRESS,2021:450-457.
[4]POURHABIBI T,ONG K L,KAM B H,et al.Fraud Detection:A Systematic Literature Review of Graph-Based Anomaly Detection Approaches[J].Decision Support Systems,2020,133:113303.
[5]CHEN D,WANG P,YUE L,et al.Anomaly Detection in Surveillance Video Based on Bidirectional Prediction[J].Image and Vision Computing,2020,98:103915.
[6]NAKAO T,HANAOKA S,NOMURA Y,et al.UnsupervisedDeep Anomaly Detection in Chest Radiographs[J].Journal of Digital Imaging,2021,34(2):418-427.
[7]SCHÖLKOPF B,WILLIAMSON R C,SMOLA A,et al.Support Vector Method for Novelty Detection[C]//International Conference on Neural Information Processing Systems.Cambridge:MIT Press,1999:582-588.
[8]PARZEN E.On Estimation of a Probability Density Functionand Mode[J].The Annals of Mathematical Statistics,1962,33(3):1065-1076.
[9]TAX D M J,DUIN R P W.Support Vector Data Description[J].Machine Learning,2004,54(1):45-66.
[10]HAWKINS S,HE H,WILLIAMS G,et al.Outlier DetectionUsing Replicator Neural Networks[C]//Data Warehousing and Knowledge Discovery.Berlin:Springer,2002:170-180.
[11]AN J,CHO S.Variational Autoencoder Based Anomaly Detection Using Reconstruction Probability[J].Special Lecture on IE,2015,2(1):1-18.
[12]ERFANI S M,RAJASEGARAR S,KARUNASEKERA S,et al.High-Dimensional and Large-Scale Anomaly Detection Using a Linear One-Class SVM with Deep Learning[J].Pattern Recognition,2016,58:121-134.
[13]RUFF L,VANDERMEULEN R,GOERNITZ N,et al.DeepOne-Class Classification[C]//International Conference on Machine Learning.Stockholm:ACM,2018,4393-4402.
[14]CHALAPATHY R,MENON A K,CHAWLA S.Anomaly Detection Using One-Class Neural Networks[J].arXiv:1802.06360,2018.
[15]ZHANG Z,DENG X.Anomaly Detection Using Improved Deep SVDD Model with Data Structure Preservation[J].Pattern Re-cognition Letters,2021,148:1-6.
[16]GHAFOORI Z,LECKIE C.Deep Multi-Sphere Support Vector Data Description[C]//SIAM International Conference on Data Mining.Cincinnati:SIAM,2020:109-117.
[17]ZHOU Y,LIANG X,ZHANG W,et al.VAE-Based Deep SVDD for Anomaly Detection[J].Neurocomputing,2021,453:131-140.
[18]HU T,GUO Q,SUN H,et al.Nontechnical Losses Detectionthrough Coordinated BiWGAN and SVDD[J].IEEE Transactions on Neural Networks and Learning Systems,2021,32(5),1866-1880.
[19]WU D,DENG Y,LI M.FL-MGVN:Federated Learning forAnomaly Detection Using Mixed Gaussian Variational Self-Encoding Network[J].Information Processing & Management,2022,59(2):102839.
[20]BISHOP C M.Pattern Recognition and Machine Learning[M].New York:springer,2006.
[21]JIANG Z,ZHENG Y,TAN H,et al.Variational Deep Embedding:An Unsupervised and Generative Approach to Clustering[C]//International Joint Conference on Artificial Intelligence.Melbourne:Morgan Kaufmann,2017:1965-1972.
[22]ZONG B,SONG Q,MIN M R,et al.Deep Autoencoding Gaus-sian Mixture Model for Unsupervised Anomaly Detection[C]//International Conference on Learning Representations.Vancouver:IEEE,2018:781-795.
[23]YANG L,FAN W,BOUGUILA N.Clustering Analysis ViaDeep Generative Models with Mixture Models[J].IEEE Transac-tions on Neural Networks and Learning Systems,2022,33(1):340-350.
[24]KINGMA D P,WELLING M.Auto-Encoding Variational Bayes[C]//International Conference on Learning Representations.Banff:IEEE,2014:1-5.
[25]LLOYD S.Least Squares Quantization in PCM[J].IEEE Transac-tions on Information Theory,1982,28(2):129-137.
[26]KINGMA D P,BA J.Adam:A Method for Stochastic Optimization[C]//International Conference on Learning Representations.San Diego:IEEE,2015:1-15.
[27]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-BasedLearning Applied to Document Recognition[J].IEEE,1998,86(11):2278-2324.
[28]XIAO H,RASUL K,VOLLGRAF R.Fashion-mnist:A Novel Image Dataset for Benchmarking Machine Learning Algorithms[J].arXiv:1708.07747,2017.
[29]KRIZHEVSKY A,HINTON G.Learning Multiple Layers ofFeatures from Tiny Images[J/OL].Handbook of Systemic Autoimmune Diseases,2009,1(4).https://scholar.google.com/scholar?hl=zhCN&as_sdt=0%2C5&q=Learning+multiple+layers+of+features+from+tiny+image&btnG=.
[30]FAWCETT T.An Introduction to ROC Analysis[J].PatternRecognition Letters,2006,27(8):861-874.
[31]LIU F T,TING K M,ZHOU Z H.Isolation Forest[C]//IEEE International Conference on Data Mining.Pisa:IEEE,2008:413-422.
[32]MASCI J,MEIER U,CIRESAN D,et al.Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction[C]//International Conference on Artificial Neural Networks.Espoo:Springer,2011:52-59.
[33]LI D,TAO Q,LIU J,et al.Center-Aware Adversarial Auto-encoder for Anomaly Detection[J].IEEE Transactions on Neural Networks and Learning Systems,2022,33(6):2480-2493.
[34]YANG Z,ZHANG T,BOZCHALOOI I S,et al.Memory-Augmented Generative Adversarial Networks for Anomaly Detection[J].IEEE Transactions on Neural Networks and Learning Systems,2022,33(6):2324-2334.
[1] WU Nannan, GUO Zehao, ZHAO Yiming, YU Wei, SUN Ying, WANG Wenjun. Study on Anomalous Evolution Pattern on Temporal Networks [J]. Computer Science, 2024, 51(6): 118-127.
[2] LI Shasha, XING Hongjie. Robust Anomaly Detection Based on Adversarial Samples and AutoEncoder [J]. Computer Science, 2024, 51(5): 363-373.
[3] ZHOU Wenhao, HU Hongtao, CHEN Xu, ZHAO Chunhui. Weakly Supervised Video Anomaly Detection Based on Dual Dynamic Memory Network [J]. Computer Science, 2024, 51(1): 243-251.
[4] LI Hui, LI Wengen, GUAN Jihong. Dually Encoded Semi-supervised Anomaly Detection [J]. Computer Science, 2023, 50(7): 53-59.
[5] HENG Hongjun, ZHOU Wenhua. Anomaly Detection Method Based on Context Information Fusion and Noise Adaptation [J]. Computer Science, 2023, 50(7): 237-245.
[6] SUN Kaiwei, WANG Zhihao, LIU Hu, RAN Xue. Maximum Overlap Single Target Tracking Algorithm Based on Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220400023-5.
[7] ZHANG Guohua, YAN Xuefeng, GUAN Donghai. Anomaly Detection of Time-series Based on Multi-modal Feature Fusion [J]. Computer Science, 2023, 50(6A): 220700094-7.
[8] SUN Xuekui, DAI Hua, ZHOU Jianguo, YANG Geng, CHEN Yanli. LTTFAD:Log Template Topic Feature-based Anomaly Detection [J]. Computer Science, 2023, 50(6): 313-321.
[9] ZHAO Song, FU Hao, WANG Hongxing. Pseudo-abnormal Sample Selection for Video Anomaly Detection [J]. Computer Science, 2023, 50(5): 146-154.
[10] ZHANG Renbin, ZUO Yicong, ZHOU Zelin, WANG Long, CUI Yuhang. Multimodal Generative Adversarial Networks Based Multivariate Time Series Anomaly Detection [J]. Computer Science, 2023, 50(5): 355-362.
[11] RAO Dan, SHI Hongwei. Study on Air Traffic Flow Recognition and Anomaly Detection Based on Deep Clustering [J]. Computer Science, 2023, 50(3): 121-128.
[12] CUI Jingsong, ZHANG Tongtong, GUO Chi, GUO Wenfei. Network Equipment Anomaly Detection Based on Time Delay Feature [J]. Computer Science, 2023, 50(3): 371-379.
[13] LI Shasha, XING Hongjie, LI Gang. Multi-temporal Hyperspectral Anomaly Change Detection Based on Dual Space Conjugate Autoencoder [J]. Computer Science, 2023, 50(12): 175-184.
[14] ZHANG Zouquan, ZHANG Hui, WU Tianyue, CHEN Tiancai. Continuous Dense Normalized Flow Model for Anomaly Detection in Industrial Images [J]. Computer Science, 2023, 50(12): 212-220.
[15] LIANG Lifang, GUAN Donghai, ZHANG Ji, YUAN Weiwei. Spatial-Temporal Attention Mechanism Based Anomaly Detection for Multivariate Times Series [J]. Computer Science, 2023, 50(11A): 230300022-8.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.