计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (6): 135-143.doi: 10.11896/jsjkx.230300194

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于混合高斯先验变分自编码器的深度多球支持向量数据描述

武慧囡1, 邢红杰1, 李刚2,3   

  1. 1 河北大学数学与信息科学学院河北省机器学习与计算智能重点实验室 河北 保定 071002
    2 华北电力大学计算机系 河北 保定 071003
    3 复杂能源系统智能计算教育部工程研究中心 河北 保定 071003
  • 收稿日期:2023-03-24 修回日期:2023-08-12 出版日期:2024-06-15 发布日期:2024-06-05
  • 通讯作者: 邢红杰(hjxing@hbu.edu.cn)
  • 作者简介:(18731259735@163.com)
  • 基金资助:
    国家自然科学基金(61672205);河北省自然科学基金(F2017201020);河北大学高层次人才科研启动项目(521100222002);复杂能源系统智能计算教育部工程研究中心开放基金(ESIC202101)

Deep Multiple-sphere Support Vector Data Description Based on Variational Autoencoder with Mixture-of-Gaussians Prior

WU Huinan1, XING Hongjie1, LI Gang2,3   

  1. 1 Hebei Key Laboratory of Machine Learning and Computational Intelligence,College of Mathematics and Information Science,Hebei University,Baoding,Hebei 071002,China
    2 Department of Computer,North China Electric Power University,Baoding,Hebei 071003,China
    3 Engineering Research Center of Intelligent Computing for Complex Energy Systems,Baoding,Hebei 071003,China
  • Received:2023-03-24 Revised:2023-08-12 Online:2024-06-15 Published:2024-06-05
  • About author:WU Huinan,born in 1997,postgra-duate.Her main research interests include anomaly detection,variational autoencoder and deep learning.
    XING Hongjie,Ph.D,professor,Ph.D supervisor.His main research interests include kernel methods,neural networks,novelty detection,and ensemble learning.
  • Supported by:
    National Natural Science Foundation of China(61672205),Natural Science Foundation of Hebei Province,China(F2017201020),High-Level Talents Research Start-Up Project of Hebei University(521100222002) and Open Foundation of Engineering Research Center of Intelligent Computing for Complex Energy Systems(ESIC202101).

PDF (PC)

513

摘要： 随着数据维度和规模的不断增加,基于深度学习的异常检测方法取得了优异的检测性能,其中深度支持向量数据描述(Deep SVDD)得到了广泛应用。然而,要缓解超球崩溃问题,就需要对Deep SVDD中映射网络的各种参数施加约束。为了进一步提高Deep SVDD中映射网络的特征学习能力,同时解决超球崩溃问题,提出了基于混合高斯先验变分自编码器的深度多球支持向量数据描述(Deep Multiple-Sphere Support Vector Data Description Based on Variational Autoencoder with Mixture-of-Gaussians Prior,DMSVDD-VAE-MoG)。首先,通过预训练初始化网络参数和多个超球中心;其次,利用映射网络获得训练数据的潜在特征,对VAE损失、多个超球的平均半径和潜在特征到所对应超球中心的平均距离进行联合优化,以获得最优网络连接权重和多个最小超球。实验结果表明,所提DMSVDD-VAE-MoG在MNIST,Fashion-MNIST和CIFAR-10上均取得了优于其他8种相关方法的检测性能。

关键词: 深度支持向量数据描述, 混合高斯先验, 变分自编码器, 异常检测, 超球崩溃

Abstract: With the continuous increase of data dimension and scale,anomaly detection methods based on deep learning have achieved excellent detection performance,among which deep support vector data description(Deep SVDD) has been widely used.However,it is necessary to impose constraints on various parameters of the mapping network in Deep SVDD to alleviate the hypersphere collapse problem.In order to further improve the feature learning ability of the mapping network in Deep SVDD and solve the hypersphere collapse problem,deep multiple-sphere support vector data description based on variational autoencoder with mixture-of-gaussians prior(DMSVDD-VAE-MoG) is proposed.First,the network parameters and multiple hypersphere centers are initialized by pre-training.Second,the latent features of the training data are obtained by mapping network.The VAE loss,the average radius of multiple hyperspheres together with the average distance between the latent features and their corres-ponding hypersphere centers are jointly optimized to obtain the optimal network connection weights and multiple minimum hyperspheres.In comparison with the other eight related methods,the experimental results show that the proposed DMSVDD-VAE-MoG achieves better detection performance upon MNIST,Fashion-MNIST and CIFAR-10.

Key words: Deep support vector data description, Mixture-of-Gaussians prior, Variational autoencoder, Anomaly detection, Hypersphere collapse

中图分类号: 

  • TP391.4
[1]CHANDOLA V,BANERJEE A,KUMAR V.Anomaly Detection:A Survey[J].ACM Computing Surveys,2009,41(3):1-58.
[2]AGGARWAL C.An Introduction to Outlier Analysis[M]//Outlier Analysis.Cham:Springer,2017:1-34.
[3]NEUSCHMIED H,WINTER M,HOFER-SCHMITZ K,et al.Two Stage Anomaly Detection for Network Intrusion Detection[C]//International Conference on Information Systems Security and Privacy.Online Streaming:SCITEPRESS,2021:450-457.
[4]POURHABIBI T,ONG K L,KAM B H,et al.Fraud Detection:A Systematic Literature Review of Graph-Based Anomaly Detection Approaches[J].Decision Support Systems,2020,133:113303.
[5]CHEN D,WANG P,YUE L,et al.Anomaly Detection in Surveillance Video Based on Bidirectional Prediction[J].Image and Vision Computing,2020,98:103915.
[6]NAKAO T,HANAOKA S,NOMURA Y,et al.UnsupervisedDeep Anomaly Detection in Chest Radiographs[J].Journal of Digital Imaging,2021,34(2):418-427.
[7]SCHÖLKOPF B,WILLIAMSON R C,SMOLA A,et al.Support Vector Method for Novelty Detection[C]//International Conference on Neural Information Processing Systems.Cambridge:MIT Press,1999:582-588.
[8]PARZEN E.On Estimation of a Probability Density Functionand Mode[J].The Annals of Mathematical Statistics,1962,33(3):1065-1076.
[9]TAX D M J,DUIN R P W.Support Vector Data Description[J].Machine Learning,2004,54(1):45-66.
[10]HAWKINS S,HE H,WILLIAMS G,et al.Outlier DetectionUsing Replicator Neural Networks[C]//Data Warehousing and Knowledge Discovery.Berlin:Springer,2002:170-180.
[11]AN J,CHO S.Variational Autoencoder Based Anomaly Detection Using Reconstruction Probability[J].Special Lecture on IE,2015,2(1):1-18.
[12]ERFANI S M,RAJASEGARAR S,KARUNASEKERA S,et al.High-Dimensional and Large-Scale Anomaly Detection Using a Linear One-Class SVM with Deep Learning[J].Pattern Recognition,2016,58:121-134.
[13]RUFF L,VANDERMEULEN R,GOERNITZ N,et al.DeepOne-Class Classification[C]//International Conference on Machine Learning.Stockholm:ACM,2018,4393-4402.
[14]CHALAPATHY R,MENON A K,CHAWLA S.Anomaly Detection Using One-Class Neural Networks[J].arXiv:1802.06360,2018.
[15]ZHANG Z,DENG X.Anomaly Detection Using Improved Deep SVDD Model with Data Structure Preservation[J].Pattern Re-cognition Letters,2021,148:1-6.
[16]GHAFOORI Z,LECKIE C.Deep Multi-Sphere Support Vector Data Description[C]//SIAM International Conference on Data Mining.Cincinnati:SIAM,2020:109-117.
[17]ZHOU Y,LIANG X,ZHANG W,et al.VAE-Based Deep SVDD for Anomaly Detection[J].Neurocomputing,2021,453:131-140.
[18]HU T,GUO Q,SUN H,et al.Nontechnical Losses Detectionthrough Coordinated BiWGAN and SVDD[J].IEEE Transactions on Neural Networks and Learning Systems,2021,32(5),1866-1880.
[19]WU D,DENG Y,LI M.FL-MGVN:Federated Learning forAnomaly Detection Using Mixed Gaussian Variational Self-Encoding Network[J].Information Processing & Management,2022,59(2):102839.
[20]BISHOP C M.Pattern Recognition and Machine Learning[M].New York:springer,2006.
[21]JIANG Z,ZHENG Y,TAN H,et al.Variational Deep Embedding:An Unsupervised and Generative Approach to Clustering[C]//International Joint Conference on Artificial Intelligence.Melbourne:Morgan Kaufmann,2017:1965-1972.
[22]ZONG B,SONG Q,MIN M R,et al.Deep Autoencoding Gaus-sian Mixture Model for Unsupervised Anomaly Detection[C]//International Conference on Learning Representations.Vancouver:IEEE,2018:781-795.
[23]YANG L,FAN W,BOUGUILA N.Clustering Analysis ViaDeep Generative Models with Mixture Models[J].IEEE Transac-tions on Neural Networks and Learning Systems,2022,33(1):340-350.
[24]KINGMA D P,WELLING M.Auto-Encoding Variational Bayes[C]//International Conference on Learning Representations.Banff:IEEE,2014:1-5.
[25]LLOYD S.Least Squares Quantization in PCM[J].IEEE Transac-tions on Information Theory,1982,28(2):129-137.
[26]KINGMA D P,BA J.Adam:A Method for Stochastic Optimization[C]//International Conference on Learning Representations.San Diego:IEEE,2015:1-15.
[27]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-BasedLearning Applied to Document Recognition[J].IEEE,1998,86(11):2278-2324.
[28]XIAO H,RASUL K,VOLLGRAF R.Fashion-mnist:A Novel Image Dataset for Benchmarking Machine Learning Algorithms[J].arXiv:1708.07747,2017.
[29]KRIZHEVSKY A,HINTON G.Learning Multiple Layers ofFeatures from Tiny Images[J/OL].Handbook of Systemic Autoimmune Diseases,2009,1(4).https://scholar.google.com/scholar?hl=zhCN&as_sdt=0%2C5&q=Learning+multiple+layers+of+features+from+tiny+image&btnG=.
[30]FAWCETT T.An Introduction to ROC Analysis[J].PatternRecognition Letters,2006,27(8):861-874.
[31]LIU F T,TING K M,ZHOU Z H.Isolation Forest[C]//IEEE International Conference on Data Mining.Pisa:IEEE,2008:413-422.
[32]MASCI J,MEIER U,CIRESAN D,et al.Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction[C]//International Conference on Artificial Neural Networks.Espoo:Springer,2011:52-59.
[33]LI D,TAO Q,LIU J,et al.Center-Aware Adversarial Auto-encoder for Anomaly Detection[J].IEEE Transactions on Neural Networks and Learning Systems,2022,33(6):2480-2493.
[34]YANG Z,ZHANG T,BOZCHALOOI I S,et al.Memory-Augmented Generative Adversarial Networks for Anomaly Detection[J].IEEE Transactions on Neural Networks and Learning Systems,2022,33(6):2324-2334.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发