计算机科学 ›› 2022, Vol. 49 ›› Issue (3): 144-151.doi: 10.11896/jsjkx.210100142

• 数据库&大数据&数据科学 • 上一篇    下一篇

单类支持向量机融合深度自编码器的异常检测模型

武玉坤, 李伟, 倪敏雅, 许志骋   

  1. 浙江工业大学计算机科学与技术学院 杭州310023
  • 收稿日期:2021-01-18 修回日期:2021-05-01 出版日期:2022-03-15 发布日期:2022-03-15
  • 作者简介:(051092296wyk@163.com)
  • 基金资助:
    国家自然科学基金(61502422,61972056);浙江省自然科学基金(LY18F020028);浙江省科技厅公益项目(2017C33108);浙江省教育厅一般科研项目(Y202044619)

Anomaly Detection Model Based on One-class Support Vector Machine Fused Deep Auto-encoder

WU Yu-kun, LI Wei, NI Min-ya, XU Zhi-cheng   

  1. College of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China
  • Received:2021-01-18 Revised:2021-05-01 Online:2022-03-15 Published:2022-03-15
  • About author:WU Yu-kun,born in 1980,Ph.D student,is a member of China Compu-ter Federation.His main research in-terests include machine learning and big data.
    LI Wei,born in 1958,Ph.D,professor.His main research interests include big data,block chain,IOT,and smart city development.
  • Supported by:
    National Natural Science Foundation of China(61502422,61972056),Natural Science Foundation of Zhejiang Province,China(LY18F020028),Public Welfare Project of Zhejiang Science and Technology Department(2017C33108) and General Research Project of Zhejiang Provincial Department of Education(Y202044619).

摘要: 大规模高维不平衡数据是异常检测中的重大挑战。单类支持向量机在处理不平衡数据方面非常有效,但不适合大规模高维数据,同时单类支持向量机的核函数对检测性能也具有重要的影响。文中提出了一个深度自编码器与单类支持向量机相结合的异常检测模型,深度自编码器不仅负责提取特征和降维,同时拟合出了一个自适应核函数。深度自编码器与单类支持向量机共享损失函数,实现了端到端的训练。作为一个整体,模型采用梯度下降法进行联合训练。在4个公开数据集上与其他异常检测方法进行了对比实验。实验结果表明,在AUC以及召回率(RECALL)方面,所提模型的性能优于单核和多核单类支持向量机以及其他模型,并且所提模型在不同异常率时是鲁棒的,在时间复杂度方面也具有非常大的优势。

关键词: 单类支持向量机, 混合模型, 深度自编码器, 异常检测

Abstract: Large-scale high-dimensional unbalanced data handling is a major challenge in anomaly detection.One-class support vector machine(OCSVM) is very efficient at handling unbalanced data,but it is not suitable for large-scale high-dimensional dataset.Meanwhile,the kernel function of OCSVM also has an important influence on the detection performance.An anomaly detection model combining a deep auto-encoder and a one-class support vector machine is proposed.The deep auto-encoder is not only responsible for extracting features and dimensionality reduction,but also mapping an adaptive kernel function.As a whole,the model adopts the gradient descent method to carry out joint training and realizes end-to-end training.Experiment is conducted on four public datasets and compared with other anomaly detection methods.Experimental results show that the proposed model has better performance than single-kernel or multi-kernel one-class support vector machines and other models in terms of AUC and RECALL,and the proposed model is robust at different anomaly rate and has great advantages in time complexity.

Key words: Anomaly detection, Deep auto-encoder, Hybrid model, One-class SVM

中图分类号: 

  • TP391
[1]TIAN Y J,MIRZABAGHERI M,BAMAKAN S M H,et al.Ramp loss one-class support vector machine:A robust and effective approach to anomaly detection problems[J].Neurocompu-ting,2018,310(6):223-235.
[2]CHANDOLA V,BANERJEE A,KUMAR V,et al.Anomalydetection:A survey[J].ACM Computing Surveys,2009,41(3):15:1-15:58.
[3]YE Q,YANG J,YIN T,et al.Can the virtual labels obtained by traditional LP approaches be well encoded in WLR[J].IEEE Transactions on Neural Networks and Learning Systems,2016,27(7):1591-1598.
[4]CANDES E J,LI X D,MA Y,et al.Robust principal component analysis[J].arXiv:0912,3599,2009.
[5]CHALAPATHY R,MENON A K,CHAWLA S,et al.Robust,deep and inductive anomaly detection[C]//Proceedings of Machine Learning and Knowledge Discovery in Databases.Skopje:Macedonia,2017:36-51.
[6]SCHÖLKOPF B,PLATT J C,TAYLOR J,et al.Estimating the support of a high-dimensional distribution[J].Neural Computation,2001,13 (7):1443-1471.
[7]TAX D M,DUIN R P.Support vector data description[J].Machine Learning,2004,54(1):45-66.
[8]LIU F T,TING K M,ZHOU Z H,et al.Isolation forest[C]//Proceedings of 2008 Eighth IEEE International Conference on Data Mining.Pisa,Italy:IEEE Computer Society,2008:413-422.
[9]KIM J,SCOTT C D.Robust kernel density estimation[J].Journal of Machine Learning Research,2012,13(9):2529-2565.
[10]ZIMEK A,SCHUBERT E,KRIEGEL H P,et al.A survey on unsupervised outlier detection in high-dimensional numerical data[J].Statistical Analysis and Data Mining,2012,5(5):363-387.
[11]GAUTAM C,BALAJI R,SUDHARSAN K,et al.LocalizedMultiple Kernel learning for Anomaly Detection:One-class Classification[J].Knowledge-Based Systems,2019,165(1):241-252.
[12]LI H Q,YING N,GUO C S,et al.High-dimensional outlier detection based on deep belief network and linear one-class SVM[J].Telecommunication Science,2018,34(1):34-42.
[13]JIN P,XIA X F,QIAO Y,et al.High-Dimensional Data Anomaly Detection for WSNs Based on Deep Belief Network[J].Chinese Journal of Sensors and Actuators,2019,32(6):892-901.
[14]ERFANI S M,RAJASEGARAR S,KARUNASEKERA S,et al.High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning[J].Pattern Recognition,2016,58(10):121-134.
[15]ZHOU C,PAFFENROTH R C.Anomaly detection with robust deep autoencoders[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,KDD17.New York:ACM,2017:665-674.
[16]XIE J Y,GIRSHICK R,FARHADI A,et al.Unsupervised deep embedding for clustering analysis[C]//Proceedings of the 33rd International Conference on Machine Learning.New York:JMLR.org,2016:478-487.
[17]GEHLER P V,NOWOZIN S.Infinite Kernel Learning[R].Tübingen,Max Planck Institute for Biological Cybernetics Technical Report,2008.
[18]BENGIO Y,LAMBLIN P,POPOVICI D,et al.Greedy layer-wise training of deep networks[C]//Proceedings of Advances in Neural Information Processing Systems 19(NIPS).Vancouver,BC,Canada:MIT Press,2007:153-160.
[19]CHALAPATHY R,MENON A K,CHAWLA S,et al.Anomaly detection using one-class neural networks[J].arXiv:1802.06360,2019.
[20]REVATHI S,MALATHI A.A detailed analysis on NSL-KDD dataset using various machine learning techniques for intrusion detection[J].International Journal of Engineering Research & Technology,2013,2(11):1848-1853.
[21]MOUSTAFA N,SLAY J.The evaluation of Network Anomaly Detection Systems:Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set[J].Information Systems Security,2016,25(1/2/3):18-31.
[22]SHARAFALDIN I,LASHKARI A H,GHORBANI A A,et al.Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization[C]//Proceedings of the 4th International Conference on Information Systems Security and Privacy.2018:108-116.
[23]PING R,ZHOU S S,LI D,et al.Cost Sensitive Random Forest Classification Algorithm for Highly Unbalanced Data[J].Pattern Recognition and Artificial Intelligence,2020,33(3):249-257.
[24]ALI R,BENJAMIN R.Random features for large-scale kernelmachines[C]//Proceedings of Advances in Neural Information Processing Systems 20.Vancouver,British Columbia,Canada:Curran Associates Inc,2008:1177-1184.
[25]SHARAFALDIN I,GHARIB A,LASHKARI A H,et al.Towards a reliable intrusion detection benchmark dataset[J].Software Networking,2018,2017(1):177-200.
[26]FAN J N,ZHANG Q R,ZHU J L,et al.Robust deep auto-encoding Gaussian process regression for unsupervised anomaly detection[J].Neurocomputing,2020,376(1):180-190.
[27]XAVIER G,BENGIO Y.Understanding the diffıculty of training deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artifıcial Intelligence and Statistics,Proceedings of Machine Learning Research.Chia Laguna Resort,Sardinia,Italy:Proceedings.mlr.press,2010:249-256.
[1] 徐天慧, 郭强, 张彩明.
基于全变分比分隔距离的时序数据异常检测
Time Series Data Anomaly Detection Based on Total Variation Ratio Separation Distance
计算机科学, 2022, 49(9): 101-110. https://doi.org/10.11896/jsjkx.210600174
[2] 李其烨, 邢红杰.
基于最大相关熵的KPCA异常检测方法
KPCA Based Novelty Detection Method Using Maximum Correntropy Criterion
计算机科学, 2022, 49(8): 267-272. https://doi.org/10.11896/jsjkx.210700175
[3] 王馨彤, 王璇, 孙知信.
基于多尺度记忆残差网络的网络流量异常检测模型
Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network
计算机科学, 2022, 49(8): 314-322. https://doi.org/10.11896/jsjkx.220200011
[4] 杜航原, 李铎, 王文剑.
一种面向电商网络的异常用户检测方法
Method for Abnormal Users Detection Oriented to E-commerce Network
计算机科学, 2022, 49(7): 170-178. https://doi.org/10.11896/jsjkx.210600092
[5] 冷佳旭, 谭明圮, 胡波, 高新波.
基于隐式视角转换的视频异常检测
Video Anomaly Detection Based on Implicit View Transformation
计算机科学, 2022, 49(2): 142-148. https://doi.org/10.11896/jsjkx.210900266
[6] 刘意, 毛莺池, 程杨堃, 高建, 王龙宝.
基于邻域一致性的异常检测序列集成方法
Locality and Consistency Based Sequential Ensemble Method for Outlier Detection
计算机科学, 2022, 49(1): 146-152. https://doi.org/10.11896/jsjkx.201000156
[7] 张叶, 李志华, 王长杰.
基于核密度估计的轻量级物联网异常流量检测方法
Kernel Density Estimation-based Lightweight IoT Anomaly Traffic Detection Method
计算机科学, 2021, 48(9): 337-344. https://doi.org/10.11896/jsjkx.200600108
[8] 郭奕杉, 刘漫丹.
基于时空轨迹数据的异常检测
Anomaly Detection Based on Spatial-temporal Trajectory Data
计算机科学, 2021, 48(6A): 213-219. https://doi.org/10.11896/jsjkx.201100193
[9] 邢红杰, 郝忠.
基于全局和局部判别对抗自编码器的异常检测方法
Novelty Detection Method Based on Global and Local Discriminative Adversarial Autoencoder
计算机科学, 2021, 48(6): 202-209. https://doi.org/10.11896/jsjkx.200400083
[10] 管文华, 林春雨, 杨尚蓉, 刘美琴, 赵耀.
基于人体关节点的低头异常行人检测
Detection of Head-bowing Abnormal Pedestrians Based on Human Joint Points
计算机科学, 2021, 48(5): 163-169. https://doi.org/10.11896/jsjkx.200800214
[11] 刘立成, 徐一凡, 谢贵才, 段磊.
面向NoSQL数据库的JSON文档异常检测与语义消歧模型
Outlier Detection and Semantic Disambiguation of JSON Document for NoSQL Database
计算机科学, 2021, 48(2): 93-99. https://doi.org/10.11896/jsjkx.200900039
[12] 邹承明, 陈德.
高维大数据分析的无监督异常检测方法
Unsupervised Anomaly Detection Method for High-dimensional Big Data Analysis
计算机科学, 2021, 48(2): 121-127. https://doi.org/10.11896/jsjkx.191100141
[13] 石琳姗, 马创, 杨云, 靳敏.
基于SSC-BP神经网络的异常检测算法
Anomaly Detection Algorithm Based on SSC-BP Neural Network
计算机科学, 2021, 48(12): 357-363. https://doi.org/10.11896/jsjkx.201000086
[14] 杨月麟, 毕宗泽.
基于深度学习的网络流量异常检测
Network Anomaly Detection Based on Deep Learning
计算机科学, 2021, 48(11A): 540-546. https://doi.org/10.11896/jsjkx.201200077
[15] 王卫东, 徐金慧, 张志峰, 杨习贝.
基于密度峰值聚类的高斯混合模型算法
Gaussian Mixture Models Algorithm Based on Density Peaks Clustering
计算机科学, 2021, 48(10): 191-196. https://doi.org/10.11896/jsjkx.200800191
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!