计算机科学 ›› 2016, Vol. 43 ›› Issue (10): 63-65.doi: 10.11896/j.issn.1002-137X.2016.10.011

• 2015 第五届全国可信计算学术会议 • 上一篇    下一篇

一种新的在线流数据异常检测方法

丁智国,莫毓昌,杨凡   

  1. 浙江师范大学数理与信息工程学院 金华321004,浙江师范大学数理与信息工程学院 金华321004,浙江师范大学数理与信息工程学院 金华321004
  • 出版日期:2018-12-01 发布日期:2018-12-01
  • 基金资助:
    本文受浙江省计算机科学与技术重中之重学科开放项目(ZC323014100),浙江省自然科学基金项目(Y13F020080)资助

Novel Anomaly Detection Method of Online Streaming Data

DING Zhi-guo, MO Yu-chang and YANG Fan   

  • Online:2018-12-01 Published:2018-12-01

摘要: 流数据的海量、无限、分布动态变化且不均衡等特征使得对流数据的在线异常检测成为当前一个研究热点。分析了异常数据的少而不同且更容易通过随机空间的分割而孤立出来的特征,基于在线集成学习理论,提出了一种基于隔离森林的在线流数据异常检测算法。在4个UCI标准数据集上的实验结果表明提出的方法有效。

关键词: 流数据,异常检测,隔离森林,在线集成学习

Abstract: Streaming data has some unique characteristics such as massive,unlimited generation,dynanmic variation distribution and unbalanced data distribution and so on,which make the anomaly detection for streaming data become a research hot spot.An obvious characteristic of anomalous sample is “few and different” compared to these normal data,which makes it more easily to be isolated than normal data by the partition of stochastic space.In this paper,a novel online anomaly detection method for streaming data was proposed based on the isolation principle and online ensemble learning theory.Experiments conducted on four UCI datasets demonstrate the effectiveness of the proposed method.

Key words: Streaming data,Anomaly detection,Isolation forest,Online ensemble learning

[1] Gupta M,Gao J,Aggarwal C C,et al.Outlier Detection for Temporal Data:A Survey[J].IEEE Transactions on Knowledge and Data Engineering,2014,6(9):2250-2267
[2] Zheng Li-ming,Zou Peng,Jia Yan.How to Extract and Train the Classifier in Traffic Anomaly Detection System[J].Chinese Journal of Computers,2012,35(4):719-730(in Chinese) 郑黎明,邹鹏,贾焰.网络流量异常检测中分类器的提取与训练方法研究[J].计算机学报,2012,35(4):719-730
[3] Shen M X,Liu D R,Shann S H.Outlier detection from vehicle trajectories to discover roaming events[J].Information Sciences,2015,294:242-254
[4] Lu You,Li Wei,Luo Jun-zhou,et al.A Network User’s Abnormal Behavior Detection Approach Based on Selective Collaborative Learning[J].Chinese Journal of Computers,2014,37(1):28-41(in Chinese) 陆悠,李伟,罗军舟,等,一种基于选择性协同学习的网络用户异常行为检测方法[J].计算机学报,2014,37(1):28-41
[5] Srinivasan U.Anomalies Detection in Healthcare Services[J].It Professional,2014,6(6):12-15
[6] Zhou Dong-hua,Wei Mu-heng,Si Xiao-sheng.A Survey on Anomaly Detection,Life Prediction and Maintenance Decision for Industrial Processes[J].Acta Automatica Sinica,2013,39(6):711-722(in Chinese) 周东华,魏慕恒,司小胜.工业过程异常检测、寿命预测与维修决策的研究进展[J].自动化学报,2013,39(6):711-722
[7] Tan S C,Ting K M,Liu T F.Fast anomaly detection for strea-ming data[C]∥Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence.AAAI Press,2011,2:1511-1516
[8] Widmer G,Kubat M.Learning in the presence of concept drift and hidden contexts[J].Machine Learning,1996,3(1):69-101
[9] Li D,Liu S L,Zhang H L.A negative selection algorithm with online adaptive learning under small samples for anomaly detection[J].Neurocomputing,2015,9:515-525
[10] Liu F T,Ting K M,Zhou Z H.Isolation-Based Anomaly Detection[J].ACM Transactions on Knowledge Discovery from Data,2012,6(1):1-39
[11] Daneshpazhouh A,Sami A.Entropybased outlier detectionusing semi-supervised approach with few positive examples[J].Pattern Recognition Letters,2014,9:77-84
[12] Quinn J A,Sugiyama M.A least-squares approach to anomaly detection in static and sequential data[J].Pattern Recognition Letters,2014,0:36-40
[13] He H,Chen S,Li K,et al.Incremental learning from stream da-ta[J].IEEE Transactions on Neural Networks and Learning Systems,2011,2(12):1901-1914
[14] Ando S,Thanomphongphan T,Seki Y,et al.Ensemble anomaly detection from multi-resolution trajectory features[J].Data Mining and Knowledge Discovery,2015,9(1):39-83
[15] Minku L L,Yao X.DDD:A New Ensemble Approach for Dea-ling with Concept Drift[J].IEEE Transactions on Knowledge and Data Engineering,2012,4(4):619-633
[16] Chang W C,Cho C W.Online Boosting for Vehicle Detection[J].IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics,2010,0(3):892-902
[17] UCI Machine Learning Repository .http://archive.ics.uci.edu/ml/datasets.html
[18] Hand D J,Tillr J.A simple generalisation of the area under the ROC curve for multiple class classification problems[J].Machine Learning,2001,5(2):171-186

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!