计算机科学 ›› 2016, Vol. 43 ›› Issue (8): 19-25.doi: 10.11896/j.issn.1002-137X.2016.08.004

• 目次 • 上一篇    下一篇

基于PCA降维的云资源状态监控数据压缩技术

洪斌,邓波,彭甫阳,包阳,冯学伟   

  1. 北京市系统工程研究所 北京 100101,北京市系统工程研究所 北京 100101,北京市系统工程研究所 北京 100101,清华大学计算机科学与技术系 北京100084,信息系统安全技术国家重点实验室 北京100101
  • 出版日期:2018-12-01 发布日期:2018-12-01
  • 基金资助:
    本文受国家高技术研究发展计划(863计划)(2013AA01A215),国家自然科学基金(61271252)资助

Data Dimension Reduction Method Based on PCA for Monitoring Data of Virtual Resources in Cloud Computing

HONG Bin, DENG Bo, PENG Fu-yang, BAO Yang and FENG Xue-wei   

  • Online:2018-12-01 Published:2018-12-01

摘要: 云计算资源状态监控作为保障云服务质量和可靠性的重要自动化手段,必须从海量的监控数据中分析出各类云资源的真实状态信息。为了减少资源监控任务自身对云计算资源的消耗,提出一种基于PCA(Principal Components Analysis)降维的监控数据的降维和筛选技术。监控数据转换利用PCA降维,将原始监控数据映射至若干主成分方向上,实现数据压缩。而监控数据筛选则着眼于在保留原始数据的前提下,筛选出关键监控指标以有效表征资源状态。基于VICCI云服务实验平台的实验结果证明,所提出的方法能够从多种监控数据中快速筛选出表征资源状态的核心数据,在保证状态监控效果的前提下,有效减少了监控任务所需处理的数据量。

关键词: 云计算,状态监控,数据降维,大数据,主成分分析

Abstract: Cloud computing has become increasingly popular and cloud providers face serious problems as resource monitoring tasks become more and more complicated.As an effective approach to enhancing availability and reliability of cloud infrastructures,state monitoring system aims to detect anomalous state in cloud by analyzing monitoring data.To reduce the processed data volume,we proposed a data dimension reduction method based on PCA(Principal Components Analysis)with high fidelity in this article.The results of experiments carried on VICCI cloud service platform show that,our method can select the kernel metrics from hundreds of monitoring data types and sharply reduce the computing overload incurred by state monitoring tasks.

Key words: Cloud computing,State monitoring,Dimension reduction,Big data,PCA

[1] Aceto G,Botta A,De Donato W,et al.Cloud monitoring:A survey[J].Computer Networks,2013,7(9):2093-2115
[2] Amazon CloudWatch developer Guide.http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/acw-dg.pdf
[3] Jolliffe I.Principal component analysis[M].John Wiley & Sons,Ltd,2002
[4] Moore B C.Principal component analysis in linear systems:Controllability,observability,and model reduction[J].IEEE Transac-tions on Automatic Control,1981,26(1):17-32
[5] Meng S,Liu L.Enhanced monitoring-as-a-service for effective cloud management[J].IEEE Transactions on Computers,2013,62(9):1705-1720
[6] Meng S,Kashyap S R,Venkatramani C,et al.Resource-aware application state monitoring[J].IEEE Transactions on Parallel and Distributed Systems,2012,23(12):2315-2329
[7] Mi H,Wang H,Zhou Y,et al.Toward fine-grained,unsuper-vised,scalable performance diagnosis for production cloud computing systems[J].IEEE Transactions on Parallel and Distributed Systems,2013,24(6):1245-1255
[8] Pannu H S,Liu J,Guan Q,et al.AFD:Adaptive failure detection system for cloud computing infrastructures[C]∥2012 IEEE 31st International Performance Computing and Communications Conference (IPCCC).IEEE,2012:71-80
[9] Zheng Pai,Cui Li-zheng,Wang Hai-yang,et al.A data place-ment Strategy for Data-Intensive Applicationa in Cloud[J].Chinese Journey of Computer,2010,33(8):1472-1480(in Chinese) 郑湃,崔立真,王海洋,等.云计算环境下面向数据密集型应用的数据布局策略与方法[J].计算机学报,2010,33(8):1472-1480
[10] Zhu Xia,Luo Jun-zhou,Song Ai-bo,et al.A multi-Dimensional Indexing for Complex Query in Cloud Computing[J].Journal of Computer Research and Development,2015,50(8):1592-1603(in Chinese) 朱夏,罗军舟,宋爱波,等.云计算环境下支持复杂查询的多维数据索引机制[J].计算机研究与发展,2015,50(8):1592-1603
[11] Chandola V,Banerjee A,Kumar V.Anomaly detection:A survey[J].ACM Computing Surveys (CSUR),2009,41(3):75-79
[12] Salfner F,Lenk M,Malek M.A survey of online failure prediction methods[J].ACM Computing Surveys (CSUR),2010,42(3):1283-1310
[13] Timusk M A.A unified method for anomaly detection in un-steady systems[M].2006
[14] McBain J,Timusk M.Feature extraction for novelty detection as applied to fault detection in machinery[J].Pattern Recognition Letters,2011,32(7):1054-1061
[15] Kinney J B,Atwal G S.Equitability,mutual information,and the maximal information coefficient[J].Proceedings of the National Academy of Sciences,2014,111(9):3354-3359

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!