Computer Science ›› 2017, Vol. 44 ›› Issue (7): 8-15.doi: 10.11896/j.issn.1002-137X.2017.07.002

Previous Articles     Next Articles

Study on Testing Cloud Computing Elasticity

HU Ya-zhou, DENG Bo, LIN Wang-qun, PENG Fu-yang and WANG Dong-xia   

  • Online:2018-11-13 Published:2018-11-13

Abstract: Elasticity is the key feature of cloud computing,which can support resources to elastically expand,fast configured,flexibly add and remove,and it can make full use of cloud resources and reduced the cost of cloud services providers and users.To evaluate whether elasticity meet the SLA (Service Level Agreement) requirements and make full use of cloud resources,we should evaluate and test elasticity of cloud computing.In this paper,we introduced the definition of elasticity,solutions and metrics for elasticity.Then we proposed the definition of cloud computing elasticity testing,and summarized some correlational researches of cloud computing elasticity testing.Next,we summarized its key technologies and frameworks.We also discussed some challenges and issues on testing cloud computing elasticity.Finally,we put forward the further research direction of cloud computing elasticity testing.

Key words: Cloud computing,Cloud testing,Elasticity,Elasticity testing

[1] ARMBRUST M,FOX O,GRIFFITH R,et al.Above theclouds:a Berkeley view of cloud computing[J].Eecs Department University of California Berkeley,2009,3(4):50-58.
[2] Amazon elastic compute cloud (Amazon EC2).http://aws.amazon.com/cn/ec2.
[3] ZAHARIEV A.Google app engine[J].Helsinki University of Technology,2009.
[4] SIMS K.IBM Blue Cloud initiative advances enterprise cloudcomputing.http://www-03.ibm.com/press/us/en/pressrelease/26642.wss.(Accessed on:January 23,2013),2009.
[5] Microsoft azure [EB/OL].http://www.microsoft.com/azure.
[6] Salesforce.com [EB/OL].http://www.force.com.
[7] MELL P,GRANCE T.The NIST definition of cloud compu-ting[J].Communications of the ACM,2011,53(6):50.
[8] MIROBI G J,AROCKIAM L.Service Level Agreement in cloud computing:An overview[C]∥International Conference on Control,Instrumentation,Communication and Computational Technologies.IEEE,2016.
[9] GAO J,BAI X,TSAI W T.Cloud testing-issues,challenges,needs and practice[J].Software Engineering:An International Journal,2011,1(1):9-23.
[10] BAI X,LI M,CHEN B,et al.Cloud testing tools[C]∥2011 IEEE 6th International Symposium on Service Oriented System Engineering (SOSE).IEEE,2011:1-12.
[11] RIUNGU-KALLIOSAARI L,TAIPALE O,S MOLANDER K.Testing in the cloud:Exploring the practice[J].Software,IEEE,2012,29(2):46-51.
[12] INCKI K,ARr I,SZER H.A survey of software testing in the cloud[C]∥2012 IEEE Sixth International Conference on Software Security and Reliability Companion (SERE-C).IEEE,2012:18-23.
[13] PRIYADARSINK K,BALASUBRAMANIAN V,KARTHIKS.Cloud testing as a service[J].International Journal of Advanced Engineering Sciences and Technologies,2011,6(2):173-177.
[14] KUMAR R,SINGH S.Cloud Testing:Perspectives and Challenges[J].International Journal of Computer Applications,2014,6(17):975-8887.
[15] COOPER B F,SILBERSTEIN A,TAM E,et al.Benchmarking cloud serving systems with YCSB[C]∥Proceedings of the 1st ACM Symposium on Cloud Computing.ACM,2010:143-154.
[16] SOASTA [EB/OL].http://www.SOASTA.com.
[17] ITKO [EB/OL].http://www.itko.com.
[18] YU L,TSAI W T,CHEN X,et al.Testing as a Service over Cloud[C]∥2010 Fifth IEEE International Symposium on Servi-ce Oriented System Engineering (SOSE).IEEE,2010:181-188.
[19] Cloud-intelligence.Cloud-testing white paper[EB/OL].[2011-03-24].http://www.cloud-intelligence.com/files/Cloud%20Tes-ting%20White%20%Paper_0.pdf.
[20] RIUNGU L M,TAIPALE O,SMOLANDER K.Research issues for software testing in the cloud[C]∥2010 IEEE Second International Conference on Cloud Computing Technology and Scien-ce (CloudCom).IEEE,2010:557-564.
[21] RIUNGU L M,TAIPALE O,SMOLANDER K.Software testing as an online service:Observations from practice[C]∥2010 Third International Conference on Software Testing,Verification,and Validation Workshops (ICSTW).IEEE,2010:418-423.
[22] LI Q,KE D L,WANG X L.Brief survey on cloud testing[J].Application Research of Computers,2012,9(12):4401-4406.(in Chiese) 李乔,柯栋梁,王小林.云测试研究现状综述[J].计算机应用研究,2012,29(12):4401-4406.
[23] GAISBAUER S,KIRSCHNICK J,E DWARDW N,et al.Vats:Virtualized-aware automated test service[C]∥Fifth Internatio-nal Conference on Quantitative Evaluation of Systems,2008(QEST’08).IEEE,2008:93-102.
[24] CIORTEA L,ZAMFIR C,BUCUR S,et al.Cloud9:A software testing service[J].ACM SIGOPS Operating Systems Review,2010,43(4):5-10.
[25] BANZAI T,KOIZUMI H,KANBAYASHI R,et al.D-cloud:Design of a software testing environment for reliable distributed systems using cloud computing technology[C]∥Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster,Cloud and Grid Computing.IEEE Computer Society,2010:631-636.
[26] ORIOL M,ULLAH F.Yeti on the cloud[C]∥2010 Third International Conference on Software Testing,Verification,and Validation Workshops (ICSTW).IEEE,2010:434-437.
[27] PARVEEN T,TILLEY S,DALEY N,et al.Towards a distributed execution framework for JUnit test cases[C]∥IEEE International Conference on Software Maintenance,2009(ICSM 2009).IEEE,2009:425-428.
[28] HANAWA T,BANZAI T,KOIZUMI H,et al.Large-Scale Sof-tware Testing Environment Using Cloud Computing Technology for Dependable Parallel and Distributed Systems[C]∥Third International Conference on Software Testing,Verification,and Validation Workshops.IEEE Computer Society,2010:428-433.
[29] TSAI W T,COLBOURN C J,LUO J,et al.Test algebra forcombinatorial testing[C]∥2013 8th International Workshop on Automation of Software Test (AST).IEEE,2013:19-25.
[30] Cloud Testing [EB/OL].http://www.CloudTesting.com.
[31] IBM.Smart business test cloud [EB/OL].http://www-935.ibm.com/service/us/index.wss/offering.midware/a1030965.
[32] Baidu.MTC [EB/OL].http://mtc.baidu.com.
[33] Testin.[EB/OL].http://www.testin.cn/portal.action?op=Portal.index.
[34] COTRONEO D,FRATTINI F,PIETRANTUONO R,et al.State-based robustness testing of IaaS cloud platforms[C]∥Proceedings of the 5th International Workshop on Cloud Data and Platforms.ACM,2015.
[35] FAGHIR F,BAZARBAYEV S,OVERHOLT M,et al.Failure scenario as a service (FSaaS) for hadoop clusters[C]∥Procee-dings of the Workshop on Secure and Dependable Middleware for Cloud Monitoring and Management.ACM,2012.
[36] GUNAWI H S,DO T,JOSHI P,et al.FATE and DESTINI:A Framework for Cloud Recovery Testing[C]∥NSDI.2011.
[37] JOSHI P,GUNAWI H S,SEN K.PREFAIL:A programmable tool for multiple-failure injection[J].ACM SIGPLAN Notices,2011,46(10):171-188.
[38] BALDUZZI M,ZADDACH J,BALZAROTTI D,et al.A security analysis of amazon’s elastic compute cloud service[C]∥Proceedings of the 27th Annual ACM Symposium on Applied Computing.ACM,2012:1427-1434.
[39] YIGITBASI N,IOSUP A,EPEMA D,et al.C-meter:A framework for performance analysis of computing clouds[C]∥Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.IEEE Computer Society,2009:472-477.
[40] MOLDOVAN D,COPIL G,TRUONG H L,et al.Mela:Monitoring and analyzing elasticity of cloud services[C]∥2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom).IEEE,2013:80-87.
[41] COPIL G,TRIHINAS D,TRUONG H L,et al.ADVISE-aFramework for Evaluating Cloud Service Elasticity Behavior[M]∥Service-Oriented Computing.Springer Berlin Heidelberg,2014:275-290.
[42] GAMBI A,HUMMER W,DUSTDAR S.Automated testing of cloud-based elastic systems with AUToCLES[C]∥2013 IEEE/ACM 28th International Conference on Automated Software Engineering (ASE).IEEE,2013:714-717.
[43] ANTONIOU A.Performance evaluation of cloud infrastructure using complex workloads[D].TU Delft,Delft University of Technology,2012.
[44] SCHOUTEN E.Rapid Elasticity and the Cloud,September2012[EB/OL].http://thoughtsoncloud.com/index.php/2012/09/rapid-elasticity-and-the-cloud.
[45] MODEL O M U.Compute Infratructure as a Service[R].Tech.rep.,Open Data Center Alliance (OCDA),2012.http://www.opendatacenteralliance.org/docs/ODCA_Compute_IaaS_Master-UM_v1.0_Nov2012.pdf.
[46] COHEN R.Defining elastic computing,September 2009.http://www.elasticvapor.com/2009/09/defining-elastic-computing.html,2013.
[47] HERBST N R,KOUNEV S,REUSSNER R.Elasticity in Cloud Computing:What It Is,and What It Is Not[C]∥ICAC.2013:23-27.
[48] AISOPOS F,TSERPES K,VARVARIGOU T.Resource mana-gement in software as a service using the knapsack problem model[J].International Journal of Production Economics,2013,141(2):465-477.
[49] LI M,YE F,KIM M,et al.A scalable and elastic publish/subscribe service[C]∥2011 IEEE International Parallel & Distribu-ted Processing Symposium (IPDPS).IEEE,2011:1254-1265.
[50] PEREZ-SORROSAL F,PATIO-MARTINEZ M,JIMENEZ-PERIS R,et al.Elastic SI-Cache:consistent and scalable caching in multi-tier architectures[J].The VLDB Journal—The International Journal on Very Large Data Bases,2011,20(6):841-865.
[51] HAN R,GHAMEN M M,GUO L,et al.Enabling cost-awareand adaptive elasticity of multi-tier cloud applications[J].Future Generation Computer Systems,2014,32:82-98.
[52] GALANTE G,DE BONA L C E.A survey on cloud computing elasticity[C]∥2012 IEEE Fifth International Conference on Utility and Cloud Computing (UCC).IEEE,2012:263-270.
[53] AGRAWAL D,EL ABBADI A,DAS S,et al.Database scalability,elasticity,and autonomy in the cloud[C]∥Database Systems for Advanced Applications.Springer Berlin Heidelberg,2011:2-15.
[54] DAS S.Scalable and elastic transactional data stores for cloud computing platforms[D].Unversity of Californai Santa Barbara,2011.
[55] Vaquero L M,Rodero-Merino L,Buyya R.Dynamically scaling ap-plications in the cloud[J].ACM SIGCOMM Computer Communication Review,2011,41(1):45-52.
[56] COUTINHO E F,SOUSA F R D C,REGO P A L,et al.Elasti-city in cloud computing:a survey[J].Annals of Telecommunications,2015,0(7):289-309.
[57] LIM H C,BABU S,CHASE J S,et al.Automated control in cloud computing:challenges and opportunities[C]∥Proceedings of the 1st Workshop on Automated Control for Datacenters and Clouds.ACM,2009:13-18.
[58] MENG S,LIU L,SOUNDARARAJAN V.Tide:achieving self-scaling in virtualized datacenter management middleware[C]∥Proceedings of the 11th International Middleware Conference Industrial track.ACM,2010:17-22.
[59] CALHEIROS R N,VECCHIOLA C,K ARUNAMOORTHY D,et al.The Aneka platform and QoS-driven resource provisioning for elastic applications on hybrid Clouds[J].Future Generation Computer Systems,2012,28(6):861-870.
[60] FIT J O,GOIRI I,GUITART J.SLA-driven elastic cloud hosting provider[C]∥2010 18th Euromicro International Confe-rence on Parallel,Distributed and Network-Based Processing (PDP).IEEE,2010:111-118.
[61] MARSHALL P,KEAHEY K,FREEMAN T.Elastic site:Using clouds to elastically extend site resources[C]∥ Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster,Cloud and Grid Computing.IEEE Computer Society,2010:43-52.
[62] DAWOUD W,TAKOUNA I,MEINEL C.Elastic vm for cloud resources provisioning optimization[M]∥Advances in Computing and Communications.Springer Berlin Heidelberg,2011:431-445.
[63] DAWOUD W, TAKOUNA I,MEINEL C.Elastic VM forCloud Resources Provisioning Optimization[C]∥Advances in Computing and Communications-First International Confe-rence(Acc 2011).Kochi,India,DBLP,2011:431-445.
[64] GONG Z,GU X,WILKES J.Press:Predictive elastic resource scaling for cloud systems[C]∥2010 International Conference on Network and Service Management (CNSM).IEEE,2010:9-16.
[65] SHEN Z M,SUBBIAH S, GU X H,et al.Cloud Scale:Elastic Resource Scaling for Multi-tenant Cloud Systems[C]∥Proc.of the 2nd ACM Symposium on Cloud Computing.New York,USA:ACM Press,2011:1353-1355.
[66] SHARMA U,SHENOY P,SAHU S,et al.A cost-aware elastici- ty provisioning system for the cloud[C]∥2011 31st Internatio-nal Conference on Distributed Computing Systems (ICDCS).IEEE,2011:559-570.
[67] KNAUTH T,FETZER C.Scaling non-elastic applications using virtual machines[C]∥2011 IEEE International Conference on Cloud Computing (CLOUD).IEEE,2011:468-475.
[68] HERBST N R.Quantifying the Impact of Platform Configuration Space for Elasticity Benchmarking[D].Informatics Institute,2011.
[69] WEBER A,HERBST N,GROENDA H,et al.Towards a Resource Elasticity Benchmark for Cloud Environments[C]∥Proceedings of the 2nd International Workshop on Hot Topics in Cloud service Scalability.ACM,2014.
[70] HERBST N R,KOUNEV S,WEBER A,et al.BUNGEE:An Elasticity Benchmark for Self-Adaptive IaaS Cloud Environments[C]∥Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015).2015.
[71] LI Z,O’BRIEN L,ZHANG H,et al.On a catalogue of metrics for evaluating commercial cloud services[C]∥Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing.IEEE Computer Society,2012:164-173.
[72] FERREIRA COUTINHO E,GONCALVES GOMESD,NEU-MAN DE SOUZA J.An analysis of elasticity in cloud computing environments based on allocation time and resources[C]∥2nd IEEE Latin American Conference on Cloud Computing and Communications (LatinCloud).IEEE,2013:7-12.
[73] SHAWKY D M,ALI A F.Defining a measure of cloud computing elasticity[C]∥2012 1st International Conference on Systems and Computer Science (ICSCS).IEEE,2012:1-5.
[74] BREBNER P C.Is your cloud elastic enough?:performancemodelling the elasticity of infrastructure as a service (iaas) cloud applications[C]∥Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering.ACM,2012:263-266.
[75] BERSANI M M,BIANCULLI D,DUSTDAR S,et al.Towards the formalization of properties of cloud-based elastic systems [C]∥Proceedings of the 6th International Workshop on Principles of Engineering Service-Oriented and Cloud Systems.ACM,2014:38-47.
[76] FOLKERTS E,ALEXANDROV A,SACHS K,et al.Bench-marking in the cloud:What it should,can,and cannot be[M]//Selected Topics in Performance Evaluation and Benchmarking.Springer Berlin Heidelberg,2013:173-188.
[77] SULEIMAN B.Elasticity economics of cloud-based applications[C]∥2012 IEEE Ninth International Conference on Services Computing (SCC).IEEE,2012:694-695.
[78] ISLAM S,LEE K,FEKETE A,et al.How a consumer canmeasure elasticity for cloud platforms[C]∥Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering.ACM,2012:85-96.
[79] TINNEFELD C,TASCHIK D,PLATTNER H.Quantifying the elasticity of a database management system[C]∥DBKDA 2014,The Sixth International Conference on Advances in Databases,Knowledge,and Data Applications.2014:125-131.
[80] MOLDOVAN D,COPIL G,TRUONG H L,et al.On Analyzing Elasticity Relationships of Cloud Services[C]∥2014 IEEE 6th International Conference on Cloud Computing Technology and Science (CloudCom).IEEE,2014:447-454.
[81] SHARMA U.Elastic resource management in cloud computing platforms[J].Dissertations & Theses-Gradworks,2013:1-175.
[82] JOGALEKAR P,WOODSIDE M.Evaluating the scalability of distributed systems[J].IEEE Transactions on Parallel and Distributed Systems,2000,11(6):589-603.
[83] JMeter [EB/OL].http://jakarta.apache.org/jmeter.
[84] Faban [EB/OL].http://java.net/projects/faban.
[85] Siege [EB/OL].http://www.joedog.org/siege-home.
[86] MENASC D A.TPC-W:A benchmark for e-commerce[J].IEEE Internet Computing,2002,6(3):83-87.
[87] Amazon Cloudwatch [EB/OL].http://aws.amazon.com/cn/cloudwatch.
[88] Yahoo Chukwa [EB/OL].http://chuka.apache.org.
[89] Ganglia [EB/OL].http://ganglia.info.
[90] Nagios [EB/OL].http://www.nagios.org.
[91] Grenchmark [EB/OL].http://grenchmark.st.ewi.tudelft.nl.
[92] NAJJAR A,SERPAGGI X,GRAVIER C,et al.Survey of elasticity management solutions in cloud computing[M]∥Continued Rise of the Cloud.Springer London,2014:235-263

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!