Computer Science ›› 2015, Vol. 42 ›› Issue (5): 24-27.doi: 10.11896/j.issn.1002-137X.2015.05.005

Previous Articles     Next Articles

Performance Testing and Analysis among Different MapReduce Runtime Systems

YI Xiu-wen, LI Tian-rui, ZHANG Jun-bo and TENG Fei   

  • Online:2018-11-14 Published:2018-11-14

Abstract: With the development of cloud computing technology,several implementations which adopt MapReduce mo-del,e.g.,Hadoop,Phoenix and twister,have been developed.Hadoop has high scalability and stability,thus is suitable for handling large-scale off-line applications.The primary advantage of Phoenix,which is especially appropriate for data-intensive tasks,is its processing speed.Twister,a lightweight iterative runtime system,is designed for iterative applications.Different applications produce different levels of performance on different MapReduce runtime systems.By testing various applications using the aforementioned runtime systems,the experimental comparison and performance analysis were presented,providing the basis for the selection of parallel programming models for big data processing.

Key words: Cloud computing,MapReduce,Hadoop,Phoenix,Twister

[1] Pan Y,Zhang J B.Parallel Programming on Cloud Computing Platforms:Challenges and Solutions[J].KITCS/FTRA Journal of Convergence,2012,3(4):23-28〖ZK)
[2] Armbrust M,Fox A,Griffith R,et al.A view of cloud computing[J].Communications of the ACM,2010,53(4):50-58
[3] Dean J,Ghemawat S.MapReduce:simplified data processing on large clusters[J].Communications of the ACM,2008,51(1):107-113
[4] White T.Hadoop:the definitive guide(2nd ed)[M].O’Reilly,2012
[5] Talbot J,Yoo R M,Kozyrakis C.Phoenix++:modular MapReduce for shared-memory systems[C]∥Proceedings of the Se-cond International Workshop on MapReduce and its Applications.ACM,2011:9-16
[6] Ekanayake J,Li H,Zhang B,et al.Twister:a runtime for iterative mapreduce[C]∥Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing.ACM,2010:810-818
[7] He B,Fang W,Luo Q,et al.Mars:a MapReduce framework on graphics processors[C]∥Proceedings of the 17th international conference on Parallel architectures and compilation techniques.ACM,2008:260-269
[8] Zaharia M,Chowdhury M,Das T,et al.Resilient distributeddatasets:A fault-tolerant abstraction for in-memory cluster computing[C]∥Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation.USENIX Association,2012:2
[9] Kang L Y,Wang X Y,Bai R J.Analysis of MapReduce Principle and Its Main Implementation Platforms[J].New Technology of Library and Information Service,2012(02):60-67
[10] Li J J,Li Q,Tian B.The Analysis and Comparison between MapReduce Implementations[J/OL].http://www.paper.edu.cn/html/releasepaper/2011/11/464.2011
[11] Li J J,C J,W D,et al.Survey of MapReduce Parallel Programming Model[J].ACTA Electronica Sinica,2011(11):2635-2642
[12] Borthakur D.The hadoop distributed file system:Architectureand design.http://hadoop.apache.ogr/docs/ro.18.1/hdsf_design.html
[13] Thusoo A,Sarma J S,Jain N,et al.Hive:a warehousing solution over a map-reduce framework[J].Proceedings of the VLDB Endowment,2009,2(2):1626-1629
[14] Olston C,Reed B,Srivastava U,et al.Pig latin:a not-so-foreign language for data processing[C]∥Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data.ACM,2008:1099-1110
[15] Abouzeid A,Bajda-Pawlikowski K,Abadi D,et al.HadoopDB:an architectural hybrid of MapReduce and DBMS technologies for analytical workloads[J].Proceedings of the VLDB Endowment,2009,2(1):922-933
[16] Anil R,Dunning T,Friedman E.Mahout in action[M].Manning publications,2011

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!