计算机科学 ›› 2015, Vol. 42 ›› Issue (5): 24-27.doi: 10.11896/j.issn.1002-137X.2015.05.005
易修文,李天瑞,张钧波,滕 飞
YI Xiu-wen, LI Tian-rui, ZHANG Jun-bo and TENG Fei
摘要: 随着云计算技术的发展,许多MapReduce运行系统被开发出来,如Hadoop、Phoenix和Twister等。直观上,Hadoop具有很强的可扩展性、稳定性,适合处理大规模离线应用;Phoenix具有运行速度快等优点,适合处理数据密集型任务;Twister是轻量级的迭代系统,非常适合迭代式的应用。不同的应用在不同的MapReduce运行系统中有着不同的性能。通过测试不同应用在这些运行系统上的性能,给出了实验比较和性能分析,从而为大数据处理时选择合适的并行编程模型提供依据。
[1] Pan Y,Zhang J B.Parallel Programming on Cloud Computing Platforms:Challenges and Solutions[J].KITCS/FTRA Journal of Convergence,2012,3(4):23-28〖ZK) [2] Armbrust M,Fox A,Griffith R,et al.A view of cloud computing[J].Communications of the ACM,2010,53(4):50-58 [3] Dean J,Ghemawat S.MapReduce:simplified data processing on large clusters[J].Communications of the ACM,2008,51(1):107-113 [4] White T.Hadoop:the definitive guide(2nd ed)[M].O’Reilly,2012 [5] Talbot J,Yoo R M,Kozyrakis C.Phoenix++:modular MapReduce for shared-memory systems[C]∥Proceedings of the Se-cond International Workshop on MapReduce and its Applications.ACM,2011:9-16 [6] Ekanayake J,Li H,Zhang B,et al.Twister:a runtime for iterative mapreduce[C]∥Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing.ACM,2010:810-818 [7] He B,Fang W,Luo Q,et al.Mars:a MapReduce framework on graphics processors[C]∥Proceedings of the 17th international conference on Parallel architectures and compilation techniques.ACM,2008:260-269 [8] Zaharia M,Chowdhury M,Das T,et al.Resilient distributeddatasets:A fault-tolerant abstraction for in-memory cluster computing[C]∥Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation.USENIX Association,2012:2 [9] Kang L Y,Wang X Y,Bai R J.Analysis of MapReduce Principle and Its Main Implementation Platforms[J].New Technology of Library and Information Service,2012(02):60-67 [10] Li J J,Li Q,Tian B.The Analysis and Comparison between MapReduce Implementations[J/OL].http://www.paper.edu.cn/html/releasepaper/2011/11/464.2011 [11] Li J J,C J,W D,et al.Survey of MapReduce Parallel Programming Model[J].ACTA Electronica Sinica,2011(11):2635-2642 [12] Borthakur D.The hadoop distributed file system:Architectureand design.http://hadoop.apache.ogr/docs/ro.18.1/hdsf_design.html [13] Thusoo A,Sarma J S,Jain N,et al.Hive:a warehousing solution over a map-reduce framework[J].Proceedings of the VLDB Endowment,2009,2(2):1626-1629 [14] Olston C,Reed B,Srivastava U,et al.Pig latin:a not-so-foreign language for data processing[C]∥Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data.ACM,2008:1099-1110 [15] Abouzeid A,Bajda-Pawlikowski K,Abadi D,et al.HadoopDB:an architectural hybrid of MapReduce and DBMS technologies for analytical workloads[J].Proceedings of the VLDB Endowment,2009,2(1):922-933 [16] Anil R,Dunning T,Friedman E.Mahout in action[M].Manning publications,2011 |
No related articles found! |
|