计算机科学 ›› 2017, Vol. 44 ›› Issue (4): 85-89.doi: 10.11896/j.issn.1002-137X.2017.04.019
杨冬菊,李青
YANG Dong-ju and LI Qing
摘要: 当集群中的部分节点是廉价主机时,采用HDFS的随机存储策略可能使访问频率高的数据存储在廉价节点上,受到廉价节点的性能影响,访问时间过长,降低了集群效率。为改善以上问题,提出一种改进的副本分级存储调度策略。为减少副本调度的次数,先根据节点的CPU、内存、网络、存储负载以及网络距离来评价节点的性能,再从中选取高性能节点进行存储。副本调度以节点中副本的访问频率为依据,结合硬件配置,把访问频率高的副本尽可能存储在高性能、高配置的节点中,以加快集群响应速度。实验结果表明,改进后的策略可以在异构集群中提高副本的访问效率,优化负载均衡。
[1] CHEN K,ZHENG W M.Clouding Computing:System Instan-ces and Current Research[J].Journal of Software,2009,20(5):1337-1348.(in Chinese) 陈康,郑纬民.云计算:系统实例与研究现状[J].软件学报,2009,20(5):1337-1348. [2] Apache Hadoop .[2013-07-10].http://hadoop.apache.org. [3] MILOJICIC D,WOLSKI R.Eucalyptus:delivering a private cloud[J].Computer,2011,44(4):102-104. [4] 蔡斌,陈湘萍.Hadoop技术内幕:深入解析Hadoop Common和HDFS架构设计与实现原理[M].北京:机械工业出版社,2013. [5] REN C,YANG D J.The two-stage Dynamic Optimized Scheduling Mechanism Based on Cloud Storage[J].Computer & Digital Engineering,2014,42(9):1553-1557,1716.(in Chinese) 任川,杨冬菊.基于云存储的二阶段动态优化调度机制[J].计算机与数字工程,2014,42(9):1553-1557,1716. [6] TAO Y C,SHI L.Research on Dynamic Management of Data Replicas of Cloud Computing in Heterogeneous Environments[J].Journal of Chinese Computer Systems,2013,34(2):97-102.(in Chinese) 陶永才,石磊.异构资源环境下的MapReduce性能优化[J].小型微型计算机系统,2013,34(2):97-102. [7] KARGER D,LEHMAN E,LEIGHTON T,et al.Consistenthashing and random trees:distributed caching protocols for relieving hot spots on the world wide web[C]∥ACM Symposium on Theory of Computing.CA,USA,1997:654-663. [8] XIE J,YIN S,RUAN X J,et al.Improving mapreduce perfor-mance through data placement in heterogeneous Hadoop clusters[C]∥IPDPS Workshops.Atlanta:IEEE Computer Society Press,2010:1-9. [9] ZAMAN S,GROSU D.A distributed algorithm for the replica placement problem[C]∥Proc.of IEEE Transaction on Parallel and Distributed System.2011:1455-1468. [10] LUO P,GONG X.Research and Improvement of Data Place-ment Strategy for HDFS [J].Computer Engineering and Design,2014,35(4):1127-1131.(in Chinese) 罗鹏,龚勋.HDFS数据存放策略的研究与改进[J].计算机工程与设计,2014,35(4):1127-1131. |
No related articles found! |
|