Computer Science ›› 2017, Vol. 44 ›› Issue (4): 85-89.doi: 10.11896/j.issn.1002-137X.2017.04.019

Previous Articles     Next Articles

Scheduling Strategy of Hierarchical Storage about Replication in Cloud Storage

YANG Dong-ju and LI Qing   

  • Online:2018-11-13 Published:2018-11-13

Abstract: HDFS takes random storage strategy,if cluster has some cheap nodes,it is possible to make high frequency data store in the low processing performance nodes,causing a long time access and poor efficiency.To solve these problems,an improved scheduling strategy of hierarchical storage about replication was proposed.In order to reduce the number of replication scheduling,firstly,the information of data node from CPU load,memory load,network load,sto-rage load and network distance are used to evaluate node availability.Secondly,the optimal one is selected.Accessing frequency and hardware configuration are used to realize the replication scheduling.The response rate of cluster is improved by making high frequency data store on the high processing performance and high configuration node.The experimental results show that the strategy can improve access efficiency of replicas and local balancing for data storage in the heterogeneous clusters.

Key words: Cloud storage,HDFS,Hierarchical storage,Replication scheduling

[1] CHEN K,ZHENG W M.Clouding Computing:System Instan-ces and Current Research[J].Journal of Software,2009,20(5):1337-1348.(in Chinese) 陈康,郑纬民.云计算:系统实例与研究现状[J].软件学报,2009,20(5):1337-1348.
[2] Apache Hadoop .[2013-07-10].http://hadoop.apache.org.
[3] MILOJICIC D,WOLSKI R.Eucalyptus:delivering a private cloud[J].Computer,2011,44(4):102-104.
[4] 蔡斌,陈湘萍.Hadoop技术内幕:深入解析Hadoop Common和HDFS架构设计与实现原理[M].北京:机械工业出版社,2013.
[5] REN C,YANG D J.The two-stage Dynamic Optimized Scheduling Mechanism Based on Cloud Storage[J].Computer & Digital Engineering,2014,42(9):1553-1557,1716.(in Chinese) 任川,杨冬菊.基于云存储的二阶段动态优化调度机制[J].计算机与数字工程,2014,42(9):1553-1557,1716.
[6] TAO Y C,SHI L.Research on Dynamic Management of Data Replicas of Cloud Computing in Heterogeneous Environments[J].Journal of Chinese Computer Systems,2013,34(2):97-102.(in Chinese) 陶永才,石磊.异构资源环境下的MapReduce性能优化[J].小型微型计算机系统,2013,34(2):97-102.
[7] KARGER D,LEHMAN E,LEIGHTON T,et al.Consistenthashing and random trees:distributed caching protocols for relieving hot spots on the world wide web[C]∥ACM Symposium on Theory of Computing.CA,USA,1997:654-663.
[8] XIE J,YIN S,RUAN X J,et al.Improving mapreduce perfor-mance through data placement in heterogeneous Hadoop clusters[C]∥IPDPS Workshops.Atlanta:IEEE Computer Society Press,2010:1-9.
[9] ZAMAN S,GROSU D.A distributed algorithm for the replica placement problem[C]∥Proc.of IEEE Transaction on Parallel and Distributed System.2011:1455-1468.
[10] LUO P,GONG X.Research and Improvement of Data Place-ment Strategy for HDFS [J].Computer Engineering and Design,2014,35(4):1127-1131.(in Chinese) 罗鹏,龚勋.HDFS数据存放策略的研究与改进[J].计算机工程与设计,2014,35(4):1127-1131.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!