计算机科学 ›› 2015, Vol. 42 ›› Issue (4): 44-50.doi: 10.11896/j.issn.1002-137X.2015.04.007

• 网络与通信 • 上一篇    下一篇

基于多目标优化的云存储副本分布策略的研究

张华伟,李志华   

  1. 江南大学物联网应用技术教育部工程研究中心 无锡214122,江南大学物联网应用技术教育部工程研究中心 无锡214122;江南大学物联网工程学院轻工过程先进控制教育部重点实验室 无锡214122
  • 出版日期:2018-11-14 发布日期:2018-11-14
  • 基金资助:
    本文受江苏省科技厅产学研前瞻项目(BY2013015-23),中央高校科研专项(JUSRP211A41)资助

Research on Data Distribution Strategy in Cloud Storage System Based on Multi-objective Optimization

ZHANG Hua-wei and LI Zhi-hua   

  • Online:2018-11-14 Published:2018-11-14

摘要: 针对现有云存储副本分布策略优化目标比较单一的不足,提出了局部最佳分布策略(Local Optimum Distribution,LODS)。LODS策略通过给出一系列新定义并利用一致性哈希函数来缩小副本分布的节点选择范围,进一步结合层次分析法,将一定决策半径内的节点作为方案层中的候选对象,通过更深入地研究云存储多目标优化准则对其优化从而最终选择出当前候选方案中的最佳目标节点。实验结果表明,通过优化的最优决策半径取值相对稳定,不随云存储系统规模的扩展和数据的增多而剧烈变化,并且当取值最佳决策半径时,LODS策略的存储负载平衡、热度负载平衡、等待时间性能高于HDFS、Amazon S3等系统中所采用的副本分布策略。

关键词: 云存储,一致性哈希,层次分析法,副本分布,多目标优化

Abstract: The data distribution strategy in cloud storage systems has shortcomings that the distribution strategy only considers the load balancing among physical storage nodes instead of the heat load balancing,waiting time and other factors.Aiming at this question,a rational and efficient data distribution strategy local optimum distribution (LODS) was proposed.By defining a series of new definitions to describe the correlations in LODS,employing the hashing method to act as the core ideal and presenting a couple of multi-objective optimization rules,LODS combines the above innovations and the AHP method to give the finally optimized storage scheme.The proposed storage scheme can search the target storage node in cloud system within a local range with a optimized decision radius.The decision radius is stable without variation while the large-scale storage amount is tested.Once the decision radius is “0”,what the LODS degenerates is almost corresponding to that of Amazon S3.Experiments show that the average advantage of the proposed cloud storage strategy overs that used in HDFS,Amazon S3 in terms of the strategies of storage load balancing,load balancing heat and waiting time.

Key words: Cloud storage,Consistent hashing,AHP,Data distribution,Multi-objective optimization

[1] Borthakur D.The Hadoop distributed file system:Architecture and design [EB/OL].[20l3-08-04].http://hadoop.apache.org/docs/r1.2.1/hdfs_design.pdf
[2] Ghemawat S,Gogioff H,Leung P T.The google file system[C]∥Proceedings of the 19th ACM Symp on Operating Systems Principles.New York:ACM,2003:29-43
[3] Amazon.comInc.Amazon simple storage service (AmazonS3)[EB/OL].[2014-04-09].http://aws.amazon.com/s3
[4] Lewn D.Consistent hashing and random trees:Algorithms for caching in distributed networks[D].Cambridge,Massachusetts:Massa-chusetts Institute of Technology,Department of Electrical Engineering and Computer science,1998
[5] 郭金玉,张忠彬,孙庆云.层次分析法的研究与应用[J].中国安全科学学报,2008,18(5):148-153
[6] Mohamed N,Al-Jaroodi J,Eid A.A dual-direction technique for fast file downloads with dynamic load balancing in the Cloud [J].Journal of Network and Computer Applications,2013,36(4):1116-1130
[7] 周敬利,周正达.改进的云存储系统数据分布策略[J].计算机应用,2012,2(2):309-312
[8] 王永洲,茅苏.HDFS中的一种数据放置策略[J].计算机技术与发展,2013,(5):90-92
[9] 董继光,陈卫卫,田浪军,等.大规模云存储系统副本布局研究[J].计算机应用,2012,2(3):620-624
[10] 董继光,陈卫卫,吴海佳,等.基于动态副本技术的云存储负载均衡研究[J].计算机应用研究,2012,9(9):3422-3424,6
[11] Chen Tao,Bahsoon R,Tawil A-R H.Scalable service-oriented replication with flexible consistency guarantee in the cloud[J].Information Sciences,2014,4:349-370
[12] Du Zhi-hui,Hu Jing-kun,Chen Yi-nong,et al.Optimized QoS-aware replica placement heuristics and applications in astronomy data grid [J].Journal of Systems and Software,2011,4(7):1224-1232
[13] 廖彬,于炯,张陶,等.基于分布式文件系统HDFS的节能算法[J].计算机学报,2013,6(5):1047-1064
[14] Gkantsidis C,Vytiniotis D,Hodson O,et al.Rhea:automatic filtering for unstructured cloud storage[C]∥Presented as part of the 10th USENIX Symposium on Networked Systems Design and Implementation.2013:343-355
[15] Freedman M J,Shaikh A.Performance isolation and fairness for multi-tenant cloud storage[C]∥Proc.10th USENIX Conference on Operating Systems Design and Implementation.2012:349-362
[16] 高尚.3种计算层次分析法中权值的方法[J].科学技术与工程,2007,7(20):5204-5207
[17] Dong Bo,Zheng Qing-hua,Tian Feng,et al.An optimized ap-proach for storing and accessing small files on cloud storage[J].Journal of Network and Computer Applications,2012,5(6):1847-1862

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!