计算机科学 ›› 2014, Vol. 41 ›› Issue (7): 30-35.doi: 10.11896/j.issn.1002-137X.2014.07.005
师明,刘轶,唐歌实
SHI Ming,LIU Yi and TANG Ge-shi
摘要： 如何为上层应用和计算提供稳定高效的文件I/O性能,是分布式文件系统性能研究的热点。文中分析分布式文件系统在设计机理上的共同特征,基于此提出一种通用型的启发式文件预取模型,并选取HDFS平台进行系统实现。启发式文件预取对上层应用透明,采用在文件系统内部建立预取线程池的方法,以组成文件块的数据存储文件为预取单位,在分布式文件系统内部实现。这种设计思路具有一定的普适性,适合推广应用于多种分布式文件系统。实验结果表明,所述的启发式文件预取,能够有效提升分布式文件系统的I/O性能。
| [1] Yue Yin-liang,Feng Dan,Wang Juang,et al．High AvailabilityStorage System Based on Two-level Metadata Management[C]∥Proceedings of Frontier of Computer Science and Technology(FCST 2007)．2007:41-48 [2] Mackey G,Sehrish S,Wang Jun．Improving Metadata Management for Small Files in HDFS[C]∥Proceedings of Cluster Computing and Workshops．2009:1-4 [3] Yu Wei-kuan,Oral H S,Canon R S,et al．Empirical Analysis of a Large-Scale Hierarchical Storage System[C]∥Euro-Par 2008-Parallel Processing．2008:130-140 [4] Yu Wei-kuan,Jeffrey S V,et al．Performance Characterizationand Optimization of Parallel I/O on the Cray XT[C]∥The 22nd IEEE International Parallel and Distributed Processing Symposium(IPDPS2008)．2008:1-11 [5] Dittrich J,Quiane R J A,Jindal A,et al．Hadoop++:Making a Yellow Elephant Run Like a Cheetah(Without It Even Noticing)[J]．Proceedings of the VLDB Endowment,2010(3):515-529 [6] OMalley O.The Anatomy of Hadoop I/O Pipeline[EB/OL]．http://developer.yahoo.com/ [7] Liu Xu-hui,Han Ji-zhong,Zhong Yun-qin,et al．Implementing WebGIS on Hadoop:A case study of improving small file I/O performance on HDFS[C]∥Cluster Computing and Workshops(CLUSTER’09)．2009:1-8 [8] Dong Bo,Qiu Jie,Zheng Qing-hua,et al．A Novel Approach to Improving the Efficiency of Storing and Accessing Small Files on Hadoop:A Case Study by PowerPoint Files[C]∥Services Computing Conference(SCC)．2009:65-72 |
| No related articles found! |
|
||
首页