计算机科学 ›› 2017, Vol. 44 ›› Issue (10): 75-79.doi: 10.11896/j.issn.1002-137X.2017.10.014
徐琪,程耀东,陈刚
XU Qi, CHENG Yao-dong and CHEN Gang
摘要: 高能物理是典型的数据密集型计算环境,数据处理包括模拟计算、重建计算以及物理分析。其中大文件计算占据较大比重,并且高能物理文件访问模式以跳读为主,因此大文件的高速访问成为整个系统性能的重要影响因素。首先剖析传统高能物理计算环境的典型架构及其文件访问模式的特点,介绍混合存储模式在高能物理计算环境中的优势,总结其数据访问方式的特点,对其各种读写方式进行数据测试;然后提出针对该环境的混合存储系统的部署设计和优化,使该环境下的数据读写性能得到明显提高;同时将成本因素考虑到系统设计中,实现了一个低成本高性能的存储系统。测试表明,混合存储系统在高能物理等大数据存储系统中具有高效的I/O性能。文中全面分析了影响其性能的各种因素,实现了最优化配置的低成本高性能混合存储系统,并对该系统的未来发展趋势进行了分析和展望。
[1] WLCG-Worldwide LHC Computing Grid.http://lcg.web.cern.ch/LCG. [2] CABRERA L,LONG D D E.Swift:Using Distributed Disk Striping to Provide High I/O Data Rates[J].Computing Systems,1991,4(4):402-441. [3] CHENG Y D,SHI J Y,CHEN G.A survey of High Energy Physics Computing System[J].e-Science Technology & Application,2014,5(3):3-10.(in Chinese) 程耀东,石京燕,陈刚.高能物理计算环境概述[J].科研信息化技术与应用,2014,5(3):3-10. [4] CHENG Y D,WANG L,HUANG Q L,et al.Design and Optimization of Storage System in HEP Computing Environment[J].Computer Science,2015,42(1):54-58.(in Chinese) 程耀东,汪璐,黄秋兰,等.高能物理计算环境中存储系统的设计与优化[J].计算机科学,2015,42(1):54-58. [5] MITUZAS D.Flashcache at Facebook From 2010 to 2013 and beyond[EB/OL].[2013-10-9].https://www.facebook.com/notes/facebook-engineering. [6] Facebook/Flashcache.https://github.com/facebook/flash-cache. [7] MORE A,GANJEWAR P.Dynamic Cache Resizing in Flash-cache[J].Advances in Intelligent Systems and Computing,2015,327:537-544. [8] 敖青云.存储技术原理分析:基于Linux 2.6内核源代码[M].北京:电子工业出版社,2011:363-476. [9] LEE D,CHOI J,KIM J H,et al.On the Existence of a Spectrum of Policies That Subsumes the Least Recently Used (LRU) and Least Frequently Used (LFU) Policies[J].SIGMETRICS Performance Evaluation Review,1999,27(1):134-143. [10] RAMOS L E,GORBATOV E,B IANCHINI R.Page placement in hybrid memory systems.[C]∥ Proceedings of the international conference on Supercomputing.ACM,2011:85-95. [11] YANG Z Y.Design and Implementation of Hybrid StorageScheme Based on Flashcache[D].Wuhan:Huazhong University of Science and Technology,2013.(in Chinese) 杨昭宇.基于Flashcache的混合存储方案设计与实现[D].武汉:华中科技大学,2013. [12] Freecode/fio.http://freecode.com/projects/fio. [13] FRHWIRT P,HUBER M,MULAZZANI M,et al.InnoDBDatabase Forensics[C]∥24th IEEE International Conference on Advanced Information Networking and Applications (AINA).IEEE,2010:1028-1036. [14] LIN T J.Principle,Composition and Application of Several Typi-cal Solid State Drives[J].Computer and External Equipment,1999(1):16-21.(in Chinese) 林天静.几种典型固态盘的原理、组成形式及应用[J].电子计算机与外部设备,1999(1):16-21. |
No related articles found! |
|