Computer Science ›› 2017, Vol. 44 ›› Issue (10): 75-79, 90.doi: 10.11896/j.issn.1002-137X.2017.10.014

Previous Articles     Next Articles

Design and Optimization of Hybrid Storage System in HEP Environment

XU Qi, CHENG Yao-dong and CHEN Gang   

  • Online:2018-12-01 Published:2018-12-01

Abstract: Computing in high energy physics (HEP) is a typical data-intensive application including simulation,reconstruction and physical analysis.Generally,the HEP experiment file is very big and the way of accessing to the files is usually skipping through large data blocks.Therefore,the performance of accessing to big files is one of decisive factors for the HEP computing system.Firstly,this paper analyzed the typical structure of the computing environment in high energy physics and the characters of accessing to files,introduced the advantages of hybrid storage system in high energy physics,summarized the characteristics of data access mode,evaluated the performance of different read/write mode,then proposed a new deployment model of hybrid storage system in high energy physics,which is proved to have higher I/O performance,at the same time the cost was considered to implement a high-performance system with low cost.The test result shows that the hybrid storage system has good performance in some fields such as HEP.Based on the analysis,it can help to get better I/O performance with lower price in High Energy Physics.At the last,the future of the hybrid storage system was analyzed.

Key words: Mass storage system,HEP,Hybrid storage system,Cache,Block device,HPC,Performance price ratio

[1] WLCG-Worldwide LHC Computing Grid.http://lcg.web.cern.ch/LCG.
[2] CABRERA L,LONG D D E.Swift:Using Distributed Disk Striping to Provide High I/O Data Rates[J].Computing Systems,1991,4(4):402-441.
[3] CHENG Y D,SHI J Y,CHEN G.A survey of High Energy Physics Computing System[J].e-Science Technology & Application,2014,5(3):3-10.(in Chinese) 程耀东,石京燕,陈刚.高能物理计算环境概述[J].科研信息化技术与应用,2014,5(3):3-10.
[4] CHENG Y D,WANG L,HUANG Q L,et al.Design and Optimization of Storage System in HEP Computing Environment[J].Computer Science,2015,42(1):54-58.(in Chinese) 程耀东,汪璐,黄秋兰,等.高能物理计算环境中存储系统的设计与优化[J].计算机科学,2015,42(1):54-58.
[5] MITUZAS D.Flashcache at Facebook From 2010 to 2013 and beyond[EB/OL].[2013-10-9].https://www.facebook.com/notes/facebook-engineering.
[6] Facebook/Flashcache.https://github.com/facebook/flash-cache.
[7] MORE A,GANJEWAR P.Dynamic Cache Resizing in Flash-cache[J].Advances in Intelligent Systems and Computing,2015,327:537-544.
[8] 敖青云.存储技术原理分析:基于Linux 2.6内核源代码[M].北京:电子工业出版社,2011:363-476.
[9] LEE D,CHOI J,KIM J H,et al.On the Existence of a Spectrum of Policies That Subsumes the Least Recently Used (LRU) and Least Frequently Used (LFU) Policies[J].SIGMETRICS Performance Evaluation Review,1999,27(1):134-143.
[10] RAMOS L E,GORBATOV E,B IANCHINI R.Page placement in hybrid memory systems.[C]∥ Proceedings of the international conference on Supercomputing.ACM,2011:85-95.
[11] YANG Z Y.Design and Implementation of Hybrid StorageScheme Based on Flashcache[D].Wuhan:Huazhong University of Science and Technology,2013.(in Chinese) 杨昭宇.基于Flashcache的混合存储方案设计与实现[D].武汉:华中科技大学,2013.
[12] Freecode/fio.http://freecode.com/projects/fio.
[13] FRHWIRT P,HUBER M,MULAZZANI M,et al.InnoDBDatabase Forensics[C]∥24th IEEE International Conference on Advanced Information Networking and Applications (AINA).IEEE,2010:1028-1036.
[14] LIN T J.Principle,Composition and Application of Several Typi-cal Solid State Drives[J].Computer and External Equipment,1999(1):16-21.(in Chinese) 林天静.几种典型固态盘的原理、组成形式及应用[J].电子计算机与外部设备,1999(1):16-21.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75, 88 .
[2] XIA Qing-xun and ZHUANG Yi. Remote Attestation Mechanism Based on Locality Principle[J]. Computer Science, 2018, 45(4): 148 -151, 162 .
[3] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .
[4] WANG Huan, ZHANG Yun-feng and ZHANG Yan. Rapid Decision Method for Repairing Sequence Based on CFDs[J]. Computer Science, 2018, 45(3): 311 -316 .
[5] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[6] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[7] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[8] LIU Qin. Study on Data Quality Based on Constraint in Computer Forensics[J]. Computer Science, 2018, 45(4): 169 -172 .
[9] ZHONG Fei and YANG Bin. License Plate Detection Based on Principal Component Analysis Network[J]. Computer Science, 2018, 45(3): 268 -273 .
[10] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99, 116 .