Computer Science ›› 2017, Vol. 44 ›› Issue (6): 17-22.doi: 10.11896/j.issn.1002-137X.2017.06.003

Previous Articles     Next Articles

Heterogeneous Storage Aware Data Placement of Ceph Storage System

LIU Fei, JIANG De-jun, ZHANG Huan, CHEN Jing, WANG Jun and XIONG Jin   

  • Online:2018-11-13 Published:2018-11-13

Abstract: Ceph distributed storage system is becoming a widely used open source cloud storage solution.Heterogeneous storage can provide large capacity and high performance storage while maintaining low cost if it uses an effective data management strategy.Using heterogeneous storage devices in Ceph currently cannot effectively exploit the performance of heterogeneous storage devices.Since multiple replicas of the data can be stored in different storage media,the performance and cost of different device combinations for multiple replicas are not the same.In this paper,a data placement method was proposed for heterogeneous storage based on Ceph.The method puts different data on different replica combination based on the access intensity and read/write ratio,which can effectively improve the system performance while controlling the cost of the system.

Key words: Heterogeneous storage,Data placement,Replicas,Ceph

[1] WEIL S A,BRANDT S A,MILLER E L,et al.Ceph:A scalable,high-performance distributed file system[C]∥Proceedings of the 7th Symposium on Operating Systems Design and Implementation.USENIX Association,2006:307-320.
[2] WEIL S A,BRANDT S A,MILLER E L,et al.CRUSH:Controlled,scalable,decentralized placement of replicated data[C]∥Proceedings of the 2006 ACM/IEEE Conference on Supercomputing.ACM,2006:122.
[3] WEIL S A,LEUNG A W,BRANDT S A,et al.Rados:a scalable,reliable storage service for petabyte-scale storage clusters[C]∥Proceedings of the 2nd International Workshop on Petascale Data Storage:Held in conjunction with Supercomputing’07.ACM,2007:35-44.
[4] PARK D,DU D H C.Hot data identification for flash-basedstorage systems using multiple bloom filters[C]∥2011 IEEE 27th Symposium on Mass Storage Systems and Technologies (MSST).IEEE,2011:1-11.
[5] CAI Y J,KANG C K,WU C H.A virtual storage environment for SSDs and HDDs in Xen hypervisor[C]∥ACM SIGBED.2014:39-44.
[6] fio.http://freecode.com/projects/fio/.
[7] TPC-WHomepage.http://www.tpc.org/tpcw/.
[8] Block I/O Traces.http://iotta.snia.org/traces/.
[9] FlashCache.https://github.com/facebook/flashcache.
[10] KLONATOS Y,MAKATOS T,MARAZAKIS M,et al.Azor:using two-level block selection to improve SSD-based I/O caches[C]∥2011 6th IEEE International Conference on En Networking,Architecture and Storage (NAS).IEEE,2011:309-318.
[11] CHEN F,JIANG S,ZHANG X.SmartSaver:turning flash drive into a disk energy saver for mobile computers[C]∥International Symposium on Low Power Electronics and Design,2006(ISLPED’06).IEEE,2006:412-417.
[12] KHATIB M G,VAN DER ZWAAG B J,Hartel P H,et al.Interposing flash between disk and DRAM to save energy for streaming workloads[C]∥IEEE/ACM/IFIP Workshop on Embedded Systems for Real-Time Multimedia,2007(ESTIMedia 2007).IEEE,2007:7-12.
[13] KIM Y,GUPTA A,URGAONKAR B,et al.HybridStore:acost-efficient,high-performance storage system combining SSDs and HDDs[C]∥2011 IEEE 19th International Symposium on Modeling,Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS).IEEE,2011:227-236.
[14] STRUNK,JON D.Hybrid Aggregates:Combining SSDs andHDDs in a single storage pool[J].ACM SIGOPS Operating Systems Review,2012,46(3):50-56.
[15] CHEN F,KOUFATY D A,ZHANG X D.Hystor:making the best use of solid state drives in high performance storage systems[C]∥Proceedings of the International Conference on Supercomputing.ACM,2011:22-32.
[16] LUO T,LEE R,MESNIER M,et al.hStorage-DB:heterogeneity-aware data management to exploit the full capability of hybrid storage systems[J].Proceedings of the VLDB Endowment,2012,5(10):1076-1087.
[17] JO H,KWON Y,KIM H,et al.SSD-HDD-hybrid virtual disk in consolidated environments[M]∥En Euro-Par 2009-Parallel Processing Workshops.Springer Berlin Heidelberg,2010:375-384.
[18] LIN L,et al.Hot random off-loading:a hybrid storage system with dynamic data migration[C]∥2011 IEEE 19th International Symposium on En Modeling,Analysis & Simulation of Compu-ter and Telecommunication Systems (MASCOTS).IEEE,2011:318-325.
[19] AWASTHI A,NANDINI A,Bhattacharya A,et al.HybridHBase:Leveraging flash SSDs to improve cost per throughput of HBase[C]∥Proceedings of the 18th International Conference on Management of Data.Computer Society of India,2012:68-79.
[20] KRISH K R,ANWAR A,BUTT A R.hatS:A heterogeneity-aware tiered storage for Hadoop[C]∥2014 14th IEEE/ACM International Symposium on Cluster,Cloud and Grid Computing (CCGrid).IEEE,2014:502-511.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!