计算机科学 ›› 2011, Vol. 38 ›› Issue (2): 63-67.

• 计算机网络与信息安全 • 上一篇    下一篇

基于存储环境感知的重复数据删除算法优化

周敬利,聂雪军,秦磊华,刘科,朱建峰,王宇   

  1. (华中科技大学计算机科学与技术学院 武汉430074)
  • 出版日期:2018-11-16 发布日期:2018-11-16
  • 基金资助:
    本文受部委基金“基于服务定制的智能存储系统研究”,!词家白然科学基金项目(606730001),国家"973"重点基础研究发展规划基金项目(2004CB318203)资助。

Optimization for Data De-duplication Algorithm Based on Storage Environment Aware

ZHOU Jing-li,NIE Xue-jun,QIN Lei-hua,LIU Ke,ZHU Jian-feng,WANG Yu   

  • Online:2018-11-16 Published:2018-11-16

摘要: 在备份和归档等存储应用中产生的重复数据在存储空间和能耗上造成的浪费问题日益突出,如何删除重复数据已成为当前存储领域中的研究热点。CDC(Content Defined Chunking)是一种适用于多种应用环境的重复数据删除算法,但缺乏针对具体应用环境的优化。通过对存储环境进行感知和分析,为CIX;算法提出了两个参数选择约束条件:(1)根据存储设备中的数据块存储方式来选择平均分块大小等参数;(2)根据数据的分块边界分布特性来选择分块边界特征值参数。实验表明,与无约束条件的CDC算法相比,这两个约束条件在4个实验数据集上平均可提高16. 3%的数据缩减比。

关键词: 重复数据删除,存储环境感知,CDC,文件系统,分块边界

Abstract: Storage applications such as backup and archive are creating more and more duplication data, which has caused increasing waste in storage space and energy consumption, and how to delete duplication data has become a hot subject in research. CI}C(Content-Defined Chunking) is a prevail algorithm for data dcduplication and can be applicable in various environment,however it does not take into account some characteristics which are specific to individual environment and can influence its result. We studied the CDC' s application in storage system and put up with two constraints for determining parameters for CDC; (1) Determining parameters such as average block size based on the block organization of storage devices; (2)Determining block boundary based on candidate boundary distribution. The result indicates that, comparing with the separate CI}C without restraint conditions, these two constraints can achieve 16. 3% higher compression ratio with 4 data sets.

Key words: Data de-duplication, Storage environment aware, Cdc, File system, Block boundary

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!