计算机科学 ›› 2015, Vol. 42 ›› Issue (7): 174-177.doi: 10.11896/j.issn.1002-137X.2015.07.038

• 网络与通信 • 上一篇    下一篇

基于分数阶Fourier变换的云存储系统重复数据删除算法

徐奕奕,唐培和   

  1. 广西科技大学计算机科学与通信工程学院 柳州545006;武汉理工大学信息工程学院 武汉430070,广西科技大学计算机科学与通信工程学院 柳州545006
  • 出版日期:2018-11-14 发布日期:2018-11-14
  • 基金资助:
    本文受广西自然科学基金青年基金项目(2013GXNSFBA019268),广西科技大学自然科学基金项目(校科自1261126),广西特色专业建设项目(GXTSZY217),广西教育厅一般项目(YB2014208),广西教育厅立项项目(LX2014182)资助

Duplicate Data Remove Algorithm of Cloud Storage System Based on Fractional Fourier Transform

XU Yi-yi and TANG Pei-he   

  • Online:2018-11-14 Published:2018-11-14

摘要: 云存储系统的重复数据作为大量冗余数据的一种,对其有效及时地删除能保证云存储系统的稳定与运行。由于云存储系统中的干扰数据较多,信噪比较低,传统的重删算法会在分数阶Fourier域出现伪峰峰值,不能有效地对重复数据进行检测滤波和删除处理,因此提出一种改进的基于分数阶Fourier变换累积量检测的云存储系统重复数据删除算法。首先分析云存储系统重复数据删除机制体系架构,定义数据存储点的适应度函数,得到云存储节点的系统子集随机概率分布;采用经验约束函数对存储节点中的校验数据块分存,通过分数阶Fourier变换对云存储系统中的幅度调制分量进行残差信号滤波预处理。采用4阶累积量切片后置算子,把每个文件分为若干个块,针对每个文件块进行重删,进行重复数据检测后置滤波处理,实现存储资源上的重复数据检测及其删除。仿真实验表明,该算法能提高集群云存储系统计算资源的利用率,重复数据准确删除率较高,有效避免了数据信息流的干扰特征造成的误删和漏删,性能优越。

关键词: 分数阶Fourier变换,云存储,重复数据

Abstract: Duplicate data of cloud storage system is taken as one of a large amount of redundant data,and the effective and timely remove can guarantee the stability and operation of cloud storage system.Because of the interference of data,the SNR is low,the traditional method has false peaks in the fractional Fourier domain,and it cannot effectively detect and remove the duplicate data.An improved duplicate data remove algorithm of cloud storage system was proposed based on fractional Fourier transform cumulant detection.Firstly,the delete system architecture for cloud storage system was taken,the fitness function of data storage point was defined,and system subset random probability distribution function of the cloud storage node was gotten.The constraint function was used for blocking the calibration data of storage nodes,the detection of duplicate data removing processing was taken,and the fractional Fourier transform was used to preprocess the residual signal filtering in cloud storage system.The 4 order cumulanted slice post operator was used to divide each file into blocks.To delete each file block,duplicated data detection post filtering was obtained,and data storage resource detection and deletion were realized.Simulation results show that this algorithm can improve the utilization efficiency of cluster cloud storage system resource,and duplicate data can be accurately removed with higher rate.It can effectively avoid the error removing caused by interference and leakage removing,and it has superior performance.

Key words: Fractional Fourier transform ,Cloud storage,Duplicate data

[1] 谢平.存储系统重复数据删除技术研究综述[J].计算机科学,2014,1(1):22-30 Xie Ping.Surey on data deduplication techniques for storage systems[J].Computer Science,2014,1(1):22-30
[2] Miorandi D,Sicari S,Pellegrini F D,et al.Internet of things:vision,applications and research challenges[J].Ad Hoc Networks,2012,10(7):1497-1516
[3] Wu T Y,Lee W T,Lin Y S,et al.Dynamic load balancing mecha-nism based on cloud storage[C]∥Computing,Communications and Applications Conference (ComComAp),2012.IEEE,2012:102-106
[4] 蒋海波,王晓京,范明钰,等.基于水平纠删码的云存储数据布局方法[J].四川大学学报(工程科学版),2013,45(2):103-109 Jiang Hai-bo,Wang Xiao-jing,Fan Ming-yu.A Data Placement Based on Level Array Codes in Cloud Storage [J].Journal of Sichuan University(Engineeging Science Edition),2013,45(2):103-109
[5] 敖莉,舒继武,李明强.重复数据删除技术[J].软件学报,2010,1(5):916-929 Ao Li,Shu Ji-wu,Li Ming-qiang.Data Deduplication Techniques[J].Journal of Software,2010,1(5):916-929
[6] 付印金,肖侬,刘芳.重复数据删除关键技术研究进展[J].计算机研究与发展,2012,9(1):12-20 Fu Ying-jin,Xiao Nong,Liu Fang.Research and Development on Key Techniques of Data Deduplicaton[J].Journal of Computer Research and Development,2012,9(1):12-20
[7] 李渊.智能PID控制区优化仿真研究[J].计算机仿真,2012,29(12):180-182 Li Yuan.Parameters Optimization of PID Controller[J].Computer Simulation,2012,29(12):180-182
[8] 谭鹏许,陈越,兰巨龙,等.用于云存储的安全容错编码[J].通信学报,2014,5(3):109-114 Tan Peng-xu,Chen Yue,Lan Ju-long,et al.Secure fault-tolerant code for cloud storage[J].Journal on Communications,2014,5(3):109-114
[9] Tang Pei-he,Xu Yi-yi.Resource Scheduling Strategy Based on Credibility in the Enterprise Gloud Strorage[J].Journal of Convergence Information Technology,2012,7(16):393-400

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!