关键词: 大数据, 副本复制, 擦除编码, 冷热数据, 存储利用率

Abstract: With the rapid development of big data technology,cloud computing,computer technology and network technology,Internet data has shown explosive growth,and efficient storage of massive data has become an urgent challenge for current Internet technology.However,traditional multi-copy redundancy mechanisms result in huge storage costs,thus drawing attention to new storage solutions.In this context,a distributed hybrid storage strategy based on erasure coding and replica replication is proposed.Based on data characteristics,this strategy uses replica replication for hot data to ensure high reliability and performance,while erasure coding is used for cold data to improve storage utilization.Based on Newton's cooling law,the data files is divided into hot files and cold files,and an adaptive data temperature identification and hot and cold data adaptive dynamic allocation algorithm are introduced,so that the system can automatically adjust the ratio of hot and cold data at runtime,and then intelligently adjust the data storage strategy according to the the hot and cold conditions of real-time data,which reflects the system's adaptability in a dynamic environment.It not only enhances the system's adaptability to dynamic workloads,but also provides a new paradigm for the efficiency and flexibility of distributed storage systems in practical applications.This innovation has important promotion significance at both the academic and practical levels.At the same time,the effectiveness and usability of the strategy have been verified through simulation experiments,which provides new ideas for the optimization of distributed storage systems.

Key words: Big data, Replica replication, Erasure coding, Hot and cold data, Storage utilization rate