Computer Science ›› 2022, Vol. 49 ›› Issue (12): 17-21.doi: 10.11896/jsjkx.220700131

• Federated Leaming • Previous Articles     Next Articles

Storage Task Allocation Algorithm in Decentralized Cloud Storage Network

SHEN Zhen, ZHAO Cheng-gui   

  1. School of Information Science,Yunnan University of Finance and Economics,Kunming 650221,China
  • Received:2022-07-13 Revised:2022-09-30 Published:2022-12-14
  • About author:SHEN Zhen,born in 1993,postgra-duate,is a member of China Computer Federation.His main research interests include blockchain,SDN/NFV,federated learning and so on.ZHAO Cheng-gui,born in 1974,Ph.D,professor.His main research interests include communication network and computer system architecture,distributed systems,algebraic graph theory and its applications,blockchain,federated learning and so on.
  • Supported by:
    National Natural Science Foundation of China(61562089).

Abstract: Constructing a novel model for the storage task allocation problem of federated learning client datasets,to ensure load balancing of decentralized cloud storage networks,shorten the storage data uploading and recovery time,and reduce the total client storage cost,a data storage task allocation algorithm——URGL_allo (allocation based on user requirements and global load) that considers client requirements and global load is proposed.In the node allocation phase,node resources such as global load,topological attributes,storage price and data recovery time concerned by clients are considered,and a new node ranking method is defined in conjunction with the law of gravity to select the best storage task allocation node.In the link allocation stage,the shortest path calculation is performed using Dijkstra’s algorithm for the client node as the center to other nodes in the network,and the path with the largest bandwidth value in the set of shortest paths between two nodes is selected for allocation.Simulation results show that the proposed algorithm reduces the load balancing index and the total client storage cost by 41.9% and 5%,respectively,compared with the random policy-based allocation algorithm (Random_allo),and the data recovery time is not much different from that of the link bandwidth-based greedy algorithm,both of which are stably maintained between (0,2],which is 1/20 of Random_allo.The combined performance of global load and service quality is better than that of the comparison algorithm.

Key words: Federated learning, Decentralized cloud storage, Storage task allocation, Global load, Client requirements, Node sequencing

CLC Number: 

  • TP311
[1]ZHANG C,XIE Y,BAI H,et al.A survey on federated learning [J].Knowledge-Based Systems,2021,216:106775.
[2]XIAO T,CUI T,ISLAM S R,et al.Joint content placement and storage allocation based on federated learning in F-RANs [J].Sensors,2020,21:215.
[3]LEMAÎTRE G,NOGUEIRA F,ARIDAS C K.Imbalanced-learn:A python toolbox to tackle the curse of imbalanced datasets in machine learning [J].The Journal of Machine Learning Research,2017,18:559-563.
[4]JUBA B,LE H S.Precision-recall versus accuracy and the role of large data sets[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:4039-4048.
[5]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tionefficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[6]WEBVIEW-LAB.Decentralized storage,the disruptor of traditional cloud storage [EB/OL].(2022-04-15) [2022-08-26].http://www.yitb.com/article-19160.
[7]CANGIR O F,CANKUR O,OZSOY A.A taxonomy for Blockchain based distributed storage technologies [J].Information Processing & Management,2021,58:102627.
[8]MAIDSAFE.NET.Safe network [EB/OL].(2014-03-26) [2022-08-26].https://github.com/maidsafe/Whitepapers/blob/master/Project-Safe.md.
[9]VAKILINIA I,VAKILINIA S,BADSHA S,et al.Pooling ap-proach for task allocation in the blockchain based decentralized storage network[C]//2019 15th International Conference on Network and Service Management(CNSM).IEEE,2019:1-6.
[10]YUAN N H.Research on user-oriented software defined cloud storage resource allocation strategy[D].Beijing:Beijing University of Posts and Telecommunications,2017.
[11]BENISI N Z,AMINIAN M,JAVADI B Blockchain-based decentralized storage networks:A survey [J].Journal of Network and Computer Applications,2020,162:102656.
[12]PROTOCOL-LABS.Filecoin:A dicentralized storage networks [EB/OL].[2022-07-06].https://ipfs.netlify.app/tutorial/whitepapera.html.
[13]FU C H.The frame of decentralized storage system based on distributed ledger [D].Chengdu:University of Electronic Science and Technology of China,2020.
[14]STORJ LABS I.Storj:A Decentralized Cloud Storage Network Framework(Version 3.0) [EB/OL].(2018-10-30) [2022-07-06].https://www.storj.io/whitepaper.
[15]CONG L,LI R,WANG H,et al.Distributed data storage allocation scheme for the industrial internet of things[J].Electric Power Information and Communication Technology,2020,18(7):52-57.
[16]WU S,YIN H,CAO H,et al.Node ranking strategy in virtual network embedding:An overview [J].China Communications,2021,18:114-136.
[17]CAO H T.Research on virtual network embedding algorithm in network virtualization environment [D].Nanjing:Nanjing University of Posts and Telecommunications,2020.
[18]GONG L,WEN Y,ZHU Z,et al.Toward profit-seeking virtual network embedding algorithm via global resource capacity[C]//IEEE INFOCOM 2014-IEEE Conference on Computer Communications.IEEE,2014:1-9.
[1] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients [J]. Computer Science, 2022, 49(9): 183-193.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] CHEN Ming-xin, ZHANG Jun-bo, LI Tian-rui. Survey on Attacks and Defenses in Federated Learning [J]. Computer Science, 2022, 49(7): 310-323.
[4] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Clustered Federated Learning Methods Based on DBSCAN Clustering [J]. Computer Science, 2022, 49(6A): 232-237.
[5] YAN Meng, LIN Ying, NIE Zhi-shen, CAO Yi-fan, PI Huan, ZHANG Lan. Training Method to Improve Robustness of Federated Learning [J]. Computer Science, 2022, 49(6A): 496-501.
[6] DU Hui, LI Zhuo, CHEN Xin. Incentive Mechanism for Hierarchical Federated Learning Based on Online Double Auction [J]. Computer Science, 2022, 49(3): 23-30.
[7] WANG Xin, ZHOU Ze-bao, YU Yun, CHEN Yu-xu, REN Hao-wen, JIANG Yi-bo, SUN Ling-yun. Reliable Incentive Mechanism for Federated Learning of Electric Metering Data [J]. Computer Science, 2022, 49(3): 31-38.
[8] ZHAO Luo-cheng, QU Zhi-hao, XIE Zai-peng. Study on Communication Optimization of Federated Learning in Multi-layer Wireless Edge Environment [J]. Computer Science, 2022, 49(3): 39-45.
[9] ZOU Sai-lan, LI Zhuo, CHEN Xin. Study on Transmission Optimization for Hierarchical Federated Learning [J]. Computer Science, 2022, 49(12): 5-16.
[10] YANG Hong-jian, HU Xue-xian, LI Ke-jia, XU Yang, WEI Jiang-hong. Study on Privacy-preserving Nonlinear Federated Support Vector Machines [J]. Computer Science, 2022, 49(12): 22-32.
[11] QU Xiang-mou, WU Ying-bo, JIANG Xiao-ling. Federated Data Augmentation Algorithm for Non-independent and Identical Distributed Data [J]. Computer Science, 2022, 49(12): 33-39.
[12] GUO Gui-juan, TIAN Hui, WANG Tian, JIA Wei-jia. Efficient Federated Learning Scheme Based on Background Optimization [J]. Computer Science, 2022, 49(12): 40-45.
[13] LIANG Wen-ya, LIU Bo, LIN Wei-wei, YAN Yuan-chao. Survey of Incentive Mechanism for Federated Learning [J]. Computer Science, 2022, 49(12): 46-52.
[14] WU Yun-han, BAI Guang-wei, SHEN Hang. Multi-dimensional Resource Dynamic Allocation Algorithm for Internet of Vehicles Based on Federated Learning [J]. Computer Science, 2022, 49(12): 59-65.
[15] GUO Yan-qing, LI Yu-hang, WANG Wan-wan, FU Hai-yan, WU Ming-kan, LI Yi. FL-GRM:Gamma Regression Algorithm Based on Federated Learning [J]. Computer Science, 2022, 49(12): 66-73.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!