Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (3): 39-45.doi: 10.11896/jsjkx.210800054

• Novel Distributed Computing Technology and System • Previous Articles     Next Articles

Study on Communication Optimization of Federated Learning in Multi-layer Wireless Edge Environment

ZHAO Luo-cheng, QU Zhi-hao, XIE Zai-peng   

  1. School of Computer and Information,Hohai University,Nanjing 211100,China
  • Received:2021-08-05 Revised:2021-09-06 Online:2022-03-15 Published:2022-03-15
  • About author:ZHAO Luo-cheng,born in 1998,postgraduate.His main research interests include distributed computing and fe-derated learning.
    QU Zhi-hao,born in 1989,assistant researcher.His main research interests include federated learning,cloud edge collaboration and distributed machine learning.
  • Supported by:
    Fundamental Research Funds for the Central Universities(B200202176,B210202079) and Project Funded by China Postdoctoral Science Foundation(2019M661709).

PDF (PC)

1042

Abstract: Existing model synchronization mechanisms of federated learning (FL) are mostly based on single-layer parameter server architecture,which are difficult to adapt to current heterogeneous wireless network scenarios.There are some problems such as excessive communication load on single-point and poor scalability of FL.In response to these problems,this paper proposes an efficient model synchronization scheme for FL in hybrid wireless edge networks.In a hybrid edge wireless network,edge devices transmit local models to nearby small base stations.After receiving local models from edge devices,small base stations exe-cute the aggregation algorithm and send the aggregated models to the macro base station to update the global model.Considering the heterogeneity of channel performance and the competitive relationship of data transmission on the wireless channel,this paper proposes a new type of grouping asynchronous model synchronization scheme and designs a transmission rate aware channel allocation algorithm.Experiments are carried out on real data sets.Experimental results show that the proposed transmission rate aware channel allocation algorithm in grouping asynchronous model synchronization scheme can reduce communication time by 25%～60% and greatly improve the training efficiency of FL.

Key words: Asynchronous update, Channel allocation, Federated learning, Heterogeneous wireless network, Model aggregation

CLC Number: 

  • TP393.1
[1]MOHANTA B K,JENA D,SATAPATHY U,et al.Survey onIoT security:Challenges and solution using machine learning,artificial intelligence and blockchain technology[J].Internet of Things,2020,11:100227-100244.
[2]LI S,DA X L,ZHAO S.5G Internet of Things:A survey[J].Journal of Industrial Information Integration,2018,10:1-9.
[3]QADRI Y A,NAUMAN A,ZIKRIA Y B,et al.The future ofhealthcare internet of things:a survey of emerging technologies[J].IEEE Communications Surveys & Tutorials,2020,22(2):1121-1167.
[4]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial intelligence and statistics.PMLR,2017:1273-1282.
[5]NIKNAM S,DHILLON H S,REED J H.Federated learning for wireless communications:Motivation,opportunities,and challenges[J].IEEE Communications Magazine,2020,58(6):46-51.
[6]CHEN M,YANG Z,SAAD W,et al.A joint learning and communications framework for federated learning over wireless networks[J].IEEE Transactions on Wireless Communications,2020,20(1):269-283.
[7]KONČ ENÝ J,MCMAHAN H B,YU F X,et al.Federatedlearning:Strategies for improving communication efficiency[J].arXiv:1610.05492,2016.
[8]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[9]CHEN M,POOR H V,SAAD W,et al.Convergence time optimization for federated learning over wireless networks[J].IEEE Transactions on Wireless Communications,2020,20(4):2457-2471.
[10]TRAN N H,BAO W,ZOMAYA A,et al.Federated learningover wireless networks:Optimization model design and analysis[C]//IEEE INFOCOM 2019-IEEE Conference on Computer Communications.IEEE,2019:1387-1395.
[11]PILLA L L.Optimal task assignment for heterogeneous federated learning devices[C]//2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS).IEEE,2021:661-670.
[12]JIANG P,AGRAWAL G.A linear speedup analysis of distributed deep learning with sparse and quantized communication[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.2018:2530-2541.
[13]WADU M M,SAMARAKOON S,BENNIS M.Joint clientscheduling and resource allocation under channel uncertainty in federated learning[J].IEEE Transactions on Communications,2021,69(9):5962-5974.
[14]YANG H H,LIU Z,QUEK T Q S,et al.Scheduling policies for federated learning in wireless networks[J].IEEE Transactions on Communications,2019,68(1):317-333.
[15]XIA W,QUEK T Q S,GUO K,et al.Multi-armed bandit-based client scheduling for federated learning[J].IEEE Transactions on Wireless Communications,2020,19(11):7108-7123.
[16]NISHIO T,YONETANI R.Client selection for federated lear-ning with heterogeneous resources in mobile edge[C]//2019 IEEE International Conference on Communications (ICC 2019).IEEE,2019:1-7.
[17]QU Z,GUO S,WANG H,et al.Partial synchronization to acce-lerate federated learning over relay-assisted edge networks[J/OL].IEEE Transactions on Mobile Computing.https://ieeexplore.ieee.org/abstract/document/9439928.
[18]ABAD M S H,OZFATURA E,GUNDUZ D,et al.Hierarchical federated learning across heterogeneous cellular networks[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2020:8866-8870.
[19]LIU L,ZHANG J,SONG S H,et al.Client-edge-cloud hierarchical federated learning[C]//2020 IEEE International Confe-rence on Communications (ICC 2020).IEEE,2020:1-6.
[20]YUAN J,XU M,MA X,et al.Hierarchical federated learning through lan-wan orchestration[J].arXiv:2010.11612,2020.
[21]LIM W Y B,NG J S,XIONG Z,et al.Decentralized edge intelligence:a dynamic resource allocation framework for hierarchical federated learning[J].IEEE Transactions on Parallel and Distributed Systems,2021,33(3):536-550.
[22]LUO S,CHEN X,WU Q,et al.Hfel:Joint edge association and resource allocation for cost-efficient hierarchical federated edge learning[J].IEEE Transactions on Wireless Communications,2020,19(10):6535-6548.
[23]WU W,HE L,LIN W,et al.SAFA:A semi-asynchronous protocol for fast federated learning with low overhead[J].IEEE Transactions on Computers,2020,70(5):655-668.
[24]XIE C,KOYEJO S,GUPTA I.Asynchronous federated optimization[J].arXiv:1903.03934,2019.
[25]CHEN Y,NING Y,SLAWSKI M,et al.Asynchronous online federated learning for edge devices with non-iid data[C]//2020 IEEE International Conference on Big Data.IEEE,2020:15-24.
[26]CHEN Y,SUN X,JIN Y.Communication-efficient federateddeep learning with layerwise asynchronous model update and temporally weighted aggregation[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(10):4229-4238.
[1] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients [J]. Computer Science, 2022, 49(9): 183-193.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] CHEN Ming-xin, ZHANG Jun-bo, LI Tian-rui. Survey on Attacks and Defenses in Federated Learning [J]. Computer Science, 2022, 49(7): 310-323.
[4] YAN Meng, LIN Ying, NIE Zhi-shen, CAO Yi-fan, PI Huan, ZHANG Lan. Training Method to Improve Robustness of Federated Learning [J]. Computer Science, 2022, 49(6A): 496-501.
[5] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Clustered Federated Learning Methods Based on DBSCAN Clustering [J]. Computer Science, 2022, 49(6A): 232-237.
[6] DU Hui, LI Zhuo, CHEN Xin. Incentive Mechanism for Hierarchical Federated Learning Based on Online Double Auction [J]. Computer Science, 2022, 49(3): 23-30.
[7] WANG Xin, ZHOU Ze-bao, YU Yun, CHEN Yu-xu, REN Hao-wen, JIANG Yi-bo, SUN Ling-yun. Reliable Incentive Mechanism for Federated Learning of Electric Metering Data [J]. Computer Science, 2022, 49(3): 31-38.
[8] LIU Wei, LI Dong-kun, XU Chang, TIAN Zhao, SHE Wei. Channel Assignment Algorithm Based on Particle Swarm Optimization in Emergency Communication Networks [J]. Computer Science, 2021, 48(5): 277-282.
[9] JI Xiao-xiang, SHEN Hang, BAI Guang-wei. Non-orthogonal Multiple Access Enabled Scalable Video Multicast in HetNets [J]. Computer Science, 2021, 48(11): 356-362.
[10] TAO Yang,JI Rui-juan,YANG Li,WANG Jin. Study on Dynamic Priority Admission Control Algorithm in Heterogeneous Wireless Networks [J]. Computer Science, 2020, 47(3): 242-247.
[11] GUO Bin, YU Dan-dan, LU Wei, HUANG Ming-he, ZENG Ya-lin. Selective Network Coding Strategy Based on Packet Loss Prediction [J]. Computer Science, 2019, 46(11A): 399-404.
[12] BIE Yu-xia, BU Rui-jie and LIU Hai-yan. Channel Allocation Algorithm of Multi-priority Satellite Network [J]. Computer Science, 2017, 44(3): 132-136.
[13] LI Xiang-yang, ZHAO Hang-sheng, ZHAO Xiao-long and ZHANG Yang. Distributed Channel Allocation Algorithm for Multi-services Systems [J]. Computer Science, 2016, 43(Z6): 272-275.
[14] ZHANG Yuan-yuan, WANG Jian and XIAO Chuang-bai. Optimal and Dynamic Resource Management Scheme for Inter-domain Heterogeneous Wireless Networks [J]. Computer Science, 2016, 43(Z11): 290-295.
[15] ZHANG Yuan-yuan and WANG Jian. Multi-domain Optimal Resource Management in Heterogeneous Wireless Networks [J]. Computer Science, 2016, 43(4): 76-80.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.