计算机科学 ›› 2022, Vol. 49 ›› Issue (3): 39-45.doi: 10.11896/jsjkx.210800054

• 新兴分布式计算技术与系统* 上一篇    下一篇

面向多层无线边缘环境下的联邦学习通信优化的研究

赵罗成, 屈志昊, 谢在鹏   

  1. 河海大学计算机与信息学院 南京211100
  • 收稿日期:2021-08-05 修回日期:2021-09-06 出版日期:2022-03-15 发布日期:2022-03-15
  • 通讯作者: 屈志昊(quzhihao@hhu.edu.cn)
  • 作者简介:(cheng114yang@hhu.edu.cn)
  • 基金资助:
    中央高校业务费(B200202176,B210202079);中国博士后基金面上项目(2019M661709)

Study on Communication Optimization of Federated Learning in Multi-layer Wireless Edge Environment

ZHAO Luo-cheng, QU Zhi-hao, XIE Zai-peng   

  1. School of Computer and Information,Hohai University,Nanjing 211100,China
  • Received:2021-08-05 Revised:2021-09-06 Online:2022-03-15 Published:2022-03-15
  • About author:ZHAO Luo-cheng,born in 1998,postgraduate.His main research interests include distributed computing and fe-derated learning.
    QU Zhi-hao,born in 1989,assistant researcher.His main research interests include federated learning,cloud edge collaboration and distributed machine learning.
  • Supported by:
    Fundamental Research Funds for the Central Universities(B200202176,B210202079) and Project Funded by China Postdoctoral Science Foundation(2019M661709).

摘要: 现有的联邦学习模型同步方法大多基于单层的参数服务器架构,难以适应当前异构无线网络场景,同时存在单点通信负载过重、系统延展性差等问题。针对这些问题,文中提出了一种面向边缘混合无线网络的联邦学习高效模型同步方法。在混合无线网络环境中,边缘移动终端将本地模型传输给附近的小型基站,小型基站收到边缘移动终端模型后执行聚合算法,并将聚合后的模型发送给宏基站完成全局模型更新。考虑到信道性能的异构性和数据传输在无线信道上的竞争关系,文中提出了一种新型的分组异步模型同步方法,并设计了基于传输速率感知的信道分配算法。在真实的数据集上进行了实验,实验结果表明,与传统的模型更新算法相比,所提基于分组异步模型更新的信道分配方法可以缩短25%~60%的训练通信时间,大幅度提升了联邦学习的训练效率。

关键词: 联邦学习, 模型聚合, 信道分配, 异步更新, 异构无线网络

Abstract: Existing model synchronization mechanisms of federated learning (FL) are mostly based on single-layer parameter server architecture,which are difficult to adapt to current heterogeneous wireless network scenarios.There are some problems such as excessive communication load on single-point and poor scalability of FL.In response to these problems,this paper proposes an efficient model synchronization scheme for FL in hybrid wireless edge networks.In a hybrid edge wireless network,edge devices transmit local models to nearby small base stations.After receiving local models from edge devices,small base stations exe-cute the aggregation algorithm and send the aggregated models to the macro base station to update the global model.Considering the heterogeneity of channel performance and the competitive relationship of data transmission on the wireless channel,this paper proposes a new type of grouping asynchronous model synchronization scheme and designs a transmission rate aware channel allocation algorithm.Experiments are carried out on real data sets.Experimental results show that the proposed transmission rate aware channel allocation algorithm in grouping asynchronous model synchronization scheme can reduce communication time by 25%~60% and greatly improve the training efficiency of FL.

Key words: Asynchronous update, Channel allocation, Federated learning, Heterogeneous wireless network, Model aggregation

中图分类号: 

  • TP393.1
[1]MOHANTA B K,JENA D,SATAPATHY U,et al.Survey onIoT security:Challenges and solution using machine learning,artificial intelligence and blockchain technology[J].Internet of Things,2020,11:100227-100244.
[2]LI S,DA X L,ZHAO S.5G Internet of Things:A survey[J].Journal of Industrial Information Integration,2018,10:1-9.
[3]QADRI Y A,NAUMAN A,ZIKRIA Y B,et al.The future ofhealthcare internet of things:a survey of emerging technologies[J].IEEE Communications Surveys & Tutorials,2020,22(2):1121-1167.
[4]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial intelligence and statistics.PMLR,2017:1273-1282.
[5]NIKNAM S,DHILLON H S,REED J H.Federated learning for wireless communications:Motivation,opportunities,and challenges[J].IEEE Communications Magazine,2020,58(6):46-51.
[6]CHEN M,YANG Z,SAAD W,et al.A joint learning and communications framework for federated learning over wireless networks[J].IEEE Transactions on Wireless Communications,2020,20(1):269-283.
[7]KONČ ENÝ J,MCMAHAN H B,YU F X,et al.Federatedlearning:Strategies for improving communication efficiency[J].arXiv:1610.05492,2016.
[8]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[9]CHEN M,POOR H V,SAAD W,et al.Convergence time optimization for federated learning over wireless networks[J].IEEE Transactions on Wireless Communications,2020,20(4):2457-2471.
[10]TRAN N H,BAO W,ZOMAYA A,et al.Federated learningover wireless networks:Optimization model design and analysis[C]//IEEE INFOCOM 2019-IEEE Conference on Computer Communications.IEEE,2019:1387-1395.
[11]PILLA L L.Optimal task assignment for heterogeneous federated learning devices[C]//2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS).IEEE,2021:661-670.
[12]JIANG P,AGRAWAL G.A linear speedup analysis of distributed deep learning with sparse and quantized communication[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.2018:2530-2541.
[13]WADU M M,SAMARAKOON S,BENNIS M.Joint clientscheduling and resource allocation under channel uncertainty in federated learning[J].IEEE Transactions on Communications,2021,69(9):5962-5974.
[14]YANG H H,LIU Z,QUEK T Q S,et al.Scheduling policies for federated learning in wireless networks[J].IEEE Transactions on Communications,2019,68(1):317-333.
[15]XIA W,QUEK T Q S,GUO K,et al.Multi-armed bandit-based client scheduling for federated learning[J].IEEE Transactions on Wireless Communications,2020,19(11):7108-7123.
[16]NISHIO T,YONETANI R.Client selection for federated lear-ning with heterogeneous resources in mobile edge[C]//2019 IEEE International Conference on Communications (ICC 2019).IEEE,2019:1-7.
[17]QU Z,GUO S,WANG H,et al.Partial synchronization to acce-lerate federated learning over relay-assisted edge networks[J/OL].IEEE Transactions on Mobile Computing.https://ieeexplore.ieee.org/abstract/document/9439928.
[18]ABAD M S H,OZFATURA E,GUNDUZ D,et al.Hierarchical federated learning across heterogeneous cellular networks[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2020:8866-8870.
[19]LIU L,ZHANG J,SONG S H,et al.Client-edge-cloud hierarchical federated learning[C]//2020 IEEE International Confe-rence on Communications (ICC 2020).IEEE,2020:1-6.
[20]YUAN J,XU M,MA X,et al.Hierarchical federated learning through lan-wan orchestration[J].arXiv:2010.11612,2020.
[21]LIM W Y B,NG J S,XIONG Z,et al.Decentralized edge intelligence:a dynamic resource allocation framework for hierarchical federated learning[J].IEEE Transactions on Parallel and Distributed Systems,2021,33(3):536-550.
[22]LUO S,CHEN X,WU Q,et al.Hfel:Joint edge association and resource allocation for cost-efficient hierarchical federated edge learning[J].IEEE Transactions on Wireless Communications,2020,19(10):6535-6548.
[23]WU W,HE L,LIN W,et al.SAFA:A semi-asynchronous protocol for fast federated learning with low overhead[J].IEEE Transactions on Computers,2020,70(5):655-668.
[24]XIE C,KOYEJO S,GUPTA I.Asynchronous federated optimization[J].arXiv:1903.03934,2019.
[25]CHEN Y,NING Y,SLAWSKI M,et al.Asynchronous online federated learning for edge devices with non-iid data[C]//2020 IEEE International Conference on Big Data.IEEE,2020:15-24.
[26]CHEN Y,SUN X,JIN Y.Communication-efficient federateddeep learning with layerwise asynchronous model update and temporally weighted aggregation[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(10):4229-4238.
[1] 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩.
基于分层抽样优化的面向异构客户端的联邦学习
Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients
计算机科学, 2022, 49(9): 183-193. https://doi.org/10.11896/jsjkx.220500263
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 陈明鑫, 张钧波, 李天瑞.
联邦学习攻防研究综述
Survey on Attacks and Defenses in Federated Learning
计算机科学, 2022, 49(7): 310-323. https://doi.org/10.11896/jsjkx.211000079
[4] 闫萌, 林英, 聂志深, 曹一凡, 皮欢, 张兰.
一种提高联邦学习模型鲁棒性的训练方法
Training Method to Improve Robustness of Federated Learning
计算机科学, 2022, 49(6A): 496-501. https://doi.org/10.11896/jsjkx.210400298
[5] 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩.
基于DBSCAN聚类的集群联邦学习方法
Clustered Federated Learning Methods Based on DBSCAN Clustering
计算机科学, 2022, 49(6A): 232-237. https://doi.org/10.11896/jsjkx.211100059
[6] 杜辉, 李卓, 陈昕.
基于在线双边拍卖的分层联邦学习激励机制
Incentive Mechanism for Hierarchical Federated Learning Based on Online Double Auction
计算机科学, 2022, 49(3): 23-30. https://doi.org/10.11896/jsjkx.210800051
[7] 王鑫, 周泽宝, 余芸, 陈禹旭, 任昊文, 蒋一波, 孙凌云.
一种面向电能量数据的联邦学习可靠性激励机制
Reliable Incentive Mechanism for Federated Learning of Electric Metering Data
计算机科学, 2022, 49(3): 31-38. https://doi.org/10.11896/jsjkx.210700195
[8] 刘炜, 李东坤, 徐畅, 田钊, 佘维.
应急通信网络中基于粒子群优化的信道分配算法
Channel Assignment Algorithm Based on Particle Swarm Optimization in Emergency Communication Networks
计算机科学, 2021, 48(5): 277-282. https://doi.org/10.11896/jsjkx.200400042
[9] 吉晓祥, 沈航, 白光伟.
异构无线网络中基于非正交多址的可伸缩视频多播机制
Non-orthogonal Multiple Access Enabled Scalable Video Multicast in HetNets
计算机科学, 2021, 48(11): 356-362. https://doi.org/10.11896/jsjkx.200900080
[10] 陶洋,纪瑞娟,杨理,王进.
异构无线网络中动态优先级接纳控制算法研究
Study on Dynamic Priority Admission Control Algorithm in Heterogeneous Wireless Networks
计算机科学, 2020, 47(3): 242-247. https://doi.org/10.11896/jsjkx.190100089
[11] 郭斌, 余丹丹, 卢伟, 黄明和, 曾雅琳.
一种基于丢包预测的选择性网络编码策略
Selective Network Coding Strategy Based on Packet Loss Prediction
计算机科学, 2019, 46(11A): 399-404.
[12] 刘萍,袁培燕.
移动自组织认知网络中的路由选择与信道分配方法
Routing Selection and Channel Assignment Method for Mobile Ad Hoc Cognitive Network
计算机科学, 2017, 44(3): 140-144. https://doi.org/10.11896/j.issn.1002-137X.2017.03.031
[13] 别玉霞,卜瑞杰,刘海燕.
多优先级的卫星网络信道分配算法
Channel Allocation Algorithm of Multi-priority Satellite Network
计算机科学, 2017, 44(3): 132-136. https://doi.org/10.11896/j.issn.1002-137X.2017.03.029
[14] 刘蔚,赵宇,陈锐.
无线Ad hoc网络中基于0-1优化的两步骤资源分配算法
Zero-One Integer Programming Based Optimization Model and Two-phase Resource Optimization Algorithm for Wireless Ad hoc Networks
计算机科学, 2017, 44(1): 103-108. https://doi.org/10.11896/j.issn.1002-137X.2017.01.020
[15] 李湘洋,赵杭生,赵小龙,张阳.
分布式多业务系统信道分配算法
Distributed Channel Allocation Algorithm for Multi-services Systems
计算机科学, 2016, 43(Z6): 272-275. https://doi.org/10.11896/j.issn.1002-137X.2016.6A.065
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!