计算机科学 ›› 2024, Vol. 51 ›› Issue (6): 354-363.doi: 10.11896/jsjkx.230400183
刘建勋, 张幸林
LIU Jianxun, ZHANG Xinglin
摘要： 联邦学习(Federated Learning,FL)是一种新兴的分布式机器学习范式,其核心思想是用户设备以分布式的方式在本地训练模型,且无需上传原始数据,仅需将训练后的模型上传到服务器进行模型聚合。现有研究大多忽略了设备的计算资源会随着用户的使用模式而发生时序性变化,这会影响FL的训练进度。文中针对异构设备具有时变计算资源的特点,使用自回归模型对时变计算资源进行建模,并提出了一个设备选择算法。首先构造了长期训练时间约束下最小化每轮FL平均训练时间的优化问题,接着采用李雅普诺夫优化理论对其进行转化,最后求解得到设备选择算法。实验结果表明,与基线算法相比,所提算法能够在基本保证模型质量的同时缩短FL的训练时间和设备的平均等待时间。
中图分类号:
| [1]LIU Y,BI S,SHI Z,et al.When machine learning meets big data:A wireless communication perspective[J].IEEE Vehicular Technology Magazine,2019,15(1):63-72. [2]SUN Y,PENG M,ZHOU Y,et al.Application of machine lear-ning in wireless networks:Key techniques and open issues[J].IEEE Communications Surveys & Tutorials,2019,21(4):3072-3108. [3]YANG Q,LIU Y,CHENG Y,et al.Federated Learning[M].Beijing:Publishing House of Electronics Industry,2020. [4]MCMAHAN B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282. [5]BONAWITZ K,EICHNER H,GRIESKAMP W,et al.Towards federated learning at scale:System design[J].Proceedings of Machine Learning and Systems,2019,1:374-388. [6]WANG H,KAPLAN Z,NIU D,et al.Optimizing federatedlearning on non-iid data with reinforcement learning[C]//IEEE Conference on Computer Communications(INFOCOM 2020).IEEE,2020:1698-1707. [7]NISHIO T,YONETANI R.Client selection for federated lear-ning with heterogeneous resources in mobile edge[C]//2019 IEEE International Conference on Communications(ICC 2019).IEEE,2019:1-7. [8]ZENG Q,DU Y,HUANG K,et al.Energy-efficient radio re-source allocation for federated edge learning[C]//2020 IEEE International Conference on Communications Workshops(ICC Workshops).IEEE,2020:1-6. [9]LI L,XIONG H,GUO Z,et al.SmartPC:Hierarchical pace control in real-time federated learning system[C]//IEEE Real-Time Systems Symposium(RTSS 2019).IEEE,2019:406-418. [10]WANG C,YANG Y,ZHOU P.Towards efficient scheduling of federated mobile devices under computational and statistical he-terogeneity[J].IEEE Transactions on Parallel and Distributed Systems,2020,32(2):394-410. [11]YU L,ALBELAIHI R,SUN X,et al.Jointly optimizing client selection and resource management in wireless federated lear-ning for Internet of things[J].IEEE Internet of Things Journal,2022,9(6):4385-4395. [12]WADU M M,SAMARAKOON S,BENNIS M.Joint clientscheduling and resource allocation under channel uncertainty in federated learning[J].IEEE Transactions on Communications,2021,69(9):5962-5974. [13]CHEN M,YANG Z,SAAD W,et al.A joint learning and communications framework for federated learning over wireless networks[J].IEEE Transactions on Wireless Communications,2021,20(1):269-283. [14]XU J,WANG H.Client selection and bandwidth allocation inwireless federated learning networks:A long-term perspective[J].IEEE Transactions on Wireless Communications,2021,20(2):1188-1200. [15]ABDULRAHMAN S,TOUT H,MOURAD A,et al.Fed-MCCS:Multicriteria client selection model for optimal IoT fe-derated learning[J].IEEE Internet of Things Journal,2021,8(6):4723-4735. [16]BRIGGS C,FAN Z,ANDRAS P.Federated learning with hie-rarchical clustering of local updates to improve training on non-IID data[C]//2020 International Joint Conference on Neural Networks(IJCNN).IEEE,2020:1-9. [17]WU H,WANG P.Node selection toward faster convergence for federated learning on non-iid data[J].IEEE Transactions on Network Science and Engineering,2022,9(5):3099-3111. [18]XIA W,QUEK T Q S,GUO K,et al.Multi-armed bandit-based client scheduling for federated learning[J].IEEE Transactions on Wireless Communications,2020,19(11):7108-7123. [19]HUANG T,LIN W,WU W,et al.An efficiency-boosting client selection scheme for federated learning with fairness guarantee[J].IEEE Transactions on Parallel and Distributed Systems,2021,32(7):1552-1564. [20]ZHU H,ZHOU Y,QIAN H,et al.Onlineclient selection forasynchronous federated learning with fairness consideration[J].IEEE Transactions on Wireless Communications,2023,22(4):2493-2506. [21]LAI F,ZHU X,MADHYASTHA H V,et al.Oort:Efficientfederated learning via guided participant selection[C]//OSDI.2021:19-35. [22]NEELY M J.Stochastic network optimization with applicationto communication and queueing systems[J].Synthesis Lectures on Communication Networks,2010,3(1):1-211. [23]WU W,HE L,LIN W,et al.Accelerating federated learningover reliability-agnostic clients in mobile edge computing systems[J].IEEE Transactions on Parallel and Distributed Systems,2020,32(7):1539-1551. [24]MA Z,XU Y,XU H,et al.Adaptive Batch Size for Federated Learning in Resource-Constrained Edge Computing[J].IEEE Transactions on Mobile Computing,2023,22(1):37-53. |
|
||
首页