计算机科学 ›› 2024, Vol. 51 ›› Issue (6A): 230800046-6.doi: 10.11896/jsjkx.230800046
周天阳, 杨磊
ZHOU Tianyang, YANG Lei
摘要: 联邦学习是分布式机器学习在现实中的应用之一。针对联邦学习中的异构性,基于FedProx算法,提出优先选择近端项较大的客户端选择策略,效果优于常见的选择局部损失值较大的客户端选择策略,可以有效提高FedProx算法在异构数据和系统下的收敛速度,提高有限聚合次数内的准确率。针对联邦学习数据异构的假设,设计了一套异构数据划分流程,得到了基于真实图像数据集的异构联邦数据集作为实验数据集。使用开源的分布式机器学习框架Edge-TB作为实验测试平台,以异构划分后的Cifar10作为数据集,实验表明,采用新的客户端选择策略的改进FedProx算法较原算法在有限的聚合轮数内准确率提升14.96%,通信开销减小6.3%;与SCAFFOLD算法相比,准确率提升3.6%,通信开销减小51.7%,训练时间减少15.4%。
中图分类号:
[1]MCMAHAN B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282. [2]LI T,SAHU A K,TALWALKAR A,et al.Federated learning:Challenges,methods,and future directions[J].IEEE Signal Processing Magazine,2020,37(3):50-60. [3]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[J].Proceedings of Machine Learning and Systems,2020,2:429-450. [4]WANG J,LIU Q,LIANG H,et al.Tackling the objective inconsistency problem in heterogeneous federated optimization[J].Advances in Neural Information Processing Systems,2020,33:7611-7623. [5]KARIMIREDDY S P,KALE S,MOHRI M,et al.Scaffold:Stochastic controlled averaging for federated learning[C]//International Conference on Machine Learning.PMLR,2020:5132-5143. [6]XIE C,KOYEJO S,GUPTA I.Asynchronous federated optimization[J].arXiv:1903.03934,2019. [7]NISHIO T,YONETANI R.Client selection for federated lear-ning with heterogeneous resources in mobile edge[C]//2019 IEEE International Conference on Communications(ICC 2019).IEEE,2019:1-7. [8]RIBERO M,VIKALO H.Communication-efficient federatedlearning via optimal client sampling[J].arXiv:2007.15197,2020. [9]CHEN W,HORVATH S,RICHTARIK P.Optimal client sampling for federated learning[J].arXiv:2010.13723,2020. [10]CHO Y J,WANG J,JOSHI G.Client selection in federatedlearning:Convergence analysis and power-of-choice selection strategies[J].arXiv:2010.01243,2020. [11]LAI F,ZHU X,MADHYASTHA H V,et al.Oort:EfficientFederated Learning via Guided Participant Selection[C]//OSDI.2021:19-35. [12]FRABONI Y,VIDAL R,KAMENI L,et al.Clustered sampling:low-variance and improved represent ativity for clients selection in federated learning[C]//International Conference on Machine Learning.New York:PMLR,2021:3407-3416. [13]WANG H,KAPLAN Z,NIU D,et al.Optimizing federatedlearning on non-iid data with reinforcement learning[C]//IEEE Conference on Computer Communications(INFOCOM 2020).IEEE,2020:1698-1707. [14]CALDAS S,DUDDU S M K,WU P,et al.Leaf:A benchmark for federated settings[J].arXiv:1812.01097,2018. [15]ZHAO Y,LI M,LAI L,et al.Federated learning with non-iiddata[J].arXiv:1806.00582,2018. [16]ZHU H,XU J,LIU S,et al.Federated learning on non-IID data:A survey[J].Neurocomputing,2021,465:371-390. [17]LI Q,DIAO Y,CHEN Q,et al.Federated learning on non-iid data silos:An experimental study[C]//2022 IEEE 38th International Conference on Data Engineering(ICDE).IEEE,2022:965-978. [18]YANG L,WEN F,CAO J,et al.Edgetb:A hybrid testbed for distributed machine learning at the edge with high fidelity[J].IEEE Transactions on Parallel and Distributed Systems,2022,33(10):2540-2553. |
|