计算机科学 ›› 2023, Vol. 50 ›› Issue (12): 343-348.doi: 10.11896/jsjkx.221100111
吴飞1, 宋一波2, 季一木2, 胥熙2, 王木森2, 荆晓远3
WU Fei1, SONG Yibo2, JI Yimu2, XU Xi2, WANG Musen2, JING Xiaoyuan3
摘要： 联邦学习在保护各方数据隐私的前提下,协同多方共同训练,提高了全局模型的精度。数据的类不平衡问题是联邦学习范式中具有挑战的问题,联邦学习中的数据不平衡问题可分为局部数据不平衡和全局数据不平衡,目前针对全局数据不平衡问题的相关研究较少。文中提出了一种面向全局不平衡问题的基于贡献度的联邦学习方法(CGIFL)。首先,设计了一种基于贡献度的全局判别损失函数,用于调整本地训练过程中的模型优化方向,使模型在训练中给予全局少数类更多的关注,以提高模型的泛化能力;然后,在全局模型更新阶段,设计了一种基于贡献度的动态联邦汇聚策略,优化了各节点的参与权重,更好地平衡了全局模型的更新方向。在MNIST,CIFAR10和CIFAR100这3个数据集上进行实验,实验结果表明了CGIFL在解决全局数据不平衡问题上的有效性。
中图分类号:
| [1]MOHAMMADI M,AL-FUQAHA A,SOROUR S,et al.Deeplearning for IoT big data and streaming analytics:A survey[J].IEEE Communications Surveys & Tutorials,2018,20(4):2923-2960. [2]YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Concept and applications[J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19. [3]WANG L,XU S,WANG X,et al.Addressing class imbalance in federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021,35(11):10165-10173. [4]KAIROUZ P,MCMAHAN H B,AVENT B,et al.Advancesand open problems in federated learning[J].Foundations and Trends in Machine Learning,2021,14(1/2):214-217. [5]ZHONG Y,DENG W,WANG M,et al.Unequal-training for deep face recognition with long-tailed noisy data[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:7812-7821. [6]LENG Z,TAN M,LIU C,et al.PolyLoss:A Polynomial Expansion Perspective of Classification Loss Functions[J].arXiv:2204.12511,2022. [7]DENG L.The mnist database of handwritten digit images formachine learning research[best of the web][J].IEEE Signal Processing Magazine,2012,29(6):141-142. [8]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[R].Technical report,University of Toronto,2009. [9]BENNIN K E,KEUNG J,PHANNACHITTA P,et al.Ma-hakil:Diversity based oversampling approach to alleviate the class imbalance issue in software defect prediction[J].IEEE Transactions on Software Engineering,2017,44(6):534-550. [10]LIU X Y,WU J,ZHOU Z H.Exploratory undersampling forclass-imbalance learning[J].IEEE Transactions on Systems,Man,and Cybernetics,Part B(Cybernetics),2008,39(2):539-550. [11]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988. [12]WU Y,LIU H,LI J,et al.Deep face recognition with center invariant loss[C]//Proceedings of the on Thematic Workshops of ACM Multimedia 2017.2017:408-414. [13]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[J].Proceedings of Machine Learning and Systems,2020,2:429-450. [14]LI Q,HE B,SONG D.Model-contrastive federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10713-10722. [15]ZHAO Y,LI M,LAI L,et al.Federated learning with non-iid data[J].arXiv:1806.00582,2018. [16]SARKAR D,NARANG A,RAI S.Fed-focal loss for imbalanced data classification in Federated Learning[J].arXiv:2011.06283,2020. [17]DUAN M,LIU D,CHEN X,et al.Self-balancing federatedlearning with global imbalanced data in mobile systems[J].IEEE Transactions on Parallel and Distributed Systems,2020,32(1):59-71. [18]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324. [19]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778. [20]MCMAHAN H B,MOORE E,RAMAGE D,et al.Federated learning of deep networks using model averaging[J].arXiv:1602.05629,2016. |
|
||
首页