关键词: 个性化联邦学习, 长尾分布, 数据异构性, 参数解耦, 特征增强, 优化策略

Abstract: Aiming at the problem of model performance degradation in federated learning caused by long tail distribution and he-terogeneity of data,a novel personalized federated learning framework called Balanced Personalized Federated Learning(BPFed) is proposed,where the whole federated learning process is divided into two stages:representation learning based on personalized fe-derated learning and personalized classifiers retraining based on global feature augmentation.In the first stage,the Mixup strategy is adopted firstly for data augmentation,and then a feature extractor training method is proposed based on the personalized fede-rated learning with parameter decoupling to optimize the performance of the feature extractor while reducing the communication cost.In the second stage,a new class-level feature augmentation method based on global covariance matrix is proposed firstly,and then the classifiers of clientsare retaines individually in balance with a proposed label-aware smoothing loss function based on sample weight to correct the overconfidence for head classes and boost the generalization ability for tail classes.Extensive experimental results show that the model accuracy of BPFed is significantly improved compared with other representative related algorithms under different settings of data long-tailed distributions and heterogeneities.Moreover,the effectiveness of the proposed methods and optimization strategies is further verified by the experiments on ablation and hyperparameter influence.

Key words: Personalized federated learning, Long-tailed distribution, Data heterogeneity, Parameter decoupling, Feature augmentation, Optimization strategy