计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (7): 433-441.doi: 10.11896/jsjkx.250900003

• 信息安全 • 上一篇    

面向概念与标签分布双偏斜的个性化联邦学习方法

平奉沁, 付晓东   

  1. 昆明理工大学信息工程与自动化学院 昆明 650500
  • 收稿日期:2025-09-01 修回日期:2025-11-20 出版日期:2026-07-15 发布日期:2026-07-10
  • 通讯作者: 付晓东(xiaodong_fu@hotmail.com)
  • 作者简介:(20232104100@stu.kust.edu.cn)
  • 基金资助:
    国家自然科学基金(62362043,62262036);云南省“兴滇英才支持计划”项目(KKXY202203008);云南省科技计划项目(202502AD080003,202503AA080013)

Personalized Federated Learning for Concept and Label Distribution Drift

PING Fengqin, FU Xiaodong   

  1. Faculty of Information Engineering and Automation,Kunming University of Science and Technology,Kunming 650500,China
  • Received:2025-09-01 Revised:2025-11-20 Published:2026-07-15 Online:2026-07-10
  • About author:PING Fengqin,born in 2001,postgra-duate,is a member of CCF(No.A02867G).His main research interests include federated learning and so on.
    FU Xiaodong,born in 1975,Ph.D,professor,Ph.D supervisor.His main research interests include services computing and federated learning.
  • Supported by:
    National Natural Science Foundation of China(62362043,62262036),Xingdian Talent Support Project(KKXY202203008) and Science and Technology Plan Projects of Yunnan Province(202502AD080003,202503AA080013).

PDF (PC)

19

摘要： 个性化联邦学习(Personalized Federated Learning,PFL)通过为各客户端保留定制模型来应对数据异构问题。现有工作主要缓解标签分布偏斜,未考虑非时序概念偏斜及两者叠加所造成的梯度冲突,因此全局模型常表现为收敛缓慢、泛化退化。为此,提出一种层次聚合-双原型负蒸馏方法,用于提升此类极端异构场景下的个性化联邦学习的性能。首先,通过短轮次训练预热、互评与多数票统计,无需额外隐私信息即可在早期准确标定概念异常客户端;随后,在可信客户端集合内利用局部模型主成分计算条件分布相似度与边缘分布互补度,构造动态权重并递归执行层次化聚合,兼顾语义一致性与特征多样性;最后,从可信与异常客户端中提取正、负类别原型生成伪样本,对全局模型施加交叉熵与基于间隔的负蒸馏联合损失,同步强化正确语义并显式抑制冲突概念,同时对异常客户端实施原型蒸馏微调以保持个性化准确率。在 Fashion-MNIST,CIFAR-10 及 CIFAR-100 的概念偏斜场景下对所提方法进行评测,其相对多项主流方法将全局准确率平均提升3.6个百分点,且通信轮次与经典平均聚合相当。研究结论表明,所提方法能够在复杂双偏斜环境下同时提升全局泛化性与本地适应性。

关键词: 联邦学习, 个性化联邦学习, 标签分布偏斜, 非时序概念偏斜, 知识蒸馏

Abstract: Personalized federated learning(PFL) combats data heterogeneity by retaining custom models for each client.Existing works primarily address label distribution drift but rarely consider non-temporal concept drift or the gradient conflicts caused by their combination,leading to slow convergence and degraded generalization in global models.This study proposes a hierarchical aggregation-dual prototype negative distillation method to enhance PFL performance in such extreme heterogeneous scenarios.The method first identifies concept-drifted clients early on through short-term training preheating,peer evaluation,and majority voting,without the need for additional private information.Then,within the trusted client set,it calculates the similarity of conditional distributions and complementarity of marginal distributions using the local model's principal components.This dynamic weighting is used in recursive hierarchical aggregation,balancing semantic consistency with feature diversity.Finally,positive and negative category prototypes are extracted from both trusted and abnormal clients to generate pseudo-samples,and a combined loss of cross-entropy and margin-based negative distillation is applied to the global model,simultaneously reinforcing correct semantics and explicitly suppressing conflicting concepts.Prototype distillation fine-tuning is also applied to abnormal clients to maintain personalized accuracy.Experiments conducted on Fashion-MNIST,CIFAR-10,and CIFAR-100 datasets in concept drift scenarios show that the proposed method achieves an average global accuracy improvement of 3.6 percentage points and the communication rounds are comparable to those of classical averaging aggregation.The research concludes that this method effectively enhances both global generalization and local adaptability in complex dual-bias environments.

Key words: Federated learning, Personalized federated learning, Label distribution drift, Non-temporal concept drift, Knowledge distillation

中图分类号: 

  • TP181
[1]GUO X,WANG D D,FENG D,et al.A Verifiable Privacy Protection Federated Learning Scheme Based on Homomorphic Encryption[J].Journal of Electronics & Information Technology,2025,47(4):1113-1125.
[2]HE Z P,XV J,DAI H,et al.A Review of Federated LearningApplication Technologies[J].Chinese Journal of NETINFO SECURITY,2024,24(12):1831-1844
[3]XU R Z,TONG Y M,DAI L P.Research on Federated Learning Adaptive Differential Privacy Method Based on Heterogeneous Data[J].Netinfo Security,2025,25(1):63-77.
[4]MARFOQ O,NEGLIA G,VIDAL R,et al.Personalized federated learning through local memorization[C]//Proceedings of the International Conference on Machine Learning.New York:ACM,2022:17-23.
[5]ZHANG J,HUA Y,WANG H,et al.Fedala:Adaptive local aggregation for personalized federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI,2023:11237-11244.
[6]LI Z,SHANG X,HE R,et al.No fear of classifier biases:Neural collapse inspired federated learning with synthetic and fixed classifier[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Piscataway,NJ:IEEE Computer Society,2023:5296-5306.
[7]JEON I,HONG M,YUN J,et al.Federated learning via meta-variational dropout[C]//Advances in Neural Information Processing Systems.New York:Curran Associates Inc.,2023:11168-11193.
[8]CASADO F E,LEMA D,CRIADO M F,et al.Concept drift detection and adaptation for federated and continual learning[J].Multimedia Tools and Applications,2022,81(3):3397-3419.
[9]CHEN J,XUE J,WANG Y,et al.Classifier Clustering and Feature Alignment for Federated Learning under Distributed Concept Drift[C]//Advances in Neural Information Processing Systems.New York:Curran Associates Inc.,2024:81360-81388.
[10]JOTHIMURUGESAN E,HSIEH K,WANG J,et al.Federated learning under distributed concept drift[C]//International Conference on Artificial Intelligence and Statistics.New York:PMLR,2023:5834-5853.
[11]TAN A Z,YU H,CUI L,et al.Towards personalized federated learning[J].IEEE Transactions on Neural Networks and Lear-ning Systems,2022,34(12):9587-9603.
[12]SMITH V,CHIANG C K,SANJABI M,et al.Federated multi-task learning[C]//Advances in Neural Information Processing Systems.New York:Curran Associates Inc.,2017:4424-4434.
[13]XU J,TONG X,HUANG S L.Personalized Federated Learning with Feature Alignment and Classifier Collaboration[C]//International Conference on Learning Representations Inc..2023.
[14]OH J,KIM S,YUN S Y.Fedbabu:Towards enhanced representation for federated image classification[C]//International Conference on Learning Representations.2022.
[15]BIETTI A,WEI C Y,DUDIK M,et al.Personalization improves privacy-accuracy tradeoffs in federated learning[C]//Procee-dings of the International Conference on Machine Learning.New York:ACM,2022:1945-1962.
[16]YANG X,HUANG W,YE M.Dynamic personalized federated learning with adaptive differential privacy[C]//Advances in Neural Information Processing Systems.New York:Curran Associates Inc.,2023:72181-72192.
[17]LI T,HU S,BEIRAMI A,et al.Ditto:Fair and robust federated learning through personalization[C]//Proceedings of the International Conference on Machine Learning. New York:ACM,2021:6357-6368.
[18]COLLINS L,HASSANI H,MOKHTARI A,et al.Exploiting shared representations for personalized federated learning[C]//Proceedings of the International Conference on Machine Lear-ning.New York:ACM,2021:2089-2099.
[19]DINH C T,TRAN N,NGUYEN J.Personalized federatedlearning with moreau envelopes[C]//Advances in Neural Information Processing Systems.New York:Curran Associates Inc.,2020:21394-21405.
[20]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.New York:PMLR,2017:1273-1282.
[21]CASADO F E,LEMA D,IGLESIAS R,et al.Ensemble and continual federated learning for classification tasks[J].Machine Learning,2023,112(9):3413-3453.
[22]KANG M,KIM S,JIN K H,et al.FedNN:Federated learning on concept drift data using weight and adaptive group normalizations[J].Pattern Recognition,2024,149:110230.
[23]CHOWT,RAZA U,MAVROMATIS I,et al.FLARE:Detection and mitigation of concept drift for federated learning based IoT deployments[C]//International Wireless Communications and Mobile Computing.Piscataway,NJ:IEEE Computer Society,2023:989-995.
[24]XU J,CHEN Z,QUEK T Q S,et al.Fedcorr:Multi-stage fede-rated learning for label noise correction[C]//IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2022:10184-10193.
[25]WU N,YU L,JIANG X,et al.FedNoRo:Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneity[C]//Proceedings of International Joint Conference on Artificial Intelligence.San Francisco:Morgan Kaufmann,2023:4424-4432.
[26]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[J].Proceedings of Machine Learning and Systems,2020,2:429-450.
[27]KARIMIREDDY S P,KALE S,MOHRI M,et al.Scaffold:Stochastic controlled averaging for federated learning[C]//Proceedings of the International Conference on Machine Learning.New York:ACM,2020:5132-5143.
[28]XIAO H,RASUL K,VOLLGRAF R.Fashion-mnist:a novelimage dataset for benchmarking machine learning algorithms[J].arXiv:1708.07747,2017.
[29]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[EB/OL].https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
[30]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2016:770-778.
[31]CANONACO G,BERGAMASCO A,MONGELLUZZO A,et al.Adaptive federated learning in presence of concept drift[C]//International Joint Conference on Neural Networks.Piscataway,NJ:IEEE Computer Society,2021:1-7.
[32]PU J,FU X D,DONG H,et al.Dynamic Adaptive FederatedLearning on Local Long-Tailed Data[J].IEEE Transactions on Services Computing,2024,17(6):3485-3498.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发