基于双重抗遗忘机制的轻量化联邦持续学习方法

doi:10.11896/jsjkx.250500116

摘要/Abstract

摘要： 联邦学习在不共享数据的前提下,通过上传并聚合客户端模型实现不同客户端之间的知识共享。现有的联邦学习方法大多假设客户端数据是已知且固定的。然而,在现实场景中,客户端会不断地接收包含新类别数据的任务并更新模型,导致模型在旧任务上的表现持续下滑,即发生灾难性遗忘问题。为有效应对这一严峻挑战,研究者将持续学习方法引入联邦学习中,衍生出联邦持续学习这一研究方向。然而,随着客户端所接收的任务数量不断增加,现有联邦持续学习方法在缓解灾难性遗忘问题上的效果逐渐变差,尤其是在针对较为久远的任务时,准确率出现了大幅下降,且数据异构程度的提升也进一步削弱了模型的准确率表现。鉴于此,设计了本地-全局双重抗遗忘机制,以缓解模型在久远任务上的遗忘问题。具体而言,在客户端层面引入特定于任务的轻量化模块,有效克服了数据变化与模型更新引发的灾难性遗忘;在服务器端通过模型反演生成并筛选得到类别均衡的伪图像,缓解了数据分布差异导致的模型性能下降的问题。在CIFAR10,CIFAR100和TinyImageNet等数据集上开展了一系列实验,实验结果有力地证实了该机制的优越性,充分表明其相较于现有各类方法,在提升模型性能、缓解灾难性遗忘等方面具有显著优势。

关键词: 联邦持续学习, 灾难性遗忘, 数据异构, 轻量化模块, 本地-全局抗遗忘

Abstract: Federated learning(FL) enables knowledge sharing among different clients by uploading and aggregating client modelswithout sharing data.However,existing FL methods generally assume that client data is known and fixed.In reality,clients continuously receive tasks with new category data and update their models,which leads to a continuous decline in model performance on old tasks,known as catastrophic forgetting.To address this severe challenge,researchers have introduced continual learning(CL) into FL,giving rise to the research direction of federated continual learning(FCL).Nevertheless,as the number of tasks received by clients increases,existing FCL methods become less effective in alleviating catastrophic forgetting,especially for tasks that are relatively distant in time,where accuracy drops significantly.Moreover,the increasing degree of data heterogeneity further weakens model accuracy.To address this issue,this paper proposes a local-global anti-forgetting mechanism to mitigate the forgetting problem on distant tasks.Specifically,it introduces task-specific lightweight modules at the client level to effectively overcome catastrophic forgetting caused by data changes and model updates.At the server level,it generates and filters category-balanced pseudo-images through model inversion to alleviate the decline in model performance due to data distribution diffe-rences.Through a series of experiments conducted on CIFAR-10,CIFAR-100,and TinyImageNet datasets,the results strongly demonstrate the superiority of the proposed mechanism.Compared with existing methods,it shows significant advantages in improving model performance and alleviating catastrophic forgetting.

Key words: Federated continual learning, Catastrophic forgetting, Data heterogeneity, Lightweight modules, Local-global anti forgetting

中图分类号:

TP301

王攀, 王吉, 钟正仪, 包卫东, 张耀鸿. 基于双重抗遗忘机制的轻量化联邦持续学习方法[J]. 计算机科学, 2026, 53(4): 424-434. https://doi.org/10.11896/jsjkx.250500116

WANG Pan, WANG Ji, ZHONG Zhengyi, BAO Weidong, ZHANG Yaohong. Lightweight Federated Continual Learning Method Based on Double Anti-forgetting Mechanism[J]. Computer Science, 2026, 53(4): 424-434. https://doi.org/10.11896/jsjkx.250500116

参考文献

[1]DE CAPITANI DI VIMERCATI S,FORESTI S,LIVRAGA G,et al.Data privacy:Definitions and techniques[J].International Journal of Uncertainty,Fuzziness and Knowledge-Based Systems,2012,20(6):793-817.
[2]PARDAU S L.The california consumer privacy act:Towards a European-style privacy regime in the united states[J].Journal of Technology Law & Policy,2018,23:68.
[3]VOIGT P,VON DEM BUSSCHE A.The EU general data protection regulation(GDPR)[M].Cham:Springer,2017.
[4]CHI J L,FENG D G,ZHANG M.,et al.Research Progress on Privacy-Preserving Ciphertext Retrieval Technology[J].Journal of Electronics and Information,2024,46(5):1-24.
[5]FIENBERG S E,SLAVKOVIĆ A B.Data privacy and confidentiality[M]//International Encyclopedia of Statistical Science.Berlin:Springer,2025:615-619.
[6]OMID P,SOREN F.The Digital Double:Data Privacy,Security,and Consent in AI Implants West[J].Journal of Dental Sciences,2025,2(1):108.
[7]BALOGUN A Y.Strengthening compliance with data privacyregulations in US healthcare cybersecurity[J].Asian Journal of Research in Computer Science,2025,18(1):154-173.
[8]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[9]YANG Q.AI and Data Privacy Protection:Solution of Federated Learning[J].Information Security Research,2019,5(11):961.
[10]DEMBANI R,KARVELAS I,AKBAR N A,et al.Agricultural data privacy and federated learning:A review of challenges and opportunities[J].Computers and Electronics in Agriculture,2025,232:110048.
[11]KEMKER R,MCCLURE M,ABITINO A,et al.Measuring catastrophic forgetting in neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[12]LI Y C,WANG H Z,XU W C,et al.Unleashing the power of continual learning on non-centralized devices:A survey[J].IEEE Communications Surveys & Tutorials,2025,28:1059-1098.
[13]QU H,RAHMANI H,XU L,et al.Recent advances of continual learning in computer vision:An overview[J].IET Computer Vision,2025,19(1):e70013.
[14]YANG X,YU H,GAO X,et al.Federated continual learning via knowledge fusion:A survey[J].IEEE Transactions on Know-ledge and Data Engineering,2024,36(8):3832-3850.
[15]YOON J,JEONG W,LEE G,et al.Federated continual learning with weighted inter-client transfer[C]//International Confe-rence on Machine Learning.PMLR,2021:12073-12086.
[16]ZHANG J,CHEN C,ZHUANG W,et al.Target:Federatedclass-continual learning via exemplar-free distillation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:4782-4793.
[17]FANG Z X,FU X D,DING J M,et al.Federated Class-incremental Learning for Unbalanced Data[J].Journal of Chinese Computer Systems,2025,46(9):2121-2129.
[18]TONG G,LI G,WU J,et al.GradMFL:Gradient memory-based federated learning for hierarchical knowledge transferring over non-iid data[C]//International Conference on Algorithms and Architectures for Parallel Processing.Cham:Springer,2021:612-626.
[19]ZHANG Z,ZHANG Y,GUO D,et al.Communication-efficient federated continual learning for distributed learning system with Non-IID data[J].Science China Information Sciences,2023,66(2):122102.
[20]CICIRELLO V,HU K,LU M,et al.A Federated Incremental Learning Algorithm Based on Dual Attention Mechanism[J].Applied Sciences,2022,12(19),10025.
[21]USMANOVA A,PORTET F,LALANDA P,et al.A distil-lation-based approach integrating continual learning and federated learning for pervasive services[J].arXiv:2109.04197,2021.
[22]BABAKNIYA S,FABIAN Z,HE C,et al.A data-free approach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks[J].Advances in Neural Information Processing Systems,2023,36:66408-66425.
[23]GUBEROVIĆ E,ALEXOPOULOS C,BOSNIĆ I,et al.Frame-work for federated learning open models in e-government applications[J].Interdisciplinary Description of Complex Systems:INDECS,2022,20(2):162-178.
[24]FATHIMAA S,BASHA S M,AHMED S T,et al.Empowering consumer healthcare through sensor-rich devices using federated learning for secure resource recommendation[J].IEEE Transactions on Consumer Electronics,2025,71(1):1563-1570.
[25]KONEČNÝ J,MCMAHAN H B,YU F X,et al.Federated learning:Strategies for improving communication efficiency[J].arXiv:1610.05492,2016.
[26]HSIEH K,HARLAP A,VIJAYKUMAR N,et al.Gaia:{Geo-Distributed} machine learning approaching {LAN} speeds[C]//14th USENIX Symposium on Networked Systems Design and Implementation(NSDI 17).2017:629-647.
[27]LUPI NG W,WEI W,BO L I.CMFL:Mitigating communication overhead for federated learning[C]//2019 IEEE 39th International Conference on Distributed Computing Systems(ICDCS).IEEE,2019:954-964.
[28]BONAWITZ K,EICHNER H,GRIESKAMP W,et al.Towards federated learning at scale:System design[J].Proceedings of Machine Learning and Systems,2019,1:374-388.
[29]LIU Y,JAMES J Q,KANG J,et al.Privacy-preserving traffic flow prediction:A federated learning approach[J].IEEE Internet of Things Journal,2020,7(8):7751-7763.
[30]TANG L T,WANG D,ZHANG L F,et al.A Federated Lear-ning Scheme Based on Secure Multi-Party Computation and Differential Privacy[J].Computer Science,2022,49(9):297-305.
[31]SHEN W,HUANG W,WAN G,et al.Label-free backdoor attacks in vertical federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2025:20389-20397.
[32]MAI Z,LI R,JEONG J,et al.Online continual learning in image classification:An empirical survey[J].Neurocomputing,2022,469:28-51.
[33]MALLYA A,LAZEBNIK S.Packnet:Adding multiple tasks to a single network by iterative pruning[C]//Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition.2018:7765-7773.
[34]ALJUNDI R,CHAKRAVARTY P,TUYTELAARS T.Expert gate:Lifelong learning with a network of experts[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3366-3375.
[35]ROLNICK D,AHUJA A,SCHWARZ J,et al.Experience replay for continual learning[C]//Proceedings of the 33rd Internatio-nal Conference on Neural Information Processing Systems.2019:350-360.
[36]SHIN H,LEE J K,KIM J,et al.Continual learning with deep generative replay[J].arXiv:1705.08690,2017.
[37]LI Z,HOIEM D.Learning without forgetting[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(12):2935-2947.
[38]KIRKPATRICK J,PASCANU R,RABINOWITZ N,et al.Overcoming catastrophic forgetting in neural networks[J].Proceedings of the National Academy of Sciences,2017,114(13):3521-3526.
[39]DONG J,WANG L,FANG Z,et al.Federated class-incremental learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:10164-10173.
[40]ALEX KRIZHEVSKY.Cifar-10 and cifar-100 datasets[EB/OL].http://www.cs.toronto.edu/kriz/cifar.html.
[41]LE Y,YANG X.Tiny imagenet visual recognition challenge[EB/OL].http://vision.stanford.edu/teaching/cs231n/reports/2015/pdfs/yle_project.pdf.
[42]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[J].Proceedings of Machine Learning and Systems,2020,2:429-450.
[43]YU H,YANG X,ZHANG L,et al.Handling spatial-temporal data heterogeneity for federated continual learning via tail anchor[C]//Proceedings of the Computer Vision and Pattern Re-cognition Conference.2025:4874-4883.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed