计算机科学 ›› 2024, Vol. 51 ›› Issue (3): 271-279.doi: 10.11896/jsjkx.230100125
黄楠, 李冬冬, 姚佳, 王喆
HUANG Nan, LI Dongdong, YAO Jia, WANG Zhe
摘要： 针对联邦增量场景中持续学习和数据安全的问题,构建了结合元学习的去中心化联邦增量学习框架。首先,为解决增量场景中持续学习带来的灾难性遗忘问题,提出了结合最近类均值样本回放的增量元学习方法NMR-cMAML,利用元训练对不同任务流的快速适应进行元更新,得到适用于新旧样本的模型。然后,为解决联邦增量场景中的数据安全问题,设计了基于对等网络架构的去中心化联邦增量学习框架,对等架构中每个客户端采用NMR-cMAML对私有的持续任务流进行增量学习。不同于传统的基于服务器-客户端的中心化架构,该去中心化架构采用客户端间通信的策略,消除了传统中央服务器易被攻击的隐患;同时,在联邦通信过程中,通过共享元学习的模型参数实现客户端间知识的有效迁移。最后在图像数据集(Cifar100和Imagenet50)上进行了不同任务场景的实验,结果表明所提方法能在提高系统的数据安全性的同时提升客户端本地性能。
中图分类号:
| [1]LIN J,YU W,ZHANG N,et al.A Survey on Internet ofThings:Architecture,Enabling Technologies,Security and Privacy,and Applications[J].IEEE Internet of Things Journal,2017,4(5):1125-1142. [2]ZHU H,XU J,LIU S,et al.Federated learning on non-IID data:A survey[J].Neurocomputing,2021,465:371-390. [3]RAHMAN S A,TOUT H,MOURAD A,et al.FedMCCS:Multicriteria Client Selection Model for Optimal IoT Federated Learning[J].IEEE Internet of Things Journal,2021,8(6):4723-4735. [4]YANG Q,LIU Y,CHEN T,et al.Federated Machine Learning:Concept and Applications[J].ACM Trans.Intell.Syst.Technol.,2019,10(2):12:1-12:19. [5]MAI Z,LI R,JEONG J,et al.Online continual learning in image classification:An empirical survey[J].Neurocomputing,2022,469:28-51. [6]LIU L Y,QIAN H,XING H J,et al.Incremental Classification Model Based on Q-Learning Algorithm[J].Computer Science,2020,47(8):171-177. [7]QI X M,WU Y B,JIANG X L.Federated Data Augmentation Algorithm for Non-independent and Identical Distributed Data[J].Computer Science,2022,49(12):33-39. [8]LI Z Z,HOLEM D.Learning without forgetting[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(2):2935-2947. [9]YAO X,SUN L.Continual Local Training For Better Initialization Of Federated Models[C]//2020 IEEE International Confe-rence on Image Processing(ICIP).Abu Dhabi,United Arab Emirates:IEEE,2020:1736-1740. [10]YOON J,JEONG W,LEE G,et al.Federated Continual Learning with Weighted Inter-client Transfer[C]//Proceedings of the 38th International Conference on Machine Learning(ICML).Virtual:PMLR,2021:12073-12086. [11]CASADO F,LEMA D,LGLESIAS R,et al.Federated and continual learning for classification tasks in a society of devices[J].arXiv:2006.07129,2020. [12]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-Efficient Learning of Deep Networks from Decentralized Data[C]//Artificial Intelligence and Statistics(AISTATS).Ft.Lauderdale,FL,USA:PMLR,2017:1273-1282. [13]LI T,SAHU A,ZAHEER M,et al.Federated optimization in heterogeneous networks[J].Proceedings of Machine Learning and Systems,2020,2:429-450. [14]ZANTEDESCHI V,BELLET A,TOMMASI M.Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs[C]//The 23rd International Conference on Artificial Intelligence and Statistics(AISTATS 2020).Palermo,Sicily,Italy:PMLR,2020:864-874. [15]HEGEDUS I,DANNER G,JELASITY M.Decentralized lear-ning works:An empirical comparison of gossip learning and fe-derated learning[J].Journal of Parallel and Distributed Computing,2021(148):109-124. [16]WARNET S,SCHULTZE H,SHASTR K L,et al.SwarmLearning for decentralized and confidential clinical machine learning[J].Nature,2021,594(7862):265-270. [17]HEGEDUS I,DANNER G,JELASITY M.Gossip Learning as a Decentralized Alternative to Federated Learning[C]//Distributed Applications and Interoperable Systems:19th IFIP WG 6.1 International Conference(DAIS).Kongens Lyngby,Denmark:Springer International Publishing,2019:74-90. [18]MOTHUKURI V,PARIZI R,POURIYEH S,et al.A survey on security and privacy of federated learning[J].Future Gener.Comput.Syst.,2021,115:619-640. [19]LIU Y,LIU J,BASAR T.Differentially Private Gossip Gradient Descent[C]//2018 IEEE Conference on Decision and Control(CDC).Fontainebleau Miami Beach,USA:IEEE,2018:2777-2782. [20]THRUN S,PRATT L.Lifelong Learning Algorithms[C]//Learning to Learn.Boston:Springer,1998:181-209. [21]VEN G,TOLAIS A.Three scenarios for continual learning[J].arXiv:1904.07734,2019. [22]LANGE M,ALJUNDI R,MASANA M,et al.A ContinualLearning Survey:Defying Forgetting in Classification Tasks[J].IEEE Trans.Pattern Anal.Mach.Intell.,2022,44(7):3366-3385. [23]MAI Z,LI R,JEONG J,et al.Online continual learning in image classification:An empirical survey[J].Neurocomputing,2022,469:28-51. [24]LIU H,YANG Y,WANG X.Overcoming Catastrophic Forgetting in Graph Neural Networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Virtual:AAAI,2021:8653-8661. [25]SERRA J,SURIS D,MIRON M,et al.Overcoming catastrophic forgetting with hard attention to the task[C]//Proceedings of the 35th International Conference on Machine Learning(IC-ML).Stockholmsmässan,Sweden:PMLR,2018:4548-4557. [26]ZHOU F,CAO C.Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Virtual:AAAI,2021:4714-4722. [27]RIEMER M,CASES I,AJEMIAN R,et al.Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference[C]//7th International Conference on Learning Representations.New Orleans,USA:ICLR,2019:1-31. [28]ZHANG C Y,SI S J,WANG J Z,et al.Federated Meta Lear-ning:A Review[J].Big Data Research,2023,9(2):112-146. [29]LI F C,LIU Y,WU P X.A Survey onRecent Advances in Meta-Learning[J].Chinese Journal of Computer,2021,44(2):422-446. [30]FINN C,ABBEL P,LEVIINE S.Model-Agnostic Meta-Lear-ning for Fast Adaptation of Deep Networks[C]//International Conference on Machine Learning(ICML).Sydney,Australia:PMLR,2017:1126-1135. |
|
||
首页