计算机科学 ›› 2024, Vol. 51 ›› Issue (10): 391-398.doi: 10.11896/jsjkx.230900050

• 信息安全 • 上一篇    下一篇

基于协同网络与度量学习的标签噪声鲁棒联邦学习方法

吴飞1, 张家宾1, 岳晓凡1, 季一木2, 荆晓远3   

  1. 1 南京邮电大学自动化学院、人工智能学院 南京 210003
    2 南京邮电大学计算机学院 南京 210003
    3 武汉大学计算机学院 武汉 430072
  • 收稿日期:2023-09-11 修回日期:2024-02-06 出版日期:2024-10-15 发布日期:2024-10-11
  • 通讯作者: 吴飞(wufei_8888@126.com)
  • 基金资助:
    国家自然科学基金(62076139);之江实验室开放课题(2021KF0AB05);未来网络科研基金项目(FNSRFP-2021-YB-15);南京邮电大学1311人才计划

Collaborative Network and Metric Learning Based Label Noise Robust Federated LearningMethod

WU Fei1, ZHANG Jiabin1, YUE Xiaofan1, JI Yimu2, JING Xiaoyuan3   

  1. 1 College of Automation,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    2 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    3 School of Computer Science,Wuhan University,Wuhan 430072,China
  • Received:2023-09-11 Revised:2024-02-06 Online:2024-10-15 Published:2024-10-11
  • About author:WU Fei,born in 1989,Ph.D,professor.His main research interests include pattern recognition and machine learning.
  • Supported by:
    National Natural Science Foundation of China(62076139),Open Research Project of Zhejiang Lab(2021KF0AB05),Future Network Scientific Research Fund Project(FNSRFP-2021-YB-15) and 1311 Talent Program of Nanjing University of Posts and Telecommunications.

摘要: 针对联邦学习中标签噪声问题的研究较少,目前的主流方法是,服务器端引入基准数据集对客户端的模型进行评估,对客户端的聚合权重、特征类中心进行控制等,但大多数方法区分噪声客户端/噪声样本的能力尚有提升空间。文中提出了一种基于协同网络与度量学习的标签噪声鲁棒联邦学习方法。该方法由以下3部分组成:1)客户端互评分机制:客户端为彼此模型评分,构建评分矩阵,进一步将其转化为邻接矩阵,以区分干净/噪声客户端。2)协同网络模块:通过构建两个协同对等的联邦网络模型,使用简森-香农散度为协同网络彼此的训练区分干净样本与噪声样本。3)联邦-协同网络三元组损失:为噪声样本设计损失函数,约束同一噪声样本协同网络的输出特征。在CIFAR-10和CIFAR-100两个公开数据集上进行实验验证,结果表明所提方法在准确性上具有优势。

关键词: 鲁棒联邦学习, 标签噪声, 协同网络, 度量学习

Abstract: Currently,there is limited research on the problem of label noise in federated learning.The main approaches involve introducing a benchmark dataset on the server side to evaluate the client's model,controlling the aggregation weights and feature class centers of the clients.However,most methods still have room for improvement in distinguishing noisy clients or noisy samples.This paper proposes a label-noise robust federated learning method based on co-networks and metric learning.The method consists of the following three parts:1) Client mutual evaluation mechanism.Clients score each other's models,construct a rating matrix,and further transform it into an adjacency matrix to differentiate clean/noisy clients.2) Collaborative network module.By constructing two collaborative equivalent federated network models,the Jensen-Shannon divergence is used to distinguish clean samples from noisy samples for the training of collaborative networks.3) Federated-collaborative network triplet loss.A loss function is designed to constrain the output features of the collaborative networks for the same noisy samples.Experimental verification is conducted on the publicly available datasets CIFAR-10 and CIFAR-100,and the results demonstrate the superiority of the proposed method in accuracy.

Key words: Robust federated learning, Label noise, Collaborative network, Metric learning

中图分类号: 

  • TP391
[1]LI T,SAHU A K,TALWALKAR A,et al.Federated learning:Challenges,methods,and future directions[J].IEEE Signal Processing Magazine,2020,37(3):50-60.
[2]MCMAHAN B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.2017:1273-1282.
[3]YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Concept and applications[J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19.
[4]CHEN Y,YANG X,QIN X,et al.Dealing with label quality dis-parity in federated learning[M]//Federated Learning:Privacy and Incentive.2020:108-121.
[5]YANG S,PARK H,BYUN J,et al.Robust federated learningwith noisy labels[J].IEEE Intelligent Systems,2022,37(2):35-43.
[6]XU J,CHEN Z,QUEK T Q S,et al.Fedcorr:Multi-stage fede-rated learning for label noise correction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:10184-10193.
[7]FUGLEDE B,TOPSOE F.Jensen-Shannon divergence and Hilbert space embedding[C]//International Symposium on Information Theory.2004:31-36.
[8]KRIZHEVSKY A.Learning multiple layers of features from tiny images[D].Toronto:University of Toronto,2009.
[9]TANAKA D,IKAMI D,YAMASAKI T,et al.Joint optimization framework for learning with noisy labels[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5552-5560.
[10]WANG Y,MA X,CHEN Z,et al.Symmetric cross entropy for robust learning with noisy labels[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:322-330.
[11]THULASIDASAN S,BHATTACHARYA T,BILMES J,et al.Combating label noise in deep learning using abstention[C].International Conference on Machine Learning.2019:6234-6243.
[12]GHOSH A,KUMAR H,SASTRY P S.Robust loss functions under label noise for deep neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2017:1919-1925.
[13]HAN B,YAO Q,YU X,et al.Robust training of deep neuralnetworks with extremely noisy labels[J]arXiv:1804.06872,2020.
[14]LI J,SOCHER R,HOI S C H.Dividemix:Learning with noisy labels as semi-supervised learning[C]//International Conference on Learning Representations.2020.
[15]GU N,FAN M,MENG D.Robust semi-supervised classification for noisy labels based on self-paced learning[J].IEEE Signal Processing Letters,2016,23(12):1806-1810.
[16]SMART B,CARNEIRO G.Bootstrapping the Relationship Between Images and Their Clean and Noisy Labels[C]//Procee-dings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2023:5344-5354.
[17]PATEL D,SASTRY P S.Adaptive sample selection for robustlearning under label noise[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2023:3932-3942.
[18]XU M L,JIANG G X,WANG W J.Label noise filtering framework based on outlier detection[J].Computer Science,2024,51(2):87-99.
[19]ISCEN A,VALMADRE J,ARNAB A,et al.Learning withneighbor consistency for noisy labels[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:4672-4681.
[20]HOULE M E.Dimensionality,discriminability,density and distance distributions[C]//International Conference on Data Mi-ning Workshops.2013:468-473.
[21]LI Y,CHEN C,LIU N,et al.A blockchain-based decentralized federated learning framework with committee consensus[J].IEEE Network,2020,35(1):234-241.
[22]CHE C,LI X,CHEN C,et al.A decentralized federated learning framework via committee mechanism with convergence guarantee[J].IEEE Transactions on Parallel and Distributed Systems,2022,33(12):4783-4800.
[23]YOU Y,CHEN T,SUI Y,et al.Graph contrastive learning with augmentations[C]//Advances in Neural Information Processing Systems.2020:5812-5823.
[24]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!