Computer Science ›› 2024, Vol. 51 ›› Issue (10): 391-398.doi: 10.11896/jsjkx.230900050

• Information Security • Previous Articles     Next Articles

Collaborative Network and Metric Learning Based Label Noise Robust Federated LearningMethod

WU Fei1, ZHANG Jiabin1, YUE Xiaofan1, JI Yimu2, JING Xiaoyuan3   

  1. 1 College of Automation,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    2 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    3 School of Computer Science,Wuhan University,Wuhan 430072,China
  • Received:2023-09-11 Revised:2024-02-06 Online:2024-10-15 Published:2024-10-11
  • About author:WU Fei,born in 1989,Ph.D,professor.His main research interests include pattern recognition and machine learning.
  • Supported by:
    National Natural Science Foundation of China(62076139),Open Research Project of Zhejiang Lab(2021KF0AB05),Future Network Scientific Research Fund Project(FNSRFP-2021-YB-15) and 1311 Talent Program of Nanjing University of Posts and Telecommunications.

Abstract: Currently,there is limited research on the problem of label noise in federated learning.The main approaches involve introducing a benchmark dataset on the server side to evaluate the client's model,controlling the aggregation weights and feature class centers of the clients.However,most methods still have room for improvement in distinguishing noisy clients or noisy samples.This paper proposes a label-noise robust federated learning method based on co-networks and metric learning.The method consists of the following three parts:1) Client mutual evaluation mechanism.Clients score each other's models,construct a rating matrix,and further transform it into an adjacency matrix to differentiate clean/noisy clients.2) Collaborative network module.By constructing two collaborative equivalent federated network models,the Jensen-Shannon divergence is used to distinguish clean samples from noisy samples for the training of collaborative networks.3) Federated-collaborative network triplet loss.A loss function is designed to constrain the output features of the collaborative networks for the same noisy samples.Experimental verification is conducted on the publicly available datasets CIFAR-10 and CIFAR-100,and the results demonstrate the superiority of the proposed method in accuracy.

Key words: Robust federated learning, Label noise, Collaborative network, Metric learning

CLC Number: 

  • TP391
[1]LI T,SAHU A K,TALWALKAR A,et al.Federated learning:Challenges,methods,and future directions[J].IEEE Signal Processing Magazine,2020,37(3):50-60.
[2]MCMAHAN B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.2017:1273-1282.
[3]YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Concept and applications[J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19.
[4]CHEN Y,YANG X,QIN X,et al.Dealing with label quality dis-parity in federated learning[M]//Federated Learning:Privacy and Incentive.2020:108-121.
[5]YANG S,PARK H,BYUN J,et al.Robust federated learningwith noisy labels[J].IEEE Intelligent Systems,2022,37(2):35-43.
[6]XU J,CHEN Z,QUEK T Q S,et al.Fedcorr:Multi-stage fede-rated learning for label noise correction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:10184-10193.
[7]FUGLEDE B,TOPSOE F.Jensen-Shannon divergence and Hilbert space embedding[C]//International Symposium on Information Theory.2004:31-36.
[8]KRIZHEVSKY A.Learning multiple layers of features from tiny images[D].Toronto:University of Toronto,2009.
[9]TANAKA D,IKAMI D,YAMASAKI T,et al.Joint optimization framework for learning with noisy labels[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5552-5560.
[10]WANG Y,MA X,CHEN Z,et al.Symmetric cross entropy for robust learning with noisy labels[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:322-330.
[11]THULASIDASAN S,BHATTACHARYA T,BILMES J,et al.Combating label noise in deep learning using abstention[C].International Conference on Machine Learning.2019:6234-6243.
[12]GHOSH A,KUMAR H,SASTRY P S.Robust loss functions under label noise for deep neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2017:1919-1925.
[13]HAN B,YAO Q,YU X,et al.Robust training of deep neuralnetworks with extremely noisy labels[J]arXiv:1804.06872,2020.
[14]LI J,SOCHER R,HOI S C H.Dividemix:Learning with noisy labels as semi-supervised learning[C]//International Conference on Learning Representations.2020.
[15]GU N,FAN M,MENG D.Robust semi-supervised classification for noisy labels based on self-paced learning[J].IEEE Signal Processing Letters,2016,23(12):1806-1810.
[16]SMART B,CARNEIRO G.Bootstrapping the Relationship Between Images and Their Clean and Noisy Labels[C]//Procee-dings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2023:5344-5354.
[17]PATEL D,SASTRY P S.Adaptive sample selection for robustlearning under label noise[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2023:3932-3942.
[18]XU M L,JIANG G X,WANG W J.Label noise filtering framework based on outlier detection[J].Computer Science,2024,51(2):87-99.
[19]ISCEN A,VALMADRE J,ARNAB A,et al.Learning withneighbor consistency for noisy labels[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:4672-4681.
[20]HOULE M E.Dimensionality,discriminability,density and distance distributions[C]//International Conference on Data Mi-ning Workshops.2013:468-473.
[21]LI Y,CHEN C,LIU N,et al.A blockchain-based decentralized federated learning framework with committee consensus[J].IEEE Network,2020,35(1):234-241.
[22]CHE C,LI X,CHEN C,et al.A decentralized federated learning framework via committee mechanism with convergence guarantee[J].IEEE Transactions on Parallel and Distributed Systems,2022,33(12):4783-4800.
[23]YOU Y,CHEN T,SUI Y,et al.Graph contrastive learning with augmentations[C]//Advances in Neural Information Processing Systems.2020:5812-5823.
[24]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[1] ZHANG Rui, WANG Ziqi, LI Yang, WANG Jiabao, CHEN Yao. Task-aware Few-shot SAR Image Classification Method Based on Multi-scale Attention Mechanism [J]. Computer Science, 2024, 51(8): 160-167.
[2] JIANG Gaoxia, WANG Fei, XU Hang, WANG Wenjian. Robust Estimation and Filtering Methods for Ordinal Label Noise [J]. Computer Science, 2024, 51(6): 144-152.
[3] XU Maolong, JIANG Gaoxia, WANG Wenjian. Label Noise Filtering Framework Based on Outlier Detection [J]. Computer Science, 2024, 51(2): 87-99.
[4] ZHANG Shuaiyu, PENG Li, DAI Feifei. Person Re-identification Method Based on Progressive Attention Pyramid [J]. Computer Science, 2023, 50(6A): 220200084-8.
[5] ZHANG Lianfu, TAN Zuowen. Robust Federated Learning Algorithm Based on Adaptive Weighting [J]. Computer Science, 2023, 50(6A): 230200188-9.
[6] JIANG Gaoxia, QIN Pei, WANG Wenjian. Noise Estimation and Filtering Methods with Limit Distance [J]. Computer Science, 2023, 50(6): 151-158.
[7] MA Jiye, ZHU Guosheng, WEI Cao, ZENG Yuxuan. Noise Tolerant Algorithm for Network Traffic Classification Method [J]. Computer Science, 2023, 50(11A): 220800120-7.
[8] HOU Xia-ye, CHEN Hai-yan, ZHANG Bing, YUAN Li-gang, JIA Yi-zhen. Active Metric Learning Based on Support Vector Machines [J]. Computer Science, 2022, 49(6A): 113-118.
[9] ZHANG Min, YU Zeng, HAN Yun-xing, LI Tian-rui. Overview of Person Re-identification for Complex Scenes [J]. Computer Science, 2022, 49(10): 138-150.
[10] TENG Jun-yuan, GAO Meng, ZHENG Xiao-meng, JIANG Yun-song. Noise Tolerable Feature Selection Method for Software Defect Prediction [J]. Computer Science, 2021, 48(12): 131-139.
[11] YU Cheng, ZHU Wan-ning, YOU Kun, ZHU Jin-fu. Prediction Model of E-sports Behavior Pattern Based on Attention Mechanism and LRUA Module [J]. Computer Science, 2019, 46(11A): 76-79.
[12] BAO Zong-ming, GONG Sheng-rong, ZHONG Shan, YAN Ran, DAI Xing-hua. Person Re-identification Algorithm Based on Bidirectional KNN Ranking Optimization [J]. Computer Science, 2019, 46(11): 267-271.
[13] WEN Jun-hao, DAI Da-wen, YU Jun-liang, GAO Min, ZHANG Yi-hao. Social Recommendation Method Integrating Matrix Factorization and Distance Metric Learning [J]. Computer Science, 2018, 45(10): 196-201.
[14] LIANG Lu, GONG Ben-long, LI Jian and TENG Shao-hua. Diffusion Method of Sample Points for Alleviating Staggered Situation of Classification [J]. Computer Science, 2017, 44(9): 286-289.
[15] WU Wei, GAO Guang-lai and NIE Jian-yun. Combination of Nearest Neighbor with Semantic Distance for Image Annotation [J]. Computer Science, 2015, 42(1): 297-302.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!