计算机科学 ›› 2024, Vol. 51 ›› Issue (10): 391-398.doi: 10.11896/jsjkx.230900050
吴飞1, 张家宾1, 岳晓凡1, 季一木2, 荆晓远3
WU Fei1, ZHANG Jiabin1, YUE Xiaofan1, JI Yimu2, JING Xiaoyuan3
摘要: 针对联邦学习中标签噪声问题的研究较少,目前的主流方法是,服务器端引入基准数据集对客户端的模型进行评估,对客户端的聚合权重、特征类中心进行控制等,但大多数方法区分噪声客户端/噪声样本的能力尚有提升空间。文中提出了一种基于协同网络与度量学习的标签噪声鲁棒联邦学习方法。该方法由以下3部分组成:1)客户端互评分机制:客户端为彼此模型评分,构建评分矩阵,进一步将其转化为邻接矩阵,以区分干净/噪声客户端。2)协同网络模块:通过构建两个协同对等的联邦网络模型,使用简森-香农散度为协同网络彼此的训练区分干净样本与噪声样本。3)联邦-协同网络三元组损失:为噪声样本设计损失函数,约束同一噪声样本协同网络的输出特征。在CIFAR-10和CIFAR-100两个公开数据集上进行实验验证,结果表明所提方法在准确性上具有优势。
中图分类号:
[1]LI T,SAHU A K,TALWALKAR A,et al.Federated learning:Challenges,methods,and future directions[J].IEEE Signal Processing Magazine,2020,37(3):50-60. [2]MCMAHAN B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.2017:1273-1282. [3]YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Concept and applications[J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19. [4]CHEN Y,YANG X,QIN X,et al.Dealing with label quality dis-parity in federated learning[M]//Federated Learning:Privacy and Incentive.2020:108-121. [5]YANG S,PARK H,BYUN J,et al.Robust federated learningwith noisy labels[J].IEEE Intelligent Systems,2022,37(2):35-43. [6]XU J,CHEN Z,QUEK T Q S,et al.Fedcorr:Multi-stage fede-rated learning for label noise correction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:10184-10193. [7]FUGLEDE B,TOPSOE F.Jensen-Shannon divergence and Hilbert space embedding[C]//International Symposium on Information Theory.2004:31-36. [8]KRIZHEVSKY A.Learning multiple layers of features from tiny images[D].Toronto:University of Toronto,2009. [9]TANAKA D,IKAMI D,YAMASAKI T,et al.Joint optimization framework for learning with noisy labels[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5552-5560. [10]WANG Y,MA X,CHEN Z,et al.Symmetric cross entropy for robust learning with noisy labels[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:322-330. [11]THULASIDASAN S,BHATTACHARYA T,BILMES J,et al.Combating label noise in deep learning using abstention[C].International Conference on Machine Learning.2019:6234-6243. [12]GHOSH A,KUMAR H,SASTRY P S.Robust loss functions under label noise for deep neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2017:1919-1925. [13]HAN B,YAO Q,YU X,et al.Robust training of deep neuralnetworks with extremely noisy labels[J]arXiv:1804.06872,2020. [14]LI J,SOCHER R,HOI S C H.Dividemix:Learning with noisy labels as semi-supervised learning[C]//International Conference on Learning Representations.2020. [15]GU N,FAN M,MENG D.Robust semi-supervised classification for noisy labels based on self-paced learning[J].IEEE Signal Processing Letters,2016,23(12):1806-1810. [16]SMART B,CARNEIRO G.Bootstrapping the Relationship Between Images and Their Clean and Noisy Labels[C]//Procee-dings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2023:5344-5354. [17]PATEL D,SASTRY P S.Adaptive sample selection for robustlearning under label noise[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2023:3932-3942. [18]XU M L,JIANG G X,WANG W J.Label noise filtering framework based on outlier detection[J].Computer Science,2024,51(2):87-99. [19]ISCEN A,VALMADRE J,ARNAB A,et al.Learning withneighbor consistency for noisy labels[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:4672-4681. [20]HOULE M E.Dimensionality,discriminability,density and distance distributions[C]//International Conference on Data Mi-ning Workshops.2013:468-473. [21]LI Y,CHEN C,LIU N,et al.A blockchain-based decentralized federated learning framework with committee consensus[J].IEEE Network,2020,35(1):234-241. [22]CHE C,LI X,CHEN C,et al.A decentralized federated learning framework via committee mechanism with convergence guarantee[J].IEEE Transactions on Parallel and Distributed Systems,2022,33(12):4783-4800. [23]YOU Y,CHEN T,SUI Y,et al.Graph contrastive learning with augmentations[C]//Advances in Neural Information Processing Systems.2020:5812-5823. [24]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778. |
|