Computer Science ›› 2025, Vol. 52 ›› Issue (3): 188-196.doi: 10.11896/jsjkx.240100213

• Database & Big Data & Data Science • Previous Articles     Next Articles

FedRCD:A Clustering Federated Learning Algorithm Based on Distribution Extraction andCommunity Detection

WANG Ruicong, BIAN Naizheng, WU Yingjun   

  1. College of Computer Science and Electronic Engineering,Hunan University,Changsha 410082,China
  • Received:2024-01-30 Revised:2024-07-02 Online:2025-03-15 Published:2025-03-07
  • About author:WANG Ruicong,born in 1998,master.His main research interests include fe-derated learning and edge computing.
    BIAN Naizheng,born in 1969,master,associate professor.His main research interests include blockchain,federated learning,software engineering,and big data.
  • Supported by:
    Science and Technology Progress and Innovation Plan of the Department of Transportation of Hunan Province in 2021(202101-E-34).

Abstract: Clustering clients and conducting federated learning within clusters is an effective method to mitigate the poor perfor-mance of traditional federated learning algorithms in non-independently and identically distributed(Non-IID) data scenarios.Such methods primarily utilize the parameters of a client’s local model to characterize data features,and evaluate similarity through the “distance” between parameters,thereby realizing client clustering.However,due to the permutation invariance of neurons in neural networks,this could lead to inaccurate clustering results.Moreover,these methods typically require a predetermined number of clusters,which might result in unreasonable clusters,or they may require clustering during the algorithmic iterative process,lea-ding to substantial communication overhead.After in-depth analysis of the shortcomings of existing methods,a novel federated learning algorithm named FedRCD is proposed.This algorithm combines autoencoders and K-Means algorithms,directly extracting distribution information from a client’s dataset to represent its characteristics,thereby reducing reliance on model parameters.FedRCD also organizes the relationships between clients into a graph structure,and employs the Louvain algorithm to construct client clustering relationships.This process does not require pre-setting the number of clusters,which makes the clustering results more reasonable.Experimental results show that FedRCD can more effectively unearth latent clustering relationships between clients.In a variety of Non-IID data scenarios,compared to other federated learning algorithms,it significantly improves the training effect of neural networks.On the CIFAR10 dataset,the accuracy of FedRCD surpasses the classical FedAvg algorithm by 37.08%,and even outperforms the newly released FeSEM algorithm by 1.89%,demonstrating superior fairness performance.

Key words: Federated learning, Non-IID, Distribution extraction, Community detection, Louvain algorithm

CLC Number: 

  • TP393.4
[1]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//PMLR.2017:1273-1282.
[2]ZHU H,XU J,LIU S,et al.Federated learning on non-IID data:A survey[J].Neurocomputing,2021,465:371-390.
[3]GUO G J,TIAN H,PI H J,et al.Advances in Federated Lear-ning for Non-independent Identically Distributed Data[J].Journal of Chinese Computer Systems,2023,44(11):2442-2449
[4]ZHAO Y,LI M,LAI L,et al.Federated learning with non-iiddata[J].arXiv:1806.00582,2018.
[5]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[C]//Proceedings of Machine Learning and Systems.2020:429-450.
[6]KARIMIREDDY S P,KALE S,MOHRI M,et al.Scaffold:Stochastic controlled averaging for federated learning[C]//International Conference on Machine Learning.PMLR,2020:5132-5143.
[7]ARIVAZHAGAN M G,AGGARWAL V,SINGH A K,et al.Federated learning with personalization layers[J].arXiv:1912.00818,2019.
[8]ZOU M H,GAN Z X.Federated Learning Algorithm for Non-IID Data with Partial Device Participation[J].Journal of Chinese Computer Systems.2023,44(6):1121-1127.
[9]HUANG Y,CHU L,ZHOU Z,et al.Personalized cross-silo fe-derated learning on non-iid data[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:7865-7873.
[10]LONG G,XIE M,SHEN T,et al.Multi-Center FederatedLearning:Clients Clustering for Better Personalization[J].ar-Xiv:2005.01026,2023.
[11]GHOSH A,CHUNG J,YIN D,et al.An efficient frameworkfor clustered federated learning[J].Advances in Neural Information Processing Systems,2020,33:19586-19597.
[12]SATTLER F,MÜLLER K R,SAMEK W.Clustered federated learning:Model-agnostic distributed multitask optimization under privacy constraints[J].IEEE transactions on neural networks and learning systems,2020,32(8):3710-3722.
[13]DENNIS D K,LI T,SMITH V.Heterogeneity for the win:One-shot federated clustering[C]//International Conference on Machine Learning.PMLR,2021:2611-2620.
[14]LIU B,GUO Y,CHEN X.PFA:Privacy-preserving federated adaptation for effective model personalization[C]//Proceedings of the Web Conference 2021.2021:923-934.
[15]YANG Z Q,ZHANG Y G,ZHENG Y,et al.FedFed:Feature distillation against data heterogeneity in federated learning[J].arXiv:2310.05077,2024.
[16]LI Q,ZHANG L Y,MENG X Y.A resource-efficient clustering collaborative federated client selection[J/OL].https://doi.org/10.13229/j.cnki.jdxbgxb.20231369.
[17]LI R J,YAN Q.Inter-cluster Optimization for Cluster Federated Learning[J].Computer Science,2023,50(S2):543-547.
[18]HINTON G E,SALAKHUTDINOV R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
[19]ZHOU Z H.Machine Learing[M].Beijing:TsingHua University Press,2016:225-242.
[20]BLONDEL V D,GUILLAUME J L,LAMBIOTTE R,et al.Fast unfolding of communities in large networks[J].Journal of Statistical Mechanics:Theory and Experiment,2008,2008(10):P10008.
[21]LIU T Y.Distributed Machine Learning theories,algorithms,and systems[M].Beijing:China Machine Press,2018:44-46.
[22]MCINNES L,HEALY J,MELVILLE J.Umap:Uniform manifold approximation and projection for dimension reduction[J].arXiv:1802.03426,2018.
[23]LECUN Y,BOSER B,DENKER J S,et al.Backpropagation applied to handwritten zip code recognition[J].Neural computation,1989,1(4):541-551.
[24]RUMELHART D E,HINTON G E,WILLIAMS R J.Learning representations by back-propagating errors[J].Nature,1986,323(6088):533-536.
[25]LI T,SANJABI M,BEIRAMI A,et al.Fair resource allocation in federated learning[J].arXiv:1905.10497,2019.
[1] LUO Zhengquan, WANG Yunlong, WANG Zilei, SUN Zhenan, ZHANG Kunbo. Study on Active Privacy Protection Method in Metaverse Gaze Communication Based on SplitFederated Learning [J]. Computer Science, 2025, 52(3): 95-103.
[2] HU Kangqi, MA Wubin, DAI Chaofan, WU Yahui, ZHOU Haohao. Federated Learning Evolutionary Multi-objective Optimization Algorithm Based on Improved NSGA-III [J]. Computer Science, 2025, 52(3): 152-160.
[3] WANG Dongzhi, LIU Yan, GUO Bin, YU Zhiwen. Edge-side Federated Continuous Learning Method Based on Brain-like Spiking Neural Networks [J]. Computer Science, 2025, 52(3): 326-337.
[4] XIE Jiachen, LIU Bo, LIN Weiwei , ZHENG Jianwen. Survey of Federated Incremental Learning [J]. Computer Science, 2025, 52(3): 377-384.
[5] ZHENG Jianwen, LIU Bo, LIN Weiwei, XIE Jiachen. Survey of Communication Efficiency for Federated Learning [J]. Computer Science, 2025, 52(2): 1-7.
[6] WANG Xin, XIONG Shubo, SUN Lingyun. Federated Graph Learning:Problems,Methods and Challenges [J]. Computer Science, 2025, 52(1): 362-373.
[7] DUN Jingbo, LI Zhuo. Survey on Transmission Optimization Technologies for Federated Large Language Model Training [J]. Computer Science, 2025, 52(1): 42-55.
[8] LIU Yuming, DAI Yu, CHEN Gongping. Review of Federated Learning in Medical Image Processing [J]. Computer Science, 2025, 52(1): 183-193.
[9] LI Zhi, LIN Sen, ZHANG Qiang. Edge Cloud Computing Approach for Intelligent Fault Detection in Rail Transit [J]. Computer Science, 2024, 51(9): 331-337.
[10] ZANG Hongrui, YANG Tingting, LIU Hongbo, MA Kai. Study on Cryptographic Verification of Distributed Federated Learning for Internet of Things [J]. Computer Science, 2024, 51(6A): 230700217-5.
[11] SUN Min, DING Xining, CHENG Qian. Federated Learning Scheme Based on Differential Privacy [J]. Computer Science, 2024, 51(6A): 230600211-6.
[12] TAN Zhiwen, XU Ruzhi, WANG Naiyu, LUO Dan. Differential Privacy Federated Learning Method Based on Knowledge Distillation [J]. Computer Science, 2024, 51(6A): 230600002-8.
[13] LIU Dongqi, ZHANG Qiong, LIANG Haolan, ZHANG Zidong, ZENG Xiangjun. Study on Smart Grid AMI Intrusion Detection Method Based on Federated Learning [J]. Computer Science, 2024, 51(6A): 230700077-8.
[14] WANG Chenzhuo, LU Yanrong, SHEN Jian. Study on Fingerprint Recognition Algorithm for Fairness in Federated Learning [J]. Computer Science, 2024, 51(6A): 230800043-9.
[15] ZHOU Tianyang, YANG Lei. Study on Client Selection Strategy and Dataset Partition in Federated Learning Basedon Edge TB [J]. Computer Science, 2024, 51(6A): 230800046-6.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!