Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (12): 343-348.doi: 10.11896/jsjkx.221100111

• Information Security • Previous Articles     Next Articles

Contribution-based Federated Learning Approach for Global Imbalanced Problem

WU Fei1, SONG Yibo2, JI Yimu2, XU Xi2, WANG Musen2, JING Xiaoyuan3   

  1. 1 College of Automation,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    2 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    3 School of Computer Science,Wuhan University,Wuhan 430072,China
  • Received:2022-11-11 Revised:2023-02-17 Online:2023-12-15 Published:2023-12-07
  • About author:WU Fei,born in 1989,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include pattern recognition and machine learning.
  • Supported by:
    National Natural Science Foundation of China(62076139),Open Research Project of Zhejiang Lab(2021KF0AB05),Future Network Scientific Research Fund Project(FNSRFP-2021-YB-15) and 1311 Talent Program of Nanjing University of Posts and Telecommunications.

PDF (PC)

2542

Abstract: Under the premise of protecting the data privacy,federated learning unites multiple parties to train together to improve the accuracy of the global model.Class imbalance of data is a challenging problem in the federated learning paradigm.Data imba-lance in federated learning can be divided into local data imbalance and global data imbalance.At present,there are few researches on global data imbalance.This paper proposes a contribution-based federated learning approach for global imbalance problem(CGIFL).First,a contribution-based global discriminant loss is designed to adjust the model optimization direction in the local training process and make models give more attention to the global minority classes in training to improve the generalization ability of models.And a contribution-based dynamic federated aggregation algorithm is designed to optimize the participation weight of each node and better balance the updating direction of the global model.Experimental results on MNIST,CIFAR10 and CIFAR100 datasets demonstrate the effectiveness of CGIFL in solving the problem of global data imbalance.

Key words: Federated learning, Data imbalance, Multi-party coordination, Image classification

CLC Number: 

  • TP391
[1]MOHAMMADI M,AL-FUQAHA A,SOROUR S,et al.Deeplearning for IoT big data and streaming analytics:A survey[J].IEEE Communications Surveys & Tutorials,2018,20(4):2923-2960.
[2]YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Concept and applications[J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19.
[3]WANG L,XU S,WANG X,et al.Addressing class imbalance in federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021,35(11):10165-10173.
[4]KAIROUZ P,MCMAHAN H B,AVENT B,et al.Advancesand open problems in federated learning[J].Foundations and Trends in Machine Learning,2021,14(1/2):214-217.
[5]ZHONG Y,DENG W,WANG M,et al.Unequal-training for deep face recognition with long-tailed noisy data[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:7812-7821.
[6]LENG Z,TAN M,LIU C,et al.PolyLoss:A Polynomial Expansion Perspective of Classification Loss Functions[J].arXiv:2204.12511,2022.
[7]DENG L.The mnist database of handwritten digit images formachine learning research[best of the web][J].IEEE Signal Processing Magazine,2012,29(6):141-142.
[8]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[R].Technical report,University of Toronto,2009.
[9]BENNIN K E,KEUNG J,PHANNACHITTA P,et al.Ma-hakil:Diversity based oversampling approach to alleviate the class imbalance issue in software defect prediction[J].IEEE Transactions on Software Engineering,2017,44(6):534-550.
[10]LIU X Y,WU J,ZHOU Z H.Exploratory undersampling forclass-imbalance learning[J].IEEE Transactions on Systems,Man,and Cybernetics,Part B(Cybernetics),2008,39(2):539-550.
[11]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988.
[12]WU Y,LIU H,LI J,et al.Deep face recognition with center invariant loss[C]//Proceedings of the on Thematic Workshops of ACM Multimedia 2017.2017:408-414.
[13]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[J].Proceedings of Machine Learning and Systems,2020,2:429-450.
[14]LI Q,HE B,SONG D.Model-contrastive federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10713-10722.
[15]ZHAO Y,LI M,LAI L,et al.Federated learning with non-iid data[J].arXiv:1806.00582,2018.
[16]SARKAR D,NARANG A,RAI S.Fed-focal loss for imbalanced data classification in Federated Learning[J].arXiv:2011.06283,2020.
[17]DUAN M,LIU D,CHEN X,et al.Self-balancing federatedlearning with global imbalanced data in mobile systems[J].IEEE Transactions on Parallel and Distributed Systems,2020,32(1):59-71.
[18]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[19]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[20]MCMAHAN H B,MOORE E,RAMAGE D,et al.Federated learning of deep networks using model averaging[J].arXiv:1602.05629,2016.
[1] ZHAO Yuhao, CHEN Siguang, SU Jian. Privacy-enhanced Federated Learning Algorithm Against Inference Attack [J]. Computer Science, 2023, 50(9): 62-67.
[2] LIN Xinyu, YAO Zewei, HU Shengxi, CHEN Zheyi, CHEN Xing. Task Offloading Algorithm Based on Federated Deep Reinforcement Learning for Internet of Vehicles [J]. Computer Science, 2023, 50(9): 347-356.
[3] LI Rongchang, ZHENG Haibin, ZHAO Wenhong, CHEN Jinyin. Data Reconstruction Attack for Vertical Graph Federated Learning [J]. Computer Science, 2023, 50(7): 332-338.
[4] LI Fan, JIA Dongli, YAO Yumin, TU Jun. Graph Neural Network Few Shot Image Classification Network Based on Residual and Self-attention Mechanism [J]. Computer Science, 2023, 50(6A): 220500104-5.
[5] ZHANG Lianfu, TAN Zuowen. Robust Federated Learning Algorithm Based on Adaptive Weighting [J]. Computer Science, 2023, 50(6A): 230200188-9.
[6] WANG Xianwang, ZHOU Hao, ZHANG Minghui, ZHU Youwei. Hyperspectral Image Classification Based on Swin Transformer and 3D Residual Multilayer Fusion Network [J]. Computer Science, 2023, 50(5): 155-160.
[7] ZHONG Jialin, WU Yahui, DENG Su, ZHOU Haohao, MA Wubin. Multi-objective Federated Learning Evolutionary Algorithm Based on Improved NSGA-III [J]. Computer Science, 2023, 50(4): 333-342.
[8] XIE Qinqin, HE Lang, XU Ruli. Classification of Oil Painting Art Style Based on Multi-feature Fusion [J]. Computer Science, 2023, 50(3): 223-230.
[9] CHEN Luoxuan, LIN Chengchuang, ZHENG Zhaoliang, MO Zefeng, HUANG Xinyi, ZHAO Gansen. Review of Transformer in Computer Vision [J]. Computer Science, 2023, 50(12): 130-147.
[10] TANG Junkun, ZHANG Hui, ZHANG Zhouquanand WU Tianyue. Image Classification for Unsupervised Domain Adaptation Based on Task Relevant FeatureSeparation Network [J]. Computer Science, 2023, 50(11A): 230100068-8.
[11] LI Renjie, YAN Qiao. Inter-cluster Optimization for Cluster Federated Learning [J]. Computer Science, 2023, 50(11A): 221000243-5.
[12] WANG Chundong, DU Yingqi, MO Xiuliang, FU Haoran. Enhanced Federated Learning Frameworks Based on CutMix [J]. Computer Science, 2023, 50(11A): 220800021-8.
[13] ZHANG Lianfu, TAN Zuowen. Federated Learning Privacy-preserving Approach for Multimodal Medical Data [J]. Computer Science, 2023, 50(11A): 230800021-8.
[14] FENG Chen, GU Jingjing. Efficient Distributed Training Framework for Federated Learning [J]. Computer Science, 2023, 50(11): 317-326.
[15] XU Wentao, WANG Binjun. Backdoor Defense of Horizontal Federated Learning Based on Random Cutting and GradientClipping [J]. Computer Science, 2023, 50(11): 356-363.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.