Computer Science ›› 2023, Vol. 50 ›› Issue (11A): 220800021-8.doi: 10.11896/jsjkx.220800021

• Information Security • Previous Articles     Next Articles

Enhanced Federated Learning Frameworks Based on CutMix

WANG Chundong, DU Yingqi, MO Xiuliang, FU Haoran   

  1. National Engineering Laboratory for Computer Virus Prevention and Control Technology,Tianjin 300384,China
    Engineering Research Center of Learning-Based Intelligent System,Ministry of Education,Tianjin 300384,China
    School of Computer Science and Engineering,Tianjin University of Technology,Tianjin 300384,China
  • Published:2023-11-09
  • About author:WANG Chundong,born in 1969,Ph.D,professor,is a senior member of China Computer Federation.His main researchinterests includecyberspace security,blockchain technology,etc.

Abstract: The emergence of federated learning solves the problem of "data silos" in traditional machine learning.Federated learning enables the training of collective models while protecting the privacy of the client's local data.When the client’s dataset is independently identically distributed(IID) data,federated learning can achieve an accuracy similar to that of centralized machine learning.However,in real scenarios,due to differences in client devices and geographic locations,there are often cases where client’s dataset contain noisy data and non-independent identical distribution(Non-IID).Therefore,this paper proposes a CutMix-based federated learning framework,namely CutMix enhanced federated learning(CEFL),which first filters out noisy data through data cleaning algorithms and then trains through CutMix-based data enhancement.Compared with the traditional federated learning algorithm,the accuracy of CutMix enhanced federated learning can be improved by 23% and 19% for the model on Non-IID dataset.

Key words: Federated learning, Non-independent identically distributed data, Data cleaning, Data augmentation, Saliency detection

CLC Number: 

  • TP183
[1]RAGHU M,SCHMIDT E.A Survey of Deep Learning forScientific Discovery[J].arXiv:2003.11755,2020.
[2]POUYANFAR S,SADI S,YAN Q Y,et al.A Survey on Deep Learning:Algorithms,Techniques,and Applications[J].ACM Computing Surveys,2019,51(5):1-36.
[3]LEVINE S,PASTOR P,KRIZHEVSKY A,et al.Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection[J].arXiv:1603.02199,2016.
[4]MCMAHAN B,MOORE E,RAMAGE D,et al.Communication-Efficient Learning of Deep Networks from Decentralized Data[C]//Proceedings of the 20th International Conference on Artificial Intelligence and Statistics.2017:1273-128.
[5]YANG Q,LIU Y,CHEN T.Federated Machine Learning:Concept and Applications[J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19.
[6]ZHU H,XU J,LIU S,et al.Federated learning on non-IID data:A survey[J].Neurocomputing,2021,465:371-390.
[7]FALLAH A,MOKHTARI A A,OZDAGLAR A A,et al.Personalized Federated Learning with Theoretical Guarantees:A Model-Agnostic Meta-Learning Approach[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems.Vancouver,BC,Canada,2020:3557-3568.
[8]ARIVAZHAGAN M G,AGGARWAL V,SINGH A K,et al.Federated Learning with Personalization Layers[J].arXiv:1912.00818,2019.
[9]GHOSH A,CHUNG A A,YIN J A,etal.An Efficient Framework for Clustered Federated Learning[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems.Vancouver,BC,Canada,2020:19586-19597.
[10]WANG Y S,LIU Y A,MA W A,et al.Iterative Learning with Open-set Noisy Labels[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:8688-8696.
[11]RETU,STAVROU G F A,LOCASTO A A,et al.Casting out Demons:Sanitizing Training Data for Anomaly Sensors[C]//2008 IEEE Symposium on Security and Privacy(sp 2008).2008:81-95.
[12]XIE C,KOYEJO O,GUPTA I.Zeno:Distributed stochastic gradient descent with suspicion-based fault-tolerance[C]//36th International Conference on Machine Learning(ICML 2019).2019:11928-11944.
[13]HAN B,YAO B A,YU Q A,et al.Co-Teaching:Robust Training of Deep Neural Networks with Extremely Noisy Labels[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems(NIPS’18).2018.
[14]LI A R,ZHANG A A,WANG L A,et al.Privacy-Preserving Efficient Federated-Learning Model Debugging[J].IEEE Transactions on Parallel and Distributed Systems,2022,33(10):2291-2303.
[15]LEMLEY,BAZRAFKAN J A,SHABAB.Smart Augmentation Learning an Optimal Data Augmentation Strategy[J].IEEE Access,2017,5:5858-586.
[16]CUBUK E D,ZOPH B,MANÉ D,et al.AutoAugment:Learning Augmentation Strategies From Data[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:113-123.
[17]ZHANG H,CISSE M,DAUPHIN Y N,et al.mixup:BeyondEmpirical Risk Minimization[J].arXiv:1710.09412,2018.
[18]YUN S,HAN D,OH S J,et al.CutMix:Regularization Strategy to Train Strong Classifiers With Localizable Features[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).2019:6022-6031.
[19]DEVRIES T,TAYLOR G W.Improved Regularization of Convolutional Neural Networks with Cutout[J].arXiv:1708.04552,2017.
[20]QIN X,ZHANG Z,HUANG C,et al.BASNet:Boundary-Aware Salient Object Detection[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2019:7471-7481.
[21]HSIEH K,PHANISHAYEE A,MUTLU O.The Non-IID Data Quagmire of Decentralized Machine Learning[C]//Proceedings of the 37th International Conference on Machine Learning.PMLR,2020:4387-4398.
[1] LIN Xinyu, YAO Zewei, HU Shengxi, CHEN Zheyi, CHEN Xing. Task Offloading Algorithm Based on Federated Deep Reinforcement Learning for Internet of Vehicles [J]. Computer Science, 2023, 50(9): 347-356.
[2] ZHAO Yuhao, CHEN Siguang, SU Jian. Privacy-enhanced Federated Learning Algorithm Against Inference Attack [J]. Computer Science, 2023, 50(9): 62-67.
[3] XU Jie, WANG Lisong. Contrastive Clustering with Consistent Structural Relations [J]. Computer Science, 2023, 50(9): 123-129.
[4] LIANG Jiayin, XIE Zhipeng. Text Paraphrase Generation Based on Pre-trained Language Model and Tag Guidance [J]. Computer Science, 2023, 50(8): 150-156.
[5] LI Rongchang, ZHENG Haibin, ZHAO Wenhong, CHEN Jinyin. Data Reconstruction Attack for Vertical Graph Federated Learning [J]. Computer Science, 2023, 50(7): 332-338.
[6] HUANG Fangwan, LU Juhong, YU Zhiyong. Data Augmentation for Cardiopulmonary Exercise Time Series of Young HypertensivePatients Based on Active Barycenter [J]. Computer Science, 2023, 50(6A): 211200233-11.
[7] ZENG Wu, MAO Guojun. Few-shot Learning Method Based on Multi-graph Feature Aggregation [J]. Computer Science, 2023, 50(6A): 220400029-10.
[8] WANG Qingyu, WANG Hairui, ZHU Guifu, MENG Shunjian. Study on SQL Injection Detection Based on FlexUDA Model [J]. Computer Science, 2023, 50(6A): 220600172-6.
[9] ZHANG Lianfu, TAN Zuowen. Robust Federated Learning Algorithm Based on Adaptive Weighting [J]. Computer Science, 2023, 50(6A): 230200188-9.
[10] ZHONG Jialin, WU Yahui, DENG Su, ZHOU Haohao, MA Wubin. Multi-objective Federated Learning Evolutionary Algorithm Based on Improved NSGA-III [J]. Computer Science, 2023, 50(4): 333-342.
[11] KANG Shuming, ZHU Yan. Text Stance Detection Based on Topic Attention and Syntactic Information [J]. Computer Science, 2023, 50(11A): 230200068-5.
[12] LI Renjie, YAN Qiao. Inter-cluster Optimization for Cluster Federated Learning [J]. Computer Science, 2023, 50(11A): 221000243-5.
[13] LU Qi, YU Yuanqiang, XU Daoming, ZHANG Qi. Improved YOLOv5 Small Drones Target Detection Algorithm [J]. Computer Science, 2023, 50(11A): 220900050-8.
[14] LUO Yuetong, LI Chao, DUAN Chang, ZHOU Bo. Hue Augmentation Method for Industrial Product Surface Defect Images [J]. Computer Science, 2023, 50(11A): 230200089-6.
[15] ZHANG Lianfu, TAN Zuowen. Federated Learning Privacy-preserving Approach for Multimodal Medical Data [J]. Computer Science, 2023, 50(11A): 230800021-8.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!