一种非独立同分布问题下的联邦数据增强算法

doi:10.11896/jsjkx.220300031

Abstract

Abstract: In federated learning,the local data distribution of users changes with the location and preferences of users,the data under the non-independent and identical distributed(Non-IID) data may lack data of some label categories,which significantly affects the update rate and the performance of the global model in federated aggregation.To solve this problem,a federated data augmentation based on conditional generative adversarial network(FDA-cGAN) algorithm is proposed,which can amplify data from participants with skewed data without compromising user privacy,and greatly improve the performance of the algorithm with Non-IID data.Experimental results show that,compared with the current mainstream federated average algorithm,under the Non-IID data setting,the prediction accuracy of MNIST and CIFAR-10 data sets improves by 1.18% and 14.6%,respectively,which demonstrates the effectiveness and practicability of the proposed algorithm for Non-IID data problems in federated learning.

Key words: Federated learning, Privacy-preserving, Generative adversarial network, Differential Privacy, Data augmentation

CLC Number:

TP391

QU Xiang-mou, WU Ying-bo, JIANG Xiao-ling. Federated Data Augmentation Algorithm for Non-independent and Identical Distributed Data[J].Computer Science, 2022, 49(12): 33-39.

References

[1]MCMAHAN H B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[2]MCMAHAN H B,MOORE E,RAMAGE D,et al.Federatedlearning of deep networks using model averaging[J].arXiv:1602.05629,2016.
[3]YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Concept and applications[J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19.
[4]JAKUB K,MCMAHAN H B,YU F X,et al.Federated lear-ning:Strategies for improving communication efficiency[J].ar-Xiv:1610.05492,2016.
[5]JAKUB K,MCMAHAN H B,DANIEL R,et al.Federated Optimization:Distributed Machine Learning for On-Device Intelligence[J].arXiv:1610.02527,2016.
[6]ZHAO Y,LI M,SUDA N,et al.Federated learning with non-iid data[J].arXiv:1806.00582,2018.
[7]BONAWITZ K,EICHNER H,GRIESKAMP W,et al.Towards federated learning at scale:System design[C]//Proceedings of Machine Learning and Systems.2019,1:374-388.
[8]LARIMIREDDY S P,KALE S,MOHRI M,et al.SCAFFOLD:Stochastic Controlled Averaging for On-Device Federated Lear-ning[C]//Proceedings of the International Conference on Machine Learning.PMLR,2020,119:5132-5143.
[9]LI X,HUANG K,YANG W,et al.On the convergence of fedavg on non-iid data[J] arXiv:1907.02189,2019.
[10]HSU T M H,QI H,BROWN M.Measuring the effects of non-identical data distribution for federated visual classification[J].arXiv:1909.06335,2019.
[11]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[C]//Proceedings of Machine Learning and Systems.2020:429-450.
[12]WANG J,LIU Q,LIANG H,et al.Tackling the objective inconsistency problem in heterogeneous federated optimization[C]//Advances in Neural Information Processing Systems.2020:7611-7623.
[13]KAIROUZ P,MCMAHAN H B,AVENT B,et al.Advances and openproblems in federated learning[J].Foundations and Trends in Machine Learning,2021,14(1／2):1-210.
[14]SATTLER F,WIREDEMANN S,MULLER KR,et al.Robust and communication-efficient federated learning from non-iid data[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(9):3400-3413.
[15]NISHIO T,YONETANI R.Client selection for federated lear-ning with heterogeneous resources in mobile edge[C]//International Conference on Communications(ICC).IEEE,2019:1-7.
[16]WANG L,WANG W,LI B.CMFL:Mitigating communication overhead for federated learning[C]//International Conference on Distributed Computing Systems(ICDCS).IEEE,2019:954-964.
[17]SMITH V,CHIANG C K,SANJABI M,et al.Federated multi-task learning[C]//Advances in Neural Information Processing Systems.2017.
[18]SATTLER F,MULLER K R,SAMEK W.Clustered federated learning:Model-agnostic distributed multitask optimization under privacy constraints[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(8):3710-3722.
[19]LI R,MA F,JIANG W,et al.Online federated multitask lear-ning[C]//International Conference on Big Data(Big Data).IEEE,2019:215-220.
[20]COLLINS L,HASSANI H,MOKHTARI A,et al.Exploiting shared representations for personalized federated learning[C]//International Conference on Machine Learning.PMLR,2021:2089-2099.
[21]PAN S J,YANG Q.A survey on transfer learning[J].IEEE Transactions on Knowledge and Data Engineering,2009,22(10):1345-1359.
[22]YANG H,HE H,ZHANG W,et al.FedSteg:A federated transfer learning framework for secure image steganalysis[J].IEEE Transactions on Network Science and Engineering,2020,8(2):1084-1094.
[23]LIU Y,KANG Y,XING C,et al.A secure federated transfer learning framework[J].IEEE Intelligent Systems,2020,35(4):70-82.
[24]XU M,LI X,WANG Y,et al.Privacy-preserving multisourcetransfer learning in intrusion detection system[J].Transactions on Emerging Telecommunications Technologies,2021,32(5):e3957.
[25]JING Q,WANG W,ZHANG J,et al.Quantifying the perfor-mance of federated transfer learning[J].arXiv:1912.12795,2019.
[26]SHARMA S,XING C,LIU Y.Secure and efficient federated transfer learning[C]//International Conference on Big Data(Big Data).IEEE,2019:2569-2576.
[27]WANG Z,SONG M,ZHANG Z,et al.Beyond inferring class representatives:User-level privacy leakage from federated lear-ning[C]//IEEE Conference on Computer Communications.IEEE,2019:2512-2520.
[28]SUN J,LI A,WANG B,et al.Soteria:Provable defense against privacy leakage in federated learning from representation perspective[C]//IEEE Conference on Computer Vision and Pattern Recognition.2021:9311-9319.
[29]GOODFELLOW I,POUGET-ABADIE J,MIRZA MEHDI,et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems.2014.
[30]DWORK C.Differential privacy:A survey of results[C]//International Conference on Theory and Applications of Models of Computation.Berlin:Springer,2008:1-19.
[31]LIU J,YIN S,LI H,et al.A Density-based Clustering Method for K-anonymity Privacy Protection[J].Journal of Information Hiding and Multimedia Signal Processing,2017,8(1):12-18.
[32]YANG Z,CHEN M,SAAD W,et al.Energy efficient federated learning over wireless communication networks[J].IEEE Transactions on Wireless Communications,2020,20(3):1935-1949.
[33]HAMER J,MOHRI M,SURESH A T.Fedboost:A communication-efficient algorithm for federated learning[C]//International Conference on Machine Learning.PMLR,2020:3973-3983.
[34]WAHAB O A,MOURAD A,OTROK H,et al.Federated machine learning:Survey,multi-level classification,desirable criteria and future directions in communication and networking systems[J].IEEE Communications Surveys & Tutorials,2021,23(2):1342-1397.

Related Articles 15

[1]	TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[2]	LYU You, WU Wen-yuan. Privacy-preserving Linear Regression Scheme and Its Application [J]. Computer Science, 2022, 49(9): 318-325.
[3]	ZHANG Jia, DONG Shou-bin. Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer [J]. Computer Science, 2022, 49(9): 41-47.
[4]	LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients [J]. Computer Science, 2022, 49(9): 183-193.
[5]	SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[6]	YANG Bing-xin, GUO Yan-rong, HAO Shi-jie, Hong Ri-chang. Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition [J]. Computer Science, 2022, 49(7): 57-63.
[7]	DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[8]	CHEN Ming-xin, ZHANG Jun-bo, LI Tian-rui. Survey on Attacks and Defenses in Federated Learning [J]. Computer Science, 2022, 49(7): 310-323.
[9]	HUANG Jue, ZHOU Chun-lai. Frequency Feature Extraction Based on Localized Differential Privacy [J]. Computer Science, 2022, 49(7): 350-356.
[10]	XU Guo-ning, CHEN Yi-peng, CHEN Yi-ming, CHEN Jin-yin, WEN Hao. Data Debiasing Method Based on Constrained Optimized Generative Adversarial Networks [J]. Computer Science, 2022, 49(6A): 184-190.
[11]	LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Clustered Federated Learning Methods Based on DBSCAN Clustering [J]. Computer Science, 2022, 49(6A): 232-237.
[12]	WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[13]	YAN Meng, LIN Ying, NIE Zhi-shen, CAO Yi-fan, PI Huan, ZHANG Lan. Training Method to Improve Robustness of Federated Learning [J]. Computer Science, 2022, 49(6A): 496-501.
[14]	YIN Wen-bing, GAO Ge, ZENG Bang, WANG Xiao, CHEN Yi. Speech Enhancement Based on Time-Frequency Domain GAN [J]. Computer Science, 2022, 49(6): 187-192.
[15]	XU Hui, KANG Jin-meng, ZHANG Jia-wan. Digital Mural Inpainting Method Based on Feature Perception [J]. Computer Science, 2022, 49(6): 217-223.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Federated Data Augmentation Algorithm for Non-independent and Identical Distributed Data

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0