Computer Science ›› 2023, Vol. 50 ›› Issue (4): 333-342.doi: 10.11896/jsjkx.220300033

• Information Security • Previous Articles     Next Articles

Multi-objective Federated Learning Evolutionary Algorithm Based on Improved NSGA-III

ZHONG Jialin, WU Yahui, DENG Su, ZHOU Haohao, MA Wubin   

  1. Science and Technology on Information System Engineering Laboratory,National University of Defense Technology,Changsha 410073,China
  • Received:2022-03-04 Revised:2022-04-22 Online:2023-04-15 Published:2023-04-06
  • About author:ZHONG Jialin,born in 1998,postgra-duate.Her main research interests include federated learning and so on.
    MA Wubin,born in 1986,Ph.D,asso-ciate researcher.His main research inte-rests include multi-objective optimization,micro service and data mining.
  • Supported by:
    General Program of National Natural Science Foundation of China(61871388).

Abstract: Federated learning technology solves the problems of data islands and privacy leakage to a certain extent.However it has shortcomings such as high communication cost,unstable communication,and uneven distribution of participant performance.In order to overcome these shortcomings and achieve a balance between model effectiveness,fairness,and communication costs,an improved NSGA-III algorithm for multi-objective optimization of federated learning is proposed.First,a federated learning multi-objective optimization model is constructed to maximize the accuracy of the global model,minimize the variance of the global mo-del accuracy distribution and minimize the communication cost of participant,and an improved NSGA-III algorithm based on fast greedy initialization is proposed,which improves the efficiency of NSGA-III for multi-objective optimization of federated learning.Experimental results show that the proposed optimization method can obtain a better Pareto solution than the classical multi-objective evolutionary algorithm.Compared with the standard model of federated learning,the optimized model can effectively lower the communication cost and the variance of the global model accuracy distribution while ensuring the accuracy of the global model.

Key words: Federated learning, Multi-objective equilibrium, Non-dominated sorted genetic algorithm-III (NSGA-III), Multi-objective optimization, Parameters optimization

CLC Number: 

  • TP301
[1]REGULATION G D P.Regulation EU 2016/679 of the Euro-pean Parliament and of the Council of 27 April 2016[J].Official Journal of the European Union,2016(59):1-88.
[2]SUN Y H.Cybersecurity Law:The Fundamental Measures toEnsure Cybersecurity——Study and implement the “Network Security Law of the People’s Republic of China”[J].China Information Security,2016(12):30-33.
[3]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[4]LI L,FAN Y,TSE M,et al.A review of applications in federated learning[J].Computers & Industrial Engineering,2020,149(5):106854.
[5]CHEN Y,SUN X,JIN Y.Communication-efficient federateddeep learning with layerwise asynchronous model update and temporally weighted aggregation[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(10):4229-4238.
[6]ZHU H,JIN Y.Multi-objective evolutionary federated learning[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(4):1310-1322.
[7]MOCANU D C,MOCANU E,STONE P,et al.Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science[J].Nature Communications,2018,9(1):1-12.
[8]HAO M,LI H,LUO X,et al.Efficient and privacy-enhancedfederated learning for industrial artificial intelligence[J].IEEE Transactions on Industrial Informatics,2019,16(10):6532-6542.
[9]KANG J,XIONG Z,NIYATO D,et al.Incentive design for efficient federated learning in mobile networks:A contract theory approach[C]//2019 IEEE VTS Asia Pacific Wireless Communications Symposium.IEEE,2019:1-5.
[10]LI T,SANJABI M,BEIRAMI A,et al.Fair resource allocation in federated learning[J].arXiv:1905.10497,2019.
[11]QOLOMANY B,AHMAD K,AL-FUQAHA A,et al.Particle swarm optimized federated learning for industrial IoT and smart city services[C]//2020 IEEE Global Communications Confe-rence(GLOBECOM 2020).IEEE,2020:1-6.
[12]DEB K,PRATAP A,AGARWAL S,et al.A fast and elitistmulti-objective genetic algorithm:NSGA-II[J].IEEE Transactions on Evolutionary Computation,2002,6(2):182-197.
[13]ZHANG Q,LI H.MOEA/D:A multi-objective evolutionary algorithm based on decomposition[J].IEEE Transactions on evolutionary computation,2007,11(6):712-731.
[14]ZITZLER E,LAUMANNS M,THIELE L.SPEA2:Improving the strength Pareto evolutionary algorithm:TIK-Report 103[R].Swiss:Swiss Federal Institute of Technology,2001.
[15]KNOWLES J,CORNE D.The pareto archived evolution strategy:A new baseline algorithm for pareto multi-objective optimization[C]//Proceedings of the 1999 Congress on Evolutionary Computation-CEC99.IEEE,1999:98-105.
[16]DEB K,JAIN H.An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach,part I:solving problems with box constraints[J].IEEE Transactions on Evolutionary Computation,2013,18(4):577-601.
[17]ERDOS P,RÉNYI A.On the evolution of random graphs[J].Publications of the Mathematical Institute of the Hungarian Academy of Sciences,1960,5(1):17-60.
[18]LECUN Y,CORTES C,BURGES C J.MNIST handwritten di-git database[J/OL].http://yann.lecun.com/exdb/mnist.
[19]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[J/OL].http://www.cs.toronto.edu/~kriz/cifar.html.
[20]ZITZLER E,THIELE L.Multi-objective evolutionary algo-rithms:a comparative case study and the strength Pareto approach[J].IEEE Transactions on Evolutionary Computation,1999,3(4):257-271.
[21]ZITZLER E,THIELE L.Multi-objective optimization using evolutionary algorithms——a comparative case study[C]//International Conference on Parallel Problem Solving From Nature.Berlin:Springer,1998:292-301.
[22]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[C]//Proceedings of Machine Learning and Systems.2020:429-450.
[1] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients [J]. Computer Science, 2022, 49(9): 183-193.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] CHEN Ming-xin, ZHANG Jun-bo, LI Tian-rui. Survey on Attacks and Defenses in Federated Learning [J]. Computer Science, 2022, 49(7): 310-323.
[4] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Clustered Federated Learning Methods Based on DBSCAN Clustering [J]. Computer Science, 2022, 49(6A): 232-237.
[5] YAN Meng, LIN Ying, NIE Zhi-shen, CAO Yi-fan, PI Huan, ZHANG Lan. Training Method to Improve Robustness of Federated Learning [J]. Computer Science, 2022, 49(6A): 496-501.
[6] SUN Gang, WU Jiang-jiang, CHEN Hao, LI Jun, XU Shi-yuan. Hidden Preference-based Multi-objective Evolutionary Algorithm Based on Chebyshev Distance [J]. Computer Science, 2022, 49(6): 297-304.
[7] LI Hao-dong, HU Jie, FAN Qin-qin. Multimodal Multi-objective Optimization Based on Parallel Zoning Search and Its Application [J]. Computer Science, 2022, 49(5): 212-220.
[8] PENG Dong-yang, WANG Rui, HU Gu-yu, ZU Jia-chen, WANG Tian-feng. Fair Joint Optimization of QoE and Energy Efficiency in Caching Strategy for Videos [J]. Computer Science, 2022, 49(4): 312-320.
[9] DU Hui, LI Zhuo, CHEN Xin. Incentive Mechanism for Hierarchical Federated Learning Based on Online Double Auction [J]. Computer Science, 2022, 49(3): 23-30.
[10] WANG Xin, ZHOU Ze-bao, YU Yun, CHEN Yu-xu, REN Hao-wen, JIANG Yi-bo, SUN Ling-yun. Reliable Incentive Mechanism for Federated Learning of Electric Metering Data [J]. Computer Science, 2022, 49(3): 31-38.
[11] ZHAO Luo-cheng, QU Zhi-hao, XIE Zai-peng. Study on Communication Optimization of Federated Learning in Multi-layer Wireless Edge Environment [J]. Computer Science, 2022, 49(3): 39-45.
[12] ZOU Sai-lan, LI Zhuo, CHEN Xin. Study on Transmission Optimization for Hierarchical Federated Learning [J]. Computer Science, 2022, 49(12): 5-16.
[13] SHEN Zhen, ZHAO Cheng-gui. Storage Task Allocation Algorithm in Decentralized Cloud Storage Network [J]. Computer Science, 2022, 49(12): 17-21.
[14] YANG Hong-jian, HU Xue-xian, LI Ke-jia, XU Yang, WEI Jiang-hong. Study on Privacy-preserving Nonlinear Federated Support Vector Machines [J]. Computer Science, 2022, 49(12): 22-32.
[15] QU Xiang-mou, WU Ying-bo, JIANG Xiao-ling. Federated Data Augmentation Algorithm for Non-independent and Identical Distributed Data [J]. Computer Science, 2022, 49(12): 33-39.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!