Computer Science ›› 2022, Vol. 49 ›› Issue (12): 22-32.doi: 10.11896/jsjkx.220500240

• Federated Leaming • Previous Articles     Next Articles

Study on Privacy-preserving Nonlinear Federated Support Vector Machines

YANG Hong-jian, HU Xue-xian, LI Ke-jia, XU Yang, WEI Jiang-hong   

  1. School of Data and Target Engineering,PLA Strategic Support Force Information Engineering University,Zhengzhou 450001,China
  • Received:2022-05-26 Revised:2022-07-07 Published:2022-12-14
  • About author:YANG Hong-jian,born in 1998,postgraduate.His main research interests include federated learning,homomorphic encryption and blockchain.HU Xue-xian,born in 1982,Ph.D,associate professor,master supervisor.His main research interests include big data security,applied cryptography and network security.
  • Supported by:
    National Natural Science Foundation of China(62172433,62172434,61862011,61872449).

Abstract: Federated learning offers new ideas for solving the problem of multiparty joint modeling in “data silos”.Federated support vector machines can realize cross-device support vector machine modeling without local data,but the existing research has some defects such as insufficient privacy protection in a training process and a lack of research on nonlinear federated support vector machines.To solve the above problems,this paper utilizes the stochastic Fourier feature method and CKKS homomorphic encryption system to propose a nonlinear federated support vector machine training(PPNLFedSVM) algorithm for privacy protection.Firstly,the same Gaussian kernel approximate mapping function is generated locally for each participant based on the random Fourier feature method,and the training data of each participant is explicitly mapped from the low-dimensional space to the high-dimensional space.Secondly,the model parameter security aggregation algorithm based on CKKS cryptography ensures the privacy of model parameters and their contributions during the model aggregation process.Moreover,the parameter aggregation process is optimized and adjusted according to the characteristics of CKKS cryptography to improve the efficiency of the security aggregation algorithm.Security analysis and experimental results show that the PPNLFedSVM algorithm can ensure the privacy of participant model parameters and their contributions to the training process without losing the model accuracy.

Key words: Federated learning, Privacy preserving, Homomorphic encryption, Support vector machines, Multi-party secure random seed negotiation, Random Fourier features

CLC Number: 

  • TP309.2
[1]ZHUANG M Q,TAN X H,FAN Y C,et al.3D animation expression generation and emotional supervision based on convolutional neural network[J].Journal of Chongqing University of Technology(Natural Science),2022,36(1):151-158.
[2]WANG Z,GUO Y,NIE Z,et al.Privacy protection and cost management of smart meters based on dueling double deep Q-learning[J].Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition),2021,33(4):554-561.
[3]WANG J,XU Y H,LI L.Data fusion privacy protection method with low energy consumption and integrity verification[J].Journal of Jilin University(Engineering and Technology Edition),2022,52(7):1657-1665.
[4]LI Q X,ZHOU Q X,WANG Z L,et al.Provable Secure Delegation Computing Protocol Based on Privacy Protection[J].Computer Engineering,2021,47(5):131-137.
[5]YANG W Q,ZHANG Y,NIE J T,et al.Energy and Information Management Strategy Based on Federated Learning for Wireless Network Nodes[J].Computer Engineering,2022,48(1):188-196,203.
[6]LIU Y X,CHEN H,LIU Y H,et al.Privacy-Preserving Techniques in Federated Learning [J].Ruan Jian Xue Bao/Journal of Software,2022,33(3):1057-1092.
[7]WEN Y L,CHEN M J.Medical Data Sharing Scheme Combined with Federal Learning and Blockchain[J].Computer Enginee-ring,2022,48(5):145-153,161.
[8]ZHU L,LIU Z,HAN S.Deep Leakage from Gradients [J].Advances in Neural Information Processing Systems,2019,32:1-11.
[9]ZHAO B,MOPURI K R,BILEN H.iDLG:Improved Deep Lea-kage from Gradients [J].arXiv:2001.02610,2020.
[10]WANG Z,SONG M,ZHANG Z,et al.Beyond Inferring Class Representatives:User-Level Privacy Leakage from Federated Learning[C]//IEEE INFOCOM 2019-IEEE Conference on Computer Communications.IEEE,2019:2512-2520.
[11]BAKOPOULOU E,TILLMAN B,MARKOPOULOU A.AFederated Learning Approach for Mobile Packet Classification [J].arXiv:1907.13113,2019.
[12]GE N,LI G H,ZHANG L,et al.Failure Prediction in Production Line Based on Federated Learning:An Empirical Study [J].arXiv:2101.11715,2021.
[13]HARTMANN V,MODI K,PUJOL J M,et al.Privacy-Preserving Classification with Secret Vector Machines[C]//Procee-dings of the 29th ACM InternationalConference on Infor-mation & Knowledge Management.2020:475-484.
[14]BURMESTER M,DESMEDT Y.A Secure and EfficientConfe-rence Key Distribution System[C]//Workshop on the Theory and Application of Cryptographic Techniques.Berlin:Springer,1994:275-286.
[15]CHEON J H,KIM A,KIM M,et al.Homomorphic Encryption for Arithmetic of Approximate Numbers[C]//International Conference on the Theory and Application of Cryptology and Information Security.Cham:Springer,2017:409-437.
[16]YU H,VAIDYA J,JIANG X.Privacy-Preserving SVM Classification on Vertically Partitioned Data[C]//Pacific-Asia Confe-rence on Knowledge Discovery and Data Mining.Berlin:Sprin-ger, 2006:647-656.
[17]YU H,JIANG X,VAIDYA J.Privacy-Preserving SVM Using Nonlinear Kernels on Horizontally Partitioned Data[C]//Proceedings of the 2006 ACM Symposium on Applied Computing.2006:603-610.
[18]VAIDYA J,YU H,JIANG X.Privacy-Preserving SVM Classification [J].Knowledge and Information Systems,2008,14(2):161-178.
[19]MANGASARIAN O L,WILD E W.Privacy-Preserving Classification of Horizontally Partitioned Data via Random Kernels[C]//Proceedings of the 2008 International Conference on Data Mining.Las Vegas,USA,2008:473-479.
[20]LEE Y J,MANGASARIAN O L.RSVM:Reduced Support Vector Machines[C]//Proceedings of the 2001 SIAM International Conference on Data Mining.Society for Industrial and Applied Mathematics.2001:1-17.
[21]SUN L,MU W S,QI B,et al.A New Privacy-Preserving Proximal Support Vector Machine for Classification of Vertically Partitioned Data [J].International Journal of Machine Learning and Cybernetics,2015,6(1):109-118.
[22]LIU X,DENG R H,CHOO K K R,et al.Privacy-PreservingOutsourced Support Vector Machine Design for Secure Drug Discovery [J].IEEE Transactions on Cloud Computing,2018,8(2):610-622.
[23]LIU X,DENG R H,CHOO K K R,et al.An Efficient Privacy-Preserving Outsourced Calculation Toolkit with Multiple Keys [J].IEEE Transactions on Information Forensics and Security,2016,11(11):2401-2414.
[24]WANG J,WU L,WANG H,et al.An Efficient and Privacy-Preserving Outsourced Support Vector Machine Training for Internet of Medical Things [J].IEEE Internet of Things Journal,2020,8(1):458-473.
[25]MCMAHAN B,MOORE E,RAMAGE D,et al.Communic-ationEfficient Learning of Deep Networks from Decentralized Data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[26]RIVEST R L,ADLEMAN L,DERTOUZOS M L.On DataBanks and Privacy Homomorphisms [J].Foundations of Secure Computation,1978,4(11):169-180.
[27]LYU L,YU H,YANG Q.Threats to Federated Learning:A Survey [J].arXiv:2003.02133,2020.
[28]RAHIMI A,RECHT B.Random Features for Large-Scale Kernel Machines [J].Advances in Neural Information Processing Systems,2007,20:1177-1184.
[29]RUDIN W.Fourier Analysis on Groups[M].New York:Courier Dover Publications,2017.
[30]GREGORY G.Predicts Random Fourier Features[EB/OL].(2019-12-23) [2022-05-24].http://gregorygundersen.com/blog/2019/12/23/random-fourier-features/.
[31]CHEON J H,HONG S,KIM D.Remark on the Security ofCKKS Scheme in Practice [EB/OL].(2020-12-21) [2022-05-26].https://eprint.iacr.org/2020/1581.pdf.
[32]ODED G.Foundations of Cryptography-Basic Applications[M].Cambridge:Cambridge University Press,2004.
[33]BOST R,POPA R A,TU S,et al.Machine Learning Classification over Encrypted Data[C]//Network and Distributed System Security Symposium.2014.
[1] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[2] LYU You, WU Wen-yuan. Privacy-preserving Linear Regression Scheme and Its Application [J]. Computer Science, 2022, 49(9): 318-325.
[3] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients [J]. Computer Science, 2022, 49(9): 183-193.
[4] CHEN Ming-xin, ZHANG Jun-bo, LI Tian-rui. Survey on Attacks and Defenses in Federated Learning [J]. Computer Science, 2022, 49(7): 310-323.
[5] WANG Jian. Back-propagation Neural Network Learning Algorithm Based on Privacy Preserving [J]. Computer Science, 2022, 49(6A): 575-580.
[6] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Clustered Federated Learning Methods Based on DBSCAN Clustering [J]. Computer Science, 2022, 49(6A): 232-237.
[7] CHEN Jing-nian. Acceleration of SVM for Multi-class Classification [J]. Computer Science, 2022, 49(6A): 297-300.
[8] YAN Meng, LIN Ying, NIE Zhi-shen, CAO Yi-fan, PI Huan, ZHANG Lan. Training Method to Improve Robustness of Federated Learning [J]. Computer Science, 2022, 49(6A): 496-501.
[9] QIN Xiao-yue, HUANG Ru-wei, YANG Bo. NTRU Type Fully Homomorphic Encryption Scheme over Prime Power Cyclotomic Rings [J]. Computer Science, 2022, 49(5): 341-346.
[10] DU Hui, LI Zhuo, CHEN Xin. Incentive Mechanism for Hierarchical Federated Learning Based on Online Double Auction [J]. Computer Science, 2022, 49(3): 23-30.
[11] WANG Xin, ZHOU Ze-bao, YU Yun, CHEN Yu-xu, REN Hao-wen, JIANG Yi-bo, SUN Ling-yun. Reliable Incentive Mechanism for Federated Learning of Electric Metering Data [J]. Computer Science, 2022, 49(3): 31-38.
[12] ZHAO Luo-cheng, QU Zhi-hao, XIE Zai-peng. Study on Communication Optimization of Federated Learning in Multi-layer Wireless Edge Environment [J]. Computer Science, 2022, 49(3): 39-45.
[13] REN Hua, NIU Shao-zhang, WANG Mao-sen, YUE Zhen, REN Ru-yong. Homomorphic and Commutative Fragile Zero-watermarking Based on SVD [J]. Computer Science, 2022, 49(3): 70-76.
[14] LYU You, WU Wen-yuan. Linear System Solving Scheme Based on Homomorphic Encryption [J]. Computer Science, 2022, 49(3): 338-345.
[15] KONG Yu-ting, TAN Fu-xiang, ZHAO Xin, ZHANG Zheng-hang, BAI Lu, QIAN Yu-rong. Review of K-means Algorithm Optimization Based on Differential Privacy [J]. Computer Science, 2022, 49(2): 162-173.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!