计算机科学 ›› 2022, Vol. 49 ›› Issue (12): 40-45.doi: 10.11896/jsjkx.220600237
郭桂娟1, 田晖1, 王田2,3, 贾维嘉2,3
GUO Gui-juan1, TIAN Hui1, WANG Tian2,3, JIA Wei-jia2,3
摘要： 联邦学习因其在客户端本地进行数据的训练,从而有效保证了数据的隐私性和安全性。对于联邦学习的研究虽然取得了很大的进展,但是,由于非独立同分布数据的存在以及数据量不平衡、数据类型不平衡等问题,客户端在利用本地数据进行训练时不可避免地存在精确度缺失、训练效率低下等问题。为了应对联邦学习背景环境的不同导致的联邦学习效率降低的问题,文中提出了一种基于背景优化的高效联邦学习方案,用于提高终端设备中本地模型的精确度,从而减小通信开销、提高整体模型的训练效率。具体来说,在不同的环境中根据精确度的差异性来选择第一设备和第二设备,将第一设备模型和全局模型的不相关性(下文统称为差异值)作为标准差异值;而第二设备是否上传本地模型则由第二设备和第一设备之间的差异值决定。实验结果表明,与传统的联邦学习相比,所提方案在普通联邦学习场景下的表现明显优于联邦平均算法,在MINIST数据集上,其精确度提高了约7.5%;在CIFAR-10数据集上,其精确度提高了约10%。
中图分类号:
| [1]WANG T,CAO Z,WANG S,et al.Privacy-enhanced data collection based on deep learning for Internet of vehicles[J].IEEE Transactions on Industrial Informatics,2019,16(10):6663-6672. [2]YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Concept and applications[J].ACM Transactions on Intelligent Systems and Technology(TIST),2019,10(2):1-19. [3]ZHANG Y L,LIANG Y Z,YIN M J,et al.A Review of Offloading Schemes in moving Edge Computing[J].Chinese Journal of Computers,2021,44(12):2406-2430. [4]SATTLER F,WIEDEMANN S,MÜLLER K R,et al.Robust and communication-efficient federated learning from non-iid data[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(9):3400-3413. [5]WANG T,BHUIYAN M Z A,WANG G,et al.Preserving ba-lance between privacy and data integrity in edge-assisted Internet of Things[J].IEEE Internet of Things Journal,2019,7(4):2679-2689. [6]BRIGGS C,FAN Z,ANDRAS P.Federated learning with hie-rarchical clustering of local updates to improve training on non-IID data[C]//2020 International Joint Conference on Neural Networks(IJCNN).IEEE,2020:1-9. [7]ZHAO Y,LI M,LAI L,et al.Federated learning with non-iid data[J].arXiv:1806.00582,2018. [8]DUAN M,LIU D,CHEN X,et al.Self-balancing federatedlearning with global imbalanced data in mobile systems[J].IEEE Transactions on Parallel and DistributedSystems,2020,32(1):59-71. [9]HUANG Y,CHU L,ZHOU Z,et al.Personalized cross-silo fe-derated learning on non-iid data[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:7865-7873. [10]ZHANG L,LUO Y,BAI Y,et al.Federated Learning for Non-IID Data via Unified Feature Learning and Optimization Objective Alignment[C]//Proceedings of the IEEE/CVF Inter-national Conference on Computer Vision.2021:4420-4428. [11]SUN Y,ZHOU S,GÜNDÜZ D.Energy-aware analog aggregation for federated learning with redundant data[C]//ICC 2020-2020 IEEE International Conference on Communications(ICC).IEEE,2020:1-7. [12]BONAWITZ K,EICHNER H,GRIESKAMP W,et al.Towards federated learning at scale:System design[J].arXiv:1902.01046,2019. [13]SHI D,LI L,CHEN R,et al.Towards Energy Efficient Federated Learning over 5G+ Mobile Devices [J].IEEE Signal Processing Magazine,2020,37(3):50-60. [14]WANG L,WANG W,LI B.CMFL:Mitigating communication overhead for federated learning[C]//Proceedings of the IEEE 39th International Conference on Distributed Computing Systems.Texas,USA,2019:954-964. [15]WANG T,WANG P,CAI S,et al.Mobile edge-enabled trustevaluation for the Internet of Things[J].Information Fusion,2021,75(4):90-100. [16]SUN B,LIU Y,WANG T,et al.Review offederated learning Efficiency Optimization in Mobile Edge Networks[EB/OL].http://kns.cnki.net/kcms/detail/11.1777.TP.20211104.2019.024.html.2022-4-16. [17]ZHANG C,XIE Y,BAI H,et al.A survey on federated learning[J].Knowledge-Based Systems,2021,216(15):1-11. [18]KAIROUZ P,MCMAHAN H B,AVENT B,et al.Advancesand open problems in federated learning[J].arXiv:1912.04977,2019. [19]WANG T,LIU Y,ZHENG X,et al.Edge-Based Communication Optimization for Distributed Federated Learning[J].IEEE Transactions on Network Science and Engineering,2022,9(4): 2015-2024. [20]LIU Y,WANG T,PENG S L,et al.Edge-based model cleaning and device clustering in federated learning[J].Chinese Journal of Computers,2021,44(12):2517-2530. [21]ABADI M,BARHAM P,CHEN J,et al.Tensorflow:A system for large-scale machine learning[C]//12th USENIX Symposium on Operating Systems Design and Implementation(OSDI 16).2016:265-283. |
| [1] | 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩. 基于分层抽样优化的面向异构客户端的联邦学习 Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients 计算机科学, 2022, 49(9): 183-193. https://doi.org/10.11896/jsjkx.220500263 |
| [2] | 汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108 |
| [3] | 陈明鑫, 张钧波, 李天瑞. 联邦学习攻防研究综述 Survey on Attacks and Defenses in Federated Learning 计算机科学, 2022, 49(7): 310-323. https://doi.org/10.11896/jsjkx.211000079 |
| [4] | 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩. 基于DBSCAN聚类的集群联邦学习方法 Clustered Federated Learning Methods Based on DBSCAN Clustering 计算机科学, 2022, 49(6A): 232-237. https://doi.org/10.11896/jsjkx.211100059 |
| [5] | 闫萌, 林英, 聂志深, 曹一凡, 皮欢, 张兰. 一种提高联邦学习模型鲁棒性的训练方法 Training Method to Improve Robustness of Federated Learning 计算机科学, 2022, 49(6A): 496-501. https://doi.org/10.11896/jsjkx.210400298 |
| [6] | 杜辉, 李卓, 陈昕. 基于在线双边拍卖的分层联邦学习激励机制 Incentive Mechanism for Hierarchical Federated Learning Based on Online Double Auction 计算机科学, 2022, 49(3): 23-30. https://doi.org/10.11896/jsjkx.210800051 |
| [7] | 王鑫, 周泽宝, 余芸, 陈禹旭, 任昊文, 蒋一波, 孙凌云. 一种面向电能量数据的联邦学习可靠性激励机制 Reliable Incentive Mechanism for Federated Learning of Electric Metering Data 计算机科学, 2022, 49(3): 31-38. https://doi.org/10.11896/jsjkx.210700195 |
| [8] | 赵罗成, 屈志昊, 谢在鹏. 面向多层无线边缘环境下的联邦学习通信优化的研究 Study on Communication Optimization of Federated Learning in Multi-layer Wireless Edge Environment 计算机科学, 2022, 49(3): 39-45. https://doi.org/10.11896/jsjkx.210800054 |
| [9] | 邹赛兰, 李卓, 陈昕. 面向分层联邦学习的传输优化研究 Study on Transmission Optimization for Hierarchical Federated Learning 计算机科学, 2022, 49(12): 5-16. https://doi.org/10.11896/jsjkx.220300204 |
| [10] | 申圳, 赵成贵. 去中心化云存储网络的存储任务分配算法 Storage Task Allocation Algorithm in Decentralized Cloud Storage Network 计算机科学, 2022, 49(12): 17-21. https://doi.org/10.11896/jsjkx.220700131 |
| [11] | 杨鸿健, 胡学先, 李可佳, 徐阳, 魏江宏. 隐私保护的非线性联邦支持向量机研究 Study on Privacy-preserving Nonlinear Federated Support Vector Machines 计算机科学, 2022, 49(12): 22-32. https://doi.org/10.11896/jsjkx.220500240 |
| [12] | 瞿祥谋, 吴映波, 蒋晓玲. 一种非独立同分布问题下的联邦数据增强算法 Federated Data Augmentation Algorithm for Non-independent and Identical Distributed Data 计算机科学, 2022, 49(12): 33-39. https://doi.org/10.11896/jsjkx.220300031 |
| [13] | 梁文雅, 刘波, 林伟伟, 严远超. 联邦学习激励机制研究综述 Survey of Incentive Mechanism for Federated Learning 计算机科学, 2022, 49(12): 46-52. https://doi.org/10.11896/jsjkx.220500272 |
| [14] | 程帆, 王瑞锦, 张凤荔. 边缘场景下动态权重的联邦学习优化方法 Federated Learning Optimization Method for Dynamic Weights in Edge Scenarios 计算机科学, 2022, 49(12): 53-58. https://doi.org/10.11896/jsjkx.220700136 |
| [15] | 吴赟寒, 白光伟, 沈航. 基于联邦学习的车联网多维资源动态分配算法 Multi-dimensional Resource Dynamic Allocation Algorithm for Internet of Vehicles Based on Federated Learning 计算机科学, 2022, 49(12): 59-65. https://doi.org/10.11896/jsjkx.211000123 |
|
||
首页