Abstract: Electronic health records(EHRs) data has become a valuable resource for biomedical research.By learning multi-dimensional features hidden in EHRs data that are difficult for humans to distinguish,machine learning methods can achieve better results.However,some existing studies only consider some privacy leaks that may be faced during or after model training,resulting in a single privacy preservation measure that cannot cover the whole life cycle of machine learning.In addition,most of the existing programs are focused on federated learning privacy preservationmethods for single-mode data.Therefore,a federated learningprivacy preservation approach for multimodal data is proposed.To prevent the adversaryfrom stealing the original data information through reverse attack,differential privacy perturbation is performed on the model parameters uploaded by each participant.To prevent the leakage of local model information of each participant in the process of model training,the Paillier cryptosystem is used for homomorphic encryption of local model parameters.The security of the method is analyzed from the theoretical point of view,the security model is defined,and the security of the subprotocol is proved.Experimental results show that this method can preserveprivacy of training data and model with almost no loss of performance.

Key words: Federated learning, Multimodal data, EHRs, Secure aggregation, Privacy-preserving