Abstract: In distributed environments,data heterogeneity manifests as discrepancies in data features.Expert model collaboration suffers from knowledge isolation and improper task allocation,leading to uneven training results among experts,preventing full exploitation of each model’s advantages,and thus limiting overall performance.To address these challenges,this paper proposes MECII.The framework integrates the mixture of experts(MoE) architecture with social learning(SL) principles,optimizing the knowledge sharing and complementarity among experts through four key components:multi-expert collaboration,gating network,adaptive information interaction,and gating selection constraints.This approach effectively resolves issues related to data heterogeneity and expert collaboration in distributed learning.By ensuring precise expert selection and task allocation,MECII facilitates information flow between experts,enhancing each expert’s accuracy when processing specific data,and consequently enhancing the overall model performance.Experimental results demonstrate that MECII significantly outperforms traditional federated learning(FL) baseline methods on the CIFAR-10 and CIFAR-100 datasets.Particularly in scenarios with data heterogeneity,MECII achieves improvements in classification accuracy of 6.69 percentage points and 5.13 percentage points,respectively,compared to the state-of-the-art FedL2P method.Moreover,individual expert accuracy is effectively optimized,validating the framework’s significant advantages in promoting expert collaboration and improving individual accuracy.

Key words: Multi-expert collaboration, Social learning, Data heterogeneity, Federated learning, Information interaction, Mixture of experts