Abstract: Personalized federated learning(PFL) combats data heterogeneity by retaining custom models for each client.Existing works primarily address label distribution drift but rarely consider non-temporal concept drift or the gradient conflicts caused by their combination,leading to slow convergence and degraded generalization in global models.This study proposes a hierarchical aggregation-dual prototype negative distillation method to enhance PFL performance in such extreme heterogeneous scenarios.The method first identifies concept-drifted clients early on through short-term training preheating,peer evaluation,and majority voting,without the need for additional private information.Then,within the trusted client set,it calculates the similarity of conditional distributions and complementarity of marginal distributions using the local model's principal components.This dynamic weighting is used in recursive hierarchical aggregation,balancing semantic consistency with feature diversity.Finally,positive and negative category prototypes are extracted from both trusted and abnormal clients to generate pseudo-samples,and a combined loss of cross-entropy and margin-based negative distillation is applied to the global model,simultaneously reinforcing correct semantics and explicitly suppressing conflicting concepts.Prototype distillation fine-tuning is also applied to abnormal clients to maintain personalized accuracy.Experiments conducted on Fashion-MNIST,CIFAR-10,and CIFAR-100 datasets in concept drift scenarios show that the proposed method achieves an average global accuracy improvement of 3.6 percentage points and the communication rounds are comparable to those of classical averaging aggregation.The research concludes that this method effectively enhances both global generalization and local adaptability in complex dual-bias environments.

Key words: Federated learning, Personalized federated learning, Label distribution drift, Non-temporal concept drift, Knowledge distillation