关键词: 软模块化方法, 多任务强化学习, 优先级加权知识蒸馏, 分层缓存, 任务嵌入

Abstract: In recent years,reinforcement learning has achieved remarkable success in various domains.However,traditional RL methods often struggle with adaptability when facing dynamic environments or multiple tasks.To address this challenge,this thesis introduces SM-PHT,a robust,scalable,and efficient method for multi-task reinforcement learning.The primary objective of this research is to enhance the adaptability and generalization capabilities of reinforcement learning agents in multi-task environments by enabling them to learn and transfer knowledge across multiple tasks.SM-PHT integrates three key mechanisms:priority-weighted knowledge distillation(PWKD),hierarchical buffer,and task embedding.PWKD leverages a weighted distillation process to assimilate knowledge from multiple high-performing models,improving the robustness and stability of the student network.Moreover,the hierarchical buffer employs dual buffers to store low-level experiential data and high-level model parameters,optimizing offline learning efficiency.Finally,task embedding enrich task representations by capturing detailed environmental characteristics,facilitating effective knowledge transfer.Experiments conducted in the Meta-World environment demonstrate SM-PHT’s superior performance compared to state-of-the-art methods.In the MT10 challenge,SM-PHT achieves double the success rate and a 30% increase in average rewards.In the more complex MT50 challenge,it improves the success rate by approximately 10% and increases average rewards by around 10%.These results highlight SM-PHT’s ability to handle complex tasks with remarkable stability and minimal fluctuation,making it a promising approach for real-world MTRL applications.

Key words: Soft modularization network, Multi-task reinforcement learning, Priority-weighted knowledge distillation, Hierarchical buffer, Task embedding