关键词: 提示微调, 软提示迁移, 特征提取, 门控代理注意力机制, 注意力分布扰动

Abstract: Soft prompt transfer,as a cross-task learning approach,aims to guide the target task model to learn more generalizable prompt representations by leveraging knowledge from source prompts.However,existing methods often ignore the impact of task-specific information embedded in target samples,which may result in biased target prompt training.To address this issue,a gate-controlled agent attention mechanism-based soft prompt transfer method(GPAPT) is proposed.Firstly,to separate and enhance both global and task-specific information in target samples,a dual-path feature extraction strategy is introduced.It employs two lightweight extraction paths to derive agent tokens that encode task-level semantics and local features that capture instance-specific characteristics.Secondly,to assess the transferability of source prompts,a gate-controlled agent attention mechanism is presented.It computes attention distributions between prompt features and both agent tokens and local features to model task similarity.To preserve task-specific information,a gating unit filters local features and integrates the two types of attention.Finally,a perturbation mechanism for attention distribution is introduced.It perturbs the attention weights based on the prediction loss of source prompts on the target task,thereby reduces the influence of source prompts that are similar in the representation space but perform poorly in prediction,and thus improving transfer robustness.Extensive experiments on the GLUE benchmark demonstrate that GPAPT achieves the best average performance compared to nine strong baseline methods,validating the effectiveness of the proposed approach.

Key words: Prompt tuning, Soft prompt transfer, Feature extraction, Gate-controlled agent attention mechanism, Attention distribution perturbation