Abstract: Existing methods for speech emotion recognition usually rely on deep learning models to extract acoustic features,but most of them focus only on modelling generic features,failing to fully explore a priori knowledge in the data that is closely related to emotion.To this end,this paper proposes an end-to-end multi-task learning framework that utilizes the self-supervised pre-training model WavLM to extract speech features rich in emotional information,and introduces gender recognition as an auxiliary task to account for the influence of gender differences on emotion recognition.To address the learning imbalance issue caused by the fixed weight calculation in traditional multi-task learning frameworks,this paper proposes a Temperature-aware Dynamic Weight Averaging(TA-DWA) method.This method balances the learning speeds of different tasks by dynamically adjusting the temperature coefficient and achieves more reasonable weight allocation by incorporating the rate of change in task losses.Experimental results on the IEMOCAP and EMODB datasets demonstrate that the proposed approach significantly improves emotion recognition accuracy.These findings validate the effectiveness of using gender recognition as an auxiliary task and highlight the advantages of the dynamic weighting strategy in multi-task learning.

Key words: Speech emotion recognition, Multi-task learning, Dynamic weight assignment, Self-supervised models