Abstract: Continuous sign language recognition is a challenging task.Most current models ignore the overall modeling ability of long sequences,resulting in lower accuracy of recognition and translation of longer sign language videos.The unique codec structure of Transformer model can be used for sign language recognition,but its position coding method and multi-head self-attention mechanism still need to be improved.Therefore,this paper proposes a continuous sign language recognition method based on the improved Transformer model.Through multiple multiplexed position codes with parameters,each word vector in the continuous hand sentence is calculated multiple times to accurately grasp the position information between each word,add learnable memory key-value pairs to the attention module to form a persistent memory module,and expand the number of attention heads and embedding dimensions through linear high-dimensional mapping and the like,to maximize the multi-head attention mechanism of the Transformer model,and the overall modeling ability of long sign language sequences,in-depth mining of key information in each frame of the video.The proposed method achieves competitive recognition results on the most authoritative continuous sign language data sets PHOENIX-Weather2014^[1] and PHOENIX-Weather2014-T^[2].

Key words: Continuous sign language recognition, Transformer, Multi-head attention, Position encoding