Abstract: With the intensification of trade frictions between China and the United States,the development of domestic accelerator chips in China has become increasingly urgent.The Hygon DCU,with its CUDA-like architecture,excellent compatibility,and cost-effectiveness,has emerged as a strong candidate to replace high-end American chips in the field of artificial intelligence.However,on the Hygon DCU platform,the performance of the GEMM kernel function,which is a critical operator in large language model training,varies significantly.This paper investigates the impact of matrix transposition on the performance of the GEMM kernel function in the rocBLAS algorithm library and proposes two optimization methods:minimizing transposition and adaptive transposition,to effectively reduce the training time of large language models.This study modifies the implementation of the linear layer in PyTorch and proposes the minimization and adaptation of transposition methods for distributed training of large language models.Experimental results show that these two optimization methods significantly reduce training time in the distri-buted training of various large-scale language models,such as OPT-6.7B,LLaMA-7B,and Bloom-7B.Among the 83 test cases,the adaptive transposition method outperformes in 72 cases,with the highest improvement of 24.27% in end-to-end training time compared to the original PyTorch-based Megatron-LM.

Key words: Large language model, Training system, Hygon DCU, Matrix transpose