Abstract: With the rapid development of the internet,various functional apps have emerged,allowing people to carry out various activities online.All kinds of goods,news,advertisements,and other information continue to be generated and spread on the internet.At the same time,engineers in the field of recommendation algorithms are constantly collecting useful features to iteratively optimize the algorithm’s effectiveness.From early collection of portrait features,evolving to user behavior logs,historical beha-vior statistics,and the current research on user behavior sequence features,the field of recommendation algorithms has now esta-blished a complete feature engineering paradigm.In recent years,it has been discovered that the historical behavior sequence of users is a very important feature.However,the semantic embeddings that can be obtained solely by item IDs are very limited and cannot automatically intersect with other related information,resulting in limited benefits in algorithm effectiveness.Since last year,the introduction of language models has achieved significant results in both the academic and industrial spheres,and engineers in the field of recommendation algorithms have also made some attempts.We propose a recommendation algorithm enhancement based on language models to enhance user behavior sequence features.By leveraging the semantic analysis and logical reasoning ability of language models,pre-training representation learning of user behavior sequence features is used to achieve feature enhancement,ultimately improving the effectiveness of recommendation algorithm models.

Key words: Recommendation algorithm, Language model, Sequence characteristics, Feature augmentation