关键词: 自动构词,统计语言模型,关联规则,最长公共子序列,文本分类

Abstract: Words are the basic elements of Chinese text,and Chinese language model plays a key role in Chinese text mining.Text classification is a data mining technology with high dimensions and most of the classifying algorithms are sensitive to the dimensions.As a result,the classification depends on the quantity of vocabularies.Besides,most of current Chinese language models are based on statistical theory,such as N-gram model and other improved models.Howe-ver,these statistical models are disadvantaged with computational complexity.In order to improve the quantity and efficiency,this paper gave Chinese words a new definition based on association rules,and proposed the Auto-word algorithm,by which a word vocabulary is constructed automatically and used for Chinese text mining.Finally,the efficiency of the Auto-word algorithm was proved by experiment.

Key words: Constructing words automatically,Statistical language model,Association rules,Longest common subsequence,Text classification