关键词: SVM,主题爬虫,爬取策略,HITS

Abstract: With the rapid development of Internet,network information comes to be massive and diversity.How to provide the information required by users rapidly and exactly is the first task of the search engine.Traditional general search engine can provide the information in the general area,but in the special area,it cannot provide the professional and in-depth information for users.In this paper,the focused crawler based on the SVM classification algorithm was proposed for a solution to the problem of information retrieval in the special area,which makes use of the topic relevance predict algorithm based on the content and partial link information,the SVM classification algorithm and the HITS algorithm.The experiment shows that the crawling strategy based on the SVM classification algorithm can distinguish the topic related pages and topic unrelated Web pages better,improve the harvest rate and recall rate,and furthermore,the retrieval efficiency of search engines is improved.

Key words: SVM,Focused crawler,Crawling strategy,HITS