计算机科学 ›› 2009, Vol. 36 ›› Issue (8): 201-204.

• 人工智能 • 上一篇    下一篇

基于LSH的中文文本快速检索

蔡衡,李舟军,孙健,李洋   

  1. (北京航空航天大学计算机学院 北京 100083);(新浪网技术(中国)有限公司研发中心一搜索新技术部 北京 100191)
  • 出版日期:2018-11-16 发布日期:2018-11-16
  • 基金资助:
    本文受国家自然科学基金项目(60573057,90718017)资助。

Fast Chinese Text Search Based on LSH

CAI Heng, LI Zhou-Jun,SUN Jian,LI Yang   

  • Online:2018-11-16 Published:2018-11-16

摘要: 目前,高维数据的快速检索问题已经受到越来越多的关注。当向量空间的维度高于10时,R-tree, Kd-tree,SR-tree的检索效率反而不如线性检索,而位置敏感的哈希((Locality Sensitive Hashing,缩写为LSH)算法成功地解决了高维近邻数据的快速检索问题,因而受到国内外学术界的高度关注。首先介绍了工.SH算法的基本原理和方法,然后使用多重探测的方法对二进制向量的LSH算法做了进一步改进。最后实现了这两种 LSH算法,并通过详细的实验验证表明:在改进后的算法中,通过增加偏移量可

关键词: 高维数据,相似性检索,位置敏感的哈希,近部,多重探测

Abstract: The query of High dimension data attracts more and more attention. When dimension of a space vector is higher than 10, R-tree, Kd-tree, SR-tree and Quadtrecs perform worse than linear query. However, Locality Sensitive hashing (LSH) algorithm successfully deals with this problem. Nowadays LSH is playing a more and more important role in high dimension query. In the paper, the basic algorithm and principle of LSH were introduced firstly, then binary vector LSH Search Algorithm was improved by means of the multi-probe. Finally, we implemented the two kinds of LSH algorithms. The experience we have designed verified that the revised algorithm has better performance than the original one in two aspects. On the one hand, as the increment of setover, the proportion of retrial recall enlarges. On the other hand, the complexity of space decreases without the change of time complexity.

Key words: High dimension data, Similarity search, Locality sensitive hashing, Near neighbor, Multi-probe

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!