计算机科学 ›› 2024, Vol. 51 ›› Issue (5): 172-178.doi: 10.11896/jsjkx.230200199
徐雪洁, 王宝会
XU Xuejie, WANG Baohui
摘要： 专利分类是专利数据挖掘领域一项非常重要的任务,该任务的目标是为给定专利文献分配若干个国际专利分类(IPC)号,近几年针对该任务的很多研究都集中在通过挖掘专利文本表示对IPC分类体系中部级或大类级分类号的多分类预测。而实际场景中,一篇专利往往有多个分类号,是一种多标签分类任务,且除了专利的文本内容外,每个专利都有对应的专利权组织,专利权组织的历史专利申请行为会有一定的业务倾向,这种申请行为的偏好表示能有效提高专利分类准确度。然而,目前专利分类的相关研究中并没有充分利用到专利的历史数据,针对IPC体系小类的多标签分类问题,提出了一个综合考虑专利内容的专利自动分类模型。首先用BERT预训练语言模型初始化专利文本表示,再利用Text-CNN捕捉局部特征获得将其输出作为专利文本的最终表示;其次,通过Bi-LSTM对历史专利文本及专利标签进行双通道聚合,学习该组织的历史专利申请行为表示;最后,将专利的文本表示与历史专利申请行为表示进行融合后做预测。在真实专利数据集上,将所提模型与基于专利文本挖掘的不同基线进行了对比实验,结果表明基于专利文本和历史数据建模的深度学习分类算法在精确度上有很大的提升。
中图分类号:
| [1]ABDELGAWAD L,KLUEGL P,GENC E,et al.Optimizingneural networks for patent classification[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Cham:Springer,2020:688-703. [2]LI X,CHEN H,ZHANG Z,et al.Automatic patent classification using citation network information:an experimental study in nanotechnology[C]//Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries.2007:419-427. [3]DERIEUX F,BOBEICA M,POIS D,et al.Combining semantics and statistics for patent classification[C]//DBLP.2010. [4]VERBERNE S,D'HONDT E.Patent classification experiments with the Linguistic Classification System LCS in CLEF-IP 2011[C]//CLEF.2011. [5]BAO X,LIU G F,CUI J H.Application of Multi Instance MultiLabel Learning in Chinese Patent Automatic Classification[J].Library and Information Service,2021,65(8):107-113. [6]FALLC J,TÖRCSVÁRI A,BENZINEB K,et al.Automatedcategorization in the international patent classification[J].ACM SIGIR Forum,2003,37(1):10-25. [7]DAI P J,HE C L,SHANYUE Y R.XGBoost-based Classification of Multi-label Texts of Pharmaceutical Patent[J].Journal of Neijiang Normal University,2021,36(10):55-60. [8]HAGHIGHIAN ROUDSARI A,AFSHAR J,LEE W,et al.PatentNet:multi-label classification of patent documents using deep learning based language understanding[J].Scientometrics,2022,127(1):207-231. [9]JUNG G,SHIN J,LEE S.Impact of preprocessing and word embedding on extreme multi-label patent classification tasks[J].Applied Intelligence,2023,53(4):4047-4062. [10]GOMEZ J C,MOENS M F.A survey of automated hierarchical classification of patents[M]//Professional Search in the Modern World.Cham:Springer,2014:215-249. [11]TIAN C,ZHAO Y J.A mapping model of patent and industry category based on similarity:A case study of International Patent Classification and Trade Classification of National Economy[J].Library and Information Service,2016,60(20):123. [12]ELMAN J L.Finding structure in time[J].Cognitive science,1990,14(2):179-211. [13]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural computation,1997,9(8):1735-1780. [14]CHO K,VAN MERRIËNBOER B,GULCEHRE C,et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[J].arXiv:1406.1078,2014. [15]GRAVES A.Generating sequences with recurrent neural networks[J].arXiv:1308.0850,2013. [16]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing SystemsDecember.2017:6000-6010. [17]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013. [18]PENNINGTON J,SOCHER R,MANNING C D.Glove:Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing(EMNLP).2014:1532-1543. [19]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018. [20]WOLF T,DEBUT L,SANH V,et al.Huggingface's transfor-mers:State-of-the-art natural language processing[J].arXiv:1910.03771,2019. [21]LIU Y,OTT M,GOYAL N,et al.Roberta:A robustly opti-mized bert pretraining approach[J].arXiv:1907.11692,2019. [22]GRAWE M F,MARTINS C A,BONFANTE A G.Automated patent classification using word embedding[C]//2017 16th IEEE International Conference on Machine Learning and Applications(ICMLA).IEEE,2017:408-411. [23]LI S,HU J,CUI Y,et al.DeepPatent:patent classification with convolutional neural networks and word embedding[J].Scientometrics,2018,117(2):721-744. [24]SHALABY M,STUTZKI J,SCHUBERT M,et al.An lstm approach to patent classification based on fixed hierarchy vectors[C]//Proceedings of the 2018 SIAM International Conference on Data Mining.Society for Industrial and Applied Mathema-tics,2018:495-503. [25]HUANG W,CHEN E,LIU Q,et al.Hierarchical multi-labeltext classification:An attention-based recurrent network approach[C]//Proceedings of the 28th ACM International Confe-rence on Information and Knowledge Management.2019:1051-1060. [26]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019,33(1):7370-7377. [27]TANG P,JIANG M,XIA B N,et al.Multi-label patent categorization with non-local attention-based graph convolutional network[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020,34(5):9024-9031. [28]ROUDSARI A H,AFSHAR J,LEE C C,et al.Multi-label patent classification using attention-aware deep learning model[C]//2020 IEEE International Conference on Big Data and Smart Computing(BigComp).IEEE,2020:558-559. [29]GOMEZ J C.Analysis of the effect of data properties in automated patent classification[J].Scientometrics,2019,121(3):1239-1268. [30]LYU L,HAN T.A comparative study of Chinese patent literature automatic classification based on deep learning[C]//2019 ACM/IEEE Joint Conference on Digital Libraries(JCDL).IEEE,2019:345-346. [31]FANG L,ZHANG L,WU H,et al.Patent2Vec:Multi-view representation learning on patent-graphs for patent classification[J].World Wide Web,2021,24(5):1791-1812. [32]SHEN J,QIU W,MENG Y,et al.TaxoClass:Hierarchicalmulti-label text classification using only class names[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2021:4239-4249. [33]ZHAO H Y,CAO J,CHEN Q K,et al.Methods for Hierarchical Multi-label Text Classification.Journal of Chinese Computer Systems.2022,43(4):673-683. |
|
||
首页