Computer Science ›› 2024, Vol. 51 ›› Issue (6A): 230700226-5.doi: 10.11896/jsjkx.230700226

• Artificial Intelligenc • Previous Articles     Next Articles

Study on Hypernymy Recognition Based on Combined Training of Attention Mechanism and Prompt Learning

BAI Yu, WANG Xinzhe   

  1. Research Center for HCI,Shenyang Aerospace University,Shenyang 110136,China
  • Published:2024-06-06
  • About author:BAI Yu,born in 1982,lecturer.His main research interests is natural language processing.
    WANG Xinzhe,born in 1998,postgra-duate.Her main research interests include natural language processing and relationship recognition.
  • Supported by:
    Liaoning Province Applied Basic Research Program(2022JH2/101300248).

Abstract: The hypernymy between patent terms is an important semantic relationship.The identification of hypernymy between terms in patent text plays an important role in patent retrieval,query expansion,knowledge graph construction and other fields.However,due to the diversity of patent field and the complexity of language expression,the task of identifying the hypernymy between terms still faces many challenges.This paper proposes a method to recognize the hypernymy of terms by integrating prompt learning and attention mechanism.This method is based on the distantly supervised framework,and uses the shortest dependent path between terms as an auxiliary feature to integrate into the prompt template.Graph neural network is used to integrate the common information between terms into the joint training process of prompt learning and attention mechanism.Expe-rimental results on the patent text test dataset show that the AUC and f1 value of our method reache 94.94% and 89.33%,respectively,which are 3.82% and 3.17% higher than the PARE model.This method effectively removes the noise of the dataset annotated using distantly supervised methods,avoids the mismatch problem between the training target of the masked language model and downstream tasks,and fully utilizes the language knowledge information existing in the pre-trained language model.

Key words: Term relationship recognition, Distant supervision, Prompt learning, Attention mechanism, Hypernymy

CLC Number: 

  • TP391
[1]CHEN Y,HUANG Y Y,FANG J G.Paten Infomation Collection and Analysis[M].Beijing:Tsinghua University Press,2006.
[2]MILLER G A.WordNet:A lexical database for English[J].Communications of the ACM,1995,38(11):39-41.
[3]ZHANG S J,CHEN J X,WU X Y.A Novel Distant supervision Relation Extraction Approach Based on Sentence Bag Attention[J].Computer Application and Software,2022,39(8):193-203.
[4]MINTZ M,BILLS S,SNOW R,et al.Distant supervision for relation extraction without labeled data[C]//Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP.2009:2-7.
[5]ZENG D,LIU K,CHEN Y,et al.Distant Supervision for Rela-tion Extraction via Piecewise Convolutional Neural Networks[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1753-1762.
[6]JIANG X T,WANG Q,LI P,et al.Relation Extraction withMulti-instance Multi-label Convolutional Neural Networks[C]//Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers.2016:1471-1480.
[7]LIN Y K,SHEN S Q,LIU Z Y,et al.Neural Relation Extraction with Selective Attention over Instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2016:2124-2133.
[8]QIN P D,XU W R,WANG W Y.DSGAN:Generative Adversarial Training for Distant Supervision Relation Extraction[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2018:496-505.
[9]YUAN Y,LIU L,TANG S,et al.Cross-Relation Cross-Bag Attention for Distantly-Supervised Relation Extraction[C]/Proceedings of the Conference on Artificial Intelligence.Association for the Advancement of Artificial Intelligence(AAAI).2019.
[10]CHEN T,SHI H Z,TANG S L,et al.CIL:Contrastive instance learning framework for distantly supervised relation extraction[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 1:Long Papers).2021:6191-6200.
[11]LI D,ZHANG T,HU N,et al.HiCLRE:A Hierarchical Con-trastive Learning Framework for Distantly Supervised Relation Extraction[C]//Proceedings of the Conference on the Association for Computational Linguistics.2022:2567-2578.
[12]RATHORE V,BADOLA K,SINGLA P,et al.PARE:A Simple and Strong Baseline for Monolingual and Multilingual Distantly Supervised Relation Extraction[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics.2022:340-354.
[1] LIU Xiaohu, CHEN Defu, LI Jun, ZHOU Xuwen, HU Shan, ZHOU Hao. Speaker Verification Network Based on Multi-scale Convolutional Encoder [J]. Computer Science, 2024, 51(6A): 230700083-6.
[2] ZHU Yuliang, LIU Juntao, RAO Ziyun, ZHANG Yi, CAO Wanhua. Knowledge Reasoning Model Combining HousE with Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230600209-8.
[3] WANG Guogang, DONG Zhihao. Lightweight Image Semantic Segmentation Based on Attention Mechanism and Densely AdjacentPrediction [J]. Computer Science, 2024, 51(6A): 230300204-8.
[4] ZHAI Yunkai, QIAO Zhengwen, QIAO Yan. Forecasting Teleconsultation Demand Based on LSTM and Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230800119-7.
[5] ZHANG Lanxin, XIANG Ling, LI Xianze, CHEN Jinpeng. Intelligent Fault Diagnosis Method for Rolling Bearing Based on SAMNV3 [J]. Computer Science, 2024, 51(6A): 230700167-6.
[6] ZHANG Le, YU Ying, GE Hao. Mural Inpainting Based on Fast Fourier Convolution and Feature Pruning Coordinate Attention [J]. Computer Science, 2024, 51(6A): 230400083-9.
[7] SUN Yang, DING Jianwei, ZHANG Qi, WEI Huiwen, TIAN Bowen. Study on Super-resolution Image Reconstruction Using Residual Feature Aggregation NetworkBased on Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230600039-6.
[8] QUE Yue, GAN Menghan, LIU Zhiwei. Object Detection with Receptive Field Expansion and Multi-branch Aggregation [J]. Computer Science, 2024, 51(6A): 230600151-6.
[9] HE Xinyu, LU Chenxin, FENG Shuyi, OUYANG Shangrong, MU Wentao. Ship Detection and Recognition of Optical Remote Sensing Images for Embedded Platform [J]. Computer Science, 2024, 51(6A): 230700117-7.
[10] WU Chunming, WANG Tiaojun. Study on Defect Detection Algorithm of Transmission Line in Complex Background [J]. Computer Science, 2024, 51(6A): 230500178-6.
[11] LYU Yiming, WANG Jiyang. Iron Ore Image Classification Method Based on Improved Efficientnetv2 [J]. Computer Science, 2024, 51(6A): 230600212-6.
[12] WU Chunming, LIU Yali. Method for Lung Nodule Detection on CT Images Using Improved YOLOv5 [J]. Computer Science, 2024, 51(6A): 230500019-6.
[13] XIAO Yahui, ZHANG Zili, HU Xinrong, PENG Tao, ZHANG Jun. Clothing Image Segmentation Method Based on Deeplabv3+ Fused with Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230900153-7.
[14] LANG Lang, CHEN Xiaoqin, LIU Sha, ZHOU Qiang. Detection of Pitting Defects on the Surface of Ball Screw Drive Based on Improved Deeplabv3+ Algorithm [J]. Computer Science, 2024, 51(6A): 240200058-6.
[15] QIAO Hong, XING Hongjie. Attention-based Multi-scale Distillation Anomaly Detection [J]. Computer Science, 2024, 51(6A): 230300223-11.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!