计算机科学 ›› 2023, Vol. 50 ›› Issue (12): 294-301.doi: 10.11896/jsjkx.221000083

• 人工智能 • 上一篇    下一篇

融合多头注意力机制和孪生网络的语义匹配方法

臧洁, 周万林, 王妍   

  1. 辽宁大学信息学院 沈阳 110036
  • 收稿日期:2022-10-11 修回日期:2023-03-25 出版日期:2023-12-15 发布日期:2023-12-07
  • 通讯作者: 王妍(wang_yan@lnu.edu.cn)
  • 作者简介:(zangjie@lnu.edu.cn)
  • 基金资助:
    国家重点研发计划(2019YFB1405804)

Semantic Matching Method Integrating Multi-head Attention Mechanism and Siamese Network

ZANG Jie, ZHOU Wanlin, WANG Yan   

  1. College of Information,Liaoning University,Shenyang 110036,China
  • Received:2022-10-11 Revised:2023-03-25 Online:2023-12-15 Published:2023-12-07
  • About author:ZANG Jie,born in 1979,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include big data,optimization theory and application.
    WANG Yan,born in 1978,Ph.D,professor,is a member of China Computer Federation.Her main research interests include big data,Internet of Things and blockchain technology.
  • Supported by:
    National Key R & D Program of China(2019YFB1405804).

摘要: 考虑企业资源与客户需求匹配问题,现有的方法存在资源和需求封装不够准确以及匹配效果无法满足用户需求等问题。为解决企业资源与需求描述的多样性和歧义性,提出了动态自定义模板封装。针对封装后的需求与资源大多都是中文短文本这一特点,兼顾句子间语义的差异性和相似性,提出了融合多头注意力机制和孪生网络的交互型文本匹配模型。模型使用字词混向量作为输入增强文本的语义信息,将孪生网络与多头注意力机制相融合,作为独立单元提取上下文的语义特征并使语义特征充分交互。为了验证模型的有效性,在经典数据集LCQMC和自我构建的CSMD数据集上对模型进行了实验,结果表明所提模型在准确率和性能等方面均有不同程度的提升,为企业资源与需求提供了更精准的匹配方法。

关键词: 自定义模板, 语义匹配, 孪生网络, 多头注意力机制, 双向GRU

Abstract: Considering the matching problem of enterprise resources and customer requirements,the existing methods have the problems that the resource and requirement encapsulation is not accurate enough and the matching effect can't satisfy uses' requirement.In order to solve the problem of diversity and ambiguity of enterprise resource and requirement description,this paper proposes the dynamic user-defined template encapsulation.According to the feature that most of the encapsulated requirements and resources are Chinese short texts,an interactive text matching model which integrates multi-head attention mechanism and sia-mese network is proposed.The semantic differences and similarities between sentences are considered in this model.It uses word mixing vectors as input to enhance the semantic information of the text,combines the Siamese network with the multi-head attention mechanism,and extractes the semantic features of the context as an independent unit to fully interact with the semantic features.In order to verify the effectiveness of the model,the classical data set LCQMC and the self-constructed CSMD data set are used to conduct experiments on the model.The results show that the accuracy and performance of the model are improved in different degrees,which provides a more accurate matching method for enterprise resources and requirements.

Key words: Custom template, Semantic matching, Siamese network, Multi-head attention mechanism, Bidirectional GRU

中图分类号: 

  • TP391
[1]CAMPANA S,FRANCOVICH R.Landscape Archaeology inTuscany:Cultural resource management,remotely sensed techniques,GIS based data integration and interpretation[J].Bar International Series,2003,1151:15-28.
[2]CHEN K,WANG D,WANG Y H,et al.Research on Soft Resource Packaging of Cloud Manufacturing[J].Informatization of China's Manufacturing Industry,2012,41(5):58-63.
[3]JIANG J,PANG T,ZHANG F,et al.Pathway to encapsulatethe surface water quality model and its applications as cloud computing services and integration with EDSS for managing urban water environments[J].Environmental Modelling & Software,2022,148:105280.
[4]KONG L J,XU W S,ZHA J Z.Manufacturing resource packaging method based on service template[J].Computer Applications,2012,32(12):3534-3539.
[5]LIU Y J,DU H,NIYATO D,et al.Slicing4Meta:An Intelligent Integration Framework with Multi-dimensional Network Resources for Metaverse-as-a-Service in Web 3.0[J].arXiv:2208.06081,2022.
[6]LUCKOW A,RATTAN K,JHA S.Pilot-Edge:Distributed Resource Management Along the Edge-to-Cloud Continuum[C]//2021 IEEE International Parallel and Distributed Processing Symposium Workshops(IPDPSW).IEEE,2021:874-878.
[7]PAWLEWSKI P,ANHOLCER M.Relational database template in the simulation modeling of manufacturing systems[J].IFAC-Papers OnLine,2019,52(13):1744-1748.
[8]BOTEA A,MULLER M,SCHAEFFER J.Using Componentabstraction for automatic generation of macro-actions[C]//ICAPS.2004:181-190.
[9]ANTEQUERA R B,CALYAM P,CHANDRASHEKARA AA,et al.Recommending heterogeneous resources for science gateway applications based on custom templates composition[J].Future Generation Computer Systems,2019,100:281-297.
[10]CHEN X,JIANG G,LI G,et al.A new knowledgeable encapsulation method of steel production scheduling model[J].Procee-dings of the Institution of Mechanical Engineers,Part B:Journal of Engineering Manufacture,2020,234(14):1673-1685.
[11]TIAN Y,QIU L Y,WU Y F,et al.Microservice based Space based Information Support Tactical Terminal Application Ser-vice Architecture[J].Computer and Network,2021,47(16):64-68.
[12]HUANG P S,HE X,GAO J,et al.Learning deep structured se-mantic models for web search using clickthrough data[C]//Proceedings of the 22nd ACM International Conference on Information & Knowledge Management.2013:2333-2338.
[13]WANG X,YANG H.MGMSN:Multi-Granularity MatchingModel Based on Siamese Neural Network[J].Frontiers in Bioengineering and Biotechnology,2022,10:839586.
[14]MUELLER J,THYAGARAJAN A.Siamese recurrent architectures for learning sentence similarity[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016.
[15]DENG W B,ZHU K,LI Y B,et al.FMNN:Text Classification Model Based on Multi neural Network[J].Computer Science,2022,49(3):281-287.
[16]DENG Y,LI X Y CUI J,et al.Multi head attentional memory network for short text emotion classification[J].Computer Application,2021,41(11):3132-3138.
[17]ZHANG X,LU W,ZHANG G,et al.Chinese sentence semantic matching based on multi-granularity fusion model[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining.Cham:Springer,2020:246-257.
[18]CHEN Q,ZHU X,LING Z,et al.Enhanced LSTM for natural language inference[J].arXiv:1609.06038,2016.
[19]WU S H,PENG D L,YUAN W W,et al.MGSC:A Short Text Semantic Matching Model with Multi-granularity Semantic Intersection[J].Small and Microcomputer Systems,2019,40(6):1148-1152.
[20]CHEN Y,QIU X Y.Multi-task text semantic matching method combined with self-supervised learning[J].Journal of Peking University(Natural Science Edition),2022,58(1):83-90.
[21]MA X Y,FAN Y X,GUO J F,et al.Research and Analysis on Generalization and Migration of Short Text Matching[J].Computer Research and Development,2022,59(1):118-126.
[22]XU W F,YANG Y,ZHANG C F.A deep text semantic ma-tching model integrating entity context features[J].Journal of Wuhan University(Science Edition),2020,66(5):483-494.
[23]JUN Y S.jieba[EB/OL].https:github.com/fxsjy/jieba.
[24]CHO K,MERRIENBOER B V,GULCEHRE C,et al.Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation[C]//Proceedings of the 2014 Confe-rence on Empirical Methods in Natural Language Processing(EMNLP).2014:1724-1734.
[25]LIU X,CHEN Q,DENG C,et al.Lcqmc:A large-scale chinese question matching corpus[C]//Proceedings of the 27th International Conference on Computational Linguistics.2018:1952-1962.
[26]CHEN Y.Convolutional neural network for sentence classification[D].Waterloo:University of Waterloo,2015.
[27]MUELLER J,THYAGARAJAN A.Siamese recurrent architectures for learning sentence similarity[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016.
[28]WANG Z,HAMZA W,FLORIAN R.Bilateral multi-perspective matching for natural language sentences[J].arXiv:1702.03814,2017.
[29]YU B H,WANG J C.Research on Chinese Semantic Matching in Siamese Network[J].Small and Microcomputer System,2021,42(2):231-234.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!