Computer Science ›› 2023, Vol. 50 ›› Issue (12): 294-301.doi: 10.11896/jsjkx.221000083

• Artificial Intelligence • Previous Articles     Next Articles

Semantic Matching Method Integrating Multi-head Attention Mechanism and Siamese Network

ZANG Jie, ZHOU Wanlin, WANG Yan   

  1. College of Information,Liaoning University,Shenyang 110036,China
  • Received:2022-10-11 Revised:2023-03-25 Online:2023-12-15 Published:2023-12-07
  • About author:ZANG Jie,born in 1979,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include big data,optimization theory and application.
    WANG Yan,born in 1978,Ph.D,professor,is a member of China Computer Federation.Her main research interests include big data,Internet of Things and blockchain technology.
  • Supported by:
    National Key R & D Program of China(2019YFB1405804).

Abstract: Considering the matching problem of enterprise resources and customer requirements,the existing methods have the problems that the resource and requirement encapsulation is not accurate enough and the matching effect can't satisfy uses' requirement.In order to solve the problem of diversity and ambiguity of enterprise resource and requirement description,this paper proposes the dynamic user-defined template encapsulation.According to the feature that most of the encapsulated requirements and resources are Chinese short texts,an interactive text matching model which integrates multi-head attention mechanism and sia-mese network is proposed.The semantic differences and similarities between sentences are considered in this model.It uses word mixing vectors as input to enhance the semantic information of the text,combines the Siamese network with the multi-head attention mechanism,and extractes the semantic features of the context as an independent unit to fully interact with the semantic features.In order to verify the effectiveness of the model,the classical data set LCQMC and the self-constructed CSMD data set are used to conduct experiments on the model.The results show that the accuracy and performance of the model are improved in different degrees,which provides a more accurate matching method for enterprise resources and requirements.

Key words: Custom template, Semantic matching, Siamese network, Multi-head attention mechanism, Bidirectional GRU

CLC Number: 

  • TP391
[1]CAMPANA S,FRANCOVICH R.Landscape Archaeology inTuscany:Cultural resource management,remotely sensed techniques,GIS based data integration and interpretation[J].Bar International Series,2003,1151:15-28.
[2]CHEN K,WANG D,WANG Y H,et al.Research on Soft Resource Packaging of Cloud Manufacturing[J].Informatization of China's Manufacturing Industry,2012,41(5):58-63.
[3]JIANG J,PANG T,ZHANG F,et al.Pathway to encapsulatethe surface water quality model and its applications as cloud computing services and integration with EDSS for managing urban water environments[J].Environmental Modelling & Software,2022,148:105280.
[4]KONG L J,XU W S,ZHA J Z.Manufacturing resource packaging method based on service template[J].Computer Applications,2012,32(12):3534-3539.
[5]LIU Y J,DU H,NIYATO D,et al.Slicing4Meta:An Intelligent Integration Framework with Multi-dimensional Network Resources for Metaverse-as-a-Service in Web 3.0[J].arXiv:2208.06081,2022.
[6]LUCKOW A,RATTAN K,JHA S.Pilot-Edge:Distributed Resource Management Along the Edge-to-Cloud Continuum[C]//2021 IEEE International Parallel and Distributed Processing Symposium Workshops(IPDPSW).IEEE,2021:874-878.
[7]PAWLEWSKI P,ANHOLCER M.Relational database template in the simulation modeling of manufacturing systems[J].IFAC-Papers OnLine,2019,52(13):1744-1748.
[8]BOTEA A,MULLER M,SCHAEFFER J.Using Componentabstraction for automatic generation of macro-actions[C]//ICAPS.2004:181-190.
[9]ANTEQUERA R B,CALYAM P,CHANDRASHEKARA AA,et al.Recommending heterogeneous resources for science gateway applications based on custom templates composition[J].Future Generation Computer Systems,2019,100:281-297.
[10]CHEN X,JIANG G,LI G,et al.A new knowledgeable encapsulation method of steel production scheduling model[J].Procee-dings of the Institution of Mechanical Engineers,Part B:Journal of Engineering Manufacture,2020,234(14):1673-1685.
[11]TIAN Y,QIU L Y,WU Y F,et al.Microservice based Space based Information Support Tactical Terminal Application Ser-vice Architecture[J].Computer and Network,2021,47(16):64-68.
[12]HUANG P S,HE X,GAO J,et al.Learning deep structured se-mantic models for web search using clickthrough data[C]//Proceedings of the 22nd ACM International Conference on Information & Knowledge Management.2013:2333-2338.
[13]WANG X,YANG H.MGMSN:Multi-Granularity MatchingModel Based on Siamese Neural Network[J].Frontiers in Bioengineering and Biotechnology,2022,10:839586.
[14]MUELLER J,THYAGARAJAN A.Siamese recurrent architectures for learning sentence similarity[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016.
[15]DENG W B,ZHU K,LI Y B,et al.FMNN:Text Classification Model Based on Multi neural Network[J].Computer Science,2022,49(3):281-287.
[16]DENG Y,LI X Y CUI J,et al.Multi head attentional memory network for short text emotion classification[J].Computer Application,2021,41(11):3132-3138.
[17]ZHANG X,LU W,ZHANG G,et al.Chinese sentence semantic matching based on multi-granularity fusion model[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining.Cham:Springer,2020:246-257.
[18]CHEN Q,ZHU X,LING Z,et al.Enhanced LSTM for natural language inference[J].arXiv:1609.06038,2016.
[19]WU S H,PENG D L,YUAN W W,et al.MGSC:A Short Text Semantic Matching Model with Multi-granularity Semantic Intersection[J].Small and Microcomputer Systems,2019,40(6):1148-1152.
[20]CHEN Y,QIU X Y.Multi-task text semantic matching method combined with self-supervised learning[J].Journal of Peking University(Natural Science Edition),2022,58(1):83-90.
[21]MA X Y,FAN Y X,GUO J F,et al.Research and Analysis on Generalization and Migration of Short Text Matching[J].Computer Research and Development,2022,59(1):118-126.
[22]XU W F,YANG Y,ZHANG C F.A deep text semantic ma-tching model integrating entity context features[J].Journal of Wuhan University(Science Edition),2020,66(5):483-494.
[23]JUN Y S.jieba[EB/OL].https:github.com/fxsjy/jieba.
[24]CHO K,MERRIENBOER B V,GULCEHRE C,et al.Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation[C]//Proceedings of the 2014 Confe-rence on Empirical Methods in Natural Language Processing(EMNLP).2014:1724-1734.
[25]LIU X,CHEN Q,DENG C,et al.Lcqmc:A large-scale chinese question matching corpus[C]//Proceedings of the 27th International Conference on Computational Linguistics.2018:1952-1962.
[26]CHEN Y.Convolutional neural network for sentence classification[D].Waterloo:University of Waterloo,2015.
[27]MUELLER J,THYAGARAJAN A.Siamese recurrent architectures for learning sentence similarity[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016.
[28]WANG Z,HAMZA W,FLORIAN R.Bilateral multi-perspective matching for natural language sentences[J].arXiv:1702.03814,2017.
[29]YU B H,WANG J C.Research on Chinese Semantic Matching in Siamese Network[J].Small and Microcomputer System,2021,42(2):231-234.
[1] LUO Yuanyuan, YANG Chunming, LI Bo, ZHANG Hui, ZHAO Xujian. Chinese Medical Named Entity Recognition Method Incorporating Machine ReadingComprehension [J]. Computer Science, 2023, 50(9): 287-294.
[2] CUI Lin, CUI Chenlu, LIU Zhengwei, XUE Kai. Speech Emotion Recognition Based on Improved MFCC and Parallel Hybrid Model [J]. Computer Science, 2023, 50(6A): 220800211-7.
[3] LUO Huilan, LONG Jun, LIANG Miaomiao. Attentional Feature Fusion Approach for Siamese Network Based Object Tracking [J]. Computer Science, 2023, 50(6A): 220300237-9.
[4] LI Haitao, WANG Ruimin, DONG Weiyu, JIANG Liehui. Semi-supervised Network Traffic Anomaly Detection Method Based on GRU [J]. Computer Science, 2023, 50(3): 380-390.
[5] LI Zhifei, ZHAO Yue, ZHANG Yan. Survey of Knowledge Graph Reasoning Based on Representation Learning [J]. Computer Science, 2023, 50(3): 94-113.
[6] WANG Chao, WANG Kai. Visual Object Tracking Based on Adaptive Search Range Adjustment [J]. Computer Science, 2023, 50(11A): 221000172-6.
[7] LI Xuehui, ZHANG Yongjun, SHI Dianxi, XU Huachi, SHI Yanyan. AFTM:Anchor-free Object Tracking Method with Attention Features [J]. Computer Science, 2023, 50(1): 138-146.
[8] CHEN Kun-feng, PAN Zhi-song, WANG Jia-bao, SHI Lei, ZHANG Jin. Moderate Clothes-Changing Person Re-identification Based on Bionics of Binocular Summation [J]. Computer Science, 2022, 49(8): 165-171.
[9] ZHAO Yue, YU Zhi-bin, LI Yong-chun. Cross-attention Guided Siamese Network Object Tracking Algorithm [J]. Computer Science, 2022, 49(3): 163-169.
[10] XIAO Ding, ZHANG Yu-fan, JI Hou-ye. Electricity Theft Detection Based on Multi-head Attention Mechanism [J]. Computer Science, 2022, 49(1): 140-145.
[11] CHENG Xu, CUI Yi-ping, SONG Chen, CHEN Bei-jing, ZHENG Yu-hui, SHI Jin-gang. Object Tracking Algorithm Based on Temporal-Spatial Attention Mechanism [J]. Computer Science, 2021, 48(4): 123-129.
[12] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[13] ZHANG Zhi-yang, ZHANG Feng-li, CHEN Xue-qin, WANG Rui-jin. Information Cascade Prediction Model Based on Hierarchical Attention [J]. Computer Science, 2020, 47(6): 201-209.
[14] LI Hao, LIU Yong-jian, XIE Qing, TANG Ling-li. Distant Supervision Relation Extraction Model Based on Multi-level Attention Mechanism [J]. Computer Science, 2019, 46(10): 252-257.
[15] . Research on Petri Net Based OWL-S Semantic Matching Mechanism [J]. Computer Science, 2012, 39(10): 174-176.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!