计算机科学 ›› 2022, Vol. 49 ›› Issue (7): 179-186.doi: 10.11896/jsjkx.210500190

• 人工智能 • 上一篇    下一篇

融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取

金方焱1, 王秀利1,2   

  1. 1 中央财经大学信息学院 北京102206
    2 国家金融安全教育部工程研究中心 北京102206
  • 收稿日期:2021-05-26 修回日期:2021-10-18 出版日期:2022-07-15 发布日期:2022-07-12
  • 通讯作者: 王秀利(wangxiuli@cufe.edu.cn)
  • 作者简介:(2020212350@email.cufe.edu.cn)

Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM

JIN Fang-yan1, WANG Xiu-li1,2   

  1. 1 College of Information,Central University of Finance and Economics,Beijing 102206,China
    2 Engineering Research Center of State Financial Security,Ministry of Education,Beijing 102206,China
  • Received:2021-05-26 Revised:2021-10-18 Online:2022-07-15 Published:2022-07-12
  • About author:JIN Fang-yan,born in 1998,postgra-duate.His main research interests include financial technology and natural language processing.
    WANG Xiu-li,born in 1977,Ph.D,professor,is a senior member of China Computer Federation.His main research interests include financial technology,artificialintelligence and security.

摘要: 金融领域的文本信息量大、价值高,尤其是其中的隐式因果关系事件包含着巨大的潜在利用价值。对金融领域文本进行隐式因果关系分析,挖掘隐式因果关系事件中隐含的重要信息,了解金融领域事件更深层的演化逻辑,进而构建金融领域知识库,对金融风险控制、风险预警等具有重要意义。为了提高金融领域中隐式因果关系事件识别的准确度,从特征挖掘的角度入手,提出了一种基于自注意力机制的融合循环注意力卷积神经网络(Recurrent Attention Convolution Neural Network,RACNN)和双向长短时记忆网络(Bidirectional Long Short-Term Memory,BiLSTM)的隐式因果关系抽取方法。该方法结合了基于迭代反馈机制能提取更重要文本局部特征的RACNN、能更好地提取文本全局特征的BiLSTM以及能更深入地挖掘融合特征语义信息的自注意力机制,在SemEval-2010 Task 8数据集和金融领域数据集上进行了实验,结果表明,评估指标F1值分别达到了72.98%和75.74%,均显著优于其他对比模型。

关键词: 迭代反馈机制, 金融领域, 双向长短时记忆网络, 循环注意力卷积神经网络, 隐式因果关系抽取, 自注意力机制

Abstract: The financial field has a large amount of information and high value,especially the implicit causal events which contains huge potential useful value.Carrying out causal analysis on financial domain text to mine the important information hidden in the implicit causal events,understanding the deeper evolutionary logic of the financial field events,to build a financial field knowledge base,which plays an important role in financial risk control and risk early warning.In order to improve the accuracy of identifying the implicit causal events in the financial field,from the perspective of feature mining,based on self-attention mechanism,an implicit causality extraction method integrating recurrent attention convolution neural network(RACNN) and bidirectional long short-term memory(BiLSTM) is proposed.This method combines RACNN that can extract more important local features of text based on an iterative feedback mechanism,BiLSTM that can better extract global features of text,and a self-attention mechanism that can more deeply dig the semantic information of fused features.Experimental results on SemEval-2010 Task 8 and financial field datasets show that the evaluation index F1 value can reach 72.98% and 75.74% respectively,which is significantly better than other comparison models.

Key words: BiLSTM, Financial field, Implicit causality extraction, Iterative feedback mechanism, RACNN, Self-attention mechanism

中图分类号: 

  • TP391
[1]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the Advances in Neural Information Processing Systems.Cambridge,MA:MIT Press,2017:5998-6008.
[2]FU J L,ZHENG H L,MEI T.Look closer to see better:Recurrent attention convolutional neural network for fine-grained image recognition[C]//Proceedings of the CVPR.Piscataway,NJ:IEEE,2017:4438-4446.
[3]HENDRICKX I,KIM S N,KOZAREVA Z,et al.Semeval-2010 task 8:Multi-way classification of semantic relations between pairs of nominals [C]//Proceedings of the Workshop on Semantic Evaluations:Recent Achievements and Future Directions(SEW-2009).Stroudsburg,PA:ACL,2009:94-99.
[4]SAKAJI H,MURONO R,SAKAI H,et al.Discovery of rarecausal knowledge from financial statement summaries[C]//Proceedings of 2017 IEEE Symp Series on Computational Intelligence(SSCI).Piscataway,NJ:IEEE,2017:1-7.
[5]IZUMI K,SAKAJI H.Economic causal-chain search using text mining technology[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence.Amsterdam:Else-vier,2019:23-35.
[6]CAO M Y,YANG Z H,LUO L,et al.Joint drug entities and relations extraction based on neural networks[J].Journal of Computer Research and Development,2019,56(7):1432-1440.
[7]XU J H,ZUO W L,LIANG S N,et al.Causal relation extraction based on graph attention networks[J].Journal of Computer Research and Development,2020,57(1):159-174.
[8]LI Z N,LI Q,ZOU X T,et al.Causality extraction based on self-attentive BiLSTM-CRF with transferred embeddings[J].Neurocomputing,2019,423:207-219.
[9]ZHONG J,YU L,TIAN S W,et al.Causal relation extraction of uyghur emergency events based on cascaded model[J].Acta Automatica Sinica,2014,40(4):771-779.
[10]ZHOU P,SHI W,TIAN J,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the ACL.Stroudsburg,PA:ACL,2016:207-212.
[11]TIAN S W,ZHOU X F,YU L,et al.Causal relation extraction of uyghur events based on bidirectional long short-term memory model[J].Journal of Electronics and Information Technology,2018,40(1):200-208.
[12]NING S M,TENG F,LI T R.Muti-channel self-attention mecha-nism for relation extraction in clinical records[J].Chinese Journal of Computers,2020,43(5):916-929.
[13]WANG J,SHI C H,ZHANG J,et al.Document-level event temporal relation extraction with context information[J].Journal of Computer Research and Development,2021,58(11):2475.
[14]TOURILLE J,FERRET O,NEVEOL A,et al.Neural architecture for temporal relation extraction:A bi-lstm approach for detecting narrative containers[C]//Proceedings of the 55th AnnualMeeting of the ACL.Stroudsburg,PA:ACL,2017:224-230.
[15]FENG X C,HUANG L F,TANG D Y,et al.A language-independent neural network for event detection[C]//Proceedings of the 54th Annual Meeting of the ACL.Stroudsburg,PA:ACL,2016:66-71.
[16]GUO F Y,HE R F,DANG J W.Implicit discourse relation reco-gnition via a BiLSTM-CNN architecture with dynamic chunk-based max pooling[J].IEEE Access,2019,7:169281-169292.
[17]WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision(ECCV).Berlin,Germany:Sprin-ger,2018:3-19.
[18]HOCHREITER S,SCHMIDHUBER J.Long short-term me-mory[J].Neural Computation,1997,9(8):1735-1780.
[19]PENNINGTON J,SOCHER R,MANNING C.Glove:Globalvectors for word representation[C]//Proceedings of Conference on the 2014 Empirical Methods in Natural Language Processing(EMNLP).Stroudsburg,PA:ACL,2014:1532-1543.
[20]LI S,ZHAO Z,HU R F,et al.Analogical reasoning on Chinese morphological and semantic relations[C]//Proceedings of the 56th Annual Meeting of the ACL.Stroudsburg,PA:ACL,2018:138-143.
[21]CAO P F,CHEN Y B,LIU K,et al.Adversarial transfer lear-ning for Chinese named entity recognition with self-attention mechanism[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing(EMNLP).Stroudsburg,PA:ACL,2018:182-192.
[22]LIU X,OU J,SONG Y,et al.On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence.Amsterdam:Elsevier,2020:3830-3836.
[23]GUO F,HE R,DANG J,et al.Working memory-driven neural networks with a novel knowledge enhancement paradigm for implicit discourse relation recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:7822-7829.
[24]JIANG D,HE J.Tree Framework With BERT Word Embedding for the Recognition of Chinese Implicit Discourse Relations[J].IEEE Access,2020,8:162004-162011.
[25]FAN Z W,ZHANG M,LI Z H.BiLSTM-based Implicit Discourse Relation Classification Combining Self-attention Mechanism and Syntactic Information[J].Computer Science,2019,46(5):214-220.
[1] 张嘉淏, 刘峰, 齐佳音.
一种基于Bottleneck Transformer的轻量级微表情识别架构
Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer
计算机科学, 2022, 49(6A): 370-377. https://doi.org/10.11896/jsjkx.210500023
[2] 赵丹丹, 黄德根, 孟佳娜, 董宇, 张攀.
基于BERT-GRU-ATT模型的中文实体关系分类
Chinese Entity Relations Classification Based on BERT-GRU-ATT
计算机科学, 2022, 49(6): 319-325. https://doi.org/10.11896/jsjkx.210600123
[3] 潘志豪, 曾碧, 廖文雄, 魏鹏飞, 文松.
基于交互注意力图卷积网络的方面情感分类
Interactive Attention Graph Convolutional Networks for Aspect-based Sentiment Classification
计算机科学, 2022, 49(3): 294-300. https://doi.org/10.11896/jsjkx.210100180
[4] 丁锋, 孙晓.
基于注意力机制和BiLSTM-CRF的消极情绪意见目标抽取
Negative-emotion Opinion Target Extraction Based on Attention and BiLSTM-CRF
计算机科学, 2022, 49(2): 223-230. https://doi.org/10.11896/jsjkx.210100046
[5] 胡艳丽, 童谭骞, 张啸宇, 彭娟.
融入自注意力机制的深度学习情感分析方法
Self-attention-based BGRU and CNN for Sentiment Analysis
计算机科学, 2022, 49(1): 252-258. https://doi.org/10.11896/jsjkx.210600063
[6] 徐少伟, 秦品乐, 曾建朝, 赵致楷, 高媛, 王丽芳.
基于多级特征和全局上下文的纵膈淋巴结分割算法
Mediastinal Lymph Node Segmentation Algorithm Based on Multi-level Features and Global Context
计算机科学, 2021, 48(6A): 95-100. https://doi.org/10.11896/jsjkx.200700067
[7] 王习, 张凯, 李军辉, 孔芳, 张熠天.
联合自注意力和循环网络的图像标题生成
Generation of Image Caption of Joint Self-attention and Recurrent Neural Network
计算机科学, 2021, 48(4): 157-163. https://doi.org/10.11896/jsjkx.200300146
[8] 周小诗, 张梓葳, 文娟.
基于神经网络机器翻译的自然语言信息隐藏
Natural Language Steganography Based on Neural Machine Translation
计算机科学, 2021, 48(11A): 557-564. https://doi.org/10.11896/jsjkx.210100015
[9] 张鹏飞, 李冠宇, 贾彩燕.
面向自然语言推理的基于截断高斯距离的自注意力机制
Truncated Gaussian Distance-based Self-attention Mechanism for Natural Language Inference
计算机科学, 2020, 47(4): 178-183. https://doi.org/10.11896/jsjkx.190600149
[10] 康雁,崔国荣,李浩,杨其越,李晋源,王沛尧.
融合自注意力机制和多路金字塔卷积的软件需求聚类算法
Software Requirements Clustering Algorithm Based on Self-attention Mechanism and Multi- channel Pyramid Convolution
计算机科学, 2020, 47(3): 48-53. https://doi.org/10.11896/jsjkx.190700146
[11] 王启发, 王中卿, 李寿山, 周国栋.
基于交叉注意力机制和新闻正文的评论情感分类
Comment Sentiment Classification Using Cross-attention Mechanism and News Content
计算机科学, 2020, 47(10): 222-227. https://doi.org/10.11896/jsjkx.190900173
[12] 张义杰, 李培峰, 朱巧明.
基于自注意力机制的事件时序关系分类方法
Event Temporal Relation Classification Method Based on Self-attention Mechanism
计算机科学, 2019, 46(8): 244-248. https://doi.org/10.11896/j.issn.1002-137X.2019.08.040
[13] 凡子威, 张民, 李正华.
基于BiLSTM并结合自注意力机制和句法信息的隐式篇章关系分类
BiLSTM-based Implicit Discourse Relation Classification Combining Self-attention
Mechanism and Syntactic Information
计算机科学, 2019, 46(5): 214-220. https://doi.org/10.11896/j.issn.1002-137X.2019.05.033
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!