Computer Science ›› 2020, Vol. 47 ›› Issue (2): 157-162.doi: 10.11896/jsjkx.190100167

• Artificial Intelligence • Previous Articles     Next Articles

Distant Supervised Relation Extraction Based on Densely Connected Convolutional Networks

QIAN Xiao-mei1,LIU Jia-yong1,CHENG Peng-sen1,2   

  1. (School of Cybersecurity,Sichuan University,Chengdu 610000,China)1;
    (Key Laboratory of Network Assessment Technology,Institute of Information Engineering,Chinese Academy of Sciences,Beijing 100093,China)2
  • Received:2019-01-23 Online:2020-02-15 Published:2020-03-18
  • About author:QIAN Xiao-mei,born in 1995,postgra-duate.Her main research interests include information content security and so on;CHENG Peng-sen,born in 1988,Ph.D candidate.His main research interests include information content security and so on.
  • Supported by:
    This work was supported by Open Research Fund of the Key Laboratory of Network Assessment Technology of Chinese Academy of Sciences (NST-18-001).

Abstract: Densely connected convolutional networks (DenseNet) is a new architecture of deep convolutional neural network.By using identity mapping for shortcut connections between different layers,it can ensure the maximum information transmission of neural network.In the distant supervised relation extraction task,precious models use shallow convolution neural networks to extract features of a sentence which can only represent partial semantic information.To enhance the representation power of network,a deep convolutional neural network model based on dense connectivity was designed to encode sentences.The proposed model consists of five layers of densely connected convolutional neural networks.It can capture more semantic information by combining different levels of lexical,syntactic,and semantic features.At the same time,it can alleviate the phenomenon of gradient disappearance of deep neural network,which makes the network more capable of characterizing natural language.The experimental results on NYT-Freebase datasets show that the mean accuracy of the proposed model achieves 82.5%,and the PR curve area achieves 0.43.Experimental results show that the proposed model can effectively utilize features and improve the accuracy of distant supervised relation extraction.

Key words: Convolutional neural network, Deep learning, Dense connectivity, Distant supervision, Relation extraction

CLC Number: 

  • TP391
[1]KUMAR S.A Survey of Deep Learning Methods for Relation Extraction[J].arXiv:1705.03645.
[2]RINK B,HARABAGIU S.Utd:Classifying semantic relations by combining lexical and semantic resources[C]∥Proceedings of the 5th International Workshop on Semantic Evaluation.Association for Computational Linguistics,ACL Anthology,Stroudsburg,PA,2010:256-259.
[3]BUNESCU R C,MOONEY R J.A shortest path dependency kernel for relation extraction[C]∥Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing.Association for Computational Linguistics(ACL),Stroudsburg,PA,2005:724-731.
[4]ZENG D,LIU K,LAI S,et al.Relation classification via convolutional deep neural network[C]∥Proceedings of COLING 2014,the 25th International Conference on Computational Linguistics:Technical Papers.Association for Computational Linguistics,ACL Anthology,Stroudsburg,PA,2014:2335-2344.
[5]EBRAHIMI J,DOU D.Chain based RNN for relation classification[C]∥Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Association for Computational Linguistics (ACL),Stroudsburg,PA,2015:1244-1249.
[6]XU Y,MOU L,LI G,et al.Classifying relations via long short term memory networks along shortest dependency paths[C]∥Proceedings of the 2015 conference on empirical methods in na-tural language processing.Association for Computational Linguistics (ACL),Stroudsburg,PA,2015:1785-1794.
[7]MINTZ M,BILLS S,SNOW R,et al.Distant supervision for relation extraction without labeled data[C]∥Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP.Association for Computational Linguistics(ACL),Stroudsburg,PA,2009:1003-1011.
[8]ZENG D,LIU K,CHEN Y,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]∥Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics (ACL),Stroudsburg,PA,2015:1753-1762.
[9]LIN Y,SHEN S,LIU Z,et al.Neural relation extraction with selective attention over instances[C]∥Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics (ACL),Stroudsburg,PA,2016:2124-2133.
[10]ZHOU P,SHI W,TIAN J,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]∥Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics (ACL),Stroudsburg,PA,2016:207-212.
[11]HUANG Y Y,WANG W Y.Deep Residual Learning for Weakly-Supervised Relation Extraction[J].arXiv:1707.08866.
[12]HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely connected convolutional networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscata-way,NJ,2017:4700-4708.
[13]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed rep-resentations of words and phrases and their compositionality[C]∥Advances in Neural Information Processing Systems.Neural Information Processing Systems Foundation.2013:3111-3119.
[14]NASRABADI N M.Pattern recognition and machine learning.[J].Journal of Electronic Imaging,2007,16(4):049901.
[15]IOFFE S,SZEGEDY C.Batch normalization:Accelerating deep network training by reducing internal covariate shift[J].arXiv:1502.03167.
[16]GLOROT X,BORDES A,BENGIO Y.Deep sparse rectifier neural networks[C]∥Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics.Microtome Publishing,Menlo Park,CA,2011:315-323.
[17]SRIVASTAVA N,HINTON G,KRIZHEVSKY A,et al.Dropout:a simple way to prevent neural networks from overfitting[J].The Journal of Machine Learning Research,2014,15(1):1929-1958.
[18]HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Socie-ty,Los Alamitos,CA,2016:770-778.
[19]DE BOER P T,KROESE D P,MANNOR S,et al.A tutorial on the cross-entropy method[J].Annals of Operations Research,2005,134(1):19-67.
[20]RIEDEL S,YAO L,MCCALLUM A.Modeling relations and their mentions without labeled text[C]∥Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Berlin:Springer,2010:148-163.
[1] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[2] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[3] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[4] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[5] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[6] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[9] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[10] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[11] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[12] DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] LIU Yue-hong, NIU Shao-hua, SHEN Xian-hao. Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(7): 127-131.
[15] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
Full text



No Suggested Reading articles found!