Event detection is an important task in information extraction field,which aims to identify trigger words in raw text and then classify them into correct event types.Neural network based methods usually regard event detection as a word-wise classification task,which suffers from the mismatch problem between words and triggers when applied to Chinese.Besides,due to the multiple word senses of a trigger word,the same trigger word in different sentences causes the ambiguity problem.To address the two problems in Chinese event detection,we propose a Chinese event detection model with hierarchical and multi-granularity semantic fusion.First,we adopt a character-based sequence labelling method to solve the mismatch problem,in which we devise a Character-Word Fusion Gate to capture the semantic information of words in different segmentation ways.Then we device a Character-Sentence Fusion Gate to learn a character-word-sentence hybrid representation of sequence,which takes the semantic information of the entire sentence into condition and solves the ambiguity problem.Finally,in order to balance the influence the label “O” and the other labels,a loss function with bias is applied to train our model.The experimental results on the widely used ACE2005 dataset show that our approach outperforms at least 3.9%,1.4% and 2.9% than other Chinese event detection models under the metrics of accuracy (Precision,P),recall (Recall,R) and F1.

Natural language is the crystallization of human wisdom,and interacting with computers in the form of natural language has long been expected.With the development of natural language processing technology and the rise of deep learning methods,human-computer dialogue systems have become a new research hotspot.Human-computer dialogue systems can be divided into task-oriented dialogue systems,chit-chat-oriented dialogue systems,and question-and-answer dialogue systems accor-ding to their functions.The task-oriented dialogue system is a typical man-machine dialogue system,which aims to help users complete certain specific tasks,and has very important academic significance and application value.This paper systematically illustrates the general framework of task-oriented dialogue systems in practical engineering applications,including natural language understanding,dialogue management,and natural language generation.Then,the classical deep learning and machine learning methods used in the above parts are introduced.Finally,the task of natural language understanding is empirically verified and analyzed.This paper can provide effective guidance for the construction of a task-oriented dialogue system.

Question generation means that the machine actively asks a natural language question by given a passage.Neural question generation is trained in a completely end-to-end training mode,using neural networks to convert documents and answers to questions,which is an emerging and important research direction in natural language processing.This paper first gives a brief introduction to neural question generation,including basic concepts,mainstream frameworks,and evaluation methods.Then,it introduces the key issues of question generation,including input modeling,long document processing,multi-task learning,and the application of machine learning,other issues and improvements.Finally,it introduces the relationship between question generation and question answering,as well as future research of question generation.

Question answering system based on knowledge base aims to extract answers directly from the knowledge base by parsing users’ natural language question sentences.Currently,most knowledge based question answering models follow the two steps of entity detection and relationship recognition,but such methods ignore the structural information contained in the know-ledge base and the connection between the two tasks.In this paper,a joint question answering model based on knowledge representation is proposed.First,the knowledge representation model is used to map the entities and relationships in the knowledge base to a low-dimensional vector space,then the question sentences are embedded into the same vector space through neural network,and the entities in the question sentences are detected at the same time.The semantic similarity between knowledge base triples and question sentences is measured in the vector space,so that knowledge base embedding and multi-task learning are introduced into the task of knowledge based question answering.The experimental results show that the proposed model can greatly improve the training speed,and the accuracies of entity detection and relationship recognition task reach the mainstream level.It is proved that knowledge embedding and multi-task learning can improve the performance of knowledge based question answe-ring task.

The rapid increase in the number of judgment documents puts forward an urgent need for automated classification.However,there is a lack of method in existing studies that use judgment results as the subject of classification in the subdivision of civil cases,and therefore they cannot achieve accurate classification of judgment results in civil cases.In this paper,we apply deep learning technology in the field of classification of judgment results of civil cases,and obtain a model with better perfor-mance in this field through horizontal comparison of multiple deep learning models.This model is further optimized based on the data characteristics of the judgment document.After experiments,the Transformer model's macro precision rate,macro recall rate and macro F1 score in the judgment result classification are all higher than other models.By adjusting the data preprocessing process and adjusting the position embedding method of the Transformer model,the performance index of the model is increased by 1％～2%.

The fine-grained sentiment analysis is one of the key problems in the area of natural language processing.By learning contextual information of the text to conduct sentiment analysis on specific aspects,it can help users and businesses to better understand the sentiment information of specific aspects of users' comments.Aiming at the task of fine-grained sentiment analysis on users' comments,a text sentiment classification model combining BiGRU-attention and Gated Mechanisms is proposed.By integrating existing sentiment resources,HOWNET evaluation sentiment dictionary is used as the seed sentiment dictionary to expand the user comment sentiment dictionary through SO-PMI algorithm,the negative dictionary and part of speech information are combined to expand the user comment sentiment knowledge as the users' comment sentiment characteristic information.Introducing word,character and sentiment characteristics as the model of input infotmation,and using BiGRU to extract deep text features,then combined with gated mechanism as well as the attention mechanism,according to the acquired aspect word information to further extract the contextual sentiment characteristics related to aspect words,the final sentiment polarity is obtained by the softmax classfier.Experimental results show that the proposed model achieves better experimental results on the AIchallenger 2018 fine-grained sentiment analysis Chinese data sets,the Macro_F1_ score value reaches 0.7218,and the performance exceeds the baseline system.

Conversation model is one of the important directions of natural language processing.Today's dialogue models are mainly divided into retrieval-based methods and generation-based methods.However,the retrieval method cannot respond to questions that do not appear in the corpus,and the generation method is prone to problems with safe responses.In view of this,a compound conversation model that combines retrieval and generation is proposed,and the retrieval method and generation method are combined to make up for their shortcomings.First,K retrieval contexts and corresponding K retrieval candidate responses are obtained through the retrieval module.In the multi-response generation module,retrieval contexts are further combined to obtain several generation candidate responses.The candidate response ranking module is divided into two steps:pre-screening and post-reranking.The pre-screening part obtains the optimal retrieval response and the optimal generated response by calculating the similarity between the input question and candidate responses,and the post-reranking part further selects the most suitable answer to the input question.Experimental results show that the BLUE index increased by 6%,and the diversity index increased by 12%.

Multilingual question answering is one of the research hotspots in the field of natural language processing,which aims to enable the model to return a correct answer based on understanding of the given questions and texts in different languages.With the rapid development of machine translation technology and the wide application of multilingual pre-training technology in the field of natural language processing,multilingual question answering has also achieved a relatively rapid development.This paper first systematically reviews the current work of multilingual question answering methods,and divides them into feature-based methods,translation-based methods,pre-training-based methods and dual encoding-based methods,and introduces the use and characteristics of each method respectively.Meanwhile,it also discusses the current work related to multilingual question answe-ring tasks,and divides them into text-based and multi-modal-based tasks and gives the basic definition of each one.Moreover,this paper summarizes the dataset statistics,evaluation metrics and multilingual question answering methods involved in these tasks.Finally,it proposes the future research prospect of multilingual question answering.

Text sentiment analysis is a hot field in natural language processing.In recent years,Chinese text sentiment analysis methods have been widely investigated.Most of the recurrent neural network and convolutional neural network models based on word vectors have insufficient ability to extract and retain text features.In this paper,a Chinese sentiment polarity analysis model combining bi-directional GRU (BGRU) and multi-scale CNN is proposed.First,BGRU is utilized to extract text serialization features filtered with attention mechanism.Then the convolution neural network with distinct convolution kernels is applied to attention mechanism to adjust the dynamic weights.The text is acquired by the Softmax emotional polarity.Experiments indicates that our model outperforms the state-of-the-art methods on Chinese datasets.The accuracy of sentiment classification is 92.94% on the online_shopping_10_cats dataset of ChineseNLPcorpus,and 92.75% on the hotel review dataset compiled by Tan Songbo of Chinese Academy of Sciences,which is significantly improved compared with the current mainstream methods.

Aspect-based sentiment analysis (ABSA) is a popular topic for natural language processing,in which opinion target extraction and sentiment polarity classification of opinion target are one of the basic subtasks of ABSA.However,few studies directly extract the opinion targets of specific emotional polarity,especially the negative emotion opinion targets with more potential value.A new ABSA subtask--negative emotion opinion target extraction (NE-OTE) is proposed,and a BiLSTM-CRF model based on attention mechanism and character and word mixture embedding (AB-CE) is proposed.On the basis of bi-directional long short-term memory (BiLSTM) learning textual semantic information and capturing long distance bi-directional semantic dependency,through the attention mechanism,the model can better pay attention to the key parts in the input sequence and capture the implied characteristics related to the opinion target and its emotional tendency.Finally,the CRF layer can be used to predict the optimal tag sequence at the sentence level,so as to extract the negative emotional opinion target.This paper builds three NE-OTE task datasets based on the mainstream ABSA task baseline datasets and conducts extensive experiments on these datasets.Experimental results show that the model proposed in this paper can effectively identify the target of negative emotional opinions,and is significantly better than other baseline models,which verifies the effectiveness of the method proposed in this paper.

Automatic question generation is a research hotspot in the field of natural language processing,which aims to generate natural questions from texts.With the continuous development of internet,a large amount of commodity reviews has been generated in the electronic commerce fields.In the face of massive review information,how to quickly mine key reviews related to pro-duct information has great research value.It is of great importance to both customers and merchants.Most of existing question generation models are based on reading comprehension type corpus and use sequence-to-sequence network to generate questions.However,for question generation tasks based on product reviews,existing models fail to incorporate the product information that users and businesses focus on into the learning process.In order to make the generated questions more in line with the attributes of the goods,a question generation model based on product is proposed in this paper.Through joint learning and training with product attribute recognition,the model strengthens the attention to feature information related to product.Compared with the existing question generation models,this model can not only strengthen the recognition ability of product attributes,but also ge-nerate contents more accurately.This paper carries out experiments on the data sets of product reviews of JD and Amazon.The results show that in the question generation task based on reviews,this model achieves a great improvement compared with the existing question generation model,which is improved by 3.26% and 2.01% respectively on BLEU,and 2.33% and 2.10% respectively on ROUGE.

With the wide application of graph neural network technology in the field of natural language processing,the research of text classification based on graph neural networks has received more and more attention.Building graph for text is an important research task in the application of graph neural networks for text classification.Existing methods cannot effectively capture the dependency of long-distance words in sentences when building graph.Short text classification is a special type of text classification task in which the classified text is generally short,so the traditional text representation is usually sparse and lacks rich semantic information.Based on this,in this paper we propose a short text classification method based on graph convolutional neural networks incorporating long-distance words dependency.Firstly,by using the co-occurrence relationship of words,the containment relationship between documents and words,and the long-distance words dependency in sentences,a text graph is constructed for the entire text corpus.Then,the text graph is input into the graph convolutional neural networks,and the category label prediction is made for each document node after 2-layer convolution.The experimental results on the three datasets of online_shopping_10_cats,summaries of Chinese papers and hotel reviews show that the proposed method achieves better results than the existing baselines.

Text classification is a basic and important task in natural language processing.Most of the text classification methods based on deep learning only focus on a single model structure.The single structure lacks the ability to simultaneously capture and utilize both global and local semantic features.Besides,the deepening of the network will lose more semantic information.In order to overcome the above problems,a text classification model FMNN which is a text classification model fused with multiple neural network is proposed in this paper.The model combines the performances of BERT,RNN,CNN and Attention while minimizing the network depth.BERT is used as the embedding layer to obtain the matrix representation of the text.BiLSTM and Attention are used to jointly extract the global semantic features of the text.CNN is used to extract the local semantic features of the text at multiple granularities.The global semantic features and local semantic features are applied to the softmax classifier respectively.The results are finally fused by arithmetic average.The experimental results on three public data sets and one judicial data set show that the proposed FMNN model achieves higher accuracy rate,and the accuracy rate on the judicial data set reaches 90.31%,which proves that the model has good practical value.