关键词: 长短期记忆网络, 词嵌入, 卷积神经网络, 视觉问答, 自然语言处理

Abstract: At present,there are various researches on the modeling of Visual Question Answering (VQA) tasks,but existing VQA models have a common drawback,i.e. training and reasoning are time-consuming.Research shows that the text processing part of the VQA model is mainly based on LSTM (Long Short Term Memory) networks,and the overall performance of the VQA model is also limited by the LSTM network used for the text processing.Due to the recurrent nature of the LSTM network,the complex data streams in the LSTM network can hardly take advantages of GPU parallel computing to accelerate.Aiming at the above problems,and for the purpose of optimizing the training speed of the model,a new model named SCMP (Simple Conv1d MaxPool1d) is proposed in this paper to replace the LSTM network to deal with incoming natural language questions.The experimental results on the VQA2.0 dataset show that the training speed of the model is 10 times faster than the existing model,and there is no loss for the accuracy of the VQA model.In addition,this paper proposes a novel method for data augmentation of question datasets in VQA2.0 datasets.Experimental results show that data augmentation can improve model prediction performance and accelerate model convergence.The model trained with enhanced data (SCMP) obtains an evaluation score of 63.46% on the validation set,which is better than the existing VQA model.

Key words: CNN, LSTM, Natural language processing, Visual question answering, Word embedding