Abstract: Aiming at the problem that traditional retrieval model on pest has a single mode,this paper uses a cross-modal retrieval method for 17 kinds of common lycium pests in image and text modal,which integrates attention mechanism and visual semantic reasoning.First,use Faster R-CNN+ResNet101 to realize the attention mechanism to extract local fine-grained information of wolfberry pest images.Then further introduce vision semantic reasoning to build the image region connections and use convolutional network GCN for region relation reasoning to enhance area representation.In addition,global semantic reasoning is performed by enhancing semantic correlation between regions,selecting discriminant features and filtering out unimportant information to capture more key semantic information.Finally,the semantic association between different modalities of lycium barbarum pest image and text is deeply explored through modal interaction.On the self-built lycium barbarum pest dataset,the average accuracy(MAP) is used as the evaluation index to carry out comparative experiment and ablation experiment.Experimental results demonstrate that the averaged MAP of the proposed method in the self-built lycium pest dataset achieves 0.522,compared with the eight mainstream methods,the average MAP of the method improves by 0.048 to 0.244,and it has better retrieval effect.

Key words: Cross-modal retrieval, Attention mechanism, Fine-grained, Visual semantic reasoning, Lycium barbarum pest