Abstract: Multimodal Name Entity Recognition(MNER) and Multimodal Relation Extraction(MRE) are two key technologies in the construction of multimodal knowledge graphs.However,the existing MNER and MRE methods still have certain limitations in feature extraction and fusion of high-dimensional data.To address these issues,this paper proposes a multimodal entity relation joint extraction method based on quantum Transformer.Firstly,a parameterized quantum circuit for text data processing is design,which utilizes the superposition and entanglement characteristics in quantum mechanics,and combines with the Transformer model to extract deep features from text;Secondly,the pyramid visual feature extraction model is designed to obtain hierarchical features from high to low,which fully considers the multi-scale information of the image.Finally,by designing a hierarchical visual prefix network,the hierarchical multi-scale image features are aligned and fused with the text features to obtain a highly robust text representation.This study provides a new research approach for multimodal entity relation joint extraction.Experimental results on three public benchmark datasets show that the multimodal entity relation extraction method based on quantum Transformer proposed in this paper is effective and stable.

Key words: MNER, MRE, Pyramid visual feature, Transformer, Feature fusion