Abstract: Currently,methods for identifying public nuisance websites,suffer from insufficient feature utilization and poor feature integration.Therefore,this paper proposes a multi-modal fusion model for identifying public nuisance websites,named RBI-RA.This model uses the ResNet50+Attention model to extract visual features from website screenshots,while utilizing OCR techno-logy to extract text from screenshots to enrich the website’s text features subsequently.The model employs the RoBERTa+Bi-LSTM+interactive attention mechanism model to extract features from HTML text and screenshot text separately,and integrates them through an interactive attention mechanism to enrich and expand the website text features.The model uses a self-attention mechanism to merge the website’s visual and text features,resulting in a multi-modal fusion classifier that leverages the complementary features across different modalities.Finally,to prove the effectiveness of the proposed model,experiments are conducted on a self-developed dataset.Experimental results show that the proposed model based on multi-modal data fusion effectively improves the performance of identifying public nuisance websites,with good precision,recall,and F1 scores.

Key words: Public nuisance website identification, Model fusion, Deep learning, Attention mechanism