Abstract: HOI detection plays a crucial role in visual scene understanding.With the advancement of deep learning technologies,vision-based interaction detection models have achieved promising performance.However,most existing methods lack the utilization of prior logical knowledge,sometimes leading to unreasonable predictions.Additionally,while some methods employ spatial information and human pose information for reasoning,they only construct losses between inference results and annotations,preventing decoders from learning accurate implicit relationships.Therefore,this paper proposes the PKHOI method,which enhances existing HOI detection algorithms by leveraging prior knowledge,effectively improving the accuracy of current HOI detection algorithms.Specifically,it constructs a logical rule table from the training set,encompassing object functionality,spatial relationships,human poses,and verb co-occurrence.These rules are transformed into first-order logic and mapped to continuous space.The prior logical rules are then incorporated into neural networks through loss functions during training and matrix multiplication during inference,enhancing model accuracy.Furthermore,this paper proposes a method to generate human-object pair queries by fusing multimodal information(spatial,semantic,and human pose information).Combined with logical loss functions,this approach guides the decoder to learn more implicit knowledge.The proposed method enhances two mainstream HOI detection algorithms,UPT and PViC,and evaluates them on V-COCO,HICO-DET,and Flickr30k datasets.Experimental results demonstrate that the proposed method can effectively improve the performance of existing approaches.

Key words: Human-object interaction detection, Prior knowledge, First-order logic, Pose information, Multi-modal information fusion