Computer Science ›› 2026, Vol. 53 ›› Issue (10): 164-173.doi: 10.11896/jsjkx.260100047

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Efficient Human-Object Interaction Detection Method for Long-range Dependency Modeling

LIN Shengjie, BAI Lin, LIU Ri, JIANG Xiaoyu   

  1. School of Computer Electronics and Information,Guangxi University,Nanning 530000, China
  • Received:2026-01-12 Revised:2026-04-25 Published:2026-10-13
  • About author:LIN Shengjie,born in 1998,postgra-duate.His main research interest is re-cognizing human-object interaction actions.
    BAI Lin,born in 1985,Ph.D,associate professor,master’s supervisor,is a member of CCF(No.A6951M).His main research interests include deep learning and computer vision.

Abstract: Human-object interaction(HOI) detection aims to simultaneously identify humans,interacting objects,and their corresponding interaction relationships in images,and constitutes a fundamental task in behavior understanding and high-level visual semantic analysis.To address the challenges of effectively modeling long-range,cross-region human-object interactions in complex scenes,as well as the high computational cost of existing methods,an efficient HOI detection framework is proposed,which reduces model complexity while maintaining high detection accuracy.The framework introduces a Mamba Encoder to model image feature sequences,enabling efficient integration of global contextual information and thereby enhancing the capability to capture long-range human-object interaction relationships.On this basis,a keypoint selection mechanism is designed to guide the model to focus on discriminative regions that are highly relevant to interaction semantics.Combined with deformable attention,the selected key features are further refined to improve interaction recognition performance under complex scene conditions.In addition,a human-object interaction pair representation that explicitly incorporates spatial relationships is constructed.By introducing spatial relation encoding and performing joint feature fusion,the robustness and discriminability of interaction semantic representations are further enhanced.Experimental results on the HICO-DET and V-COCO benchmark datasets demonstrate that the proposed approach achieves a favorable balance between detection accuracy and computational efficiency.Compared with existing methods,it exhibits clear performance advantages across multiple evaluation metrics while maintaining lower parameter counts and computational complexity.Both qualitative and quantitative analyses further validate the effectiveness of the proposed approach in mo-deling long-range dependencies,adapting to complex scenes,and enabling efficient inference.

Key words: Human-object interaction detection, Long-range dependency modeling, Mamba Encoder, Deformable attention, Spatial relationship modeling

CLC Number: 

  • TP391
[1] XIE C,ZENG F,HU Y,et al.Category query learning for human-object interaction classification [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:15275-15284.
[2] YANG J,LI B,YANG F,et al.Boosting human-object interaction detection with text-to-image diffusion model [J].arXiv:2305.12252,2023.
[3] LIM J Y,BASKARAN V M,LIM J M Y,et al.ERNet:An efficient and reliable human-object interaction detection network [J].IEEE Transactions on Image Processing,2023,32:964-979.
[4] ZHANG F Z,YUAN Y,CAMPBELL D,et al.Exploring predicate visual context in detecting of human-object interactions [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:10411-10421.
[5] GU A,DAO T.Mamba:Linear-time sequence modeling with selective state spaces [C]//Proceedings of the First Conference on Language Modeling.2024.
[6] VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need [C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[7] BANSAL A,RAMBHATLA S S,SHRIVASTAVA A,et al.Detecting human-object interactions via functional generalization [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:10460-10469.
[8] GKIOXARI G,GIRSHICK R,DOLLÁR P,et al.Detecting and recognizing human-object interactions [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8359-8367.
[9] GU A.Modeling sequences with structured state spaces [M].California:Stanford University,2023.
[10] BENGIO Y,DUCHARME R,VINCENT P,et al.A neuralprobabilistic language model [J].Journal of Machine Learning Research,2003,3:1137-1155.
[11] BOX G E P,JENKINS G M,REINSEL G C,et al.Time series analysis:Forecasting and control [M].Hoboken:John Wiley & Sons,2015.
[12] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks [C]//Advances in Neural Information Processing Systems.2012:1097-1105.
[13] ZHU L,LIAO B,ZHANG Q,et al.Vision Mamba:Efficient vi-sual representation learning with bidirectional state space model [J].arXiv:2401.09417,2024.
[14] BUI T M,BOUGOURZI F,DORNAIKA F,et al.MambaCA-FU:Hybrid multi-scale and multi-attention model with Mamba-based fusion for medical image segmentation [J].arXiv:2510.03786,2025.
[15] MUNIR M,ZHANG A,MARCOLESCU R.VCMamba:Brid-ging convolutions with multi-directional Mamba for efficient vi-sual representation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2025:3037-3046.
[16] WANG Z,LI C,XU H,et al.Mamba YOLO:A simple baseline for object detection with state space model [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2025:8205-8213.
[17] LECUN Y,BENGIO Y,HINTON G.Deep learning [J].Na-ture,2015,521(7553):436-444.
[18] PASZKE A,GROSS S,MASSA F,et al.PyTorch:An imperative style,high-performance deep learning library [C]//Ad-vances in Neural Information Processing Systems.2019:8024-8035.
[19] HUBER P J.Robust estimation of a location parameter[M]//Breakthroughs in Statistics:Methodology and Distribution.New York:Springer,1992:492-518.
[20] REZATOFIGHI H,TSOI N,GWAK J Y,et al.Generalized intersection over union:A metric and a loss for bounding box regression [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:658-666.
[21] LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection [C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988.
[22] LOSHCHILOV I,HUTTER F.Decoupled weight decay regularization [J].arXiv:1711.05101,2017.
[23] TAMURA M,OHASHI H,YOSHINAGA T.QPIC:Query-based pairwise human-object interaction detection with image-wide contextual information [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10410-10419.
[24] ZHANG F Z,CAMPBELL D,GOULD S.Efficient two-stagedetection of human-object interactions with a novel unary-pairwise transformer [C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2022:20104-20112.
[25] PARK J,PARK J W,LEE J S.ViPLO:Vision transformerbased pose-conditioned self-loop graph for human-object interaction detection [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:17152-17162.
[26] ZHANG A,LIAO Y,LIU S,et al.Mining the benefits of two-stage and one-stage HOI detection [C]//Advances in Neural Information Processing Systems.2021:17209-17220.
[27] ZHANG F Z,CAMPBELL D,GOULD S.Spatially conditioned graphs for detecting human-object interactions [C]//Procee-dings of the IEEE/CVF International Conference on Computer Vision.2021:13319-13327.
[28] CHENG Y,DUAN H,WANG C,et al.Parallel disentanglingnetwork for human-object interaction detection [J].Pattern Recognition,2024,146:110021.
[29] KIM S,JUNG D,CHO M.Relational context learning for human-object interaction detection [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:2925-2934.
[30] SU Z,YANG H.A novel part refinement tandem transformer for human-object interaction detection [J].Sensors,2024,24(13):4278.
[31] YAMADA M,DHARAMSHI N,KOHLI A,et al.Unleashing potentials of vision-language models for zero-shot HOI detection [C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2025:5751-5760.
[32] ZHU M,HO E S L,CHEN S,et al.Geometric features en-hanced human-object interaction detection [J].IEEE Transactions on Instrumentation and Measurement,2024,73:1-14.
[1] ZHAO Wenhao, MEI Meng, WANG Xiaoping, LUO Hangyu. PKHOI:Enhancing Human-Object Interaction Detection Algorithms with Prior Knowledge [J]. Computer Science, 2026, 53(1): 141-152.
[2] DING Yuanbo, BAI Lin, LI Taoshen. Human-Object Interaction Detection Based on Fine-grained Attention Mechanism [J]. Computer Science, 2025, 52(11): 141-149.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!