计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (10): 164-173.doi: 10.11896/jsjkx.260100047

• 计算机图形学 & 多媒体 • 上一篇    下一篇

面向长程依赖建模的高效人-物交互检测方法

林胜杰, 白琳, 刘日, 蒋晓昱   

  1. 广西大学计算机与电子信息学院 南宁 530000
  • 收稿日期:2026-01-12 修回日期:2026-04-25 发布日期:2026-10-13
  • 通讯作者: 白琳(bailin@gxu.edu.cn)
  • 作者简介:(2313301032@gxu.edu.cn)

Efficient Human-Object Interaction Detection Method for Long-range Dependency Modeling

LIN Shengjie, BAI Lin, LIU Ri, JIANG Xiaoyu   

  1. School of Computer Electronics and Information,Guangxi University,Nanning 530000, China
  • Received:2026-01-12 Revised:2026-04-25 Online:2026-10-13
  • About author:LIN Shengjie,born in 1998,postgra-duate.His main research interest is re-cognizing human-object interaction actions.
    BAI Lin,born in 1985,Ph.D,associate professor,master’s supervisor,is a member of CCF(No.A6951M).His main research interests include deep learning and computer vision.

PDF (PC)

83

摘要： 人-物交互检测旨在同时识别图像中人与物、交互物体及其对应的交互关系,是行为理解与高层视觉语义分析中的关键任务。针对复杂场景下普遍存在的远距离、跨区域人-物交互关系难以有效建模,以及现有方法计算开销较大的问题,提出了一种高效的人-物交互检测框架,在保证检测精度的同时降低了模型复杂度。该框架引入Mamba Encoder对图像特征序列进行建模,以高效整合全局上下文信息,从而增强对长程人-物交互关系的建模能力。在此基础上,设计关键点选择机制,引导模型聚焦于与交互语义高度相关的判别性区域,并结合可变形注意力对关键特征进行精细建模,以提升复杂场景条件下的交互识别性能。此外,构建了一种融合空间关系的人-物交互对表示方式,通过显式引入空间关系编码并进行联合特征融合,进一步增强交互语义表征的鲁棒性与判别性。在HICO-DET和V-COCO等主流HOI检测数据集上的实验结果表明,该方法在检测精度与计算效率之间取得了良好的平衡,相较于现有方法在多个评价指标上表现出明显优势,同时在参数量和计算量方面保持了更优的复杂度水平。定性与定量分析进一步验证了该方法在长程依赖建模、复杂场景适应性以及高效推理方面的有效性。

关键词: 人-物交互检测, 长程依赖建模, Mamba 编码器, 可变形注意力, 空间关系建模

Abstract: Human-object interaction(HOI) detection aims to simultaneously identify humans,interacting objects,and their corresponding interaction relationships in images,and constitutes a fundamental task in behavior understanding and high-level visual semantic analysis.To address the challenges of effectively modeling long-range,cross-region human-object interactions in complex scenes,as well as the high computational cost of existing methods,an efficient HOI detection framework is proposed,which reduces model complexity while maintaining high detection accuracy.The framework introduces a Mamba Encoder to model image feature sequences,enabling efficient integration of global contextual information and thereby enhancing the capability to capture long-range human-object interaction relationships.On this basis,a keypoint selection mechanism is designed to guide the model to focus on discriminative regions that are highly relevant to interaction semantics.Combined with deformable attention,the selected key features are further refined to improve interaction recognition performance under complex scene conditions.In addition,a human-object interaction pair representation that explicitly incorporates spatial relationships is constructed.By introducing spatial relation encoding and performing joint feature fusion,the robustness and discriminability of interaction semantic representations are further enhanced.Experimental results on the HICO-DET and V-COCO benchmark datasets demonstrate that the proposed approach achieves a favorable balance between detection accuracy and computational efficiency.Compared with existing methods,it exhibits clear performance advantages across multiple evaluation metrics while maintaining lower parameter counts and computational complexity.Both qualitative and quantitative analyses further validate the effectiveness of the proposed approach in mo-deling long-range dependencies,adapting to complex scenes,and enabling efficient inference.

Key words: Human-object interaction detection, Long-range dependency modeling, Mamba Encoder, Deformable attention, Spatial relationship modeling

中图分类号: 

  • TP391
[1] XIE C,ZENG F,HU Y,et al.Category query learning for human-object interaction classification [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:15275-15284.
[2] YANG J,LI B,YANG F,et al.Boosting human-object interaction detection with text-to-image diffusion model [J].arXiv:2305.12252,2023.
[3] LIM J Y,BASKARAN V M,LIM J M Y,et al.ERNet:An efficient and reliable human-object interaction detection network [J].IEEE Transactions on Image Processing,2023,32:964-979.
[4] ZHANG F Z,YUAN Y,CAMPBELL D,et al.Exploring predicate visual context in detecting of human-object interactions [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:10411-10421.
[5] GU A,DAO T.Mamba:Linear-time sequence modeling with selective state spaces [C]//Proceedings of the First Conference on Language Modeling.2024.
[6] VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need [C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[7] BANSAL A,RAMBHATLA S S,SHRIVASTAVA A,et al.Detecting human-object interactions via functional generalization [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:10460-10469.
[8] GKIOXARI G,GIRSHICK R,DOLLÁR P,et al.Detecting and recognizing human-object interactions [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8359-8367.
[9] GU A.Modeling sequences with structured state spaces [M].California:Stanford University,2023.
[10] BENGIO Y,DUCHARME R,VINCENT P,et al.A neuralprobabilistic language model [J].Journal of Machine Learning Research,2003,3:1137-1155.
[11] BOX G E P,JENKINS G M,REINSEL G C,et al.Time series analysis:Forecasting and control [M].Hoboken:John Wiley & Sons,2015.
[12] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks [C]//Advances in Neural Information Processing Systems.2012:1097-1105.
[13] ZHU L,LIAO B,ZHANG Q,et al.Vision Mamba:Efficient vi-sual representation learning with bidirectional state space model [J].arXiv:2401.09417,2024.
[14] BUI T M,BOUGOURZI F,DORNAIKA F,et al.MambaCA-FU:Hybrid multi-scale and multi-attention model with Mamba-based fusion for medical image segmentation [J].arXiv:2510.03786,2025.
[15] MUNIR M,ZHANG A,MARCOLESCU R.VCMamba:Brid-ging convolutions with multi-directional Mamba for efficient vi-sual representation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2025:3037-3046.
[16] WANG Z,LI C,XU H,et al.Mamba YOLO:A simple baseline for object detection with state space model [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2025:8205-8213.
[17] LECUN Y,BENGIO Y,HINTON G.Deep learning [J].Na-ture,2015,521(7553):436-444.
[18] PASZKE A,GROSS S,MASSA F,et al.PyTorch:An imperative style,high-performance deep learning library [C]//Ad-vances in Neural Information Processing Systems.2019:8024-8035.
[19] HUBER P J.Robust estimation of a location parameter[M]//Breakthroughs in Statistics:Methodology and Distribution.New York:Springer,1992:492-518.
[20] REZATOFIGHI H,TSOI N,GWAK J Y,et al.Generalized intersection over union:A metric and a loss for bounding box regression [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:658-666.
[21] LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection [C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988.
[22] LOSHCHILOV I,HUTTER F.Decoupled weight decay regularization [J].arXiv:1711.05101,2017.
[23] TAMURA M,OHASHI H,YOSHINAGA T.QPIC:Query-based pairwise human-object interaction detection with image-wide contextual information [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10410-10419.
[24] ZHANG F Z,CAMPBELL D,GOULD S.Efficient two-stagedetection of human-object interactions with a novel unary-pairwise transformer [C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2022:20104-20112.
[25] PARK J,PARK J W,LEE J S.ViPLO:Vision transformerbased pose-conditioned self-loop graph for human-object interaction detection [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:17152-17162.
[26] ZHANG A,LIAO Y,LIU S,et al.Mining the benefits of two-stage and one-stage HOI detection [C]//Advances in Neural Information Processing Systems.2021:17209-17220.
[27] ZHANG F Z,CAMPBELL D,GOULD S.Spatially conditioned graphs for detecting human-object interactions [C]//Procee-dings of the IEEE/CVF International Conference on Computer Vision.2021:13319-13327.
[28] CHENG Y,DUAN H,WANG C,et al.Parallel disentanglingnetwork for human-object interaction detection [J].Pattern Recognition,2024,146:110021.
[29] KIM S,JUNG D,CHO M.Relational context learning for human-object interaction detection [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:2925-2934.
[30] SU Z,YANG H.A novel part refinement tandem transformer for human-object interaction detection [J].Sensors,2024,24(13):4278.
[31] YAMADA M,DHARAMSHI N,KOHLI A,et al.Unleashing potentials of vision-language models for zero-shot HOI detection [C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2025:5751-5760.
[32] ZHU M,HO E S L,CHEN S,et al.Geometric features en-hanced human-object interaction detection [J].IEEE Transactions on Instrumentation and Measurement,2024,73:1-14.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发