计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (2): 236-244.doi: 10.11896/jsjkx.250300103

• 计算机图形学&多媒体 • 上一篇    下一篇

双支特征融合的带约束的多损失视频异常检测

韩磊1, 商浩宇1, 钱小燕2, 顾妍2, 刘青松2, 王闯1   

  1. 1 南京邮电大学计算机学院、软件学院、网络空间安全学院 南京 210023
    2 南京航空航天大学民航学院 南京 211106
  • 收稿日期:2025-03-20 修回日期:2025-05-27 发布日期:2026-02-10
  • 通讯作者: 韩磊(hanlei@njupt.edu.cn)
  • 基金资助:
    国家重点研发计划项目重点专项(2024YFB2906704);国家自然科学基金项目(62202237,62372248,U2033207);江苏省自然科学基金重点项目(24KJA520006)

Constrained Multi-loss Video Anomaly Detection with Dual-branch Feature Fusion

HAN Lei1, SHANG Haoyu1, QIAN Xiaoyan2, GU Yan2, LIU Qingsong2, WANG Chuang1   

  1. 1 School of Computer Science,School of Software,School of Cyberspace Security,Nanjing University of Posts and Telecommunications, Nanjing 210023,China
    2 College of Civil Aviation,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China
  • Received:2025-03-20 Revised:2025-05-27 Online:2026-02-10
  • About author:HAN Lei,born in 1979,Ph.D,senior engineer.His main research interests include computer vision and data center networking.
  • Supported by:
    National Key Research and Development Program of China(2024YFB2906704),National Natural Science Foundation of China(62202237,62372248,U2033207) and Key Program of Natural Science Foundation of Jiangsu,China(24KJA520006).

PDF (PC)

81

摘要： 针对视频异常事件的时空相关性学习对检测性能存在重要影响的问题,提出了基于融合双支特征的带约束损失的视频异常检测方法(Dual-branch Feature Fusion Based Constrained Multi-loss Video Anomaly Detection,DBF-CML-transMIL)。该方法考虑多示例学习中片段的显著性和相关性,利用多层线性神经网络学习各片段的空间显著性特征,并设计级联Transformer融合模块来学习示例间的多层时序相关性;然后利用多损失模型对融合特征进行多loss监督学习,以丰富预测的多样性;针对现有top-k的离散性问题,提出了带约束机制的滑窗top-k强化异常事件的相关性。在ShanghaiTech和UCF-Crime数据集上的对比实验与消融实验表明,DBF-CML-transMIL的异常检测曲线下面积(Area Under Curve,AUC)分别达到97.33%和83.82%;各模块都能有效提升视频异常事件检测的性能。

关键词: 视频异常检测, 多示例学习, 级联注意力机制, 多损失函数, 滑窗top-k

Abstract: To address the significant impact of spatiotemporal correlation learning on video anomaly detection performance,this paper proposes a dual-branch feature fusion-based constrained multi-loss video anomaly detection method(DBF-CML-transMIL).This method considers the saliency and correlation of segments in multiple instance learning(MIL),utilizing a multi-layer linear neural network to learn the spatial saliency features of each segment.A cascaded Transformer fusion module is designed to capture multi-level temporal correlations among instances.Then,a multi-loss model is employed to perform supervised learning on the fused features,enriching prediction diversity.To address the discreteness issue of the existing top-k method,a constrained sliding window top-k mechanism is introduced to enhance the correlation of anomalous events.Comparative and ablation experiments conducted on the ShanghaiTech and UCF-Crime datasets demonstrate that DBF-CML-transMIL achieves AUC scores of 97.33% and 83.82%,respectively.Furthermore,each module effectively enhances the performance of video anomaly detection.

Key words: Video anomaly detection, Multiple instance learning, Cascaded attention mechanism, Multi-loss function, Sliding window top-k

中图分类号: 

  • TP183
[1]CHEN J,LI L,SU L,et al.Prompt-enhanced multiple instance learning for weakly supervised video anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:18319-18329.
[2]WU P,ZHOU X,PANG G,et al.Weakly supervised videoanomaly detection and localization with spatio-temporal prompts[C]//Proceedings of the 32nd ACM International Conference on Multimedia.2024:9301-9310.
[3]SULTANI W,CHEN C,SHAH M.Real-world anomaly detection in surveillance videos[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:6479-6488.
[4]WAN B,FANG Y,XIA X,et al.Weakly supervised video anomaly detection via center-guided discriminative learning[C]//2020 IEEE International Conference on Multimedia and Expo(ICME).IEEE,2020:1-6.
[5]LI S,LIU F,JIAO L.Self-training multi-sequence learning with transformer for weakly supervised video anomaly detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:1395-1403.
[6]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[7]ZHOU H,ZHAN Y Z,MAO Q R.Video Anomaly Detection Based on Space-Time Fusion Graph Network Learning[J].Journal of Computer Research and Development,2021,58(1):48-59.
[8]TIAN Y,PANG G,CHEN Y,et al.Weakly-supervised videoanomaly detection with robust temporal feature magnitude learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:4975-4986.
[9]SHAO Z,BIAN H,CHEN Y,et al.Transmil:Transformerbased correlated multiple instance learning for whole slide image classification[J].Advances in Neural Information Processing Systems,2021,34:2136-2147.
[10]ZHU X R,QIAN X Y,SHI Y Z,et al.Video anomaly detection with long-and-short term time series correlations[J].Journal of Image and Graphics,2024,29(7):1998-2010.
[11]SHI RN,HE Q,WANG H Y,et al.Dynamic motion constraints for video anomaly detection[J].Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition),2025,37(1):110-120.
[12]WU P,LIU J,SHI Y,et al.Not only look,but also listen:Learning multimodal violence detection under weak supervision[C]//Computer Vision-ECCV 2020:16th European Conference.Springer International Publishing,2020:322-339.
[13]HUSSAIN A,ULLAH W,KHAN N,et al.TDS-Net:Trans-former enhanced dual-stream network for video Anomaly Detection[J].Expert Systems with Applications,2024,256:124846.
[14]MA H,ZHANG L.Attention-based framework for weakly supervised video anomaly detection[J].The Journal of Supercomputing,2022,78(6):8409-8429.
[15]ZHONG J X,LI N,KONG W,et al.Graph convolutional label noise cleaner:Train a plug-and-play action classifier for anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:1237-1246.
[16]FENG J C,HONG F T,ZHENG W S.Mist:Multiple instance self-training framework for video anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14009-14018.
[17]LI G,CAI G,ZENG X,et al.Scale-aware spatio-temporal relation learning for video anomaly detection[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:333-350.
[18]CHEN H,MEI X,MA Z,et al.Spatial-temporal graph attention network for video anomaly detection[J].Image and Vision Computing,2023,131:104629.
[19]ZHANG J,QING L,MIAO J.Temporal convolutional network with complementary inner bag loss for weakly supervised anomaly detection[C]//2019 IEEE International Conference on Image Processing(ICIP).IEEE,2019:4030-4034.
[20]GONG Y,WANG C,DAI X,et al.Multi-scale continuity-aware refinement network for weakly supervised video anomaly detection[C]//2022 IEEE International Conference on Multimedia and Expo(ICME).IEEE,2022:1-6.
[21]ZHEN Y,GUO Y,WEI J,et al.Multi-scale background sup-pression anomaly detection in surveillance videos[C]//2021 IEEE International Conference on Image Processing(ICIP).IEEE,2021:1114-1118.
[22]ULLAH W,ULLAH A,HAQ I U,et al.CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks[J].Multimedia Tools and Applications,2021,80:16979-16995.
[23]UL AMIN S,KIM Y,SAMI I,et al.An Efficient Attention-Based Strategy for Anomaly Detection in Surveillance Video[J].Computer Systems Science & Engineering,2023,46(3):3939-3958.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发