计算机科学 ›› 2022, Vol. 49 ›› Issue (8): 172-177.doi: 10.11896/jsjkx.210600061

• 计算机图形学& 多媒体 • 上一篇    下一篇

基于非局部注意力生成对抗网络的视频异常事件检测方法

孙奇, 吉根林, 张杰   

  1. 南京师范大学计算机与电子信息学院/人工智能学院 南京 210023
  • 收稿日期:2021-06-04 修回日期:2021-09-11 发布日期:2022-08-02
  • 通讯作者: 吉根林(glji@njnu.edu.cn)
  • 作者简介:(192202001@njnu.edu.cn)
  • 基金资助:
    国家自然科学基金(41971343)

Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection

SUN Qi, JI Gen-lin, ZHANG Jie   

  1. School of Computer and Electronic Information/School of Artificial Intelligence,Nanjing Normal University,Nanjing,210023,China
  • Received:2021-06-04 Revised:2021-09-11 Published:2022-08-02
  • About author:SUN Qi,born in 1995,master.Her main research interests include big data ana-lysis and mining technology.
    JI Gen-lin,born in 1964,Ph.D,professor.His main research interests include big data analysis and mining techno-logy.
  • Supported by:
    National Natural Science Foundation of China(41971343).

摘要: 针对异常事件的不确定性,文中选择使用未来帧预测的方式对视频进行异常事件检测。通过正常样本对预测模型进行训练,使模型能够准确预测不包含异常事件的未来帧,但对于包含未知事件的视频帧,模型无法进行预测,利用生成对抗网络以及表观约束和运动约束对用于预测的生成器模型进行训练。为了减少相关目标特征丢失,提出了非局部注意力U型网络生成器(Nonlocal Attention Unet Generator,NA-UnetG)模型,提升了生成器的预测精度,同时提升了视频异常事件检测的准确度。通过公开数据集CUHK Avenue和UCSD Ped2对所提方法进行实验验证,实验结果表明,所提方法的AUC指标优于其他方法,AUC分别达到了83.4%和96.3%。

关键词: 非局部注意力机制, 深度学习, 生成对抗网络, 视频异常事件检测, 视频预测

Abstract: As the uncertainty of abnormal events,the method of future frame prediction is chosen to detect abnormal events in video.The prediction model is trained with normal samples,so that the model can accurately predict the future frames without abnormal events.However,it cannot predict video frames with unknown events.Combining with apparent constraints and motion constraints,generative adversarial network is used to train the generator model for prediction.In order to reduce the loss of relative target features,a nonlocal attention Unet generator (NA-UnetG) model is proposed to improve the prediction accuracy of generator and the accuracy of abnormal video event detection.Experiments on datasets CUHK Avenue and UCSD Ped2 validate the effectiveness of the proposed method.The results show that the AUC of the proposed method is better than that of other methods,reaches 83.4% and 96.3%,respectively.

Key words: Deep learning, Generative adversarial network, Non-local attention mechanism, Video anomaly event detection, Video prediction

中图分类号: 

  • TP391
[1]WANG Z G,ZHANG Y J.Anomaly detection in surveillancevideos:A survey [J].Journal of Tsinghua University(Science and Technology),2020,60(6):518-529.
[2]COLQUE R V H M,CAETANO C,DE ANDRADE M T L. Histograms of Optical Flow Orientation and Magnitude to Detect Anomalous Events in Videos[C]//The 28th SIBGRAPI Conference on Graphics,Patterns and Images.IEEE Computer Society.2015:673-682.
[3]CHANDOLA V,BANERJEE A,KUMAR V.Anomaly detec-tion:A survey [J].ACM Computer,2009,41(3):1-15.
[4]MAHADEVAN V,LI W X.Anomaly detection in crowdedscenes[C]//The twenty-third IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2010:1975-1981.
[5]WANG X L,GIRSHICK R B,GUPTA A,et al.Non-Local neural networks[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2018:7794-7803.
[6]DALA N,TRIGGS B.Histograms of oriented gradients for human detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE Computer Socie-ty,2005:886-893.
[7]DALA N,TRIGGS B.Human detection using oriented histo-grams of flow and appearance[C]//The 9th European Confe-rence on Computer Vision.Lecture Notes In Computer Science,2006:428-441.
[8]ZHANG J Y,XIE W X,ZHI K L.Abnormal crowd behavior detection based on motion effect map features of moving foregrounds [J].Journal of Signal Processing,2018,34(3):296-304.
[9]ADAM A,RIVLIN E,SHIMSHONI I,et al.Robust real-time unusual event detection using multiple fixed-location monitors [J].IEEE Trans.Pattern Anal,2008,30(3):555-560.
[10]KIM J,GRAUMAN K.Observe locally,infer globally:A space-time MRF for detecting abnormal activities with incremental updates[C]//2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2009:2921-2928.
[11]MEHRAN R,OYAMA A,SHAH M.Abnormal crowd behavior detection using social force model [C]//Conference on Compu-ter Vision and Pattern Recognition.IEEE Computer Society,2009:935-942.
[12]XU D,RICCI E,YAN Y,et al.Learning deep representations of appearance and motion for anomalous event detection[C]//British Machine Vision Conference 2015.BMVC,2015:1-12.
[13]HASAN M,CHOI J,NEUMANN J,et al.Learning temporalregularity in video sequences[C]//Proceedings of the Confe-rence on Computer Vision and Pattern Recognition.IEEE Computer Society,2016:733-742.
[14]LUO W X,LIU W,GAO S H.A revisit of sparse coding based anomaly detection in stacked RNN framework[C]//IEEE International Conference on Computer Vision.IEEE Computer Society,2017:341-349.
[15]MORAIS R,LE V,TRAN T,et al.Learning regularity in skeleton trajectories for anomaly detection in videos[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2019:11996-12004.
[16]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial nets[C]//Neural Information Processing Systems 27,NIPS.2014:2672-2680.
[17]REED S,AKATA Z,YAN X C,et al.Generative adversarial text to image synthesis[C]//The 33nd International Conference on Machine Learning.JMLR Workshop and Conference Procee-dings,2016:1060-1069.
[18]VILLEGAS R,YANG J M,ZOU Y L,et al.Learning to gene-rate long-term future via hierarchical prediction[C]//The 34th International Conference on Machine Learning.PMLR,2017:3560-3569.
[19]KWON Y H,PARK M.Predicting future frames using retrospective cycle GAN[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2019:1810-1820.
[20]ILG E,MAYER N.FlowNet 2.0:evolution of optical flow estimation with deep networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2017:1647-1655.
[21]RONNEBERGER O,FISCHER P,BROX T.U-Net:convolu-tional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention.Lecture Notes in Computer Science,2015:234-241.
[22]WANG X B,GIRSHICK R,GUPTA A,et al.Non-Local Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2018:1768-1777.
[23]LIU W,LUO W X,LIAN D Z,et al.Future frame prediction for anomaly detection-A new baseline[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer So-ciety,2018:6536-6545.
[24]LU C W,SHI J P,JIA J Y.Abnormalevent detection at 150 FPS in MATLAB[C]//IEEE International Conference on Computer Vision.IEEE Computer Society,2013:2720-2727.
[25]GONG D,LIU L Q,LE V,et al.Memorizing normality to detect anomaly:memory-augmented deep autoencoder for unsupervised anomaly detection[C]//IEEE/CVF International Conference on Computer Vision.IEEE Computer Society,2019:1705-1714.
[1] 张佳, 董守斌.
基于评论方面级用户偏好迁移的跨领域推荐算法
Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer
计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131
[2] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[3] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[4] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[5] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[6] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[7] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[8] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[9] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[10] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[11] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[12] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[13] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[14] 刘伟业, 鲁慧民, 李玉鹏, 马宁.
指静脉识别技术研究综述
Survey on Finger Vein Recognition Research
计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056
[15] 孙福权, 崔志清, 邹彭, 张琨.
基于多尺度特征的脑肿瘤分割算法
Brain Tumor Segmentation Algorithm Based on Multi-scale Features
计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!