计算机科学

计算机科学 ›› 2021, Vol. 48 ›› Issue (9): 125-134.doi: 10.11896/jsjkx.201100015

• 计算机图形学&多媒体 • 上一篇    下一篇

基于深度学习的人群异常行为检测综述

徐涛, 田崇阳, 刘才华   

  1. 中国民航大学计算机科学与技术学院 天津300300中国民航大学民航信息技术科研基地 天津300300
  • 收稿日期:2020-11-01 修回日期:2021-01-04 出版日期:2021-09-15 发布日期:2021-09-10
  • 通讯作者: 刘才华(chliu@cauc.edu.cn)
  • 作者简介:txu@cauc.edu.cn
  • 基金资助:
    中央高校基本科研业务经费中国民航大学专项资金项目(3122018C024);天津市自然科学基金(18JCYBJC885100);中国民航大学科研启动项目(2017QD16X)

Deep Learning for Abnormal Crowd Behavior Detection:A Review

XU Tao, TIAN Chong-yang, LIU Cai-hua   

  1. College of Computer Science and Technology,Civil Aviation University of China,Tianjin 300300,China Civil Aviation Information Technology Research Base,Civil Aviation University of China,Tianjin 300300,China
  • Received:2020-11-01 Revised:2021-01-04 Online:2021-09-15 Published:2021-09-10
  • About author:XU Tao,born in 1962,Ph.D,professor,is a member of China Computer Federation.His main research interests include intelligent information processing and image processing.
    LIU Cai-hua,born in 1986,Ph.D,lectu-rer.Her main research interests include computer vision and machine learning.
  • Supported by:
    Fundamental Research Funds for the Central Universities from Civil Aviation University of China(3122018C024),Natural Science Foundation of Tianjin,China(18JCYBJC885100) and Research Initiation Project of Civil Aviation University of China(2017QD16X)

PDF (PC)

3025

摘要： 随着安防需求的日益增长,人群异常行为检测已经成为计算机视觉的研究热点。人群异常行为检测旨在对监控视频中行人的行为进行建模和分析,区分出人群中的正常行为和异常行为,及时发现灾难和意外事件。文中对基于深度学习的人群异常行为检测算法进行了梳理总结。首先,针对人群异常行为检测任务及其现状进行介绍;其次,重点探讨卷积神经网络、自编码网络和生成对抗网络在人群异常行为检测任务中的研究进展;然后,列举该领域常用的数据集,并比较和分析了深度学习方法在UCSD行人数据集上的性能;最后,总结人群异常行为检测的任务难点,并对该领域的未来发展趋势进行了展望。

关键词: 卷积神经网络, 深度学习, 生成对抗网络, 异常行为检测, 自编码网络

Abstract: With the increasing demand of security industry,abnormal crowd behavior detection has become a hot research issue in computer vision.Abnormal crowd behavior detection aims to model and analyze the behavior of pedestrians in surveillance videos,distinguish between normal and abnormal behaviors in the crowd,and discover disasters and accidents in time.A large number of algorithms for abnormal crowd behavior detection based on deep learning are summarized in this paper.First,abnormal crowd behavior detection task and its current research situation are briefly introduced.Second,the research progress of convolutional neural networks,auto-encoder and generative adversarial networks on abnormal crowd behavior detection are discussed separately.Then,some commonly used datasets are listed,and the performance of deep learning methods on UCSD pedestrian datasets are compared and analyzed.Finally,the development difficulties of abnormal crowd behavior detection tasks are summarized,and its future research directions are discussed.

Key words: Abnormal behavior detection, Autoencoder, Convolutional neural network, Deep learning, Generative adversarial networks

中图分类号: 

  • TP391
[1]HU Y.Design and Implementation of Abnormal Behavior Detection Based on Deep Intelligent Analysis Algorithms in Massive Video Surveillance[J].Journal of Grid Computing,2020,18(2):227-237.
[2]ALI K,MOHAMMAD S M.Improved Anomaly Detection in Surveillance Videos Based on a Deep Learning Method[C]//2018 8th Conference of AI & Robotics and 10th RoboCup Iranopen International Symposium.2018:73-81.
[3]BERA A,KIM S,MANOCHA D.Realtime Anomaly Detection Using Trajectory-Level Crowd Behavior Learning[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).2016:1289-1296.
[4]MARCO B,ALBERTO D B,LORENZO S.Multi-Scale andReal-Time Non-Parametric Approach for Anomaly Detection and Localization[J].Computer Vision and ImageUnderstan-ding,2012,116(3):320-329.
[5]YANG C,YUAN J S,LIU J.Abnormal Event Detection inCrowded Scenes Using Sparse Representation[J].Pattern Re-cognition,2013,46(7):1851-1864.
[6]VENKATESH S,ZHU C Z.Video Anomaly DetectionBased on Local Statistical Aggregates[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.2012:2112-2119.
[7]MAHADEVAN V,LI W X,BHALODIA V,et al.Anomaly Detection in Crowded Scenes[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2010:1975-1981.
[8]MEHRAN R,OYAMA A,SHAH M.Abnormal Crowd Beha-vior Detection Using Social Force Model[C]//2009 IEEE Confe-rence on Computer Vision and Pattern Recognition.2009:935-942.
[9]ANTIC B,OMMER B.Video Parsing for Abnormality Detection[C]//2011 International Conference on Computer Vision.2011:2415-2422.
[10]MABROUK A B,ZAGROUBA E.Abnormal Behavior Recognition for Intelligent Video Surveillance Systems:A Review[J].Expert Systems With Applications,2018,91:480-491.
[11]AFIQ A A,ZAKARIYA M A,SAAD M N,et al.A Review on Classifying Abnormal Behavior in Crowd Scene[J].Journal of Visual Communication and Image Representation,2019,58:285-303.
[12]TRIPATHI G,SINGH K,VISHWAKARMA D K.Convolu-tional Neural Networks for Crowd Behaviour Analysis:A Survey[J].The Visual Computer,2019,35(5):753-776.
[13]RAMACHANDRA B,VATSAVAI R R.A Survey of Single-Scene Video Anomaly Detection[J].arXiv:2004.05993,2020.
[14]LIU W B,WANG Z D,LIU X H,et al.A survey of deep neural network architectures and their applications[J].Neurocompu-ting,2017,234:11-16.
[15]TAY N C,CONNIE T,ONG T S,et al.A Robust Abnormal Behavior Detection Method Using Convolutional Neural Network[C]//International Conference on Omputational Science and Technology.2019:37-47.
[16]SMEUREANU S,IONESCU R T,POPESCU M,et al.Deep Appearance Features for Abnormal Behavior Detection in Video[C]//International Conference on Image Analysis and Proces-sing.2017:779-789.
[17]SINGH K,RAJORA S,VISHWAKARMA D K,et al.Crowd Anomaly Detection Using Aggregation of Ensembles of Fine-Tuned ConvNets[J].Neurocomputing,2020,371:188-198.
[18]NAZARE T S,MELLO R F,PONTI M A.Are Pre-Trained CNNs Good Feature Extractors for Anomaly Detection in Surveillance Videos?[J].arXiv:1811.08495,2018.
[19] SABOKROU M,FAYYAZ M,FATHY M,et al.Deep-Anomaly:Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes[J].Computer Vision and Image Understanding,2018,172:88-97.
[20]PANG G S,YAN C,SHEN C H,et al.Self-Trained Deep Ordinal Regression for End-to-End Video Anomaly Detection[C]//CVPR 2020:Computer Vision and Pattern Recognition.2020:12173-12182.
[21]HINAMI R,MEI T,SATOH S.Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge[C]//2017 IEEE International Conference on Computer Vision (ICCV).2017:3639-3647.
[22]XIA L M,LI Z M.A new method of abnormal behavior detection using LSTM network with temporal attention mechanism[J].The Journal of Supercomputing,2021,77:3223-3241.
[23]RAVANBAKHSH M,NABI M,MOUSAVI H,et al.Plug-and-Play CNN for Crowd Motion Analysis:An Application in Abnormal Event[C]//Detection 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).2018:1689-1698.
[24]HU X,DAI J,HUANG Y P,et al.A Weakly Supervised Framework for Abnormal Behavior Detection and Localization in Crowded Scenes[J].Neurocomputing,2020,383:270-281.
[25]RAMACHANDRA B,JONES M J,VATSAVAI R R.Learning a Distance Function with a Siamese Network to Localize Ano-malies in Videos[C]//2020 IEEE Winter Conference on Applications of Computer Vision (WACV).2020:2598-2607.
[26]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[27]SUDHAKARAN S,LANZ O.Learning to Detect Violent VideosUsing Convolutional Long Short-Term Memory[C]//2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).2017:1-6.
[28]SHI X J,CHEN Z R,WANG H,et al.Convolutional LSTM Network:A Machine Learning Approach for Precipitation Nowcasting[C]//NIPS'15 Proceedings of the 28th International Conference on Neural Information Processing Systems.2015:802-810.
[29]KO K,SIM K.Deep Convolutional Framework for AbnormalBehavior Detection in a Smart Surveillance System[J].Engineering Applications of Artificial Intelligence,2018,67:226-234.
[30]TAY N C,TEE C,ONG T S,et al.Abnormal Behavior Recognition Using CNN-LSTM with Attention Mechanism[C]//2019 1st International Conference on Electrical,Control and Instrumentation Engineering (ICECIE).2019:1-5.
[31]TRAN D,BOURDEV L,FERGUS R,et al.Learning Spatiotemporal Features with 3D Convolutional Networks[C]//2015 IEEE International Conference on Computer Vision (ICCV).2015:4489-4497.
[32]ZHOU S F,SHEN W,ZENG D,et al.Spatial-Temporal Convolutional Neural Networks for Anomaly Detection and Localization in Crowded Scenes[J].Signal Processing-Image Communication,2016,47:358-368.
[33]GONG M G,ZENG H M,XIE Y,et al.Local Distinguishability Aggrandizing Network for Human Anomaly Detection[J].Neural Networks,2020,122:364-373.
[34]SULTANI W,CHEN C,SHAH M.Real-World Anomaly Detection in Surveillance Videos[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:6479-6488.
[35]DUBEY S,BORAGULE A,JEON M.3D ResNet with Ranking Loss Function for Abnormal Activity Detection in Videos[C]//2019 International Conference on Control,Automation and Information Sciences (ICCAIS).2019:1-6.
[36]ZHANG J G,QING L Y,MIAO J.Temporal ConvolutionalNetwork with Complementary Inner Bag Loss for Weakly Supervised Anomaly Detection[C]//2019 IEEE International Conference on Image Processing (ICIP).2019:4030-4034.
[37]KOSHTI D,KAMOJI S,KALNAD N,et al.Video Anomaly Detection Using Inflated 3D Convolution Network[C]//2020 International Conference on Inventive Computation Technologies (ICICT).2020:729-733.
[38]HINTON G E,SALAKHUTDINOV R R.Reducing the Dimensionality of Data with Neural Networks[J].Science,2006,313(5786):504-507.
[39]RIBEIRO M,LAZZARETTI A E,LOPES H S.A Study of Deep Convolutional Auto-Encoders for Anomaly Detection in Videos[J].Pattern Recognition Letters,2018,105:13-22.
[40]BGUYEN T,MEUNIER J.Anomaly Detection in Video Se-quence With Appearance-Motion Correspondence[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).2019:1273-1283.
[41]HASAN M,CHOI J,NEUMANN J,et al.Learning Temporal Regularity in Video Sequences[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2016:733-742.
[42]YANG B,CAO J M,WANG N,et al.Anomalous Behaviors Detection in Moving Crowds Based on a Weighted Convolutional Autoencoder-Long Short-Term Memory Network[J].IEEE Transactions on Cognitive and Developmental Systems,2019,11(4):473-482.
[43]CHONG Y S,TAY Y H.Abnormal Event Detection in Videos Using Spatiotemporal Autoencoder[C]//International Sympo-sium on Neural Networks.2017:189-196.
[44]WANG L,ZHOU F Q,LI Z X,et al.Abnormal Event Detection in Videos Using Hybrid Spatio-Temporal Autoencoder[C]//2018 25th IEEE International Conference on Image Processing (ICIP).2018:2276-2280.
[45]YAN S Y,SMITH J S,LU W J,et al.Abnormal Event Detection From Videos Using a Two-Stream Recurrent Variational Autoencoder[J].IEEE Transactions on Cognitive and Developmental Systems,2020,12(1):30-42.
[46]NAWARATNE R,ALAHAKOON D,SILVA D D,et al.Spa-tiotemporal Anomaly Detection Using Deep Learning for Real-Time Video Surveillance[J].IEEE Transactions on Industrial Informatics,2020,16(1):393-402.
[47]ZHAO Y R,DENG B,SHEN C,et al.Spatio-Temporal AutoEncoder for Video Anomaly Detection[C]// Proceedings of the 25th ACM International Conference on Multimedia.2017:1933-1941.
[48]FU J P,FAN W T,BOUGUILA N.A Novel Approach forAnomaly Event Detection in Videos Based on Autoencoders and SE Networks[C]//2018 9th International Symposium on Signal,Image,Video and Communications (ISIVC).2018:179-184.
[49]GONG D,LIU L Q,LE V,et al.Memorizing Normality to Detect Anomaly:Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).2019:1705-1714.
[50]WU P,LIU J,SHEN F.A Deep One-Class Neural Network for Anomalous Event Detection in Complex Scenes[J].IEEE Transactions on Neural Networks,2020,31(7):2609-2622.
[51]IONESCU R T,KHAN F S,GEORGESCU M,et al.Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection in Video[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2019:7842-7851.
[52]ABATI D,PORRELLO A,CALDERARA S,et al.Latent Space Autoregression for Novelty Detection[C] //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2019:481-490.
[53]PARK H,NOH J,HAM B.Learning Memory-Guided Normality for Anomaly Detection[C]//Computer Vision and Pattern Recognition(CVPR).2020:14372-14381.
[54]XU D,YAN Y,RICCI E,et al.Detecting Anomalous Events in Videos by Learning Deep Representations of Appearance and Motion[J].Computer Vision and Image Understanding,2017,156:117-127.
[55]FAN Y X,WEN G J,LI D R,et al.Video Anomaly Detection and Localization via Gaussian Mixture Fully ConvolutionalVaria-tional Autoencoder[J].Computer Vision and Image Understanding,2020,195:1-13.
[56]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative Adversarial Nets[C]//Advances in Neural Information Processing Systems 27.2014:2672-2680.
[57]ISOLA P,ZHU J Y,ZHOU T H,et al.Image-to-Image Translation with Conditional Adversarial Networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2017:5967-5976.
[58]RAVANBAKHSH M,NABI M,SANGINETO E,et al.Abnormal Event Detection in Videos Using Generative Adversarial Nets[C]//2017 IEEE International Conference on Image Processing (ICIP).2017:1-5.
[59]VU H,NGUYEN T D,LE T,et al.Robust Anomaly Detection in Videos Using Multilevel Representations[C]//AAAI 2019:Thirty-Third AAAI Conference on Artificial Intelligence.2019:5216-5223.
[60]GOLDA T,MURZYN N,QU C C,et al.What Goes aroundComes around:Cycle-Consistency-Based Short-Term Motion Prediction for Anomaly Detection Using Generative Adversarial Networks[C]//2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).2019:1-8.
[61]SONG H,SUN C,WU X X,et al.Learning Normal Patterns via Adversarial Attention-Based Autoencoder for Abnormal Event Detection in Videos[J].IEEE Transactions on Multimedia,2019,22(8):2138-2148.
[62]LIU W,LUO W X,LING D Z,et al.Future Frame Prediction for Anomaly Detection - A New Baseline[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:6536-6545.
[63]TANG Y,ZHAO L,ZHANG S S,et al.Integrating Prediction and Reconstruction for Anomaly Detection[J].Pattern Recognition Letters,2019,129:123-130.
[64]LEI Z,DENG F,YANG X D.Spatial Temporal Balanced Gene-rative Adversarial Autoencoder for Anomaly Detection[C]//Proceedings of the 2019 International Conference on Image,Vi-deo and Signal Processing.2019:1-7.
[65]LEE S,KIM H G,RO Y M.BMAN:Bidirectional Multi-Scale Aggregation Networks for Abnormal Event Detection[J].IEEE Transactions on Image Processing,2020,29:2395-2408.
[66]SABOKROU M,POURREZA M,FAYYZA M,et al.AVID:Adversarial Visual Irregularity Detection[C]//Asian Confe-rence on Computer Vision.2018:488-505.
[67]SABOKROU M,KHALOOEI M,FATHY M,et al.Adversarial-ly Learned One-Class Classifier for Novelty Detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:3379-3388.
[68]ZAHEER M Z,LEE J,ASTRID M,et al.Old Is Gold:Redefi-ning the Adversarially Learned One-Class Classifier Training Paradigm[J].CVPR 2020:Computer Vision and Pattern Reco-gnition,2020:14183-14193.
[69]LEE S,KIM H G,RO Y M.Stan:Spatio-Temporal Adversarial Networks for Abnormal Event Detection[C]//2018 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).2018:1323-1327.
[70]RAVANBAKHSH M,SANGINETO E,NABI M,et al.Trai-ning Adversarial Discriminators for Cross-Channel Abnormal Event Detection in Crowds[C]//2019 IEEE Winter Conference on Applications of Computer Vision (WACV).2019:1896-1904.
[71]YAN M J,JIANG X D,YUAN J S.3D Convolutional Generative Adversarial Networks for Detecting Temporal Irregularities in Videos[C]//2018 24th International Conference on Pattern Recognition (ICPR).2018:2522-2527.
[72]MATHIEU M,COUPRIE C,LECUN Y.Deep Multi-Scale Vi-deo Prediction beyond Mean Square Error[C]//International Conference on Learning Representations(ICLR 2016).2016:1-14.
[73]DHIMAN C,VISHWAKARMA D K.A Review of State-of-the-Art Techniques for Abnormal Human Activity Recognition[J].Engineering Applications of Artificial Intelligence,2019,77:21-45.
[1] 张佳, 董守斌.
基于评论方面级用户偏好迁移的跨领域推荐算法
Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer
计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131
[2] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[3] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[5] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[6] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[7] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[8] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[9] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[10] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[11] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[12] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[13] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[14] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[15] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发