Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (9): 125-134.doi: 10.11896/jsjkx.201100015

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Deep Learning for Abnormal Crowd Behavior Detection:A Review

XU Tao, TIAN Chong-yang, LIU Cai-hua   

  1. College of Computer Science and Technology,Civil Aviation University of China,Tianjin 300300,China Civil Aviation Information Technology Research Base,Civil Aviation University of China,Tianjin 300300,China
  • Received:2020-11-01 Revised:2021-01-04 Online:2021-09-15 Published:2021-09-10
  • About author:XU Tao,born in 1962,Ph.D,professor,is a member of China Computer Federation.His main research interests include intelligent information processing and image processing.
    LIU Cai-hua,born in 1986,Ph.D,lectu-rer.Her main research interests include computer vision and machine learning.
  • Supported by:
    Fundamental Research Funds for the Central Universities from Civil Aviation University of China(3122018C024),Natural Science Foundation of Tianjin,China(18JCYBJC885100) and Research Initiation Project of Civil Aviation University of China(2017QD16X)

PDF (PC)

2972

Abstract: With the increasing demand of security industry,abnormal crowd behavior detection has become a hot research issue in computer vision.Abnormal crowd behavior detection aims to model and analyze the behavior of pedestrians in surveillance videos,distinguish between normal and abnormal behaviors in the crowd,and discover disasters and accidents in time.A large number of algorithms for abnormal crowd behavior detection based on deep learning are summarized in this paper.First,abnormal crowd behavior detection task and its current research situation are briefly introduced.Second,the research progress of convolutional neural networks,auto-encoder and generative adversarial networks on abnormal crowd behavior detection are discussed separately.Then,some commonly used datasets are listed,and the performance of deep learning methods on UCSD pedestrian datasets are compared and analyzed.Finally,the development difficulties of abnormal crowd behavior detection tasks are summarized,and its future research directions are discussed.

Key words: Abnormal behavior detection, Autoencoder, Convolutional neural network, Deep learning, Generative adversarial networks

CLC Number: 

  • TP391
[1]HU Y.Design and Implementation of Abnormal Behavior Detection Based on Deep Intelligent Analysis Algorithms in Massive Video Surveillance[J].Journal of Grid Computing,2020,18(2):227-237.
[2]ALI K,MOHAMMAD S M.Improved Anomaly Detection in Surveillance Videos Based on a Deep Learning Method[C]//2018 8th Conference of AI & Robotics and 10th RoboCup Iranopen International Symposium.2018:73-81.
[3]BERA A,KIM S,MANOCHA D.Realtime Anomaly Detection Using Trajectory-Level Crowd Behavior Learning[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).2016:1289-1296.
[4]MARCO B,ALBERTO D B,LORENZO S.Multi-Scale andReal-Time Non-Parametric Approach for Anomaly Detection and Localization[J].Computer Vision and ImageUnderstan-ding,2012,116(3):320-329.
[5]YANG C,YUAN J S,LIU J.Abnormal Event Detection inCrowded Scenes Using Sparse Representation[J].Pattern Re-cognition,2013,46(7):1851-1864.
[6]VENKATESH S,ZHU C Z.Video Anomaly DetectionBased on Local Statistical Aggregates[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.2012:2112-2119.
[7]MAHADEVAN V,LI W X,BHALODIA V,et al.Anomaly Detection in Crowded Scenes[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2010:1975-1981.
[8]MEHRAN R,OYAMA A,SHAH M.Abnormal Crowd Beha-vior Detection Using Social Force Model[C]//2009 IEEE Confe-rence on Computer Vision and Pattern Recognition.2009:935-942.
[9]ANTIC B,OMMER B.Video Parsing for Abnormality Detection[C]//2011 International Conference on Computer Vision.2011:2415-2422.
[10]MABROUK A B,ZAGROUBA E.Abnormal Behavior Recognition for Intelligent Video Surveillance Systems:A Review[J].Expert Systems With Applications,2018,91:480-491.
[11]AFIQ A A,ZAKARIYA M A,SAAD M N,et al.A Review on Classifying Abnormal Behavior in Crowd Scene[J].Journal of Visual Communication and Image Representation,2019,58:285-303.
[12]TRIPATHI G,SINGH K,VISHWAKARMA D K.Convolu-tional Neural Networks for Crowd Behaviour Analysis:A Survey[J].The Visual Computer,2019,35(5):753-776.
[13]RAMACHANDRA B,VATSAVAI R R.A Survey of Single-Scene Video Anomaly Detection[J].arXiv:2004.05993,2020.
[14]LIU W B,WANG Z D,LIU X H,et al.A survey of deep neural network architectures and their applications[J].Neurocompu-ting,2017,234:11-16.
[15]TAY N C,CONNIE T,ONG T S,et al.A Robust Abnormal Behavior Detection Method Using Convolutional Neural Network[C]//International Conference on Omputational Science and Technology.2019:37-47.
[16]SMEUREANU S,IONESCU R T,POPESCU M,et al.Deep Appearance Features for Abnormal Behavior Detection in Video[C]//International Conference on Image Analysis and Proces-sing.2017:779-789.
[17]SINGH K,RAJORA S,VISHWAKARMA D K,et al.Crowd Anomaly Detection Using Aggregation of Ensembles of Fine-Tuned ConvNets[J].Neurocomputing,2020,371:188-198.
[18]NAZARE T S,MELLO R F,PONTI M A.Are Pre-Trained CNNs Good Feature Extractors for Anomaly Detection in Surveillance Videos?[J].arXiv:1811.08495,2018.
[19] SABOKROU M,FAYYAZ M,FATHY M,et al.Deep-Anomaly:Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes[J].Computer Vision and Image Understanding,2018,172:88-97.
[20]PANG G S,YAN C,SHEN C H,et al.Self-Trained Deep Ordinal Regression for End-to-End Video Anomaly Detection[C]//CVPR 2020:Computer Vision and Pattern Recognition.2020:12173-12182.
[21]HINAMI R,MEI T,SATOH S.Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge[C]//2017 IEEE International Conference on Computer Vision (ICCV).2017:3639-3647.
[22]XIA L M,LI Z M.A new method of abnormal behavior detection using LSTM network with temporal attention mechanism[J].The Journal of Supercomputing,2021,77:3223-3241.
[23]RAVANBAKHSH M,NABI M,MOUSAVI H,et al.Plug-and-Play CNN for Crowd Motion Analysis:An Application in Abnormal Event[C]//Detection 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).2018:1689-1698.
[24]HU X,DAI J,HUANG Y P,et al.A Weakly Supervised Framework for Abnormal Behavior Detection and Localization in Crowded Scenes[J].Neurocomputing,2020,383:270-281.
[25]RAMACHANDRA B,JONES M J,VATSAVAI R R.Learning a Distance Function with a Siamese Network to Localize Ano-malies in Videos[C]//2020 IEEE Winter Conference on Applications of Computer Vision (WACV).2020:2598-2607.
[26]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[27]SUDHAKARAN S,LANZ O.Learning to Detect Violent VideosUsing Convolutional Long Short-Term Memory[C]//2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).2017:1-6.
[28]SHI X J,CHEN Z R,WANG H,et al.Convolutional LSTM Network:A Machine Learning Approach for Precipitation Nowcasting[C]//NIPS'15 Proceedings of the 28th International Conference on Neural Information Processing Systems.2015:802-810.
[29]KO K,SIM K.Deep Convolutional Framework for AbnormalBehavior Detection in a Smart Surveillance System[J].Engineering Applications of Artificial Intelligence,2018,67:226-234.
[30]TAY N C,TEE C,ONG T S,et al.Abnormal Behavior Recognition Using CNN-LSTM with Attention Mechanism[C]//2019 1st International Conference on Electrical,Control and Instrumentation Engineering (ICECIE).2019:1-5.
[31]TRAN D,BOURDEV L,FERGUS R,et al.Learning Spatiotemporal Features with 3D Convolutional Networks[C]//2015 IEEE International Conference on Computer Vision (ICCV).2015:4489-4497.
[32]ZHOU S F,SHEN W,ZENG D,et al.Spatial-Temporal Convolutional Neural Networks for Anomaly Detection and Localization in Crowded Scenes[J].Signal Processing-Image Communication,2016,47:358-368.
[33]GONG M G,ZENG H M,XIE Y,et al.Local Distinguishability Aggrandizing Network for Human Anomaly Detection[J].Neural Networks,2020,122:364-373.
[34]SULTANI W,CHEN C,SHAH M.Real-World Anomaly Detection in Surveillance Videos[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:6479-6488.
[35]DUBEY S,BORAGULE A,JEON M.3D ResNet with Ranking Loss Function for Abnormal Activity Detection in Videos[C]//2019 International Conference on Control,Automation and Information Sciences (ICCAIS).2019:1-6.
[36]ZHANG J G,QING L Y,MIAO J.Temporal ConvolutionalNetwork with Complementary Inner Bag Loss for Weakly Supervised Anomaly Detection[C]//2019 IEEE International Conference on Image Processing (ICIP).2019:4030-4034.
[37]KOSHTI D,KAMOJI S,KALNAD N,et al.Video Anomaly Detection Using Inflated 3D Convolution Network[C]//2020 International Conference on Inventive Computation Technologies (ICICT).2020:729-733.
[38]HINTON G E,SALAKHUTDINOV R R.Reducing the Dimensionality of Data with Neural Networks[J].Science,2006,313(5786):504-507.
[39]RIBEIRO M,LAZZARETTI A E,LOPES H S.A Study of Deep Convolutional Auto-Encoders for Anomaly Detection in Videos[J].Pattern Recognition Letters,2018,105:13-22.
[40]BGUYEN T,MEUNIER J.Anomaly Detection in Video Se-quence With Appearance-Motion Correspondence[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).2019:1273-1283.
[41]HASAN M,CHOI J,NEUMANN J,et al.Learning Temporal Regularity in Video Sequences[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2016:733-742.
[42]YANG B,CAO J M,WANG N,et al.Anomalous Behaviors Detection in Moving Crowds Based on a Weighted Convolutional Autoencoder-Long Short-Term Memory Network[J].IEEE Transactions on Cognitive and Developmental Systems,2019,11(4):473-482.
[43]CHONG Y S,TAY Y H.Abnormal Event Detection in Videos Using Spatiotemporal Autoencoder[C]//International Sympo-sium on Neural Networks.2017:189-196.
[44]WANG L,ZHOU F Q,LI Z X,et al.Abnormal Event Detection in Videos Using Hybrid Spatio-Temporal Autoencoder[C]//2018 25th IEEE International Conference on Image Processing (ICIP).2018:2276-2280.
[45]YAN S Y,SMITH J S,LU W J,et al.Abnormal Event Detection From Videos Using a Two-Stream Recurrent Variational Autoencoder[J].IEEE Transactions on Cognitive and Developmental Systems,2020,12(1):30-42.
[46]NAWARATNE R,ALAHAKOON D,SILVA D D,et al.Spa-tiotemporal Anomaly Detection Using Deep Learning for Real-Time Video Surveillance[J].IEEE Transactions on Industrial Informatics,2020,16(1):393-402.
[47]ZHAO Y R,DENG B,SHEN C,et al.Spatio-Temporal AutoEncoder for Video Anomaly Detection[C]// Proceedings of the 25th ACM International Conference on Multimedia.2017:1933-1941.
[48]FU J P,FAN W T,BOUGUILA N.A Novel Approach forAnomaly Event Detection in Videos Based on Autoencoders and SE Networks[C]//2018 9th International Symposium on Signal,Image,Video and Communications (ISIVC).2018:179-184.
[49]GONG D,LIU L Q,LE V,et al.Memorizing Normality to Detect Anomaly:Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).2019:1705-1714.
[50]WU P,LIU J,SHEN F.A Deep One-Class Neural Network for Anomalous Event Detection in Complex Scenes[J].IEEE Transactions on Neural Networks,2020,31(7):2609-2622.
[51]IONESCU R T,KHAN F S,GEORGESCU M,et al.Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection in Video[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2019:7842-7851.
[52]ABATI D,PORRELLO A,CALDERARA S,et al.Latent Space Autoregression for Novelty Detection[C] //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2019:481-490.
[53]PARK H,NOH J,HAM B.Learning Memory-Guided Normality for Anomaly Detection[C]//Computer Vision and Pattern Recognition(CVPR).2020:14372-14381.
[54]XU D,YAN Y,RICCI E,et al.Detecting Anomalous Events in Videos by Learning Deep Representations of Appearance and Motion[J].Computer Vision and Image Understanding,2017,156:117-127.
[55]FAN Y X,WEN G J,LI D R,et al.Video Anomaly Detection and Localization via Gaussian Mixture Fully ConvolutionalVaria-tional Autoencoder[J].Computer Vision and Image Understanding,2020,195:1-13.
[56]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative Adversarial Nets[C]//Advances in Neural Information Processing Systems 27.2014:2672-2680.
[57]ISOLA P,ZHU J Y,ZHOU T H,et al.Image-to-Image Translation with Conditional Adversarial Networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2017:5967-5976.
[58]RAVANBAKHSH M,NABI M,SANGINETO E,et al.Abnormal Event Detection in Videos Using Generative Adversarial Nets[C]//2017 IEEE International Conference on Image Processing (ICIP).2017:1-5.
[59]VU H,NGUYEN T D,LE T,et al.Robust Anomaly Detection in Videos Using Multilevel Representations[C]//AAAI 2019:Thirty-Third AAAI Conference on Artificial Intelligence.2019:5216-5223.
[60]GOLDA T,MURZYN N,QU C C,et al.What Goes aroundComes around:Cycle-Consistency-Based Short-Term Motion Prediction for Anomaly Detection Using Generative Adversarial Networks[C]//2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).2019:1-8.
[61]SONG H,SUN C,WU X X,et al.Learning Normal Patterns via Adversarial Attention-Based Autoencoder for Abnormal Event Detection in Videos[J].IEEE Transactions on Multimedia,2019,22(8):2138-2148.
[62]LIU W,LUO W X,LING D Z,et al.Future Frame Prediction for Anomaly Detection - A New Baseline[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:6536-6545.
[63]TANG Y,ZHAO L,ZHANG S S,et al.Integrating Prediction and Reconstruction for Anomaly Detection[J].Pattern Recognition Letters,2019,129:123-130.
[64]LEI Z,DENG F,YANG X D.Spatial Temporal Balanced Gene-rative Adversarial Autoencoder for Anomaly Detection[C]//Proceedings of the 2019 International Conference on Image,Vi-deo and Signal Processing.2019:1-7.
[65]LEE S,KIM H G,RO Y M.BMAN:Bidirectional Multi-Scale Aggregation Networks for Abnormal Event Detection[J].IEEE Transactions on Image Processing,2020,29:2395-2408.
[66]SABOKROU M,POURREZA M,FAYYZA M,et al.AVID:Adversarial Visual Irregularity Detection[C]//Asian Confe-rence on Computer Vision.2018:488-505.
[67]SABOKROU M,KHALOOEI M,FATHY M,et al.Adversarial-ly Learned One-Class Classifier for Novelty Detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:3379-3388.
[68]ZAHEER M Z,LEE J,ASTRID M,et al.Old Is Gold:Redefi-ning the Adversarially Learned One-Class Classifier Training Paradigm[J].CVPR 2020:Computer Vision and Pattern Reco-gnition,2020:14183-14193.
[69]LEE S,KIM H G,RO Y M.Stan:Spatio-Temporal Adversarial Networks for Abnormal Event Detection[C]//2018 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).2018:1323-1327.
[70]RAVANBAKHSH M,SANGINETO E,NABI M,et al.Trai-ning Adversarial Discriminators for Cross-Channel Abnormal Event Detection in Crowds[C]//2019 IEEE Winter Conference on Applications of Computer Vision (WACV).2019:1896-1904.
[71]YAN M J,JIANG X D,YUAN J S.3D Convolutional Generative Adversarial Networks for Detecting Temporal Irregularities in Videos[C]//2018 24th International Conference on Pattern Recognition (ICPR).2018:2522-2527.
[72]MATHIEU M,COUPRIE C,LECUN Y.Deep Multi-Scale Vi-deo Prediction beyond Mean Square Error[C]//International Conference on Learning Representations(ICLR 2016).2016:1-14.
[73]DHIMAN C,VISHWAKARMA D K.A Review of State-of-the-Art Techniques for Abnormal Human Activity Recognition[J].Engineering Applications of Artificial Intelligence,2019,77:21-45.
[1] WANG Guan-yu, ZHONG Ting, FENG Yu, ZHOU Fan. Collaborative Filtering Recommendation Method Based on Vector Quantization Coding [J]. Computer Science, 2022, 49(9): 48-54.
[2] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[5] LIU Xin, WANG Jun, SONG Qiao-feng, LIU Jia-hao. Collaborative Multicast Proactive Caching Scheme Based on AAE [J]. Computer Science, 2022, 49(9): 260-267.
[6] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[7] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[8] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[9] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[10] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[11] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[12] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[13] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[14] DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[15] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.