Computer Science ›› 2022, Vol. 49 ›› Issue (2): 142-148.doi: 10.11896/jsjkx.210900266

• Computer Vision: Theory and Application • Previous Articles     Next Articles

Video Anomaly Detection Based on Implicit View Transformation

LENG Jia-xu1,2, TAN Ming-pi1,3, HU Bo1, GAO Xin-bo1   

  1. 1 Key Laboratory of Image Cognition,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 Jiangsu Key Laboratory of Image and Video Understanding for Social Safety,Nanjing University of Science and Technology,Nanjing 210094,China
    3 School of Optoelectronic Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
  • Received:2021-09-29 Revised:2021-11-08 Online:2022-02-15 Published:2022-02-23
  • About author:LENG Jia-xu,born in 1989,Ph.D,lecture,is a member of China Computer Federation.His main research interests include computer vision and object detection.
    GAO Xin-bo,born in 1972,Ph.D,professor,is one of the talent bank of 10 000 advisor for innovation and entrepreneurship,is a board member of China Computer Federation.His main research interests include artificial intelligence and pattern recognition.
  • Supported by:
    National Natural Science Foundation of China(62036007,62050175,62102057) and Science and Technology Research Program of Chongqing Municipal Education Commission(KJQN-202100627).

Abstract: Existing deep learning-based video anomaly detection methods all detect anomalies in video clips under a single view,ignoring the importance of view information in video anomaly detection.Under a single view,when anomalies are occluded or not obvious,the performance of existing algorithms will suffer drops.To avoid this problem,the author firstly introduces the concept of view transformation into video anomaly detection,which improves the robustness of the model by judging abnormalities from multiple views.However,due to the lack of multi-view supervision information in the dataset,it is difficult to achieve explicit view transformation.Specifically,in order to reflect the idea of view transformation,the author proposes a video anomaly detection method based on implicit view transformation,using the optical flow information between frames to warp the implicit view information of the previous frame to the target frame,so as to realize the implicit view transformation from the target frame to the previous frame.And then,the method performs secondary anomaly detection on the target frame after view transformation.Experimental results show that the proposed method responds more sensitively to abnormal data and has a more robust normal data fitting ability.The AUC values on the UCSD Ped2 and CUHK Avenue datasets reached 97.0% and 88.9%,respectively.

Key words: Deep learning, Implicit view transformation, Multi-view detection, Optical flow warp, Video anomaly detection

CLC Number: 

  • TP391.41
[1]PENG J L,ZHAO Y L,WANG L M.An Overview of VideoAnomaly Behavior Detection Based on Deep Learning[J].Laser &Optoelectronics Progress,2020,58(6):1-17.
[2]KINGMA D P,WELLING M.Auto-encoding variational bayes[C]//Proceedings of the International Conference on Learning Representations.Banff,Canada,2014.
[3]HASAN M,CHOI J,NEUMANN J,et al.Learning TemporalRegularity in Video Sequences[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,USA,2016:733-742.
[4]LIU W,LUO W,LIAN D,et al.Future frame prediction foranomaly detection-a new baseline[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Utah,USA,2018:6536-6545.
[5]YE M C,PENG X J,GAN W H,et al.Anopcn:Video anomalydetection via deep predictive coding network[C]//Proceedings of the ACM International Conference on Multimedia.Nice,France,2019:1805-1813.
[6]GONG D,LIU L,LE V,et al.Memorizing normality to detectanomaly:Memory-augmented deep autoencoder for unsupervised anomaly detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Seoul,Korea,2019:1705-1714.
[7]PARK H,NOH J,HAM B.Learning memory-guided normality for anomaly detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Seattle,USA,2020:14372-14381.
[8]CAI R C,ZHANG H,LIU W,et al.Appearance-Motion Memory Consistency Network for Video Anomaly Detection[J].AAAI Conference on Artificial Intelligence,2021,35(2):938-946.
[9]ZHU Z,WU W,ZOU W,et al.End-to-End Flow CorrelationTracking with Spatial-Temporal Attention[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Utah,USA,2018:548-557.
[10]ZHAO Y R,DENG B,SHEN C,et al.Spatio-Temporal AutoEncoder for Video Anomaly Detection[C]//Proceedings of the ACM International Conference on Multimedia.New York,USA,2017:1933-1941.
[11]LUO W X,LIU W,GAO S H.Remembering history with con-volutional LSTM for anomaly detection[C]//Proceedings of the IEEE International Conference on Multimedia and Expo.Hong Kong,China,2017:439-444.
[12]WANG X Z,CHE Z P,YANG K,et al.Robust UnsupervisedVideo Anomaly Detection by Multi-Path Frame Prediction[J].IEEE Transactions on Neural Networks and Learning Systems,2021.
[13]LIU Z A,NIE Y W,LONG C J,et al.A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction[C]//Proceedings of the IEEE International Conference on Computer Vision.Montreal,Canada,2021:13588-13597.
[14]SALIGRAMA V,KONRAD J,JODOIN P.Video AnomalyIdentification[J].IEEE Signal Processing Magazine,2010,27(5):18-33.
[15]LEYVA R,SANCHEZ V,LI C T.Video Anomaly Detectionwith Compact Feature Sets for Online Performance[J].IEEE Transactions on Image Processing,2017,26(7):3463-3478.
[16]LU C,SHI J,JIA J.Abnormal Event Detection at 150 FPS in MATLAB[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.Sydney,Australia,2013:2720-2727.
[17]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention.Cham:Springer,2015:234-241.
[18]ZHU X Z,XIONG Y W,DAI J F,et al.Deep Feature Flow for Video Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Hawaii,USA,2017:4141-4150.
[19]LIU L,ZHANG J N,HE R F,et al.Learning by Analogy:Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Seattle,USA,2020:6488-6497.
[20]ILG E,MAYER N,SAIKIA T,et al.FlowNet 2.0:Evolution of Optical Flow Estimation with Deep Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Hawaii,USA,2017:1647-1655.
[21]FISCHER P,DOSOVITSKIY A,ILG E,et al.FlowNet:Lear-ning Optical Flow with Convolutional Networks[C]//Procee-dings of the IEEE International Conference on Computer Vision.Santiago,Chile,2016:2758-2766.
[22]LI W X,MAHADEVAN V,VASCONCELOS N.Anomaly de-tection and localization in crowded scenes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(1);18-32.
[23]PASZKE A,GROSS S,MASSA F,et al.PyTorch:An imperative style,high-performance deep learning library[C]//Procee-dings of the Annual Conference on Neural Information Proces-sing Systems.Vancouver,Canada,2019:8026-8037.
[24]KINGMA D P,BA J.Adam:A method for stochastic optimization[C]//Proceedings of the International Conference on Lear-ning Representations.San Diego,USA,2015:1-15.
[25]LUO W X,LIU W,GAO S H.A revisit of sparse coding based anomaly detection in stacked RNN framework[C]//Proceedings of the IEEE International Conference on Computer Vision.Ve-nice,Italy,2017:341-349.
[26]IONESCU R T,SMEUREANU S,ALEXE B,et al.Unmasking the abnormal events in video[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy,2017:2914-2922.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[9] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[10] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[11] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[12] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[13] ZHU Wen-tao, LAN Xian-chao, LUO Huan-lin, YUE Bing, WANG Yang. Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN [J]. Computer Science, 2022, 49(6A): 378-383.
[14] WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[15] MAO Dian-hui, HUANG Hui-yu, ZHAO Shuang. Study on Automatic Synthetic News Detection Method Complying with Regulatory Compliance [J]. Computer Science, 2022, 49(6A): 523-530.
Full text



No Suggested Reading articles found!