Computer Science ›› 2021, Vol. 48 ›› Issue (7): 206-212.doi: 10.11896/jsjkx.200900093

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Temporal Modeling for Online Anomaly Detection

QING Lai-yun1, ZHANG Jian-gong1, MIAO Jun2   

  1. 1 School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 100049,China
    2 Beijing Key Laboratory Internet Culture Digital Dissemination Research,Beijing Information Science & Technology University,Beijing 100101,China
  • Received:2020-09-13 Revised:2020-10-25 Online:2021-07-15 Published:2021-07-02
  • About author:QING Lai-yun,born in 1974,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.Her main research interests include multimedia,computer vision and machine learning.
  • Supported by:
    NSFC (61872333Y),Research Fund from Beijing Innovation Center for Future Chips (KYJJ2018004),Beijing Municipal Education Commission Project (KM201911232003) and Beijing Natural Science Foundation (4202025).

Abstract: Weakly supervised anomaly detection (WSAD) is a challenging task in that there is only normal and anomaly video label supervision but it is required to localize intervals where anomalies take place.We employ multiple instance learning (MIL) network for weakly supervised anomaly detection,which regards the input video as a bag and the segments chunked from the vi-deo as instances in it.We train the instance classifier with only label of video level (bag level),while the label of instance level is unknown.As there is strong temporal information in videos,we focus on temporal relationship for online anomaly detection in surveillance videos.We consider both global and local perspective and use self-attention module to learn each instance weight.We get the linear weighted sum of self-attention score and instance anomaly score,which represents video level anomaly score.Then the mean square error loss is employed to train the self-attention module.Online constraints allow us to use historical and current video clips only,without future frames.In order to model the temporal structure of video,we introduce LSTM and temporal con-volutional network (TCN) into WSAD problem.We explore single rate dilated temporal convolutional network,and pyramid dilated temporal convolutional network (PDTCN) which fuses multi-scale feature with different rates.Experiments show that the AUC of PDTCN with complementary inner and outer bag loss is higher than that of the baseline method without temporal mode-ling by 3.2% on UCF-Crime dataset.

Key words: Anomaly detection, Attention module, Multiple instance learning, Temporal convolutional network, Weakly-supervised learning

CLC Number: 

  • TP391
[1]BAI S,KOLTER J Z,KOLTUN V.An empirical evaluation of generic convolutional and recurrent networks for sequence mo-deling[J].arXiv:1803.01271,2018.
[2]SULTANI W,CHEN C,SHAH M.Real-world anomaly detection in surveillance videos[J].arXiv:1801.04264,2018.
[3]BILEN H,VEDALDI A.Weakly supervised deep detection networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2846-2854.
[4]TANG P,WANG X,BAI X,et al.Multiple instance detectionnetwork with online instance classifier refinement[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2843-2851.
[5]LI D,HUANG J,LI Y,et al.Weakly supervised object localization with progressive domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:3512-3520.
[6]ZHANG Y,BAI Y,DING M,et al.W2f:A weakly-supervised to fully-supervised framework for object detection[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:928-936.
[7]NGUYEN P,HAN B,LIU T,et al.Weakly supervised action localization by sparse temporal pooling network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018.
[8]ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning deepfeatures for discriminative localization[C]//2016 IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).2016
[9]PAUL S,ROY S,ROY-CHOWDHURY A K.W-talc:Weakly-supervised temporal activity localization and classification[C]//Proceedings of the European Conference on Computer Vision.2018:563-579.
[10]LEE P,UH Y,BYUN H.Background suppression network for weakly-supervised temporal action localization[J].arXiv:1911.09963.2019.
[11]HASAN M,CHOI J,NEUMANN J,et al.Learning temporal regularity in video sequences[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:733-742.
[12]LU C,SHI J,JIA J.Abnormal event detection at 150 fps in matlab[C]//Proceedings of the IEEE International Conference on Computer Vision.2013:2720-2727.
[13]ZHAO Y,DENG B,SHEN C,et al.Spatio-temporal autoencoder for video anomaly detection [C]//Proceedings of the 2017 ACM on Multimedia Conference.ACM,2017:1933-1941.
[14]LIU W,LUO W,LIAN D,et al.Future frame prediction foranomaly detection-a new baseline [J].arXiv:1712.09867,2017.
[15]DOSOVITSKIY A,FISCHER P,ILG E,et al.Flownet:Lear-ning optical flow with convolutional networks[C]//2015 IEEE International Conference on Computer Vision (ICCV).2015.
[16]IONESCU R T,KHAN F S,GEORGESCU M I,et al.Object-centric auto-encoders and dummy anomalies for abnormal event detection in video[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2019.
[17]GONG D,LIU L,LE V,et al.Memorizing normality to detectanomaly:Memory-augmented deep autoencoder for unsupervised anomaly detection[C]//2019 IEEE/CVF International Confe-rence on Computer Vision (ICCV).2019.
[18]ZHANG J G,QING L Y,MIAO J.Temporal convolutional network with complementary inner bag loss for weakly supervised anomaly detection[C]//Proceedings of IEEE International Conference on Image Processing.2019:4030-4034.
[19]TRAN D,BOURDEV L,FERGUS R,et al.Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:4489-4497.
[20]ZHU Y,NEWSAM S.Motion-aware feature for improved video anomaly detection[J].arXiv:1907.10211,2019.
[21]WANG W,PENG X,QIAO Y,et al.A comprehensive study on temporal modeling for online action detection[J].arXiv:2001.07501,2020.
[22]OORD A V D,DIELEMAN S,ZEN H,et al.Wavenet:A gene-rative model for raw audio[J].arXiv:1609.03499,2016.
[23]LI J,ZHANG S,WANG J,et al.Global-local temporal representations for video person reidentification[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).2019.
[24]DUCHI J,HAZAN E,SINGER Y.Adaptive subgradient me-thods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12(7):2121-2159.
[1] XU Tian-hui, GUO Qiang, ZHANG Cai-ming. Time Series Data Anomaly Detection Based on Total Variation Ratio Separation Distance [J]. Computer Science, 2022, 49(9): 101-110.
[2] WANG Xin-tong, WANG Xuan, SUN Zhi-xin. Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network [J]. Computer Science, 2022, 49(8): 314-322.
[3] WEI Kai-xuan, FU Ying. Re-parameterized Multi-scale Fusion Network for Efficient Extreme Low-light Raw Denoising [J]. Computer Science, 2022, 49(8): 120-126.
[4] LIU Dong-mei, XU Yang, WU Ze-bin, LIU Qian, SONG Bin, WEI Zhi-hui. Incremental Object Detection Method Based on Border Distance Measurement [J]. Computer Science, 2022, 49(8): 136-142.
[5] DU Hang-yuan, LI Duo, WANG Wen-jian. Method for Abnormal Users Detection Oriented to E-commerce Network [J]. Computer Science, 2022, 49(7): 170-178.
[6] WU Lin, SUN Jing-yu. Multi-branch RA Capsule Network and Its Application in Image Classification [J]. Computer Science, 2022, 49(6): 224-230.
[7] SHEN Shao-peng, MA Hong-jiang, ZHANG Zhi-heng, ZHOU Xiang-bing, ZHU Chun-man, WEN Zuo-cheng. Three-way Drift Detection for State Transition Pattern on Multivariate Time Series [J]. Computer Science, 2022, 49(4): 144-151.
[8] XU Hua-jie, QIN Yuan-zhuo, YANG Yang. Scene Recognition Method Based on Multi-level Feature Fusion and Attention Module [J]. Computer Science, 2022, 49(4): 209-214.
[9] WU Yu-kun, LI Wei, NI Min-ya, XU Zhi-cheng. Anomaly Detection Model Based on One-class Support Vector Machine Fused Deep Auto-encoder [J]. Computer Science, 2022, 49(3): 144-151.
[10] ZHAO Yue, YU Zhi-bin, LI Yong-chun. Cross-attention Guided Siamese Network Object Tracking Algorithm [J]. Computer Science, 2022, 49(3): 163-169.
[11] LENG Jia-xu, TAN Ming-pi, HU Bo, GAO Xin-bo. Video Anomaly Detection Based on Implicit View Transformation [J]. Computer Science, 2022, 49(2): 142-148.
[12] ZHANG Ye, LI Zhi-hua, WANG Chang-jie. Kernel Density Estimation-based Lightweight IoT Anomaly Traffic Detection Method [J]. Computer Science, 2021, 48(9): 337-344.
[13] ZHOU Wen-hui, SHI Min, ZHU Deng-ming, ZHOU Jun. Seismic Data Super-resolution Method Based on Residual Attention Network [J]. Computer Science, 2021, 48(8): 24-31.
[14] GUO Yi-shan, LIU Man-dan. Anomaly Detection Based on Spatial-temporal Trajectory Data [J]. Computer Science, 2021, 48(6A): 213-219.
[15] XING Hong-jie, HAO ZhongHebei. Novelty Detection Method Based on Global and Local Discriminative Adversarial Autoencoder [J]. Computer Science, 2021, 48(6): 202-209.
Full text



No Suggested Reading articles found!