Computer Science ›› 2019, Vol. 46 ›› Issue (10): 299-306.doi: 10.11896/jsjkx.180901750

• Graphics,Image & Pattern Recognition • Previous Articles     Next Articles

Parking Anomaly Behavior Recognition Method Based on Key Sentence of Behavior Sequence Features

WANG Hong-nian1, SU Han1,2, LONG Gang1, WANG Yan-fei1, YIN Kuan1   

  1. (School of Computer Science,Sichuan Normal University,Chengdu 610101,China)1
    (Visual Computing and Virtual Reality Key Laboratory of Sichuan Province,Chengdu 610066,China)2
  • Received:2018-09-16 Revised:2018-12-17 Online:2019-10-15 Published:2019-10-21

Abstract: With the development of technology and the popularity of cameras,people’s demands on intelligent video surveillance are increasing.Anomaly behavior recognition is a key part of intelligent monitoring systems and plays an important role in maintaining social security.Aiming at the spatio-temporal feature of video data,this paper proposed a method of characterizing behavior as a key sentence with time series,termed Key Sentence of Behavior Sequence (KSBS),and realized the anomaly behavior recognition in the parking scenes by learning key sentences of behaviors.Firstly,the motion sequence is segmented,the foreground target is extracted,and the Motion Period Curve (MPC) of the foreground target is calculated.Then,according to the motion cycle curve,the MPC and DTW method are used to extract the behavior key frames.Finally,based on the semantic understanding method in the field of natural language proces-sing,the behavior key frames are characterized as a series of behavior key sentence.In light of time series features of key sentences,LSTM,which is expert in dealing with time series data,is used to classify the key statements of behaviors.In order to solve the existing data imbalance problem,GAN is used to expand the training set,thus increasing the sample space and balancing the difference between different types of data.Validation results on CASIA behavior database and self-built behavior database show that the average recognition rate of the proposed method for anomaly behavior is 97%.It is proved that the Key Sentece of Behavior Sequence can better represent the behavior information and the LSTM model is more suitable for learning the patterns behind the time series data,verifying the effectiveness of the proposed method on anomaly behavior recognition in parking scenes.

Key words: Anomaly behavior recognition, Features of deep learning, Dynamic time warping, Generative adversarial networks, Long Short-term memory neural network

CLC Number: 

  • TP391
[1]FAN Z,LING S,JIN X,et al.From handcrafted to learned representations for human action recognition:A survey[J].Image and Vision Computing,2016,55(P2):42-52.
[2]ZOU J Y.Research on abnormal activity recognition in parking[D].Chengdu:Sichuan Normal University,2014.(in Chinese)
[3]ZIVKOVIC Z,VAN DER HEIJDEN F.Efficient adaptive density estimation per image pixel for the task of background subtraction[J].Pattern Recognition Letters,2006,27(7):773-780.
[4]KIM K,CHALIDABHONGSE T H,HARWOOD D,et al.Real-time foreground-background segmentation using codebook model[J].Real-time Imaging,2005,11(3):172-185.
[5]BARNICH O,VAN DROOGENBROECK M.ViBe:A universal background subtraction algorithm for video sequences[J].IEEE Transactions on Image processing,2011,20(6):1709-1724.
[6]ZHANG D X,DAI K R.Adaptive Target Extraction and Trac-king Method for Complex Image Sequences[J].Chinese Journal of Electronics,1994,22(10):46-53.(in Chinese)
[7]BOBICK A F,DAVIS J W.The recognition of human movement using temporal templates[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(3):257-267.
[8]WANG Y,HUANG K,TAN T.Human activity recognition based on r transform[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2007:1-8.
[9]CHEN H S,CHEN H T,CHEN Y W,et al.Human action reco-gnition using star skeleton[C]//Proceedings of the 4th ACM International Workshop on Video Surveillance and Sensor Networks.ACM,2006:171-178.
[10]SOUVENIR R,BABBS J.Learning the viewpoint manifold for action recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2008:1-7.
[11]GORELICK L,BLANK M,SHECHTMAN E,et al.Actions as space-time shapes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(12):2247-2253.
[12]ERFANI S M,RAJASEGARAR S,KARUNASEKERA S,et al.High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning[J].Pattern Recognition,2016,58(C):121-134.
[13]LIU C,XU W S,WU Q D.Spatiotemporal Convolutional Neural Networks and its Application in Action Recognition[J].Computer Science,2015,42(7):245-249.(in Chinese)
[14]TRAN D,BOURDEV L,FERGUS R,et al.Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015:4489-4497.
[15]TRAN D,WANG H,TORRESANI L,et al.A Closer Look at Spatiotemporal Convolutions for Action Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:6450-6459.
[16]SULTANI W,CHEN C,SHAH M.Real-world Anomaly Detection in Surveillance Videos[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:6479-6488.
[17]RAVANBAKHSH M,NABI M,SANGINETO E,et al.Abnormal event detection in videos using generative adversarial nets[C]//IEEE International Conference on Image Processing.IEEE,2017:1577-1581.
[18]KAR A,RAI N,SIKKA K,et al.Adascan:Adaptive scan pooling in deep convolutional neural networks for human action reco-gnition in videos[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017:3376-3385.
[19]GAO X.Research on abnormal behavior ofpedestrians in video surveillance [D].Chengdu:University of Electronic Science and Technology,2018.(in Chinese)
[20]WANG H N,SU H.STAR:A Concise Deep Learning Framework for Citywide Human Mobility Prediction [C]//IEEE International Conference on Mobile Data Management.IEEE,2019:304-309.
[21]KEOGH E J,PAZZANI M J.Derivative dynamic time warping[C]//Proceedings of the 2001 SIAM International Conference on Data Mining.Philadelphia:SIAM,2001:1-11.
[22]SU H,HUANG F G.A Method of Gait Recognition UsingSpatio-Temporal Analysis[J].Pattern Recognition & Artificial Intelligence,2007,20(2):281-286.(in Chinese)
[23]RATLIFF L J,BURDEN S A,SASTRY S S.Characterization and computation of localnash equilibria in continuous games[C]//Communication,Control,and Computing (Allerton).IEEE,2013:917-924.
[24]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial nets[C]//Advances in neural information processing systems.New York:Curran Associates,2014:2672-2680.
[25]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[26]ZHANG X R,JU X Z,SONG P,et al.Feature Fusion Based on DBN for Cross-Corpus Speech Emotion Recognition[J].Signal Processing,2017,33(5):649-660.(in Chinese)
张昕然,巨晓正,宋鹏,等.用于跨库语音情感识别的 DBN 特征融合方法[J].信号处理,2017,33(5):649-660.
[27]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[1] ZHANG Yang, MA Xiao-hu. Anime Character Portrait Generation Algorithm Based on Improved Generative Adversarial Networks [J]. Computer Science, 2021, 48(1): 182-189.
[2] MENG Li-sha, REN Kun, FAN Chun-qi, HUANG Long. Dense Convolution Generative Adversarial Networks Based Image Inpainting [J]. Computer Science, 2020, 47(8): 202-207.
[3] LI Ze-wen, LI Zi-ming, FEI Tian-lu, WANG Rui-lin and XIE Zai-peng. Face Image Restoration Based on Residual Generative Adversarial Network [J]. Computer Science, 2020, 47(6A): 230-236.
[4] ZHENG Zhe, HU Qing-hao, LIU Qing-shan, LENG Cong. Quantizing Weights and Activations in Generative Adversarial Networks [J]. Computer Science, 2020, 47(5): 144-148.
[5] TANG Hao-feng, DONG Yuan-fang, ZHANG Yi-tong, SUN Juan-juan. Survey of Image Inpainting Algorithms Based on Deep Learning [J]. Computer Science, 2020, 47(11A): 151-164.
[6] LIU Hai-bo,WU Tian-bo,SHEN Jing,SHI Chang-ting. Advanced Persistent Threat Detection Based on Generative Adversarial Networks and Long Short-term Memory [J]. Computer Science, 2020, 47(1): 281-286.
[7] LIU Jian, JIN Ze-qun. Facial Expression Transfer Method Based on Deep Learning [J]. Computer Science, 2019, 46(6A): 250-253.
[8] CHEN Jian-ping, ZOU Feng, LIU Quan, WU Hong-jie, HU Fu-yuan, FU Qi-ming. Reinforcement Learning Algorithm Based on Generative Adversarial Networks [J]. Computer Science, 2019, 46(10): 265-272.
[9] FENG Yu-bo,DING Cheng-jun,GAO Xue,ZHU Xue-hong,LIU Qiang. Time Series Similarity Based on Moving Average and Piecewise Linear Regression [J]. Computer Science, 2018, 45(6A): 110-113.
[10] SUN Quan, ZENG Xiao-qin. Image Inpainting Based on Generative Adversarial Networks [J]. Computer Science, 2018, 45(12): 229-234.
[11] XU Jian-feng, HE Yu-fan, ZHANG Yuan-jian and TANG Tao. Similarity Algorithm Based on Three Way Decision of Time Warping Distance [J]. Computer Science, 2017, 44(9): 40-44.
[12] LIANG Wen-le, HUANG Yuan-yuan and HU Zuo-jin. Real-time Dynamic Sign Language Recognition Based on Hierarchical Matching Strategy [J]. Computer Science, 2017, 44(7): 299-303.
[13] LUO Zheng-ping, LIU Yan-jun and YANG Tian-qi. Gait Recognition Based on Decomposition of Optical Flow Components [J]. Computer Science, 2016, 43(9): 295-300.
[14] LI Hai-lin and YANG Li-bin. Similarity Measure for Time Series Based on Incremental Dynamic Time Warping [J]. Computer Science, 2013, 40(4): 227-230.
[15] . New Leaning Method for Optimal Warping Window of DTW [J]. Computer Science, 2012, 39(8): 191-195.
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .