基于行为关键语句特征的停车场异常行为识别方法

doi:10.11896/jsjkx.180901750

计算机科学 ›› 2019, Vol. 46 ›› Issue (10): 299-306.doi: 10.11896/jsjkx.180901750

基于行为关键语句特征的停车场异常行为识别方法

汪鸿年¹, 苏菡^1,2, 龙刚¹, 王雁飞¹, 尹宽¹

(四川师范大学计算机科学学院成都610101)¹
(可视化计算与虚拟现实四川省重点实验室成都610066)²

收稿日期:2018-09-16 修回日期:2018-12-17 出版日期:2019-10-15 发布日期:2019-10-21
通讯作者: 苏菡(1979-),女,博士,教授,主要研究方向为模式识别与图像处理,E-mail:jkxy_sh@sicnu.edu.cn。
作者简介:汪鸿年(1994-),男,硕士生,CCF会员,主要研究方向为时空数据挖掘;龙刚(1992-),男,硕士生,主要研究方向为图像处理;王雁飞(1990-),男,硕士生,主要研究方向为图像处理;尹宽(1995-),男,硕士生,主要研究方向为目标跟踪。
基金资助:
本文受国家自然科学基金(61403266,61403196),人社部留学回国人员科技活动择优项目重点项目,四川省可视化与虚拟现实重点实验室项目(KJ201419),成都大熊猫繁育研究基地项目(CPB2018-02)资助。

Parking Anomaly Behavior Recognition Method Based on Key Sentence of Behavior Sequence Features

WANG Hong-nian¹, SU Han^1,2, LONG Gang¹, WANG Yan-fei¹, YIN Kuan¹

(School of Computer Science,Sichuan Normal University,Chengdu 610101,China)¹
(Visual Computing and Virtual Reality Key Laboratory of Sichuan Province,Chengdu 610066,China)²

Received:2018-09-16 Revised:2018-12-17 Online:2019-10-15 Published:2019-10-21

摘要/Abstract

摘要： 随着技术的发展和摄像头的普及,人们对智能视频监控的需求越来越高,其中异常行为识别是智能监控系统的关键部分,对维护社会安全有着重要的作用。针对视频数据的时空特性,文中提出了将行为表示为具有时间序列性的关键语句的方法,并将这些关键语句称为行为关键语句。通过对行为关键语句的学习,实现了对停车场场景的异常行为识别。首先,对行为图像序列进行分割,提取前景目标并计算前景目标的运动周期曲线;然后,依据运动周期曲线采用动态时间规整(Dynamic Time Warping,DTW)的方法提取行为关键帧;最后,基于自然语言处理领域中的语义理解的方法,将行为关键帧表征为一系列行为关键语句进行识别。针对关键语句的时序性,采用擅长处理时序数据的长短时记忆神经网络(Long Short-Term Memory Network,LSTM)对行为关键语句进行分类。此外,为解决现有的数据不平衡问题,采用生成对抗网络(Generative Adversarial Networks,GAN)等方法扩充训练集,以增大样本空间,平衡不同类别数据量的差异。在中国科学院CASIA行为数据库和自建行为数据库上的验证结果表明,所提方法对异常行为的平均识别率达到了97%,相比于以前的方法有了明显的提升,证明了行为关键语句能更好地表征行为信息且LSTM模型更适用于学习时序数据背后的模式,因此该方法在停车场场景的异常行为识别任务上具有有效性。

关键词: 长短时记忆神经网络, 动态时间规整, 深度学习特征, 生成对抗网络, 异常行为识别

Abstract: With the development of technology and the popularity of cameras,people’s demands on intelligent video surveillance are increasing.Anomaly behavior recognition is a key part of intelligent monitoring systems and plays an important role in maintaining social security.Aiming at the spatio-temporal feature of video data,this paper proposed a method of characterizing behavior as a key sentence with time series,termed Key Sentence of Behavior Sequence (KSBS),and realized the anomaly behavior recognition in the parking scenes by learning key sentences of behaviors.Firstly,the motion sequence is segmented,the foreground target is extracted,and the Motion Period Curve (MPC) of the foreground target is calculated.Then,according to the motion cycle curve,the MPC and DTW method are used to extract the behavior key frames.Finally,based on the semantic understanding method in the field of natural language proces-sing,the behavior key frames are characterized as a series of behavior key sentence.In light of time series features of key sentences,LSTM,which is expert in dealing with time series data,is used to classify the key statements of behaviors.In order to solve the existing data imbalance problem,GAN is used to expand the training set,thus increasing the sample space and balancing the difference between different types of data.Validation results on CASIA behavior database and self-built behavior database show that the average recognition rate of the proposed method for anomaly behavior is 97%.It is proved that the Key Sentece of Behavior Sequence can better represent the behavior information and the LSTM model is more suitable for learning the patterns behind the time series data,verifying the effectiveness of the proposed method on anomaly behavior recognition in parking scenes.

Key words: Anomaly behavior recognition, Dynamic time warping, Features of deep learning, Generative adversarial networks, Long Short-term memory neural network

中图分类号:

TP391

汪鸿年, 苏菡, 龙刚, 王雁飞, 尹宽. 基于行为关键语句特征的停车场异常行为识别方法[J]. 计算机科学, 2019, 46(10): 299-306. https://doi.org/10.11896/jsjkx.180901750

WANG Hong-nian, SU Han, LONG Gang, WANG Yan-fei, YIN Kuan. Parking Anomaly Behavior Recognition Method Based on Key Sentence of Behavior Sequence Features[J]. Computer Science, 2019, 46(10): 299-306. https://doi.org/10.11896/jsjkx.180901750

参考文献

[1]FAN Z,LING S,JIN X,et al.From handcrafted to learned representations for human action recognition:A survey[J].Image and Vision Computing,2016,55(P2):42-52.
[2]ZOU J Y.Research on abnormal activity recognition in parking[D].Chengdu:Sichuan Normal University,2014.(in Chinese)
邹佳运.停车场异常行为识别方法研究[D].成都:四川师范大学,2014.
[3]ZIVKOVIC Z,VAN DER HEIJDEN F.Efficient adaptive density estimation per image pixel for the task of background subtraction[J].Pattern Recognition Letters,2006,27(7):773-780.
[4]KIM K,CHALIDABHONGSE T H,HARWOOD D,et al.Real-time foreground-background segmentation using codebook model[J].Real-time Imaging,2005,11(3):172-185.
[5]BARNICH O,VAN DROOGENBROECK M.ViBe:A universal background subtraction algorithm for video sequences[J].IEEE Transactions on Image processing,2011,20(6):1709-1724.
[6]ZHANG D X,DAI K R.Adaptive Target Extraction and Trac-king Method for Complex Image Sequences[J].Chinese Journal of Electronics,1994,22(10):46-53.(in Chinese)
张天序,戴可荣.复杂图象序列的自适应目标提取和跟踪方法[J].电子学报,1994,22(10):46-53.
[7]BOBICK A F,DAVIS J W.The recognition of human movement using temporal templates[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(3):257-267.
[8]WANG Y,HUANG K,TAN T.Human activity recognition based on r transform[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2007:1-8.
[9]CHEN H S,CHEN H T,CHEN Y W,et al.Human action reco-gnition using star skeleton[C]//Proceedings of the 4th ACM International Workshop on Video Surveillance and Sensor Networks.ACM,2006:171-178.
[10]SOUVENIR R,BABBS J.Learning the viewpoint manifold for action recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2008:1-7.
[11]GORELICK L,BLANK M,SHECHTMAN E,et al.Actions as space-time shapes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(12):2247-2253.
[12]ERFANI S M,RAJASEGARAR S,KARUNASEKERA S,et al.High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning[J].Pattern Recognition,2016,58(C):121-134.
[13]LIU C,XU W S,WU Q D.Spatiotemporal Convolutional Neural Networks and its Application in Action Recognition[J].Computer Science,2015,42(7):245-249.(in Chinese)
刘琮,许维胜,吴启迪.时空域深度卷积神经网络及其在行为识别上的应用[J].计算机科学,2015,42(7):245-249.
[14]TRAN D,BOURDEV L,FERGUS R,et al.Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015:4489-4497.
[15]TRAN D,WANG H,TORRESANI L,et al.A Closer Look at Spatiotemporal Convolutions for Action Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:6450-6459.
[16]SULTANI W,CHEN C,SHAH M.Real-world Anomaly Detection in Surveillance Videos[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:6479-6488.
[17]RAVANBAKHSH M,NABI M,SANGINETO E,et al.Abnormal event detection in videos using generative adversarial nets[C]//IEEE International Conference on Image Processing.IEEE,2017:1577-1581.
[18]KAR A,RAI N,SIKKA K,et al.Adascan:Adaptive scan pooling in deep convolutional neural networks for human action reco-gnition in videos[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017:3376-3385.
[19]GAO X.Research on abnormal behavior ofpedestrians in video surveillance [D].Chengdu:University of Electronic Science and Technology,2018.(in Chinese)
高翔.视频监控中行人异常行为分析研究[D].成都:电子科技大学,2018.
[20]WANG H N,SU H.STAR:A Concise Deep Learning Framework for Citywide Human Mobility Prediction [C]//IEEE International Conference on Mobile Data Management.IEEE,2019:304-309.
[21]KEOGH E J,PAZZANI M J.Derivative dynamic time warping[C]//Proceedings of the 2001 SIAM International Conference on Data Mining.Philadelphia:SIAM,2001:1-11.
[22]SU H,HUANG F G.A Method of Gait Recognition UsingSpatio-Temporal Analysis[J].Pattern Recognition & Artificial Intelligence,2007,20(2):281-286.(in Chinese)
苏菡,黄凤岗.一种基于时空分析的步态识别方法[J].模式识别与人工智能,2007,20(2):281-286.
[23]RATLIFF L J,BURDEN S A,SASTRY S S.Characterization and computation of localnash equilibria in continuous games[C]//Communication,Control,and Computing (Allerton).IEEE,2013:917-924.
[24]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial nets[C]//Advances in neural information processing systems.New York:Curran Associates,2014:2672-2680.
[25]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[26]ZHANG X R,JU X Z,SONG P,et al.Feature Fusion Based on DBN for Cross-Corpus Speech Emotion Recognition[J].Signal Processing,2017,33(5):649-660.(in Chinese)
张昕然,巨晓正,宋鹏,等.用于跨库语音情感识别的 DBN 特征融合方法[J].信号处理,2017,33(5):649-660.
[27]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.

相关文章 15

[1]	张佳, 董守斌. 基于评论方面级用户偏好迁移的跨领域推荐算法 Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer 计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131
[2]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[3]	戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[4]	尹文兵, 高戈, 曾邦, 王霄, 陈怡. 基于时频域生成对抗网络的语音增强算法 Speech Enhancement Based on Time-Frequency Domain GAN 计算机科学, 2022, 49(6): 187-192. https://doi.org/10.11896/jsjkx.210500114
[5]	徐辉, 康金梦, 张加万. 基于特征感知的数字壁画复原方法 Digital Mural Inpainting Method Based on Feature Perception 计算机科学, 2022, 49(6): 217-223. https://doi.org/10.11896/jsjkx.210500105
[6]	高志宇, 王天荆, 汪悦, 沈航, 白光伟. 基于生成对抗网络的5G网络流量预测方法 Traffic Prediction Method for 5G Network Based on Generative Adversarial Network 计算机科学, 2022, 49(4): 321-328. https://doi.org/10.11896/jsjkx.210300240
[7]	黎思泉, 万永菁, 蒋翠玲. 基于生成对抗网络去影像的多基频估计算法 Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal 计算机科学, 2022, 49(3): 179-184. https://doi.org/10.11896/jsjkx.201200081
[8]	石达, 芦天亮, 杜彦辉, 张建岭, 暴雨轩. 基于改进CycleGAN的人脸性别伪造图像生成模型 Generation Model of Gender-forged Face Image Based on Improved CycleGAN 计算机科学, 2022, 49(2): 31-39. https://doi.org/10.11896/jsjkx.210600012
[9]	唐雨潇, 王斌君. 基于深度生成模型的人脸编辑研究进展 Research Progress of Face Editing Based on Deep Generative Model 计算机科学, 2022, 49(2): 51-61. https://doi.org/10.11896/jsjkx.210400108
[10]	李建, 郭延明, 于天元, 武与伦, 王翔汉, 老松杨. 基于生成对抗网络的多目标类别对抗样本生成算法 Multi-target Category Adversarial Example Generating Algorithm Based on GAN 计算机科学, 2022, 49(2): 83-91. https://doi.org/10.11896/jsjkx.210800130
[11]	谈馨悦, 何小海, 王正勇, 罗晓东, 卿粼波. 基于Transformer交叉注意力的文本生成图像技术 Text-to-Image Generation Technology Based on Transformer Cross Attention 计算机科学, 2022, 49(2): 107-115. https://doi.org/10.11896/jsjkx.210600085
[12]	陈贵强, 何军. 自然场景下遥感图像超分辨率重建算法研究 Study on Super-resolution Reconstruction Algorithm of Remote Sensing Images in Natural Scene 计算机科学, 2022, 49(2): 116-122. https://doi.org/10.11896/jsjkx.210700095
[13]	蒋宗礼, 樊珂, 张津丽. 基于生成对抗网络和元路径的异质网络表示学习 Generative Adversarial Network and Meta-path Based Heterogeneous Network Representation Learning 计算机科学, 2022, 49(1): 133-139. https://doi.org/10.11896/jsjkx.201000179
[14]	张玮琪, 汤轶丰, 李林燕, 胡伏原. 基于场景图的段落生成序列图像方法 Image Stream From Paragraph Method Based on Scene Graph 计算机科学, 2022, 49(1): 233-240. https://doi.org/10.11896/jsjkx.201100207
[15]	徐涛, 田崇阳, 刘才华. 基于深度学习的人群异常行为检测综述 Deep Learning for Abnormal Crowd Behavior Detection:A Review 计算机科学, 2021, 48(9): 125-134. https://doi.org/10.11896/jsjkx.201100015

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于行为关键语句特征的停车场异常行为识别方法

Parking Anomaly Behavior Recognition Method Based on Key Sentence of Behavior Sequence Features

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0