计算机科学 ›› 2022, Vol. 49 ›› Issue (2): 142-148.doi: 10.11896/jsjkx.210900266

• 计算机视觉:理论与应用 • 上一篇    下一篇

基于隐式视角转换的视频异常检测

冷佳旭1,2, 谭明圮1,3, 胡波1, 高新波1   

  1. 1 重庆邮电大学图像认知重庆市重点实验室 重庆400065
    2 南京理工大学江苏省社会安全图像与视频理解重点实验室 南京210094
    3 重庆邮电大学光电工程学院 重庆400065
  • 收稿日期:2021-09-29 修回日期:2021-11-08 出版日期:2022-02-15 发布日期:2022-02-23
  • 通讯作者: 高新波(gaoxb@cqupt.edu.cn)
  • 作者简介:lengjx@cqupt.edu.cn
  • 基金资助:
    国家自然科学基金(62036007,62050175,62102057);重庆市教委科学技术研究项目(KJQN-202100627)

Video Anomaly Detection Based on Implicit View Transformation

LENG Jia-xu1,2, TAN Ming-pi1,3, HU Bo1, GAO Xin-bo1   

  1. 1 Key Laboratory of Image Cognition,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 Jiangsu Key Laboratory of Image and Video Understanding for Social Safety,Nanjing University of Science and Technology,Nanjing 210094,China
    3 School of Optoelectronic Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
  • Received:2021-09-29 Revised:2021-11-08 Online:2022-02-15 Published:2022-02-23
  • About author:LENG Jia-xu,born in 1989,Ph.D,lecture,is a member of China Computer Federation.His main research interests include computer vision and object detection.
    GAO Xin-bo,born in 1972,Ph.D,professor,is one of the talent bank of 10 000 advisor for innovation and entrepreneurship,is a board member of China Computer Federation.His main research interests include artificial intelligence and pattern recognition.
  • Supported by:
    National Natural Science Foundation of China(62036007,62050175,62102057) and Science and Technology Research Program of Chongqing Municipal Education Commission(KJQN-202100627).

摘要: 目前,基于深度学习的视频异常检测方法都是在单一视角下对视频片段中的异常行为或异常事物进行检测,忽视了视角信息在视频异常检测中的重要性。在单一视角下,当异常事物被遮挡或异常行为不明显时,现有算法的性能将难以得到保证。为此,文中首次将视角转换的概念引入到视频异常检测中,通过级联网络结构在多视角下进行异常判断来提升模型的鲁棒性。针对受限于数据集没有多视角的监督信息,难以实现真正的显式的视角转换问题,提出了一种基于隐式视角转换的视频异常检测方法.对初步检测结果为正常的目标帧,利用其与特定帧的光流信息,通过光流映射实现目标帧到特定帧视角的隐式视角转换,并对视角转换后的目标帧进行二次异常检测。通过多个视角来判定目标帧是否异常,为视频异常检测提供了一种新的思路。实验结果表明,所提方法对异常数据的反应更灵敏,具有更鲁棒的正常数据拟合能力,在UCSD Ped2和CUHK Avenue数据集上的AUC值分别达到了97.0%和88.9%。

关键词: 多视角检测, 光流映射, 深度学习, 视频异常检测, 隐式视角转换

Abstract: Existing deep learning-based video anomaly detection methods all detect anomalies in video clips under a single view,ignoring the importance of view information in video anomaly detection.Under a single view,when anomalies are occluded or not obvious,the performance of existing algorithms will suffer drops.To avoid this problem,the author firstly introduces the concept of view transformation into video anomaly detection,which improves the robustness of the model by judging abnormalities from multiple views.However,due to the lack of multi-view supervision information in the dataset,it is difficult to achieve explicit view transformation.Specifically,in order to reflect the idea of view transformation,the author proposes a video anomaly detection method based on implicit view transformation,using the optical flow information between frames to warp the implicit view information of the previous frame to the target frame,so as to realize the implicit view transformation from the target frame to the previous frame.And then,the method performs secondary anomaly detection on the target frame after view transformation.Experimental results show that the proposed method responds more sensitively to abnormal data and has a more robust normal data fitting ability.The AUC values on the UCSD Ped2 and CUHK Avenue datasets reached 97.0% and 88.9%,respectively.

Key words: Deep learning, Implicit view transformation, Multi-view detection, Optical flow warp, Video anomaly detection

中图分类号: 

  • TP391.41
[1]PENG J L,ZHAO Y L,WANG L M.An Overview of VideoAnomaly Behavior Detection Based on Deep Learning[J].Laser &Optoelectronics Progress,2020,58(6):1-17.
[2]KINGMA D P,WELLING M.Auto-encoding variational bayes[C]//Proceedings of the International Conference on Learning Representations.Banff,Canada,2014.
[3]HASAN M,CHOI J,NEUMANN J,et al.Learning TemporalRegularity in Video Sequences[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,USA,2016:733-742.
[4]LIU W,LUO W,LIAN D,et al.Future frame prediction foranomaly detection-a new baseline[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Utah,USA,2018:6536-6545.
[5]YE M C,PENG X J,GAN W H,et al.Anopcn:Video anomalydetection via deep predictive coding network[C]//Proceedings of the ACM International Conference on Multimedia.Nice,France,2019:1805-1813.
[6]GONG D,LIU L,LE V,et al.Memorizing normality to detectanomaly:Memory-augmented deep autoencoder for unsupervised anomaly detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Seoul,Korea,2019:1705-1714.
[7]PARK H,NOH J,HAM B.Learning memory-guided normality for anomaly detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Seattle,USA,2020:14372-14381.
[8]CAI R C,ZHANG H,LIU W,et al.Appearance-Motion Memory Consistency Network for Video Anomaly Detection[J].AAAI Conference on Artificial Intelligence,2021,35(2):938-946.
[9]ZHU Z,WU W,ZOU W,et al.End-to-End Flow CorrelationTracking with Spatial-Temporal Attention[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Utah,USA,2018:548-557.
[10]ZHAO Y R,DENG B,SHEN C,et al.Spatio-Temporal AutoEncoder for Video Anomaly Detection[C]//Proceedings of the ACM International Conference on Multimedia.New York,USA,2017:1933-1941.
[11]LUO W X,LIU W,GAO S H.Remembering history with con-volutional LSTM for anomaly detection[C]//Proceedings of the IEEE International Conference on Multimedia and Expo.Hong Kong,China,2017:439-444.
[12]WANG X Z,CHE Z P,YANG K,et al.Robust UnsupervisedVideo Anomaly Detection by Multi-Path Frame Prediction[J].IEEE Transactions on Neural Networks and Learning Systems,2021.
[13]LIU Z A,NIE Y W,LONG C J,et al.A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction[C]//Proceedings of the IEEE International Conference on Computer Vision.Montreal,Canada,2021:13588-13597.
[14]SALIGRAMA V,KONRAD J,JODOIN P.Video AnomalyIdentification[J].IEEE Signal Processing Magazine,2010,27(5):18-33.
[15]LEYVA R,SANCHEZ V,LI C T.Video Anomaly Detectionwith Compact Feature Sets for Online Performance[J].IEEE Transactions on Image Processing,2017,26(7):3463-3478.
[16]LU C,SHI J,JIA J.Abnormal Event Detection at 150 FPS in MATLAB[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.Sydney,Australia,2013:2720-2727.
[17]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention.Cham:Springer,2015:234-241.
[18]ZHU X Z,XIONG Y W,DAI J F,et al.Deep Feature Flow for Video Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Hawaii,USA,2017:4141-4150.
[19]LIU L,ZHANG J N,HE R F,et al.Learning by Analogy:Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Seattle,USA,2020:6488-6497.
[20]ILG E,MAYER N,SAIKIA T,et al.FlowNet 2.0:Evolution of Optical Flow Estimation with Deep Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Hawaii,USA,2017:1647-1655.
[21]FISCHER P,DOSOVITSKIY A,ILG E,et al.FlowNet:Lear-ning Optical Flow with Convolutional Networks[C]//Procee-dings of the IEEE International Conference on Computer Vision.Santiago,Chile,2016:2758-2766.
[22]LI W X,MAHADEVAN V,VASCONCELOS N.Anomaly de-tection and localization in crowded scenes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(1);18-32.
[23]PASZKE A,GROSS S,MASSA F,et al.PyTorch:An imperative style,high-performance deep learning library[C]//Procee-dings of the Annual Conference on Neural Information Proces-sing Systems.Vancouver,Canada,2019:8026-8037.
[24]KINGMA D P,BA J.Adam:A method for stochastic optimization[C]//Proceedings of the International Conference on Lear-ning Representations.San Diego,USA,2015:1-15.
[25]LUO W X,LIU W,GAO S H.A revisit of sparse coding based anomaly detection in stacked RNN framework[C]//Proceedings of the IEEE International Conference on Computer Vision.Ve-nice,Italy,2017:341-349.
[26]IONESCU R T,SMEUREANU S,ALEXE B,et al.Unmasking the abnormal events in video[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy,2017:2914-2922.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[9] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[10] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[11] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[12] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[13] 祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋.
改进Faster R-CNN的光学遥感飞机目标检测
Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN
计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
[14] 王建明, 陈响育, 杨自忠, 史晨阳, 张宇航, 钱正坤.
不同数据增强方法对模型识别精度的影响
Influence of Different Data Augmentation Methods on Model Recognition Accuracy
计算机科学, 2022, 49(6A): 418-423. https://doi.org/10.11896/jsjkx.210700210
[15] 毛典辉, 黄晖煜, 赵爽.
符合监管合规性的自动合成新闻检测方法研究
Study on Automatic Synthetic News Detection Method Complying with Regulatory Compliance
计算机科学, 2022, 49(6A): 523-530. https://doi.org/10.11896/jsjkx.210300083
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!