Computer Science ›› 2023, Vol. 50 ›› Issue (11A): 230300022-8.doi: 10.11896/jsjkx.230300022

• Big Data & Data Science • Previous Articles     Next Articles

Spatial-Temporal Attention Mechanism Based Anomaly Detection for Multivariate Times Series

LIANG Lifang1, GUAN Donghai1, ZHANG Ji2, YUAN Weiwei1   

  1. 1 School of Computer Science, Technology, Nanjing University of Aeronautics, Astronautics, Nanjing 211106, China
    2 University of Southern Queensland,Toowoomba,Queensland 4350,Australia
  • Published:2023-11-09
  • About author:LIANG Lifang,born in 1999,postgra-duate.Her main research interests include anomaly detection for time series and so on.
    GUAN Donghai,born in 1981,Ph.D,associate professor,postgraduate supervisor.His main research interests include data mining,knowledge inference and so on.
  • Supported by:
    National Defense Basic Research Program(JCKY2020204C009).

Abstract: Internet of Things systems are widely used in a variety of infrastructure,involving many interconnected sensors that generate large amounts of multivariate time series data.Since the Internet of Things systems are vulnerable to network attacks,multivariate time series anomaly detection methods are used to timely monitor anomalies occurring in the system,which is crucial for securing the system.However,due to the complex relationships of high-dimensional sensor data,most existing anomaly detection methods have difficulty in learning the correlation of multivariate time series explicitly,resulting in low accuracy of anomaly detection.Therefore,a multivariate time series anomaly detection method(STA) based on spatial and temporal attention mechanism is proposed.STA first learns the relationship between sensors in the form of a graph structure and then uses a multi-hop graph attention network to assign different attention weights to the multi-hop neighbor nodes of each sensor node in the graph for capturing the spatial correlation of the sequence.Secondly,STA use a temporal attention mechanism-based long short-term me-mory network to adaptively select the corresponding time sequences to study the temporal correlation of sequences.Experimental results on four real-world sensor datasets show that STA can detect anomalies in time series more accurately than the baseline approach,with its F1 score outperforms the optimal baseline by 31.03%,14.29%,15.91% and 21.74%,respectively.In addition,ablation experiments and sensitivity analysis validate the effectiveness of the key components in the model.In general,STA can effectively capture the spatial and temporal correlations in multivariate time series and improve the anomaly detection performance of the model.

Key words: Multivariate times series, Attention mechanism, Graph attention network, Long short-term memory network, Temporal correlation, Spatial correlation, Anomaly detection

CLC Number: 

  • TP391
[1]MATHUR A P.TIPPENHAUER N O.SWaT:a water treatment testbed for research and training on ICS security[C]//2016 International Workshop on Cyber-physical Systems for Smart Water Networks(CySWater).IEEE,2016:31-36.
[2]ZHANG C,SONG D,CHEN Y,et al.A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:1409-1416.
[3]LIN S,CLARK R,BIRKE R,et al.Anomaly detection for time series using vae-lstm hybrid model[C]//2020 IEEE InternationalConference on Acoustics,Speech and Signal Processing(ICASSP 2020).IEEE,2020:4322-4326.
[4]SCHLICHTKRULL M,KIPF T N,BLOEM P,et al.Modelingrelational data with graph convolutional networks[C]//The Semantic Web:15th International Conference(ESWC 2018).Herak-lion,Crete,Greece,Springer International Publishing,2018:593-607.
[5]VELIČKOVIČ P,CUCURULL G,CASANOVA A,et al.Graph attention networks[J].arXiv:1710.10903,2017.
[6]ZHANG Z,LI M,LIN X,et al.Multistep speed prediction on traffic networks:A deep learning approach considering spatio-temporaldependencies[J].Transportation Research Part C:Emerging Technologies,2019,105:297-322.
[7]ZHENG C,FAN X,WANG C,et al.GMAN:A Graph Multi-Attention Network for Traffic Prediction[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:1234-1241.
[8]DENG A,HOOI B.Graph neural network-based anomaly detection in multivariate time series[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence.2021.
[9]WANG X,HE X,CAOY,et al.Kgat:Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2019:950-958.
[10]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural computation,1997,9(8):1735-1780.
[11]PANG G,SHEN C,CAO L,et al.Deep learning for anomaly detection:A review[J].ACM Computing Surveys(CSUR),2021,54(2):1-38.
[12]HAWKINS S,HE H,WILLIAMS G,et al.Outlier detectionusing replicator neural networks[C]//Data Warehousing and Knowledge Discovery.Springer Berlin Heidelberg,2002:170-180.
[13]GARG A,ZHANG W,SAMARANJ,et al.An evaluation of anomaly detection and diagnosis in multivariate time series[J].IEEE Transactions on Neural Networks and Learning Systems,2021,33(6):2508-2517.
[14]MALHOTRA P,RAMAKRISHNAN A,ANANDG,et al.LSTM-based encoder-decoder for multi-sensor anomaly detection[J].arXiv:1607.00148,2016.
[15]SHYU M L,CHEN S C,SARINNAPAKORN K,et al.A novel anomaly detection scheme based on principal component classifier[R].Miami Univ Coral Gables Fl Dept of Electrical and Computer Engineering,2003.
[16]SU Y,ZHAO Y,NIU C,et al.Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2019:2828-2837.
[17]KIM S,CHOI K,CHOI H S,et al.Towards a rigorous evaluation of time-series anomaly detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:7194-7201.
[18]HUNDMAN K,CONSTANTINOU V,LAPORTE C,et al.De-tecting spacecraft anomalies using lstms and nonparametric dynamic thresholding[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mi-ning.2018:387-395.
[19]BAI S,KOLTER J Z,KOLTUNV.An empirical evaluation ofgeneric convolutional and recurrent networks for sequence mo-deling[J].arXiv:1803.01271,2018.
[20]PARK D,HOSHI Y,KEMP C C.A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder[J].IEEE Robotics and Automation Letters,2018,3(3):1544-1551.
[21]SU Y,ZHAO Y,NIU C,et al.Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2019:2828-2837.
[22]AHMED C M,PALLETI V R,MATHUR A P.WADI:a water distribution testbed for research in the design of secure cyber physical systems[C]//Proceedings of the 3rd International Workshop on Cyber-physical Systems for Smart Water Networks.2017:25-28.
[23]BARTYŚ M,PATTON R,SYFERT M,et al.Introduction to the DAMADICS actuator FDI benchmark study[J].Control Engineering Practice,2006,14(6):577-596.
[1] LI Ke, YANG Ling, ZHAO Yanbo, CHEN Yonglong, LUO Shouxi. EGCN-CeDML:A Distributed Machine Learning Framework for Vehicle Driving Behavior Prediction [J]. Computer Science, 2023, 50(9): 318-330.
[2] WANG Luo, LI Biao, FU Ruigang. Infrared Ground Multi-object Tracking Method Based on Improved ByteTrack Algorithm [J]. Computer Science, 2023, 50(9): 176-183.
[3] YI Liu, GENG Xinyu, BAI Jing. Hierarchical Multi-label Text Classification Algorithm Based on Parallel Convolutional Network Information Fusion [J]. Computer Science, 2023, 50(9): 278-286.
[4] LUO Yuanyuan, YANG Chunming, LI Bo, ZHANG Hui, ZHAO Xujian. Chinese Medical Named Entity Recognition Method Incorporating Machine ReadingComprehension [J]. Computer Science, 2023, 50(9): 287-294.
[5] ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[6] TENG Sihang, WANG Lie, LI Ya. Non-autoregressive Transformer Chinese Speech Recognition Incorporating Pronunciation- Character Representation Conversion [J]. Computer Science, 2023, 50(8): 111-117.
[7] YANG Zhizhuo, XU Lingling, Zhang Hu, LI Ru. Answer Extraction Method for Reading Comprehension Based on Frame Semantics and GraphStructure [J]. Computer Science, 2023, 50(8): 170-176.
[8] WANG Jiahao, ZHONG Xin, LI Wenxiong, ZHAO Dexin. Human Activity Recognition with Meta-learning and Attention [J]. Computer Science, 2023, 50(8): 193-201.
[9] WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[10] LI Hui, LI Wengen, GUAN Jihong. Dually Encoded Semi-supervised Anomaly Detection [J]. Computer Science, 2023, 50(7): 53-59.
[11] YAN Mingqiang, YU Pengfei, LI Haiyan, LI Hongsong. Arbitrary Image Style Transfer with Consistent Semantic Style [J]. Computer Science, 2023, 50(7): 129-136.
[12] HENG Hongjun, ZHOU Wenhua. Anomaly Detection Method Based on Context Information Fusion and Noise Adaptation [J]. Computer Science, 2023, 50(7): 237-245.
[13] GAO Xiang, TANG Jiqiang, ZHU Junwu, LIANG Mingxuan, LI Yang. Study on Named Entity Recognition Method Based on Knowledge Graph Enhancement [J]. Computer Science, 2023, 50(6A): 220700153-6.
[14] ZHANG Tao, CHENG Yifei, SUN Xinxu. Graph Attention Networks Based on Causal Inference [J]. Computer Science, 2023, 50(6A): 220600230-9.
[15] CUI Lin, CUI Chenlu, LIU Zhengwei, XUE Kai. Speech Emotion Recognition Based on Improved MFCC and Parallel Hybrid Model [J]. Computer Science, 2023, 50(6A): 220800211-7.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!