基于多尺度层级LSTM网络的时间序列预测分析

摘要/Abstract

摘要： 现有的深度学习研究都依赖于网络的自发学习能力,在训练过程中力求避免或尽量减少人为先验知识的设定,导致网络训练过程完全“黑盒”,研究人员很难从语义上进行阐述。针对这种情况,文中提出了一种基于原始LSTM网络的改进——多尺度层级LSTM(Multi-Scale Hierarchical Long Short-Term Memory,MSH-LSTM)网络。该网络保留了神经网络的常规实现流程,在网络学习过程中将层级网络结构与人的经验知识有机结合,使网络在人为指引下有目的地训练,不再是完全的“黑盒”,同时对时间序列更好地进行分析预测。为说明MSH-LSTM网络结构的有效性,实验选取了两种时间序列数据(气温、股票),结果表明,相较于ANN网络、LSTM网络及GRU网络,MSH-LSTM网络在保证网络适用性的同时更具分析预测优势。在气温实验中,由于MSH-LSTM与常规LSTM,GRU网络都利用了序列数据的时间因素,因此,它们的效果明显优于ANN;在股票实验中,MSH-LSTM的MAPE误差相对于常规LSTM,GRU,ANN网络分别平均提升了约19.65%,24.35%,46.30%。

关键词: LSTM, 层级网络, 短期预测, 时间序列, 循环神经网络

Abstract: Nowadays,most researches in deep learning depend on the self-learning capacity of used neural network.Specifically,they focus on using as less human-knowledge priors as possible during the training step,which leads to totally “black-box” and is hard to clarify the training process semantically for researchers.In light of this situation,this paper proposes an improved structure of primitive LSTM (Multi-Scale Hierarchical Long Short-Term Memory,MSH-LSTM).It retains the common procedure widely used in neural network,combines the structure of neural network and human’s prior knowledge,enables the network to train purposefully under the guidance and solving the problem of “black-box” in a way,ultimately resulting in much better analytic results on time series data.To illustrate the effectiveness of MSH-LSTM,two groups of experiments (temperature and stock-price respectively) were carried out.Experimental results demonstrate that the proposed MSH-LSTM outperforms primitive ANN,LSTM and GRU without loss of network’s applicability.In temperature experiment,MSH-LSTM,primitive LSTM and primitive GRU use temporal information to get approximately results,which are better than primitive ANN.In stock price experiment,MSH-LSTM’ssuperiority is more obvious.The error of MAPE of MSH-LSTM is increaced by an average of 19.65%,24.35%,46.3% compared with that of primitive LSTM,GRU and ANN,respectively.

Key words: Hierarchical network, LSTM, Recursive neural network, Short-term forecast, Time series

中图分类号:

TP391

张旭东, 杜家浩, 黄宇方, 石东贤, 缪永伟. 基于多尺度层级LSTM网络的时间序列预测分析[J]. 计算机科学, 2019, 46(11A): 52-57. https://doi.org/

ZHANG Xu-dong, DU Jia-hao, HUANG Yu-fang, SHI Dong-xian, MIAO Yong-wei. Time Series Analysis Based on MSH-LSTM[J]. Computer Science, 2019, 46(11A): 52-57. https://doi.org/

参考文献

[1]SELVIN S,VINAYAKUMAR R,GOPALAKRISHNAN E A,et al.Stock price prediction using LSTM,RNN and CNN-sliding window model[C]∥2017 International Conference on Advances in Computing,Communications and Informatics (ICACCI).IEEE,2017.
[2]SHI S,ZHAO M,GUAN J,et al.A hierarchical lstm model with multiple features for sentiment analysis of sina weibo texts[C]∥International Conference on Asian Language Processing.IEEE,2017:379-382.
[3]HU Y,SUN X,NIE X,et al.An Enhanced LSTM for Trend Following of Time Series[J].IEEE Access,2019,7:34020-34030.
[4]ZHANG Q,LI F,LONG F,et al.Vehicle Emission Forecasting Based on Wavelet Transform and Long Short-Term Memory Network[J].IEEE Access,2018,6:56984-56994.
[5]WANG R,LIANG X,ZHU X,et al.A Feasibility of Respiration Prediction Based on Deep Bi-LSTM for Real-Time Tumor Tracking[J].IEEE Access,2018,6:51262-51268.
[6]SAAD W.Comparative study of stock trend prediction usingtime delay,recurrent and probabilistic neural network[J].IEEE Transactions on Neural Networks,1998,9(6):1456-70.
[7]LIN S,RUNGER G C.GCRNN:Group-Constrained Convolutional Recurrent Neural Network[J].IEEE Transactions on Neural Networks and Learning Systems,2018,29(10):4709-4718.
[8]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[9]ZHANG Q,WANG H,DONG J,et al.Prediction of Sea Surface Temperature Using Long Short-Term Memory[J].IEEE Geoscience and Remote Sensing Letters,2017,14(10):1745-1749.
[10]YANG Y,DONG J,SUN X,et al.A CFCC-LSTM Model forSea Surface Temperature Prediction[J].IEEE Geoscience and Remote Sensing Letters,2018,15(2):207-211.
[11]KARIM F,MAJUMDAR S,DARABI H,et al.LSTM FullyConvolutional Networks for Time Series Classification[J].IEEE Access,2017,6(99):1662-1669.
[12]LIU F,CAI M,WANG L,et al.An Ensemble Model Based on Adaptive Noise Reducer and Over-Fitting Prevention LSTM for Multivariate Time Series Forecasting[J].IEEE Access,2019,6:26102-26115.
[13]ZHANG S,WANG Y,LIU M,et al.Data-based Line Trip Fault Prediction in Power Systems Using LSTM Networks and SVM[J].IEEE Access,2017,6:1-1.
[14]CHO K,VAN MERRINBOER B,GULCEHRE C,et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[J].arXiv:1406.1078,2014.
[15]CHEN W,ZHANG Y,YEO C K,et al.Stock market prediction using neural network through news on online social networks[C]∥2017 International Smart Cities Conference (ISC2).IEEE,2017:1-6.
[16]SUN L,SU T,ZHOU S,et al.GMU:A Novel RNN Neuron and Its Application to Handwriting Recognition[C]∥2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).IEEE Computer Society,2017.
[17]HASIB M M,NAYAK T,HUANG Y.A hierarchical LSTM model with attention for modeling EEG non-stationarity for human decision prediction[C]∥IEEE EMBS International Confe-rence on Biomedical & Health Informatics (BHI).IEEE,2018:104-107.
[18]TANG X.Large-Scale Computing Systems Workload Prediction Using Parallel Improved LSTM Neural Network[J].IEEE Access,2019,7:40525-40533.
[19]LIU Y,DONG S,LU M,et al.LSTM based reserve prediction for bank outlets[J].Tsinghua Science and Technology,2018,24(1):77-85.
[20]SU P,DING X R,ZHANG Y T,et al.Long-term blood pressure prediction with deep recurrent neural networks[C]∥2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI).IEEE,2018:323-328.

相关文章 15

[1]	彭双, 伍江江, 陈浩, 杜春, 李军. 基于注意力神经网络的对地观测卫星星上自主任务规划方法 Satellite Onboard Observation Task Planning Based on Attention Neural Network 计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093
[2]	张源, 康乐, 宫朝辉, 张志鸿. 基于Bi-LSTM的期货市场关联交易行为检测方法 Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM 计算机科学, 2022, 49(7): 31-39. https://doi.org/10.11896/jsjkx.210400304
[3]	高振卓, 王志海, 刘海洋. 嵌入典型时间序列特征的随机Shapelet森林算法 Random Shapelet Forest Algorithm Embedded with Canonical Time Series Features 计算机科学, 2022, 49(7): 40-49. https://doi.org/10.11896/jsjkx.210700226
[4]	刘宝宝, 杨菁菁, 陶露, 王贺应. 基于DE-LSTM模型的教育统计数据预测研究 Study on Prediction of Educational Statistical Data Based on DE-LSTM Model 计算机科学, 2022, 49(6A): 261-266. https://doi.org/10.11896/jsjkx.220300120
[5]	于家畦, 康晓东, 白程程, 刘汉卿. 一种新的中文电子病历文本检索模型 New Text Retrieval Model of Chinese Electronic Medical Records 计算机科学, 2022, 49(6A): 32-38. https://doi.org/10.11896/jsjkx.210400198
[6]	林夕, 陈孜卓, 王中卿. 基于不平衡数据与集成学习的属性级情感分类 Aspect-level Sentiment Classification Based on Imbalanced Data and Ensemble Learning 计算机科学, 2022, 49(6A): 144-149. https://doi.org/10.11896/jsjkx.210500205
[7]	王杉, 徐楚怡, 师春香, 张瑛. 基于CNN-LSTM的卫星云图云分类方法研究 Study on Cloud Classification Method of Satellite Cloud Images Based on CNN-LSTM 计算机科学, 2022, 49(6A): 675-679. https://doi.org/10.11896/jsjkx.210300177
[8]	喻昕, 林植良. 解决一类非光滑伪凸优化问题的新型神经网络 Novel Neural Network for Dealing with a Kind of Non-smooth Pseudoconvex Optimization Problems 计算机科学, 2022, 49(5): 227-234. https://doi.org/10.11896/jsjkx.210400179
[9]	安鑫, 代子彪, 李阳, 孙晓, 任福继. 基于BERT的端到端语音合成方法 End-to-End Speech Synthesis Based on BERT 计算机科学, 2022, 49(4): 221-226. https://doi.org/10.11896/jsjkx.210300071
[10]	时雨涛, 孙晓. 一种会话理解模型的问题生成方法 Conversational Comprehension Model for Question Generation 计算机科学, 2022, 49(3): 232-238. https://doi.org/10.11896/jsjkx.210200153
[11]	高堰泸, 徐圆, 朱群雄. 基于A-DLSTM夹层网络结构的电能消耗预测方法 Predicting Electric Energy Consumption Using Sandwich Structure of Attention in Double -LSTM 计算机科学, 2022, 49(3): 269-275. https://doi.org/10.11896/jsjkx.210100006
[12]	李昊, 曹书瑜, 陈亚青, 张敏. 基于注意力机制的用户轨迹识别模型 User Trajectory Identification Model via Attention Mechanism 计算机科学, 2022, 49(3): 308-312. https://doi.org/10.11896/jsjkx.210300231
[13]	肖丁, 张玙璠, 纪厚业. 基于多头注意力机制的用户窃电行为检测 Electricity Theft Detection Based on Multi-head Attention Mechanism 计算机科学, 2022, 49(1): 140-145. https://doi.org/10.11896/jsjkx.210100177
[14]	袁景凌, 丁远远, 盛德明, 李琳. 基于视觉方面注意力的图像文本情感分析模型 Image-Text Sentiment Analysis Model Based on Visual Aspect Attention 计算机科学, 2022, 49(1): 219-224. https://doi.org/10.11896/jsjkx.201000074
[15]	程思伟, 葛唯益, 王羽, 徐建. BGCN:基于BERT和图卷积网络的触发词检测 BGCN:Trigger Detection Based on BERT and Graph Convolution Network 计算机科学, 2021, 48(7): 292-298. https://doi.org/10.11896/jsjkx.200500133

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed