Computer Science ›› 2022, Vol. 49 ›› Issue (3): 269-275.doi: 10.11896/jsjkx.210100006

• Artificial Intelligence • Previous Articles     Next Articles

Predicting Electric Energy Consumption Using Sandwich Structure of Attention in Double -LSTM

GAO Yan-lu, XU Yuan, ZHU Qun-xiong   

  1. College of Information Science & Technology,Beijing University of Chemical Technology,Beijing 100029,China
    Engineering Research Center of Intelligent PSE,Ministry of Education of China,Beijing 100029,China
  • Received:2021-01-01 Revised:2021-05-16 Online:2022-03-15 Published:2022-03-15
  • About author:GAO Yan-lu,born in 1996,postgra-duate.His main research interests include time series forecasting and artificial intelligence.
    ZHU Qun-xiong,born in 1960,Ph.D,professor,Ph.D supervisor.His main research interests include computational intelligence and industrial applications,intelligent modeling and optimization,fault diagnosis and alarm management.
  • Supported by:
    National Natural Science Foundation of China(61973024 ,61973022).

Abstract: The rapid growth of the global population and technological progress has significantly increased the world’s total power generation.Electric energy consumption forecasts play an essential role in power system dispatch and power generation management.Aim at the complex characteristics of time series of energy consumption data,and to improve the prediction accuracy of power consumption,a novel sandwich structure is proposed,in which an Attention mechanism is placed in the double layer long short-term memory artificial neural network,namely A-DLSTM.This network structure uses the attention mechanism in the mezzanine to adaptively focus on different features in each single time unit and uses the two-layer LSTM network to capture the time information in the sequence to predict the sequence data.The experimental data comes from the UCI machine learning data set,and it is the electricity consumption of a family in the past five years.The parameters of the experiment are adjusting by the grid search method.The experiment compares the prediction performance of A-DLSTM and the existing model on energy consumption data.The network of this article reaches the state-of-the-art in terms of mean square error,root mean square error,average absolute error,and average absolute percentage error.By analyzing the heat map’s attention layer,the factor that has the most significant impact on electricity consumption forecasting is determined.

Key words: Attention mechanism, Energy consumption prediction, Long and short-term memory network, Time series

CLC Number: 

  • TP183
[1]DEB C,ZHANG F,YANG J,et al.A review on time series forecasting techniques for building energy consumption[J].Rene-wable and Sustainable Energy Reviews,2017,74:902-924.
[2]LIU L,SHEN J,ZHANG M,et al.Learning the joint representation of heterogeneous temporal events for clinical endpoint prediction[J].arXiv:1803.04837,2018.
[3]CAO W,HU L,CAO L.Deep modeling complex couplings wi-thin financial markets[C]//Proceedings of the National Confe-rence on Artificial Intelligence.2015.
[4]HULOT P,ALOISE D,JENA S D.Towards station-level de-mand prediction for effective rebalancing in bike-sharing systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2018:378-386.
[5]YULE G U.Vii.on a method of investigating periodicities dis-turbed series,with special reference to wolfer’s sunspot numbers[J].Philosophical Transactions of the Royal Society of London.Series A,Containing Papers of a Mathematical or Physical Character,1927,226(636-646):267-298.
[6]BOX G E,JENKINS G M,REINSEL G C,et al.Time seriesanalysis:forecasting and control[M].John Wiley & Sons,2015.
[7]DRUCKER H,BURGES C J,KAUFMAN L,et al.Support vector regression machines[J].Advances in Neural Information Processing Systems,1996,9:155-161.
[8]KE G,MENG Q,FINLEY T,et al.Lightgbm:A highly efficient gradient boosting decision tree[C]//Advances in Neural Information Processing Systems.2017:3146-3154.
[9]HOPFIELD J J.Neural networks and physical systems withemergent collective computational abilities[J].Proceedings of the National Academy of Sciences,1982,79(8):2554-2558.
[10]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[11]ZHANG Y,JI B,LU H X,et al.Short-term High Voltage Load Current Prediction Method Based on LSTM Neural Network[J].Computer Science,2019,46(6):49-54.
[12]PARK D,KIM S,AN Y,et al.LiReD:A Light-Weight Real-Time Fault Detection System for Edge Computing Using LSTM Recurrent Neural Networks[J].Sensors,2018,18(7):2110.
[13]LI Y,ZHU Z,KONG D,et al.EA-LSTM:Evolutionary attention-based LSTM for time series prediction[J].Knowledge-Based Systems,2019,181:104785.
[14]CINAR Y G,MIRISAEE H,GOSWAMI P,et al.Position-based content attention for time series forecasting with sequence-to-sequence rnns [C]//International Conference on Neural Information Processing.Springer,2017:533-544.
[15]QIN Y,SONG D,CHEN H,et al.A dual-stage attention-based recurrent neural network for time series prediction[J].arXiv:1704.02971,2017.
[16]YANG M,TU W,WANG J,et al.Attention-based LSTM for target dependent sentiment classification[C]//Proceedings of the thirty-first AAAI Conference on Artificial Intelligence.2017:5013-5014.
[17]KIM S,KANG M.Financial series prediction using attentionLSTM[J].arXiv:1902.10877,2019.
[18]ZHANG H L,KANG X D,LI B,et al.Medical name entity recognition based on Bi-LSTM-CRF and attention mechanism [J].Computer Application,2020,40(S1):5.
[19]PENG W,WANG J R,YIN S Q.Short-term Load Forecasting Model Based on Attention-LSTM in Electricity Market[J].Power System Technology,2019,43(5):1745-1751.
[20]LIU J C,QING X L,ZHU R Z.Prediction of RFID Mobile Object Location Based on LSTM-Attention[J].Computer Science,2021,48(3):188-195.
[21]IBRAHIM H,ILINCA A,PERRON J.Energy storage systems characteristics and comparisons[J].Renewable and Sustainable Energy Reviews,2008,12(5):1221-1250.
[22]MURALITHARAN K,SAKTHIVEL R,VISHNUVARTHAN R.Neural network based optimization approach for energy demand prediction in smart grid[J].Neurocomputing,2018,273:199-208.
[23]BOUKTIF S,FIAZ A,OUNI A,et al.Optimal deep learning LSTM model for electric load forecasting using feature selection and genetic algorithm:Comparison with machine learning approaches[J].Energies,2018,11(7):1636.
[24]KIM T Y,CHO S B.Predicting residential energy consumption using CNN-LSTM neural networks[J].Energy,2019,182:72-81.
[25]LE T,VO M T,VO B,et al.Improving electric energy consump-tion prediction using CNN and Bi-LSTM [J].Applied Sciences,2019,9(20):4237.
[26]LIPTON Z C,BERKOWITZ J,ELKAN C.A critical review of recurrent neural networks for sequence learning[J].arXiv:1506.00019,2015.
[27]MNIH V,HEESS N,GRAVES A,et al.Recurrent models ofvisual attention[J].Advances in Neural Information Processing Systems,2014,27:2204-2212.
[28]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[29]CHO K,MERRIËNBOER B V,GULCEHRE C,et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[J].arXiv:1406.1078,2014.
[30]HEBRAIL G.Individual household electric power consumption data set.UCI machine learning repository[EB/OL].https://archive.ics.uci.edu/ml/datasets/individual+household+electric+power+consumption/.
[31]YAO Y,HUANG Z.Bi-directional LSTM recurrent neural net-work for Chineseword segmentation[C]//International Confe-rence on Neural Information Processing.Springer,2016:345-353.
[32]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTMfor Aspect-level Sentiment Classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[3] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[4] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[5] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[8] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[9] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[10] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[11] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[12] XIONG Luo-geng, ZHENG Shang, ZOU Hai-tao, YU Hua-long, GAO Shang. Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism [J]. Computer Science, 2022, 49(7): 212-219.
[13] PENG Shuang, WU Jiang-jiang, CHEN Hao, DU Chun, LI Jun. Satellite Onboard Observation Task Planning Based on Attention Neural Network [J]. Computer Science, 2022, 49(7): 242-247.
[14] GAO Zhen-zhuo, WANG Zhi-hai, LIU Hai-yang. Random Shapelet Forest Algorithm Embedded with Canonical Time Series Features [J]. Computer Science, 2022, 49(7): 40-49.
[15] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!