计算机科学 ›› 2022, Vol. 49 ›› Issue (3): 269-275.doi: 10.11896/jsjkx.210100006

• 人工智能 • 上一篇    下一篇

基于A-DLSTM夹层网络结构的电能消耗预测方法

高堰泸, 徐圆, 朱群雄   

  1. 北京化工大学信息科学与技术学院 北京100029
    智能过程系统工程教育部工程研究中心 北京100029
  • 收稿日期:2021-01-01 修回日期:2021-05-16 出版日期:2022-03-15 发布日期:2022-03-15
  • 通讯作者: 朱群雄(zhuqx@mail.buct.edu.cn)
  • 作者简介:(872213775@qq.com)
  • 基金资助:
    国家自然科学基金(61973024,61973022)

Predicting Electric Energy Consumption Using Sandwich Structure of Attention in Double -LSTM

GAO Yan-lu, XU Yuan, ZHU Qun-xiong   

  1. College of Information Science & Technology,Beijing University of Chemical Technology,Beijing 100029,China
    Engineering Research Center of Intelligent PSE,Ministry of Education of China,Beijing 100029,China
  • Received:2021-01-01 Revised:2021-05-16 Online:2022-03-15 Published:2022-03-15
  • About author:GAO Yan-lu,born in 1996,postgra-duate.His main research interests include time series forecasting and artificial intelligence.
    ZHU Qun-xiong,born in 1960,Ph.D,professor,Ph.D supervisor.His main research interests include computational intelligence and industrial applications,intelligent modeling and optimization,fault diagnosis and alarm management.
  • Supported by:
    National Natural Science Foundation of China(61973024 ,61973022).

摘要: 全球人口的快速增长和技术进步极大地提高了世界的总发电量,电能消耗预测对于电力系统调度和发电量管理发挥着重要的作用,为了提高电能消耗的预测精度,针对能耗数据的复杂时序特性,文中提出了一种将注意力机制(Attention)放置于双层长短期记忆人工神经网络(Double layer Long Short-Term Memory,DLSTM)中的新颖夹层结构,即A-DLSTM。该网络结构利用夹层中的注意力机制自适应地关注单个时间单元中不同的特征量,通过双层LSTM网络对序列中的时间信息进行抓取,以对序列数据进行预测。文中的实验数据为UCI机器学习数据集上某家庭近5年的用电量,采用网格搜索法进行调参,实验对比了A-DLSTM与现有的模型在能耗数据上的预测性能,文中的网络结构在均方误差、均方根误差、平均绝对误差、平均绝对百分比误差上均达到了最优,且通过热力图对注意力层进行了分析,确定了对用电量预测影响最大的因素。

关键词: 长短期记忆网络, 能耗预测, 时间序列, 注意力机制

Abstract: The rapid growth of the global population and technological progress has significantly increased the world’s total power generation.Electric energy consumption forecasts play an essential role in power system dispatch and power generation management.Aim at the complex characteristics of time series of energy consumption data,and to improve the prediction accuracy of power consumption,a novel sandwich structure is proposed,in which an Attention mechanism is placed in the double layer long short-term memory artificial neural network,namely A-DLSTM.This network structure uses the attention mechanism in the mezzanine to adaptively focus on different features in each single time unit and uses the two-layer LSTM network to capture the time information in the sequence to predict the sequence data.The experimental data comes from the UCI machine learning data set,and it is the electricity consumption of a family in the past five years.The parameters of the experiment are adjusting by the grid search method.The experiment compares the prediction performance of A-DLSTM and the existing model on energy consumption data.The network of this article reaches the state-of-the-art in terms of mean square error,root mean square error,average absolute error,and average absolute percentage error.By analyzing the heat map’s attention layer,the factor that has the most significant impact on electricity consumption forecasting is determined.

Key words: Attention mechanism, Energy consumption prediction, Long and short-term memory network, Time series

中图分类号: 

  • TP183
[1]DEB C,ZHANG F,YANG J,et al.A review on time series forecasting techniques for building energy consumption[J].Rene-wable and Sustainable Energy Reviews,2017,74:902-924.
[2]LIU L,SHEN J,ZHANG M,et al.Learning the joint representation of heterogeneous temporal events for clinical endpoint prediction[J].arXiv:1803.04837,2018.
[3]CAO W,HU L,CAO L.Deep modeling complex couplings wi-thin financial markets[C]//Proceedings of the National Confe-rence on Artificial Intelligence.2015.
[4]HULOT P,ALOISE D,JENA S D.Towards station-level de-mand prediction for effective rebalancing in bike-sharing systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2018:378-386.
[5]YULE G U.Vii.on a method of investigating periodicities dis-turbed series,with special reference to wolfer’s sunspot numbers[J].Philosophical Transactions of the Royal Society of London.Series A,Containing Papers of a Mathematical or Physical Character,1927,226(636-646):267-298.
[6]BOX G E,JENKINS G M,REINSEL G C,et al.Time seriesanalysis:forecasting and control[M].John Wiley & Sons,2015.
[7]DRUCKER H,BURGES C J,KAUFMAN L,et al.Support vector regression machines[J].Advances in Neural Information Processing Systems,1996,9:155-161.
[8]KE G,MENG Q,FINLEY T,et al.Lightgbm:A highly efficient gradient boosting decision tree[C]//Advances in Neural Information Processing Systems.2017:3146-3154.
[9]HOPFIELD J J.Neural networks and physical systems withemergent collective computational abilities[J].Proceedings of the National Academy of Sciences,1982,79(8):2554-2558.
[10]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[11]ZHANG Y,JI B,LU H X,et al.Short-term High Voltage Load Current Prediction Method Based on LSTM Neural Network[J].Computer Science,2019,46(6):49-54.
[12]PARK D,KIM S,AN Y,et al.LiReD:A Light-Weight Real-Time Fault Detection System for Edge Computing Using LSTM Recurrent Neural Networks[J].Sensors,2018,18(7):2110.
[13]LI Y,ZHU Z,KONG D,et al.EA-LSTM:Evolutionary attention-based LSTM for time series prediction[J].Knowledge-Based Systems,2019,181:104785.
[14]CINAR Y G,MIRISAEE H,GOSWAMI P,et al.Position-based content attention for time series forecasting with sequence-to-sequence rnns [C]//International Conference on Neural Information Processing.Springer,2017:533-544.
[15]QIN Y,SONG D,CHEN H,et al.A dual-stage attention-based recurrent neural network for time series prediction[J].arXiv:1704.02971,2017.
[16]YANG M,TU W,WANG J,et al.Attention-based LSTM for target dependent sentiment classification[C]//Proceedings of the thirty-first AAAI Conference on Artificial Intelligence.2017:5013-5014.
[17]KIM S,KANG M.Financial series prediction using attentionLSTM[J].arXiv:1902.10877,2019.
[18]ZHANG H L,KANG X D,LI B,et al.Medical name entity recognition based on Bi-LSTM-CRF and attention mechanism [J].Computer Application,2020,40(S1):5.
[19]PENG W,WANG J R,YIN S Q.Short-term Load Forecasting Model Based on Attention-LSTM in Electricity Market[J].Power System Technology,2019,43(5):1745-1751.
[20]LIU J C,QING X L,ZHU R Z.Prediction of RFID Mobile Object Location Based on LSTM-Attention[J].Computer Science,2021,48(3):188-195.
[21]IBRAHIM H,ILINCA A,PERRON J.Energy storage systems characteristics and comparisons[J].Renewable and Sustainable Energy Reviews,2008,12(5):1221-1250.
[22]MURALITHARAN K,SAKTHIVEL R,VISHNUVARTHAN R.Neural network based optimization approach for energy demand prediction in smart grid[J].Neurocomputing,2018,273:199-208.
[23]BOUKTIF S,FIAZ A,OUNI A,et al.Optimal deep learning LSTM model for electric load forecasting using feature selection and genetic algorithm:Comparison with machine learning approaches[J].Energies,2018,11(7):1636.
[24]KIM T Y,CHO S B.Predicting residential energy consumption using CNN-LSTM neural networks[J].Energy,2019,182:72-81.
[25]LE T,VO M T,VO B,et al.Improving electric energy consump-tion prediction using CNN and Bi-LSTM [J].Applied Sciences,2019,9(20):4237.
[26]LIPTON Z C,BERKOWITZ J,ELKAN C.A critical review of recurrent neural networks for sequence learning[J].arXiv:1506.00019,2015.
[27]MNIH V,HEESS N,GRAVES A,et al.Recurrent models ofvisual attention[J].Advances in Neural Information Processing Systems,2014,27:2204-2212.
[28]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[29]CHO K,MERRIËNBOER B V,GULCEHRE C,et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[J].arXiv:1406.1078,2014.
[30]HEBRAIL G.Individual household electric power consumption data set.UCI machine learning repository[EB/OL].https://archive.ics.uci.edu/ml/datasets/individual+household+electric+power+consumption/.
[31]YAO Y,HUANG Z.Bi-directional LSTM recurrent neural net-work for Chineseword segmentation[C]//International Confe-rence on Neural Information Processing.Springer,2016:345-353.
[32]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTMfor Aspect-level Sentiment Classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[3] 戴禹, 许林峰.
基于文本行匹配的跨图文本阅读方法
Cross-image Text Reading Method Based on Text Line Matching
计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[4] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[5] 熊丽琴, 曹雷, 赖俊, 陈希亮.
基于值分解的多智能体深度强化学习综述
Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization
计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[6] 王馨彤, 王璇, 孙知信.
基于多尺度记忆残差网络的网络流量异常检测模型
Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network
计算机科学, 2022, 49(8): 314-322. https://doi.org/10.11896/jsjkx.220200011
[7] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[8] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[9] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[10] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[11] 汪鸣, 彭舰, 黄飞虎.
基于多时间尺度时空图网络的交通流量预测模型
Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction
计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[12] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[13] 熊罗庚, 郑尚, 邹海涛, 于化龙, 高尚.
融合双向门控循环单元和注意力机制的软件自承认技术债识别方法
Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism
计算机科学, 2022, 49(7): 212-219. https://doi.org/10.11896/jsjkx.210500075
[14] 彭双, 伍江江, 陈浩, 杜春, 李军.
基于注意力神经网络的对地观测卫星星上自主任务规划方法
Satellite Onboard Observation Task Planning Based on Attention Neural Network
计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093
[15] 赵冬梅, 吴亚星, 张红斌.
基于IPSO-BiLSTM的网络安全态势预测
Network Security Situation Prediction Based on IPSO-BiLSTM
计算机科学, 2022, 49(7): 357-362. https://doi.org/10.11896/jsjkx.210900103
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!