计算机科学 ›› 2020, Vol. 47 ›› Issue (9): 105-109.doi: 10.11896/jsjkx.190800030

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于Zoneout的跨尺度循环神经网络及其在短期电力负荷预测中的应用

庄世杰, 於志勇, 郭文忠, 黄昉菀   

  1. 福州大学数学与计算机科学学院 福州350116
    福州大学福建省网络计算与智能信息处理重点实验室 福州350116
  • 收稿日期:2019-08-07 发布日期:2020-09-10
  • 通讯作者: 黄昉菀(hfw@fzu.edu.cn)
  • 作者简介:2942265521@qq.com
  • 基金资助:
    国家自然科学基金(61772136,61672159);福建省中青年教师教育科研项目(JT180045)

Short Term Load Forecasting via Zoneout-based Multi-time Scale Recurrent Neural Network

ZHUANG Shi-jie, YU Zhi-yong, GUO Wen-zhong, HUANG Fang-wan   

  1. College of Mathematics and Computer Science,Fuzhou University,Fuzhou 350116,China
    Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing,Fuzhou University,Fuzhou 350116,China
  • Received:2019-08-07 Published:2020-09-10
  • About author:ZHUANG Shi-jie,born in 1995,undergraduate.His main research interests include computational intelligence,machine learning and deep learning.
    HUANG Fang-wan,born in 1980,senior lecturer,is a member of China Computer Federation.Her main research interests include computational intelligence,machine learning and big data analysis.
  • Supported by:
    National Natural Science Foundation of China (61772136,61672159) and Research Project for Young and Middle-aged Teachers of Fujian Province (JT180045).

摘要: 通过精确的电力负荷预测,智能电网可以提供比传统电网更高效、可靠和环保的电力服务。现实生活中,电力负荷数据往往存在着与历史数据较高的时间相关性,而传统的神经网络却很少关注它。近年来,循环神经网络(Recurrent Neural Network,RNN)由于可以很好地捕获在时间上距离很远的数据之间的相关性,因此在电力负荷预测中受到越来越多的关注。但是,由于RNN特有的自循环结构,当采用随时间的反向传播算法(Back-Propagation Through Time,BPTT)进行网络训练时,随着网络层数的增加,很容易发生梯度消失等问题,从而导致预测精度下降。目前已有多种解决梯度消失问题的RNN架构,如长短期记忆网络(Long Short-Term Memory,LSTM)和门控制单元(Gated Recurrent Unit,GRU),但其复杂的内部结构会增加训练时长。为了解决上述问题,文中首先对目前流行的各种RNN架构进行了研究和分析,其次结合最新提出的Zoneout技术,设计了一种跨时间尺度的分模块循环神经网络架构,重点研究了隐藏层模块的随机更新策略,不仅有效解决了梯度消失问题,而且大幅度减少了待训练的网络参数。基于基准数据集和实际负载数据集的实验结果表明,该结构可以获得比目前流行的RNN架构更好的性能。

关键词: 短期电力负荷预测, 循环神经网络, 跨时间尺度, Zoneout

Abstract: Because accurate power load forecasting,smart grids can provide more efficient,reliable and environmentally friendly power services than traditional grids.In real life,power load data often has a high temporal correlation with historical data,while traditional neural networks pay little attention to it.In recent years,the recurrent neural network (RNN) has received more and more attention in power load forecasting,because it can well capture the correlation between data with large cross-time scale.However,due to the unique self-connections structure of RNN,when the back-propagation through time(BPTT) is adopted for network training,the problems such as vanishing gradient are prone to occur with the number of network layers increases,resulting in a decrease in prediction accuracy.There are varieties of RNN architectures that can solve the vanishing gradient problem,such as long short-term memory (LSTM) and gated recurrent unit (GRU),but their complex internal structure will increase the training time.In order to solve the above problems,this paper first analyzes and studies RNN and itsvariants,and then combines the Zoneout function to design a multi-time scale modularized RNN architecture,focuses on the update strategy of hidden layer modules.It not only effectively solves the vanishing gradient problem,but also greatly reduces the number of network parameters that need to be trained.Experimental results based on the benchmark dataset and the real-worldload dataset show that this architecture can achieve better performance than the current popular RNN architecture.

Key words: Short term load forecast, Recurrent neural networks, Multi-time scale, Zoneout

中图分类号: 

  • TP183
[1] DU D,CHEN R,LI X,et al.Malicious data deception attacksagainst power systems:A new case and its detection method[J].Transactions of the Institute of Measurement and Control,2019,41(6):1590-1599.
[2] NATARAJA C,GORAWAR M B,SHILPA G N,et al.Shortterm load forecasting using time series analysis:a case study for Karnataka,India[J].Int.J.Eng.Sci.Innov.Technol,2012,1(2):45-53.
[3] HERNÁNDEZ L,BALADRÓN C,AGUIAR J,et al.Artificial neural network for short-term load forecasting in distribution systems[J].Energies,2014,7(3):1576-1598.
[4] SONG K B,BAEK Y S,HONG D H,et al.Short-term load forecasting for the holidays using fuzzy linear regression method[J].IEEE Transactions on Power Systems,2005,20(1):96-101.
[5] WANG Z,YANG F,HO D W C,et al.Stochastic dynamic mo-deling of short gene expression time-series data[J].IEEE Tran-sactions on Manobioscience,2008,7(1):44-55.
[6] WEI G,WANG Z,SHU H,et al.Robust filtering for gene expression time series data with variance constraints[J].International Journal of Computer Mathematics,2007,84(5):619-633.
[7] AL-HAMADI H M,SOLIMAN S A.Short-term electric load forecasting based on Kalman filtering algorithm with moving window weather and load model[J].Electric Power Systems Research,2004,68(1):47-59.
[8] RAHMAN S,BHATNAGAR R.An expert system based algorithm for short term load forecast[J].IEEE Transactions on Power Systems,1988,3(2):392-399.
[9] WEI G,FENG G,WANG Z.Robust Control for Discrete-Time Fuzzy Systems With Infinite-Distributed Delays[J].IEEE Transactions on Fuzzy Systems,2008,17(1):224-232.
[10] LEE C M,KO C N.Short-term load forecasting using liftingscheme and ARIMA models[J].Expert Systems with Applications,2011,38(5):5902-5911.
[11] LIU Y,WANG Z,LIU X.Asymptotic stability for neural networks with mixed time-delays:The discrete-time case[J].Neural Networks,2009,22(1):67-74.
[12] LIU L,SHEN B,WANG X.Research on kernel function of support vector machine[M]//Advanced Technologies,Embedded and Multimedia for Human-centric Computing.Springer,Dordrecht,2014:827-834.
[13] BAI Y.Design of Cluster Analysis Model Based on Load CharacteristicCurve of Power Consumers [J].Journal of Chongqing University of Technology(Natural Science),2018, 32(12):181-185.
[14] HIPPERT H S,PEDREIRA C E,SOUZA R C.Neural networks for short-term load forecasting:A review and evaluation[J].IEEE Transactions on Power Systems,2001,16(1):44-55.
[15] LÄNGKVIST M,KARLSSON L,LOUTFI A.A review of unsupervised feature learning and deep learning for time-series modeling[J].Pattern Recognition Letters,2014,42:11-24.
[16] BENGIO Y,SIMARD P,FRASCONI P.Learning long-term dependencies with gradient descent is difficult[J].IEEE Transactions on Neural Networks,1994,5(2):157-166.
[17] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[18] CHUNG J,GULCEHRE C,CHO K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[J].arXiv:1412.3555,2014.
[19] KOUTNIK J,GREFF K,GOMEZ F,et al.A clockwork rnn[J].arXiv:1402.3511,2014.
[20] CHANG S,ZHANG Y,HAN W,et al.Dilated recurrent neural networks[C]//Advances in Neural Information Processing Systems.2017:77-87.
[21] LI S,LI W,COOK C,et al.Independently recurrent neural network (indrnn):Building a longer and deeper rnn[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5457-5466.
[22] KRUEGER D,MAHARAJ T,KRAMÁR J,et al.Zoneout:Re-gularizing rnns by randomly preserving hidden activations[J].arXiv:1606.01305,2016.
[1] 李亚男, 胡宇佳, 甘伟, 朱敏. 基于深度学习的miRNA靶位点预测研究综述[J]. 计算机科学, 2021, 48(1): 209-216.
[2] 游兰, 韩雪薇, 何正伟, 肖丝雨, 何渡, 潘筱萌. 基于改进Seq2Seq的短时AIS轨迹序列预测模型[J]. 计算机科学, 2020, 47(9): 169-174.
[3] 赫磊, 邵展鹏, 张剑华, 周小龙. 基于深度学习的行为识别算法综述[J]. 计算机科学, 2020, 47(6A): 139-147.
[4] 张志扬, 张凤荔, 陈学勤, 王瑞锦. 基于分层注意力的信息级联预测模型[J]. 计算机科学, 2020, 47(6): 201-209.
[5] 李太松,贺泽宇,王冰,颜永红,唐向红. 基于循环时间卷积网络的序列流推荐算法[J]. 计算机科学, 2020, 47(3): 103-109.
[6] 励益韬, 孙未未. 基于循环神经网络的轨迹压缩算法[J]. 计算机科学, 2020, 47(10): 102-107.
[7] 陈俊航, 徐小平, 杨恒泓. 基于Seq2seq模型的推荐应用研究[J]. 计算机科学, 2019, 46(6A): 493-496.
[8] 毛莺池, 曹海, 何进锋. 面向大坝变形监测的时空一体化预测算法[J]. 计算机科学, 2019, 46(2): 223-229.
[9] 敬颉, 陈潭, 杜文丽, 刘志康, 尹皓. 自动驾驶场景中增强深度学习的时空特征提取方法[J]. 计算机科学, 2019, 46(11A): 1-4.
[10] 肖锐, 蒋家琪, 张云春. 多义词语义拓扑及有监督的词义消歧研究[J]. 计算机科学, 2019, 46(11A): 13-18.
[11] 张旭东, 杜家浩, 黄宇方, 石东贤, 缪永伟. 基于多尺度层级LSTM网络的时间序列预测分析[J]. 计算机科学, 2019, 46(11A): 52-57.
[12] 贾宁, 郑纯军. 基于注意力LSTM的音乐主题推荐模型[J]. 计算机科学, 2019, 46(11A): 230-235.
[13] 张献, 贲可荣. 改进的神经语言模型及其在代码提示中的应用[J]. 计算机科学, 2019, 46(11): 168-175.
[14] 刘世昌,金敏. 多尺度分析与数据互迁移相结合的短期电力负荷预测方法[J]. 计算机科学, 2018, 45(7): 315-321.
[15] 权波, 杨博辰, 胡可奇, 郭晨萱, 李巧勤. 基于LSTM的船舶航迹预测模型[J]. 计算机科学, 2018, 45(11A): 126-131.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 雷丽晖,王静. 可能性测度下的LTL模型检测并行化研究[J]. 计算机科学, 2018, 45(4): 71 -75 .
[2] 孙启,金燕,何琨,徐凌轩. 用于求解混合车辆路径问题的混合进化算法[J]. 计算机科学, 2018, 45(4): 76 -82 .
[3] 张佳男,肖鸣宇. 带权混合支配问题的近似算法研究[J]. 计算机科学, 2018, 45(4): 83 -88 .
[4] 伍建辉,黄中祥,李武,吴健辉,彭鑫,张生. 城市道路建设时序决策的鲁棒优化[J]. 计算机科学, 2018, 45(4): 89 -93 .
[5] 崔琼,李建华,王宏,南明莉. 基于节点修复的网络化指挥信息系统弹性分析模型[J]. 计算机科学, 2018, 45(4): 117 -121 .
[6] 杨羽琦,章国安,金喜龙. 车载自组织网络中基于车辆密度的双簇头路由协议[J]. 计算机科学, 2018, 45(4): 126 -130 .
[7] 施超,谢在鹏,柳晗,吕鑫. 基于稳定匹配的容器部署策略的优化[J]. 计算机科学, 2018, 45(4): 131 -136 .
[8] 庞博,金乾坤,合尼古力·吾买尔,齐兴斌. 软件定义网络中基于网络切片和ILP模型的路由方案[J]. 计算机科学, 2018, 45(4): 143 -147 .
[9] 郑秀林,宋海燕,付伊鹏. MORUS-1280-128算法的区分分析[J]. 计算机科学, 2018, 45(4): 152 -156 .
[10] 厉柏伸,李领治,孙涌,朱艳琴. 基于伪梯度提升决策树的内网防御算法[J]. 计算机科学, 2018, 45(4): 157 -162 .