Computer Science ›› 2020, Vol. 47 ›› Issue (9): 105-109.doi: 10.11896/jsjkx.190800030

• Database & Big Data & Data Science • Previous Articles     Next Articles

Short Term Load Forecasting via Zoneout-based Multi-time Scale Recurrent Neural Network

ZHUANG Shi-jie, YU Zhi-yong, GUO Wen-zhong, HUANG Fang-wan   

  1. College of Mathematics and Computer Science,Fuzhou University,Fuzhou 350116,China
    Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing,Fuzhou University,Fuzhou 350116,China
  • Received:2019-08-07 Published:2020-09-10
  • About author:ZHUANG Shi-jie,born in 1995,undergraduate.His main research interests include computational intelligence,machine learning and deep learning.
    HUANG Fang-wan,born in 1980,senior lecturer,is a member of China Computer Federation.Her main research interests include computational intelligence,machine learning and big data analysis.
  • Supported by:
    National Natural Science Foundation of China (61772136,61672159) and Research Project for Young and Middle-aged Teachers of Fujian Province (JT180045).

Abstract: Because accurate power load forecasting,smart grids can provide more efficient,reliable and environmentally friendly power services than traditional grids.In real life,power load data often has a high temporal correlation with historical data,while traditional neural networks pay little attention to it.In recent years,the recurrent neural network (RNN) has received more and more attention in power load forecasting,because it can well capture the correlation between data with large cross-time scale.However,due to the unique self-connections structure of RNN,when the back-propagation through time(BPTT) is adopted for network training,the problems such as vanishing gradient are prone to occur with the number of network layers increases,resulting in a decrease in prediction accuracy.There are varieties of RNN architectures that can solve the vanishing gradient problem,such as long short-term memory (LSTM) and gated recurrent unit (GRU),but their complex internal structure will increase the training time.In order to solve the above problems,this paper first analyzes and studies RNN and itsvariants,and then combines the Zoneout function to design a multi-time scale modularized RNN architecture,focuses on the update strategy of hidden layer modules.It not only effectively solves the vanishing gradient problem,but also greatly reduces the number of network parameters that need to be trained.Experimental results based on the benchmark dataset and the real-worldload dataset show that this architecture can achieve better performance than the current popular RNN architecture.

Key words: Short term load forecast, Recurrent neural networks, Multi-time scale, Zoneout

CLC Number: 

  • TP183
[1] DU D,CHEN R,LI X,et al.Malicious data deception attacksagainst power systems:A new case and its detection method[J].Transactions of the Institute of Measurement and Control,2019,41(6):1590-1599.
[2] NATARAJA C,GORAWAR M B,SHILPA G N,et al.Shortterm load forecasting using time series analysis:a case study for Karnataka,India[J].Int.J.Eng.Sci.Innov.Technol,2012,1(2):45-53.
[3] HERNÁNDEZ L,BALADRÓN C,AGUIAR J,et al.Artificial neural network for short-term load forecasting in distribution systems[J].Energies,2014,7(3):1576-1598.
[4] SONG K B,BAEK Y S,HONG D H,et al.Short-term load forecasting for the holidays using fuzzy linear regression method[J].IEEE Transactions on Power Systems,2005,20(1):96-101.
[5] WANG Z,YANG F,HO D W C,et al.Stochastic dynamic mo-deling of short gene expression time-series data[J].IEEE Tran-sactions on Manobioscience,2008,7(1):44-55.
[6] WEI G,WANG Z,SHU H,et al.Robust filtering for gene expression time series data with variance constraints[J].International Journal of Computer Mathematics,2007,84(5):619-633.
[7] AL-HAMADI H M,SOLIMAN S A.Short-term electric load forecasting based on Kalman filtering algorithm with moving window weather and load model[J].Electric Power Systems Research,2004,68(1):47-59.
[8] RAHMAN S,BHATNAGAR R.An expert system based algorithm for short term load forecast[J].IEEE Transactions on Power Systems,1988,3(2):392-399.
[9] WEI G,FENG G,WANG Z.Robust Control for Discrete-Time Fuzzy Systems With Infinite-Distributed Delays[J].IEEE Transactions on Fuzzy Systems,2008,17(1):224-232.
[10] LEE C M,KO C N.Short-term load forecasting using liftingscheme and ARIMA models[J].Expert Systems with Applications,2011,38(5):5902-5911.
[11] LIU Y,WANG Z,LIU X.Asymptotic stability for neural networks with mixed time-delays:The discrete-time case[J].Neural Networks,2009,22(1):67-74.
[12] LIU L,SHEN B,WANG X.Research on kernel function of support vector machine[M]//Advanced Technologies,Embedded and Multimedia for Human-centric Computing.Springer,Dordrecht,2014:827-834.
[13] BAI Y.Design of Cluster Analysis Model Based on Load CharacteristicCurve of Power Consumers [J].Journal of Chongqing University of Technology(Natural Science),2018, 32(12):181-185.
[14] HIPPERT H S,PEDREIRA C E,SOUZA R C.Neural networks for short-term load forecasting:A review and evaluation[J].IEEE Transactions on Power Systems,2001,16(1):44-55.
[15] LÄNGKVIST M,KARLSSON L,LOUTFI A.A review of unsupervised feature learning and deep learning for time-series modeling[J].Pattern Recognition Letters,2014,42:11-24.
[16] BENGIO Y,SIMARD P,FRASCONI P.Learning long-term dependencies with gradient descent is difficult[J].IEEE Transactions on Neural Networks,1994,5(2):157-166.
[17] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[18] CHUNG J,GULCEHRE C,CHO K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[J].arXiv:1412.3555,2014.
[19] KOUTNIK J,GREFF K,GOMEZ F,et al.A clockwork rnn[J].arXiv:1402.3511,2014.
[20] CHANG S,ZHANG Y,HAN W,et al.Dilated recurrent neural networks[C]//Advances in Neural Information Processing Systems.2017:77-87.
[21] LI S,LI W,COOK C,et al.Independently recurrent neural network (indrnn):Building a longer and deeper rnn[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5457-5466.
[22] KRUEGER D,MAHARAJ T,KRAMÁR J,et al.Zoneout:Re-gularizing rnns by randomly preserving hidden activations[J].arXiv:1606.01305,2016.
[1] JING Jie, CHEN Tan, DU Wen-li, LIU Zhi-kang, YIN Hao. Spatio-temporal Features Extraction of Traffic Based on Deep Neural Network [J]. Computer Science, 2019, 46(11A): 1-4.
[2] ZENG An, NIE Wen-jun. Stock Recommendation System Based on Deep Bidirectional LSTM [J]. Computer Science, 2019, 46(10): 84-89.
[3] PANG Chao and YIN Chuan-huan. Chinese Text Summarization Based on Classification [J]. Computer Science, 2018, 45(1): 144-147.
[4] DAI Xiao-hong,WANG Guang-li. Application of L-M Optimized BP Algorithm in Short-term Power Load Forecast [J]. Computer Science, 2011, 38(7): 265-267.
[5] REN Hai-jun,ZHANG Xiao-xing,SUN Cai-xin,WEN Jun-hao. Regulation Regression Local Forecasting Method of Multivariable Chaotic Time Series in Short-term Electrical Load [J]. Computer Science, 2010, 37(7): 220-224.
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[6] YANG Yu-qi, ZHANG Guo-an and JIN Xi-long. Dual-cluster-head Routing Protocol Based on Vehicle Density in VANETs[J]. Computer Science, 2018, 45(4): 126 -130 .
[7] SHI Chao, XIE Zai-peng, LIU Han and LV Xin. Optimization of Container Deployment Strategy Based on Stable Matching[J]. Computer Science, 2018, 45(4): 131 -136 .
[8] PANG Bo, JIN Qian-kun, HENIGULI·Wu Mai Er and QI Xing-bin. Routing Scheme Based on Network Slicing and ILP Model in SDN[J]. Computer Science, 2018, 45(4): 143 -147 .
[9] ZHENG Xiu-lin, SONG Hai-yan and FU Yi-peng. Distinguishing Attack of MORUS-1280-128[J]. Computer Science, 2018, 45(4): 152 -156 .
[10] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .