Computer Science ›› 2020, Vol. 47 ›› Issue (9): 105-109.doi: 10.11896/jsjkx.190800030

• Database & Big Data & Data Science • Previous Articles     Next Articles

Short Term Load Forecasting via Zoneout-based Multi-time Scale Recurrent Neural Network

ZHUANG Shi-jie, YU Zhi-yong, GUO Wen-zhong, HUANG Fang-wan   

  1. College of Mathematics and Computer Science,Fuzhou University,Fuzhou 350116,China
    Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing,Fuzhou University,Fuzhou 350116,China
  • Received:2019-08-07 Published:2020-09-10
  • About author:ZHUANG Shi-jie,born in 1995,undergraduate.His main research interests include computational intelligence,machine learning and deep learning.
    HUANG Fang-wan,born in 1980,senior lecturer,is a member of China Computer Federation.Her main research interests include computational intelligence,machine learning and big data analysis.
  • Supported by:
    National Natural Science Foundation of China (61772136,61672159) and Research Project for Young and Middle-aged Teachers of Fujian Province (JT180045).

Abstract: Because accurate power load forecasting,smart grids can provide more efficient,reliable and environmentally friendly power services than traditional grids.In real life,power load data often has a high temporal correlation with historical data,while traditional neural networks pay little attention to it.In recent years,the recurrent neural network (RNN) has received more and more attention in power load forecasting,because it can well capture the correlation between data with large cross-time scale.However,due to the unique self-connections structure of RNN,when the back-propagation through time(BPTT) is adopted for network training,the problems such as vanishing gradient are prone to occur with the number of network layers increases,resulting in a decrease in prediction accuracy.There are varieties of RNN architectures that can solve the vanishing gradient problem,such as long short-term memory (LSTM) and gated recurrent unit (GRU),but their complex internal structure will increase the training time.In order to solve the above problems,this paper first analyzes and studies RNN and itsvariants,and then combines the Zoneout function to design a multi-time scale modularized RNN architecture,focuses on the update strategy of hidden layer modules.It not only effectively solves the vanishing gradient problem,but also greatly reduces the number of network parameters that need to be trained.Experimental results based on the benchmark dataset and the real-worldload dataset show that this architecture can achieve better performance than the current popular RNN architecture.

Key words: Multi-time scale, Recurrent neural networks, Short term load forecast, Zoneout

CLC Number: 

  • TP183
[1] DU D,CHEN R,LI X,et al.Malicious data deception attacksagainst power systems:A new case and its detection method[J].Transactions of the Institute of Measurement and Control,2019,41(6):1590-1599.
[2] NATARAJA C,GORAWAR M B,SHILPA G N,et al.Shortterm load forecasting using time series analysis:a case study for Karnataka,India[J].Int.J.Eng.Sci.Innov.Technol,2012,1(2):45-53.
[3] HERNÁNDEZ L,BALADRÓN C,AGUIAR J,et al.Artificial neural network for short-term load forecasting in distribution systems[J].Energies,2014,7(3):1576-1598.
[4] SONG K B,BAEK Y S,HONG D H,et al.Short-term load forecasting for the holidays using fuzzy linear regression method[J].IEEE Transactions on Power Systems,2005,20(1):96-101.
[5] WANG Z,YANG F,HO D W C,et al.Stochastic dynamic mo-deling of short gene expression time-series data[J].IEEE Tran-sactions on Manobioscience,2008,7(1):44-55.
[6] WEI G,WANG Z,SHU H,et al.Robust filtering for gene expression time series data with variance constraints[J].International Journal of Computer Mathematics,2007,84(5):619-633.
[7] AL-HAMADI H M,SOLIMAN S A.Short-term electric load forecasting based on Kalman filtering algorithm with moving window weather and load model[J].Electric Power Systems Research,2004,68(1):47-59.
[8] RAHMAN S,BHATNAGAR R.An expert system based algorithm for short term load forecast[J].IEEE Transactions on Power Systems,1988,3(2):392-399.
[9] WEI G,FENG G,WANG Z.Robust Control for Discrete-Time Fuzzy Systems With Infinite-Distributed Delays[J].IEEE Transactions on Fuzzy Systems,2008,17(1):224-232.
[10] LEE C M,KO C N.Short-term load forecasting using liftingscheme and ARIMA models[J].Expert Systems with Applications,2011,38(5):5902-5911.
[11] LIU Y,WANG Z,LIU X.Asymptotic stability for neural networks with mixed time-delays:The discrete-time case[J].Neural Networks,2009,22(1):67-74.
[12] LIU L,SHEN B,WANG X.Research on kernel function of support vector machine[M]//Advanced Technologies,Embedded and Multimedia for Human-centric Computing.Springer,Dordrecht,2014:827-834.
[13] BAI Y.Design of Cluster Analysis Model Based on Load CharacteristicCurve of Power Consumers [J].Journal of Chongqing University of Technology(Natural Science),2018, 32(12):181-185.
[14] HIPPERT H S,PEDREIRA C E,SOUZA R C.Neural networks for short-term load forecasting:A review and evaluation[J].IEEE Transactions on Power Systems,2001,16(1):44-55.
[15] LÄNGKVIST M,KARLSSON L,LOUTFI A.A review of unsupervised feature learning and deep learning for time-series modeling[J].Pattern Recognition Letters,2014,42:11-24.
[16] BENGIO Y,SIMARD P,FRASCONI P.Learning long-term dependencies with gradient descent is difficult[J].IEEE Transactions on Neural Networks,1994,5(2):157-166.
[17] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[18] CHUNG J,GULCEHRE C,CHO K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[J].arXiv:1412.3555,2014.
[19] KOUTNIK J,GREFF K,GOMEZ F,et al.A clockwork rnn[J].arXiv:1402.3511,2014.
[20] CHANG S,ZHANG Y,HAN W,et al.Dilated recurrent neural networks[C]//Advances in Neural Information Processing Systems.2017:77-87.
[21] LI S,LI W,COOK C,et al.Independently recurrent neural network (indrnn):Building a longer and deeper rnn[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5457-5466.
[22] KRUEGER D,MAHARAJ T,KRAMÁR J,et al.Zoneout:Re-gularizing rnns by randomly preserving hidden activations[J].arXiv:1606.01305,2016.
[1] JING Jie, CHEN Tan, DU Wen-li, LIU Zhi-kang, YIN Hao. Spatio-temporal Features Extraction of Traffic Based on Deep Neural Network [J]. Computer Science, 2019, 46(11A): 1-4.
[2] ZENG An, NIE Wen-jun. Stock Recommendation System Based on Deep Bidirectional LSTM [J]. Computer Science, 2019, 46(10): 84-89.
[3] PANG Chao and YIN Chuan-huan. Chinese Text Summarization Based on Classification [J]. Computer Science, 2018, 45(1): 144-147.
[4] DAI Xiao-hong,WANG Guang-li. Application of L-M Optimized BP Algorithm in Short-term Power Load Forecast [J]. Computer Science, 2011, 38(7): 265-267.
[5] REN Hai-jun,ZHANG Xiao-xing,SUN Cai-xin,WEN Jun-hao. Regulation Regression Local Forecasting Method of Multivariable Chaotic Time Series in Short-term Electrical Load [J]. Computer Science, 2010, 37(7): 220-224.
Full text



No Suggested Reading articles found!