Computer Science ›› 2016, Vol. 43 ›› Issue (12): 125-129.doi: 10.11896/j.issn.1002-137X.2016.12.022

Previous Articles     Next Articles

Experimental Research on Effects of Random Weight Distributions on Performance of Extreme Learning Machine

ZHAI Jun-hai, ZANG Li-guang and ZHANG Su-fang   

  • Online:2018-12-01 Published:2018-12-01

Abstract: Extreme learning machine (ELM) is an algorithm for training single-hidden layer feed-forward neural networks (SLFNs).ELM firstly employs randomization method to generate the input weights and hidden nodes biases,and then determines the output weights analytically.ELM has fast learning speed and good generalization ability.All metho-ds published in literatures usually initialize the weights of input layer and biases of hidden nodes with a uniform distribution over the interval [-1,1].However,there are no studies on the rationality of this setting in literatures.This paper investigated this problem by experimental approach.Specifically,the effects of random weights with uniform distribution,Gaussian distribution and exponential distribution were studied.We found that the random weight distributions do have impact on the performance of the extreme learning machine.For different problems or different data sets,the random weights with uniform distribution in [-1,1] are not necessarily optimal.The results of this paper can be used for reference by the researchers engaged in the study of ELM.

Key words: Random weight distributions,Extreme learning machine,Uniform distribution,Gaussian distribution,Exponential distribution

[1] Huang Guang-bin,Zhu Qin-yu,Siew C K.Extreme learning machine:A new learning scheme of feedforward neural networks [C]∥Proceedings of International Joint Conference on Neural Networks(IJCNN2004).2004:985-990
[2] Huang Guang-bin,Wang Dian-hui,Lan Yuan.Extreme learning machines:a survey [J].International Journal of Machine Lear-ning and Cybernetics,2011,2(2):107-122
[3] Huang Guang-bin,Zhu Qin-yu,Siew C K.Extreme learning machine:Theory and applications [J].Neurocomputing,2006,70(1-3):489-501
[4] Huang Guang-bin,Chen L,Siew C K.Universal approximation using incremental constructive feedforward networks with random hidden nodes [J].IEEE Transactions on Neural Networks,2006,17(4):879-892
[5] Huang Guang-bin,Chen Lei.Enhanced random search based incremental extreme learning machine [J].Neurocomputing,2008,71(16-18):3460-3468
[6] Huang Guang-bin,Chen Lei.Convex incremental extreme lear-ning machine [J].Neurocomputing,2007,70(16):3056-3062
[7] Liang Nan-ning,Huang Guang-bin,Saratchandran P,et al.Afast and accurate on-line sequential learning algorithm for feedforward networks [J].IEEE Transactions on Neural Networks,2006,7(6):1411-1423
[8] Liu Qiu-ge,He Qing,Shi Zhong-zhi.Extreme support vectormachine classifier [C]∥ Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining.Springer-Verlag.2008:222-233
[9] Huang Guang-bin,Ding Xiao-jian,Zhou Hong-ming.Optimiza-tion method based extreme learning machine for classification [J].Neurocomputing,2010,74(1-3):155-163
[10] Huang Guang-bin,Zhou Hong-ming,Ding Xiao-jian,et al.Ex-treme Learning Machine for Regression and Multiclass Classification [J].IEEE Transactions on Systems,Man,and Cyberne-tics,Part B,2012,42(2):513-529
[11] Emilio S O,Juan G S,Martín J D,et al.BELM:Bayesian Extreme Learning Machine [J].IEEE Transactions on Neural Networks,2011,22(3):505-509
[12] Feng Guo-rui,Huang Guang-bin,Lin Qing-ping,et al.ErrorMinimized extreme learning machine with growth of hidden nodes and incremental learning [J].IEEE Trans.Neural Networks,2009,20(8):1352-1357
[13] Rong Hai-jun,Ong Y S,Tan A H,et al.A fast pruned-extreme learning machine for classification problem [J].Neurocompu-ting,2008,72(1-3):359-366
[14] Miche Y,Sorjamaa A,Bas P,et al.OP-ELM:Optimally pruned extreme lear-ning machine [J].IEEE Transactions on Neural Networks,2010,21(1):158-162
[15] Mohammed A A,Minhas R,Jonathan Q M,et al.Human face recognition based on multidimensional PCA and extreme lear-ning machine [J].Pattern Recognition,2011,44(10/11):2588-2597
[16] Iosifidis A,Tefas A,Pitas I.Minimum Class Variance Extreme Learning Machine for Human Action Recognition [J].IEEE Transactions on Circuits and Systems for Video Technology,2013,23(11):1968-1979
[17] Frank A,Asuncion A.UCI machine learning repository .http://archive.ics.uci.edu/ml
[18] Zhai Jun-hai,Li Ta,Zhai Meng-yao,et al.Experimental Re-search on Random Mapping Function in ELM Algorithm [J].Computer Engineering,2012,38(20):164-168(in Chinese) 翟俊海,李塔,翟梦尧,等.ELM中随机映射作用的实验研究[J].计算机工程,2012,38(20):164-168

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!