基于互信息的RBF神经网络结构优化设计

计算机科学 ›› 2013, Vol. 40 ›› Issue (6): 252-255.

基于互信息的RBF神经网络结构优化设计

郭伟

辽宁工程技术大学电子与信息工程学院葫芦岛125105

出版日期:2018-11-16 发布日期:2018-11-16
基金资助:
本文受国家自然科学基金(60971048),辽宁省科技厅基金(2009S051)资助

Structural Optimization Algorithm for RBF Neural Network Based on Mutual Information

GUO Wei

Online:2018-11-16 Published:2018-11-16

摘要/Abstract

摘要： 以设计最小RBF网络结构为着眼点,提出了一种基于互信息的RBF神经网络结构优化算法。该算法用k近邻统计法估计隐节点输出矩阵与输出节点输出矩阵之间的互信息,获得每个隐节点与输出节点之间的相关性度量,删除相关性最小的隐节点,进而达到优化网络结构的目的。该算法具有自恢复机制,在简化网络结构的同时能有效保证网络的信息处理能力。在人工数据集和真实基准数据集上的仿真实验验证了该算法的有效性与稳定性。

关键词: RBF神经网络,结构优化,k-近邻统计法,互信息

Abstract: Aiming at designing the simplest RBF neural network architecture,a RBF neural network structure design algorithm based on mutual information was proposed in this paper．The relevance measure between each hidden and output units can be acquired by estimating the mutual information between the output matrix of the hidden unit and output unit,using k-nearest-neighbor statistics．And the simplest RBF neural network architecture can be achieved by removing the least related hidden units from the trained neural network one after another according to the relevance measure．This algorithm has the self-recovery mechanism,and the information processing capacity of the neural network can be ensured in the process of the simplification of the network’s architecture．The simulation results on the artificial datasets and the real-world benchmark datasets show the effectiveness and stability of the algorithm．

Key words: RBF neural network,Structural optimization,k-nearest-neighbor statistics,Mutual information

郭伟. 基于互信息的RBF神经网络结构优化设计[J]. 计算机科学, 2013, 40(6): 252-255. https://doi.org/

GUO Wei. Structural Optimization Algorithm for RBF Neural Network Based on Mutual Information[J]. Computer Science, 2013, 40(6): 252-255. https://doi.org/

参考文献

[1] 姜慧研,宗茂,刘相莹．基于ACO-SVM的软件缺陷预测模型的研究[J]．计算机学报,2011,34(6):1148-1154
[2] Chen S,Wang X X,Brown D J．Sparse incremental regression modeling using correlation criterion with boosting search[J]．IEEE Signal Processing Letters,2005,2(3):198-201
[3] 孟锦,马驰,何家浪,等．基于HHGA-RBF神经网络的网络安全态势预测模型[J]．计算机科学,2011,38(7):70-75
[4] Lu Ying-wei,Sundararajan N,Saratchandran P．A sequentiallearning scheme for function approximation using minimal radial basis function (RBF) neural networks[J]．Neural Computation,1997,9:461-478
[5] Panchapakesan C,Palaniswami M,Ralph D,et al．Effects ofmoving the centers in an RBF network[J]．IEEE Transaction on Neural Networks,2002,3(6):1299-1307
[6] Feng H M．Self-generation RBFNs using evolutional PSO lear-ning[J]．Neurocomputing,2006,0(13):241-251
[7] Huang De-shuang,Du Ji-xiang．A constructive hybrid structure optimization methodology for radial basis probabilistic neural networks[J]．IEEE Transaction on Neural Networks,2008,9(12):2099-2115
[8] Chen Sheng,Hong Xia,Luk B L,et al．Construction of tunable radial basis function networks using orthogonal forward selection[J]．IEEE Transaction on Systems,Man,and Cybernetics—Part B:Cybernetics,2009,9(2):457-466
[9] Stogbauer H,Kraskov A,Astakhov S A,et al．Least dependent component analysis based on mutual information [J]．Physical Review E,2004,0(6):066123
[10] Xing Hong-jie,Hu Bao-gang．Two-Phase Construction of Multilayer Perceptions Using Information Theory[J]．IEEE Transactions on Neural Networks,2009,0(4):542-550
[11] Rossi F,Lendasse A,Francois D,et al．Mutual information for the selection of relevant variables in spectrometric nonlinear modeling[J]．Chemometrics and Interlligent Laboratory Systems,2006,80:215-226
[12] Huang Guang-bin,et al．A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation[J]．IEEE Transaction on Neural Networks,2005,16(1):57-67
[13] Xu Jin-hua,Ho D W C．A new training and pruning algorithm based on node dependence and Jacobian rank deficiency[J]．Neurocomputing,2006,70(1-3):544-558
[14] Ridella A,Rovetta S,Zunino R．Circular back propagation forclassification[J]．IEEE Transaction on Neural Networks,1997,8(1):84-97

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed