计算机科学 ›› 2015, Vol. 42 ›› Issue (2): 224-227.doi: 10.11896/j.issn.1002-137X.2015.02.046

• 人工智能 • 上一篇    下一篇

基于AP二次聚类的神经网络集成算法研究

李辉,丁世飞   

  1. 中国矿业大学计算机科学与技术学院 徐州221116;江苏师范大学计算机科学与技术学院 徐州221116,中国矿业大学计算机科学与技术学院 徐州221116;中国科学院智能信息处理重点实验室 北京100190
  • 出版日期:2018-11-14 发布日期:2018-11-14
  • 基金资助:
    本文受国家重点基础研究计划(973计划)项目(2013CB329502),国家自然科学基金(41074003,61379101)资助

AP Twice Clustering Based Neural Network Ensemble Algorithm

LI Hui and DING Shi-fei   

  • Online:2018-11-14 Published:2018-11-14

摘要: 为了提高个体神经网络精度及差异度进而提高神经网络集成(Neural Network Ensemble,NNE)的泛化性能,提出了一种基于二次聚类的神经网络集成方法。首先对所有样本进行聚类,得到第一次聚类样本子集;然后对每一类样本子集进行二次聚类,得到每一子类的样本子集,通过Affinity Propagation(AP)聚类使得“类内相似,类间相异”的准则最大化,类内样本能够反映真实的数据分布;最后按照排列组合的方式,从二次聚类的每个样本子集中选取一类样本构成训练集来构造一个个体神经网络。这样从不同类中选择样本集构造的个体神经网络差异性较大,既能使数据的规模较小,又能反映真实的数据分布,用这种方法产生的个体神经网络进行集成具有较高的性能。仿真实验表明,该方法能够取得较好的性能。

关键词: Affinity Propagation(AP)聚类,神经网络集成,二次聚类,个体神经网络

Abstract: In order to improve the precision and diversity of individual networks to improve the generalization perfor-mance of neural network ensemble (NNE),this paper proposed a method to generate individual neural network for ensembling based on twice clustering.Firstly,all the samples are chosen to cluster for the first time to form once clustering subclass,and then the twice clustering is performed for each type of subclass to form the sample subsets of each subclass.Affinity Propagation(AP) clustering makes the otherness criterion of “similar in classes,diversity between classes” maxization,and the samples in the class can response the real data distribution.Finally,according to the permutation and combination a subset is selected from each twice clustering of each subclass to construct a trainset to train an individual neural network.So the trainset with the smaller size of the data and the real data distribution can train the individual neural networks with the bigger diversity,and the ensemble of these individual neural networks can get better performance.Simulation experiments show that our proposed method here is effective.

Key words: Affinity propagation(AP) clustering,Neural network ensemble(NNE),Twice clustering,Individual neural network

[1] Hansen L K,Salamon P.Neural network ensemble[J].IEEETrans on Pattern Analysis and Machine Intelligence,1990,12(10):993-1001
[2] Valentini G,Masulli F.Ensembles of learning machines[C]∥Lecture Notes in Computer Sciences.Heidelberg,Germany:SpringerVerlag,2002
[3] Yu Shi-xin.Feature Selection and Classifier Ensembles:A study on hyperspectral remote sensing data.2003.http://143.129.203.3/visielab/theses/shixin/thesis_yu.pdf
[4] Tesauro G,Touretzky D S,Leen T K,et al.Neural network ensembles,Crosses validation,and active learning[C]∥Advances in Neural Information Processing System.Cambridge,MA:MIT Press,1995:231-238
[5] Zhou Zhi-hua,Chen Shi-fu.Neural network ensemble[J].Chinese Journal of Computers,2002,25(1):1-8 (下转第252页)(上接第227页)
[6] Li Hui,Ding Shi-fei.Research of Individual Neural NetworkGeneration and Ensemble Algorithm Based on Quotient Space Granularity Clustering[J].Applied Mathematics & Information Sciences,2013,7(2):701-708
[7] Xu Xin-zheng,Ding Shi-fei,Jia Wei-kuan,et al.Research of assembling optimized classification algorithm by neural network based on Ordinary Least Squares (OLS) [J].Neural Comput & Applic,2013(22):187-193
[8] Schapire R E.The strength of weak 1earnability[J].Machine Learning,1990,5(2):197-227
[9] Breiman L.Bagging predictors[J].Machine Learning,1996,24(2):123-140
[10] Frey B J,Dueck D.Clustering by passing messages between data points[J].Science,2007,315(5814):972-976
[11] Liu Xiao-yong,Fu Hui.A fast affinity propagation clustering algorithm[J].Journal of Shandong University (Engineering Science),2011,41(4):20-28
[12] Xia Ding-yin,Wu Fei,Zhang Xu-qing,et al.Local and global ap-proaches of affinity propagation clustering for large scale data[J].Journal of Zhejiang University-Science A,2008,9(10):1373-1381
[13] Lazic N,Givoni I,Aarabi P,et al.FLoSS:Facility location for subspace segmentation[C]∥Proceedings of 12th InternationalConference on Computer Vision ( ICCV ).Kyoto:IEEE Press,2009:825-832
[14] Lazic N,Frey B J,Aarabi P.Solving the uncapacitated facility location problem using message passing algorithms[C]∥Procee-dings of 13th International Conference on Artificial Intelligence and Statistics(AISTATS).Sardinia:Microtome Publishing,2010:429-436
[15] Tang Dong-ming,Zhu Qing-xin,Yang Fan,et al.Solving largescale location problem using affinity propagation clustering[J].Application Research of Computers,2010,27(3):841-844
[16] Dueck D,Frey B J.Nonmetric affinity propagation for unsupervised image categorization[C]∥Proceedings of 11th International Conference on Computer Vision (ICCV).Riode Janeiro:IEEE Press,2007:1-8
[17] Zhang Ren-yan,Zhao Hong-liang,Lu Xiao,et al.Grey imagesegmentation method based on affinity propagation clustering[J].Journal of Naval University of Engineering,2009,21(3):33-37
[18] Givoni I E,Frey B J.Semi-supervised affinity propagation with instance-level constraints[C]∥Proceedings of 12th International Conference on Artificial Intelligence and Statistics (AISTATS).Florida:Microtome Publishing,2009:161-168
[19] Guan Ren-chu,Pei Zhi-li,Shi Xiao-hu,et al.Weight affinity progagation and its application to text clustering[J].Journal of Computer Research and Development,2010,47(10):1733-1740
[20] Dueck D,Frey B J,Jojic N,et al.Constructing treatment portfolios using affinity propagation[C]∥Proceedings of International Conference on Research in Computational Molecular Biology (RECOMB).Singapore:Springer,2008:360-371
[21] Xu Wen-zhu,Xu Li-hong.Adaptive key-frame extraction based on affinity propagation clustering[J].Computer Science,2010(1):268-270
[22] Xiang Pei-su.New CBIR system based on the affinity propagation clustering algorithm[J].Journal of Southwest University for Nationalities:Natural Science Edition,2010,36(4):624-627
[23] Wang Kai-jun,Zhang Jun-ying,Li Dan,et al.Adaptive afinitypropagation clustering[J].Acta Automatica Sinica,2007,33(12):1242-1246
[24] Wang Kai-jun,Li Jian,Zhang Jun-ying,et al.Semi-supervised affinity propagation clustering[J].Computer Engineering,2007,33(23):197-198,201
[25] Wang Kai-jun,Zheng Jie.Fast algorithm of affinity propagation clustering under given number of clusters[J].Computer Systems & Applications,2010,19(7):207-209
[26] Xie Xin-xi,Wang Shi-tong.Affinity propagation clustering forsymbolic interval data based on mutual distance[J].Computer Application,2008,28(6):1441-1443
[27] Xiao Yu,Yu Jian.Semi-Supervised clustering based on affinity propagation algorithm[J].Journal of Software,2008,19(11):2803-2813
[28] Gu Rui-jun,Wang Jia-cai,Chen Geng,et al.Affinity propagation clustering for large scale dataset[J].Computer Engineering,2010,36(23):22-24
[29] Dong Jun,Wang Suo-ping,Xiong Fan-lun.Affinity propagationclustering based on variable-similarity measure[J].Journal of Electronics & Information Technology,2010,32(3):509-514
[30] Li Ya-qin,Yang Hui-zhong.Multi-model modeling method basedon affinity propagation clustering and Gaussian processes[J].Computers and Applied Chemistry,2010,27(1):51-54
[31] Frey B J,Dueck D.Clustering by passing messages between data points[J].Science,2007,315 (5814):972-976
[32] Lin J,Zhu B Z.Improved Principal Component Analysis andNeural Network Ensemble Based Economic Forecasting[J].Lecture Notes in Computer Science,2006,4113:135-145
[33] Lin Jian,Peng Min-jing.GDP forecasting odel based on neuralnetworks ensemble[J].Chinese Journal of Management,2005,2(4):434-436
[34] LinJian,Zhu Bang-zhu.Neural network ensemble based onforecasting effective measure and its application[J].Journal of Computational Information Systems,2005,1(4):781-787
[35] Perrone M P,Cooper L N.When Networks Disagree:Ensemblemethod for neural networks[C]∥Mammone R J,eds.Artificial Neural Networks for Speed and Vision.New Yorks:Chapman & Hall,1993:126-142
[36] Li Ling-ling,Liu Xi-yu,Lu Shu-qiang.Constructive methods for parallel learning neural network ensemble based on particle swarm optimization[J].ShanDong Science,2007,20(4):16-20
[37] Opitz D,Vedelsby J.Actively searching for an efficient neural network ensemble[J].Connection Science,1996,8(3/4):337-353

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!