带启发信息的蚁群神经网络训练算法

doi:10.11896/j.issn.1002-137X.2017.11.043

计算机科学 ›› 2017, Vol. 44 ›› Issue (11): 284-288.doi: 10.11896/j.issn.1002-137X.2017.11.043

带启发信息的蚁群神经网络训练算法

赵章明,冯径,施恩,舒晓村

国防科技大学气象海洋学院南京211101,国防科技大学气象海洋学院南京211101,国防科技大学气象海洋学院南京211101,国防科技大学气象海洋学院南京211101

出版日期:2018-12-01 发布日期:2018-12-01
基金资助:
本文受国家自然科学基金(61371119)资助

h-ACOR:An ACOR Algorithm with Heuristic Information for Neural Network Training

ZHAO Zhang-ming, FENG Jing, SHI En and SHU Xiao-cun

Online:2018-12-01 Published:2018-12-01

摘要/Abstract

摘要： ACOR算法是一种应用于连续域实值优化的蚁群算法(Ant Colony Optimization,ACO)。ACOR算法可用于训练神经网络。与常规的蚁群算法不同,ACOR在训练神经网络时未考虑启发式信息(Heuristic Information)。在ACOR的基础上,提出了一种将启发式信息与ACOR相结合的神经网络训练算法——h-ACOR。其中,启发式信息是通过计算神经网络的误差关于网络的权值向量的偏导数而得到的梯度向量。通过十折交叉验证方法,将h-ACOR 应用于UCI数据集中的zoo,iris和tic-tac-toe 3组数据的模式分类问题中来训练神经网络。与ACOR相比,h-ACOR算法在减小分类误差的同时能够提高收敛速度,其收敛的代数约为ACOR算法的1/2,且经过完全训练,对zoo,iris和tic-tac-toe 3组数据的分类准确率分别为91.1%,93.3%和95.6%,高于ACOR算法的83.1%,88.7%和91.9%。

关键词: 蚁群算法,启发式信息,人工神经网络,神经网络训练

Abstract: The ACOR algorithm is an ant colony optimization(ACO) algorithm for real-valued optimization.The ACOR can be used for training neural network.Unlike most of the conventional ACO algorithms,ACOR does not consider heuristic information when training neural networks.So in this work,a new algorithm named h-ACOR that incorporates the heuristic information into the framework of ACOR was proposed for neural network training.The heuristic information in h-ACOR is a gradient vector,which is obtained by computing the partial derivative of error term of the neural network with respect to weight vector.Using 10-fold cross-validation method,h-ACOR is applied to train neural networks for pattern classification problems of zoo,iris and tic-tac-toe in UCI datasets.Compared with ACOR,h-ACOR can reduce classification errors while speeding up the convergence process,with the average training generations of h-ACOR being nearly 1/2 of that of ACOR.After completely training by h-ACOR,the classification accuracy about zoo,iris and tic-tac-toe are respectively 91.1%,93.3% and 95.6%,which have better performance than that of ACOR’s 83.1%,88.7% and 91.9%.

Key words: ACO,Heuristic information,Artificial neural network,Neural network training

赵章明,冯径,施恩,舒晓村. 带启发信息的蚁群神经网络训练算法[J]. 计算机科学, 2017, 44(11): 284-288. https://doi.org/10.11896/j.issn.1002-137X.2017.11.043

ZHAO Zhang-ming, FENG Jing, SHI En and SHU Xiao-cun. h-ACOR:An ACOR Algorithm with Heuristic Information for Neural Network Training[J]. Computer Science, 2017, 44(11): 284-288. https://doi.org/10.11896/j.issn.1002-137X.2017.11.043

参考文献

[1] HAYKIN S.Neural Networks and Learning Machines(3rd ed)[M].New York,NY,USA:Pearson Education,2009.
[2] HORNIK K,STINCHCOMBE M,WHITE H.Mutilayer feed-forward networks are universal approximators[J].Neural Networks,1989,2(5):359-366.
[3] RUMELHART D,DHINTON G,WILLIAMS J.Learning representations by back propagating errors[J].Nature,1986,323(6088):533-536.
[4] GAO H B,GAO L,ZHOU C,et al.Particle Swarm Optimization Based Algorithm for Neural Network Learning[J].Acta Electronica Sinica,2004,32(9):1572-1574.(in Chinese) 高海兵,高亮,周驰,等.基于粒子群优化的神经网络训练算法研究[J].电子学报,2004,32(9):1572-1574.
[5] DORIGO M,BIRATTARI M,STIITZLE T.Ant Colony Optimization:Artificial Ants as a Computational Intelligence Technique[J].IEEE Computational Intellgence Magazine,2006,1(4):28-39.
[6] SOCHA K,DORIGO M.Ant Optimizationn for Continuous Domains[J].European Journal of Operational Research,2008,185:1155-1173.
[7] SOCHA K,BLUM C.An ant Colony Optimization Algorithm for Continuous Optimization:Application to Feed-Forward Neural Network Training[J].Neural Computing & Applications,2007,16(3):235-247.
[8] WRITTEN I H,FRANK E.Data Mining-Practical MachineLearning Tools and Techniques(3rd ed)[M].Morgan Kaufmann Publishers,2012.
[9] LI Y M,WANG W J,XU Z B.About ACO Algorithm For Hard Combinatorial Optimization Problem[J].Computer Science,2002,29(3):115-118.(in Chinese) 李有梅,王文剑,徐宗本.关于求解难组合优化问题的蚁群优化算法[J].计算机科学,2002,29(3):115-118.
[10] XIA Y M,CHENG B,CHEN J L,et al.Optimizing Services Composition Based on Improved Ant Colony Algorithm[J].Chinese Journal of Computers,2012,35(2):270-281.(in Chinese) 夏亚梅,程渤,陈俊亮,等.基于改进蚁群算法的服务组合优化[J].计算机学报,2012,35(2):270-281.
[11] RAZAVI S,TOLSON B A.A New Formulation for Feedfor-ward Neural Networks[J].IEEE Transactions on Neural Networks,2011,22(10):1588-1598.
[12] WU X,WILAMOWSKI B M.Advantage Analysis of Sigmoid Based RBF Networks[C]∥IEEE 17th International Conference on Intelligent Engineering Systems.Costa Rica,2013.
[13] MULLER B.A note on the generation of random normal de-viates[J].Annals of Mathematical Statistics,1958,29:610-611.
[14] SAMPAIO L H D,ADANIYA M H A C, DE PAULA MARQUES M,et al.Ant colony optimization for resource allocation and anomaly detection in communication networks[M].INTECH Open Access Publisher,2013.
[15] KARIMI A,NOBAHARI H,SIARRY P.Continuous ant colony system and tabu search algorithms hybridized for global minimization of continuous multi-minima functions[J].Computational Optimization and Applications,2010,45(3):639-661.
[16] HUANG C L,HUANG W C,CHANG H Y,et al.Hybridization strategies for continuous ant colony optimization and particle swarm optimization applied to data clustering[J].Applied Soft Computing,2013,13(9):3864-3872.
[17] BACHE K,LICHMAN M.UCI machine learning repository.URL:http://archive.ics.uci.edu/ml,2013.
[18] LIAO T,DORIGO M.Ant colony optimization for mixed-variable optimization problems[J].IEEE Transactions on Evolutio-nary Computation,2014,18(4):503-518.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

带启发信息的蚁群神经网络训练算法

h-ACOR:An ACOR Algorithm with Heuristic Information for Neural Network Training

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

Metrics

本文评价

推荐阅读 0