Computer Science ›› 2017, Vol. 44 ›› Issue (11): 284-288.doi: 10.11896/j.issn.1002-137X.2017.11.043

Previous Articles     Next Articles

h-ACOR:An ACOR Algorithm with Heuristic Information for Neural Network Training

ZHAO Zhang-ming, FENG Jing, SHI En and SHU Xiao-cun   

  • Online:2018-12-01 Published:2018-12-01

Abstract: The ACOR algorithm is an ant colony optimization(ACO) algorithm for real-valued optimization.The ACOR can be used for training neural network.Unlike most of the conventional ACO algorithms,ACOR does not consider heuristic information when training neural networks.So in this work,a new algorithm named h-ACOR that incorporates the heuristic information into the framework of ACOR was proposed for neural network training.The heuristic information in h-ACOR is a gradient vector,which is obtained by computing the partial derivative of error term of the neural network with respect to weight vector.Using 10-fold cross-validation method,h-ACOR is applied to train neural networks for pattern classification problems of zoo,iris and tic-tac-toe in UCI datasets.Compared with ACOR,h-ACOR can reduce classification errors while speeding up the convergence process,with the average training generations of h-ACOR being nearly 1/2 of that of ACOR.After completely training by h-ACOR,the classification accuracy about zoo,iris and tic-tac-toe are respectively 91.1%,93.3% and 95.6%,which have better performance than that of ACOR’s 83.1%,88.7% and 91.9%.

Key words: ACO,Heuristic information,Artificial neural network,Neural network training

[1] HAYKIN S.Neural Networks and Learning Machines(3rd ed)[M].New York,NY,USA:Pearson Education,2009.
[2] HORNIK K,STINCHCOMBE M,WHITE H.Mutilayer feed-forward networks are universal approximators[J].Neural Networks,1989,2(5):359-366.
[3] RUMELHART D,DHINTON G,WILLIAMS J.Learning representations by back propagating errors[J].Nature,1986,323(6088):533-536.
[4] GAO H B,GAO L,ZHOU C,et al.Particle Swarm Optimization Based Algorithm for Neural Network Learning[J].Acta Electronica Sinica,2004,32(9):1572-1574.(in Chinese) 高海兵,高亮,周驰,等.基于粒子群优化的神经网络训练算法研究[J].电子学报,2004,32(9):1572-1574.
[5] DORIGO M,BIRATTARI M,STIITZLE T.Ant Colony Optimization:Artificial Ants as a Computational Intelligence Technique[J].IEEE Computational Intellgence Magazine,2006,1(4):28-39.
[6] SOCHA K,DORIGO M.Ant Optimizationn for Continuous Domains[J].European Journal of Operational Research,2008,185:1155-1173.
[7] SOCHA K,BLUM C.An ant Colony Optimization Algorithm for Continuous Optimization:Application to Feed-Forward Neural Network Training[J].Neural Computing & Applications,2007,16(3):235-247.
[8] WRITTEN I H,FRANK E.Data Mining-Practical MachineLearning Tools and Techniques(3rd ed)[M].Morgan Kaufmann Publishers,2012.
[9] LI Y M,WANG W J,XU Z B.About ACO Algorithm For Hard Combinatorial Optimization Problem[J].Computer Science,2002,29(3):115-118.(in Chinese) 李有梅,王文剑,徐宗本.关于求解难组合优化问题的蚁群优化算法[J].计算机科学,2002,29(3):115-118.
[10] XIA Y M,CHENG B,CHEN J L,et al.Optimizing Services Composition Based on Improved Ant Colony Algorithm[J].Chinese Journal of Computers,2012,35(2):270-281.(in Chinese) 夏亚梅,程渤,陈俊亮,等.基于改进蚁群算法的服务组合优化[J].计算机学报,2012,35(2):270-281.
[11] RAZAVI S,TOLSON B A.A New Formulation for Feedfor-ward Neural Networks[J].IEEE Transactions on Neural Networks,2011,22(10):1588-1598.
[12] WU X,WILAMOWSKI B M.Advantage Analysis of Sigmoid Based RBF Networks[C]∥IEEE 17th International Conference on Intelligent Engineering Systems.Costa Rica,2013.
[13] MULLER B.A note on the generation of random normal de-viates[J].Annals of Mathematical Statistics,1958,29:610-611.
[14] SAMPAIO L H D,ADANIYA M H A C, DE PAULA MARQUES M,et al.Ant colony optimization for resource allocation and anomaly detection in communication networks[M].INTECH Open Access Publisher,2013.
[15] KARIMI A,NOBAHARI H,SIARRY P.Continuous ant colony system and tabu search algorithms hybridized for global minimization of continuous multi-minima functions[J].Computational Optimization and Applications,2010,45(3):639-661.
[16] HUANG C L,HUANG W C,CHANG H Y,et al.Hybridization strategies for continuous ant colony optimization and particle swarm optimization applied to data clustering[J].Applied Soft Computing,2013,13(9):3864-3872.
[17] BACHE K,LICHMAN M.UCI machine learning repository.URL:,2013.
[18] LIAO T,DORIGO M.Ant colony optimization for mixed-variable optimization problems[J].IEEE Transactions on Evolutio-nary Computation,2014,18(4):503-518.

No related articles found!
Full text



No Suggested Reading articles found!