计算机科学 ›› 2013, Vol. 40 ›› Issue (8): 249-251.

• 人工智能 • 上一篇    下一篇

XCSG在多机器人强化学习中的应用

邵杰,杜丽娟,杨静宇   

  1. 郑州成功财经学院信息工程系 郑州451200;商丘工学院信息与电子学院 商丘476000;南京理工大学计算机科学与技术学院 南京210094
  • 出版日期:2018-11-16 发布日期:2018-11-16
  • 基金资助:
    本文受国家自然科学基金(90820004)资助

Applications of XCSG in Multi-robot Reinforcement Learning

SHAO Jie,DU Li-juan and YANG Jing-yu   

  • Online:2018-11-16 Published:2018-11-16

摘要: XCS分类器在解决机器人强化学习方面已显示出较强的能力,但在多机器人领域仅局限于MDP环境,只能解决环境空间较小的学习问题。提出了XCSG来解决多机器人的强化学习问题。XCSG建立低维的逼近函数,梯度下降技术利用在线知识建立稳定的逼近函数,使Q-表格一直保持在稳定低维状态。逼近函数Q不仅所需的存储空间更小,而且允许机器人在线对已获得的知识进行归纳一般化。仿真实验表明,XCSG算法很好地解决了多机器人学习空间大、学习速度慢、学习效果不确定等问题。

关键词: 强化学习,多机器人,学习分类器,梯度下降法的学习分类器

Abstract: XCS classifier system has been shown to solve machine-learning problems in a competitive way.However,in multi-robot problems,XCS is restricted to solve very small problems modeled by a Markov decision process.In this paper a new learning technique XCSG that combines XCS and gradient descent methods was proposed to solve multi-robot machine-learning problems.XCSG builds low-dimensional approximation of the function,and gradient descent techniques use on-line knowledge to establish a stable approximation of functions,so that the Q-form has been maintained at a low-dimensional stable state.Approximate of the function not only requires smaller storage space,but also allows the robot online knowledge is summarized on the generalization.Simulation results show that XCSG algorithm solves the multi-robot reinforcement learning in a large space,slow learning,learning uncertainty and other issues.

Key words: Reinforcement learning,Multi-robot,Accuracy-based learning classifier system(XCS),Accuracy-based learning classifier system with gradient descent method(XCSG)

[1] 邵杰,杨静宇.基于多LCS和人工势场法的机器人行为控制[J].计算机科学,2011,8(1):264-267
[2] 朱美强,程玉虎,李明,等.一类基于谱方法的强化学习混合迁移算法[J].自动化学报,2012,38(11):1765-1776
[3] Wiering M.Multi-agent reinforcement learning for traffic light control[C]∥Proc.17th Int.Conf.Mach.Learn.(ICML-00).Stanford Univ.Stanford,CA,2009:1151-1158
[4] Dixon P W,Corne D W,Oates M J.Apreliminary investigation of modified XCS as a generic data mining tool[C]∥Lanzi P L,Stolzmann W,Wilson S W,eds.LNAI,Advances in Learning Classifier Systems.vol.2321,Berlin,Germany:Springer-Verlag,2002:133-150
[5] 欧世峰,高颖,赵晓晖.基于随机梯度的变动量因子自适应白化算法[J].自动化学报,2012,38(8):1370-1374
[6] Butz M V,Goldberg D E,Lanzi P L.Gradient descent methods in learning classifier systems:Improving XCS performance in multistep problems[J].IEEE Trans.Evol.Comput.,2005,9(5):452-473
[7] Bernad E,o-Mansilla,Garrell J.Accuracy-based Learning Classifier Systems:Models,analysis and applications to classification tasks[J].Evolutionar Computation,2003,11(3):209-238
[8] Hung K-T,Liu J-S,Chang Y-Z.Smooth path planning for a mobile robot by evolutionary multiobjective optimization[C]∥IEEE Int.Symposium on Computational Intelligence in Robotics and Automation.Jacksonville,Florida,June 2007
[9] Butz M V,Lanzi P L,Wilson S W.Function approximation with XCS:Hyperellipsoidal conditions,recursive least squares,and compaction[J].IEEE Trans.Evol.Comput.,2008,12(3):355-376
[10] Bagnall A J,Cawley G C.Learning classifier systems for data mining:A comparison of XCS with other classifiers for the Fo-rest Cover dataset[C]∥Proc.IEEE/INNS Int.Joint Conf.Artificial Neural Netw.vol.3,Portland,OR,2003:1802-1807

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!