Computer Science ›› 2018, Vol. 45 ›› Issue (11A): 143-145.

• Intelligent Computing • Previous Articles     Next Articles

Q-learning with Feature-based Approximation for Traffic Light Control

LI Min-shuo1,2, YAO Ming-hai2   

  1. College of Mathematics,Physics and Information Engineering,Zhejiang Normal University,Jinhua,Zhejiang 321004,China1
    College of Information Engineering,Zhejiang University of Technology,Hangzhou 310000,China2
  • Online:2019-02-26 Published:2019-02-26

Abstract: Reinforcement learning(RL) learns the policy through interaction with the environment.RL algorithms are online,incremental,and easy to implement.This paper proposed a Q-learning algorithm with function approximation for adaptive traffic light control (TLC).The application of table-based Q-learning to traffic signal control requires full-state representations and cannot be implemented,even in moderate-sized road networks,because the computational complexity exponentially grows in the numbers of lanes and junctions.This paper tackledthe dimension disaster problem by effectively using feature-based state representations and used a broad characterization of the levels of congestion.The experiment results show that the proposed method is effective and feasible.

Key words: Adaptive traffic light control, Q-learning, Reinforcement learning

CLC Number: 

  • TP181
[1]ADAM I,WAHAB A,YAAKOP M,et al.Adaptive fuzzy logic traffic light management system[C]∥2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T).IEEE,2014:340-343.
[2]COOLS S B,GERSHENSON C,D’HOOGHE B.Self-Organi-zing Traffic Lights:A Realistic Simulation[J].Advances in Applied Self-Organizing Systems,2016,17(4):45-55.
[3]KAUR T,AGRAWAL S.Adaptive Traffic Lights Based on Hy-brid of Neural Network and Genetic Algorithm for Reduced Traffic Congestion[C]∥Recent Advances in Engineering and Computational Sciences (RAECS).2014:1-5.
[4]SRINIVASAN D,CHOY M C,CHEU R L.Neural Networks for Real-Time Traffic Signal Control[J].IEEE Transactions on Intelligent Transportation Systems,2006,7(3):261-272.
[5]SUTTON R S,BARTO A G.Introduction to reinforcement learning [J].IEEE Transactions on Neural Networks,1992,8(3/4):225-227.
[6]高阳,陈世福,陆鑫.强化学习研究综述[J].自动化学报,2004,30(1):86-100.
[7]刘忠,李海红,刘全.强化学习算法研究[J].计算机工程与设计,2008,29(22):5805-5809.
[8]SALKHAM A,CUNNINGHAM R,GARG A,et al.A Collaborative Reinforcement Learning Approach to Urban Traffic Control Optimization[C].IEEE/WIC/ACM International Conferent on Web Intelligence and Intelligent Agent Technology.2008:560-566.
[9]XIE Y C.Development and evaluation of an arterial adaptive traffic signal control system using reinforcement learning[OL].http://holl.hardle.net/1969.1/ETD-TAMU-2480.
[10]WATKINS C,DAYAN P.Q-learning [J].Machine Learning, 1992,8(3/4):279-292.
[1] LIU Xing-guang, ZHOU Li, LIU Yan, ZHANG Xiao-ying, TAN Xiang, WEI Ji-bo. Construction and Distribution Method of REM Based on Edge Intelligence [J]. Computer Science, 2022, 49(9): 236-241.
[2] SHI Dian-xi, ZHAO Chen-ran, ZHANG Yao-wen, YANG Shao-wu, ZHANG Yong-jun. Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning [J]. Computer Science, 2022, 49(8): 247-256.
[3] YUAN Wei-lin, LUO Jun-ren, LU Li-na, CHEN Jia-xing, ZHANG Wan-peng, CHEN Jing. Methods in Adversarial Intelligent Game:A Holistic Comparative Analysis from Perspective of Game Theory and Reinforcement Learning [J]. Computer Science, 2022, 49(8): 191-204.
[4] YU Bin, LI Xue-hua, PAN Chun-yu, LI Na. Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2022, 49(7): 248-253.
[5] LI Meng-fei, MAO Ying-chi, TU Zi-jian, WANG Xuan, XU Shu-fang. Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient [J]. Computer Science, 2022, 49(7): 271-279.
[6] GUO Yu-xin, CHEN Xiu-hong. Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement [J]. Computer Science, 2022, 49(6): 313-318.
[7] FAN Jing-yu, LIU Quan. Off-policy Maximum Entropy Deep Reinforcement Learning Algorithm Based on RandomlyWeighted Triple Q -Learning [J]. Computer Science, 2022, 49(6): 335-341.
[8] XIE Wan-cheng, LI Bin, DAI Yue-yue. PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing [J]. Computer Science, 2022, 49(6): 3-11.
[9] HONG Zhi-li, LAI Jun, CAO Lei, CHEN Xi-liang, XU Zhi-xiong. Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration [J]. Computer Science, 2022, 49(6): 149-157.
[10] ZHANG Jia-neng, LI Hui, WU Hao-lin, WANG Zhuang. Exploration and Exploitation Balanced Experience Replay [J]. Computer Science, 2022, 49(5): 179-185.
[11] LI Peng, YI Xiu-wen, QI De-kang, DUAN Zhe-wen, LI Tian-rui. Heating Strategy Optimization Method Based on Deep Learning [J]. Computer Science, 2022, 49(4): 263-268.
[12] OUYANG Zhuo, ZHOU Si-yuan, LYU Yong, TAN Guo-ping, ZHANG Yue, XIANG Liang-liang. DRL-based Vehicle Control Strategy for Signal-free Intersections [J]. Computer Science, 2022, 49(3): 46-51.
[13] ZHOU Qin, LUO Fei, DING Wei-chao, GU Chun-hua, ZHENG Shuai. Double Speedy Q-Learning Based on Successive Over Relaxation [J]. Computer Science, 2022, 49(3): 239-245.
[14] LI Su, SONG Bao-yan, LI Dong, WANG Jun-lu. Composite Blockchain Associated Event Tracing Method for Financial Activities [J]. Computer Science, 2022, 49(3): 346-353.
[15] HUANG Xin-quan, LIU Ai-jun, LIANG Xiao-hu, WANG Heng. Load-balanced Geographic Routing Protocol in Aerial Sensor Network [J]. Computer Science, 2022, 49(2): 342-352.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!