摘要
Reinforcement Learning (RL) has been recognized as one of the most effective methods to optimize traffic signal control. However, due to the inappropriate design of RL elements (i.e., reward and state) for complex traffic dynamics, existing RL-based approaches suffer from slow convergence to optimal traffic signal plans. Meanwhile, to simplify the traffic modeling, most optimization methods assume that the phase duration of traffic signals is constant, which strongly limits the RL capability to search for traffic signal control policies with shorter average vehicle travel time and better GreenWave control. To address these issues, this paper proposes a novel intensity- and phase duration-aware RL-based method named IPDALight for the optimization of traffic signal control. Inspired by the Max Pressure (MP)-based traffic control strategy used in the transportation field, we introduce a new concept named intensity, which ensures that our reward design and state representation can accurately reflect the status of vehicles. By taking the coordination of neighboring intersections into account, our approach enables the fine-tuning of phase duration of traffic signals to adapt to dynamic traffic situations. Comprehensive experimental results on both synthetic and real-world traffic scenarios show that, compared with the state-of-the-art RL methods, IPDALight can not only achieve better average vehicle travel time and greenwave control for various multi-intersection scenarios, but also converge to optimal solutions much faster.
学科与关键词
- MonitorLight: Reinforcement Learning-based Traffic Signal Control Using Mixed Pressure MonitoringFang, Zekuan*; Zhang, Fan; Wang, Ting; Lian, Xiang; Chen, Mingsong31st ACM International Conference on Information and Knowledge Management (CIKM), Atlanta, GA, 2022-10-17 To 2022-10-21.被引 3
- OAM: An Option-Action Reinforcement Learning Framework for Universal Multi-Intersection ControlLiang, Enming; Su, Zicheng; Fang, Chilin; Zhong, Renxin*36th AAAI Conference on Artificial Intelligence / 34th Conference on Innovative Applications of Artificial Intelligence / 12th Symposium on Educational Advances in Artificial Intelligence, ELECTR NETWORK, 2022-02-22 To 2022.被引 3
- FairLight: Fairness-Aware Autonomous Traffic Signal Control With Hierarchical Action SpaceYe, Yutong; Ding, Jiepin; Wang, Ting; Zhou, Junlong; Wei, Xian; Chen, Mingsong*IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2023, 42(8): 2434-2446.
