[1]聂雷,张明萱,黄庆涵,等.基于Double DQN的双模式多目标信号配时方法[J].计算机技术与发展,2024,34(08):143-150.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0124]
　NIE Lei,ZHANG Ming-xuan,HUANG Qing-han,et al.A Dual-mode Multi-objective Signal Timing Method Based on Double DQN[J].,2024,34(08):143-150.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0124]
点击复制

基于Double DQN的双模式多目标信号配时方法

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
34
期数:
2024年08期
页码:
143-150
栏目:
人工智能
出版日期:
2024-08-10

文章信息/Info

Title:
A Dual-mode Multi-objective Signal Timing Method Based on Double DQN
文章编号:
1673-629X(2024)08-0143-08
作者:
聂雷1; 2; 张明萱1; 2; 黄庆涵1; 2; 鲍海洲1; 2
1. 武汉科技大学 计算机科学与技术学院,湖北 武汉 430065; 2. 智能信息处理与实时工业系统湖北省重点实验室,湖北 武汉 430065
Author(s):
NIE Lei1; 2; ZHANG Ming-xuan1; 2; HUANG Qing-han1; 2; BAO Hai-zhou1; 2
1. School of Computer Science and Technology,Wuhan University of Science and Technology,Wuhan 430065,China; 2. Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial System,Wuhan 430065,China
关键词:
交通信号配时; 深度强化学习; 双模式多目标; Double DQN; SUMO
Keywords:
traffic signal timing; deep reinforcement learning; dual-mode multi-objective; Double DQN; Simulation of Urban Mobility
分类号:
TP393
DOI:
10.20165/j.cnki.ISSN1673-629X.2024.0124
摘要:
近年来深度强化学习作为一种高效可靠的机器学习方法被广泛应用在交通信号控制领域。 目前,现有交通信号 配时方法通常忽略了特殊车辆(例如救护车、消防车等)的优先通行;此外,基于传统深度强化学习的信号配时方法优化目标较为单一,导致其在复杂交通场景中性能不佳。 针对上述问题,基于 Double DQN 提出一种融合特殊车辆优先通行的双模式多目标信号配时方法(Dual-mode Multi-objective signal timing method based on Double DQN,DMDD),以提高不同交通场景下路口的通行效率。 该方法首先基于路口的饱和状态选择信号控制模式,特殊车辆在紧急控制模式下被赋予更高的通行权重,有利于其更快通过路口;接着针对等待时长、队列长度和 CO2排放量 3 个指标分别设计神经网络进行奖励计算;最后利用 Double DQN 进行最优信号相位的选择,通过灵活切换信号相位以提升通行效率。 基于 SUMO 的实验结果表明,DMDD 与对比方法相比能有效缩短路口处特殊车辆的等待时长、队列长度和 CO2排放量,特殊车辆能够更快通过路口,有效地提高了通行效率。
Abstract:
In recent years,deep reinforcement learning has been widely used as an efficient and reliable machine learning method in the field of traffic signal control. Currently,existing traffic signal timing methods usually ignore the priority of special vehicles (e. g. , am-bulances,fire engines,etc. ); in addition,the optimization objectives of signal timing methods based on traditional deep reinforcement learning are often relatively single,resulting in poor performance in complex traffic scenarios. To address the above problems,we propose a Dual-mode Multi-objective signal timing method based on Double DQN (DMDD) that incorporates the priority of special vehicles for improving the traffic efficiency of intersections under different scenarios. The method first decides the signal control mode based on the saturation state of the intersection and gives higher weights to special vehicles when in emergency control mode so that they can pass through the intersection faster. Then,neural networks are designed to calculate the rewards for the three metrics of waiting time,queue length and CO2 emission. Finally,Double DQN is utilized to select the optimal signal phase,and the signal phase is flexibly switched to improve the traffic efficiency. The experimental results based on SUMO show that the DMDD can effectively reduce the waiting time,queue length and CO2 emission of special vehicles at the intersection compared with other methods,and special vehicles can pass through the intersection faster,which effectively improves the efficiency of traffic.

相似文献/References:

[1]赵 纯,董小明.基于深度 Q-Learning 的信号灯配时优化研究[J].计算机技术与发展,2021,31(08):198.[doi:10. 3969 / j. issn. 1673-629X. 2021. 08. 034]
　ZHAO Chun,DONG Xiao-ming.Research on Signal Timing Optimization Based on Deep Q-Learning[J].,2021,31(08):198.[doi:10. 3969 / j. issn. 1673-629X. 2021. 08. 034]
[2]况立群,冯 利,韩 燮,等.基于双深度 Q 网络的智能决策系统研究[J].计算机技术与发展,2022,32(02):137.[doi:10. 3969 / j. issn. 1673-629X. 2022. 02. 022]
　KUANG Li-qun,FENG Li,HAN Xie,et al.Research on Intelligent Decision-making System Based on Double Deep Q-Network[J].,2022,32(08):137.[doi:10. 3969 / j. issn. 1673-629X. 2022. 02. 022]
[3]高文斌,王 睿,王田丰,等.基于深度强化学习的 QoS 感知 Web 服务组合[J].计算机技术与发展,2022,32(06):92.[doi:10. 3969 / j. issn. 1673-629X. 2022. 06. 016]
　GAO Wen-bin,WANG Rui,WANG Tian-feng,et al.QoS-aware Service Composition Based on Deep Reinforcement Learning[J].,2022,32(08):92.[doi:10. 3969 / j. issn. 1673-629X. 2022. 06. 016]
[4]詹 御,张郭健,彭麟杰,等.基于 DRL 的 MEC 卸载网络竞争窗口优化[J].计算机技术与发展,2022,32(06):99.[doi:10. 3969 / j. issn. 1673-629X. 2022. 06. 017]
　ZHAN Yu,ZHANG Guo-jian,PENG Lin-jie,et al.Optimization of Contention Window of MEC Offloading Network Based on DRL[J].,2022,32(08):99.[doi:10. 3969 / j. issn. 1673-629X. 2022. 06. 017]
[5]牟轩庭,张宏军,廖湘琳,等.规则引导的智能体决策框架[J].计算机技术与发展,2022,32(10):156.[doi:10. 3969 / j. issn. 1673-629X. 2022. 10. 026]
　MU Xuan-ting,ZHANG Hong-jun,LIAO Xiang-lin,et al.Rule-guided Agent Decision-Making Framework[J].,2022,32(08):156.[doi:10. 3969 / j. issn. 1673-629X. 2022. 10. 026]
[6]林泽阳,赖 俊,陈希亮.基于课程学习的深度强化学习研究综述[J].计算机技术与发展,2022,32(11):16.[doi:10. 3969 / j. issn. 1673-629X. 2022. 11. 003]
　LIN Ze-yang,LAI Jun,CHEN Xi-liang.An Overview of Deep Reinforcement Learning Based on Curriculum Learning[J].,2022,32(08):16.[doi:10. 3969 / j. issn. 1673-629X. 2022. 11. 003]
[7]吕相霖,臧兆祥,李思博,等.基于注意力的循环 PPO 算法及其应用[J].计算机技术与发展,2024,34(01):136.[doi:10. 3969 / j. issn. 1673-629X. 2024. 01. 020]
　LYU Xiang-lin,ZANG Zhao-xiang,LI Si-bo,et al.Attention-based Recurrent PPO Algorithm and Its Application[J].,2024,34(08):136.[doi:10. 3969 / j. issn. 1673-629X. 2024. 01. 020]
[8]龚亮亮,张 影,张俊尧,等.基于深度强化学习的任务卸载和资源分配优化[J].计算机技术与发展,2024,34(04):116.[doi:10. 3969 / j. issn. 1673-629X. 2024. 04. 018]
　GONG Liang-liang,ZHANG Ying,ZHANG Jun-yao,et al.Joint Optimization of Task Offloading and Resource Allocation Based on Deep Reinforcement Learning[J].,2024,34(08):116.[doi:10. 3969 / j. issn. 1673-629X. 2024. 04. 018]
[9]王钰童,顾进广*.边缘场景下基于DDQN的容器组调度策略[J].计算机技术与发展,2024,34(09):16.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0163]
　WANG Yu-tong,GU Jin-guang*.Container Group Scheduling Optimization Strategy Based on DDQN in Edge Scenarios[J].,2024,34(08):16.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0163]
[10]王宇轩,鲍海洲*,喻国荣,等.基于PER-MATD3的任务卸载和资源优化方法[J].计算机技术与发展,2024,34(12):57.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0254]
　WANG Yu-xuan,BAO Hai-zhou*,YU Guo-rong,et al.Task Offloading and Resource Optimization Method Based on PER-MATD3[J].,2024,34(08):57.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0254]

更新日期/Last Update: 2024-08-10