Bayesian Critique-Tune-Based Reinforcement Learning with Adaptive Pressure for Multi-Intersection Traffic Signal Control

Jinguo Xian; Jiwan He; Wenchang Duan; Zhenguo Gao

REVIEW 4 major objections 5 minor 46 references

Bayesian Critique-Tune-Based Reinforcement Learning with Adaptive Pressure for Multi-Intersection Traffic Signal Control

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A Bayesian critique-tune layer plus attention-weighted adaptive pressure outperforms state-of-the-art traffic signal control on seven real-world networks, cutting average queue length by 9.60% and average waiting time by 15.28%.

desk verdict The adaptive pressure mechanism is the real contribution; the Bayesian Tune Layer's theory does not survive contact with its own equation, but the paper deserves a serious referee. read the letter →

arxiv 2412.16225 v2 pith:PTPJDOQ6 submitted 2024-12-18 eess.SY cs.AIcs.LGcs.MAcs.SY

Wenchang Duan , Zhenguo Gao , Jiwan He , Jinguo Xian This is my paper · ORCID

classification eess.SYcs.AIcs.LGcs.MAcs.SY

keywords trafficsignalcontrolreinforcementlearningBayesiancritique-tuneadaptivepressuremulti-intersectionDQNposteriorriskSARIMA

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that reinforcement-learning traffic signal control can be made more efficient and more plausible by adding a two-layer Bayesian guardrail on top of the RL agent, together with an attention-based weighting of lane queues it calls Adaptive Pressure. The Critique Layer builds a 95% Bayesian credible interval from a SARIMA model of past rewards and flags the RL policy when the predicted next reward falls outside that interval. The Tune Layer then re-selects the signal phase by minimizing the posterior risk of each phase's Q-value. On seven real-world CityFlow datasets spanning Jinan, Hangzhou, and New York, the combined method (BCT-APLight) is claimed to beat the best prior method, Advanced-CoLight, cutting average queue length by 9.60% and average waiting time by 15.28%. A sympathetic reader would care because the paper addresses the common failure of RL-based signal controllers to produce stable, reasonable phase choices, and it does so with a statistical wrapper rather than a larger network.

What carries the argument

The load-bearing objects are the Bayesian credible interval in the Critique Layer, the posterior risk functional $R_{\text{post}} = \int_\Theta (Q_{\text{cur}} - Q)^2 \, p(Q \mid Q_{\text{cur}}, \{Q_t\}) \, dQ$ in the Tune Layer, and the attention-weighted adaptive pressure $p_e(X_y, X'_{y'}) = x(l_k) - \sum_j \omega^j_k x(m_j)$. The posterior risk under squared loss is what connects Bayesian decision theory to phase selection; its minimum over phases is the claimed alignment with the 'global optima'. The AP mechanism's multi-head attention weights replace the fixed equal averaging of downstream lanes, which the paper argues over-weights low-traffic lanes and under-responds to high-traffic ones.

What would settle it

Compare, on the same CityFlow episodes, the phase selected by minimizing posterior risk with the phase that maximizes the one-step-ahead SARIMA-forecast reward; if the posterior-risk choices do not match or beat the forecast-maximizing choices, the Tune Layer's claimed link to the global optimum is not operating.

Watch

Extended reading notes

Core claim

BCT-APLight is a DQN-based multi-intersection traffic signal controller whose two new components are (1) a Critique-Tune (CT) framework that uses Bayesian statistics to police the RL policy and (2) an attention-based Adaptive Pressure (AP) mechanism for lane-queue representation. The Critique Layer fits a SARIMA model to each intersection's episode reward history, samples posterior predictions for future rewards, forms a 95% Bayesian credible interval, and compares the prediction network's output to that interval; a prediction outside the interval triggers the Tune Layer. The Tune Layer maintains a non-parametric KDE-based prior over each phase's historical Q-values, constructs a likelihood that includes both the history and a confidence-weighted current Q-value, and selects the phase minimizing the expected squared-loss posterior risk. The AP mechanism replaces the equal-weight 'efficient pressure' of prior work with per-upstream-to-downstream attention weights, giving each lane's queue a dynamically learned influence on the pressure signal. The paper reports that the full method outperforms all baselines on seven real-world datasets, with average reductions of 9.60% in queue length and 15.28% in waiting time relative to Advanced-CoLight, and that both components contribute in ablation.

Load-bearing premise

The whole Critique-Tune gain rests on the assumption that choosing the signal phase with the lowest posterior-risk score actually improves long-run traffic performance; the appendix proof only shows that a Bayes rule minimizes a statistical risk, not that this risk matches the road-network reward.

Editorial extensions

If this is right

On the five Jinan and Hangzhou datasets, BCT-APLight reduces average travel time by 3.01%, average queue length by 9.60%, and average waiting time by 15.28% relative to Advanced-CoLight, the strongest traditional RL baseline.
On the two New York networks, the method cuts average travel time by 5.73% and 7.09% and average waiting time by 9.17% and 10.34%, indicating the gains scale to larger networks.
The attention-based adaptive pressure alone (AP-based DQN) already outperforms all traditional baselines, cutting ATT by 26.62%, AQL by 52.36%, and AWT by 58.21% over plain DQN.
Adding the CT framework to Advanced-CoLight yields further gains of 1.74% in ATT, 4.83% in AQL, and 7.01% in AWT, showing the critique-tune refinement helps even a strong baseline.
Removing either the AP mechanism or the CT framework degrades performance, so the paper claims both components contribute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the ablation attributes the large majority of the improvement to the adaptive-pressure mechanism, a sympathetic reading suggests the Bayesian Tune Layer is an optional safety net rather than the engine of the reported gains; one testable prediction is that replacing the Tune Layer with a simpler rule, such as falling back to the second-best DQN action, would retain most of the CT framework
The credible-interval critique depends on SARIMA's ability to model reward histories as stationary seasonal series; on datasets with nonstationary traffic regimes, the critique could fire too often or too rarely, and the 95% interval might need recalibration, an implicit hyperparameter the paper does not explore.
The AP mechanism weights upstream-downstream lane pairs with a multi-head attention; this same weighting could be reused as a reward-shaping term or as an interpretability probe to identify which turning movements dominate congestion at an intersection.
If the CT framework were applied to other RL-based control domains with expensive action switches, such as elevator dispatch or energy scheduling, the same pattern—Bayesian critique of predicted rewards followed by posterior-risk phase reselection—would be a drop-in module; the paper's experiments only cover traffic signal control.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know about 2412.16225. First, the attention-based Adaptive Pressure mechanism is the real story: it gives large, consistent gains across the CityFlow benchmarks. Second, the Bayesian Critique-Tune layer does not do what the paper claims; the posterior-risk minimization selects the phase with the most self-consistent Q-value, not the phase with the highest expected value. The stress-test note is right, and the paper's own ablation supports that reading: AP accounts for most of the improvement, while CT adds only a few percent.

What's genuinely new: the AP formulation, weighting downstream queues per upstream lane via multi-head attention, is a reasonable extension of the max-pressure literature, and it seems to work. The empirical coverage is good: seven real-world datasets, standard baselines, consistent SOTA numbers. The ablation is honest enough to show the components' contributions separately.

The soft spots are real but not evenly distributed. The CT theory is the load-bearing weakness. Equation (22) minimizes per-phase posterior risk under squared loss, which equals posterior variance plus (posterior mean minus current Q)^2. Minimizing that across phases has no term for the level of the posterior mean; a low-value but accurately estimated phase can beat a high-value uncertain one. Appendix B proves only the standard Bayes decision theorem; it never connects that to Bellman-consistent policy improvement. The claim that this 'ensures the updated policy is aligned with the global optima' is unsupported. Appendix A also has a concrete math error: the identity sum phi_k = 1 does not hold for stationary AR models, and Eq. (29) is malformed. On the experimental side, the tables report single numbers with no standard deviations or seed counts, so the 15% AWT reduction might not be robust. The abstract says codes are open-sourced, but there is no repository link.

Who gets value from this: people working on pressure-based signal control will find the AP design worth trying. The CT framework as placed is not ready to be adopted. The paper deserves a serious referee — the empirical result is substantial even if the interpretation needs major revision. I would not desk-reject; I'd send it out with a request to fix or drop the theoretical overclaim, add error bars, correct Appendix A, and actually release the code.

Referee Report

4 major / 5 minor

Summary. The paper proposes BCT-APLight, a reinforcement-learning method for multi-intersection adaptive traffic signal control. It combines a DQN backbone with two additions: a Bayesian Critique-Tune (CT) framework and an attention-based Adaptive Pressure (AP) mechanism. The Critique Layer builds a SARIMA-based Bayesian credible interval from historical rewards to flag suspicious predicted rewards, and the Tune Layer then chooses the signal phase that minimizes a posterior risk computed from current and historical Q-values. The AP mechanism replaces the standard efficient-pressure formulation with attention-weighted upstream-downstream lane influences. Experiments in the CityFlow simulator on five Jinan/Hangzhou networks and two New York networks compare against traditional, RL, and LLM-based baselines; the paper reports that BCT-APLight reduces average queue length by 9.60% and average waiting time by 15.28% relative to Advanced-CoLight, with additional large-scale experiments and an ablation study.

Significance. If the reported gains are robust, the work is practically significant for ATSC: it evaluates on seven real-world datasets, includes large-scale 28x7 New York networks, covers a broad baseline set, and states that code is open-sourced. The AP mechanism is well motivated and is shown in the ablation to be the dominant contributor to the gains. However, the central theoretical claim for the CT framework—that posterior-risk minimization aligns the policy with global optima—is not established by the provided argument, and the empirical tables report only single values without variance or seed counts. The paper is therefore a promising empirical systems contribution whose main advertised mechanism needs either a correct justification or a substantially weakened claim.

major comments (4)

[III-C and Appendix B] The Tune Layer's optimality claim is not supported. Under the squared loss L(Q_cur^(i(j)), Q) = (Q_cur^(i(j)) - Q)^2, Eq. (21) gives R_post(j) = Var(Q | data_j) + (E[Q | data_j] - Q_cur^(i(j)))^2. Minimizing this quantity across phases selects the phase whose current Q-value is most precise or closest to its own posterior mean; it does not select the phase with the highest expected Q-value or the highest expected cumulative reward. Appendix B proves only the textbook result that a Bayes rule minimizes posterior risk for a fixed decision problem; it does not show that comparing posterior risks across eight phase-specific posteriors is equivalent to maximizing a Bellman-consistent objective. The statement in Section III-A2 that the Tune Layer 'ensures the updated policy aligned with the global optima' is therefore unsupported. A concrete counterexample would be a low but accurately estimated phase beating a high but uncertain phase under Eq. (22).
[V-B, V-C, Tables I and II] The experimental results are reported as single numbers without standard deviations, confidence intervals, or the number of seeds. This affects the headline claims (9.60% AQL and 15.28% AWT improvements in Table I) and the CT ablation results (e.g., CT-Based Advanced-CoLight reduces AWT from 43.51 to 40.04 on Jinan-1 in Table II, about an 8% gain that could be within run-to-run noise). The box plots in Fig. 7 appear to summarize data over intersections or episodes, not over independent training runs. Please report multi-seed means with variance measures or equivalent statistical significance information for all tables, including the New York experiments in Fig. 8.
[Appendix A] The derivation that the zero-mean-transformed episodes follow identical ARIMA models contains a false step. The text before Eq. (34) asserts that the AR coefficients satisfy sum_{k=1}^p phi_k = 1. This is not a property of stationary AR models; for AR(1), stationarity requires |phi| < 1, not phi = 1. Consequently Eq. (34) does not follow, and the theoretical justification for treating each episode as generated by the same ARIMA model is invalid. The Critique Layer relies on this SARIMA modeling, so the issue should be addressed or the claim revised.
[III-A and Algorithm 1] The prediction network that produces the Predictive Reward is underspecified. The paper does not state the loss function, the target labels used for supervised training, the training schedule, or how the horizon h is chosen. Since the Critique Layer's decision to trigger the Tune Layer depends entirely on whether the predicted reward falls inside the SARIMA credible interval, the mechanism cannot be reproduced or validated from the manuscript as it stands.

minor comments (5)

[Fig. 8] Some New York panels use the label 'BCT-APRL' while the model is called BCT-APLight elsewhere; please unify the notation.
[Fig. 7 and Fig. 8 captions] The captions read 'The comparative results obtained using only the CT framework,' which conflicts with the content comparing full BCT-APLight against baselines; these captions appear to be copy-paste errors and should be corrected.
[Eq. (11)] Equation (11) defines Phi(x) as the standard normal density 1/sqrt(2 pi) exp(-x^2/2), but the truncated normal density requires the standard normal CDF in the denominator; use phi for the density and Phi for the CDF.
[Abstract and Section V-A] The abstract states that 'Codes are open-sourced,' but no repository URL or access instruction is provided in the manuscript; please add a link or reference.
[Section V-B] The sentence 'BCT-APLight average decreases 7.83% in ATT, 21.74% in AQL, and 20.69% in AWT' does not specify the comparison baseline, making the claim ambiguous; state explicitly that the comparison is with MaxPressure.

Circularity Check

1 steps flagged · score 4.0 of 10

Tune Layer's 'global optima' guarantee reduces to minimizing a posterior risk built from the very Q-values it is supposed to correct; the empirical benchmarks remain external.

fitted input called prediction [Section III-C, Eqs. (21)-(22); claim in Section III-A]
"Rpost = ∫Θ L(Qi(j)cur, Q) p(Q|Qi(j)cur, {Qi(j)t}) dQ. (21) ... minRpost = inf_j ∫Θ L(Qi(j)cur, Q) p(Q|Qi(j)cur, {Qi(j)t}) dQ. (22) ... Via selecting the phase with the minimal posterior risk minRpost, the Tune Layer ensures the updated policy πi_upd aligned with the global optima."

With square loss L(Q_cur,Q)=(Q_cur−Q)^2, Rpost(j)=Var(Q|data_j)+(Q_cur^(j)−E[Q|data_j])^2. The phase selected by Eq. (22) is therefore the one whose current Q estimate is most self-consistent, not the one with the largest Q or highest expected cumulative reward. The posterior is built from the same Q-value stream (KDE prior from Qi(j)_his and weighted likelihood from Qi(j)_cur) that the Tune Layer is supposed to refine. The claim that this 'ensures the updated policy aligned with the global optima' is a restatement of minimizing this self-referential risk, not a derivation from the Bellman objective; Appendix B proves only that a Bayes rule minimizes posterior risk, which is true by construction of Eq. (21). The optimality prediction is thus statistically forced by the input Q-values.

full rationale

The paper's numerical headline results are anchored to external baselines and CityFlow simulations on seven real-world datasets, so the main performance claims are not fitted constants and do not reduce to the paper's own definitions. The AP mechanism is an independently testable attention-weighted pressure representation, and its ablation shows large gains over DQN. The only load-bearing step with a self-referential structure is the Tune Layer: its posterior risk is defined using the current Q-value both as data and as the squared-loss reference point, and the 'global optima' guarantee is asserted rather than connected to the MDP objective of Section II-B. Appendix B proves the textbook Bayes-rule result but not that cross-phase comparison of phase-specific posterior risks maximizes expected cumulative reward. This is a partial circularity in the theoretical justification of one component, not in the empirical comparison, so the overall score is moderate rather than high.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central empirical result rests on many unspecified hyperparameters and on two unproven modeling assumptions: the SARIMA episode-equivalence and the posterior-risk action-selection. These do not by themselves falsify the empirical claim, but they shift the burden to the experimental comparison, which lacks error bars and code.

free parameters (6)

sigma_cur^2 (current Q-value likelihood variance) = not specified
Eq (18) defines a confidence-weighted likelihood for the current Q-value; sigma_cur^2 controls how much the Tune Layer trusts the current Q-value, and no value or selection method is given.
KDE bandwidth bw = not specified
Eqs (16)-(17) require a bandwidth for the Gaussian kernel density estimate of history Q-values; the value is not reported.
SARIMA prior hyperparameters (mu_c, sigma_c^2, a, b, b_phi, b_theta, a_sigma2, b_sigma2) = not specified
Eqs (11)-(14) specify priors over the SARIMA intercept, AR/MA coefficients, and noise variance; no numerical values or fitting routine are stated.
SARIMA orders (p, d, q, P, D, Q, s) = selected by BIC per data window
Section III-B uses BIC to choose model orders; the chosen orders are not reported for the experiments, making the credible interval construction hard to reproduce.
Prediction network horizon h and sampling count N_t = not specified
Section III-B uses future horizon h and N_t posterior samples to build the credible interval; neither is stated.
Attention head count and key dimension d_k = not specified
Eqs (23)-(24) define multi-head attention for the adaptive pressure mechanism; head count and d_k are not specified.

assumptions (5)

domain assumption History rewards in RL-based ATSC can be fit by a SARIMA model with each episode sharing identical ARIMA coefficients.
Section III-B and Appendix A. The proof in Appendix A relies on the incorrect property sum of AR coefficients equals 1, which is not generally true for ARIMA models.
ad hoc to paper Minimizing posterior risk under squared loss yields a better signal-phase policy.
Section III-C, Eq (22). No theorem connects this phase selection to the RL return or to Bellman optimality; Appendix B only proves a generic Bayes decision rule.
domain assumption A predicted reward falling outside the Bayesian credible interval indicates an unreasonable policy that needs tuning.
Section III-B3. The interval is descriptive, and no evidence shows that an out-of-interval prediction corresponds to a suboptimal action.
domain assumption Attention weights over upstream-downstream lane features produce a valid traffic-pressure weighting.
Section IV-A. The learned weights are not validated against true turning fractions or flow proportions.
domain assumption The prediction network can forecast the next reward from state and current Q-value.
Section III-A1. The MLP prediction network is assumed to be predictive, but no prediction accuracy metrics are reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Bayesian Critique-Tune-Based Reinforcement Learning with Adaptive Pressure for Multi-Intersection Traffic Signal Control." pith.science (2026). https://pith.science/paper/PTPJDOQ6

@misc{pith2026241216225,
  author       = {Pith},
  title        = {Pith review of: Bayesian Critique-Tune-Based Reinforcement Learning with Adaptive Pressure for Multi-Intersection Traffic Signal Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PTPJDOQ6}},
  note         = {Machine review of arXiv:2412.16225}
}

abstract

Adaptive Traffic Signal Control (ATSC) system is a critical component of intelligent transportation, with the capability to significantly alleviate urban traffic congestion. Although reinforcement learning (RL)-based methods have demonstrated promising performance in achieving ATSC, existing methods are still prone to making unreasonable policies. Therefore, this paper proposes a novel Bayesian Critique-Tune-Based Reinforcement Learning with Adaptive Pressure for multi-intersection signal control (BCT-APLight). In BCT-APLight, the Critique-Tune (CT) framework, a two-layer Bayesian structure is designed to refine the excessive trust of RL policies. Specifically, the Bayesian inference-based Critique Layer provides effective evaluations of the credibility of policies; the Bayesian decision-based Tune Layer fine-tunes policies by minimizing the posterior risks when the evaluations are negative. Meanwhile, an attention-based Adaptive Pressure (AP) mechanism is designed to effectively weight the vehicle queues in each lane, thereby enhancing the rationality of traffic movement representation within the network. Equipped with the CT framework and AP mechanism, BCT-APLight effectively enhances the reasonableness of RL policies. Extensive experiments conducted with a simulator across a range of intersection layouts demonstrate that BCT-APLight is superior to other state-of-the-art (SOTA) methods on seven real-world datasets. Specifically, BCT-APLight decreases average queue length by \textbf{$\boldsymbol{9.60\%}$} and average waiting time by \textbf{$\boldsymbol{15.28\%}$}.

Figures

Figures reproduced from arXiv: 2412.16225 by the authors.

**Figure 2.** The traditional efficient pressure and the attention [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Framework of Bayesian Critique-Tune for RL. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Architecture of attention-based adaptive pressure extraction for each intersection direction. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: The road network systems of the datasets from Jinan, [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: The twin line chart of the rewards. The upper section illustrates the overall trends of each method over 2000 episodes, [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 7.** Figure 7: The comparative results obtained using only the CT [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 36 canonical work pages

[1]

A review of sustainable transportation solutions: Innovations, challenges, and future directions,

E. A. Etukudoh, A. Adefemi, V . I. Ilojianya, A. A. Umoh, K. I. Ibekwe, and Z. Q. S. Nwokediegwu, “A review of sustainable transportation solutions: Innovations, challenges, and future directions,” World Journal of Advanced Research and Reviews, vol. 21, no. 1, pp. 1440–1452, 2024

work page 2024
[2]

Urban travel time and residential location choice: The impacts of traffic congestion,

M. Zhang, Z. Li, H. Si, L. Cheng, X. Zhou, and B. Wang, “Urban travel time and residential location choice: The impacts of traffic congestion,” Sustainable Cities and Society , vol. 99, p. 104975, 2023

work page 2023
[3]

Changes in traffic congestion and air pollution due to major roadway infrastructure improvements in texas,

M. Harleman, L. Harris, M. D. Willis, B. Ritz, P. Hystad, and E. L. Hill, “Changes in traffic congestion and air pollution due to major roadway infrastructure improvements in texas,” Science of The Total Environment, vol. 898, p. 165463, 2023

work page 2023
[4]

Traffic management system and traffic light control in smart city to reduce traffic congestion,

A. Wided, B. Assia, and B. Fatima, “Traffic management system and traffic light control in smart city to reduce traffic congestion,” International Journal of Automation and Smart Technology , vol. 13, no. 1, pp. 2464–2464, 2023

work page 2023
[5]

Troubleshooting so- lution for traffic congestion control,

V . Tong, S. Souihi, H. A. Tran, and A. Mellouk, “Troubleshooting so- lution for traffic congestion control,” Journal of Network and Computer Applications, p. 103923, 2024

work page 2024
[6]

What is the root cause of congestion in urban traffic networks: Road infrastructure or signal control?

W. Yue, C. Li, Y . Chen, P. Duan, and G. Mao, “What is the root cause of congestion in urban traffic networks: Road infrastructure or signal control?” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 7, pp. 8662–8679, 2022. 14

work page 2022
[7]

Deep reinforcement learning for intelligent transportation systems: A survey,

A. Haydari and Y . Yılmaz, “Deep reinforcement learning for intelligent transportation systems: A survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 11–32, 2022

2022
[8]

Reinforcement learning in urban network traffic signal control: A systematic literature review,

M. Noaeen, A. Naik, L. Goodman, J. Crebo, T. Abrar, Z. S. H. Abad, A. L. Bazzan, and B. Far, “Reinforcement learning in urban network traffic signal control: A systematic literature review,” Expert Systems with Applications, vol. 199, p. 116830, 2022

work page 2022

Show all 46 references

[9]

Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implemen- tation,

Y . Han, M. Wang, and L. Leclercq, “Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implemen- tation,” Communications in Transportation Research, vol. 3, p. 100104, 2023

2023
[10]

Redefining counterfactual explanations for reinforcement learning: Overview, challenges and opportunities,

J. Gajcin and I. Dusparic, “Redefining counterfactual explanations for reinforcement learning: Overview, challenges and opportunities,” ACM Comput. Surv. , vol. 56, no. 9, apr 2024. [Online]. Available: https://doi.org/10.1145/3648472

2024 doi
[11]

Challenges and opportunities in deep reinforcement learning with graph neural networks: A comprehensive review of algorithms and applications,

S. Munikoti, D. Agarwal, L. Das, M. Halappanavar, and B. Natarajan, “Challenges and opportunities in deep reinforcement learning with graph neural networks: A comprehensive review of algorithms and applications,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1...

2023
[13]

Multi-agent broad reinforcement learning for intelligent traffic light control,

R. Zhu, L. Li, S. Wu, P. Lv, Y . Li, and M. Xu, “Multi-agent broad reinforcement learning for intelligent traffic light control,” Information Sciences, vol. 619, pp. 509–525, 2023

2023
[14]

Reinforcement learning on graphs: A survey,

M. Nie, D. Chen, and D. Wang, “Reinforcement learning on graphs: A survey,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 7, no. 4, pp. 1065–1082, 2023

2023
[15]

New challenges in reinforcement learning: a survey of security and privacy,

Y . Lei, D. Ye, S. Shen, Y . Sui, T. Zhu, and W. Zhou, “New challenges in reinforcement learning: a survey of security and privacy,” Artificial Intelligence Review, vol. 56, no. 7, pp. 7195–7236, 2023

2023
[16]

Chal- lenges, evaluation and opportunities for open-world learning,

M. Kejriwal, E. Kildebeck, R. Steininger, and A. Shrivastava, “Chal- lenges, evaluation and opportunities for open-world learning,” Nature Machine Intelligence, pp. 1–9, 2024

2024
[17]

Reinforcement learning- assisted evolutionary algorithm: A survey and research opportunities,

Y . Song, Y . Wu, Y . Guo, R. Yan, P. N. Suganthan, Y . Zhang, W. Pedrycz, S. Das, R. Mallipeddi, O. S. Ajani et al. , “Reinforcement learning- assisted evolutionary algorithm: A survey and research opportunities,” Swarm and Evolutionary Computation , vol. 86, p. 101517, 2024

2024
[18]

Reinforcement learning algorithms: A brief survey,

A. K. Shakya, G. Pillai, and S. Chakrabarty, “Reinforcement learning algorithms: A brief survey,” Expert Systems with Applications , vol. 231, p. 120495, 2023

2023
[19]

Integrated real-time signal control and routing optimization: A two-stage rolling horizon framework with decentralized solution,

S. Lin, J. Hu, W. Ma, C. Zheng, and R. Li, “Integrated real-time signal control and routing optimization: A two-stage rolling horizon framework with decentralized solution,” Transportation Research Part C: Emerging Technologies, vol. 165, p. 104734, 2024

2024
[20]

Cooperative traffic signal control using a distributed agent-based deep reinforcement learning with incentive communication,

B. Zhou, Q. Zhou, S. Hu, D. Ma, S. Jin, and D.-H. Lee, “Cooperative traffic signal control using a distributed agent-based deep reinforcement learning with incentive communication,” IEEE Transactions on Intelli- gent Transportation Systems, 2024

2024
[22]

A large-scale traffic signal control algorithm based on multi-layer graph deep reinforcement learning,

T. Wang, Z. Zhu, J. Zhang, J. Tian, and W. Zhang, “A large-scale traffic signal control algorithm based on multi-layer graph deep reinforcement learning,” Transportation Research Part C: Emerging Technologies, vol. 162, p. 104582, 2024

2024
[23]

Prompt to transfer: Sim-to-real transfer for traffic signal control with prompt learning,

L. Da, M. Gao, H. Mei, and H. Wei, “Prompt to transfer: Sim-to-real transfer for traffic signal control with prompt learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 1, 2024, pp. 82–90

2024
[24]

A survey on deep reinforcement learning approaches for traffic signal control,

H. Zhao, C. Dong, J. Cao, and Q. Chen, “A survey on deep reinforcement learning approaches for traffic signal control,” Engineering Applications of Artificial Intelligence , vol. 133, p. 108100, 2024

2024
[25]

Large-scale traffic signal control using a novel multiagent reinforcement learning,

X. Wang, L. Ke, Z. Qiao, and X. Chai, “Large-scale traffic signal control using a novel multiagent reinforcement learning,” IEEE Transactions on Cybernetics, vol. 51, no. 1, pp. 174–187, 2021

2021
[26]

Distilling the knowledge in a neural network,

G. Hinton, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015

2015 arXiv
[27]

Differential advising in multiagent reinforcement learning,

D. Ye, T. Zhu, Z. Cheng, W. Zhou, and P. S. Yu, “Differential advising in multiagent reinforcement learning,” IEEE Transactions on Cybernetics , vol. 52, no. 6, pp. 5508–5521, 2022

2022
[28]

Location-based real-time updated advising method for traffic signal control,

C. Zhu, D. Ye, T. Zhu, and W. Zhou, “Location-based real-time updated advising method for traffic signal control,” IEEE Internet of Things Journal, vol. 11, no. 8, pp. 14 551–14 562, 2024

2024
[29]

Mastering arterial traffic signal control with multi-agent attention-based soft actor-critic model,

F. Mao, Z. Li, Y . Lin, and L. Li, “Mastering arterial traffic signal control with multi-agent attention-based soft actor-critic model,” IEEE Transactions on Intelligent Transportation Systems , vol. 24, no. 3, pp. 3129–3144, 2023

2023
[30]

Multi-agent attention double actor-critic framework for intelligent traffic light control in urban scenarios with hybrid traffic,

B. Liu, W. Han, E. Wang, S. Xiong, L. Wu, Q. Wang, J. Wang, and C. Qiao, “Multi-agent attention double actor-critic framework for intelligent traffic light control in urban scenarios with hybrid traffic,” IEEE Transactions on Mobile Computing , vol. 23, no. 1, pp. 660–672, 2024

2024
[31]

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach,

X. B. Song, B. Zhou, and D. Ma, “Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach,” Trans- portation Research Part C: Emerging Technologies, vol. 160, p. 104528, 2024

2024
[32]

Robust model-free reinforce- ment learning with multi-objective bayesian optimization,

M. Turchetta, A. Krause, and S. Trimpe, “Robust model-free reinforce- ment learning with multi-objective bayesian optimization,” in2020 IEEE International Conference on Robotics and Automation (ICRA), 2020, pp. 10 702–10 708

2020
[33]

Approximate inference of traffic flow state at signalized intersections using a bayesian learning framework,

N. Zhang, X. Yang, H. Guo, H. Dong, and W. Ma, “Approximate inference of traffic flow state at signalized intersections using a bayesian learning framework,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 5, pp. 4765–4776, 2023

2023
[34]

Presslight: Learning max pressure control to coordinate traffic signals in arterial network,

H. Wei, C. Chen, G. Zheng, K. Wu, and Z. Li, “Presslight: Learning max pressure control to coordinate traffic signals in arterial network,” Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , p. 1290––1298, 2019

2019
[35]

Smoothing-mp: A novel max- pressure signal control considering signal coordination to smooth traffic in urban networks,

T. Xu, S. Barman, and M. W. Levin, “Smoothing-mp: A novel max- pressure signal control considering signal coordination to smooth traffic in urban networks,” Transportation Research Part C: Emerging Tech- nologies, vol. 166, p. 104760, 2024

2024
[36]

Two-layer adaptive sig- nal control framework for large-scale dynamically-congested networks: Combining efficient max pressure with perimeter control,

D. Tsitsokas, A. Kouvelas, and N. Geroliminis, “Two-layer adaptive sig- nal control framework for large-scale dynamically-congested networks: Combining efficient max pressure with perimeter control,” Transporta- tion Research Part C: Emerging Technologies, vol. 152, p. 104128, 2023

2023
[37]

Gp- light: Grouped multi-agent reinforcement learning for large-scale traffic signal control

Y . Liu, G. Luo, Q. Yuan, J. Li, L. Jin, B. Chen, and R. Pan, “Gp- light: Grouped multi-agent reinforcement learning for large-scale traffic signal control.” in Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23) , 2023, pp. 199–207

2023
[38]

Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control,

L. Zhang, Q. Wu, J. Shen, L. L ¨u, B. Du, and J. Wu, “Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control,” in International Conference on Machine Learning. PMLR, 2022, pp. 26 645–26 654

2022
[39]

Q-learning,

C. J. Watkins and P. Dayan, “Q-learning,” Machine learning, vol. 8, pp. 279–292, 1992

1992
[40]

Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario,

H. Zhang, S. Feng, C. Liu, Y . Ding, Y . Zhu, Z. Zhou, W. Zhang, Y . Yu, H. Jin, and Z. Li, “Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario,” in The world wide web conference, 2019, pp. 3620–3624

2019
[41]

Traffic signal timing manual,

P. Koonce et al., “Traffic signal timing manual,” United States. Federal Highway Administration, Tech. Rep., 2008

2008
[42]

Max pressure control of a network of signalized intersec- tions,

P. Varaiya, “Max pressure control of a network of signalized intersec- tions,” Transportation Research Part C: Emerging Technologies, vol. 36, pp. 177–195, 2013

2013
[43]

Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control,

C. Chen, H. Wei, N. Xu, G. Zheng, M. Yang, Y . Xiong, K. Xu, and Z. Li, “Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, no. 04, 2020, pp. 3414–3421

2020
[44]

Attendlight: Universal attention-based reinforcement learning model for traffic signal control,

A. Oroojlooy, M. Nazari, D. Hajinezhad, and J. Silva, “Attendlight: Universal attention-based reinforcement learning model for traffic signal control,” Advances in Neural Information Processing Systems , vol. 33, pp. 4079–4090, 2020

2020
[45]

Presslight: Learning max pressure control to coordinate traffic signals in arterial network,

H. Wei, C. Chen, G. Zheng, K. Wu, V . Gayah, K. Xu, and Z. Li, “Presslight: Learning max pressure control to coordinate traffic signals in arterial network,” in Proceedings of the 25th ACM SIGKDD inter- national conference on knowledge discovery & data mining , 2019, pp. 1290–1298

2019
[46]

Colight: Learning network-level cooperation for traffic signal control,

H. Wei, N. Xu, H. Zhang, G. Zheng, X. Zang, C. Chen, W. Zhang, Y . Zhu, K. Xu, and Z. Li, “Colight: Learning network-level cooperation for traffic signal control,” in Proceedings of the 28th ACM international conference on information and knowledge management, 2019, pp. 1913– 1922

2019
[47]

Efficient pres- sure: Improving efficiency for signalized intersections,

Q. Wu, L. Zhang, J. Shen, L. L ¨u, B. Du, and J. Wu, “Efficient pres- sure: Improving efficiency for signalized intersections,” arXiv preprint arXiv:2112.02336, 2021

2021 arXiv
[48]

Large language models as traffic signal control agents: Capacity and opportunity,

S. Lai, Z. Xu, W. Zhang, H. Liu, and H. Xiong, “Large language models as traffic signal control agents: Capacity and opportunity,”arXiv preprint arXiv:2312.16044, 2023. 15 Wenchang Duan (Graduate Student Member, IEEE) received the B.S. degree in statistics from the Xi- angtan ...

2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

A review of sustainable transportation solutions: Innovations, challenges, and future directions,

E. A. Etukudoh, A. Adefemi, V . I. Ilojianya, A. A. Umoh, K. I. Ibekwe, and Z. Q. S. Nwokediegwu, “A review of sustainable transportation solutions: Innovations, challenges, and future directions,” World Journal of Advanced Research and Reviews, vol. 21, no. 1, pp. 1440–1452, 2024

work page 2024

[2] [2]

Urban travel time and residential location choice: The impacts of traffic congestion,

M. Zhang, Z. Li, H. Si, L. Cheng, X. Zhou, and B. Wang, “Urban travel time and residential location choice: The impacts of traffic congestion,” Sustainable Cities and Society , vol. 99, p. 104975, 2023

work page 2023

[3] [3]

Changes in traffic congestion and air pollution due to major roadway infrastructure improvements in texas,

M. Harleman, L. Harris, M. D. Willis, B. Ritz, P. Hystad, and E. L. Hill, “Changes in traffic congestion and air pollution due to major roadway infrastructure improvements in texas,” Science of The Total Environment, vol. 898, p. 165463, 2023

work page 2023

[4] [4]

Traffic management system and traffic light control in smart city to reduce traffic congestion,

A. Wided, B. Assia, and B. Fatima, “Traffic management system and traffic light control in smart city to reduce traffic congestion,” International Journal of Automation and Smart Technology , vol. 13, no. 1, pp. 2464–2464, 2023

work page 2023

[5] [5]

Troubleshooting so- lution for traffic congestion control,

V . Tong, S. Souihi, H. A. Tran, and A. Mellouk, “Troubleshooting so- lution for traffic congestion control,” Journal of Network and Computer Applications, p. 103923, 2024

work page 2024

[6] [6]

What is the root cause of congestion in urban traffic networks: Road infrastructure or signal control?

W. Yue, C. Li, Y . Chen, P. Duan, and G. Mao, “What is the root cause of congestion in urban traffic networks: Road infrastructure or signal control?” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 7, pp. 8662–8679, 2022. 14

work page 2022

[7] [7]

Deep reinforcement learning for intelligent transportation systems: A survey,

A. Haydari and Y . Yılmaz, “Deep reinforcement learning for intelligent transportation systems: A survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 11–32, 2022

2022

[8] [8]

Reinforcement learning in urban network traffic signal control: A systematic literature review,

M. Noaeen, A. Naik, L. Goodman, J. Crebo, T. Abrar, Z. S. H. Abad, A. L. Bazzan, and B. Far, “Reinforcement learning in urban network traffic signal control: A systematic literature review,” Expert Systems with Applications, vol. 199, p. 116830, 2022

work page 2022