{"id":"4d7f68e3-e403-497e-8d9e-19aca7008112","arxiv_id":"2506.13836","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"T-REX, a new incident benchmark, shows that RL traffic signal controllers are not robust to incident-induced distribution shifts, and that no method dominates across network types.","lead":"This paper introduces T-REX, an open-source SUMO-based simulator that injects realistic traffic incidents into signal-control simulations, and uses it to compare four reinforcement learning methods under disruption. It finds that no RL controller is robust across all networks: independent and pressure-based methods degrade sharply under incidents, while a hierarchical method is more stable in large irregular networks but slower to train.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central comparative claim is not yet supported by the reported numbers: Table 2 contradicts the 'FMA2C most robust' conclusion, Table 3 and §4.2 contradict the 'incident training does not help' conclusion, and FMA2C's 14x larger training budget confounds the architecture comparison.","rationale":"Good-faith reading: the paper builds a plausible T-REX framework, uses standard SUMO benchmarks and four established RL-TSC algorithms, and reports extensive multi-network experiments. Those are real contributions. But the central claim is comparative: FMA2C is more robust and incident training does not transfer. That claim rests entirely on the robustness tables. The reported numbers contradict the narrative in the specific ways listed above, so the claim is not currently demonstrated. The simulation-realism concern raised by the reader is a real external-validity issue, but it is second-order: even if the ICM parameters were perfect, the paper would still need internally consistent metrics to conclude anything. The unequal training budgets especially matter because FMA2C's main reported advantage, stability and robustness, could be an artifact of 14x more training episodes rather than the feudal architecture. A recomputation from raw logs or code would settle this. If the recomputation confirms the paper's narrative, the conditional acceptance is appropriate; if it does not, the central conclusion should be revised. I therefore keep the reader's CONDITIONAL verdict but flag that the condition should include metric re-derivation, not only artifact release and behavioral calibration.","tokens_in":21748,"tokens_out":9204,"duration_ms":96332,"concrete_test":"Obtain the raw training and testing episode curves (the paper promises release) and recompute LSI, FPD, CR, AUC, RAUC, and PDI exactly as defined in §3.3, correcting the apparent Table D1 copy-paste rows for Ingolstadt Region. Then run two checks: (1) match training budgets by evaluating FMA2C and IPPO at 100 episodes and IDQN and MPLight at 1,400 episodes, and verify whether FMA2C still has the lowest RAUC in Ingolstadt Region and Cologne Corridor; (2) compare Base-incident versus Incident-incident PDI across all five networks and verify whether base-trained policies are better in a majority of cells. If either check fails, the headline conclusions about hierarchical robustness and the ineffectiveness of incident exposure are unsupported by the reported data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the headline claim, the robustness metrics must show that (a) FMA2C is more stable under incidents and (b) incident exposure during training does not improve incident-test performance. The paper's own tables fail both checks. (i) Training budgets are unequal: §3.4 trains IDQN and MPLight for 100 episodes but IPPO and FMA2C for 1,400 episodes, so FMA2C's lower RAUC or PDI could reflect 14x more environment interactions rather than hierarchical coordination. (ii) In Table 2, RAUC is defined as lower-is-better, yet in Ingolstadt Region FMA2C (9.384) is said to achieve the lowest RAUC while MPLight (-10.584) and IPPO (-21.715) are lower; in Cologne Corridor IPPO (-39.830) is the minimum, not FMA2C (40.056). Thus the claim that FMA2C 'consistently demonstrates the highest robustness' is contradicted by its own table. (iii) In Table 3, comparing Base-incident versus Incident-incident PDI (the relevant columns for whether incident training helps), incident-trained policies have lower PDI in 12 of 15 method-network cells; this is the opposite of the §4.2 claim that base-trained policies generalize better, and the paper itself warns that PDI is 'less suited for comparing across training-testing pairs' before using it that way. (iv) Table D1 contains apparent copy-paste errors: the Ingolstadt Region Base-Base rows for IDQN, MPLight, and FMA2C are identical to the Ingolstadt Corridor rows, so the Ingolstadt Region PDI evidence, which is the key large-irregular-network case, is unreliable. These are internal inconsistencies, not merely calibration concerns, so the central claim is not established until they are resolved.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces T-REX, an open-source SUMO-based framework for training and evaluating RL-based traffic signal control under lane-blocking incidents. T-REX models driver rerouting with the Information Comply Model, speed adaptation via stopping-sight-distance, and modified lane-changing behavior, and it proposes robustness metrics (LSI, FPD, CR, AUC, RAUC, PDI). The authors compare IDQN, IPPO, MPLight, and FMA2C with rule-based baselines on one synthetic grid and four real-world networks. The paper's central claims are that no RL-TSC paradigm dominates across all robustness dimensions, that independent value-based and decentralized pressure-based methods degrade sharply under incidents, that hierarchical coordination (FMA2C) is more stable in large irregular networks at the cost of slower convergence, and that incident exposure during training does not improve incident-test performance. The framework and metric suite are useful and the code release is a clear strength, but the empirical evidence as reported contains internal contradictions and confounds that prevent the central claims from being accepted as stated.","tokens_in":22177,"tokens_out":5625,"duration_ms":57948,"significance":"If the claims were fully supported, the paper would make a valuable contribution: it addresses a real gap in RL-TSC evaluation by moving from localized incident modeling to network-level congestion propagation with rerouting and behavioral responses, and it provides a standardized, open benchmarking platform. The proposed metrics go beyond conventional travel-time comparisons and target deployment-relevant dimensions such as learning stability and transferability. The release of source code and the use of established baseline implementations are commendable. However, the central comparative claims are currently weakened by unequal training budgets, a numerical contradiction in the main robustness table, and a conclusion about incident training that is opposite to the paper's own PDI table. These issues would need to be resolved before the result can be relied on by the community.","major_comments":[{"comment":"The learning-performance comparison is confounded by unequal training budgets: IDQN and MPLight are trained for 100 episodes, while IPPO and FMA2C are trained for 1,400 episodes. Since RAUC and AUC are integrated over the training trajectory, FMA2C's lower RAUC (e.g., 9.384 in Ingolstadt Region) cannot be attributed to hierarchical coordination alone; it may simply reflect 14 times more environment interactions. The claim in §4.1 that FMA2C 'consistently demonstrates the highest robustness' is therefore not supported as reported. Please either use matched training budgets, report sample-efficiency-adjusted metrics, or explicitly acknowledge and control for the confound.","section":"§3.4 and Table 2"},{"comment":"The statement that FMA2C achieves the lowest RAUC in several networks is contradicted by Table 2's own numbers. With RAUC defined so that lower is better, in Ingolstadt Region MPLight (-10.584) and IPPO (-21.715) have lower RAUC than FMA2C (9.384), and in Cologne Corridor IPPO (-39.830) has the minimum value while FMA2C is 40.056. The text's example ('only 9.38% in the Ingolstadt Region') is factually incorrect. The robustness ranking needs to be corrected, and ideally accompanied by confidence intervals or significance tests given the large variances visible in the underlying results.","section":"Table 2 and §4.1"},{"comment":"The conclusion that 'base-trained policies generalize better than incident-trained ones' is not supported by Table 3. Comparing the Base-incident and Incident-incident PDI columns—the relevant contrast for whether incident training helps at incident test time—incident-trained policies have lower PDI in 12 of 15 method-network cells, which is the opposite of the stated finding. Furthermore, the manuscript itself notes that PDI is 'less suited for comparing across training-testing pairs' immediately before using it to make that cross-pair comparison. The PDI values are reported without error bars, despite the very large standard deviations in Table D1, so even the ordinal comparisons are not statistically grounded. Please re-analyze this experiment with appropriate comparison criteria and uncertainty quantification.","section":"§4.2 and Table 3"},{"comment":"The Ingolstadt Region rows in Table D1 contain apparent copy-paste errors: the Base-Base entries for IDQN (72.64 ±1.02, 73.41 ±1.95), MPLight (101.01 ±62.35, 82.81 ±16.34), and FMA2C (90.65 ±9.76, 140.90 ±18.09) are identical to the Ingolstadt Corridor entries in the same table. Since Ingolstadt Region is the key large, irregular network supporting the FMA2C-robustness narrative, these errors invalidate the corresponding PDI values in Table 3 and the conclusions drawn from them. The experiments need to be re-run or correctly reported.","section":"Appendix D, Table D1"},{"comment":"The realism of the claim that T-REX models 'realistic network-level performance' rests on behavioral parameters—ICM awareness and rerouting parameters (µ_FTI=0.7, µ_ON=0.5, µ_OS=0.8, β_gain=2.5, β_loss=2.5, β_0=-5), driver-type proportions, the conservative 8 km/h crawl speed, and modified SUMO lane-changing—which are 'inferred or assumed' according to the paper's own limitation paragraph. No sensitivity analysis or calibration against empirical incident data is provided. Because all method rankings are generated under these assumptions, the real-world deployment-readiness conclusions should either be tempered or accompanied by a sensitivity study over these parameters.","section":"Appendix B and §5"}],"minor_comments":[{"comment":"Reporting the best 10 consecutive episodes for IDQN/MPLight and the best 100 for IPPO/FMA2C is a post-hoc selection that overstates final performance; please also report the mean and standard deviation over fixed final windows or the full learning curves.","section":"§3.4"},{"comment":"In the Ingolstadt Corridor panel, Greedy appears twice (rows with 79.79 s and 72.43 s in base conditions); the second row is likely a different method and should be corrected.","section":"Table 1"},{"comment":"The rolling-average windows differ between panels (5 episodes for IDQN/MPLight, 30 for FMA2C), which makes visual comparison misleading; a consistent smoothing window should be used.","section":"§4.3, Figures 4–5"},{"comment":"The phrase 'a open-source' appears in the abstract and contributions; it should read 'an open-source'.","section":"Abstract and Section 1"},{"comment":"The column header 'Test (avg. 100)' is inconsistent with the description of the selection protocol, and it is unclear whether the reported values are means over test episodes, over seeds, or both; this should be stated explicitly.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong framework component and the open-source release is valuable, but the empirical sections as written do not yet support the central comparative claims. The internal contradictions in Tables 2 and 3, combined with the Ingolstadt Region copy-paste error in Table D1, mean that a careful re-analysis and re-reporting is required rather than a simple textual revision. I would condition acceptance on the authors verifying and correcting the numerical claims, matching training budgets or explicitly adjusting conclusions, and providing uncertainty estimates for the proposed robustness metrics."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nI read the paper on RL-TSC robustness under incidents. The idea is good: build a SUMO-based benchmark with realistic incident responses (probabilistic rerouting, SSD-based speed adaptation, contextual lane-changing), propose robustness metrics, and compare four RL methods across five networks. That is genuinely useful for a subfield that usually tests only stable traffic. The framework and the metric suite are the main contributions, and the authors are honest that behavioral parameters are inferred or assumed.\n\nThe problem is that the headline results don't survive contact with the paper's own tables. The claim that FMA2C 'consistently demonstrates the highest robustness' is contradicted by Table 2: in Ingolstadt Region, IPPO has RAUC -21.7 and MPLight -10.6, both lower than FMA2C's 9.4; in Cologne Corridor, IPPO is -39.8 and FMA2C is 40.1. The claim that 'base-trained policies generalize better than incident-trained ones' is contradicted by Table 3: comparing Base-incident PDI with Incident-incident PDI, incident training gives lower degradation in 12 of 15 method-network cells. The paper itself warns that PDI is 'less suited for comparing across training-testing pairs,' then uses it exactly that way.\n\nThere are also structural confounds. Value-based methods get 100 training episodes; policy-gradient methods get 1,400. So any FMA2C advantage could just be 14x more environment interactions. Table D1 has copy-paste errors: the Ingolstadt Region Base-Base rows for IDQN, MPLight, and FMA2C are identical to the Ingolstadt Corridor rows, which undermines the key large-network PDI evidence. And the reproducibility statement says code will be released 'upon publication,' despite the 'open-source' claim in the abstract.\n\nThese are fixable, but they are not cosmetic. The comparative rankings and the central robustness conclusions are currently unsupported by the reported numbers. What survives is the framework itself and the metric definitions, which could become community assets if the code is actually released and the analyses corrected.\n\nMy recommendation: send to peer review, but the referee should insist on corrected tables, released code, and an honest handling of the training-budget confound (e.g., matched budgets in a robustness comparison). If the authors do that, this becomes a citable benchmark. As it stands, I would not cite the comparative results.","headline":"A useful incident-robustness benchmark, but the paper's own tables contradict its headline rankings; major revisions needed before the comparative claims can be trusted.","tokens_in":22718,"tokens_out":3349,"would_cite":false,"duration_ms":29971,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Current RL traffic-signal controllers are not robust to incidents, and no single method dominates across networks or robustness metrics.","keywords":["reinforcement learning","traffic signal control","incident robustness","multi-agent reinforcement learning","simulation benchmarking","distribution shift","rerouting model","traffic network"],"falsifier":"Compare T-REX's predicted network response to a real incident of known location and duration, or to a high-fidelity simulation calibrated with GPS and loop-detector data: measure the share of rerouting drivers, speed profiles near the blocked lane, and queue spillback over time. If these quantities deviate substantially from the Information Comply Model and Stopping Sight Distance assumptions, the method rankings from the paper's experiments would not reproduce in reality.","tokens_in":21550,"feed_emoji":"🚦","tokens_out":8206,"duration_ms":81054,"temperature":0.7,"pith_summary":"Reinforcement-learning traffic signal control (RL-TSC) is usually trained and evaluated on stable, incident-free traffic, so its behavior under real disruptions such as lane-blocking incidents is largely unknown. This paper introduces T-REX, an open simulation framework that injects incidents into traffic networks with probabilistic driver rerouting, speed adaptation, and lane-changing, and uses it to compare four representative RL-TSC methods plus rule-based baselines across five networks. The central finding is that robustness is not a property of any one RL paradigm: independent value-based and decentralized pressure-based methods converge quickly and perform well under normal conditions but degrade sharply under incident-driven distribution shifts, while hierarchical coordination is more stable in large, irregular networks at the cost of slower convergence. A secondary result is that training with incident exposure does not improve test-time performance unless the method explicitly accounts for the uncertainty behind incidents. If these results hold, current RL-TSC methods are not yet reliable for deployment under disruptions, and robustness must be designed in rather than assumed.","feed_headline":"RL traffic signal controllers fail incident stress tests","feed_subtitle":"Independent and pressure-based controllers degrade under incidents; hierarchical coordination is steadier in big cities.","key_machinery":"The load-bearing mechanism is T-REX itself: a simulation framework that deploys incidents as blocking vehicles at chosen edges and then models driver behavior through three coupled modules—an Information Comply Model for probabilistic rerouting (awareness via radio, roadside signs, online sources, and direct observation, followed by a logistic rerouting choice), a Stopping Sight Distance-based speed adaptation that makes vehicles slow to about 8 km/h near the incident, and modified lane-changing that makes drivers move away from blocked lanes early. These modules produce network-level congestion propagation, including rerouting and queue spillback, that simpler capacity-reduction incident models miss. The robustness evaluation also rests on a suite of metrics—Learning Stability Index, Final Performance Deviation, Convergence Rate, Area Under the Curve, Relative AUC Difference, and Performance Degradation Index—that measure learning stability and generalization beyond raw travel time.","core_discovery":"The paper claims that incident-induced distribution shifts break the generalization of most current RL-TSC controllers, and that the severity of the break depends on network topology and on the controller's coordination architecture. Using T-REX, the authors show that IDQN (each intersection learns its own deep Q-network) and MPLight (decentralized pressure-based coordination) suffer large performance losses under incidents, while FMA2C, a hierarchical manager-worker actor-critic, maintains more stable travel times in large irregular networks such as the Ingolstadt region, although it learns much more slowly. The paper also reports that policies trained in incident-free base scenarios often generalize better to incidents than policies trained with incidents, because incident-trained agents overfit to the specific incident configurations they saw; exposure to incidents alone is therefore not a robustness strategy. The authors conclude that no single method dominates across all robustness metrics and networks, and that current RL-TSC methods are not ready for reliable deployment under incidents.","pith_inferences":["The reported rankings are conditional on the assumed driver-response parameters, such as rerouting probabilities and the 8 km/h crawl speed; calibrating these against real incident data could change which architecture looks best.","A natural next step the authors leave implicit is to use their metrics to test robustness to other disruptions, such as sensor failures, demand surges, or adversarial traffic, within the same framework.","The counterintuitive result that base-trained policies beat incident-trained ones suggests a testable hypothesis: domain randomization over incident configurations should outperform targeted incident training, which could be checked directly with T-REX.","If incident robustness becomes a training objective, value-based methods might regain competitiveness with explicit out-of-distribution regularization or uncertainty-aware exploration, since their fast convergence is otherwise attractive."],"forward_implications":["Independent and decentralized RL-TSC methods cannot be assumed to survive lane-blocking incidents; their favorable normal-condition results do not transfer to disrupted networks.","Hierarchical coordination architectures are the most defensible choice for large, irregular networks under incidents, but only where their slower training and sample inefficiency can be afforded.","Training on incidents alone can be counterproductive: without explicit incident-awareness or uncertainty handling, it overfits to specific disruption configurations.","Robustness comparisons need standardized incident dynamics and multi-dimensional metrics; raw travel-time improvements in clean conditions are insufficient evidence for deployment.","T-REX provides a reproducible way to benchmark future RL-TSC methods under incidents, supporting pre-training, online-learning, and hybrid deployment evaluations."],"supporting_citations":[{"why":"Supplies the Information Comply Model that drives probabilistic rerouting in T-REX.","marker":"Kucharski and Gentile (2019)"},{"why":"Provides the RESCO RL-TSC benchmark, network scenarios, and open-source implementations of IDQN, MPLight, IPPO, and FMA2C.","marker":"Ault and Sharon (2021)"},{"why":"Contributes the 4x4 grid network and the MPLight pressure-based coordination method.","marker":"Chen et al. (2020)"},{"why":"Defines the FMA2C hierarchical manager-worker actor-critic method whose robustness the paper highlights.","marker":"Ma and Wu (2020)"},{"why":"Provides the Stopping Sight Distance formulas used for the speed-adaptation module.","marker":"AASHTO (2018)"},{"why":"Supplies real-world incident-duration data used to sample durations in the Initializer.","marker":"Pereira et al. (2013)"},{"why":"Underlies the microscopic traffic simulator on which T-REX is built.","marker":"Lopez et al. (2018)"},{"why":"Defines the lane-changing model that T-REX modifies for contextual incident avoidance.","marker":"Erdmann (2015)"}],"fun_headline_variants":["Incident shifts break RL traffic signal controllers","Hierarchical RL traffic control steady under incidents, independent methods fail","Incident training can overfit traffic lights, harming robustness","RL traffic signal robustness varies by network and coordination","No single RL traffic controller dominates incident disruptions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the simulated drivers—how they learn about an incident, decide to reroute, slow to 8 km/h, and change lanes—match real driver behavior; if that behavior differs, the congestion patterns and the ranking of control methods will not transfer to the real world.","fun_headline_variants_meta":{"raw":{"variants":["Incident shifts break RL traffic signal controllers","Hierarchical RL traffic control steady under incidents, independent methods fail","Incident training can overfit traffic lights, harming robustness","RL traffic signal robustness varies by network and coordination","No single RL traffic controller dominates incident disruptions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001035,"raw_usage":{"total_tokens":4382,"prompt_tokens":996,"completion_tokens":3386,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":3312}},"tokens_in":612,"tokens_out":3386,"duration_ms":27381,"temperature":1.0,"reasoning_tokens":3312,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:36:48.438182+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare T-REX's predicted network response to a real incident of known location and duration, or to a high-fidelity simulation calibrated with GPS and loop-detector data: measure the share of rerouting drivers, speed profiles near the blocked lane, and queue spillback over time. If these quantities deviate substantially from the Information Comply Model and Stopping Sight Distance assumptions, the method rankings from the paper's experiments would not reproduce in reality.","supporting_citations":[{"cited_title":"Transportation Research Part B: Methodological 126:414--441","cited_arxiv_id":null,"evidence_quote":"Supplies the Information Comply Model that drives probabilistic rerouting in T-REX."},{"cited_title":"In: Proc","cited_arxiv_id":null,"evidence_quote":"Provides the RESCO RL-TSC benchmark, network scenarios, and open-source implementations of IDQN, MPLight, IPPO, and FMA2C."},{"cited_title":"In: Proc","cited_arxiv_id":null,"evidence_quote":"Contributes the 4x4 grid network and the MPLight pressure-based coordination method."},{"cited_title":"AASHTO, Washington, D.C., commonly referred to as the AASHTO Green Book","cited_arxiv_id":null,"evidence_quote":"Provides the Stopping Sight Distance formulas used for the speed-adaptation module."},{"cited_title":"Transportation Research Part C: Emerging Technologies 37:177--192","cited_arxiv_id":null,"evidence_quote":"Supplies real-world incident-duration data used to sample durations in the Initializer."},{"cited_title":"In: 2018 21st international conference on intelligent transportation systems (ITSC), Ieee, pp 2575--2582","cited_arxiv_id":null,"evidence_quote":"Underlies the microscopic traffic simulator on which T-REX is built."},{"cited_title":"In: Modeling Mobility with Open Data: 2nd SUMO Conference 2014 Berlin, Germany, May 15-16, 2014, Springer, pp 105--123","cited_arxiv_id":null,"evidence_quote":"Defines the lane-changing model that T-REX modifies for contextual incident avoidance."}],"review_version":1}