{"id":"599a9654-dadd-4420-b57d-146870868ac2","arxiv_id":"2411.19359","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Coordinated multi-agent reinforcement learning transit signal priority reduces simulated bus travel time by 27% across two intersections, outperforming independent agents (22%) while keeping side street delay increases small.","lead":"This paper trains multi-agent reinforcement learning agents to control traffic signals at two coordinated intersections, then adds separate event-based agents that give buses priority when they approach. Across simulation tests, coordinated bus-priority agents cut bus travel time by 27% versus no priority, with only modest increases in side street delay.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The coordinated-vs-independent TSP comparison is confounded by different reward functions (Eq. 7 vs Eq. 8), so the 27% vs 22% difference cannot be attributed to coordination alone.","rationale":"The reader's weakest assumption focuses on the fixed training seed and generalization to test seeds, which is a valid robustness concern. However, the most load-bearing issue is more basic: the independent and coordinated TSP agents are not trained with the same objective, so the 27% vs 22% comparison does not isolate coordination. The paper itself states that the coordinated reward function is 'specifically focused on multi-intersections operations,' which is precisely the confound. The reported side-street delays are measured only for a 300-second window after bus check-in and lack statistical tests, but the reward mismatch directly undermines the central comparative claim. This is addressable by a matched-reward ablation, so the appropriate verdict remains conditional (as the reader concluded) rather than accept or reject. The reader's rationale mentions 'confounded experimental design' and the need to 'isolate coordination as the only varying factor,' which is partial agreement with our concern, even though the reader's formal weakest assumption was the fixed seed.","tokens_in":12688,"tokens_out":6233,"duration_ms":53257,"concrete_test":"Retrain the independent TSP agents under DTDE using exactly the same reward function as the coordinated agents (Eq. 8) with the same W_bd and W_bv values, while keeping all other settings unchanged (episode length, random seed, activation logic). Then compare the Inter A&B_EB bus travel time reduction against the no-TSP baseline. If the independent agents' reduction rises to approximately 27%, the reported coordination benefit is an artifact of the reward mismatch; if it remains near 22%, coordination itself is the cause.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central comparative claim (Section 4.3b) is that coordinated CTDE-based TSP agents reduce bus travel time across the two intersections by 27%, versus 22% for independent DTDE agents, implying coordination is the cause. However, the two frameworks are trained with different reward functions. Independent TSP agents use Eq. 7, which depends only on bus delay, bus speed, and a side-street queue penalty; coordinated TSP agents use Eq. 8, which additionally includes the average general-traffic delay (sum d_i / n) and weighting factors W_bd and W_bv that are never reported. Because the objectives differ, the independent agents may be optimizing purely for bus progress while the coordinated agents balance bus priority against general traffic delay; any difference in measured bus travel time could stem from this objective mismatch rather than from multi-agent coordination. The comparison is also complicated by different training episode lengths (4 hours for independent vs 1 hour for coordinated) and different activation logic (independent agents activate only at the intersection the bus is approaching; coordinated agents activate at the upstream intersection). The paper lists the reward difference itself as a 'contributing factor' (Section 4.3b), acknowledging that the experimental design does not isolate coordination. Without a matched-reward ablation, the headline 27% vs 22% claim is not supported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript integrates Transit Signal Priority (TSP) into multi-agent reinforcement learning (MARL) based traffic signal control for a two-intersection corridor in Vissim. The first part trains a VDN-based CTDE background signal controller and compares it with coordinated actuated signal control (ASC), finding slightly better overall intersection delay at v/c = 0.95. The second part develops event-based TSP agents in two variants: independent agents trained under DTDE and coordinated agents trained under CTDE with VDN. The paper reports that, relative to a no-TSP baseline, independent TSP agents reduce bus travel time across both intersections by 22% and coordinated TSP agents by 27%, with only slight side-street delay increases for most movements. The manuscript also reports that coordinated training is more stable than independent training, although both converge to similar bus delay values.","tokens_in":12991,"tokens_out":2485,"duration_ms":23937,"significance":"If the results are robust, the paper makes a useful empirical contribution by demonstrating an event-based TSP formulation that decouples TSP from background signal control, and by comparing DTDE and CTDE training paradigms in a microscopic simulation environment. The paper ships a concrete experimental setup with explicit state, action, and reward definitions, and it reports side-street delay impacts rather than only bus benefits, which is a strength. The main comparative claim, however, that coordinated TSP outperforms independent TSP (27% versus 22% bus travel time reduction), is not supported as a statement about coordination per se, because the two TSP variants differ in reward functions, episode lengths, and activation logic. The more basic claim that TSP reduces bus delay relative to no TSP is visible in the training and test plots and is likely sound, pending statistical validation.","major_comments":[{"comment":"The headline comparison between coordinated and independent TSP agents is confounded. The coordinated agents are trained with a reward that includes general traffic delay and weighting factors W_bd and W_bv (Equation 8), while the independent agents use a reward that depends only on bus delay, bus speed, and a side-street queue penalty (Equation 7). The two variants also use different episode lengths (4 hours for independent, 1 hour for coordinated) and different activation logic. The manuscript itself lists the reward difference as a 'contributing factor' (Section 4.3b), which acknowledges that the experimental design does not isolate coordination. To support the claim that coordination causes the additional 5% travel time reduction, the authors should either run a matched-reward ablation or explicitly reframe the result as a comparison of two system designs rather than a test of coordination.","section":"Section 4.3(b), Equations (7) and (8)"},{"comment":"The weighting factors W_bd and W_bv in the coordinated TSP reward are never reported, and neither are the queue thresholds qlTh1, qlTh2, or the offset tolerance Delta theta. These parameters are load-bearing for the coordinated TSP agent's behavior and for reproducibility. Without them, a reader cannot reconstruct the training setup, and the comparison between the two TSP variants is further obscured because the independent and coordinated agents optimize different objectives with unknown relative weights.","section":"Section 3.3.4, Equation (8)"},{"comment":"All training episodes use the same random seed ('the same random seed utilized for all runs' in Section 3.3.3, repeated for the TSP training in Sections 4.2 and 4.3). The test runs use random arrival profiles, but the paper reports only average values over 10 runs without confidence intervals or statistical tests. The claim that TSP 'significantly reduces' bus travel time (Section 4.2b) is therefore not statistically supported, and the robustness of the trained policies to the specific training seed is not established. The authors should report per-run variability, perform a paired statistical test, and ideally retrain with multiple seeds to show that the delay reductions are not artifacts of the single training realization.","section":"Sections 3.3.3, 4.2, and 4.3"},{"comment":"The side-street delay analysis covers only four selected movements (A_SB_TH, A_SB_LT, B_SB_TH, B_SB_LT) and a five-minute post-check-in window. The abstract's statement that 'there is only a slight increase in delay for a majority of the side street movements' is not supported by a complete movement-level analysis, and the reported 18% increase for B_SB_LT in the independent-TSP case is not negligible. A full enumeration of side-street movements and a quantitative statement of the distribution of delay impacts would be needed to substantiate the claim.","section":"Section 4.2(c) and 4.3(c)"}],"minor_comments":[{"comment":"The manuscript contains several typographical and formatting errors, including 'und er' in Section 3.3.1 and inconsistent spacing around equations; a careful proofread is needed.","section":"General"},{"comment":"The reward in Equation (4) uses penalty magnitudes of -9999 and +100, but the text does not explain how these values were chosen or whether the results are sensitive to them.","section":"Section 3.3.2"},{"comment":"The comparison between VDN MARL and ASC is described as 'slightly better' without a statistical test; the box plots show overlap and high variability, so a formal comparison (e.g., paired t-test or Wilcoxon test over the 10 runs) would strengthen the claim.","section":"Section 4.1"},{"comment":"Figure 8 does not include error bars or confidence intervals, unlike the box plots in Figure 3, making it difficult to assess whether the 27% versus 22% difference is within run-to-run variability.","section":"Section 4.3(b)"},{"comment":"References [34] and [44] appear to be the same work (Long and Chung, 2023) cited under different numbers; the duplication should be resolved.","section":"Section 2.3(b)"}],"recommendation":"major_revision","confidential_remarks":"The paper's central comparative claim is not yet supported because the independent versus coordinated TSP comparison is confounded by multiple factors, principally the different reward functions. The authors appear aware of this to some degree, but the abstract and conclusions still assert coordination as the cause of the improvement. The paper would be acceptable after a matched-reward ablation or a carefully worded reinterpretation. The fixed-seed training issue is also worth raising editorially, as it affects the generalizability of all reported results, though the test runs do provide some limited evidence of generalization."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a solid simulation study that builds on the authors' prior single-agent TSP work and extends it to two intersections with MARL. The separation of background signal control from event-based TSP agents is a practical and sensible design; it avoids the sparse-reward problem of training one agent for both tasks. The side street impact analysis, using a 5-minute post-check-in window, is a nice touch. The paper shows that both independent and coordinated TSP agents reduce bus travel time substantially compared with no TSP (22% and 27%), with only small side street increases for most movements.\n\nThe soft spots are real but not fatal. The headline coordination claim—that CTDE beats DTDE—is not supported by the experiments as designed. The two TSP variants differ in reward functions (Eq. 7 vs 8), episode length (4h vs 1h), and activation logic. The paper itself acknowledges the reward difference as a contributing factor. So the 27% vs 22% gap cannot be attributed to coordination alone. A matched reward ablation would be needed. That said, the main result—TSP reduces bus delay relative to no TSP—is visible in the learning curves and test plots, and the paper doesn't overclaim much in the conclusions.\n\nA second concern is training on a single random seed. All episodes use the same seed, so the agents see one arrival pattern during training and are tested on different seeds. That's a generalization gap, and the paper doesn't address it. At least one multi-seed training experiment would strengthen the robustness claim. Also missing: significance tests or confidence intervals for the 22% and 27% reductions, and the values of Wbd, Wbs, and the queue thresholds. These are easy to report.\n\nThe math and reinforcement learning machinery are standard and appear sound. The citation pattern is appropriate; the authors build on their own prior work and the related MARL TSP literature.\n\nWho should read this: people working on RL-based TSP or adaptive signal control, and transit agencies evaluating TSP strategies. It's a useful data point, even if the coordination comparison needs redoing.\n\nRecommendation: send it to peer review, but with major revisions. The authors should isolate coordination as the only variable, report the missing parameters, and add seed-generalization evidence. A serious referee would catch these, so it should not be desk-rejected.","headline":"Useful event-based TSP design with a confounded coordination comparison; the bus-delay reduction vs no-TSP is credible, but the 27% vs 22% claim needs matched experiments.","tokens_in":13541,"tokens_out":1915,"would_cite":true,"duration_ms":16660,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Coordinated multi-agent transit signal priority cuts bus travel time by 27% across two intersections, versus 22% for independent agents.","keywords":["transit signal priority","multi-agent reinforcement learning","value decomposition networks","traffic signal control","connected vehicles","bus delay","traffic microsimulation"],"falsifier":"Retrain independent and coordinated TSP agents on several random seeds, or on several distinct demand patterns, and evaluate on held-out seeds; if the reported 22% and 27% bus-travel-time reductions disappear outside the single training seed, the central result is specific to that traffic realization.","tokens_in":12463,"feed_emoji":"🚌","tokens_out":4768,"duration_ms":39596,"temperature":0.7,"pith_summary":"This paper tries to show that transit signal priority (TSP) can be built on top of a multi-agent reinforcement learning traffic controller, and that coordinating the two intersection agents during training is better than training them independently. It first trains one background controller per intersection with a value decomposition network, matching or slightly beating coordinated actuated control at a volume-to-capacity ratio of 0.95. It then adds event-based TSP agents that activate only when a connected bus approaches. In test runs, coordinated TSP agents reduce bus travel time across the two intersections by 27% compared with no priority, while independently trained agents achieve 22%, with only slight delays added to most side-street movements. The paper's practical claim is that a centralized-but-decentralized-execution framework gives stable training and measurably better bus progression than independent learning.","feed_headline":"Coordinated MARL transit priority cuts bus travel time 27%","feed_subtitle":"Two-agent value-decomposition training beats independent agents (22%) while raising side-street delay only slightly.","key_machinery":"The central machinery is the Value Decomposition Network (VDN), which factorizes the joint action-value function as the sum of per-agent values, $Q_{\\mathrm{tot}} = \\sum_i Q_i(h_i, a_i)$, enabling centralized training with decentralized execution. TSP itself is event-based: it activates only when a connected bus enters an 800-foot communication zone, with bus position and speed encoded in 25-foot cells, and the TSP agent selects the next signal phase from the same four-phase set used by the background controller. Independent TSP agents use a local reward of bus delay, bus speed, and a side-street queue penalty; coordinated TSP agents use a global reward that averages local rewards combining general traffic delay, weighted bus delay, and bus speed. This lets the downstream intersection receive advance notice and adjust its timing before the bus arrives.","core_discovery":"On its own terms, the paper establishes that event-based transit signal priority can be layered onto a learned multi-agent signal controller without retraining the background traffic control agents. Two TSP formulations are compared: independent agents trained and executed separately at each intersection, and coordinated agents centrally trained under a value decomposition network but executed locally. Both converge to the same bus delay value of about 25 seconds, yet coordinated agents achieve a 27% reduction in bus travel time across the two intersections versus 22% for independent agents, and do so with markedly more stable training. Side-street delay increases only slightly for most movements within a five-minute window after bus check-in, indicating that the coordinated reward design limits the traffic cost of giving buses priority.","pith_inferences":["Extrapolating from the two-intersection result, the coordination advantage should grow with corridor length and with midblock bus stops, because downstream signals get more lead time; this is a testable extension the paper does not run.","The single-seed training design is the main unaddressed risk: if the policies memorized one arrival pattern, both the 22% and 27% numbers could shrink under other demand realizations.","A natural next step is to replace delay and speed rewards with schedule-adherence or headway-equalization rewards, which would connect TSP to transit reliability rather than just travel time.","The assumption that all buses broadcast basic safety messages could be relaxed to test partial connected-vehicle penetration, where the state vectors would have missing entries."],"forward_implications":["TSP can be added as a separate event-triggered layer over an existing learned signal controller, so the background controller does not need to be retrained for bus priority.","Centralized training with VDN stabilizes multi-intersection TSP learning, where independent agents show large bus-delay variability even after convergence.","Coordinated TSP gives the downstream intersection advance notice of an approaching bus, which is one mechanism for the extra 5% travel-time reduction.","The reward structure keeps side-street impact bounded through queue penalties, so priority does not simply shift delay onto cross streets.","The same architecture should extend to longer corridors by giving each agent signal-state information from immediate neighbors."],"supporting_citations":[{"why":"Supplies the value decomposition network architecture used for centralized training of both background and coordinated TSP agents.","marker":"(45)"},{"why":"The authors' previous single-intersection event-based TSP study that this work extends to multiple coordinated intersections.","marker":"(36)"},{"why":"Provides the double deep Q-network architecture used for all agents in the study.","marker":"(49)"},{"why":"Supplies the invalid action masking algorithm that restricts agent phase choices to legal signal transitions.","marker":"(50)"},{"why":"Prior multi-agent reinforcement learning TSP implementation on an arterial using VDN, the immediate predecessor of this coordinated approach.","marker":"(44)"},{"why":"Empirical basis for the 300-second window used to measure side-street delay persistence after bus check-in.","marker":"(51)"},{"why":"A deep reinforcement learning TSP method in a connected environment that motivates the event-based TSP formulation.","marker":"(22)"}],"fun_headline_variants":["Coordinated TSP agents beat independent in bus delay: 27% vs 22%","MARL transit priority: coordinated agents cut bus delay 27%","Coordinated VDN TSP yields 27% bus delay cut, stable training","Independent TSP agents: 22% bus cut. Coordinated: 27%. Stability matters","Event-based TSP with MARL: coordinated agents shave 27% off bus delay"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All training episodes use the same random seed, so a learned policy might be matching one specific traffic pattern rather than generalizing to the different seeds used in testing.","fun_headline_variants_meta":{"raw":{"variants":["Coordinated TSP agents beat independent in bus delay: 27% vs 22%","MARL transit priority: coordinated agents cut bus delay 27%","Coordinated VDN TSP yields 27% bus delay cut, stable training","Independent TSP agents: 22% bus cut. Coordinated: 27%. Stability matters","Event-based TSP with MARL: coordinated agents shave 27% off bus delay"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000543,"raw_usage":{"total_tokens":2599,"prompt_tokens":945,"completion_tokens":1654,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":1544}},"tokens_in":561,"tokens_out":1654,"duration_ms":11127,"temperature":1.0,"reasoning_tokens":1544,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:17:46.394078+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain independent and coordinated TSP agents on several random seeds, or on several distinct demand patterns, and evaluate on held-out seeds; if the reported 22% and 27% bus-travel-time reductions disappear outside the single training seed, the central result is specific to that traffic realization.","supporting_citations":[],"review_version":1}