{"id":"c26838f8-d945-4483-aed2-027eaf06136e","arxiv_id":"2505.16833","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Strategic link scores quantify dependencies between planned actions as the drop in probability of a set-up action when its pay-off action is constrained, with applications to RL explanation, safe recommendations, and planning-horizon measurement.","lead":"This paper defines a strategic link score that measures how much one planned action is taken only to enable a later action, computed as the drop in its probability when the later action is blocked. It shows the score can explain RL agent behavior, make policy recommendations safer, and reveal how far ahead simulated drivers plan in a traffic simulator.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Road closure changes environment dynamics, so the pre/post flow differences in §5.3 are not the Eq. (2) policy-constraint score; the planning-horizon conclusion is not established.","rationale":"The core definition of the strategic link score is clear and the GridWorld and Shortcuts experiments support the first two applications. The traffic experiment, however, is the only demonstration of the claimed generality to non-RL agents and the sole basis for the 'planning horizon' result. Its identification of a physical road closure with the policy constraint π(J10)=0 is not valid: the closure changes the environment dynamics, not merely the planner's policy, and the emergent behavior of drivers is not a replanning planner. The reader's weakest assumption identifies exactly this issue, and I agree. Because this concern is load-bearing for the third application but does not undermine the formal definition or the controlled experiments, the appropriate verdict remains CONDITIONAL: the central claim requires either a corrected experiment or a clear argument that the road closure is equivalent to the policy constraint despite the dynamics changes. The absence of code or data makes this harder to resolve by inspection, reinforcing the conditional status.","tokens_in":15637,"tokens_out":5801,"duration_ms":53391,"concrete_test":"Re-run the §5.3 experiment with the physical road closure replaced by a route-guidance intervention that forces vehicles arriving at J10 to take the highway while leaving all road segments open and all segment speeds unchanged. If the resulting flow-rate differences at J1–J9 are not identical to Figure 10c (within simulation noise), then the road closure does not isolate the policy-constraint counterfactual of Eq. (2), and the reported strategic link scores are contaminated by environmental dynamics changes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation (2) defines the strategic link score for a planner P† as the drop in the likelihood of a set-up decision when a pay-off decision is made unavailable via the policy constraint π(ã|s̃)=0, keeping the environment E fixed. Section 5.3 instead intervenes on the environment: the arterial road past J10 is physically closed, which removes an action, alters transition dynamics, and changes travel times and congestion throughout the network. The emergent 'policy' is not a planner that replans under a constraint; it is the aggregate of individual drivers responding to a changed world. Consequently, the measured pre/post difference in flow rates (Figure 10c) conflates the counterfactual policy change with direct dynamics effects (e.g., forced exits, spillback, and reassignment of traffic that previously continued past J10). The strong score at J9 may simply reflect a local reaction to a blocked link, not a strategic dependency as defined in Eq. (2). The continuous-region extension in Section 4 suggests constraining π(J10) to a low-flow region, but the simulation does not implement that constraint; it closes the road segment. Thus Figure 10c is not an instance of the proposed score, and the claim that it measures drivers' effective planning horizon is unsupported by the definition.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a formal quantity called the strategic link score, defined in Eq. (2) as the drop in the probability of a set-up decision (s,a) when a pay-off decision (s̃,ã) is made unavailable through the policy constraint π(ã|s̃)=0. The score is meant to capture whether an early action is taken mainly to enable a later action. The authors illustrate the definition with a toy MDP, then present three applications: planning-level explanations of soft-optimal RL agents in GridWorld, strategy-aware safe recommendations in a procedurally generated Shortcuts environment, and measurement of the planning horizon of drivers in a UXsim traffic simulation through road closures. The paper also studies how well the scores can be recovered from demonstrations via maximum entropy IRL.","tokens_in":15861,"tokens_out":8057,"duration_ms":75194,"significance":"If the central definition holds, the strategic link score is a simple and broadly applicable tool for planning-level analysis, and the first two applications are convincing demonstrations: the toy example cleanly separates two planners with identical policies but different strategic dependencies, the GridWorld experiments show that Eq. (2) produces interpretable explanations, and the Shortcuts experiments show that strategy-aware grouping can improve both average and worst-case recommendation outcomes. The IRL experiments are also a useful practical check. However, the third application does not implement Eq. (2): the traffic intervention changes the environment dynamics rather than constraining a fixed planner's policy, and the continuous-action treatment is internally inconsistent with Section 4. The planning-horizon conclusion is therefore not established by the formal definition.","major_comments":[{"comment":"The traffic experiment does not implement Eq. (2). The strategic link score in Eq. (2) compares two policies of the same planner in the same environment E: the unconstrained policy π† and the policy planned under the constraint π(ã|s̃)=0. In Figures 9 and 10, the post-intervention routing policy is instead measured after physically closing the arterial road past J10, which changes the transition dynamics, travel times, congestion, and information available to drivers. Even though the closure removes the J10 arterial action, it does so by altering the environment, so the pre/post flow differences in Figure 10c conflate the counterfactual policy drop with direct dynamics effects such as spillback and forced diversions. Consequently, the claim that the strongest link at J9 reveals the drivers' effective planning horizon is not a consequence of the proposed score, and the negative link at J1 is not shown to be a strategic avoidance rather than a congestion response. This identification is load-bearing for the third application; the authors should either implement a genuine policy-constraint intervention or substantially reframe the traffic analysis as an environment-intervention study that is outside Eq. (2).","section":"Section 5.3"},{"comment":"The continuous-action treatment is internally inconsistent. Section 4 states that for continuous states and actions a point constraint such as π(ã|s̃)=0 'would practically have no impact' and therefore requires constraining a region S̃×Ã, yet Section 5.3 defines S†_{JX→J10}=π†(JX)-π†:{π(J10)=0}(JX) with a point constraint and reports that the J10 score is one 'by definition.' A measure-zero constraint on a continuous action distribution is not equivalent to the road closure used in the simulation. The paper should either specify the region constraint used in the traffic example (for example, constraining π(J10) to a low-flow interval, as suggested in Section 4) or justify why the point constraint is well-defined for this emergent policy; as written, the traffic scores are not an instance of the formal definition.","section":"Section 4 and Section 5.3"},{"comment":"The strategy-aware recommendation algorithm depends on a significance threshold, set in Appendix A.2 to 'halfway between 0 and 1/5,' and the safety and effectiveness comparison in Figure 7 depends on the groups produced by that threshold. The paper gives a rationale for the 1/5 bound but no sensitivity analysis, so it is unclear how robust the claimed advantage of Strategy-Aware over Pick-and-Choose and All-or-Nothing is to the threshold choice. Please add a sensitivity study or replace the heuristic threshold with a more principled criterion.","section":"Section 5.2 and Appendix A.2"}],"minor_comments":[{"comment":"There is a typo in 'maximum entorpy IRL (Ziebart et al., 2008)' in Section 5.1, Approach 2; it should be 'maximum entropy.'","section":"Section 5.1"},{"comment":"The appendix says that when constraining a decision the reward is set to r(i,j,{f'k},a)←∞, but the main text and the intended constraint require −∞; this sign error should be corrected.","section":"Appendix A.1"},{"comment":"The caption refers to 'the second layout (Figure 4a)' when describing the correlated-keys layout; it should refer to Figure 4c.","section":"Figure 4 caption"},{"comment":"The sentence 'explainability in RL, which can be address at state, policy, or planning levels' contains a grammatical error; 'address' should be 'addressed.'","section":"Section 2"},{"comment":"The paper does not include a reproducibility statement or a link to the code and data used for the simulations; adding these would make the experimental results easier to verify, especially for the UXsim traffic experiment.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The formal definition and the first two applications are sound and likely to be of interest to the RL explainability and safe policy improvement communities. The traffic section, however, is currently overclaimed: the road-closure intervention changes the environment, so the measured quantities are not strategic link scores as defined. This is fixable by reframing the experiment as an interventional study of an emergent policy rather than an instance of Eq. (2), or by implementing a true policy constraint, but the revision needs to address the identification issue before the paper can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this paper has one genuinely useful idea, and most of the experiments support it. Second thing: the traffic experiment in Section 5.3 does not measure the quantity the paper defines, and the headline conclusion about planning horizon is not established.\n\nThe strategic link score in Eq (2) is simple: the drop in the probability of a set-up action when a pay-off action is constrained to probability zero. That simplicity is a strength. The toy example with two reward functions that yield identical policies but different link scores makes the point cleanly, and the GridWorld key/door experiments confirm that the score picks out the right dependencies. The Shortcuts recommendation grouping is also well executed; grouping recommendations by high link scores and requiring groups to be adopted together is a sensible, practical way to make partial adoption safer. The IRL experiment has a nice touch: when policies are nearly uniform, inferred rewards become unidentifiable but strategic link scores stay accurate, because recognizing that all decisions are unstrategic is enough.\n\nThe soft spot is real. In Section 5.3, Eq (2) is defined for a fixed environment and a planner that replans under a policy constraint. The road closure is an intervention on the environment: it removes a link, changes transition dynamics, congestion spillback, and route availability. The paper itself suggests the continuous-region extension where flow at J10 is constrained to a low interval, but the simulator does not implement that constraint; it closes the road. So the pre/post flow difference in Figure 10c conflates a counterfactual policy change with direct dynamics effects. The reported strong score at J9 may just be a local reaction to a blocked link, not a strategic dependency as defined. Consequently, the claim that drivers have a short planning horizon is not supported by the proposed measure. The negative link at J1 is an interesting phenomenon, but it is explained by congestion dynamics, not by Eq (2). There is also no code or data release, which matters for a paper whose contribution is a measurement tool.\n\nWho gets value: researchers in RL explainability and decision support will find the core score and the recommendation algorithm worth engaging with. It deserves a serious referee; I would send it. I would ask for the traffic section to be reframed honestly or dropped, and for code and data to be released.","headline":"A clean, simple measure of strategic dependence between decisions; the GridWorld and Shortcuts results are convincing, but the traffic 'road closure' experiment does not actually instantiate the score as defined.","tokens_in":16387,"tokens_out":2982,"would_cite":false,"duration_ms":23958,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A strategic link score quantifies when decisions exist only to enable later decisions.","keywords":["strategic link score","long-term planning","reinforcement learning","planning-level explanations","safe policy improvement","inverse reinforcement learning","planning horizon","interventional measurement"],"falsifier":"Set the simulated drivers in UXsim to follow a known optimal routing policy and close the arterial road past J10; the optimal policy yields a peak strategic link at J1. If the measured pre/post intervention flow differences do not match the link scores computed by directly constraining the optimal policy, then the road closure is not implementing the abstract pay-off constraint and the traffic-based horizon estimate does not measure Equation (2).","tokens_in":15391,"feed_emoji":"🔗","tokens_out":6392,"duration_ms":52517,"temperature":0.7,"pith_summary":"The paper proposes that long-horizon strategy in decision-making can be measured decision by decision. Its strategic link score defines the strength of the dependency between a set-up action and a later pay-off action as the drop in the probability of the set-up when the pay-off is forbidden. The paper argues this score is not tied to a particular planner or to rewards: it can be computed for any agent whose planning can be intervened on, and it supports three uses—explaining black-box policies, making action recommendations safer by grouping linked changes, and estimating how far ahead an emergent system plans, as demonstrated in a traffic simulator. A reader should care because, if the score works as claimed, it gives a general, intervention-based language for the structure of strategies, not just for the policies they produce.","feed_headline":"One score exposes the strategy hidden inside long-term plans","feed_subtitle":"It measures the drop in a decision's likelihood when its planned follow-up disappears, and reads planning horizons from interventions.","key_machinery":"The central object is the strategic link score of Equation (2): for a planner $P^{\\dagger}$, the score between a set-up decision $(s,a)$ and a pay-off decision $(\\tilde s,\\tilde a)$ is $\\pi^{\\dagger}(a|s) - \\pi^{\\dagger:\\{\\pi(\\tilde a|\\tilde s)=0\\}}(a|s)$, the drop in the set-up action's probability when the pay-off action is constrained to have probability zero. This object carries the argument by turning the counterfactual question \"what would the planner do if the follow-up were unavailable\" into a single number. Higher-order applications build on it: scoring all pairs along a high-probability trajectory yields planning-level explanations; grouping recommendations whose pairwise scores are large yields strategy-aware recommendation sets; and measuring pre-intervention versus post-intervention flow rates in a continuous-action traffic setting yields the same score, with continuous decisions handled by constraining a region of pay-off decisions instead of a single point.","core_discovery":"The central claim is that the strategic link score $S^{\\dagger}_{(s,a)\\to(\\tilde s,\\tilde a)} = \\pi^{\\dagger}(a|s) - \\pi^{\\dagger:\\{\\pi(\\tilde a|\\tilde s)=0\\}}(a|s)$ measures how much a decision is made as a setup for a later decision. The paper calls $(s,a)$ the set-up decision and $(\\tilde s,\\tilde a)$ the pay-off decision, and interprets the difference in the planner's action probabilities as the drop in the set-up's likelihood once the pay-off is unavailable. A key demonstration is that two planners can produce exactly the same optimal policy while carrying completely different strategic links, so link scores expose the planning process rather than just the policy. The paper then shows the score works across three applications: explaining agents by scoring pairs along a likely trajectory, improving worst-case recommendation safety by grouping strategically linked changes, and measuring the planning horizon of emergent driver behavior through road-closure interventions in a traffic simulator.","pith_inferences":["Beyond the paper, pairwise link scores from demonstrations will inherit the assumptions of the planner used during inverse RL; the same demonstrations could support different strategic-link conclusions under a different planner model, so explanations should be reported together with the assumed counterfactual planner.","The paper's own limitation suggests a natural extension: strategies that are redundant across multiple pay-offs, such as 'A is taken to enable either B or C', are missed by pairwise constraints, and defining joint constraints over sets of pay-off decisions would capture such higher-order structure.","Negative link scores, such as the one measured at J1 in the traffic experiment, point to feedback-mediated strategies—actions taken to avoid congestion caused by the very route the pay-off decision depends on, suggesting the score could detect indirect environment-mediated dependencies in emergent multi-agent behavior.","Because the score is an interventional quantity, applying it to real systems requires interventions that block only the pay-off decision rather than changing the environment's dynamics; when that separation fails, the measured quantity is not the strategic link score as defined."],"forward_implications":["Explanations become planning-level: a user can see not only what a policy does, but which actions exist purely to unlock later actions, including dependencies that are invisible from the policy alone.","Recommendation systems become safer under partial adoption: strategically linked changes are presented as a single all-or-nothing group, avoiding the worst case where one change is adopted alone and performance drops, while independent recommendations can still be chosen individually.","The planning horizon of an unobservable agent can be estimated from interventions alone: the time distance between strongly linked decisions, measured through road closures or analogous prohibitions, characterizes how far ahead the agent plans.","Demonstrations are sufficient to recover strategic structure: inverse reinforcement learning infers a reward function that supports counterfactual policies, and the resulting link scores remain accurate even when the reward function itself becomes unidentifiable, because uniformly random policies imply no strategic links.","The score applies beyond reward-based RL, so emergent, multi-agent, or otherwise opaque planning systems can be analyzed as long as one can intervene to remove a follow-up decision."],"supporting_citations":[{"why":"Supplies soft value iteration, the planner whose policies are explained and whose constrained versions define the strategic link scores in the GridWorld and Shortcuts experiments.","marker":"Haarnoja et al., 2017"},{"why":"Supplies maximum-entropy inverse reinforcement learning, the method used to infer a reward function from demonstrations before computing counterfactual policies and link scores.","marker":"Ziebart et al., 2008"},{"why":"Supplies the EPIC distance used to evaluate how well inferred reward functions match the true rewards in the demonstration experiments.","marker":"Gleave et al., 2021"},{"why":"Supplies the UXsim traffic simulator in which road closures are used as interventions to measure the strategic link scores of emergent driver behavior.","marker":"Seo, 2025"},{"why":"Provides the safe-policy-improvement baseline that strategy-aware recommendations are contrasted against.","marker":"Laroche et al., 2019"}],"fun_headline_variants":["Strategic link scores: the hidden glue of long-term plans","Same policy, different strategy? A score tells them apart","How to tell if a decision is really a setup for later","Road closures expose how far ahead drivers really plan","A new metric reveals the dependencies between planned actions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The traffic experiment treats closing the arterial road past J10 as equivalent to imposing the policy constraint that the pay-off decision at J10 is unavailable to a single emergent planner, while a real road closure also changes the road network's transition dynamics and each driver's alternatives, so the measured flow-rate difference is not automatically the strategic link score defined by Equation (2).","fun_headline_variants_meta":{"raw":{"variants":["Strategic link scores: the hidden glue of long-term plans","Same policy, different strategy? A score tells them apart","How to tell if a decision is really a setup for later","Road closures expose how far ahead drivers really plan","A new metric reveals the dependencies between planned actions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000494,"raw_usage":{"total_tokens":2435,"prompt_tokens":963,"completion_tokens":1472,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":1394}},"tokens_in":579,"tokens_out":1472,"duration_ms":11587,"temperature":1.0,"reasoning_tokens":1394,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:53:37.241278+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Set the simulated drivers in UXsim to follow a known optimal routing policy and close the arterial road past J10; the optimal policy yields a peak strategic link at J1. If the measured pre/post intervention flow differences do not match the link scores computed by directly constraining the optimal policy, then the road closure is not implementing the abstract pay-off constraint and the traffic-based horizon estimate does not measure Equation (2).","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies soft value iteration, the planner whose policies are explained and whose constrained versions define the strategic link scores in the GridWorld and Shortcuts experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies maximum-entropy inverse reinforcement learning, the method used to infer a reward function from demonstrations before computing counterfactual policies and link scores."},{"cited_title":"Dennis, S","cited_arxiv_id":null,"evidence_quote":"Supplies the EPIC distance used to evaluate how well inferred reward functions match the true rewards in the demonstration experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the UXsim traffic simulator in which road closures are used as interventions to measure the strategic link scores of emergent driver behavior."},{"cited_title":"Trichelair, and R","cited_arxiv_id":null,"evidence_quote":"Provides the safe-policy-improvement baseline that strategy-aware recommendations are contrasted against."}],"review_version":1}