{"id":"6e1b173c-85bf-41fc-8cd2-0b7a679ed1c2","arxiv_id":"2502.05526","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A policy-gradient neural network can learn obstacle-avoiding target navigation in a continuous robosoccer domain, with partial transfer from static training to dynamic multi-agent evaluation.","lead":"This paper trains a small neural network with policy gradients to steer robots around obstacles toward targets in a continuous 2D soccer-like environment. It demonstrates the approach on simple static and moving obstacle cases, though the learned policy does not beat a straight-line baseline in the simplest setting.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transfer claim in §IV is not yet tested: no matched static control or direct training baseline separates learned transfer from task difficulty.","rationale":"The reader's conditional verdict is appropriate, and the weakest assumption identified is the same one I find most load-bearing: the paper asserts static-to-dynamic transfer without a control that isolates transfer from task difficulty. The paper does provide a new environment, a clear reward formulation, and qualitative evidence that the policy learns something, so there is no basis for rejection. However, the central claim is weaker than the abstract implies because the evaluation conflates changes in the number of agents, number of targets, obstacle motion, and observation truncation. The proposed matched-control experiment would directly test whether the static-trained policy generalizes to dynamic multi-agent settings or whether the observed behavior is simply a reactive response to the reward signal in each new task configuration. Since the reader's own concern matches this, and the requested revision would resolve it without invalidating the paper, the verdict should remain conditional and unchanged.","tokens_in":7138,"tokens_out":7285,"duration_ms":75616,"concrete_test":"Run the published environment with the same three seeds and architecture in four matched conditions: (1) 1 agent/3 targets/10 static obstacles; (2) 3 agents/3 targets/10 static obstacles; (3) 3 agents/3 targets/10 obstacles moving as in §IV toward agents; (4) 3 agents/10 targets/10 moving obstacles. Report mean±std for collisions, targets reached, and Eq. (4) weighted score for the existing static-trained checkpoint. Then train the same architecture from scratch directly on condition (3). If zero-shot static-trained performance in conditions (2) and (3) is statistically indistinguishable, the transfer claim fails; if direct training on (3) substantially outperforms the zero-shot policy, the paper's transfer evidence is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the static-trained policy transfers to dynamic multi-agent settings (Abstract; §IV). For this to be supported, the evaluation must separate 'the policy has learned a generalizable obstacle-avoidance skill' from 'the dynamic multi-agent tasks are simply easier or harder for the same reactive policy.' It does not. All three test domains differ simultaneously in the number of agents, number of targets, and obstacle motion, so the Multi-Agent Dynamic Domain result cannot be attributed to transfer of static obstacle-avoidance. The Target-to-Target baseline always reaches all targets by construction, making target-reach a trivial comparison; collisions and weighted score are reported only graphically, with no numerical means, error bars, or significance tests across seeds. The authors also attribute the Large-domain drop to 'lack of full-observability' (§IV), a fixed N=10 observation-window limitation, which is a representation-capacity issue rather than evidence about transfer. Because no domain randomization, replay of dynamic episodes, or explicit mechanism for moving obstacles was included during training, the sentence 'training learned on simple path planning in static environments is transferable to dynamic domains' (§IV) is an assertion, not a demonstrated result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a policy-gradient-based motion planning policy for decentralized multi-agent navigation in continuous Robosoccer-like domains. The model is a simple neural network mapping agent, target, and up to the N closest obstacle observations to a velocity action, and it is trained on a single-agent static environment with 3 targets and 10 obstacles using a hand-designed reward. The authors evaluate the trained policy on three domains: a static single-agent domain, a dynamic multi-agent domain with 3 agents, 3 targets, and 10 moving obstacles, and a larger dynamic domain with 3 agents and 10 targets. They compare against a straight-line target-to-target baseline using collisions, targets reached, and a weighted score, reporting results graphically. The central claim is that static-environment training transfers to dynamic multi-agent settings. The paper also discusses limitations and proposes GNN-based extensions for scalability.","tokens_in":7392,"tokens_out":5658,"duration_ms":53759,"significance":"If the transfer claim were established, the contribution would be a simple, decentralized, end-to-end trainable policy for continuous-space navigation with moving obstacles, together with a reusable environment for Robosoccer-style multi-agent motion planning. Strengths of the paper are the clean problem formulation as an MDP, the use of a pure policy-gradient method rather than a learned heuristic for a classical planner, and an honest discussion of the observability limitation. However, the experimental evaluation as presented does not support the transfer claim: there are no matched static controls, no numerical summaries or error bars, no significance tests, and no comparison with the classical planners named in the introduction. The paper is a reasonable starting point, but its central claim is currently under-supported.","major_comments":[{"comment":"The sentence 'training learned on simple path planning in static environments is transferable to dynamic domains' (Section IV) is an assertion rather than a demonstrated result: the three test domains differ simultaneously in number of agents, number of targets, and obstacle motion, so the dynamic-domain results cannot be attributed to transfer of static obstacle-avoidance behavior. A matched static control with the same numbers of agents and targets but static obstacles, and ideally a policy trained directly on the dynamic task, are needed to separate transfer from task difficulty.","section":"IV, Multi-Agent Dynamic Domain"},{"comment":"The evaluation reports no numerical means, standard deviations, or significance tests across the three seeds and 100 test problems per domain; the figures show only graphical curves and the x-axis is not labeled. Because the straight-line baseline reaches every target by construction, the target-reach metric is trivially favorable to the baseline, and the collision and weighted-score curves alone do not establish the model's advantage. The introduction motivates the method by speed and smoothness relative to RRT* and A*, yet no comparison to those planners or to the learning-based methods in Table I is provided.","section":"IV, Figs. 2-4 and metrics"},{"comment":"The evaluation score in Eq. (4) uses the same constant alpha=10 as the reward in Eq. (1), so the weighted score is partly aligned with the training objective by construction and is not an independent evaluation measure. The authors should report the raw metrics separately and either vary alpha in evaluation or use a separately chosen score weight; otherwise the weighted-score comparisons are hard to interpret.","section":"Eq. (4) vs. Eq. (1)"},{"comment":"The attribution of the large-domain performance drop to 'lack of full-observability' is a plausible hypothesis but is not tested: no experiment varies the observation window N while holding the number of obstacles fixed, and the fixed N=10 window is a representation-capacity choice rather than evidence about transfer. An ablation over N, or over obstacle count with N matched, is needed before this explanation is used.","section":"IV, Multi-Agent Dynamic Domain (Large)"}],"minor_comments":[{"comment":"The phrase 'avoiding moving targets' appears to be a typo for 'avoiding moving obstacles'; as written, it conflicts with the stated objective of reaching targets.","section":"Abstract"},{"comment":"The problem is called a fully observable MDP in Section III-B, but the observation consists only of the N closest obstacles (Section III-C); this inconsistency, and the conclusion's claim that the model 'is able to perform under partial observability,' should be clarified because partial observability is neither formalized nor explicitly tested.","section":"III-B / III-C"},{"comment":"The piecewise condition in Eq. (2) should specify the second branch as the case D-r < 0; 'otherwise' is ambiguous, and the sign convention for beta2 should be stated explicitly.","section":"III-D, Eq. (2)"},{"comment":"Equation (3) uses an advantage term A^pi_t that is not formally defined; the text says it is computed by subtracting the mean batch reward, but the notation and the policy-gradient estimator should be stated more precisely.","section":"III-D, Eq. (3)"},{"comment":"The x-axes of Figures 2-4 are not labeled; specify whether they show training steps, evaluation episodes, or environment time steps, and add axis labels and legends for the baseline and model curves.","section":"IV"},{"comment":"There are repeated typographical errors, including 'heterogenous' for 'heterogeneous', 'optimally' for 'optimality' in the abstract, and 'unmatured' in the introduction; a careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"This is a workshop-level empirical study whose central transfer claim is currently under-supported. The main risk is not circularity or methodological misconduct but an evaluation that conflates task difficulty with transfer; I believe this is fixable with additional controlled experiments and careful rewriting. The paper does not compare with the baselines it cites, so the authors should either add such comparisons or narrow the claims to match the evidence. I see no citation or novelty red flags."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the two-minute version: this is a small, honest RL paper with a standard policy-gradient method and a new custom environment. The environment is the main asset; the transfer claim is not actually tested.\n\nThe environment is clearly described: single agent, three targets, ten obstacles, continuous space, later expanded to three agents and moving obstacles. The training is vanilla REINFORCE with a hand-shaped reward. The authors are upfront that this is a simple testbed and that scalability is future work. That honesty counts.\n\nWhat the paper does well: the problem setup is lucid, the figures show training progress, and there is a straight-line baseline. It's a readable demonstration that a policy-gradient agent can learn to reach targets and avoid static obstacles in this particular setup.\n\nNow the soft spots. The central claim, that static training transfers to dynamic multi-agent settings, is asserted rather than supported. The three test domains change multiple variables at once—agent count, target count, obstacle motion—so the large-domain result can't be attributed to transfer of a learned skill. The baseline always reaches all targets by construction, so 'targets reached' is a meaningless comparison for it. Collisions and weighted score are plotted but not summarized numerically, and there are no error bars or significance tests across the three seeds. The weighted score also reuses the same alpha (10) from the reward function, so the evaluation metric is partly aligned with the training signal. That's mild but worth noting.\n\nThe paper also contains a formal inconsistency: it defines the problem as a fully observable MDP but then claims in the conclusion that the model 'is able to perform under partial observability.' N=10 closest obstacles is a fixed observation window, and the performance drop in the large domain is attributed to limited observability. That's a capacity limitation, not evidence about transfer, and the partial-observability claim is never directly tested.\n\nThe stress-test note from your reader is right on target: the sentence in Section IV claiming that static training 'is transferable to dynamic domains' is an assertion, not a demonstrated result.\n\nWho should read this? Someone building a simple continuous-space testbed for learning-based navigation, or a group discussing evaluation pitfalls in RL. As a contribution, it's too preliminary for a full venue in its current form. I'd desk reject for a major conference but suggest a workshop submission or a major revision with matched comparisons, error bars, and a proper ablation separating task difficulty from learned transfer. The environment has potential; the evidence doesn't yet back the claims.","headline":"A small, honest RL demo on a new robosoccer-like testbed; the transfer claim is asserted more than tested.","tokens_in":7870,"tokens_out":4922,"would_cite":false,"duration_ms":46222,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a policy-gradient motion planner trained on one static agent generalizes to small dynamic Robosoccer teams, trading some collisions for target coverage.","keywords":["motion planning","multi-agent systems","policy optimization","reinforcement learning","dynamic obstacle avoidance","Robosoccer","decentralized navigation","scalable planning"],"falsifier":"Run the trained policy in a dynamic three-agent domain while increasing the number of obstacles beyond ten, so the nearest-ten observation necessarily hides some threatening obstacle; if the learned policy's collision rate then reaches or exceeds the straight-line baseline's rate, or its target completion falls below baseline in problems the baseline can finish, the static-to-dynamic transfer claim fails outside the tested range.","tokens_in":6965,"feed_emoji":"⚽","tokens_out":6441,"duration_ms":58129,"temperature":0.7,"pith_summary":"The paper sets out to show that a simple end-to-end trainable neural policy performs continuous-space motion planning for a multi-agent Robosoccer-like game, steering agents to ordered targets while avoiding static and moving obstacles. Its central claim is that a policy trained with policy-gradient optimization on a single agent in a static environment—three targets and ten fixed obstacles—transfers to decentralized use by several agents in dynamic domains with mobile obstacles. If that transfer holds, an agile collision-avoiding controller can be learned without multi-agent training or replanning, then copied to each agent at runtime. The authors also state plainly that scalability is not yet solved and is left to future graph-based work.","feed_headline":"One agent's static training steers three-agent soccer dodges","feed_subtitle":"A policy-gradient planner trained alone with fixed obstacles reaches targets in small dynamic multi-agent soccer domains.","key_machinery":"The load-bearing object is a reward-shaped policy-gradient controller. The reward is $R = -\\alpha D(u,v) + \\sum_i g(D(u,o_i), r_{o_i})$, where $D$ is Euclidean distance, $\\alpha = 10$, and the obstacle term is asymmetric: $g(D,r) = \\beta_1(D-r)$ outside the keep-out radius and $\\beta_2(D-r)$ inside it, with $\\beta_1 = 1$ and $\\beta_2 = 100$. That asymmetry makes collisions dominate the learning signal only when an obstacle is inside the keep-out radius, while target distance keeps pulling the agent forward. Each agent samples a velocity offset $\\langle \\delta x, \\delta y \\rangle$ from a small neural policy, and the policy is trained with a policy-gradient objective in which the advantage is the reward for a sampled action minus the batch mean reward. The training MDP is fully observable, but deployment is decentralized: each agent sees only its own target sequence and the ten nearest obstacles.","core_discovery":"The paper's core discovery is that one policy-gradient model, receiving the agent's location, its current target, and the positions, radii, and distances of the ten nearest obstacles, can learn to travel through a sequence of targets in continuous space without collisions, and the same weights work when copied to each of three agents facing moving obstacles. This is demonstrated in a static single-agent domain, a dynamic multi-agent domain with three agents, three targets, and ten mobile obstacles, and a larger dynamic domain with three agents and ten targets. In the two smaller domains the learned policy reaches targets at a rate near the no-obstacle-knowledge baseline and collisions decrease over training; in the larger domain the paper reports a performance drop that it attributes to observing only the nearest ten obstacles.","pith_inferences":["If the transfer claim is right, it suggests a curriculum: dynamic multi-agent skills can be bootstrapped from static single-agent practice, and adding modest domain randomization during that practice should close most of the remaining gap to the baseline.","The asymmetric obstacle penalty is a plausible mechanism for the behavior; an ablation that sets $\\beta_1 = \\beta_2$ would reveal whether the sharp inside-radius penalty is what teaches avoidance.","The reported large-domain drop predicts that any observation representation invariant to obstacle count, such as a set- or graph-based encoder, should recover the performance loss in the ten-target domain.","An adversarial variant in which moving obstacles actively pursue the agent, rather than moving in a straight line at constant speed, would test whether the transferred policy's avoidance behavior persists under harder dynamics."],"forward_implications":["A single trained policy can be duplicated across multiple heterogeneous agents and executed decentralized, with no per-agent retraining.","Because the model outputs continuous displacement deltas each time step, it avoids the per-step cost of rerunning a classical planner such as RRT* or A*.","Because training happens in a simple static single-agent environment, acquiring the dynamic collision-avoidance behavior is cheaper than training directly in the multi-agent dynamic setting.","The reported drop in the larger dynamic domain implies that the nearest-ten-obstacle observation, not the policy itself, is the current scaling bottleneck.","The method can be paired with a higher-level scheduler that supplies target sequences, so task allocation and motion planning remain separable."],"supporting_citations":[{"why":"Defines the RoboCup synthetic-agent challenge that motivates the Robosoccer environment.","marker":"[5]"},{"why":"Neural RRT*, an example of a learning model that still relies on a classical planner, the contrast motivating a purely learned generator.","marker":"[12]"},{"why":"Neural A*, another learning-plus-classical approach whose residual cost motivates the end-to-end model.","marker":"[13]"},{"why":"GRSTAPS task allocation and scheduling, which supplies the target sequences the motion policy follows.","marker":"[22]"},{"why":"Source of the distance-to-target and distance-to-obstacle reward formulation used in Equation 1.","marker":"[23]"},{"why":"Policy-gradient method used to optimize the model parameters.","marker":"[24]"},{"why":"Adam optimizer used in training with the stated learning rate and weight decay.","marker":"[25]"},{"why":"Identifies multi-agent scalability as an open challenge, framing the paper's stated limitation and future work.","marker":"[26]"}],"fun_headline_variants":["One policy, trained alone, dodges for three soccer agents","Static training, dynamic dodges: a single policy for team soccer","Solitary training steers a multi-agent robot soccer team","Policy gradient: one agent's learning moves three on the field","Single-agent training scales to multi-agent obstacle dodging"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The transfer claim rests on the assumption that a policy learned by one agent moving among three static targets and ten static obstacles keeps working when the same weights are used by three agents moving among mobile obstacles and, in the large case, ten targets.","fun_headline_variants_meta":{"raw":{"variants":["One policy, trained alone, dodges for three soccer agents","Static training, dynamic dodges: a single policy for team soccer","Solitary training steers a multi-agent robot soccer team","Policy gradient: one agent's learning moves three on the field","Single-agent training scales to multi-agent obstacle dodging"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000242,"raw_usage":{"total_tokens":1514,"prompt_tokens":926,"completion_tokens":588,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":504}},"tokens_in":542,"tokens_out":588,"duration_ms":6384,"temperature":1.0,"reasoning_tokens":504,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:57:25.584271+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained policy in a dynamic three-agent domain while increasing the number of obstacles beyond ten, so the nearest-ten observation necessarily hides some threatening obstacle; if the learned policy's collision rate then reaches or exceeds the straight-line baseline's rate, or its target completion falls below baseline in problems the baseline can finish, the static-to-dynamic transfer claim fails outside the tested range.","supporting_citations":[{"cited_title":"The RoboCup synthetic agent challenge 97,","cited_arxiv_id":null,"evidence_quote":"Defines the RoboCup synthetic-agent challenge that motivates the Robosoccer environment."},{"cited_title":"Neural RRT*: Learning-Based Optimal Path Planning,","cited_arxiv_id":null,"evidence_quote":"Neural RRT*, an example of a learning model that still relies on a classical planner, the contrast motivating a purely learned generator."},{"cited_title":"GRSTAPS: Graphically Recursive Simultaneous Task Allocation, Planning, and Scheduling,","cited_arxiv_id":null,"evidence_quote":"GRSTAPS task allocation and scheduling, which supplies the target sequences the motion policy follows."},{"cited_title":"Policy gradient methods,","cited_arxiv_id":null,"evidence_quote":"Policy-gradient method used to optimize the model parameters."},{"cited_title":"Human-Robot Teaming: Grand Challenges,","cited_arxiv_id":null,"evidence_quote":"Identifies multi-agent scalability as an open challenge, framing the paper's stated limitation and future work."}],"review_version":1}