{"id":"2a7d9fb8-a028-4db8-ae97-c3d9c8de7e3a","arxiv_id":"2412.20049","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A decentralized multi-robot exploration system using A*-based frontier density features and a learned proximity-limited map-sharing action reaches roughly 80 to 100 percent coverage in simulation and lab tests.","lead":"A research team trained four TurtleBot3 robots with reinforcement learning to explore a room while choosing when to share map data with nearby teammates. In simulation and in a 6-meter lab arena, the robots reached high coverage, but the work does not compare against existing exploration methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No baselines are reported; in a 12x12 arena with 4 robots and 13-cell FOVs, 80% coverage at 150 steps may be matched by random exploration, so the claimed effectiveness is unanchored.","rationale":"The reader's weakest_assumption focuses on generalization to larger or differently distributed arenas, which is a real concern. However, I find a more immediate and more load-bearing issue: the empirical effectiveness claim is not anchored by any baseline, even in the exact evaluation distribution. If a random walk or a simple greedy frontier policy performs equally well on the same 200 arenas, then the entire argument for the proposed architecture's benefits collapses, regardless of generalization. The reader did list 'absence of baselines' in the rationale, so we partially agree, but the reader did not elevate it to the weakest assumption. My recommended verdict remains CONDITIONAL (UNCHANGED) because the missing baselines are straightforward to add and the paper does describe a working system; a comparison against simple controls is the required condition for accepting the effectiveness claim.","tokens_in":8847,"tokens_out":6762,"duration_ms":75280,"concrete_test":"Re-run the 200-arena evaluation from Section IV-A with two control policies: (1) a random policy using the same action mask and communication action; (2) a greedy frontier policy that moves toward the A*-nearest frontier and communicates when a teammate is within communication range. Compare the mean and distribution of max-map coverage at steps 50, 150, and 300, using identical arena generation and seeds. If either control matches or exceeds the RL curves, the claim that the proposed learning and communication improve exploration is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the proposed RL framework is robust and effective rests on absolute coverage numbers in Fig. 3a (Section IV-A) and Fig. 6 (Section V). No comparison is reported against any baseline: no random policy, no greedy frontier-following, no ablation without communication. This is load-bearing because the arena is only 12x12, the four agents each have a 13-cell FOV, and the map has only 144 cells total. Over 150 steps the agents collectively make 600 moves and accumulate up to roughly 7800 cell-sightings, so a random walk could plausibly cover most of the free space in the same number of steps. The reported 80% at step 150 is therefore not self-evidently strong, and without a baseline the observed performance cannot be attributed to the A*-based frontier density features or the learned communication policy. The real-world validation similarly consists of a single run with no control comparison. Thus the effectiveness part of the central claim is unsupported even within the training distribution, independent of the generalization concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a decentralized multi-agent exploration framework based on reinforcement learning. Each agent observes a local field-of-view occupancy grid, an A*-derived frontier-density feature table, and a vector of nearby communicating agents. Agents choose among eight movements, staying, and an explicit proximity-limited communication action that broadcasts their reconstructed map to all agents in the same network. Policies are trained with HAPPO under centralized training and decentralized execution, with two reward formulations and either MLP or CNN policy architectures. The method is evaluated in 200 randomized 12x12 grid simulations and in a single real-world experiment with four TurtleBot3 robots in a 6m x 6m Vicon-instrumented arena. The paper reports at least 80% exploration after 150 steps in simulation, up to 99% for the best CNN/Case-2 policy, and near-100% merged-map coverage after 60 steps in the real-world run.","tokens_in":9076,"tokens_out":5044,"duration_ms":58099,"significance":"If the results were properly anchored, the paper would be a useful systems contribution: the A*-based frontier-density feature extraction (Algorithm 1) is clearly specified, the network architectures and hyperparameters are reported in detail, and the ROS2/Vicon real-world deployment on four TurtleBot3 robots is a nontrivial engineering effort. The use of HAPPO with a shared critic and the explicit communication action is a sensible extension of the authors' prior work. However, the central claim of robust and effective exploration is currently supported only by absolute coverage numbers in a small arena, with no baseline comparisons, no confidence intervals, no ablation of the communication mechanism, and only a single real-world run. The paper is therefore not yet ready for publication in its present form, but the underlying architecture and experimental setup are sound enough that the required additional evidence could be produced within the scope of the manuscript.","major_comments":[{"comment":"The central effectiveness claim rests on absolute exploration ratios: 'all types of agents achieve at least 80% exploration after 150 time steps.' No comparison is reported against any baseline, such as a random policy, a greedy frontier-following heuristic, or a no-communication variant of the same framework. Because the arena has only 144 cells and each of the four agents has a 13-cell field of view, the agents accumulate a large number of cell sightings over 150 steps, so the reported absolute coverage is not self-evidently strong. Additionally, Fig. 3a and Fig. 4 show no confidence intervals or significance tests across the 200 simulations. These omissions are load-bearing for the claim that the learned features and communication mechanism, rather than the sheer number of observations, drive the reported performance.","section":"Section IV-A, Fig. 3a"},{"comment":"The real-world validation consists of a single run with four robots in one fixed arena. The near-100% merged-map coverage achieved in that run is a promising feasibility demonstration, but a single successful run cannot support the paper's robustness claims. At minimum, several repeated runs with randomized initial conditions, and ideally a comparison with a baseline policy running under the same ROS2/Vicon infrastructure, would be needed to establish repeatability and to attribute the result to the proposed method.","section":"Section V, Fig. 6"},{"comment":"The evaluation is entirely in-distribution: the training and test environments use the same 12x12 grid size, the same obstacle ratio rho_obstacle = 0.1, the same four-agent team, and the same sensing and communication ranges. No experiment varies the arena size, obstacle density, team size, or communication range. The abstract's claim of 'scalable and resilient robotic exploration systems' is therefore not supported by the reported evidence, and the reader cannot assess how the method behaves in larger or differently structured environments.","section":"Section IV-A and Section V"},{"comment":"The conclusion states that 'inter-agent proximity-based communication significantly reduces exploration time,' but no experiment disables communication or compares against a variant that cannot transmit maps. Figs. 3b and 4 compare only the two reward functions and two network architectures, not the presence versus absence of the communication action. Since the communication mechanism is a principal claimed contribution, an ablation that removes it, or an equivalent baseline that merges maps without requiring a learned communication action, is necessary to support this conclusion.","section":"Section VI and Figs. 3b/4"},{"comment":"The abstract and introduction say the approach addresses 'static and dynamic obstacles,' but the simulation and real-world experiments contain only static obstacles. The only dynamic obstacles in the model are the agents themselves (Section III-A). No experiment with moving non-agent obstacles is reported. The claims about dynamic obstacles should either be backed by dedicated experiments or removed from the scope of the paper.","section":"Abstract and Section I"}],"minor_comments":[{"comment":"The text contains a likely typo: it states that 'reward function Case 1 leads to a higher frequency of communication actions' and then says 'Conversely, agents trained with reward function Case 1 show reduced communication in favor of exploration.' The second occurrence should presumably be Case 2.","section":"Section IV-A"},{"comment":"In the description of the critic update, 'the predicted state value nad the target return' should read 'and the target return.'","section":"Section III-D"},{"comment":"The definition of p_i is notationally ambiguous: p_i is also used for the agent's position, and the fraction involving q_ij over m x n plus 0.8 is not parenthesized clearly. Please rewrite this expression to avoid confusion between the communication-incentive factor and the position vector.","section":"Section III-E, Eq. (3)"},{"comment":"Both figures would benefit from explicit axis labels, units, and a legend. For the histograms in Fig. 4, the number of bins and the handling of outliers should be specified.","section":"Figs. 3b and 4"},{"comment":"The real-world experiment does not state which of the four trained policies (MLP or CNN, Case 1 or Case 2) was deployed on the TurtleBot3 robots. This should be reported, since the simulation results show meaningful differences among the policy types.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The paper extends the authors' own accepted IROS 2024 work [4], and the incremental contribution is mainly the A*-based frontier-density feature extractor and the CNN policy variant. The editor may wish to ask the authors to more clearly delineate the novelty relative to [4]. The lack of any baseline comparison is the main concern; if the authors can add random and no-communication baselines, confidence intervals, repeated real-world trials, and at least one out-of-distribution arena size, the revised paper would be much stronger."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about arXiv 2412.20049 before deciding what to do with it. First, the genuinely new piece is the FPR feature: for each of the eight movement directions, the policy sees the number of A*-computed frontier trajectories whose first move is that direction, plus the mean and standard deviation of those trajectory lengths, all normalized. That is a compact, fixed-size way to inject frontier density into a learned policy, and I have not seen it before. Second, the empirical section does not report a single baseline. No random policy, no greedy frontier-following, no ablation without the communication action. In a 12x12 grid with four agents and 13-cell fields of view, 80% coverage after 150 steps is not self-evidently impressive; a random walk could plausibly hit that. So the paper's central claim—that the learned communication and frontier features explain the performance—is essentially unanchored.\n\nThe real-world experiment with four TurtleBots is a plus. It shows the whole pipeline works: FOV grids from laser scans, Vicon poses, the learned policy choosing movements and map-sharing, and the merged map reaching near 100% in 60 steps. That is reproducible engineering and worth acknowledging. The architecture is a direct extension of the authors' own D-MARL, which they cite; the new contribution is incremental but real.\n\nThe soft spots beyond the missing baseline are mostly addressable. Figures 3 and 4 have no confidence intervals or significance tests; the evaluation is entirely in-distribution (same 12x12 arena, same 0.1 obstacle ratio), so the word 'scalable' in the abstract is doing work the data cannot support. The real robot run is a single sample with no comparison. There are also two internal inconsistencies: the Fig. 3b text contradicts itself about whether Case 1 or Case 2 produces more communication, and the stated MLP input dimension of 37 does not obviously match the components of the observation described in III-B (13-cell FOV, 24 FPR entries, and a network vector). The reward functions have hand-tuned constants, but those are acceptable in this line of work.\n\nThis is not a paper with a load-bearing mathematical flaw; the framework is coherent and the experiments, whatever they lack, are real. The circularity concern is minor: the communication-effectiveness metric tracks the training objective, but that is not unusual.\n\nWho gets value: researchers working on decentralized MARL exploration who want a concrete, hardware-validated recipe and a new feature representation. It deserves a serious referee, though that referee should insist on baselines and ablations, plus at least one out-of-distribution test or a proper statistical treatment. If I were the editor I would send it out, expecting major revisions. I would not cite it myself in the next twelve months.","headline":"A genuine new frontier-density feature and a hardware demo, but the effectiveness claim is unanchored by any baseline and needs major revision before it can be believed.","tokens_in":9616,"tokens_out":3552,"would_cite":false,"duration_ms":36186,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A decentralized RL framework trains robot teams to map unknown arenas to 99% coverage.","keywords":["multi-robot exploration","reinforcement learning","decentralized control","frontier-based exploration","A* pathfinding","occupancy grid","map merging","communication action"],"falsifier":"Run the trained policies on a 30x30 grid or with 30% obstacle density and check whether the 150-step coverage remains above the reported 80% threshold; or repeat the four-robot experiment with randomized obstacle layouts and starting positions and observe whether merged-map coverage consistently reaches near 100% by 60 steps. A significant drop would contradict the paper's robustness claim.","tokens_in":8678,"feed_emoji":"🤖","tokens_out":8165,"duration_ms":76407,"temperature":0.7,"pith_summary":"Decentralized teams of robots can map unknown, obstacle-filled arenas nearly as completely as centrally coordinated teams if each robot is trained to decide when to explore and when to broadcast its local map to nearby teammates. This paper demonstrates the idea in simulation and with real robots. Each agent sees only its own sensor field of view, plus a compact description of where unexplored frontier cells are and how reachable they are via A* path lengths, and can choose a communication action that merges maps with any teammates in range. In 200 randomized 12x12 grid simulations all four policy variants reach at least 80% coverage by 150 steps, and CNN-based policies reach up to 99%; a four-robot physical experiment reaches nearly 100% merged-map coverage in 60 steps.","feed_headline":"Robot teams map unknown arenas to 99% coverage","feed_subtitle":"Decentralized RL policies let each robot decide when to explore and when to share its map with nearby teammates.","key_machinery":"The load-bearing mechanism is the frontier-point-reachability (FPR) feature table. For each of the eight directions an agent could move, the algorithm runs A* from the agent's current cell to every frontier cell in its reconstructed map, counts how many trajectories begin with that direction, and records the average and standard deviation of those trajectory lengths. These features are normalized to [0,1] and concatenated with the field-of-view grid and the network-neighbor indicator to form a fixed-length observation that is independent of map size. The second load-bearing element is the communication action: selecting it broadcasts the agent's reconstructed occupancy grid to all agents in the same proximity-limited network, and the receiving agents replace their maps with the union. During training, a shared critic sees all agents' observations and rewards, but during execution each agent uses only its local policy.","core_discovery":"The paper's central claim is that a fully decentralized reinforcement-learning policy can explore unknown environments efficiently by combining three local observations: an agent-centered field-of-view occupancy grid, a fixed-length feature vector describing the density and distance of frontier cells reachable in each of eight movement directions (computed from A* path lengths), and an indicator of which teammates are within communication range. The policy learns a communication action that broadcasts the agent's reconstructed map to all agents in its proximity-limited network, and the maps are merged. The authors report that all trained policies, with either MLP or CNN encoders and either of two reward formulations, exceed 80% exploration after 150 steps across 200 randomized arenas, that CNN-based policies trained with the Case 2 reward reach up to 99% coverage, and that a four-robot experiment with TurtleBot3 robots achieves nearly 100% merged-map coverage in 60 steps in a 6 m by 6 m arena. The framework operates under centralized training with decentralized execution, meaning each agent acts only on its own observations during deployment.","pith_inferences":["The arena-size invariance of the observation suggests the policy could be fine-tuned for larger, more cluttered environments, but the paper only validates on 12x12 grids with 10% obstacles, so this remains an extrapolation.","The A*-based frontier features could be augmented with semantic labels or risk maps to steer exploration toward task-relevant regions, which the authors do not investigate.","A fully field-deployable system would need to replace the external motion-capture localization used in the experiment with onboard odometry or SLAM, which is not addressed.","The map-merging communication is simple and interpretable; other message types, such as compressed frontier maps or learned embeddings, could be more bandwidth-efficient, but the paper's explicit map broadcast is sufficient for the demonstrated scale."],"forward_implications":["Exploration missions can be run by robot teams that communicate only when they are physically close, removing the need for a central coordinator or persistent network connectivity.","Because the feature vector does not grow with the map, a single trained policy may transfer across arenas of different sizes without retraining.","The reward design controls the exploration-versus-communication trade-off: Case 1 rewards favor frequent sharing, while Case 2 rewards produce fewer but more impactful communication events.","Standard off-the-shelf robots with a laser scanner and a positioning system can execute the learned policy, as demonstrated by the four-robot experiment."],"supporting_citations":[{"why":"The prior architecture this work extends, and the source of the Case 2 reward formulation and the communication-action concept.","marker":"[4]"},{"why":"Provides the A* pathfinding algorithm used to compute the frontier trajectory features in the FPR table.","marker":"[21]"},{"why":"Defines the multi-agent policy optimization algorithm used to train the actors and the shared critic.","marker":"[22]"},{"why":"Provides the standard interface for the reinforcement learning simulation environment in which the policies are trained and evaluated.","marker":"[23]"},{"why":"Provides the parallel API for the multi-agent reinforcement learning environment used to run the 200-arena simulations.","marker":"[24]"},{"why":"Formalizes the decentralized partially observable Markov decision process model that frames the exploration problem.","marker":"[19]"}],"fun_headline_variants":["RL robots with explicit communication map unknown arenas to 99%","Decentralized multi-robot RL hits 99% coverage with map sharing","Density-based frontier RL helps robot teams map 99% of unknown areas","Robots learn when to share maps to boost collaborative exploration"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework is assumed to generalize from 12x12 arenas with 10% static obstacles and fixed sensing and communication ranges to real unknown environments, and a single successful lab run is taken as evidence of robustness.","fun_headline_variants_meta":{"raw":{"variants":["RL robots with explicit communication map unknown arenas to 99%","Decentralized multi-robot RL hits 99% coverage with map sharing","Density-based frontier RL helps robot teams map 99% of unknown areas","Robots learn when to share maps to boost collaborative exploration"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000549,"raw_usage":{"total_tokens":2606,"prompt_tokens":917,"completion_tokens":1689,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":1612}},"tokens_in":533,"tokens_out":1689,"duration_ms":13795,"temperature":1.0,"reasoning_tokens":1612,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:36:23.654086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained policies on a 30x30 grid or with 30% obstacle density and check whether the 150-step coverage remains above the reported 80% threshold; or repeat the four-robot experiment with randomized obstacle layouts and starting positions and observe whether merged-map coverage consistently reaches near 100% by 60 steps. A significant drop would contradict the paper's robustness claim.","supporting_citations":[{"cited_title":"D- MARL: A dynamic communication-based action space enhancement for multi agent reinforcement learning exploration of large scale unknown environments,","cited_arxiv_id":null,"evidence_quote":"The prior architecture this work extends, and the source of the Case 2 reward formulation and the communication-action concept."},{"cited_title":"A systematic literature review of a* pathfinding,","cited_arxiv_id":null,"evidence_quote":"Provides the A* pathfinding algorithm used to compute the frontier trajectory features in the FPR table."},{"cited_title":"Heterogeneous-agent reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Defines the multi-agent policy optimization algorithm used to train the actors and the shared critic."},{"cited_title":"Pettingzoo: Gym for multi-agent reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the parallel API for the multi-agent reinforcement learning environment used to run the 200-arena simulations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Formalizes the decentralized partially observable Markov decision process model that frames the exploration problem."}],"review_version":1}