{"id":"f6f9a44a-419f-47b3-8e67-d6e8146e69b6","arxiv_id":"2411.15027","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A particle filter improves object localization in a dynamically updated semantic scene graph used by an LLM-driven robot planner.","lead":"This paper describes a robot control architecture that builds and continuously updates a semantic map of the environment using a scene graph generator and a particle filter, with an LLM planner for task execution. It is relevant because it targets real-time adaptation to human-induced changes in human-robot interaction.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No object-motion model or resampling in the particle filter, and the only test has a static object; the 'dynamic' tracking claim is not established.","rationale":"The reader's weakest assumption identifies the single-object static test and unspecified parameters as the main weakness. My stress-test sharpens this into a concrete algorithmic objection: the filter as specified cannot track object motion because it lacks an object-motion model and resampling, and Table 1's constant 'real position' means the experiment never exercises the dynamic regime claimed in the title and abstract. A simple displacement experiment would settle the question. This does not change the conditional verdict; it strengthens the conditions under which acceptance is warranted.","tokens_in":6570,"tokens_out":4709,"duration_ms":47083,"concrete_test":"Re-implement Algorithm 1 exactly as written, including the s0_i addition and no resampling, on the reported setup. Then repeat the 30-frame sequence with the target object physically moved by 0.5 m in x after frame 15. If the estimated position does not approach the new ground truth within 5 frames, the dynamic-tracking claim is refuted. Also report the effective sample size over the sequence to quantify particle degeneracy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 and Algorithm 1 specify the prediction step s_t_i = T_{t-1,t} * s_{t-1}_i + s0_i, where s0_i is drawn from the initial-position distribution, and then the weight update w_t_i = 1/(1 + d_t_i), with no resampling step anywhere. Because the only transformation is the camera motion T_{t-1,t}, and the additive term is not a zero-mean process noise, the particle cloud is never relocated to follow a physically moved object. In the human-robot scenario the paper emphasizes, an object displaced between frames will leave every particle far from the new observation; with no resampling, no particles are spawned near the new location, so the weighted mean cannot converge to the moved object. Table 1 supports only a static object: the 'Real position' is identical across all 30 measurements, so the reported error reduction could come from simple averaging of repeated static observations rather than from dynamic tracking. The central claim that the architecture handles human-induced changes through this particle filter is thus unsupported and, as specified, mechanistically implausible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a robot control architecture in which a Large Language Model (LLM) planner receives an up-to-date semantic scene graph built by a Perception Module from RGB-D images. The Perception Module uses the PSGTR panoptic scene-graph model to extract objects, masks, and relationships, then applies a particle filter to refine the 3D position estimates of detected objects. The Planner Module is described at a high level as translating natural-language commands into skills and monitoring execution. The only quantitative evaluation is Table 1, which reports that over 30 measurements the particle filter reduces the mean absolute position error from (0.07, 0.18, 0.02) m to (0.02, 0.02, 0.01) m for a single object whose true position is constant.","tokens_in":6752,"tokens_out":4310,"duration_ms":42349,"significance":"The problem addressed is timely: enabling an LLM-based planner to act on a continuously updated semantic map is important for human-robot interaction in changing environments. The paper's strengths are its clear description of the scene-graph representation, the choice of a relatively lightweight scene-graph generator (PSGTR) with a motivated runtime trade-off, and the integration with a real humanoid robot platform. The particle-filter refinement is a simple, computationally cheap idea that could be useful if properly validated. However, the quantitative evidence is extremely limited, the filter as specified cannot track moving objects, and no experimental details are given for reproducibility. The claims in the abstract about adapting to human-induced changes and improving task efficiency are therefore not supported by the presented data.","major_comments":[{"comment":"The particle filter as specified cannot track object motion. In the prediction step s_t^i = T_{t-1,t} s_{t-1}^i + s0^i, the additive term s0^i is a fixed sample from the initial distribution, not a time-varying process noise, and Algorithm 1 contains no resampling step. If an object is displaced by human action, all particles remain near the previous location; the weight update w_t^i = 1/(1 + d_t^i) only reweights the existing cloud, so the weighted mean cannot converge to the new observation. This directly contradicts the abstract's claim that the architecture adapts to human-induced changes. To support the dynamic tracking claim, the filter would need an object-motion model (e.g., a random-walk or constant-velocity model), a resampling step, and an experiment with a moving object.","section":"Section 3.1, Algorithm 1, prediction and weight update equations"},{"comment":"The quantitative evidence is a single table reporting 30 measurements of one object whose 'Real position' is identical in every row (0.67, 0.10, 0.95). This shows improvement on a static object, not on a dynamic scene. The table does not report the number of particles N, the initial covariance Sigma0, the IoU threshold lambda_IoU, the distance filter threshold, or the relationship probability threshold, so the result is not reproducible. Since the prediction step does not include any object-motion noise, the filter effectively averages repeated observations of a static object, which can trivially reduce mean absolute error. The authors should provide a dynamic test (e.g., an object displaced during the measurement sequence) and report the full experimental protocol, including confidence intervals or statistical tests.","section":"Table 1 and experimental setup"},{"comment":"The paper's title and abstract claim an 'LLM-driven robot' with dynamic environment perception, but the Planner Module is described in a single paragraph and the authors state that 'details on the planner will be provided in a separate article'. No task-planning or execution experiment is reported, so the system-level claims about improved adaptability, task efficiency, and human-robot collaboration are unsupported by any evaluation. If the intended contribution is limited to the Perception Module, the framing of the paper should be narrowed accordingly; otherwise, at least one integrated planning experiment is needed to substantiate the central claims.","section":"Section 3, Planner Module and overall evaluation completeness"}],"minor_comments":[{"comment":"There is a typo in the error standard deviation row: '0.12. 0.02' should read '0.12, 0.02'.","section":"Table 1"},{"comment":"The abbreviation PSGTR is inconsistently written as 'PSTGR' in the sentence 'The Perception Module uses the result of PSTGR and builds the semantic map'.","section":"Section 3.1, step 3"},{"comment":"The author email in the footer contains a garbled fragment 'envel⌢pe-⌢pensimone...'; this should be corrected to a standard email address.","section":"Author footer"},{"comment":"Line 3 applies the transformation to masks but is placed inside a loop 'for each object k'; clarify whether the loop is over masks or objects, and specify how the transformed mask is associated with an object.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"This manuscript is in its current form a system description with a very thin experimental section. The missing resampling and the absence of a motion model are not merely presentation issues: they invalidate the central claim of dynamic tracking as stated. The paper would need substantial new experimental work (dynamic objects, full parameter reporting, and ideally a planner integration test) to meet the standards of a journal publication. The scope should also be clarified, since the planner, which appears in the title, is deferred to a separate article. If the authors can provide such additions or explicitly restrict the claims to static-object localization, the paper could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper is a sensible system integration with a load-bearing claim that, as written, does not hold up. What's new is the specific loop: PSGTR scene graph -> particle filter -> semantic map, feeding an LLM planner. I don't see that exact combination in SayPlan, DELTA, or SayNav, so the integration itself is a legitimate incremental contribution. The system description is clear, the hardware context is concrete, and the authors are honest that the planner details are deferred to a separate article.\n\nThe soft spot is not minor. Section 3.1 and Algorithm 1 specify a particle filter with no resampling, and the prediction step adds s0_i drawn from the initial-position distribution rather than zero-mean process noise. That means the particle cloud can only be dragged by camera motion; it has no mechanism to relocate toward an object that a human actually moved. So the abstract's central claim — that the system adapts to human-induced changes — is mechanistically implausible as specified. Table 1 is consistent with this: the 'real position' row is identical across all 30 measurements, so the reported error reduction is likely averaging of a static object, not dynamic tracking. No confidence intervals, no baseline, and none of the key parameters (N, initial covariance, IoU threshold, distance threshold) are given. This is a workshop-length paper, so I wouldn't demand a full benchmark, but the main claim needs at least one displaced-object trial and a proper motion model with resampling.\n\nWhat the paper does well: the semantic map structure with room nodes and object nodes is practical, the choice of PSGTR is justified by inference speed, and the related work correctly situates the gap. No code or data is released, which is disappointing but normal for a workshop contribution. The citation pattern looks fair.\n\nBottom line: a reader interested in HRI system architectures can get a useful sketch of one way to combine scene graphs, particle filtering, and LLM planning. A reader looking for evidence that the architecture tracks dynamic scenes won't find it. I would send it to a serious referee only if the venue expects heavy revision; as is, I would not accept it. If I were the editor, I'd ask for a resampling-aware filter, moving-object experiments, and parameter disclosure before any further consideration.","headline":"A plausible scene-graph-plus-particle-filter integration that, as written, cannot track moved objects; the central dynamic claim needs resampling and real moving-object evidence.","tokens_in":7309,"tokens_out":3973,"would_cite":false,"duration_ms":41169,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A particle filter updating a semantic scene graph keeps a robot's object positions accurate in dynamic scenes, cutting the mean absolute position error from (0.07, 0.18, 0.02) m to (0.02, 0.02, 0.01) m.","keywords":["scene graphs","particle filter","semantic mapping","large language models","robot task planning","human-robot interaction","RGB-D perception","dynamic environments"],"falsifier":"Place two visually similar objects in the same room, move one across the other's path, and record whether the IoU-based association swaps their identities; if labels swap, the scene graph will misrepresent the scene and the LLM planner will act on the wrong object. Repeat the 30-measurement protocol and compare the resulting position error with the reported $(0.02, 0.02, 0.01)$ m.","tokens_in":6355,"feed_emoji":"🤖","tokens_out":11513,"duration_ms":98575,"temperature":0.7,"pith_summary":"The paper is trying to establish that a robot can keep its map of a dynamic room truthful by pairing a lightweight scene-graph generator with a particle filter, and that a large language model (LLM) planner should read that updated graph rather than a static map. This matters because natural-language commands like \"pick the blue bottle on the table\" produce correct robot actions only when the object positions the planner sees match the current scene. The paper's evidence is a 30-measurement comparison in which the particle filter drops the mean absolute position error from $(0.07, 0.18, 0.02)$ m to $(0.02, 0.02, 0.01)$ m. The architecture also closes a loop between perception and planning: when a task fails, the controller and explainer feed the failure back so the planner can replan against the updated semantic map.","feed_headline":"Particle filter shrinks robot object-position error from 18 to 2 cm","feed_subtitle":"A scene graph updated by the filter keeps the LLM planner grounded in a moving room.","key_machinery":"The mechanism that carries the argument is the particle filter used to track each object's 3D position inside the semantic map. A particle filter is a Monte Carlo state estimator: each object's position is represented by many weighted point hypotheses, here initialized as a Gaussian around the first observed position and predicted by applying the camera-motion transformation plus noise. The new observation comes from the scene-graph generator's object mask; the filter associates the new mask with a tracked object using the Intersection over Union (IoU) between the transformed previous mask and the current mask. Weights are updated as $w_i = 1/(1+d_i)$ where $d_i$ is the Euclidean distance between a particle's predicted position and the observed position, then normalized, and the object's estimated position is the weighted mean over particles. This turns noisy per-frame detections into a stable map coordinate, which is precisely what the planner reads when it translates a natural-language command into skills.","core_discovery":"The paper's central claim is that a robot perception module can keep a semantic scene graph current in a dynamic environment by fusing each new RGB-D frame into the graph through a particle filter, and that this continuously updated graph is what makes LLM-based planning reliable in human-inhabited spaces. The author would summarize the contribution as showing that real-time sensor data, a lightweight scene-graph generator, and a particle filter can be combined so that object positions in the semantic map track reality well enough for a planner to act on them. The supporting measurement is Table 1: over 30 readings, the mean absolute position error falls from $(0.07, 0.18, 0.02)$ m without the filter to $(0.02, 0.02, 0.01)$ m with it, and the error standard deviation falls from $(0.35, 0.24, 0.03)$ m to $(0.17, 0.12, 0.02)$ m. The paper also claims that when an action fails, the controller and explainer can return that information to the task planner, which replans against the updated graph.","pith_inferences":["Extension: replacing the inverse-distance weight with a motion-aware likelihood could extend the same filter to handle brief occlusion and crossing objects, conditions the paper's 30-measurement test does not cover.","The graph could carry temporal attributes such as last-updated time or a velocity estimate per object, letting the planner prefer recently observed positions; the paper only anticipates new spatial relationships, not temporal ones.","A sensitivity analysis varying the particle count, the initial covariance, and the IoU threshold would show whether the reported error reduction is intrinsic to the method or depends on hand-set parameters that the paper does not report."],"forward_implications":["A continuously updated semantic graph keeps the LLM planner's object references current, so a command like 'pick the blue bottle on the table' still names the right object after the scene has been rearranged.","Because the graph contains room nodes and object nodes, the same representation supports distinguishing same-named objects by room and applying graph-search algorithms for navigation between rooms.","The closed perception-planning loop lets the robot notice human-induced changes and replan after a failed action rather than continuing with a stale world model.","The reported inference time of about 400 ms per 480p frame for the scene-graph generator makes continuous updating feasible on modest hardware, not only on dedicated research setups."],"supporting_citations":[{"why":"Supplies the scene-graph generation model whose masks, labels, and relationships feed node and edge construction.","marker":"[12]"},{"why":"Provides the attention-based architecture underlying the scene-graph generator, grounding the real-time inference speed claim.","marker":"[13]"},{"why":"Establishes semantic maps as the representation that lets a planner turn environment knowledge into robot tasks.","marker":"[4]"},{"why":"Defines the 3D scene graph structure of rooms, objects, and edges that the semantic map instantiates as a directed graph.","marker":"[6]"}],"fun_headline_variants":["Particle filter slashes robot position error from 18 to 2 cm","LLM robot planner gets real-time scene graph grounding","Dynamic robot perception: particle filter keeps LLM plans on track","Scene graph filter reduces robot object error 9x","Live scene graph + particle filter = reliable LLM robot control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise, supported only by one object over 30 measurements, is that overlap-based mask association plus inverse-distance particle weighting keeps object locations correct in general dynamic scenes without a motion model, without resampling, and without the paper reporting its IoU threshold, particle count, or initial covariance.","fun_headline_variants_meta":{"raw":{"variants":["Particle filter slashes robot position error from 18 to 2 cm","LLM robot planner gets real-time scene graph grounding","Dynamic robot perception: particle filter keeps LLM plans on track","Scene graph filter reduces robot object error 9x","Live scene graph + particle filter = reliable LLM robot control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000703,"raw_usage":{"total_tokens":3231,"prompt_tokens":1061,"completion_tokens":2170,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":677,"completion_tokens_details":{"reasoning_tokens":2085}},"tokens_in":677,"tokens_out":2170,"duration_ms":14846,"temperature":1.0,"reasoning_tokens":2085,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:34:38.292266+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Place two visually similar objects in the same room, move one across the other's path, and record whether the IoU-based association swaps their identities; if labels swap, the scene graph will misrepresent the scene and the LLM planner will act on the wrong object. Repeat the 30-measurement protocol and compare the resulting position error with the reported $(0.02, 0.02, 0.01)$ m.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the scene-graph generation model whose masks, labels, and relationships feed node and edge construction."},{"cited_title":"Vaswani, N","cited_arxiv_id":null,"evidence_quote":"Provides the attention-based architecture underlying the scene-graph generator, grounding the real-time inference speed claim."},{"cited_title":"Galindo, J.-A","cited_arxiv_id":null,"evidence_quote":"Establishes semantic maps as the representation that lets a planner turn environment knowledge into robot tasks."}],"review_version":1}