{"id":"01d9f27f-57dd-48e4-a70a-97044a7609e8","arxiv_id":"2505.08444","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Vis2Plan extracts object symbols from unlabeled play videos with vision models, plans symbolically with A* search, and retrieves reachable real images as subgoals for a goal-conditioned robot policy.","lead":"Vis2Plan turns unlabeled robot play videos into a small symbolic map, then plans tasks by searching that map and selecting real image subgoals from the videos. It reports faster and more successful long-horizon manipulation than video-generation planners, with steps that humans can inspect.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 3's reachability objective and threshold have the wrong sign relative to the definition of Rψ, so the specified planner would minimize reachability; the reported real-robot results cannot come from the mechanism as written.","rationale":"The reader's weakest assumption (peak detection in §3.1) is a legitimate empirical fragility, and the reader's rationale already lists the Eq. 3 inconsistency as one of several issues. I agree the paper is CONDITIONAL, but I would elevate Eq. 3 to the primary concern: it is not merely an unclear threshold but a sign contradiction in the only formal statement of the method's central filtering mechanism. If Rψ is higher-is-better, then the printed objective and constraint both select against reachability. The fact that the real-robot results are strong suggests the implemented system probably used a corrected sign; but because no code or data are released, a reader cannot tell whether the paper's stated algorithm is the one that was evaluated. The stable-state segmentation risk, by contrast, is an empirical assumption whose failure would be visible in final success rates and is at least partially validated by the reported high success. The plan-quality metric in Table 3 is also circular (same Rψ used for selection and evaluation), which compounds the problem: the paper's only explicit evidence of 'physical consistency' is generated by the same estimator whose specification is inconsistent. I therefore keep the reader's CONDITIONAL verdict: acceptance should require correcting Eq. 3, releasing or detailing the filtering procedure, and ideally validating Table 3 with human annotation.","tokens_in":20431,"tokens_out":10479,"duration_ms":108251,"concrete_test":"On a held-out subset of the 15 real-robot trajectories, run the planner's visual subgoal selection in two variants: (A) exactly as printed in Eq. 3 (maximize −∑Rψ, constraint Rψ≤δ) and (B) the sign-corrected variant (maximize ∑Rψ, constraint Rψ≥δ), using the same trained Rψ. Also compute, over the offline dataset, the fraction of demonstrated adjacent pairs satisfying Rψ≤δ vs. Rψ≥δ. If (A) yields success rates near zero or (B) fails to reproduce Table 2's 0.71 average, the paper does not specify the method that produced its headline results. This requires no new robot data if the trained reachability features are released; otherwise the authors should release code and trained checkpoints.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3's Eq. 3 is the only formal specification of the reachability-filtered image retrieval that the central claim depends on, and as written it is internally inconsistent with the definition of Rψ in Section 3.2 and Appendix B.2. Rψ is a contrastive-RL state-occupancy estimate; Section 3.2 states \"higher values indicate better physical reachability.\" Eq. 3 instead maximizes −∑ Rψ (i.e., minimizes the sum of reachability scores) and imposes Rψ(Oi,Oi+1) ≤ δ, where δ = min(Rψ(ot,ot+)) over adjacent training pairs. Since δ is the lowest score observed for a reachable transition, the constraint R≤δ only admits pairs at or below the worst reachable score, and the objective actively selects the lowest-scoring candidates. Literally implemented, this should select the least reachable subgoal images and collapse task success, not produce the reported 0.71 real-robot average. Either Eq. 3 has a sign error (it should be arg max ∑Rψ with Rψ≥δ), or the threshold/objective are defined differently. Without released code or data, neither possibility can be checked, and the central claim that reachability-filtered retrieval yields physically consistent visual subgoals is not reproducible as stated. This is more load-bearing than the segmentation-peak risk in §3.1: a fragile peak detector is an empirical risk, whereas Eq. 3 is an internal contradiction in the specified mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Vis2Plan, a hierarchical visual planning framework that extracts discrete symbolic state transitions from unlabeled play videos using pretrained vision foundation models, builds a directed symbolic transition graph, performs A* search over symbolic states, and then retrieves and filters real image frames as visual subgoals using a contrastive-RL reachability estimator. At execution time, a goal-conditioned low-level policy follows the resulting image subgoal sequence. The authors evaluate Vis2Plan on LIBERO simulation tasks and on a real robot kitchen setup, reporting higher success rates than the AVDC video-generation planner and other graph-based baselines, as well as much faster plan generation. The central claim is that symbolic guidance plus reachability-filtered image retrieval replaces video-generation planners with a faster, more interpretable, and physically consistent visual planner, without requiring language labels on the play data.","tokens_in":20746,"tokens_out":4561,"duration_ms":47778,"significance":"If the reported behavior is real, Vis2Plan would be a useful contribution: it removes the need for task labels, gives inspectable symbolic plans, and produces photo-realistic subgoal images from the dataset rather than hallucinated generated frames. The final task-success evaluation is external to the planning modules, so the headline comparison is not circular. The real-robot results and the 35x speed advantage over AVDC are concrete, falsifiable claims. However, the paper provides no code, and the formal specification of the reachability-filtered subgoal selection is internally inconsistent, which makes it impossible to verify that the reported experiments were produced by the mechanism as written. The plan-quality analysis in Table 3 is also partially circular because it uses the paper's own reachability estimator, which is also used to augment the GSR and UVD-graph baselines.","major_comments":[{"comment":"The reachability-filtered subgoal selection is internally inconsistent. Section 3.2 defines Rψ so that higher values indicate better physical reachability, and defines δ = min(Rψ(ot, ot+)) over adjacent training pairs. Eq. (3) then maximizes −Σ Rψ, i.e., it minimizes the sum of reachability scores, and imposes Rψ(Oi, Oi+1) ≤ δ. Since δ is the lowest score observed for a reachable transition, the constraint admits only pairs at or below the worst reachable score, which is the opposite of a reachability filter. Literally implemented, Eq. (3) should select the least reachable subgoal images and should collapse task success, not produce the reported 0.71 real-robot average. Algorithm 1 (line 15) contradicts Eq. (3) by maximizing Σ Rψ subject to Rψ ≤ δ. The authors must correct Eq. (3) and reconcile it with Algorithm 1 (the likely intended form is arg max Σ Rψ with constraint Rψ ≥ δ), and should clarify which objective was actually used in the experiments.","section":"Section 3.3, Eq. (3); Algorithm 1"},{"comment":"The high-level plan-quality evaluation uses the paper's own reachability estimator to determine whether a plan is 'meaningful'. The text says that Rψ and δ from Eq. (3) are used to decide whether every adjacent subgoal pair is physically achievable, and Table 3 reports 'high-level plan success rate' for all methods. Because Rψ is trained on the same play data and because the GSR and UVD-graph baselines are augmented with that same estimator in their graph construction, this metric is not an independent measure of physical consistency. At best it measures agreement with Vis2Plan's own reachability model. The claim of 'physically consistent' subgoals should be supported by an external criterion, such as human annotation or downstream task success, rather than by the model used to build the plans.","section":"Section 4, Table 3; Section 3.2"},{"comment":"The entire symbolic transition graph depends on stable-state identification via non-maximum suppression on summed SigLIP2 cosine similarities, but the paper provides no quantitative evaluation of segmentation quality. If peaks are missed or spurious, the symbolic graph contains incorrect edges and the high-level plan fails regardless of the low-level policy. Please report segmentation precision/recall against known subskill boundaries in LIBERO, or a sensitivity analysis over the smoothing and NMS parameters, to support the claim that the discovered symbols reliably capture the relevant stable states.","section":"Section 3.1; Appendix B.1, Eq. (4)"}],"minor_comments":[{"comment":"The abstract's '53% higher aggregate success rate' is ambiguous: comparing Vis2Plan's 0.71 average with AVDC's 0.18 in Table 2 is a 53-percentage-point difference, not a 53% relative improvement. Please state the comparison basis explicitly.","section":"Abstract and Section 4"},{"comment":"Table 1 contains many entries rendered as '0.±0.', which appear to be truncated '0.00±0.00' values. The missing digits should be restored, and the text should refer to Table 1 (not Table 3) when discussing the simulation success rates.","section":"Section 4, Table 1"},{"comment":"Equation (1) is missing an '=' sign after L_MC-InfoNCE(Rψ), and the E subscript is not separated from the loss name. Please fix the equation formatting.","section":"Section 3.2, Eq. (1)"},{"comment":"In Eq. (4), the symbol K is used both for the number of object categories and for the set of NMS-selected peaks. This is confusing and should be renamed.","section":"Appendix B.1, Eq. (4)"},{"comment":"The real-robot section refers to 'GC-Diffuser' instead of 'GC-Diffusion'; Section 3.1 has a doubled colon ('challenging due to::'); and the OptiTrack reference in [93] contains the placeholder text 'or whichever you used', which should be removed before publication.","section":"References and prose"}],"recommendation":"major_revision","confidential_remarks":"The Eq. (3) sign/threshold inconsistency is the decisive issue: it is a load-bearing internal contradiction, and because no code is released, the reported numbers cannot be checked against the stated mechanism. I would like the authors to provide the corrected equation, reconcile it with Algorithm 1, and ideally release code or a detailed reproducibility appendix; otherwise the paper cannot be verified. The paper fits CoRL's scope and the idea is promising, but this revision is necessary before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: a genuinely useful systems paper with one formal inconsistency that must be fixed before it's publishable. Eq. 3, the only specification of the reachability-filtered subgoal selection, has the wrong sign and the wrong threshold direction relative to the paper's own definition of Rψ; Algorithm 1, in the same paper, specifies the opposite. This is not a cosmetic typo because the whole central claim hangs on that selection mechanism.\n\nWhat's actually new: Vis2Plan combines frozen-VFM object-centric symbol extraction from unlabeled play data, symbolic A* over the discovered transition graph, and reachability-filtered image retrieval to produce visual subgoals. The real-robot results are decent: it beats AVDC on most tasks and the reported 0.03s plan generation is a real speed advantage. Final task success is measured externally, so the headline results aren't circular. The limitations section is honest about needing object names and weak low-level generalization.\n\nSoft spots, in decreasing order. First, Eq. 3 vs Algorithm 1. Section 3.2 defines Rψ so that higher means more reachable and δ as the minimum score over adjacent training pairs. Eq. 3 maximizes −ΣRψ and constrains Rψ ≤ δ, i.e., it selects the least reachable pairs at or below the worst observed score. Algorithm 1 line 15 maximizes +ΣRψ with Rψ ≤ δ. The stress-test note is right: literally implementing Eq. 3 would collapse task success. Since no code is released, I can't tell which one ran. Fixing Eq. 3 is mandatory. Second, Table 3's plan-quality metric uses Rψ and δ, the same estimator used to build the plans, and the GSR and UVD-graph baselines are augmented with that same estimator. That's circular and inflates Vis2Plan's 1.0 scores. Third, the abstract's '53% higher aggregate success rate' is 53 percentage points (0.71 vs 0.18), not 53% relative. Minor wording, but in the abstract it matters. Fourth, Appendix C literally stops mid-sentence; the supplementary PDF is not the paper, and a referee can't evaluate the missing visual results. The peak-detection assumption in §3.1 is a real empirical risk but secondary: if segmentation fails, the graph is wrong, but that's a robustness issue rather than an internal contradiction.\n\nWho is this for: anyone working on long-horizon manipulation from unlabeled play data. It deserves a serious referee, but I would not cite or build on it until Eq. 3 is corrected and code or data is released. My recommendation: send it to review, with a mandatory fix of Eq. 3 and the plan-quality evaluation, plus a released artifact.","headline":"Useful systems paper with a wrong-signed Eq. 3 that must be fixed; otherwise the central mechanism is not reproducible as written.","tokens_in":21289,"tokens_out":2777,"would_cite":false,"duration_ms":27727,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"From unlabeled play video, symbolic A* planning plus image retrieval outperforms diffusion video-generation planners for robot manipulation, with 53% higher aggregate success on real-robot tasks and 35x faster plan generation.","keywords":["visual planning","symbolic abstraction","unlabeled play data","goal-conditioned policy","reachability estimation","long-horizon manipulation","vision foundation models","A* search"],"falsifier":"Record a play trajectory in which a task-relevant object's visual appearance has no obvious change between two skills (for example, a bowl is moved across an empty table and then a pot is picked up while the bowl stays put). If the summed similarity peak detector finds no boundary at that transition, the symbolic graph will merge the two skills into one edge, and the visual plan will skip the second subgoal; that failure would falsify the segmentation premise.","tokens_in":20230,"feed_emoji":"🤖","tokens_out":6481,"duration_ms":59232,"temperature":0.7,"pith_summary":"The paper tries to establish that long-horizon visual planning for robot manipulation can be done without video generation, language labels, or task annotations, using only unlabeled play data. It claims that vision foundation models can turn raw demonstration videos into a compact set of object-centric symbols, that those symbols form a discrete transition graph, and that A* search over that graph plus image retrieval filtered by a learned reachability score yields physically consistent visual subgoals. If true, this would replace slow, hallucination-prone video-generation planners with a fast, inspectable white-box planner: 53% higher aggregate success in real-robot experiments and $35\\times$ faster plan generation.","feed_headline":"Symbolic planning tops video-generation robot planners by 53%","feed_subtitle":"A* search over learned object-state symbols yields physical subgoal images 35x faster than diffusion planners.","key_machinery":"The load-bearing mechanism is the symbolic transition graph constructed from stable-state segmentation. Stable states are identified by summing object-centric frame-to-frame cosine similarities from a pretrained vision-language encoder, smoothing, and applying non-maximum suppression to find peaks; each skill is a transition between two peaks. Agglomerative clustering with Silhouette-score model selection turns the peak frames into per-object discrete symbolic states, and nearest-neighbor classifiers label every frame, so a demonstration becomes a chain of (before, after) symbolic transitions. At test time A* search over this graph produces a symbolic path, and a contrastively trained reachability estimator $R(s,s_g)$, trained with a state-only InfoNCE objective, scores whether the current state can reach a candidate goal; beam search then selects image subgoals satisfying a reachability threshold. This is the identity that carries the argument: planning happens in a small discrete symbol space, while physical consistency is delegated to retrieval plus a learned reachability filter.","core_discovery":"The central claim, stated on the paper's own terms, is that a symbolic abstraction of object states can carry the full burden of high-level visual planning. From unlabeled play data, Vis2Plan detects stable states by peak-finding on summed cosine similarities of object-centric features, clusters those states per object via unsupervised clustering, and labels each frame with a nearest-neighbor classifier, yielding symbolic transitions of the form (before, after). These transitions form a directed graph; at test time a next-symbolic-state predictor proposes candidate successors from the current image, A* search selects the shortest symbolic path to the user's goal, and a beam search over dataset images maximizes cumulative learned reachability under a threshold to select a physically consistent image plan. A goal-conditioned policy then executes action chunks toward each image subgoal in closed loop. The paper reports that this pipeline outperforms a diffusion video-generation planner by 53% aggregate success in real-robot settings and is $35\\times$ faster at generating visual plans.","pith_inferences":["If the symbolic extraction scales, the same graph could answer counterfactuals, such as which sequence of object states would realize a goal combination not observed in the data, by checking whether that combination exists as a node set.","The learned reachability score could double as a progress reward for downstream reinforcement learning or as an execution-time safety check, since it already encodes which states are reachable from a given observation.","A natural extension is to let a vision-language model propose object names and goal symbols, replacing the human-provided object list and the user-specified symbolic goal; this would push toward open-vocabulary play data but would inherit vision-language hallucination risk.","The paper's reachability threshold could serve as a cheap automated proxy for physical plausibility in future visual-planning evaluations, independent of task success metrics."],"forward_implications":["Without any language labels, task annotations, or video generators, a robot can assemble a multi-goal visual plan by searching a learned symbolic graph and retrieving dataset images.","Physical consistency of subgoals comes from a learned reachability filter over real images, not from generative sampling, so planned subgoals are photo-realistic and inspectable by construction.","Because planning is A* over a compact graph, plan generation is near real-time in the reported setups, making closed-loop hierarchical control practical.","The approach inherits its ceiling from the low-level goal-conditioned policy; improving that policy or collecting more training data directly raises end-to-end success, as the paper itself notes."],"supporting_citations":[{"why":"Supplies the unlabeled play dataset used in simulation and the low-level goal-conditioned policy used in real-robot experiments.","marker":"[71]"},{"why":"Provides the vision-language encoder whose object-centric cosine similarities drive stable-state detection and symbolic clustering.","marker":"[74]"},{"why":"Detects task-relevant objects in the first frame to initialize object-centric video tracks.","marker":"[72]"},{"why":"Tracks the detected object bounding boxes across frames to produce object-centric videos.","marker":"[73]"},{"why":"Supplies the contrastive reinforcement learning formulation that the state-only reachability estimator builds on.","marker":"[84]"},{"why":"Provides the contrastive RL implementation and hyperparameters followed for the reachability network.","marker":"[85]"},{"why":"Gives the flow-based goal-conditioned policy used as the low-level controller in simulation.","marker":"[87]"},{"why":"Is the diffusion video-generation visual planner baseline that Vis2Plan compares against and outperforms.","marker":"[1]"},{"why":"Is the graph-search-retrieval baseline that represents image-latent graph planning, a main comparison.","marker":"[90]"},{"why":"Is the UVD-based segmentation baseline used to build the UVD-graph planner, another main comparison.","marker":"[91]"}],"fun_headline_variants":["Symbolic planning: 53% higher success, 35x faster than video","Symbols instead of video: robot planning 53% more successful","Vis2Plan: symbolic subgoals 53% better, 35x faster","White-box robot planning: symbols beat diffusion video","No video generation needed: symbol-guided plan wins 53%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline treats every stable-state boundary found by peak detection over summed object-centric feature similarities as a genuine skill boundary; if any sub-skill ends or begins without producing a clear similarity peak, the symbolic graph receives a wrong or missing edge and the downstream plan fails regardless of the low-level policy.","fun_headline_variants_meta":{"raw":{"variants":["Symbolic planning: 53% higher success, 35x faster than video","Symbols instead of video: robot planning 53% more successful","Vis2Plan: symbolic subgoals 53% better, 35x faster","White-box robot planning: symbols beat diffusion video","No video generation needed: symbol-guided plan wins 53%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001094,"raw_usage":{"total_tokens":4562,"prompt_tokens":933,"completion_tokens":3629,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":3536}},"tokens_in":549,"tokens_out":3629,"duration_ms":22020,"temperature":1.0,"reasoning_tokens":3536,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:55:17.843089+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record a play trajectory in which a task-relevant object's visual appearance has no obvious change between two skills (for example, a bowl is moved across an empty table and then a pot is picked up while the bowl stays put). If the summed similarity peak detector finds no boundary at that transition, the symbolic graph will merge the two skills into one edge, and the visual plan will skip the second subgoal; that failure would falsify the segmentation premise.","supporting_citations":[{"cited_title":"Eysenbach, T","cited_arxiv_id":null,"evidence_quote":"Supplies the contrastive reinforcement learning formulation that the state-only reachability estimator builds on."},{"cited_title":"Zhang, Y","cited_arxiv_id":null,"evidence_quote":"Is the UVD-based segmentation baseline used to build the UVD-graph planner, another main comparison."}],"review_version":1}