{"id":"e89d748d-1480-4b84-a6a5-ffd85ff4e041","arxiv_id":"2504.20720","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A graph-based garment state representation and a learned dynamics model let a robot perceive and fold jackets with self-occlusion.","lead":"This paper proposes a topological graph representation of garments, built from keypoint detection and occlusion-aware segmentation, plus a learned neural-network dynamics model that predicts how the graph deforms during folding. The authors validate the pipeline on a Franka robot folding jackets, including cases where sleeves self-occlude.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central 'general model' claim rests on an untested sim-to-real transfer of the topological dynamics; real-world prediction error is never measured, so the Jacobian controller may plan from false predictions.","rationale":"The reader's identified weakest assumption matches my own analysis: the paper explicitly asserts sim-to-real transfer in Sec. IV-C but provides no evidence for it. The central claim of a general, effective folding algorithm depends on the learned dynamics being accurate on real fabric, which is the foundation of the Jacobian-based controller. The reported simulated MSE of 0.017 is the only quantitative validation, and it is not compared to any real measurement. The two qualitative demos, while supportive, are insufficient to establish the claimed generality. This concern is distinct from a consensus disagreement; it is a correctness risk that can be resolved only by measuring real-world predictive error or by a controlled comparison of folding success with and without the learned model. The paper is otherwise coherent and a conditional verdict is appropriate, so I recommend no change to the reader's verdict.","tokens_in":9429,"tokens_out":4040,"duration_ms":43424,"concrete_test":"Run a set of real folding trials (e.g., 10 or more) with the jacket in varied initial configurations. During each trial, after each action, record the GNN-predicted node positions for the next step and compare them to the CoTracker-updated node positions, without applying the corrective model. Compute the mean squared error in normalized units and compare it with the simulated validation MSE of 0.017. If the real error is more than, say, 5x the simulated error, the sim-to-real transfer assumption is falsified and the Jacobian controller's predictions are unreliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the GNN dynamics model is 'general' and that 'models trained in simulation are expected to exhibit similar performance in real-world applications' (Sec. IV-C). This expectation is never tested: no real-world prediction error is reported, and the only evidence is two qualitative folding demos. The Jacobian controller (Eq. 9) relies on the predictive model's gradient; if the model's predictions in real conditions are inaccurate, the controller will generate erroneous trajectories. Moreover, the training graphs are extracted directly from meshes, while at inference the graphs come from a perception pipeline (ViTPose, Mask2Former, and hand-crafted occlusion rules), so a distribution shift exists both in physics and in graph feature statistics. The paper provides no quantitative success metric (the defined area threshold is never reported), so the two demos cannot rule out that the successes are due to the open-loop lifting-placing pattern rather than the learned dynamics. Therefore, the load-bearing assumption is the correctness of the sim-to-real transfer, and it is entirely unvalidated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an end-to-end framework for folding complex garments, centered on a low-dimensional topological graph representation of clothing state. The graph is constructed from RGB observations by combining ViTPose keypoint detection, Mask2Former occlusion-aware semantic segmentation, and two hand-defined occlusion decomposition rules. A GNN-based forward dynamics model and a corrective model are trained on topological graphs extracted from meshes in a PyBullet dataset, and the predictive model's Jacobian is used in a feedback controller to execute multi-stage folding. Experiments on a real Franka Research 3 platform with jackets show qualitative perception outputs and two image sequences of folding; the dynamics model reports a single validation MSE of 0.017. The central claim is that this constitutes a general model capable of recognizing and folding complex clothing with self-occlusion.","tokens_in":9649,"tokens_out":6906,"duration_ms":69948,"significance":"If the claims were fully substantiated, the topological graph representation would be a valuable step toward reducing the dimensionality of deformable-object state for manipulation, and the combination of occlusion-aware perception, learned graph dynamics, and Jacobian-based control is sensible and coherent. The paper also makes a falsifiable prediction—sim-to-real transfer of the learned dynamics—which is, however, never tested. The contribution is potentially significant for cloth manipulation, but the current evidence is predominantly qualitative: perception results are illustrative, the dynamics model is validated by a single simulation number, and the folding demonstrations are two sequences without success-rate statistics, error bars, or ablations. The strengths are the novel graph formulation, the integration of perception/dynamics/control, and the demonstration on a real robot; the main weakness is the absence of quantitative validation of the load-bearing sim-to-real and perception-to-graph steps.","major_comments":[{"comment":"The central claim of a general model rests on the assertion in Section IV-C that 'models trained in simulation are expected to exhibit similar performance in real-world applications,' but this expectation is never tested. The GNN is trained on topological graphs extracted directly from garment meshes in a PyBullet dataset, whereas at inference the graphs come from ViTPose keypoints, Mask2Former segmentation, and the hand-crafted occlusion rules of Section III-B. These are different input distributions in both graph topology and feature statistics, and the physical dynamics of real fabric are not guaranteed to match the simulator. Because the Jacobian controller in Eq. (9) is the gradient of the learned predictive model, a systematic sim-to-real gap would produce incorrect planned trajectories. The paper reports no real-world prediction error, no comparison of graph feature distributions, and no sim-to-real adaptation; without such evidence, the folding demos in Fig. 7 cannot support the claimed generality.","section":"Section IV-C and Eq. (5)"},{"comment":"The quantitative evaluation of the dynamics model is a single validation MSE of 0.017 on an unspecified subset of the GarmentTracking dataset. No standard deviation across seeds, per-node or per-edge error, ablation of the region-aggregation term in Eq. (5), or comparison against a standard GNN or another dynamics baseline is reported. The corrective model ffeedback (Eq. 4) is never evaluated in isolation. Consequently, the claimed improvement of the message-passing mechanism and the 'general dynamics' of the topological graph are not substantiated; the low MSE could reflect memorization of simulation-specific patterns rather than a general cloth model. The authors should provide a rigorous evaluation with ablations, baselines, and error decomposition.","section":"Section IV-C, Fig. 7"},{"comment":"The folding validation consists of two image sequences. The success criterion defined in Section III ('area of clothing smaller than a specified threshold') is never operationalized or reported, and no success rate, error bars, or repeated trials are given. The two demos do not include a baseline controller (e.g., an open-loop lifting-placing strategy without the learned Jacobian), so it is impossible to determine whether the outcome is due to the learned dynamics model or to the generic pick-and-place pattern. To support the central claim, the authors should report a quantitative success rate over multiple starting configurations, measured final garment area, and a comparison against a non-predictive or open-loop baseline.","section":"Section III-B, Fig. 6"},{"comment":"The topological graph extraction relies on hand-defined occlusion decomposition rules and the keypoint detection threshold in Eq. (3), but no quantitative metrics are provided for graph extraction accuracy. The comparison with 'traditional semantic keypoint detection' in Fig. 6 is qualitative only; there is no precision/recall for keypoint detection, no graph-structure accuracy (e.g., correct edge connectivity), and no failure-rate analysis across the 280-image dataset. Because graph extraction errors directly propagate into the dynamics model and the controller, the claim that the representation can be robustly extracted from self-occluded garments needs quantitative validation.","section":"Section IV-A"}],"minor_comments":[{"comment":"In Eq. (1), G0 is used both as the initial graph and as the observation-to-graph mapping function; please disambiguate these two uses.","section":"Eq. (1)"},{"comment":"The phrase 'li is the number of the region that vi belongs to' is ambiguous; please specify whether li is the number of nodes in the region and correct the wording accordingly.","section":"Section III-A, Eq. (5)"},{"comment":"The name 'Cotracker' is used inconsistently with 'CoTracker3' in the text and in reference [23]; please use a single consistent name.","section":"Section IV-C"},{"comment":"There is a missing space in 'wherekh is a positive coefficient'; please correct this typographical issue.","section":"Section III-B, Eq. (3)"},{"comment":"The expected trajectory for the lifting stage is described verbally as 'circles about the crease,' but the geometric construction (e.g., how the circle center and radius are chosen) is not specified; an equation or figure would improve reproducibility.","section":"Section III-D"},{"comment":"The assumption that the gripper can stably grasp all layers of the clothing is strong and is not validated in the experiments; it should be discussed as a limitation of the current work.","section":"Section IV-A"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of a robotics/automation journal, but the gap between the broad claims and the empirical evidence is large. I support a major revision rather than rejection because the topological representation is interesting and the architecture is coherent; however, the authors must add quantitative experiments on perception accuracy, dynamics prediction (including real-world error), and repeated folding trials, and they must include a baseline comparison for the folding task."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read on arXiv:2504.20720. The genuinely new thing here is the topological graph for garment state: region nodes, crease nodes, sleeve nodes, with an occlusion-aware extraction pipeline that combines semantic segmentation and keypoint detection. That's a real step beyond the usual towel/T-shirt flattening stuff, and the representation is thoughtful—low-dimensional, category-agnostic in principle, and it gives you a dynamics model and Jacobian control in one package. The region-level message aggregation in the GNN is a minor but sensible twist. This is the part worth taking seriously.\n\nThe soft spots are exactly where the reader's report puts them. The perception evaluation is qualitative only; the dynamics model reports a single simulation MSE (0.017) with no real-world predictive error; the folding validation is two image sequences with no success rate, no error bars, no ablations, no baselines. The \"general model\" claim rests on an untested assumption that simulation-trained dynamics transfer to real fabric. The paper literally says \"models trained in simulation are expected to exhibit similar performance\"—that expectation is never measured. Given the Jacobian controller plans from the learned model, a sim-to-real gap would break the whole pipeline. Also, training graphs come from meshes while inference graphs come from perception, so there's a feature-distribution shift that is unexamined. No code or data released, which makes it hard to verify anything beyond the qualitative claims.\n\nThat said, the stress-test note is fair but not fatal. The paper doesn't claim a statistically validated system; it claims a representation and a plausible pipeline, with illustrative demos. The core idea is coherent, and the limitations are at least implicitly acknowledged. It's a solid workshop-quality or early-stage paper that needs real experimental work before the \"general\" claim is supported.\n\nWho should read it: people working on cloth manipulation, especially those interested in compact state representations and model-based control for deformables. The representation itself is worth citing even if the evaluation is thin.\n\nI'd send this to peer review—it's novel enough and the idea is worth developing—but I'd brace the authors for a request for quantitative real-world evaluation, ablations, and a clearer account of the sim-to-real transfer.","headline":"A genuinely new topological representation for garment state, paired with a plausible pipeline, but the 'general' claim rests on an untested sim-to-real assumption and almost no quantitative real-world evaluation.","tokens_in":10155,"tokens_out":2058,"would_cite":true,"duration_ms":20564,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a topological graph of a garment's visible creases, sleeves, and folds can support a learned dynamics model that folds complex jackets, even with self-occlusion.","keywords":["clothing manipulation","topological graph","graph neural network","deformable object manipulation","self-occlusion","Jacobian control","robot folding","sim-to-real transfer"],"falsifier":"Measure the one-step predictive error of the graph neural network on a real jacket during the lifting phase by comparing predicted node positions with the tracked node positions; if the mean error exceeds the width of the jacket sleeve, the Jacobian controller would be planning from states that no longer match the cloth, and the claimed generality would fail.","tokens_in":9212,"feed_emoji":"👕","tokens_out":6563,"duration_ms":60507,"temperature":0.7,"pith_summary":"The authors set out to make clothing folding tractable by replacing the garment's enormous physical state with a small topological graph. They argue that the visible folding structure—crease endpoints, region corners, and sleeve axes—captures the constraints that actually govern how cloth moves. Using this graph, a graph neural network learns a general deformation model from simulated folding data, and the model's Jacobian drives a feedback controller that executes the fold. The paper reports real-robot experiments folding jackets, including configurations where sleeves occlude the body, supporting the claim of generality.","feed_headline":"One learned graph model folds jackets with overlapping sleeves","feed_subtitle":"Instead of simulating every thread, the robot learns how creases and sleeves move and plans folds from that.","key_machinery":"The central object is the topological graph $\\langle V,E\\rangle$, whose nodes come in three types: region vertices ($N_0$), crease endpoints ($N_1$), and sleeve-axis samples ($N_2$). Each node stores position, velocity, height above the folding plane, type, and the center of its fabric region, while edges connect nodes along region contours and carry features that describe outline deformation. The graph is generated by iterative keypoint detection and occlusion-layer semantic segmentation, and is kept current during manipulation by a corrective graph neural network and a predictive graph neural network. The mechanism that turns learning into control is the deformation Jacobian $J_f$, obtained by backpropagating through the predictive model and inserted into the controller $v_r = k_v (J_f^T J_f)^\\dagger J_f^T \\frac{d}{d\\tau}s_f(\\tau)$.","core_discovery":"The central discovery is that a garment's folding state can be represented by a low-dimensional heterogeneous graph, and that the dynamics of this graph can be learned and controlled without modeling every thread. The graph is extracted from a single RGB image by detecting keypoints and semantically segmenting occlusive layers, which lets the method reconstruct the connected clothing parts even when one layer hides another. Two graph neural networks—a corrective model and a predictive model—update the graph state during manipulation, and backpropagation through the predictive model yields a deformation Jacobian used for feedback control. The authors validate the full pipeline on a physical robot folding jackets from several initial configurations, including a self-occluded case.","pith_inferences":["The topological graph's dependence on visible creases suggests the method may degrade on garments with many wrinkles or very flexible fabrics where the crease structure is not stable; an obvious test is to vary fabric stiffness and wrinkle density.","A stronger validation of the 'general' claim would be to report quantitative success rates over many jacket configurations rather than the two illustrative examples shown.","The same graph-dynamics idea could be applied to other articulated soft objects, such as bags or tents, where a small set of semantic nodes and occlusion relations defines the relevant state."],"forward_implications":["If the model is general, the same learned dynamics can fold a jacket from any initial configuration from which the topological graph can be extracted, without retraining.","Because the state is a low-dimensional graph rather than a mesh, model-based cloth control becomes computationally light enough to run on a laptop GPU in real time.","The method removes the need for a predefined clothing template, so it should extend to other structured garments (shirts, cardigans) once the keypoint and segmentation networks are trained on them.","The same Jacobian-based feedback loop could be applied to other deformable-object tasks, such as unfolding or hanging, as long as the topological graph can be defined."],"supporting_citations":[{"why":"Supplies the keypoint detection network that produces heatmaps from which the graph's nodes are extracted.","marker":"[21]"},{"why":"Supplies the semantic segmentation model that separates occlusive layers of the garment so the graph can be built under self-occlusion.","marker":"[22]"},{"why":"Provides the simulated garment-folding dataset used to train the graph neural network dynamics model.","marker":"[12]"},{"why":"Provides the message-passing graph network architecture that the corrective and predictive dynamics models are built on.","marker":"[24]"},{"why":"Provides the real-time point tracker used to update visible graph nodes during manipulation.","marker":"[23]"},{"why":"Supplies the flinging action used to initially unfold the jacket before folding begins.","marker":"[25]"}],"fun_headline_variants":["Graph dynamics teach robot to fold jackets","Low-dim graph model folds self-occluded clothing","Robot learns jacket folding from topological graph","One graph predicts and controls clothing folds","Semantic graph enables robot to fold jackets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach rests on the assumption that cloth behaviour in simulation matches cloth behaviour in reality well enough that a dynamics model trained only on simulated topological graphs can predict real fabric deformation without retraining or domain adaptation.","fun_headline_variants_meta":{"raw":{"variants":["Graph dynamics teach robot to fold jackets","Low-dim graph model folds self-occluded clothing","Robot learns jacket folding from topological graph","One graph predicts and controls clothing folds","Semantic graph enables robot to fold jackets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00017,"raw_usage":{"total_tokens":1215,"prompt_tokens":837,"completion_tokens":378,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":313}},"tokens_in":453,"tokens_out":378,"duration_ms":4647,"temperature":1.0,"reasoning_tokens":313,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:21:23.568859+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the one-step predictive error of the graph neural network on a real jacket during the lifting phase by comparing predicted node positions with the tracked node positions; if the mean error exceeds the width of the jacket sleeve, the Jacobian controller would be planning from states that no longer match the cloth, and the claimed generality would fail.","supporting_citations":[{"cited_title":"ViTPose: Simple vision transformer baselines for human pose estimation,","cited_arxiv_id":null,"evidence_quote":"Supplies the keypoint detection network that produces heatmaps from which the graph's nodes are extracted."},{"cited_title":"Per-pixel classification is not all you need for semantic segmentation,","cited_arxiv_id":null,"evidence_quote":"Supplies the semantic segmentation model that separates occlusive layers of the garment so the graph can be built under self-occlusion."},{"cited_title":"GarmentTracking: Category-Level Garment Pose Tracking,","cited_arxiv_id":null,"evidence_quote":"Provides the simulated garment-folding dataset used to train the graph neural network dynamics model."},{"cited_title":"Learning to simulate complex physics with graph networks,","cited_arxiv_id":null,"evidence_quote":"Provides the message-passing graph network architecture that the corrective and predictive dynamics models are built on."},{"cited_title":"FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding,","cited_arxiv_id":null,"evidence_quote":"Supplies the flinging action used to initially unfold the jacket before folding begins."}],"review_version":1}