{"id":"0cbe8346-7051-447d-8e3c-930c829c8fd5","arxiv_id":"2412.02676","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"GLIDE trains one point-cloud diffusion policy on planner-generated simulation trajectories and successfully reorients diverse objects, including out-of-distribution shapes, with two robot arms in the real world.","lead":"A robot learns to reorient bulky objects with two arms, using synthetic demonstrations generated by a motion planner in simulation instead of human teleoperation. A single vision-based diffusion policy trained this way transfers to real boxes, containers, and even soft toys, showing a practical path to generalizable bimanual manipulation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Task-conditioning c_t may be computed from clean simulator poses during training but from noisy keypoint tracking at deployment; the OOD real-world results may reflect tracking quality, not manipulation generalization.","rationale":"I read the paper as a contribution in scalable planning-guided data synthesis plus behavior cloning, with the empirical core being sim-to-real transfer. The weakest link is not contact dynamics per se but the fact that the goal signal itself goes through an unvalidated perception pipeline. The ablations in Tab. III cover residual actions and flying point augmentation, but not the task-representation pathway described in Sec. III-C. Thus the architecture's success on OOD objects could be due to robust manipulation or to robust tracking, and the reported numbers do not separate the two. This is a concrete, testable concern rather than a rejection: the reader's conditional verdict remains appropriate, and my concern does not change it. I am not claiming the authors intentionally misled; the omission is that the provenance of c_t during training is unspecified, which is exactly the kind of missing support a stress-test should flag.","tokens_in":12607,"tokens_out":7195,"duration_ms":78499,"concrete_test":"Train two GLIDE policies in DRAKE on the same filtered 12,000-demonstration set: variant A computes c_t from ground-truth object pose; variant B computes c_t by running the same open-vocabulary segmentation and keypoint tracking on rendered point clouds with simulated depth noise. Evaluate both in simulation on held-out boxes and on deformable-like objects, and additionally report tracking error of c_t against ground truth. If variant B matches variant A on all tasks, the concern is resolved; if not, the real-world OOD numbers are confounded by task-representation error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing point is the task specification input c_t. In Sec. III-C the policy receives a transformation from the current object pose to the target pose, obtained by segmenting the target in o0 and tracking keypoints with [56,57]. The paper never states whether this segmentation-plus-tracking pipeline is also used to produce c_t for the 12,000 synthetic demonstrations, or whether c_t is computed from ground-truth object poses in DRAKE. Because all other policy inputs are rendered point clouds, there is a real possibility of a train/test shift in the very variable that encodes the goal: exact c_t in training, noisy c_t at deployment. This matters most for the paper's headline OOD results: soft rubber/fabric containers and inflatable toys have no stable rigid pose, so keypoint tracking can be poor, and an incorrect c_t guarantees failure regardless of how well the policy has learned contact-rich manipulation. The observed drop from 0.80 fixed-rotation in-distribution to 0.28 random-rotation OOD is consistent with this confound, but the paper reports no tracking accuracy and no ablation of task-representation noise. The central claim of generalization across physical properties therefore rests on an unmeasured perception component.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GLIDE, a planning-guided diffusion policy approach for contact-rich bimanual object reorientation. Synthetic demonstration trajectories are generated in the DRAKE simulator using an efficient contact-implicit planner, filtered by high-fidelity rollouts, and used to train a single task-conditioned point-cloud diffusion policy via behavior cloning. The method introduces several sim-to-real design choices, including residual joint action prediction, flying-point augmentation, and a goal transformation computed from keypoint tracking. The paper evaluates GLIDE on in-distribution boxes in simulation and the real world, and on out-of-distribution containers and inflatable toys in the real world. Results show strong in-distribution success (e.g., 0.80 on fixed 45-degree rotation) and lower but positive OOD success (0.657 on fixed rotation, 0.282 on random rotation for containers).","tokens_in":12813,"tokens_out":5503,"duration_ms":62751,"significance":"If the central claim holds, GLIDE would be a valuable contribution: it avoids expensive real-world teleoperation, trains a single policy for user-specified target poses, and shows some real-world generalization to objects with geometries and physical properties not seen in training. The paper's strengths include a concrete planning-to-learning pipeline, real hardware deployment, ablations of the proposed design choices, and evaluation on genuinely challenging OOD objects such as soft containers and inflatable toys. However, the significance is tempered by two gaps: the task-conditioning input c_t may be computed from ground-truth poses in simulation but from noisy keypoint tracking at deployment, and the real-world evidence consists of small trial counts without uncertainty quantification. These gaps directly affect the paper's headline claim of generalizable manipulation across physical properties.","major_comments":[{"comment":"The paper does not state how the task specification c_t is computed for the 12,000 synthetic demonstrations. Section III-C describes segmenting the target object in o0, selecting keypoints via farthest point sampling, and tracking them with TAPIR/CoTracker, but if this perception pipeline is used only at deployment while simulation training uses ground-truth DRAKE poses, then the policy is trained with exact task specifications and evaluated with noisy ones. Since an incorrect c_t guarantees failure regardless of manipulation skill, the out-of-distribution results in Table II may reflect tracking quality on deformable objects rather than generalization of contact-rich manipulation. Please state explicitly whether c_t in training is derived from ground-truth object poses or from the perception pipeline, and provide either per-condition tracking error statistics or an ablation that injects c_t noise at evaluation.","section":"III-C"},{"comment":"Real-world success rates are reported from 25 trials per in-distribution condition and 16 trials per OOD condition, without confidence intervals or exact trial counts per cell. For example, the OOD fixed-rotation rates 0.688 (empty containers) and 0.625 (overfilled containers) differ by one trial out of 16, so the statement that performance is \"similar across both empty and overfilled containers\" is not quantitatively supported. Please report exact numerators and denominators and binomial confidence intervals for all real-world conditions, especially where claims of robustness depend on small differences.","section":"IV-C and IV-D, Tables I-II"},{"comment":"The success criterion \"final object pose within 10 cm and 0.2 rad\" presupposes a rigid object pose, but the OOD evaluation includes fabric and rubber containers as well as inflatable toys whose geometry deforms during manipulation. The manuscript does not specify how the final pose is measured for these non-rigid objects, nor how tracked keypoints are converted to an SE(2) pose. Without a stated measurement protocol, the OOD results are not reproducible and remain entangled with the tracking-based c_t computation discussed above.","section":"IV-A and IV-D"}],"minor_comments":[{"comment":"The description of demonstration filtering is underspecified: the manuscript says trajectories that \"don't reach the goal\" and \"take too long\" are discarded, but no quantitative thresholds, timeout horizon, or filter rollout success criterion are given. This is needed to reproduce the 12,000-demonstration dataset.","section":"III-B"},{"comment":"The notation for residual actions, a_{t+1:t+Ta} = {q_i - q_t}, is unclear because the index i is not defined. Please clarify whether q_i denotes a planned future joint position and q_t the current measured joint position.","section":"III-C"},{"comment":"The sentence \"We use a larger Ta = 20 at test time and train our policy with Ta = 64\" is confusing alongside Table IV, which ablates Ta at evaluation time. Please specify that Ta is the action-chunk length used at inference and explain how a policy trained with chunk length 64 can be evaluated with shorter chunks.","section":"III-C"},{"comment":"The phrase \"vanilla DP3 baseline\" in the Table I caption is not a defined baseline. The last row of Table III and the first column of Table IV combine several ablations and do not clearly correspond to the DP3 method of [9] or [53]. Please add an explicit description of the baseline configuration.","section":"Tables I and III"},{"comment":"The claim that performance \"doesn't plateau as we scale up the number of demonstrations\" is based on four points without error bars. Please soften this claim or provide repeated-seed variance.","section":"IV-E"},{"comment":"There is a typo in the first paragraph of the demonstration synthesis section: \"Compared with with the sampling-based planner\" should read \"Compared with the sampling-based planner.\"","section":"III-B"}],"recommendation":"major_revision","confidential_remarks":"This is a competent systems paper with a plausible central claim, but the missing statement about how c_t is computed in training versus deployment is a real confound for the OOD results, and the small real-world trial counts make the quantitative claims fragile. I see no evidence of misconduct; the issues are fixable with additional reporting and targeted experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read on 2412.02676. The core result is real and useful: they use a contact-implicit planner in Drake to synthesize ~12k demonstrations for two 7-DoF arms reorienting randomized boxes, filter them by rollout, and train a single task-conditioned point-cloud diffusion policy that transfers to real hardware with zero real training data. The residual joint action head and the flying-point augmentation are the two design choices that make the sim-to-real transfer work; their ablation table shows real-world success jumping from 0.0 to 0.8 when both are present. That is a genuinely useful data point for the robot learning community.\n\nThe paper is not circular. The planner generates the data and the policy is evaluated on held-out objects, including soft containers and inflatable toys in the real world. The OOD results are real but modest: 0.66 fixed-rotation on containers, 0.28 random-rotation, and 0.52/0.28 on toys. The abstract's claim of handling diverse objects is fair if you read the numbers carefully, but the headline success rates hide wide confidence intervals (25 real trials in-distribution, 16 per OOD condition) and no error bars appear anywhere.\n\nThe soft spot that bothers me most is the task conditioning input c_t. In Sec III-C they describe computing c_t at deployment by segmenting the object in the initial frame and tracking keypoints with CoTracker. But they never state how c_t is computed during training. If it's computed from ground-truth poses in Drake (which the planner has), then the policy sees exact c_t during training and noisy c_t at deployment. That is a train/test shift on the very variable that encodes the goal. The drop from 0.80 in-distribution to 0.28 random-rotation OOD is consistent with tracking degrading on soft containers, and the paper reports no tracking accuracy and no ablation with noise added to c_t. This is a missing detail and a missing experiment, not a fatal flaw, but it is load-bearing for the generalization claim.\n\nOther annoyances: the DP3 baseline is referenced but the tables don't make clear which rows are DP3; no code or data release; the planner reference in Tab I is the raw planner success rate, which the policy beats—that's fine but should be explained. The scaling experiment (500 to 12k demos) is a nice addition.\n\nOverall, this is a serious piece of empirical work that deserves peer review. I'd recommend a conditional accept with requests for the c_t training detail, error bars, and a tracking-quality ablation. If the authors clarify that c_t is computed the same way in sim and real, the main concern dissolves.","headline":"GLIDE shows planning-generated synthetic data can train a transferable bimanual diffusion policy, but the paper hides a likely train/test shift in its goal-conditioning input and leans on small real-world trials.","tokens_in":13359,"tokens_out":3585,"would_cite":true,"duration_ms":36606,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GLIDE trains a task-conditioned diffusion policy on planner-generated simulation demonstrations so a two-arm robot can reorient novel boxes, containers, and toys from point clouds alone.","keywords":["bimanual manipulation","diffusion policy","contact-rich manipulation","planning-guided learning","sim-to-real transfer","point cloud policy","object reorientation","synthetic demonstrations"],"falsifier":"Deploy the same GLIDE policy on a set of real objects with measured friction coefficients spanning a wide range outside the simulated distribution, and record success as a function of friction: if success collapses smoothly as friction deviates while simulation success stays high, the central transfer claim would be disproved.","tokens_in":12389,"feed_emoji":"🤖","tokens_out":3783,"duration_ms":41266,"temperature":0.7,"pith_summary":"This paper claims that contact-rich bimanual reorientation of bulky objects can be learned without any real-world demonstrations, by training a single task-conditioned diffusion policy on trajectories synthesized by a model-based motion planner in simulation. The authors argue that efficient planning in randomized environments provides enough high-quality data to replace teleoperation, and they introduce design choices in point-cloud processing, task encoding, and action prediction that close the sim-to-real gap. If right, a robot with two arms and a depth camera can be taught new whole-arm manipulation skills in about two days of offline planning and then handle boxes, soft containers, and inflated toys it has never seen. The measured support is real-world success of 0.80 for fixed 45-degree rotations of in-distribution boxes, 0.657 for out-of-distribution containers under the same rotation, and 0.28 for random rotations of those containers.","feed_headline":"Planner-made data teach two arms to reorient unfamiliar objects","feed_subtitle":"A diffusion policy trained only in simulation reorients boxes and soft containers in the real world.","key_machinery":"The load-bearing mechanism is the demonstration-synthesis pipeline: a greedy contact-rich planner that alternates between collision-free approach motions and single-step contact trajectory optimization to move the object toward its goal, followed by filtering of trajectories via rollouts in a high-fidelity simulator. This turns model-based planning into a data source that is fast enough to produce 12,000 training demonstrations in about two days on a 96-CPU machine. The second half is a task-conditioned diffusion policy over point clouds and proprioception that predicts sequences of residual joint commands; the residual action parameterization is the main adaptation that keeps the actions smooth and consistent across heterogeneous demonstrations.","core_discovery":"GLIDE is the finding that a contact-rich motion planner can act as a scalable data engine for bimanual manipulation, and that the resulting policy can match or exceed the planner's own success rate while operating from point clouds without privileged object information. A diffusion policy is trained by behavior cloning on planner-generated demonstrations that are filtered through high-fidelity simulator rollouts, and the policy is conditioned on the relative transformation from the current object pose to a user-specified goal. The authors identify the essential design choices that make sim-to-real transfer work: predicting residual joint position actions instead of absolute actions, adding flying-point noise augmentation, and computing task specifications online from tracked keypoints. With these choices, the policy reorients objects whose geometry and physical properties are outside the training distribution of box primitives, including soft rubber and fabric containers and inflatable toys.","pith_inferences":["The same residual-action and flying-point-augmentation recipes could transfer to other contact-rich sim-to-real policies beyond diffusion policies, and they are cheap to test on existing benchmarks.","The drop to 0.28 success on random rotations of out-of-distribution containers suggests a ceiling tied to simulator fidelity for deformable objects; adding deformable bodies to the training distribution is a natural next step.","Because failures concentrate in poor joint configurations and torque-limit squeezing, collecting planner demonstrations that start from near-failure states could improve the hard-rotation regime more than simply adding more nominal trajectories."],"forward_implications":["A single policy, not one model per task, can handle arbitrary target object poses because the task is encoded as a delta transformation recomputed at each time step.","Because training data come from a planner, scaling data generation is a matter of CPU time rather than human teleoperation, and the reported scaling curve does not plateau at 12,000 demonstrations.","Real-world deployment works without knowing object shape or pose in advance, using only processed point clouds and tracked keypoints to infer the required manipulation.","Out-of-distribution containers and inflatable toys are handled at lower but non-trivial success rates, showing that the policy transfers beyond the rigid box primitives used for training."],"supporting_citations":[{"why":"Supplies the global contact-rich planning framework that the demonstration planner builds on.","marker":"[3]"},{"why":"Provides the smoothed linear contact-dynamics approximation that makes planning efficient enough to synthesize 12,000 demonstrations.","marker":"[4]"},{"why":"Provides the diffusion policy architecture and action-sequence prediction that GLIDE adapts.","marker":"[9]"},{"why":"Serves as the high-fidelity simulator used both to filter demonstrations and to evaluate the learned policy.","marker":"[52]"},{"why":"Supplies the point-cloud feature extraction backbone for the visuomotor policy.","marker":"[53]"},{"why":"Motivates actuating only three joints per arm, simplifying the real-world command space.","marker":"[58]"}],"fun_headline_variants":["Planner-made demos train diffusion policy for bimanual reorientation","Diffusion policy learns from planner demos to handle new objects","Sim-only planner data teach bimanual policy to reorient novel objects","From planner data to real-world bimanual reorientation","Planner data power a diffusion policy for bimanual reorientation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The sim-to-real transfer works only if the simulator's contact, friction, and object compliance faithfully predict how the real arms and objects behave under the policy's commands.","fun_headline_variants_meta":{"raw":{"variants":["Planner-made demos train diffusion policy for bimanual reorientation","Diffusion policy learns from planner demos to handle new objects","Sim-only planner data teach bimanual policy to reorient novel objects","From planner data to real-world bimanual reorientation","Planner data power a diffusion policy for bimanual reorientation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000725,"raw_usage":{"total_tokens":3239,"prompt_tokens":920,"completion_tokens":2319,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":2229}},"tokens_in":536,"tokens_out":2319,"duration_ms":15649,"temperature":1.0,"reasoning_tokens":2229,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:11:05.130319+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Deploy the same GLIDE policy on a set of real objects with measured friction coefficients spanning a wide range outside the simulated distribution, and record success as a function of friction: if success collapses smoothly as friction deviates while simulation success stays high, the central transfer claim would be disproved.","supporting_citations":[{"cited_title":"Dexterous contact-rich manipulation via the contact trust region,","cited_arxiv_id":null,"evidence_quote":"Provides the smoothed linear contact-dynamics approximation that makes planning efficient enough to synthesize 12,000 demonstrations."},{"cited_title":"Diffusion policy: Visuomotor policy learning via action diffusion,","cited_arxiv_id":null,"evidence_quote":"Provides the diffusion policy architecture and action-sequence prediction that GLIDE adapts."},{"cited_title":"Learning goal-conditioned diffusion policy for contact-rich bimanual manipulation through planning-guided data synthesis,","cited_arxiv_id":null,"evidence_quote":"Motivates actuating only three joints per arm, simplifying the real-world command space."}],"review_version":1}