Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GLIDE trains a task-conditioned diffusion policy on planner-generated simulation demonstrations so a two-arm robot can reorient novel boxes, containers, and toys from point clouds alone.

desk verdict GLIDE shows planning-generated synthetic data can train a transferable bimanual diffusion policy, but the paper hides a likely train/test shift in its goal-conditioning input and leans on small real-world trials. read the letter →

arxiv 2412.02676 v2 pith:6F2LKU32 submitted 2024-12-03 cs.RO cs.CVcs.LG

Xuanlin Li , Tong Zhao , Xinghao Zhu , Jiuguang Wang , Tao Pang , Kuan Fang This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords bimanualmanipulationdiffusionpolicycontact-richplanning-guidedlearningsim-to-realtransferpointcloudobjectreorientationsyntheticdemonstrations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that contact-rich bimanual reorientation of bulky objects can be learned without any real-world demonstrations, by training a single task-conditioned diffusion policy on trajectories synthesized by a model-based motion planner in simulation. The authors argue that efficient planning in randomized environments provides enough high-quality data to replace teleoperation, and they introduce design choices in point-cloud processing, task encoding, and action prediction that close the sim-to-real gap. If right, a robot with two arms and a depth camera can be taught new whole-arm manipulation skills in about two days of offline planning and then handle boxes, soft containers, and inflated toys it has never seen. The measured support is real-world success of 0.80 for fixed 45-degree rotations of in-distribution boxes, 0.657 for out-of-distribution containers under the same rotation, and 0.28 for random rotations of those containers.

What carries the argument

The load-bearing mechanism is the demonstration-synthesis pipeline: a greedy contact-rich planner that alternates between collision-free approach motions and single-step contact trajectory optimization to move the object toward its goal, followed by filtering of trajectories via rollouts in a high-fidelity simulator. This turns model-based planning into a data source that is fast enough to produce 12,000 training demonstrations in about two days on a 96-CPU machine. The second half is a task-conditioned diffusion policy over point clouds and proprioception that predicts sequences of residual joint commands; the residual action parameterization is the main adaptation that keeps the actions smooth and consistent across heterogeneous demonstrations.

What would settle it

Deploy the same GLIDE policy on a set of real objects with measured friction coefficients spanning a wide range outside the simulated distribution, and record success as a function of friction: if success collapses smoothly as friction deviates while simulation success stays high, the central transfer claim would be disproved.

Watch

Extended reading notes

Core claim

GLIDE is the finding that a contact-rich motion planner can act as a scalable data engine for bimanual manipulation, and that the resulting policy can match or exceed the planner's own success rate while operating from point clouds without privileged object information. A diffusion policy is trained by behavior cloning on planner-generated demonstrations that are filtered through high-fidelity simulator rollouts, and the policy is conditioned on the relative transformation from the current object pose to a user-specified goal. The authors identify the essential design choices that make sim-to-real transfer work: predicting residual joint position actions instead of absolute actions, adding flying-point noise augmentation, and computing task specifications online from tracked keypoints. With these choices, the policy reorients objects whose geometry and physical properties are outside the training distribution of box primitives, including soft rubber and fabric containers and inflatable toys.

Load-bearing premise

The sim-to-real transfer works only if the simulator's contact, friction, and object compliance faithfully predict how the real arms and objects behave under the policy's commands.

Editorial extensions

If this is right

  • A single policy, not one model per task, can handle arbitrary target object poses because the task is encoded as a delta transformation recomputed at each time step.
  • Because training data come from a planner, scaling data generation is a matter of CPU time rather than human teleoperation, and the reported scaling curve does not plateau at 12,000 demonstrations.
  • Real-world deployment works without knowing object shape or pose in advance, using only processed point clouds and tracked keypoints to infer the required manipulation.
  • Out-of-distribution containers and inflatable toys are handled at lower but non-trivial success rates, showing that the policy transfers beyond the rigid box primitives used for training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same residual-action and flying-point-augmentation recipes could transfer to other contact-rich sim-to-real policies beyond diffusion policies, and they are cheap to test on existing benchmarks.
  • The drop to 0.28 success on random rotations of out-of-distribution containers suggests a ceiling tied to simulator fidelity for deformable objects; adding deformable bodies to the training distribution is a natural next step.
  • Because failures concentrate in poor joint configurations and torque-limit squeezing, collecting planner demonstrations that start from near-failure states could improve the hard-rotation regime more than simply adding more nominal trajectories.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes GLIDE, a planning-guided diffusion policy approach for contact-rich bimanual object reorientation. Synthetic demonstration trajectories are generated in the DRAKE simulator using an efficient contact-implicit planner, filtered by high-fidelity rollouts, and used to train a single task-conditioned point-cloud diffusion policy via behavior cloning. The method introduces several sim-to-real design choices, including residual joint action prediction, flying-point augmentation, and a goal transformation computed from keypoint tracking. The paper evaluates GLIDE on in-distribution boxes in simulation and the real world, and on out-of-distribution containers and inflatable toys in the real world. Results show strong in-distribution success (e.g., 0.80 on fixed 45-degree rotation) and lower but positive OOD success (0.657 on fixed rotation, 0.282 on random rotation for containers).

Significance. If the central claim holds, GLIDE would be a valuable contribution: it avoids expensive real-world teleoperation, trains a single policy for user-specified target poses, and shows some real-world generalization to objects with geometries and physical properties not seen in training. The paper's strengths include a concrete planning-to-learning pipeline, real hardware deployment, ablations of the proposed design choices, and evaluation on genuinely challenging OOD objects such as soft containers and inflatable toys. However, the significance is tempered by two gaps: the task-conditioning input c_t may be computed from ground-truth poses in simulation but from noisy keypoint tracking at deployment, and the real-world evidence consists of small trial counts without uncertainty quantification. These gaps directly affect the paper's headline claim of generalizable manipulation across physical properties.

major comments (3)
  1. [III-C] The paper does not state how the task specification c_t is computed for the 12,000 synthetic demonstrations. Section III-C describes segmenting the target object in o0, selecting keypoints via farthest point sampling, and tracking them with TAPIR/CoTracker, but if this perception pipeline is used only at deployment while simulation training uses ground-truth DRAKE poses, then the policy is trained with exact task specifications and evaluated with noisy ones. Since an incorrect c_t guarantees failure regardless of manipulation skill, the out-of-distribution results in Table II may reflect tracking quality on deformable objects rather than generalization of contact-rich manipulation. Please state explicitly whether c_t in training is derived from ground-truth object poses or from the perception pipeline, and provide either per-condition tracking error statistics or an ablation that injects c_t noise at evaluation.
  2. [IV-C and IV-D, Tables I-II] Real-world success rates are reported from 25 trials per in-distribution condition and 16 trials per OOD condition, without confidence intervals or exact trial counts per cell. For example, the OOD fixed-rotation rates 0.688 (empty containers) and 0.625 (overfilled containers) differ by one trial out of 16, so the statement that performance is "similar across both empty and overfilled containers" is not quantitatively supported. Please report exact numerators and denominators and binomial confidence intervals for all real-world conditions, especially where claims of robustness depend on small differences.
  3. [IV-A and IV-D] The success criterion "final object pose within 10 cm and 0.2 rad" presupposes a rigid object pose, but the OOD evaluation includes fabric and rubber containers as well as inflatable toys whose geometry deforms during manipulation. The manuscript does not specify how the final pose is measured for these non-rigid objects, nor how tracked keypoints are converted to an SE(2) pose. Without a stated measurement protocol, the OOD results are not reproducible and remain entangled with the tracking-based c_t computation discussed above.
minor comments (6)
  1. [III-B] The description of demonstration filtering is underspecified: the manuscript says trajectories that "don't reach the goal" and "take too long" are discarded, but no quantitative thresholds, timeout horizon, or filter rollout success criterion are given. This is needed to reproduce the 12,000-demonstration dataset.
  2. [III-C] The notation for residual actions, a_{t+1:t+Ta} = {q_i - q_t}, is unclear because the index i is not defined. Please clarify whether q_i denotes a planned future joint position and q_t the current measured joint position.
  3. [III-C] The sentence "We use a larger Ta = 20 at test time and train our policy with Ta = 64" is confusing alongside Table IV, which ablates Ta at evaluation time. Please specify that Ta is the action-chunk length used at inference and explain how a policy trained with chunk length 64 can be evaluated with shorter chunks.
  4. [Tables I and III] The phrase "vanilla DP3 baseline" in the Table I caption is not a defined baseline. The last row of Table III and the first column of Table IV combine several ablations and do not clearly correspond to the DP3 method of [9] or [53]. Please add an explicit description of the baseline configuration.
  5. [IV-E] The claim that performance "doesn't plateau as we scale up the number of demonstrations" is based on four points without error bars. Please soften this claim or provide repeated-seed variance.
  6. [III-B] There is a typo in the first paragraph of the demonstration synthesis section: "Compared with with the sampling-based planner" should read "Compared with the sampling-based planner."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central claims are supported by external simulation and real-world evaluations.

full rationale

GLIDE's derivation chain is empirical rather than analytic: a model-based contact planner synthesizes demonstrations in Drake, trajectories are filtered by high-fidelity rollouts, a diffusion policy is trained by behavior cloning, and the policy is evaluated on held-out simulation objects and on real-world in-distribution and out-of-distribution objects. None of these steps is equivalent to its input by construction. The planner is reported as a reference baseline, and the learned policy outperforms the planner (sim fixed-rotation success 0.740 vs. 0.337), so the policy is not a renamed planner output. The task-conditioning input c_t is computed from observations via segmentation and tracking, and any train/test mismatch in that pipeline would be a robustness or evaluation concern, not circularity. Self-citations such as [3], [4], and [58] are prior planning tools and design precedents; they are not invoked as the proof of the generalization claim, which rests on external sim and real-world results. Ablations of action representation, augmentation, prediction horizon, and dataset size provide independent evidence that the reported behaviors depend on the proposed components. No fitted parameter is relabeled as a prediction, and no uniqueness theorem or definitional equivalence is used to force the result. The paper is therefore self-contained against external benchmarks, and no circular step is identifiable.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method is empirical and does not derive a closed-form result. The central claim rests on hand-chosen hyperparameters (planner cost matrices, augmentation probability, action horizon, success threshold, filtering thresholds) and domain assumptions inherited from simulation and prior planning work. No new physical entities are introduced.

free parameters (5)
  • Planner cost matrices Q and R
    User-specified weights in the contact planner objective (Sec III-B) that shape trajectory optimality and hence demonstration quality; values are not reported.
  • Flying point augmentation probability = 0.5%
    Probability of adding large Gaussian noise to point clouds during training (Sec III-C), chosen by hand to improve real-world robustness.
  • Action prediction horizon at test time = Ta=20 (training Ta=64)
    Number of future joint commands the policy predicts; the ablation in Table IV shows Ta=20 is best at test time, yet training uses Ta=64, an unexplained mismatch.
  • Success threshold = 10 cm, 0.2 rad
    Episode success is defined as final object pose within 10 cm and 0.2 rad of the goal (Sec IV-A); this task definition sets the measured success rates.
  • Demonstration filtering thresholds
    Trajectories where the object does not reach the goal or takes too long are discarded (Sec III-B); the specific time threshold is not reported.
assumptions (4)
  • domain assumption Smoothed contact dynamics from [3,4] accurately approximate contact-rich behavior in the planner
    Demonstration synthesis relies on the quasi-dynamic contact model and contact trust region solver from prior work; if these models fail, generated trajectories are poor.
  • domain assumption The DRAKE simulator is a high-fidelity proxy for the real world for filtering and policy evaluation
    Trajectories are filtered via rollouts in DRAKE and sim success is used as a proxy; the real world may differ in friction, deformation, and sensing.
  • domain assumption Open-vocabulary segmentation and keypoint tracking reliably estimate the object's delta pose without known geometry
    Task conditioning uses segmentation and 3D tracking (Sec III-C); failures here would corrupt the goal input to the policy.
  • domain assumption Behavior cloning from filtered expert trajectories yields a policy that generalizes beyond the training distribution
    The whole approach postulates that imitation of planner demos generalizes to unseen objects; the OOD results provide partial evidence but not a guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation." pith.science (2026). https://pith.science/paper/6F2LKU32

@misc{pith2026241202676,
  author       = {Pith},
  title        = {Pith review of: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6F2LKU32}},
  note         = {Machine review of arXiv:2412.02676}
}
read the original abstract

Contact-rich bimanual manipulation involves precise coordination of two arms to change object states through strategically selected contacts and motions. Due to the inherent complexity of these tasks, acquiring sufficient demonstration data and training policies that generalize to unseen scenarios remain a largely unresolved challenge. Building on recent advances in planning through contacts, we introduce Generalizable Planning-Guided Diffusion Policy Learning (GLIDE), an approach that effectively learns to solve contact-rich bimanual manipulation tasks by leveraging model-based motion planners to generate demonstration data in high-fidelity physics simulation. Through efficient planning in randomized environments, our approach generates large-scale and high-quality synthetic motion trajectories for tasks involving diverse objects and transformations. We then train a task-conditioned diffusion policy via behavior cloning using these demonstrations. To tackle the sim-to-real gap, we propose a set of essential design options in feature extraction, task representation, action prediction, and data augmentation that enable learning robust prediction of smooth action sequences and generalization to unseen scenarios. Through experiments in both simulation and the real world, we demonstrate that our approach can enable a bimanual robotic system to effectively manipulate objects of diverse geometries, dimensions, and physical properties. Website: https://glide-manip.github.io/

Figures

Figures reproduced from arXiv: 2412.02676 by the authors.

Figure 1
Figure 1. Contact-rich bimanual manipulation of objects with diverse geometries and physical properties performed by our method. Each [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Generalizable PLanning-GuIded Diffusion Policy LEarning (GLIDE) for contact-rich bimanual object manipulation. Left: Scalable synthetic demonstration generation using efficient motion planning in physical simulator (Sec. III-B). Right: Our GLIDE policy is a task-conditioned point cloud diffusion policy trained through behavior cloning. We further introduce essential design options that significantly enha… view at source ↗
Figure 3
Figure 3. Objects used for real-world evaluation: (Left) Boxes whose dimensions and physical properties are within our policy’s training distribution; (Right) Out-of-distribution (OOD) containers, some of which are made with soft materials like rubber (top right, bottom left) and fabric (bottom right). For half of OOD evaluations, we evaluate our policy using empty containers; for the other half, we evaluate using containers … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of our bimanual box reorientation environment [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Examples of policy evaluation on in-distribution (left, [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Visualization of policy evaluation under a challenging OOD [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations

    cs.RO 2025-08 conditional novelty 6.0 of 10

    LodeStar combines automatic skill segmentation with simulation-based reinforcement learning augmentation and a learned routing transformer to let a robotic hand complete long-horizon dexterous tasks from a few human demos.

  2. Diffusion-Based Imaginative Coordination for Bimanual Manipulation

    cs.RO 2025-07 conditional novelty 6.0 of 10

    A diffusion-based policy that jointly predicts future video latents and actions improves bimanual manipulation success, with video prediction used only during training.

Reference graph

Works this paper leans on

58 extracted references · 31 canonical work pages · cited by 2 Pith papers

  1. [1]

    Synthesis and stabilization of complex behaviors through online trajectory optimization,

    Y . Tassa, T. Erez, and E. Todorov, “Synthesis and stabilization of complex behaviors through online trajectory optimization,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 4906–4913

    work page 2012

  2. [2]

    A direct method for trajectory op- timization of rigid bodies through contact,

    M. Posa, C. Cantu, and R. Tedrake, “A direct method for trajectory op- timization of rigid bodies through contact,” The International Journal of Robotics Research , vol. 33, no. 1, pp. 69–81, 2014

    work page 2014

  3. [3]

    Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models,

    T. Pang, H. T. Suh, L. Yang, and R. Tedrake, “Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models,” IEEE Transactions on Robotics , 2023

    2023

  4. [4]

    Dexterous contact-rich manipulation via the contact trust region,

    H. T. Suh, T. Pang, T. Zhao, and R. Tedrake, “Dexterous contact-rich manipulation via the contact trust region,” Preprint, 2025

    work page 2025

  5. [5]

    Learning dexterous in-hand manipulation,

    O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. Mc- Grew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray et al., “Learning dexterous in-hand manipulation,” The International Journal of Robotics Research , vol. 39, no. 1, pp. 3–20, 2020

    2020

  6. [6]

    Visual dexterity: In-hand reorientation of novel and complex object shapes,

    T. Chen, M. Tippur, S. Wu, V . Kumar, E. Adelson, and P. Agrawal, “Visual dexterity: In-hand reorientation of novel and complex object shapes,” Science Robotics, vol. 8, no. 84, p. eadc9244, 2023

    2023

  7. [7]

    Dextreme: Transfer of agile in-hand manipulation from simulation to reality,

    A. Handa, A. Allshire, V . Makoviychuk, A. Petrenko, R. Singh, J. Liu, D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingamet al., “Dextreme: Transfer of agile in-hand manipulation from simulation to reality,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 5977–5984

    2023

  8. [8]

    Dexpoint: Gener- alizable point cloud reinforcement learning for sim-to-real dexterous manipulation,

    Y . Qin, B. Huang, Z.-H. Yin, H. Su, and X. Wang, “Dexpoint: Gener- alizable point cloud reinforcement learning for sim-to-real dexterous manipulation,” in Conference on Robot Learning . PMLR, 2023, pp. 594–605

    work page 2023

Show all 58 references
  1. [9]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” The International Journal of Robotics Research , 2023

    2023

  2. [10]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” in Robotics: Science and Systems, 2023

    2023

  3. [11]

    Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,

    Z. Fu, T. Z. Zhao, and C. Finn, “Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,” arXiv preprint arXiv:2401.02117, 2024

    2024 arXiv

  4. [12]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots,

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song, “Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots,”arXiv preprint arXiv:2402.10329, 2024

    2024 arXiv

  5. [13]

    General in-hand object rotation with vision and touch,

    H. Qi, B. Yi, S. Suresh, M. Lambeta, Y . Ma, R. Calandra, and J. Malik, “General in-hand object rotation with vision and touch,” in Conference on Robot Learning . PMLR, 2023, pp. 2549–2564

    2023

  6. [14]

    Approximate hybrid model predictive control for multi-contact push recovery in complex environments,

    T. Marcucci, R. Deits, M. Gabiccini, A. Bicchi, and R. Tedrake, “Approximate hybrid model predictive control for multi-contact push recovery in complex environments,” in 2017 IEEE-RAS 17th interna- tional conference on humanoid robotics (Humanoids) . IEEE, 2017, pp. 31–38

    2017

  7. [15]

    Feedback control of the pusher-slider system: A story of hybrid and underactuated contact dynamics,

    F. R. Hogan and A. Rodriguez, “Feedback control of the pusher-slider system: A story of hybrid and underactuated contact dynamics,” in Algorithmic Foundations of Robotics XII: Proceedings of the Twelfth Workshop on the Algorithmic Foundations of Robotics . Springer, 2020, pp. 800–815

    2020

  8. [16]

    Consensus complementarity control for multicontact mpc,

    A. Aydinoglu, A. Wei, W.-C. Huang, and M. Posa, “Consensus complementarity control for multicontact mpc,” IEEE Transactions on Robotics, vol. 40, pp. 3879–3896, 2024

    2024

  9. [17]

    Dojo: A differentiable physics engine for robotics,

    T. A. Howell, S. L. Cleac’h, J. Br ¨udigam, J. Z. Kolter, M. Schwa- ger, and Z. Manchester, “Dojo: A differentiable physics engine for robotics,” arXiv preprint arXiv:2203.00806 , 2022

    2022 arXiv

  10. [18]

    Bertsekas, Reinforcement learning and optimal control

    D. Bertsekas, Reinforcement learning and optimal control . Athena Scientific, 2019, vol. 1

    2019

  11. [19]

    Stabilize to act: Learning to coordinate for bimanual manipulation,

    J. Grannen, Y . Wu, B. Vu, and D. Sadigh, “Stabilize to act: Learning to coordinate for bimanual manipulation,” in 7th Annual Conference on Robot Learning , 2023

    2023

  12. [20]

    Bi-touch: Bimanual tactile manipulation with sim-to-real deep reinforcement learning,

    Y . Lin, A. Church, M. Yang, H. Li, J. Lloyd, D. Zhang, and N. F. Lepora, “Bi-touch: Bimanual tactile manipulation with sim-to-real deep reinforcement learning,” IEEE Robotics and Automation Letters , vol. 8, no. 9, pp. 5472–5479, 2023

    2023

  13. [21]

    Roboturk: A crowdsourcing platform for robotic skill learning through imitation,

    A. Mandlekar, Y . Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay et al. , “Roboturk: A crowdsourcing platform for robotic skill learning through imitation,” in Conference on Robot Learning . PMLR, 2018, pp. 879–893

    2018

  14. [22]

    Scaling robot supervision to hundreds of hours with RoboTurk: Robotic manipulation dataset through human reasoning and dexterity,

    A. Mandlekar, J. Booher, M. Spero, A. Tung, A. Gupta, Y . Zhu, A. Garg, S. Savarese, and L. Fei-Fei, “Scaling robot supervision to hundreds of hours with RoboTurk: Robotic manipulation dataset through human reasoning and dexterity,” in 2019 IEEE/RSJ Interna- tional Conference ...

    2019

  15. [23]

    Acronym: A large-scale grasp dataset based on simulation,

    C. Eppner, A. Mousavian, and D. Fox, “Acronym: A large-scale grasp dataset based on simulation,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 6222–6227

    2021

  16. [24]

    Bridge data: Boosting gen- eralization of robotic skills with cross-domain datasets,

    F. Ebert, Y . Yang, K. Schmeckpeper, B. Bucher, G. Georgakis, K. Daniilidis, C. Finn, and S. Levine, “Bridge data: Boosting gen- eralization of robotic skills with cross-domain datasets,” in Robotics: Science and Systems (RSS) XVIII , 2022

    2022

  17. [25]

    Learn to grasp with less supervision: A data-efficient maximum likelihood grasp sampling loss,

    X. Zhu, Y . Zhou, Y . Fan, L. Sun, J. Chen, and M. Tomizuka, “Learn to grasp with less supervision: A data-efficient maximum likelihood grasp sampling loss,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 721–727

    2022

  18. [26]

    Frame mining: a free lunch for learning robotic manipulation from 3d point clouds,

    M. Liu, X. Li, Z. Ling, Y . Li, and H. Su, “Frame mining: a free lunch for learning robotic manipulation from 3d point clouds,” in 6th Annual Conference on Robot Learning , 2022. [Online]. Available: https://openreview.net/forum?id=d-JYso87y6s

    2022

  19. [27]

    Learning insertion primitives with discrete-continuous hybrid action space for robotic assembly tasks,

    X. Zhang, S. Jin, C. Wang, X. Zhu, and M. Tomizuka, “Learning insertion primitives with discrete-continuous hybrid action space for robotic assembly tasks,” in 2022 International conference on robotics and automation (ICRA) . IEEE, 2022, pp. 9881–9887

    2022

  20. [28]

    ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Clouds,

    D. Seita, Y . Wang, S. Shetty, E. Li, Z. Erickson, and D. Held, “ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Clouds,” in Conference on Robot Learning (CoRL) , 2022

    2022

  21. [29]

    Is conditional generative modeling all you need for decision- making?

    A. Ajay, Y . Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal, “Is conditional generative modeling all you need for decision- making?” arXiv preprint arXiv:2211.15657 , 2022

    2022 arXiv

  22. [30]

    Decision transformer: Reinforcement learning via sequence modeling,

    L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch, “Decision transformer: Reinforcement learning via sequence modeling,” Advances in neural information processing systems , vol. 34, pp. 15 084–15 097, 2021

    2021

  23. [31]

    Plan- ning with diffusion for flexible behavior synthesis,

    M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine, “Plan- ning with diffusion for flexible behavior synthesis,” arXiv preprint arXiv:2205.09991, 2022

    2022 arXiv

  24. [32]

    Idql: Implicit q-learning as an actor-critic method with diffusion policies,

    P. Hansen-Estruch, I. Kostrikov, M. Janner, J. G. Kuba, and S. Levine, “Idql: Implicit q-learning as an actor-critic method with diffusion policies,” 2023. [Online]. Available: https://arxiv.org/abs/2304.10573

    2023 arXiv

  25. [33]

    Goal-conditioned imitation learning using score-based diffusion policies,

    M. Reuss, M. Li, X. Jia, and R. Lioutikov, “Goal-conditioned imitation learning using score-based diffusion policies,” 2023. [Online]. Available: https://arxiv.org/abs/2304.02532

    2023 arXiv

  26. [34]

    Imitating human behaviour with diffusion models,

    T. Pearce, T. Rashid, A. Kanervisto, D. Bignell, M. Sun, R. Georgescu, S. V . Macua, S. Z. Tan, I. Momennejad, K. Hofmann, and S. Devlin, “Imitating human behaviour with diffusion models,” 2023. [Online]. Available: https://arxiv.org/abs/2301.10677

    2023 arXiv

  27. [35]

    Motion policy networks,

    A. Fishman, A. Murali, C. Eppner, B. Peele, B. Boots, and D. Fox, “Motion policy networks,” in Conference on Robot Learning. PMLR, 2023, pp. 967–977

    2023

  28. [36]

    Neural mp: A generalist neural motion planner,

    M. Dalal, J. Yang, R. Mendonca, Y . Khaky, R. Salakhutdinov, and D. Pathak, “Neural mp: A generalist neural motion planner,” arXiv preprint arXiv:2409.05864, 2024

    2024 arXiv

  29. [37]

    Imitating Task and Motion Planning with Visuomotor Transformers,

    M. Dalal, A. Mandlekar, C. Garrett, A. Handa, R. Salakhutdinov, and D. Fox, “Imitating Task and Motion Planning with Visuomotor Transformers,” in Conference on Robot Learning , 2023

    2023

  30. [38]

    Pixels to plans: Learning non-prehensile manipulation by imitating a planner,

    T. Tosun, E. Mitchell, B. Eisner, J. Huh, B. Lee, D. Lee, V . Isler, H. S. Seung, and D. Lee, “Pixels to plans: Learning non-prehensile manipulation by imitating a planner,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2019, pp. 7431–7438

    2019

  31. [39]

    Motion planner augmented reinforcement learning for robot manipulation in obstructed environments,

    J. Yamada, Y . Lee, G. Salhotra, K. Pertsch, M. Pflueger, G. Sukhatme, J. Lim, and P. Englert, “Motion planner augmented reinforcement learning for robot manipulation in obstructed environments,” in Con- ference on Robot Learning . PMLR, 2021, pp. 589–603

    2021

  32. [40]

    Mpc-net: A first principles guided policy search,

    J. Carius, F. Farshidian, and M. Hutter, “Mpc-net: A first principles guided policy search,” IEEE Robotics and Automation Letters , vol. 5, no. 2, pp. 2897–2904, 2020

    2020

  33. [41]

    Bootstrap- ping motor skill learning with motion planning,

    B. Abbatematteo, E. Rosen, S. Tellex, and G. Konidaris, “Bootstrap- ping motor skill learning with motion planning,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2021, pp. 4926–4933

    2021

  34. [42]

    Agile autonomous driving using end-to-end deep imitation learning,

    Y . Pan, C.-A. Cheng, K. Saigol, K. Lee, X. Yan, E. A. Theodorou, and B. Boots, “Agile autonomous driving using end-to-end deep imitation learning,” Robotics: Science and Systems XIV , 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:53873353

    2017

  35. [43]

    Harnessing reinforcement learning for neural motion planning,

    T. Jurgenson and A. Tamar, “Harnessing reinforcement learning for neural motion planning,” in Robotics: Science and Systems , 2019

    2019

  36. [44]

    Learning a decentralized multi-arm motion planner,

    H. Ha, J. Xu, and S. Song, “Learning a decentralized multi-arm motion planner,” in Conference on Robot Learning , 2021

    2021

  37. [45]

    Guided policy search,

    S. Levine and V . Koltun, “Guided policy search,” in International conference on machine learning . PMLR, 2013, pp. 1–9

    2013

  38. [46]

    Plan-guided rein- forcement learning for whole-body manipulation,

    M. Zhang, J. Barreiros, and A. O. Onol, “Plan-guided rein- forcement learning for whole-body manipulation,” arXiv preprint arXiv:2310.12263, 2023

    2023 arXiv

  39. [47]

    Dtc: Deep tracking control,

    F. Jenelten, J. He, F. Farshidian, and M. Hutter, “Dtc: Deep tracking control,” Science Robotics, vol. 9, no. 86, p. eadh5401, 2024

    2024

  40. [48]

    Jacta: A versatile planner for learning dexterous and whole-body manipulation,

    J. Br ¨udigam, A.-A. Abbas, M. Sorokin, K. Fang, B. Hung, M. Guru, S. Sosnowski, J. Wang, S. Hirche, and S. L. Cleac’h, “Jacta: A versatile planner for learning dexterous and whole-body manipulation,” in Conference on Robot Learning , 2024

    2024

  41. [49]

    S. M. LaValle, Planning algorithms . Cambridge university press, 2006

    2006

  42. [50]

    Fast contact-implicit model predic- tive control,

    S. Le Cleac’h, T. A. Howell, S. Yang, C.-Y . Lee, J. Zhang, A. Bishop, M. Schwager, and Z. Manchester, “Fast contact-implicit model predic- tive control,” IEEE Transactions on Robotics, vol. 40, pp. 1617–1629, 2024

    2024

  43. [51]

    Bundled gradients through contact via randomized smoothing,

    H. J. T. Suh, T. Pang, and R. Tedrake, “Bundled gradients through contact via randomized smoothing,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4000–4007, 2022

    2022

  44. [52]

    Drake: Model-based design and verification for robotics,

    R. Tedrake and the Drake Development Team, “Drake: Model-based design and verification for robotics,” 2019. [Online]. Available: https://drake.mit.edu

    2019

  45. [53]

    3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,” in Proceedings of Robotics: Science and Systems (RSS), 2024

    2024

  46. [54]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu et al., “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” European Conference on Computer Vision, 2024

    2024

  47. [55]

    Efficientvit: Lightweight multi-scale attention for high-resolution dense prediction,

    H. Cai, J. Li, M. Hu, C. Gan, and S. Han, “Efficientvit: Lightweight multi-scale attention for high-resolution dense prediction,” in Proceed- ings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 17 302–17 313

    2023

  48. [56]

    Tapir: Tracking any point with per- frame initialization and temporal refinement,

    C. Doersch, Y . Yang, M. Vecerik, D. Gokay, A. Gupta, Y . Aytar, J. Carreira, and A. Zisserman, “Tapir: Tracking any point with per- frame initialization and temporal refinement,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 10 061–10 072

    2023

  49. [57]

    Cotracker: It is better to track together,

    N. Karaev, I. Rocco, B. Graham, N. Neverova, A. Vedaldi, and C. Rup- precht, “Cotracker: It is better to track together,” arXiv:2307.07635, 2023

    2023 arXiv

  50. [58]

    Learning goal-conditioned diffusion policy for contact-rich bimanual manipulation through planning-guided data synthesis,

    H. Zhu, T. Pang, X. Ni, T. Zhao, K. Fang, L. Righetti, and J. Wang, “Learning goal-conditioned diffusion policy for contact-rich bimanual manipulation through planning-guided data synthesis,” in ICRA 2024 Workshop on Future Roadmap for Manipulation Skills , 2023

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.