Pith. sign in

REVIEW 3 major objections 5 minor 5 cited by

WildLMa: Long Horizon Loco-Manipulation in the Wild

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Quadruped robot cleans trash, grasps unseen objects from few demos

desk verdict Solid real-robot systems paper for legged loco-manipulation, but the language-conditioning mechanism takes more credit for OOD generalization than the ablations support. read the letter →

arxiv 2411.15131 v2 pith:WOXRKODQ submitted 2024-11-22 cs.RO cs.CVcs.LG

Ri-Zhao Qiu , Yuchen Song , Xuanbin Peng , Sai Aneesh Suryadevara , Ge Yang , Minghuan Liu , Mazeyu Ji , Chengzhe Jia
show 3 more authors
Ruihan Yang Xueyan Zou Xiaolong Wang
This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords loco-manipulationquadrupedrobotimitationlearninglanguage-conditionedpolicyCLIPwhole-bodycontrollong-horizontaskplanningout-of-distributiongeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

WildLMa claims that a quadruped robot with a manipulator can learn reusable manipulation skills from as few as tens of VR-collected demonstrations and then compose those skills with an LLM planner to execute long-horizon mobile manipulation tasks. The key step is making imitation learning language-conditioned: a frozen CLIP encoder is reparameterized with MaskCLIP to yield per-pixel image-text similarity maps, and these maps are fed through cross-attention into an Action Chunking Transformer policy. In real-world tests, this improves average skill success to 71.2% with out-of-distribution objects, compared with 40.8% for a vanilla ACT baseline, and the whole system completes a collect-and-drop-trash task in 7 of 10 runs. The paper also shows that combining a learned whole-body controller with VR teleoperation cuts demonstration collection time by 26.9% and enables non-prehensile skills such as button pressing.

What carries the argument

The load-bearing component is the MaskCLIP reparameterization of a frozen CLIP ViT-B/16 encoder, which converts the CLIP image encoder into a dense feature map aligned with the shared image-text embedding space. Task-specific text embeddings are compared with this map via cosine similarity (Eq. 2) to produce a pixel-wise probability map of the queried object, which is used as an auxiliary cross-attention input to the policy. The second essential component is the learned low-level whole-body controller, which translates operator wrist and base commands into coordinated arm and base joint actions, making VR teleoperation on a quadruped practical and reducing demonstration cost by 26.9%.

What would settle it

Evaluate the tabletop-grasping skill on a held-out set of 30 unseen household objects under the same OOD protocol and record both the per-object success rate and the location of the cross-attention peak relative to the object's true image position; if the success rate falls more than 10 points below the reported 75%, or if the peak is misplaced on more than a fifth of trials, the central claim is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that dense, text-queryable features from a frozen CLIP encoder, extracted via the MaskCLIP reparameterization, provide exactly the localization signal that behavior-cloned visuomotor policies need to generalize outside their training distribution. By computing cosine-similarity cross-attention between the CLIP feature map and a task-specific text embedding, the policy receives a probability map of the queried object as an auxiliary input; applying dropout to this cross-attention prevents the policy from depending too heavily on it. The result is that a single ACT architecture conditioned this way outperforms the same architecture with a ResNet or DINOv2 backbone on out-of-distribution tabletop grasping, and it also enables automatic episode termination so skills can be chained by a planner. Combining this skill library with a hierarchical LLM planner over a scene graph lets the robot follow high-level instructions like 'clean the trash in the hallway' end to end.

Load-bearing premise

The out-of-distribution generalization claim rests on the assumption that the MaskCLIP similarity maps from a frozen CLIP encoder remain well aligned with the robot's camera views under novel lighting, textures, and object placements; if these maps fail to localize the queried object, the language-conditioned policy loses its grounding and the reported OOD success rates are not attainable.

Editorial extensions

If this is right

  • If correct, imitation-learned skill libraries for legged manipulators can be built from tens of human demonstrations per skill rather than thousands, and the skills will transfer to unseen object instances.
  • Language-conditioning via cross-attention means a single policy can be repurposed by changing the text query, so the same ACT backbone can serve different tasks without retraining the visual encoder.
  • The whole-body controller makes VR teleoperation of a quadruped practical enough that non-prehensile skills, such as button pressing and shelf rearrangement, become collectable, expanding the task repertoire beyond pick-and-place.
  • The hierarchical LLM planner with scene graphs turns high-level instructions into executable skill sequences, so long-horizon tasks can be addressed by composing atomic skills instead of learning end-to-end policies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The success of dense CLIP grounding suggests that other dense feature extractors (for instance DINOv2 or SAM-based features) might be directly comparable, and a systematic comparison across backbones and text prompting strategies could reveal whether CLIP's language alignment or its dense localization is the main driver of the out-of-distribution gains.
  • The paper's OOD protocol permutes object placement, texture, and background but keeps camera viewpoints plausible; the method's true boundary would show up under heavy occlusion or extreme lighting, where the cross-attention maps are likely to degrade, so adversarial clutter is a natural next stress test.
  • Manual scene-graph annotation is an unstated cost in the planner; the paper notes that off-the-shelf construction methods could replace it, so measuring how planner performance degrades with annotated-graph noise would quantify this limitation.
  • The 7/10 long-horizon success rate is reported in a single indoor hallway; a harder stress test would be outdoor trash pickup with dynamic pedestrians, which the paper only shows qualitatively.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. WildLMa proposes a framework for legged loco-manipulation that combines whole-body VR teleoperation, an imitation-learned skill library built on CLIP/MaskCLIP cross-attention with autonomous termination, and an LLM-based hierarchical planner for long-horizon tasks. Real-robot experiments on a Unitree B1/Z1 platform report success rates for tabletop grasping, ground grasping, button pressing, shelf rearrangement, and two long-horizon tasks, and the paper claims that WildLMa-Skill generalizes to out-of-distribution objects with only tens of demonstrations, outperforming ACT, OpenTV, VBC, and GeFF.

Significance. If the claims hold, the paper provides a practically valuable recipe: a modest number of VR demonstrations, a frozen CLIP backbone with text-conditioned cross-attention, and a whole-body controller can yield reusable skills that generalize to unseen objects and environments, plus an LLM planner can compose those skills for long-horizon mobile manipulation. The strengths are the real-robot deployment, the breadth of evaluated skills, the comparison with multiple recent baselines, and the teleoperation efficiency study. However, the statistical evidence is incomplete (no trial counts or confidence intervals) and the ablation design does not establish that the language-conditioned cross-attention, rather than the frozen pretrained visual backbone, is responsible for the reported OOD generalization.

major comments (3)
  1. [Sec. IV-A, Tables I, III, VI] The central mechanism claim—that CLIP-based language-conditioned cross-attention drives OOD generalization—is not supported by the ablation design. Table III shows that a frozen DinoV2 visual encoder without any language conditioning reaches 88.9% I.D. and 77.8% OOD tabletop grasping, which matches the OpenTV row in Table I and slightly exceeds WildLMa's 75% OOD in the same task. Table VI only compares CLIP features with and without cross-attention; it never varies the backbone while holding the attention mechanism fixed, nor tests a strong non-language backbone with the same policy head and training protocol. The data are therefore equally consistent with the hypothesis that frozen pretrained visual features, not the text-conditioned cross-attention map of Eq. (2), drive OOD transfer. The authors should add an ablation of a strong non-language backbone (e.g., DinoV2) under the identical ACT/termination protocol, and they should temper the abstract's causal language unless the language conditioning yields a clear gain over that backbone.
  2. [Tables I, II, III, IV, V, VI] No trial counts, raw numerators, or confidence intervals are reported for any success rate. Several headline comparisons are small differences that can easily fall within binomial noise, such as WildLMa 94.4% vs OpenTV 88.9% on I.D. tabletop grasping, WildLMa 75% vs DinoV2/OpenTV 77.8% on OOD tabletop grasping, and 80% vs 75% on I.D. button pressing. With per-cell counts of 18–36 trials, these differences are not significant, and even the cross-attention ablation (75% vs 69.4% OOD in Table VI) may be within noise. Table II's 7/10 vs 0/10 is more robust, but 3/10 for shelf rearrangement has a wide interval. Please report per-cell trial counts and exact binomial confidence intervals (or raw counts) for every percentage, and restrict comparative statements to differences that survive this uncertainty.
  3. [Table I and Sec. IV-A, A2] The comparison with RL and zero-shot baselines is partial: VBC is not evaluated on button pressing (NA), its starred entries are averaged over I.D. and O.O.D. object sets, and GeFF is reported only for tabletop grasping. The 'Avg. Succ.' column mixes different task sets across methods—WildLMa and ACT/OpenTV average over three skills, VBC over two, and GeFF over one—so the statement that WildLMa 'outperforms' VBC and GeFF is not a like-for-like comparison. Please either evaluate VBC and GeFF on all tasks where they are applicable, or present separate per-task comparisons and explicitly state the task coverage of each baseline.
minor comments (5)
  1. [Sec. III-A, Eq. (1)] The notation for the right-hand pose T_right is not fully defined as an SE(3) matrix, and the choice of translation scale sc is reported as a constant without stating how it was selected; please clarify the notation and whether sc is a tuned free parameter.
  2. [Sec. IV-A, A3 vs Sec. IV Implementation Details] A3 states that the long-horizon experiments include 20 training sequences, whereas the Implementation Details state 30–60 demonstrations per skill; these numbers should be reconciled or clarified.
  3. [Table IV] The 'W/o Whole-body (Arm Only)' condition has no reported average time for the shelf rearrangement task; please either provide the missing value or mark it explicitly as not applicable.
  4. [Sec. I and Table IV] The abstract's '26.9% demonstration cost reduction' appears to follow from averaging completion times in Table IV across ground and shelf tasks, but the calculation is not shown; please state the exact computation.
  5. [Sec. III-B] The dropout applied to cross-attention and the hyperparameters for autonomous termination (n = 10, tau = 0.8, deadzone xth = 5 cm) are not ablated; a brief robustness statement would help readers understand their sensitivity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central skill and long-horizon claims are tested against held-out real-world objects and external baselines, so they are not equivalent by construction to the method's inputs.

full rationale

Walking the claimed derivation chain, I find no prediction that reduces by construction to its inputs. The core skill results (Tables I, III, VI) are measured success rates on held-out real-world object sets and environments; the OOD numbers are externally falsifiable and are not defined in terms of WildLMa's own outputs. Equation (2) is a standard cosine-similarity map between a frozen CLIP feature map and a text embedding; the paper's claim that cross-attention improves OOD performance is an empirical ablation that could have failed, and in fact DinoV2 without language conditioning reaches 77.8% OOD in Table III. That is a real causal-attribution gap -- the Table VI ablation holds the backbone fixed at CLIP, so language conditioning is not isolated from backbone choice -- but it is a correctness/statistical-support concern, not a circular reduction. Same-group components and baselines (OpenTV [9], VBC [31], GeFF [44]) are external comparisons or an empirically ablated low-level controller (Table IV); none is used to define the headline result, and the paper reports per-cell cases where those baselines outperform WildLMa. No uniqueness theorem or ansatz is imported from the authors' prior work to force a conclusion. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on domain assumptions about transfer of pretrained models and data collection quality, but the paper introduces no new physical entities. The hand-chosen teleoperation and termination constants are free parameters in the pipeline, though they are not fitted to the evaluation data.

free parameters (5)
  • Translation scale sc (Eq. 1) = not reported
    Hand-chosen scaling constant mapping human wrist translations to Z1 arm end-effector translations during VR teleoperation; no sensitivity analysis.
  • Left-hand joystick deadzone xth = 5 cm
    Hand-chosen threshold for planar base velocity commands in whole-body teleoperation (Sec. III-A).
  • End-signal buffer size n = 10 frames
    Hand-chosen number of trailing demonstration frames that carry the end signal, to make termination supervision dense enough for ACT training (Sec. III-B).
  • End-signal termination threshold tau = 0.8
    Hand-chosen prediction threshold for the sliding-window termination detector during deployment (Sec. III-B).
  • Number of demonstrations per skill = 30 to 60
    Data budget chosen in Implementation Details; the 'tens of demonstrations' claim depends on this range, but no scaling study is reported.
assumptions (5)
  • domain assumption The MaskCLIP reparameterization of frozen CLIP yields dense pixel-aligned features whose cosine similarity with text embeddings is a valid probability map for object localization (Sec. III-B, Eq. 2).
    Borrowed from MaskCLIP [73]; the paper assumes this transfers to the robot's head and wrist camera views without fine-tuning.
  • domain assumption The VBC low-level whole-body controller, trained with sim2real RL, transfers to the Unitree B1/Z1 with a custom 3D-printed gripper and provides stable base-arm coordination under VR teleoperation (Sec. III-A).
    The paper adapts a pretrained controller from Liu et al. [31] without retraining; the success of the teleoperation interface depends on this transfer.
  • domain assumption ACT with CLIP features can learn accurate closed-loop visuomotor policies from 30-60 demonstrations per skill on this robot (Sec. III-B).
    Relies on the empirical success of ACT [69] and the authors' prior OpenTV [9]; no formal guarantee or large-scale data study is provided.
  • domain assumption GPT-4V provides accurate semantic scene descriptions and node-utility estimates for BFS planning in the hierarchical scene graph (Sec. III-C).
    The fine-grained planner uses an LLM as a heuristic evaluator; planning success depends on that model's reliability.
  • domain assumption FAST-LIO and DLO provide consistent robot pose estimation for the manually annotated waypoint map (Sec. III-C).
    The planner navigation assumes the SLAM stack outputs a world frame consistent with the hand-annotated nodes and edges.

how reviews work

0 comments
Cite this review

Pith. "Pith review of WildLMa: Long Horizon Loco-Manipulation in the Wild." pith.science (2026). https://pith.science/paper/WOXRKODQ

@misc{pith2026241115131,
  author       = {Pith},
  title        = {Pith review of: WildLMa: Long Horizon Loco-Manipulation in the Wild},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WOXRKODQ}},
  note         = {Machine review of arXiv:2411.15131}
}
read the original abstract

'In-the-wild' mobile manipulation aims to deploy robots in diverse real-world environments, which requires the robot to (1) have skills that generalize across object configurations; (2) be capable of long-horizon task execution in diverse environments; and (3) perform complex manipulation beyond pick-and-place. Quadruped robots with manipulators hold promise for extending the workspace and enabling robust locomotion, but existing results do not investigate such a capability. This paper proposes WildLMa with three components to address these issues: (1) adaptation of learned low-level controller for VR-enabled whole-body teleoperation and traversability; (2) WildLMa-Skill -- a library of generalizable visuomotor skills acquired via imitation learning or heuristics and (3) WildLMa-Planner -- an interface of learned skills that allow LLM planners to coordinate skills for long-horizon tasks. We demonstrate the importance of high-quality training data by achieving higher grasping success rate over existing RL baselines using only tens of demonstrations. WildLMa exploits CLIP for language-conditioned imitation learning that empirically generalizes to objects unseen in training demonstrations. Besides extensive quantitative evaluation, we qualitatively demonstrate practical robot applications, such as cleaning up trash in university hallways or outdoor terrains, operating articulated objects, and rearranging items on a bookshelf.

Figures

Figures reproduced from arXiv: 2411.15131 by the authors.

Figure 1
Figure 1. WildLMa implements a framework for in-the-wild manipulation with a quadruped robot, which combines a whole [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of WildLMa models and robot setups. (a) WildLMa takes a frozen CLIP model to encode task-specific texts and visual observations; (b) Our robot platform is a Unitree B1 quadruped combined with a Unitree Z1 arm and a 3D-printed gripper, with two RGBD cameras and one lidar mounted on. focus on execution of short tasks; while we investigate in-the￾wild mobile manipulation with long horizon task execution. III. … view at source ↗
Figure 3
Figure 3. Overview of WildLMa-planner. Given a constructed hierarchical scene graph, WildLMa-planner adopts a coarse-to [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative illustrations of some evaluated tasks. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. InCoM: Intent-Driven Perception and Structured Coordination for Mobile Manipulation

    cs.RO 2026-02 unverdicted novelty 6.0 of 10

    InCoM reports 23–28 percentage-point success-rate gains in mobile manipulation benchmarks by dynamically reweighting multi-scale perception via inferred motion intent and decoupling base-arm action generation with flo...

  2. Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining

    cs.RO 2025-06 conditional novelty 6.0 of 10

    Pretraining a modular transformer policy on human demonstrations then finetuning on a small robot dataset improves success on six real quadruped manipulation tasks, including out-of-distribution objects.

  3. Representative Volume Element: Existence and Extent in Cracked Heterogeneous Medium

    cs.CE 2025-08 unverdicted novelty 5.0 of 10

    Modified periodic boundary conditions that add strain periodicity to displacement periodicity are claimed to reduce mesh and size sensitivity in cracked-composite RVE simulations, tested on 1,200 samples.

  4. AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation

    cs.RO 2025-07 conditional novelty 5.0 of 10

    AC-DiT adds mobility-to-body conditioning and perception-aware 2D/3D weighting to a diffusion transformer, improving success rates on simulated and real-world mobile manipulation tasks.

  5. Towards Terrain-Aware Task-Driven 3D Scene Graph Generation in Outdoor Environments

    cs.RO 2025-06 conditional novelty 5.0 of 10

    An outdoor 3D scene graph pipeline using LiDAR-camera fusion, CLIP embeddings, and per-terrain Voronoi graphs is demonstrated on a campus dataset with qualitative results.

Reference graph

Works this paper leans on

74 extracted references · 48 canonical work pages · cited by 5 Pith papers

  1. [1]

    Do as i can, not as i say: Ground- ing language in robotic affordances,

    M. Ahn et al. , “Do as i can, not as i say: Ground- ing language in robotic affordances,” arXiv preprint arXiv:2204.01691, 2022

    arXiv 2022

  2. [2]

    Bayesian multi-task learning mpc for robotic mobile manipulation,

    E. Arcari et al. , “Bayesian multi-task learning mpc for robotic mobile manipulation,” IEEE Robotics and Automation Letters, 2023

    work page 2023

  3. [3]

    Alma-articulated locomotion and manipulation for a torque-controllable robot,

    C. D. Bellicoso et al. , “Alma-articulated locomotion and manipulation for a torque-controllable robot,” in 2019 International conference on robotics and au- tomation (ICRA) , 2019

    work page 2019

  4. [4]

    Track2act: Predicting point tracks from internet videos enables diverse zero-shot robot manipulation,

    H. Bharadhwaj, R. Mottaghi, A. Gupta, and S. Tul- siani, “Track2act: Predicting point tracks from internet videos enables diverse zero-shot robot manipulation,” in ECCV, 2024

    work page 2024

  5. [5]

    End to end learning for self-driving cars,

    M. Bojarski, “End to end learning for self-driving cars,” in arXiv preprint arXiv:1604.07316 , 2016

    arXiv 2016

  6. [6]

    Goat: Go to any thing,

    M. Chang et al. , “Goat: Go to any thing,” in RSS, 2024

    work page 2024

  7. [7]

    Direct lidar odometry: Fast localization with dense point clouds,

    K. Chen, B. T. Lopez, A.-a. Agha-mohammadi, and A. Mehta, “Direct lidar odometry: Fast localization with dense point clouds,” IEEE Robotics and Automation Letters, 2022

    work page 2022

  8. [8]

    Spatialrgpt: Grounded spatial reasoning in vision language model,

    A.-C. Cheng et al. , “Spatialrgpt: Grounded spatial reasoning in vision language model,” arXiv preprint arXiv:2406.01584, 2024

    arXiv 2024

Show all 74 references
  1. [9]

    Open-television: Teleoperation with immersive active visual feedback,

    X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang, “Open-television: Teleoperation with immersive active visual feedback,” in CoRL, 2024

    2024

  2. [10]

    Ex- treme parkour with legged robots,

    X. Cheng, K. Shi, A. Agarwal, and D. Pathak, “Ex- treme parkour with legged robots,” in ICRA, 2024

    2024

  3. [11]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi et al. , “Diffusion policy: Visuomotor policy learning via action diffusion,” in RSS, 2023

    2023

  4. [12]

    Universal manipulation interface: In- the-wild robot teaching without in-the-wild robots,

    C. Chi et al. , “Universal manipulation interface: In- the-wild robot teaching without in-the-wild robots,” in RSS, 2024

    2024

  5. [13]

    Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning,

    R. Ding et al., “Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning,” arXiv preprint arXiv:2407.03162, 2024

    2024 arXiv

  6. [14]

    Graspnet- 1billion: A large-scale benchmark for general object grasping,

    H.-S. Fang, C. Wang, M. Gou, and C. Lu, “Graspnet- 1billion: A large-scale benchmark for general object grasping,” in CVPR, 2020

    2020

  7. [15]

    Deep whole-body control: Learning a unified policy for manipulation and locomotion,

    Z. Fu, X. Cheng, and D. Pathak, “Deep whole-body control: Learning a unified policy for manipulation and locomotion,” in Conference on Robot Learning , 2023

    2023

  8. [16]

    Humanplus: Humanoid shadowing and imita- tion from humans,

    Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn, “Humanplus: Humanoid shadowing and imita- tion from humans,” in CoRL, 2024

    2024

  9. [17]

    Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,

    Z. Fu, T. Z. Zhao, and C. Finn, “Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,” arXiv, 2024

    2024

  10. [18]

    Pddlstream: Integrating symbolic planners and black- box samplers via optimistic adaptive planning,

    C. R. Garrett, T. Lozano-P ´erez, and L. P. Kaelbling, “Pddlstream: Integrating symbolic planners and black- box samplers via optimistic adaptive planning,” arXiv, 2020

    2020

  11. [19]

    Multi- skill mobile manipulation for object rearrangement,

    J. Gu, D. S. Chaplot, H. Su, and J. Malik, “Multi- skill mobile manipulation for object rearrangement,” in The Eleventh International Conference on Learning Representations, 2023

    2023

  12. [20]

    Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning,

    Q. Gu et al. , “Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning,” in ICRA, 2024

    2024

  13. [21]

    Open- vocabulary object detection via vision and language knowledge distillation,

    X. Gu, T.-Y . Lin, W. Kuo, and Y . Cui, “Open- vocabulary object detection via vision and language knowledge distillation,” arXiv preprint arXiv:2104.13921, 2021

    2021 arXiv

  14. [22]

    Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers,

    H. Ha, Y . Gao, Z. Fu, J. Tan, and S. Song, “Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers,” in CoRL, 2024

    2024

  15. [23]

    Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning,

    T. He et al. , “Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning,” in CoRL, 2024

    2024

  16. [24]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,

    W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” in Inter- national conference on machine learning , 2022

    2022

  17. [25]

    Hydra: A real-time spatial perception system for 3d scene graph construction and optimization,

    N. Hughes, Y . Chang, and L. Carlone, “Hydra: A real-time spatial perception system for 3d scene graph construction and optimization,” arXiv preprint arXiv:2201.13360, 2022

    2022 arXiv

  18. [26]

    Open teach: A versatile teleopera- tion system for robotic manipulation,

    A. Iyer et al. , “Open teach: A versatile teleopera- tion system for robotic manipulation,” arXiv preprint arXiv:2403.07870, 2024

    2024 arXiv

  19. [27]

    Graspsplats: Efficient manipulation with 3d feature splatting,

    M. Ji, R.-Z. Qiu, X. Zou, and X. Wang, “Graspsplats: Efficient manipulation with 3d feature splatting,” in CoRL, 2024

    2024

  20. [28]

    Segment anything,

    A. Kirillov et al., “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Com- puter Vision, 2023

    2023

  21. [29]

    Robotic table wiping via reinforcement learning and whole-body trajectory optimization,

    T. Lew et al., “Robotic table wiping via reinforcement learning and whole-body trajectory optimization,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) , 2023

    2023

  22. [30]

    Skilldiffuser: Interpretable hierarchical planning via skill abstractions in diffusion-based task execution,

    Z. Liang, Y . Mu, H. Ma, M. Tomizuka, M. Ding, and P. Luo, “Skilldiffuser: Interpretable hierarchical planning via skill abstractions in diffusion-based task execution,” in CVPR, 2024

    2024

  23. [31]

    Visual whole-body control for legged loco-manipulation,

    M. Liu et al. , “Visual whole-body control for legged loco-manipulation,” in CoRL, 2024

    2024

  24. [32]

    Ok-robot: What really matters in integrating open-knowledge models for robotics,

    P. Liu, Y . Orru, C. Paxton, N. M. M. Shafiullah, and L. Pinto, “Ok-robot: What really matters in integrating open-knowledge models for robotics,” arXiv preprint arXiv:2401.12202, 2024

    2024 arXiv

  25. [33]

    Dynamem: Online dynamic spatio- semantic memory for open world mobile manipula- tion,

    P. Liu et al. , “Dynamem: Online dynamic spatio- semantic memory for open world mobile manipula- tion,” arXiv preprint arXiv:2411.04999 , 2024

    2024 arXiv

  26. [34]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu et al. , “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” arXiv preprint arXiv:2303.05499 , 2023

    2023 arXiv

  27. [35]

    Combining learning-based locomotion policy with model-based manipulation for legged mobile manipu- lators,

    Y . Ma, F. Farshidian, T. Miki, J. Lee, and M. Hutter, “Combining learning-based locomotion policy with model-based manipulation for legged mobile manipu- lators,” IEEE Robotics and Automation Letters , 2022

    2022

  28. [36]

    Clio: Real-time task-driven open-set 3d scene graphs,

    D. Maggio et al. , “Clio: Real-time task-driven open-set 3d scene graphs,” arXiv preprint arXiv:2404.13696, 2024

    2024 arXiv

  29. [37]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,” OpenAI, Tech. Rep., 2023

    2023

  30. [38]

    Dinov2: Learning robust vi- sual features without supervision,

    M. Oquab et al. , “Dinov2: Learning robust vi- sual features without supervision,” arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  31. [39]

    Open x-embodiment: Robotic learning datasets and rt-x models,

    A. Padalkar et al. , “Open x-embodiment: Robotic learning datasets and rt-x models,” arXiv preprint arXiv:2310.08864, 2023

    2023 arXiv

  32. [40]

    Roboduet: A framework affording mobile-manipulation and cross-embodiment,

    G. Pan et al. , “Roboduet: A framework affording mobile-manipulation and cross-embodiment,” arXiv preprint arXiv:2403.17367, 2024

    2024 arXiv

  33. [41]

    Alvinn: An autonomous land ve- hicle in a neural network,

    D. A. Pomerleau, “Alvinn: An autonomous land ve- hicle in a neural network,” in Advances in neural information processing systems , 1988

    1988

  34. [42]

    Learning force control for legged manipulation,

    T. Portela, G. B. Margolis, Y . Ji, and P. Agrawal, “Learning force control for legged manipulation,” in ICRA, 2024

    2024

  35. [43]

    Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system,

    Y . Qin et al. , “Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system,” arXiv preprint arXiv:2307.04577, 2023

    2023 arXiv

  36. [44]

    Learning generalizable feature fields for mobile manipulation,

    R.-Z. Qiu et al. , “Learning generalizable feature fields for mobile manipulation,” arXiv preprint arXiv:2403.07563, 2024

    2024 arXiv

  37. [45]

    Learning transferable visual mod- els from natural language supervision,

    A. Radford et al. , “Learning transferable visual mod- els from natural language supervision,” in ICML, PMLR, 2021

    2021

  38. [46]

    Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning,

    K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suenderhauf, “Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning,” in CoRL, 2023

    2023

  39. [47]

    Ros moveit motion planning framework , https:// moveit.ros.org/, Accessed: 2024-09-13

    2024

  40. [48]

    On bringing robots home,

    N. M. M. Shafiullah et al., “On bringing robots home,” arXiv preprint arXiv:2311.16098 , 2023

    2023 arXiv

  41. [49]

    Distilled feature fields enable few-shot language-guided manipulation,

    W. Shen, G. Yang, A. Yu, J. Wong, L. P. Kaelbling, and P. Isola, “Distilled feature fields enable few-shot language-guided manipulation,” in CoRL, 2023

    2023

  42. [50]

    Versatile multicontact planning and control for legged loco- manipulation,

    J.-P. Sleiman, F. Farshidian, and M. Hutter, “Versatile multicontact planning and control for legged loco- manipulation,” Science Robotics , 2023

    2023

  43. [51]

    Llm-planner: Few-shot grounded planning for embodied agents with large language models,

    C. H. Song, J. Wu, C. Washington, B. M. Sadler, W.-L. Chao, and Y . Su, “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” in ICCV, 2023

    2023

  44. [52]

    Dropout: A simple way to prevent neural networks from overfitting,

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” The journal of machine learning research , 2014

    2014

  45. [53]

    Combined task and motion planning through an extensible planner-independent interface layer,

    S. Srivastava, E. Fang, L. Riano, R. Chitnis, S. Russell, and P. Abbeel, “Combined task and motion planning through an extensible planner-independent interface layer,” in ICRA, 2014

    2014

  46. [54]

    com / stretch - 3 - product, Accessed: 2024-09-01

    Stretch open source mobile manipulator - hello robot , https : / / hello - robot . com / stretch - 3 - product, Accessed: 2024-09-01

    2024

  47. [55]

    Fully autonomous real-world reinforce- ment learning with applications to mobile manipula- tion,

    C. Sun et al., “Fully autonomous real-world reinforce- ment learning with applications to mobile manipula- tion,” in Conference on Robot Learning , 2022

    2022

  48. [56]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei et al. , “Chain-of-thought prompting elicits reasoning in large language models,” NeurIPS, 2022

    2022

  49. [57]

    Error-aware imitation learning from teleoperation data for mobile manipulation,

    J. Wong et al. , “Error-aware imitation learning from teleoperation data for mobile manipulation,” in Con- ference on Robot Learning , 2022

    2022

  50. [58]

    Tidybot: Personalized robot assistance with large language models,

    J. Wu et al. , “Tidybot: Personalized robot assistance with large language models,” Autonomous Robots , 2023

    2023

  51. [59]

    Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators,

    P. Wu, Y . Shentu, Z. Yi, X. Lin, and P. Abbeel, “Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators,” arXiv preprint arXiv:2309.13037, 2023

    2023 arXiv

  52. [60]

    Relmogen: Integrating motion generation in reinforcement learning for mobile ma- nipulation,

    F. Xia, C. Li, R. Mart ´ın-Mart´ın, O. Litany, A. To- shev, and S. Savarese, “Relmogen: Integrating motion generation in reinforcement learning for mobile ma- nipulation,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) , 2021

    2021

  53. [61]

    Adaptive mobile manipulation for articulated objects in the open world,

    H. Xiong, R. Mendonca, K. Shaw, and D. Pathak, “Adaptive mobile manipulation for articulated objects in the open world,” arXiv preprint arXiv:2401.14403 , 2024

    2024 arXiv

  54. [62]

    Fast-lio: A fast, robust lidar- inertial odometry package by tightly-coupled iterated kalman filter,

    W. Xu and F. Zhang, “Fast-lio: A fast, robust lidar- inertial odometry package by tightly-coupled iterated kalman filter,” IEEE Robotics and Automation Letters , 2021

    2021

  55. [63]

    Generalized animal imitator: Agile lo- comotion with versatile motion prior,

    R. Yang et al., “Generalized animal imitator: Agile lo- comotion with versatile motion prior,” arXiv preprint arXiv:2310.01408, 2023

    2023 arXiv

  56. [64]

    Ace: A cross-platform visual- exoskeletons system for low-cost dexterous teleoper- ation,

    S. Yang et al. , “Ace: A cross-platform visual- exoskeletons system for low-cost dexterous teleoper- ation,” in CoRL, 2024

    2024

  57. [65]

    Homerobot: Open- vocabulary mobile manipulation,

    S. Yenamandra et al. , “Homerobot: Open- vocabulary mobile manipulation,” arXiv preprint arXiv:2306.11565, 2023

    2023 arXiv

  58. [66]

    Asc: Adaptive skill coordination for robotic mobile manipulation,

    N. Yokoyama et al., “Asc: Adaptive skill coordination for robotic mobile manipulation,” IEEE Robotics and Automation Letters, 2023

    2023

  59. [67]

    Gamma: Graspability-aware mobile manipulation policy learning based on online grasping pose fusion,

    J. Zhang et al. , “Gamma: Graspability-aware mobile manipulation policy learning based on online grasping pose fusion,” arXiv preprint arXiv:2309.15459 , 2023

    2023 arXiv

  60. [68]

    Adaptigraph: Material-adaptive graph-based neural dynamics for robotic manipulation,

    K. Zhang, B. Li, K. Hauser, and Y . Li, “Adaptigraph: Material-adaptive graph-based neural dynamics for robotic manipulation,” in RSS, 2024

    2024

  61. [69]

    Learn- ing fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learn- ing fine-grained bimanual manipulation with low-cost hardware,” in arXiv preprint arXiv:2304.13705 , 2023

    2023 arXiv

  62. [70]

    Aloha unleashed: A simple recipe for robot dexterity,

    T. Z. Zhao et al. , “Aloha unleashed: A simple recipe for robot dexterity,” in CoRL, 2024

    2024

  63. [71]

    Closed-loop open-vocabulary mo- bile manipulation with gpt-4v,

    P. Zhi et al. , “Closed-loop open-vocabulary mo- bile manipulation with gpt-4v,” arXiv preprint arXiv:2404.10220, 2024

    2024 arXiv

  64. [72]

    Teleman: Teleoperation for legged robot loco-manipulation using wearable imu-based motion capture,

    C. Zhou, C. Peers, Y . Wan, R. Richardson, and D. Kanoulas, “Teleman: Teleoperation for legged robot loco-manipulation using wearable imu-based motion capture,” arXiv preprint arXiv:2209.10314 , 2022

    2022 arXiv

  65. [73]

    Extract free dense labels from clip,

    C. Zhou, C. C. Loy, and B. Dai, “Extract free dense labels from clip,” in ECCV, 2022

    2022

  66. [74]

    Go fetch!-dynamic grasps using boston dynamics spot with external robotic arm,

    S. Zimmermann, R. Poranne, and S. Coros, “Go fetch!-dynamic grasps using boston dynamics spot with external robotic arm,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) , 2021

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.