Pith. sign in

REVIEW 2 major objections 4 minor 2 cited by

MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos

T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MimicDroid claims that a humanoid can learn a new manipulation task from 1–3 human demonstration videos, with no parameter updates, after training only on unlabeled human play videos.

desk verdict A credible, well-ablated demonstration that retrieval-based meta-training on unlabeled human play video can yield few-shot in-context manipulation on a humanoid, with the main caveat that simulation trains on clean teleop actions while reality depends on WiLoR. read the letter →

arxiv 2509.09769 v1 pith:6QJIGCPX submitted 2025-09-11 cs.RO

Rutav Shah , Shuijing Liu , Qi Wang , Zhenyu Jiang , Sateesh Kumar , Mingyo Seo , Roberto Martín-Martín , Yuke Zhu This is my paper
classification cs.RO
keywords in-contextlearninghumanoidmanipulationhumanplayvideofew-shotimitationbehaviorcloninghandposeestimationembodimentgapmeta-ICL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MimicDroid aims to show that in-context learning for humanoid manipulation can be trained entirely from unlabeled human play videos—continuous, freely recorded interactions—rather than from expensive teleoperated robot data. The paper constructs meta-training pairs by retrieving trajectory segments with similar manipulation behaviors from play video and training a policy to predict one segment's actions given the others. At test time, the policy uses 1–3 new human demonstration videos as context to perform a novel task on a humanoid robot without any gradient updates. If correct, this would make few-shot robot learning scalable to web-scale human video and remove a major data bottleneck in humanoid manipulation.

What carries the argument

The load-bearing mechanism is the self-supervised construction of context–target training pairs from unlabeled human play video: for each target trajectory segment, the top-k most similar segments (by cosine similarity between an embedding that concatenates mean-pooled visual features and the sequence of estimated hand actions) serve as context, and the policy is trained to predict the target's future wrist poses conditioned on that context. This meta-training objective, combined with wrist-pose retargeting to the humanoid's task space and random patch masking to reduce dependence on human-specific visual cues, is what endows the policy with test-time few-shot adaptation.

What would settle it

Run a real-world evaluation of a task where the demonstrator's hand is occluded or disappears for a substantial part of the motion (for example, reaching into a closed cupboard or behind furniture). If the policy fails on such a task while succeeding on tasks where hands stay visible, that would confirm the central claim's dependence on the hand pose estimator; conversely, if it succeeds despite the occlusion, the claim is more robust. A direct check is to compare the hand-pose estimator's wrist predictions against ground truth on those frames and measure the corruption rate.

Watch

Extended reading notes

Core claim

The central claim is that a long-context transformer policy, trained by behavior cloning on self-generated context–target pairs from unlabeled human play video, acquires a genuine in-context learning ability that transfers from humans to a humanoid robot. Training uses the future human wrist pose (from an off-the-shelf hand pose estimator) as the action label, and retrieves the most similar segments based on a cosine-similarity score that combines mean-pooled visual features and action sequences. The policy is conditioned on these retrieved segments at training time, and at test time on a few human demonstrations of a novel task; it then predicts action chunks for the robot without adaptatio

Load-bearing premise

The entire real-world pipeline relies on an off-the-shelf hand pose estimator to provide action labels during training and to parse test-time demonstrations; if that estimator fails (e.g., when hands vanish), the policy is trained on corrupted actions and deployment fails, and this failure mode is never exercised in simulation because simulation uses ground-truth teleoperated actions.

Editorial extensions

If this is right

  • Few-shot humanoid manipulation could be trained from human-only RGB video, eliminating the need for teleoperated robot demonstrations.
  • The method's performance scales with the volume of play data, suggesting that web-scale human video may further improve in-context learning capability.
  • Since adaptation is gradient-free and instantaneous, deployment is faster and avoids catastrophic forgetting seen with test-time fine-tuning.
  • The approach opens a path to training generalist humanoid policies by mining the repetitive manipulation patterns naturally present in human play.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's real-world results are only as reliable as the hand pose estimator: because that estimator also supplies the training actions, any task where hands disappear from view (e.g., reaching into a cupboard) will likely produce corrupted supervision and deployment failure, even though simulation results—trained on ground-truth teleoperation—do not reflect this.
  • The retrieval-based context construction depends on the assumption that similar manipulation behaviors share observation-action patterns across different objects and scenes; if that assumption fails for a task class (e.g., tasks requiring novel motion sequences with no visual similarity to training), in-context learning may not transfer, consistent with the reported lower L3 success rates.
  • The paper treats demonstrations as low-level state-action sequences, so it cannot generalize across semantically equivalent tasks; integrating language-conditioned context would be a natural next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper proposes MimicDroid, a method for few-shot humanoid manipulation through in-context learning (ICL) trained from human play videos. It constructs meta-training samples by retrieving trajectory segments similar to a randomly sampled target segment, using cosine similarity over mean-pooled visual features and action sequences (Eq. 2), and trains a long-context transformer with behavior cloning to predict future wrist poses on the target segment conditioned on context segments (Eq. 3). At test time, the policy is given 1-3 human demonstration videos of a novel task and acts without gradient updates. The authors introduce a RoboCasa-based simulation benchmark with three generalization levels and report real-world experiments on the GR1 humanoid, claiming a nearly twofold improvement over baselines. The paper includes ablations of visual masking, retrieval count, data scale, and number of in-context examples, and concludes with limitations around the WiLoR hand-pose estimator.

Significance. If the central claim holds, the paper is significant: it would show that ICL for humanoid manipulation can be trained from scalable, unlabeled human RGB video without teleoperated robot demonstrations. The ablations are well designed and broadly support the design choices (masking, retrieval, data scaling, context count), and the proposed simulation benchmark is a useful community resource. Credit is due for the self-supervised context-target construction and the open release of the benchmark. However, the evidence does not yet establish the central claim: the simulation benchmark trains on ground-truth teleoperated actions rather than the WiLoR-estimated actions used in deployment, and the real-world results are based on only 10 rollouts per task with no error bars. The paper's own limitation section concedes the WiLoR failure mode, which is not stress-tested in the main experiments.

major comments (2)
  1. [Sec. IV / Appendix B] The simulation training data that produces Table I uses ground-truth teleoperated actions ('Simulation: Action information (absolute wrt robot base frame) is obtained from teleoperation'), not WiLoR-estimated wrist poses on RGB human video. The real pipeline trains Eq. (3) and retrieves contexts via Eq. (2) using WiLoR outputs, and Sec. VI concedes WiLoR fails when hands vanish. Hence Table I validates Meta-ICL under clean actions, not the deployed 'human play video only' pipeline. Please add an experimental condition in which simulation training (or real training) uses WiLoR-estimated actions, or otherwise quantify sensitivity to action-estimation noise.
  2. [Sec. V / Fig. 5] The twofold improvement rests on 10 rollouts per task with no error bars or per-task counts; L3 success is 0.08. The failure analysis does not include action-estimation corruption, despite Sec. VI naming WiLoR failures (e.g., hands vanishing) as a limitation. Please report exact counts/confidence intervals and provide a diagnostic of WiLoR-induced error (e.g., task success split by hand-visibility duration, or an ablation on simulated actions corrupted with WiLoR-style noise). Without these, the real-world central claim is not statistically supported.
minor comments (4)
  1. [Sec. IV / Fig. 7] The Implementation Details say 'we find the top k=10 similar trajectories to serve as the context', but Fig. 7 reports k=50 as the best setting and the text concludes k=50 strikes a balance. Please specify which k is used for the final model and align the analysis.
  2. [Fig. 6] The caption says performance 'plateaus beyond 3', but the plotted success rates decline at 4-6 examples (0.38, 0.38, 0.33 vs. 0.39 at 3). Please rephrase to reflect the drop rather than a plateau.
  3. [Abstract / Real-world results] The phrase 'nearly twofold higher success rates' should be qualified as an average over L1-L3; the L3 comparison is 0.08 vs. 0.00, which is not a twofold improvement.
  4. [Eq. (1)] The notation \(\sigma^{ctx}_T, \sigma^{tgt}_T \sim T\) is confusing because \(T\) is used both for the task distribution and for trajectory length elsewhere. Please use distinct symbols.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: evaluation is physical task success; WiLoR dependence and minor self-citations are limitations, not circularity.

full rationale

The central derivation is not circular by construction. MimicDroid takes unlabeled play video tau, estimates future wrist poses h_{t+k}=f(s_{t+k}) as pseudo-actions (Sec. III-B), retrieves similar segments via Eq. 2 (cosine similarity in DINOv2 feature+action space), and trains a transformer with L1 behavior-cloning loss Eq. 3 on target segments conditioned on those contexts. Evaluation, both in the RoboCasa-based simulation benchmark and in real-world GR1 rollouts, scores physical task completion (e.g., pick-and-place, faucet turn-on), which is measured independently of the training loss, the retrieval scores, and the pseudo-action values. No parameter is fitted to the success metric; hyperparameters such as k are tuned on L2 success (Fig. 7), which is standard model selection. The one self-referential element is that the pseudo-actions in Eq. 3 and the action component of the retrieval features in Eq. 2 both come from the same off-the-shelf WiLoR estimator; Sec. VI explicitly concedes WiLoR 'fails when hands vanish (e.g., reaching into cupboards or behind furniture).' This makes real-world performance dependent on an action-estimation bottleneck that the simulation benchmark (Appendix B: ground-truth teleoperated actions) does not exercise—a real limitation, but not a circular reduction of the claimed result to its inputs. Self-citations (e.g., [20] MimicPlay for play-data collection statistics, [23] OKAMI for retargeting, [39] RRL for visual representations, [55] Legato for cross-embodiment) appear but are non-load-bearing background supports with their own external evaluations; they are not used as an unverified uniqueness theorem or to smuggle in an ansatz. Hence circularity score 2: no significant circularity, with minor self-citation and pipeline self-reference noted as limitations.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical or algorithmic entities are postulated; the contribution is a training-data construction, not an entity. The main hidden inputs are the tuned hyperparameters above plus two load-bearing assumptions: the retrieval similarity metric defines the pseudo-task distribution, and WiLoR hand pose estimates define the real-world action labels. The 'implicit task distribution' is framing, not a falsifiable entity.

free parameters (5)
  • Retrieval count k for meta-training context segments = 50 (best on GR1 L2)
    Swept over {3, 10, 50, 100} and evaluated on the GR1 L2 benchmark (Fig. 7); the reported Table I numbers use this tuned value.
  • Random patch masking probability p and patch budget = p=0.8, 1-16 patches, area 1-4% per frame
    Chosen to bridge the visual embodiment gap (Sec. III-C, Table IV); masking strength is tuned, only the final choice is reported.
  • Test-time context demonstration count = 3 (matching a 512-token training context)
    Fig. 6 shows gains up to 3 examples and saturation beyond; the count is bounded by the training-time context length (Appendix C).
  • Action chunk length l and execution stride = l=32, execute 16 of them
    Multi-step action chunking is a hyperparameter from prior work [53]; the paper fixes chunk 32 and executes the first 16.
  • Action horizon for future-hand-pose labels = k=1 in the real world
    The action label at time t is the estimated hand pose at t+1 (Appendix B); this horizon directly defines the supervision target.
assumptions (5)
  • domain assumption Meta-ICL transfers: training on context-target pairs from an implicit task distribution confers test-time in-context learning to novel tasks without weight updates.
    Adopted from Meta-ICL (Min et al. [22]) and prior ICL robot policies (ICRT [11], Instant Policy [12]); the paper extends it to an implicit distribution induced by its own retrieval (Sec. III-A, Eq. 1).
  • ad hoc to paper Cosine similarity between mean-pooled DINOv2 visual features concatenated with estimated actions identifies trajectory segments with genuinely similar manipulation behavior.
    This retrieval defines the pseudo-task distribution for meta-training (Sec. III-B, Eq. 2). If retrieval pairs dissimilar behaviors, the context teaches noise; the paper's own k-sweep (Fig. 7) shows performance is sensitive to retrieval quality, so the assumption is load-bearing.
  • domain assumption WiLoR hand pose estimates are accurate enough to serve as action labels and proprioception in real-world training and at test time.
    The real-world supervision in Eq. 3 and retrieval in Eq. 2 depend entirely on these estimates, and the paper concedes WiLoR 'fails when hands vanish' (Sec. VI). Simulation training uses ground-truth teleop actions (Appendix B), so this assumption is never exercised by the simulation.
  • domain assumption Retargeting human wrist poses to the GR1 humanoid wrist in task space preserves task intent.
    The kinematic-gap solution assumes humanoid-human kinematic similarity is sufficient (Sec. III-C, citing [23-25]); the paper reports failure cases where novel robot motions are required (Sec. V), which marks the boundary of this assumption.
  • ad hoc to paper Random patch masking during training reduces reliance on human-specific visual cues without destroying task-relevant information.
    Heuristic regularization for cross-embodiment transfer (Sec. III-C). The ablation supports it on transfer (Table I, GR1) but it slightly hurts source-domain performance (L2 abstract 0.51 with masking vs 0.39 without), leaving the mechanism only partially characterized.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos." pith.science (2026). https://pith.science/paper/6QJIGCPX

@misc{pith2026250909769,
  author       = {Pith},
  title        = {Pith review of: MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6QJIGCPX}},
  note         = {Machine review of arXiv:2509.09769}
}
read the original abstract

We aim to enable humanoid robots to efficiently solve new manipulation tasks from a few video examples. In-context learning (ICL) is a promising framework for achieving this goal due to its test-time data efficiency and rapid adaptability. However, current ICL methods rely on labor-intensive teleoperated data for training, which restricts scalability. We propose using human play videos -- continuous, unlabeled videos of people interacting freely with their environment -- as a scalable and diverse training data source. We introduce MimicDroid, which enables humanoids to perform ICL using human play videos as the only training data. MimicDroid extracts trajectory pairs with similar manipulation behaviors and trains the policy to predict the actions of one trajectory conditioned on the other. Through this process, the model acquired ICL capabilities for adapting to novel objects and environments at test time. To bridge the embodiment gap, MimicDroid first retargets human wrist poses estimated from RGB videos to the humanoid, leveraging kinematic similarity. It also applies random patch masking during training to reduce overfitting to human-specific cues and improve robustness to visual differences. To evaluate few-shot learning for humanoids, we introduce an open-source simulation benchmark with increasing levels of generalization difficulty. MimicDroid outperformed state-of-the-art methods and achieved nearly twofold higher success rates in the real world. Additional materials can be found on: ut-austin-rpl.github.io/MimicDroid

Figures

Figures reproduced from arXiv: 2509.09769 by the authors.

Figure 1
Figure 1. Overview. MIMICDROID enables few-shot learning for humanoid manipulation by training solely on human play videos—a scalable and diverse data source. At test time, it observes human videos of novel tasks and uses in-context learning to perform the same tasks under different object placements. Abstract— We aim to enable humanoid robots to efficiently solve new manipulation tasks from a few video examples. In￾context l… view at source ↗
Figure 2
Figure 2. Method Overview. MIMICDROID performs meta-training for in-context learning (Meta-ICL) by constructing context￾target pairs from human play videos. For a target segment, we retrieve the top-k most similar trajectory segments (bottom-left) based on observation-action similarity (bottom-right) to serve as context. These context-target pairs are used to teach the policy in-context learning (top-left). To overcome the hu… view at source ↗
Figure 3
Figure 3. Examples of target and retrieved context examples. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Overview of our simulation benchmark. We introduce a simulation benchmark to evaluate few-shot learning for humanoid manipulation. It contains 8 hours of play data collected using free-floating hands (Left) across 30 objects and 8 kitchen environments. Evaluation is st…
Figure 5
Figure 5. Figure 5: MIMICDROID reduces task misiden￾tification and grasping errors using ICL compared to the video-conditioned baseline. man play videos. (1) Task-conditioned methods. We use Vid2Robot [28], which conditions on human videos, and H2R [29], which conditions on final goal ima…
Figure 6
Figure 6. Figure 6: Performance rises with more in￾context examples but plateaus beyond 3 due to training-time context length. 3 10 50 100 # of Retrieved Trajectories (k) for Training 0.35 0.40 0.45 Success Rate 0.34 0.44 0.49 0.41 Performance on GR1 (L2) MimicDroid [PITH_FULL_IMAGE:figu…
Figure 9
Figure 9. Figure 9: Examples of real-world evaluations (L1-L3). MIMICDROID generalizes to both seen (e.g., chips, bread, and apples) and unseen objects (e.g., potatoes, garlic, and cloth), as well as novel environments. Evaluation tasks include pick-and-place to different receptacles and …

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. In-Context World Modeling for Robotic Control

    cs.RO 2026-06 unverdicted novelty 7.0 of 10

    Prepending a few self-generated random interaction clips as context lets VLA policies identify novel camera viewpoints and morphologies at test time and outperform multi-view baselines without parameter updates.

  2. WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A meta-trained test-time memory lets frozen world-action models absorb unlabeled human videos and outperform in-context video conditioning on real multi-embodiment manipulation.

Reference graph

Works this paper leans on

59 extracted references · 30 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Model-agnostic meta-learning for fast adaptation of deep networks,

    Finn et al., “Model-agnostic meta-learning for fast adaptation of deep networks,” inInternational conference on machine learning, PMLR, 2017

    2017

  2. [2]

    One-shot imitation from observing humans via domain-adaptive meta-learning,

    Yu et al., “One-shot imitation from observing humans via domain-adaptive meta-learning,”preprint arXiv:1802.01557, 2018

    arXiv 2018

  3. [3]

    Task-embedded control networks for few-shot imitation learning,

    James et al., “Task-embedded control networks for few-shot imitation learning,” inConference on robot learning, PMLR, 2018

    2018

  4. [4]

    Language models are few-shot learners,

    Brown et al., “Language models are few-shot learners,” Advances in neural information processing systems, 2020

    2020

  5. [5]

    Flamingo: A visual language model for few- shot learning,

    Alayrac et al., “Flamingo: A visual language model for few- shot learning,”Advances in neural information processing systems, 2022

    2022

  6. [6]

    Prompting decision transformer for few-shot pol- icy generalization,

    Xu et al., “Prompting decision transformer for few-shot pol- icy generalization,” ininternational conference on machine learning, PMLR, 2022

    2022

  7. [7]

    Amago: Scalable in-context reinforcement learning for adaptive agents,

    Grigsby et al., “Amago: Scalable in-context reinforcement learning for adaptive agents,”preprint arXiv:2310.09971, 2023

    arXiv 2023

  8. [8]

    In-context reinforcement learning with algo- rithm distillation,

    Laskin et al., “In-context reinforcement learning with algo- rithm distillation,”preprint arXiv:2210.14215, 2022

    arXiv 2022

Show all 59 references
  1. [9]

    Regent: A retrieval-augmented generalist agent that can act in-context in new environments,

    Sridhar et al., “Regent: A retrieval-augmented generalist agent that can act in-context in new environments,”preprint arXiv:2412.04759, 2024

    2024 arXiv

  2. [10]

    Keypoint action tokens enable in-context imitation learning in robotics,

    Di Palo et al., “Keypoint action tokens enable in-context imitation learning in robotics,”preprint arXiv:2403.19578, 2024

    2024 arXiv

  3. [11]

    In-context imitation learning via next-token pre- diction,

    Fu et al., “In-context imitation learning via next-token pre- diction,”preprint arXiv:2408.15980, 2024

    2024 arXiv

  4. [12]

    Instant policy: In-context imitation learning via graph diffusion,

    V osylius et al., “Instant policy: In-context imitation learning via graph diffusion,”preprint arXiv:2411.12633, 2024

    2024 arXiv

  5. [13]

    Generalization to new sequential de- cision making tasks with in-context learning,

    Raparthy et al., “Generalization to new sequential de- cision making tasks with in-context learning,”preprint arXiv:2312.03801, 2023

    2023 arXiv

  6. [14]

    Benchmarking general-purpose in-context learning,

    Wang et al., “Benchmarking general-purpose in-context learning,”preprint arXiv:2405.17234, 2024

    2024 arXiv

  7. [15]

    General-purpose in-context learning by meta- learning transformers,

    Kirsch et al., “General-purpose in-context learning by meta- learning transformers,”preprint arXiv:2212.04458, 2022

    2022 arXiv

  8. [16]

    Ricl: Adding in-context adaptability to pre-trained vision-language-action models,

    Sridhar et al., “Ricl: Adding in-context adaptability to pre-trained vision-language-action models,”preprint arXiv:2508.02062, 2025

    2025 arXiv

  9. [17]

    Roboturk: A crowdsourcing platform for robotic skill learning through imitation,

    Mandlekar et al., “Roboturk: A crowdsourcing platform for robotic skill learning through imitation,” inConference on Robot Learning, PMLR, 2018

    2018

  10. [18]

    Collaboration,Open X-Embodiment: Robotic learning datasets and RT-X models,https://arxiv.org/abs/ 2310.08864, 2023

    2023 arXiv

  11. [19]

    Droid: A large-scale in-the-wild robot manipulation dataset,

    al, “Droid: A large-scale in-the-wild robot manipulation dataset,” 2024

    2024

  12. [20]

    Mimicplay: Long-horizon imitation learning by watching human play,

    Wang et al., “Mimicplay: Long-horizon imitation learning by watching human play,”preprint arXiv:2302.12422, 2023

    2023 arXiv

  13. [21]

    Learning latent plans from play,

    Lynch et al., “Learning latent plans from play,” inConfer- ence on robot learning, PMLR, 2020

    2020

  14. [22]

    Metaicl: Learning to learn in context,

    Min et al., “Metaicl: Learning to learn in context,”preprint arXiv:2110.15943, 2021

    2021 arXiv

  15. [23]

    Okami: Teaching humanoid robots manipulation skills through single video imitation,

    Li et al., “Okami: Teaching humanoid robots manipulation skills through single video imitation,” in8th Annual Confer- ence on Robot Learning, 2024

    2024

  16. [24]

    Humanoid policy˜ human policy,

    Qiu et al., “Humanoid policy˜ human policy,”preprint arXiv:2503.13441, 2025

    2025

  17. [25]

    Representation and control of the task space in humans and humanoid robots,

    Mistry et al., “Representation and control of the task space in humans and humanoid robots,”Humanoid Robotics and Neuroscience: Science, Engineering and Society, 2015

    2015

  18. [26]

    Masked autoencoders are scalable vision learners,

    He et al., “Masked autoencoders are scalable vision learners,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022

    2022

  19. [27]

    Rethinking patch dependence for masked autoen- coders,

    Fu et al., “Rethinking patch dependence for masked autoen- coders,”preprint arXiv:2401.14391, 2024

    2024 arXiv

  20. [28]

    Vid2robot: End-to-end video-conditioned pol- icy learning with cross-attention transformers,

    Jain et al., “Vid2robot: End-to-end video-conditioned pol- icy learning with cross-attention transformers,”preprint arXiv:2403.12943, 2024

    2024 arXiv

  21. [29]

    Zero-shot robot manipulation from pas- sive human videos,

    Bharadhwaj et al., “Zero-shot robot manipulation from pas- sive human videos,”preprint arXiv:2302.02011, 2023

    2023 arXiv

  22. [30]

    Hand me the data: Fast robot adaptation via hand path retrieval,

    Hong et al., “Hand me the data: Fast robot adaptation via hand path retrieval,”preprint arXiv:2505.20455, 2025

    2025

  23. [31]

    Robocasa: Large-scale simulation of ev- eryday tasks for generalist robots,

    Nasiriany et al., “Robocasa: Large-scale simulation of ev- eryday tasks for generalist robots,” inRobotics: Science and Systems, 2024

    2024

  24. [32]

    Robosuite: A modular simulation frame- work and benchmark for robot learning,

    Zhu et al., “Robosuite: A modular simulation frame- work and benchmark for robot learning,” inpreprint arXiv:2009.12293, 2020

    2009 arXiv

  25. [33]

    Evolutionary principles in self-referential learning,

    Schmidhuber, “Evolutionary principles in self-referential learning,”On learning how to learn: The meta-meta-... hook.) Diploma thesis, Institut f. Informatik, Tech. Univ. Munich, 1987

    1987

  26. [34]

    Meta-neural networks that learn by learning,

    Naik et al., “Meta-neural networks that learn by learning,” in[Proceedings 1992] IJCNN International Joint Conference on Neural Networks, IEEE, 1992

    1992

  27. [35]

    Meta-learning with memory-augmented neu- ral networks,

    Santoro et al., “Meta-learning with memory-augmented neu- ral networks,” inInternational conference on machine learn- ing, PMLR, 2016

    2016

  28. [36]

    Learning to learn using gradient descent,

    Hochreiter et al., “Learning to learn using gradient descent,” inInternational conference on artificial neural networks, Springer, 2001

    2001

  29. [37]

    Rl2: Fast reinforcement learning via slow reinforcement learning,

    Duan et al., “Rl2: Fast reinforcement learning via slow reinforcement learning,”preprint arXiv:1611.02779, 2016

    2016 arXiv

  30. [38]

    Attention is all you need,

    Vaswani et al., “Attention is all you need,”Advances in neural information processing systems, 2017

    2017

  31. [39]

    Rrl: Resnet as representation for reinforcement learning,

    Shah et al., “Rrl: Resnet as representation for reinforcement learning,”preprint arXiv:2107.03380, 2021

    2021 arXiv

  32. [40]

    R3m: A universal visual representation for robot manipulation,

    Nair et al., “R3m: A universal visual representation for robot manipulation,”preprint arXiv:2203.12601, 2022

    2022 arXiv

  33. [41]

    Where are we in the search for an artificial visual cortex for embodied intelligence?

    Majumdar et al., “Where are we in the search for an artificial visual cortex for embodied intelligence?”Advances in Neural Information Processing Systems, 2023

    2023

  34. [42]

    Concept2robot: Learning manipulation concepts from instructions and human demonstrations,

    Shao et al., “Concept2robot: Learning manipulation concepts from instructions and human demonstrations,”The Interna- tional Journal of Robotics Research, 2021

    2021

  35. [43]

    Liv: Language-image representations and rewards for robotic control,

    Ma et al., “Liv: Language-image representations and rewards for robotic control,” inInternational Conference on Machine Learning, PMLR, 2023

    2023

  36. [44]

    Vip: Towards universal visual reward and representation via value-implicit pre-training,

    Ma et al., “Vip: Towards universal visual reward and representation via value-implicit pre-training,”preprint arXiv:2210.00030, 2022

    2022 arXiv

  37. [45]

    Bridging the human to robot dex- terity gap through object-oriented rewards,

    Guzey et al., “Bridging the human to robot dex- terity gap through object-oriented rewards,”preprint arXiv:2410.23289, 2024

    2024 arXiv

  38. [46]

    Screwmimic: Bimanual imitation from human videos with screw space projection,

    Bahety et al., “Screwmimic: Bimanual imitation from human videos with screw space projection,”preprint arXiv:2405.03666, 2024

    2024 arXiv

  39. [47]

    Egomimic: Scaling imitation learning via egocentric video,

    Kareer et al., “Egomimic: Scaling imitation learning via egocentric video,”preprint arXiv:2410.24221, 2024

    2024 arXiv

  40. [48]

    Potamias et al.,Wilor: End-to-end 3d hand localization and reconstruction in-the-wild, 2024

    2024

  41. [49]

    Reconstructing hands in 3D with transform- ers,

    Pavlakos et al., “Reconstructing hands in 3D with transform- ers,” inCVPR, 2024

    2024

  42. [50]

    Phantom: Training robots without robots using only human videos,

    Lepert et al., “Phantom: Training robots without robots using only human videos,”preprint arXiv:2503.00779, 2025

    2025 arXiv

  43. [51]

    Vision-based manipulation from single hu- man video with open-world object graphs,

    Zhu et al., “Vision-based manipulation from single hu- man video with open-world object graphs,”preprint arXiv:2405.20321, 2024

    2024 arXiv

  44. [52]

    Dinov2: Learning robust visual features with- out supervision,

    Oquab et al., “Dinov2: Learning robust visual features with- out supervision,”preprint arXiv:2304.07193, 2023

    2023 arXiv

  45. [53]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    Zhao et al., “Learning fine-grained bimanual manipulation with low-cost hardware,”preprint arXiv:2304.13705, 2023

    2023 arXiv

  46. [54]

    Robocasa: Large-scale simulation of ev- eryday tasks for generalist robots,

    Nasiriany et al., “Robocasa: Large-scale simulation of ev- eryday tasks for generalist robots,” inRobotics: Science and Systems (RSS), 2024

    2024

  47. [55]

    Legato: Cross-embodiment imitation using a grasping tool,

    Seo et al., “Legato: Cross-embodiment imitation using a grasping tool,”IEEE Robotics and Automation Letters, 2025

    2025

  48. [56]

    Self-distillation bridges distribution gap in lan- guage model fine-tuning,

    Yang et al., “Self-distillation bridges distribution gap in lan- guage model fine-tuning,”preprint arXiv:2402.13669, 2024

    2024 arXiv

  49. [57]

    Does continual learning equally forget all pa- rameters?

    Zhao et al., “Does continual learning equally forget all pa- rameters?” InInternational Conference on Machine Learn- ing, PMLR, 2023

    2023

  50. [58]

    Segment anything,

    Kirillov et al., “Segment anything,” inProceedings of the IEEE/CVF international conference on computer vision, 2023

    2023

  51. [59]

    Decoupling human and camera motion from videos in the wild,

    Ye et al., “Decoupling human and camera motion from videos in the wild,” inIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. APPENDIX A. Data Collection We collect human play data in both real-world and simu- lated settings to train our in-context learni...

    2023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.