Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Ag2x2: Robust Agent-Agnostic Visual Representations for Zero-Shot Bimanual Manipulation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that keeping the 2D positions of the hands, while erasing the human body, makes agent-agnostic visual representations work for zero-shot bimanual manipulation, reaching 73.5% success on 13 simulated tasks and beating…

desk verdict A solid incremental extension with a real empirical win, but the reward channel is partly doing task-specific work that the 'no engineered rewards' framing understates. read the letter →

arxiv 2507.19817 v1 pith:ULC7YK2G submitted 2025-07-26 cs.RO

Ziyin Xiong , Yinghan Chen , Puhao Li , Yixin Zhu , Tengyu Liu , Siyuan Huang This is my paper · ORCID
classification cs.RO
keywords bimanualmanipulationagent-agnosticrepresentationvisualrewardshapingtime-contrastivelearningzero-shotskillacquisitionhandpositiontokensdeformableobjectreinforcement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ag2x2's central claim is that agent-agnostic visual representations, which erase the human from demonstration videos so that skills transfer to robots, can be extended to bimanual manipulation if the 2D positions of the two hands are preserved as extra input tokens. The paper trains a time-contrastive encoder on inpainted egocentric cooking frames augmented with hand-position coordinates, then uses embedding similarity between the current frame and a goal image as a reward for two simulated robot arms. Across 13 tasks from Bi-DexHands and PerAct2, it reports a 73.5% average success rate, above its strongest zero-shot baseline and above a policy trained with expert-designed rewards. The ablation without hand positions drops to 57.3%, isolating the hand-position channel as the source of the coordination signal.

What carries the argument

The load-bearing object is the hand-aware agent-agnostic dataset: egocentric human video frames with the person segmented and inpainted away, augmented by a 4-channel signal giving the 2D position of each hand, computed as the mean of 21 projected hand keypoints, with learned variables for occluded or out-of-frame hands. A vision transformer encoder maps the concatenation of image and hand-position tokens into a shared latent space, trained with a time-contrastive loss so that temporally close states and goal states embed close together. During policy learning, embedding similarity to the goal image with a tilted reward shapes exploration, and two free-floating proxy spheres act as the end-effectors whose trajectories are converted to robot joint commands by inverse kinematics. The hand-position token is what lets the representation encode where each effector is without encoding what the human looks like, which the paper identifies as the missing ingredient for bimanual coordination.

What would settle it

A direct test is to retrain Ag2x2 on the same videos while jittering the hand-position coordinates with Gaussian noise of increasing variance; if success stays near 73.5%, precise hand positions are not the active ingredient, while a collapse toward the 57.3% ablation level would confirm they are. A complementary test swaps the two simulated arms for arms with different kinematics or re-projects the goal end-effector positions through a different camera view and checks whether the 73.5% figure survives; both tests target the human-to-robot correspondence that the reward channel depends on.

Watch

Extended reading notes

Core claim

On the paper's own terms, Ag2x2 establishes that a coordination-aware but agent-agnostic representation—human bodies removed, but two 2D hand-position coordinates injected as learned tokens—suffices to acquire bimanual manipulation skills without demonstrations or engineered rewards. With goal images plus projected end-effector positions as the only task specification, reinforcement learning on proxy agents reaches a 73.5% average success rate on 13 diverse tasks, outperforming expert-designed reward functions (63.2%) and the previous agent-agnostic method (56.4%). The paper also reports that removing the hand channel causes a consistent 16.2 percentage point drop and that the learned policies can generate demonstration data: 12 trajectories of rope straightening suffice to train an imitation policy that generalizes to unseen rope configurations.

Load-bearing premise

The load-bearing premise is that 2D hand positions extracted from egocentric human cooking videos, projected through estimated camera poses, serve as a meaningful reward signal for coordinating two simulated robot arms whose body and kinematics are completely different from a human's.

Editorial extensions

If this is right

  • Zero-shot bimanual skill acquisition becomes feasible from unlabeled human video alone, removing the two main scaling bottlenecks: expert demonstrations and task-specific reward engineering.
  • Hand-position information, not backbone choice or proprioception during policy learning, is the decisive factor: removing it costs 16.2 percentage points, while adding explicit proprioceptive rewards does not close the gap.
  • Deformable-object bimanual tasks such as rope straightening, typically hard for engineered rewards, become learnable from the same representation.
  • Ag2x2-generated rollouts are usable as training data: 12 trajectories produced an imitation policy that generalizes to unseen rope configurations, offering a route to scalable data collection.
  • On most PerAct2 tasks, the method exceeds the original benchmark's reported success despite not using the benchmark's expert waypoints, suggesting the embedding-similarity reward is a strong substitute for intermediate waypoint guidance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test, not run in the paper, is to corrupt the hand positions with increasing noise during pretraining; if performance is robust to jitter, the claim that precise positions carry coordination would need refinement, and if it collapses, the positional channel is doing the work.
  • The reward signal is defined in 2D camera space, so deploying Ag2x2 on a real robot would require projecting the robot's own end-effectors through a calibrated camera; the paper's simulated experiments do not test this correspondence under real camera noise or occlusion.
  • The time-contrastive embedding can be discontinuous for discrete state transitions (the paper notes a negative progress correlation on button pressing despite 100% success), so tasks with sudden visual changes may need a supplementary discrete-state signal.
  • Because only end-effector positions are retained, the same 'erase the agent, keep the positions' recipe could in principle extend to articulated fingers, though the paper explicitly leaves dexterous manipulation to future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Ag2x2, a two-stage framework for bimanual manipulation. In the first stage, a ViT-Large encoder is fine-tuned with a time-contrastive objective on human egocentric videos from EpicKitchen in which the human body is inpainted, while the 2D positions of the two hands are preserved and injected as learned position tokens. In the second stage, a PPO policy is trained in IsaacGym with proxy-sphere agents and a tilted reward that measures embedding similarity between the current observation and a goal specification consisting of a goal image plus 2D projections of desired end-effector positions. The policy outputs proxy actions that are converted to joint commands for two Franka arms via inverse kinematics. The paper evaluates the method on 13 bimanual tasks from Bi-DexHands and PerAct2, reporting a 73.5% overall success rate, surpassing baselines (R3M, VIP, Ag2Manip, Eureka) and an expert-reward baseline, with an ablation (Ag2x2-H) that removes hand position information and drops to 57.3%. The paper also demonstrates that 12 generated trajectories can train an imitation policy that generalizes to novel rope configurations.

Significance. If the reported results are robust, the paper makes a useful contribution: it provides evidence that time-contrastive pretraining on human videos can transfer to bimanual robot control, that preserving hand-position cues helps coordination, and that autonomously generated demonstrations can seed imitation learning. The breadth of the evaluation (13 tasks, including deformable-object manipulation) and the auxiliary analyses (trajectory smoothness, task-progress consistency, and a proprioception control) are strengths. However, the central claims of zero-shot skill acquisition, absence of engineered rewards, and superiority over expert-designed rewards are weakened by the fact that the goal specification contains task-specific end-effector coordinates that enter the same embedding used by the reward, and by the absence of variance reporting in the headline results. These issues are addressable with additional controls and more detailed statistics, but they are load-bearing for the paper's main conclusions.

major comments (4)
  1. [Section III.C, Eq. (2)] The reward is computed as a monotone function of embedding similarity S(z_t, z_g), where z_t = F_phi(o_t) and z_g = F_phi(g). According to Section III.B, F_phi takes as input the RGB image concatenated with 2D hand coordinates, and according to Section III.C the goal g explicitly contains "2D projections of desired end-effector positions in camera space." Therefore, whenever the current observation includes current end-effector positions, the reward directly rewards matching those positions to the desired end-effector projections. This is a task-specific, engineered coordinate-matching signal, and it is not a consequence of the coordination-aware representation learned from human videos. The 16.2-point gain of Ag2x2 over Ag2x2-H in Table I therefore conflates two effects: (a) transferable coordination information learned from human videos, and (b) direct reward feedback on end-effector goal coordinates. To support the abstract's claim of learning "without ... engineered rewards" and to attribute the ablation gain correctly, the paper should provide a control in which hand-position tokens are present in the observation but the goal contains no hand coordinates, or remove desired end-effector projections from the goal entirely. It should also explain how the desired end-effector projections are obtained for a new task, since the current text gives no procedure.
  2. [Table I, Section IV.B] Success rates are reported only as counts out of 9 per task, with no variance, confidence intervals, or per-seed breakdown. The headline comparison of 73.5% (Ag2x2) versus 63.2% (expert reward) and 56.4% (Ag2Manip) involves differences of roughly 10-17 successes across 117 trials, and several task-level differences are only 1-2 successes (e.g., task a: Ag2x2 7 vs. Ag2x2-H 7; task g: Ag2x2 6 vs. expert 5). Without repeated-seed statistics or a significance test, it is not possible to conclude that Ag2x2 reliably outperforms the expert-reward baseline or the hand-removed ablation. The paper should report mean and standard deviation over the three seeds, per-seed counts, and an appropriate statistical comparison, especially for the central claim that Ag2x2 surpasses expert-designed rewards.
  3. [Section IV.A] The PerAct2 tasks are ported to IsaacGym with assets converted from .ttm to .urdf/.dae, and when "direct asset migration was not possible, we substituted comparable alternatives to maintain task integrity." Because both the assets and the simulation environment differ from the original PerAct2 benchmark, the statement that Ag2x2 achieves higher success rates than those reported in PerAct2 is not a direct comparison. The paper should specify which assets were substituted, quantify how the task geometry changed, and provide the exact success criteria and error margins used, so that the 13-task results are reproducible and comparable to the original benchmarks.
  4. [Section III.D] The reward scaling parameter alpha is set to 3.0 "based on ablation studies" conducted on the same 13-task evaluation suite. This means the reward function contains at least one hyperparameter tuned on the benchmark used to demonstrate the "no engineered rewards" claim. The paper should report the ablation over alpha (including per-task success rates) and discuss the sensitivity of the main results to this choice. Without this, it is unclear whether the strong performance depends on a carefully selected constant that was chosen using the test tasks.
minor comments (5)
  1. [Eq. (1)] The notation in the contrastive loss, particularly the subscript "o‰c_l", is confusing; please make explicit that l indexes a frame from a different video than i, j, and k, and define all symbols in one place.
  2. [Table I and Table IV] The column headers are inconsistent: the "Overall" row appears above the per-task columns, and the meaning of "Avg." is not uniform across tables. Clarify the table structure so that the reader can map each column to a task.
  3. [Throughout] The name "PerAct 2" is written inconsistently as "PerAct2" and "PerAct 2" in the text, captions, and references; please standardize.
  4. [Section IV.E] The Spearman rank correlation in Table III is described as measuring "task progress consistency," but it actually measures correlation between frame index and visual similarity to the final goal image, which is a property of the reward landscape, not of the policy's causal progress. Please rephrase to avoid implying that the policy itself is being directly measured.
  5. [Section V] The limitation about single static goal images is clearly stated and appreciated; consider also mentioning that the current framework requires the final end-effector positions to be specified, consistent with Major Comment 1, and discuss how those positions could be inferred automatically for truly zero-shot operation.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the hand-position goal channel is a task input rather than a self-referential prediction, and the Ag2Manip self-citation is not load-bearing.

full rationale

The derivation chain is: (i) F_phi is trained on EpicKitchen human videos with a time-contrastive loss (Eq. 1) that does not include the benchmark tasks; (ii) the same F_phi defines a goal-embedding reward (Eq. 2) used to train a policy in IsaacGym; (iii) success is measured by object-configuration thresholds. The skeptical concern that Eq. (2) rewards direct coordinate matching is real but is not a circularity: the goal's desired end-effector positions are a task input, not an output of the representation. The representation's hand-position tokens are learned from external human videos, and the proprioception ablation (Table IV) attempts to separate pre-training gains from policy-time reward effects. The only overlapping-author citation is Ag2Manip [12], used as an architectural foundation; this is not load-bearing because the paper contributes its own experiments and ablations. No prediction is forced by construction, and no fitted parameter is renamed as a prediction; therefore there is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

Ag2x2 introduces proxy sphere agents and hand-position tokens, but these are algorithmic constructs rather than new physical entities. The central free parameter is the reward scaling alpha, tuned on ablations; the main domain assumptions involve transfer of hand positions across embodiments and the proxy-grasp abstraction.

free parameters (3)
  • reward scaling parameter alpha = 3.0
    Set based on ablation studies (Section III.D); it controls the shape of the tilted reward in Eq. (2) and is tuned against task performance.
  • grasp validity threshold = 5 cm
    A proxy grasp is considered valid when the proxy sphere is within 5 cm of a GraspNet pose (Section III.C); this threshold is chosen by hand and directly affects success definition.
  • proxy sphere collision and interaction radii = 2 cm collision, 5 cm interaction
    Hand-chosen radii for the free-floating proxy agents (Section III.D). They determine when contacts count and are not derived from data.
assumptions (4)
  • domain assumption Time-contrastive embeddings trained on human videos align object and hand state changes with task progress in robot simulation.
    Invoked in Sections III.B and III.C when the learned embedding similarity is used as the RL reward.
  • domain assumption 2D hand positions from HaMeR on EpicKitchen, projected with estimated camera poses, are stable enough to serve as robot end-effector goal coordinates.
    Introduced in Section III.B; the whole hand-position channel depends on this cross-embodiment correspondence.
  • domain assumption A proxy sphere within 5 cm of a GraspNet pose, followed by inverse kinematics, is a valid model of grasping for the bimanual tasks.
    Stated in Section III.C; this abstraction replaces true contact and finger-level grasping.
  • domain assumption PPO with 68 parallel environments and 200 episodes per task is sufficient to converge for all tasks.
    Training setup in Sections III.D and IV.A; no convergence diagnostics are shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ag2x2: Robust Agent-Agnostic Visual Representations for Zero-Shot Bimanual Manipulation." pith.science (2026). https://pith.science/paper/ULC7YK2G

@misc{pith2026250719817,
  author       = {Pith},
  title        = {Pith review of: Ag2x2: Robust Agent-Agnostic Visual Representations for Zero-Shot Bimanual Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ULC7YK2G}},
  note         = {Machine review of arXiv:2507.19817}
}
read the original abstract

Bimanual manipulation, fundamental to human daily activities, remains a challenging task due to its inherent complexity of coordinated control. Recent advances have enabled zero-shot learning of single-arm manipulation skills through agent-agnostic visual representations derived from human videos; however, these methods overlook crucial agent-specific information necessary for bimanual coordination, such as end-effector positions. We propose Ag2x2, a computational framework for bimanual manipulation through coordination-aware visual representations that jointly encode object states and hand motion patterns while maintaining agent-agnosticism. Extensive experiments demonstrate that Ag2x2 achieves a 73.5% success rate across 13 diverse bimanual tasks from Bi-DexHands and PerAct2, including challenging scenarios with deformable objects like ropes. This performance outperforms baseline methods and even surpasses the success rate of policies trained with expert-engineered rewards. Furthermore, we show that representations learned through Ag2x2 can be effectively leveraged for imitation learning, establishing a scalable pipeline for skill acquisition without expert supervision. By maintaining robust performance across diverse tasks without human demonstrations or engineered rewards, Ag2x2 represents a step toward scalable learning of complex bimanual robotic skills.

Figures

Figures reproduced from arXiv: 2507.19817 by the authors.

Figure 1
Figure 1. Ag2x2 enables zero-shot acquisition of bimanual manipulation skills without relying on expert demonstrations or engineered rewards. The framework operates in two key stages: (left) learning coordination-aware visual representations directly from human manipulation videos (shown in sequential frames of cooking with highlighted hand) while preserving critical hand position data despite domain differences; and (right) … view at source ↗
Figure 2
Figure 2. Framework of Ag2x2. Our approach consists of two main components: (a) Representation Learning via time-contrastive learning on agent-agnostic human demonstrations with preserved hand position information, visualized through sequential cooking frames with erased hands but highlighted hand positions; and (b) Skill Learning through RL with agent-agnostic action representations, illustrated by a robot learning to place … view at source ↗
Figure 3
Figure 3. Qualitative results of bimanual manipulation. Temporal sequences showing successful task execution by our learned policies across six representative tasks: swinging a cup, closing scissors, pushing a box, placing a cube into a microwave, pressing buttons, and sweeping dirt. Each row presents the progression of a single task from left to right. Our method generates coordinated trajectories for both end-effectors, whi… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Generalization of learned manipulation skills through imitation policy rollout. This sequence demonstrates an imitation policy successfully straightening a rope from an unseen initial configuration. The policy was trained on just 12 expert trajectories generated by Ag2…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 39 canonical work pages

  1. [1]

    Bi-dexhands: Towards human-level bimanual dexterous manipulation,

    Y . Chen, Y . Geng, F. Zhong, J. Ji, J. Jiang, Z. Lu, H. Dong, and Y . Yang, “Bi-dexhands: Towards human-level bimanual dexterous manipulation,” Transactions on Pattern Analysis and Machine Intelli- gence (TPAMI), vol. 46, no. 5, pp. 2804–2818, 2024

    work page 2024

  2. [2]

    A review of robot learning for manipulation: Challenges, representations, and algorithms,

    O. Kroemer, S. Niekum, and G. Konidaris, “A review of robot learning for manipulation: Challenges, representations, and algorithms,” Jour- nal of Machine Learning Research (JMLR) , vol. 22, no. 30, pp. 1–82, 2021

    work page 2021

  3. [3]

    Human–robot skill transmission for mobile robot via learning by demonstration,

    J. Li, J. Wang, S. Wang, and C. Yang, “Human–robot skill transmission for mobile robot via learning by demonstration,” Neural Computing and Applications , vol. 35, no. 32, pp. 23 441–23 451, 2023

    work page 2023

  4. [4]

    Controlvla: Few-shot object-centric adap- tation for pre-trained vision-language-action models,

    P. Li, Y . Wu, Z. Xi, W. Li, Y . Huang, Z. Zhang, Y . Chen, J. Wang, S.-C. Zhu, T. Liu et al. , “Controlvla: Few-shot object-centric adap- tation for pre-trained vision-language-action models,” arXiv preprint arXiv:2506.16211, 2025

    arXiv 2025

  5. [5]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” in Robotics: Science and Systems (RSS) , 2023

    work page 2023

  6. [6]

    Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,

    Z. Fu, T. Z. Zhao, and C. Finn, “Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,” in Conference on Robot Learning (CoRL) , 2024

    work page 2024

  7. [7]

    Stabilize to act: Learning to coordinate for bimanual manipulation,

    J. Grannen, Y . Wu, B. Vu, and D. Sadigh, “Stabilize to act: Learning to coordinate for bimanual manipulation,” in Conference on Robot Learning (CoRL) , 2023

    work page 2023

  8. [8]

    Bi-kvil: Keypoints- based visual imitation learning of bimanual manipulation tasks,

    J. Gao, X. Jin, F. Krebs, N. Jaquier, and T. Asfour, “Bi-kvil: Keypoints- based visual imitation learning of bimanual manipulation tasks,” in IEEE International Conference on Robotics and Automation (ICRA) , 2024

    work page 2024

Show all 41 references
  1. [9]

    Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning,

    K. Li, P. Li, T. Liu, Y . Li, and S. Huang, “Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning,” in Proceedings of Conference on Computer Vision and Pattern Recogni- tion (CVPR) , 2025

    2025

  2. [10]

    VIP: towards universal visual reward and representation via value-implicit pre-training,

    Y . J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V . Kumar, and A. Zhang, “VIP: towards universal visual reward and representation via value-implicit pre-training,” in Proceedings of International Con- ference on Learning Representations (ICLR) , 2023

    2023

  3. [11]

    R3M: A universal visual representation for robot manipulation,

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta, “R3M: A universal visual representation for robot manipulation,” in Conference on Robot Learning (CoRL) , 2022

    2022

  4. [12]

    Ag2manip: Learning novel manipulation skills with agent- agnostic visual and action representations,

    P. Li, T. Liu, Y . Li, M. Han, H. Geng, S. Wang, Y . Zhu, S.-C. Zhu, and S. Huang, “Ag2manip: Learning novel manipulation skills with agent- agnostic visual and action representations,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2024

    2024

  5. [13]

    Peract2: Benchmarking and learning for robotic bimanual manipulation tasks,

    M. Grotz, M. Shridhar, T. Asfour, and D. Fox, “Peract2: Benchmarking and learning for robotic bimanual manipulation tasks,” CoRL 2024 Workshop on Whole-Body Control and Bimanual Manipulation (CoRL 2024 WCBM) , 2024

    2024

  6. [14]

    Dual arm manipulation—a survey,

    C. Smith, Y . Karayiannidis, L. Nalpantidis, X. Gratal, P. Qi, D. V . Dimarogonas, and D. Kragic, “Dual arm manipulation—a survey,” Robotics and Autonomous Systems , vol. 60, no. 10, pp. 1340–1353, 2012

    2012

  7. [15]

    Coordinated multi-arm motion planning: Reaching for moving ob- jects in the face of uncertainty,

    S. S. Mirrazavi Salehian, N. B. Figueroa Fernandez, and A. Billard, “Coordinated multi-arm motion planning: Reaching for moving ob- jects in the face of uncertainty,” in Robotics: Science and Systems (RSS), 2016

    2016

  8. [16]

    A dual-arm robot cooperation framework based on a nonlinear model predictive cooperative control,

    X. Zhao, Y . Zhang, W. Ding, B. Tao, and H. Ding, “A dual-arm robot cooperation framework based on a nonlinear model predictive cooperative control,” IEEE/ASME Transactions on Mechatronics (T- MECH), vol. 29, pp. 3993–4005, 2024

    2024

  9. [17]

    2handedaf- forder: Learning precise actionable bimanual affordances from human videos,

    M. Heidinger, S. Jauhri, V . Prasad, and G. Chalvatzaki, “2handedaf- forder: Learning precise actionable bimanual affordances from human videos,” in Conference on Robot Learning (CoRL) , 2024

    2024

  10. [18]

    Bi-touch: Bimanual tactile manipulation with sim-to-real deep reinforcement learning,

    Y . Lin, A. Church, M. Yang, H. Li, J. Lloyd, D. Zhang, and N. F. Lepora, “Bi-touch: Bimanual tactile manipulation with sim-to-real deep reinforcement learning,” IEEE Robotics and Automation Letters (RA-L), vol. 8, no. 9, pp. 5472–5479, 2023

    2023

  11. [19]

    A comparison of imitation learning algorithms for bimanual manipulation,

    M. Drolet, S. Stepputtis, S. Kailas, A. Jain, J. Peters, S. Schaal, and H. B. Amor, “A comparison of imitation learning algorithms for bimanual manipulation,” IEEE Robotics and Automation Letters (RA- L), 2024

    2024

  12. [20]

    Deep imitation learning for bimanual robotic manipulation,

    F. Xie, A. Chowdhury, M. C. De Paolis Kaluza, L. Zhao, L. L. Wong, and R. Yu, “Deep imitation learning for bimanual robotic manipulation,” in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) , 2020

    2020

  13. [21]

    Screwmimic: Bimanual imitation from human videos with screw space projection,

    A. Bahety, P. Mandikal, B. Abbatematteo, and R. Martín-Martín, “Screwmimic: Bimanual imitation from human videos with screw space projection,” in Robotics: Science and Systems (RSS) , 2024

    2024

  14. [22]

    Curl: Contrastive unsuper- vised representations for reinforcement learning,

    M. Laskin, A. Srinivas, and P. Abbeel, “Curl: Contrastive unsuper- vised representations for reinforcement learning,” in Proceedings of International Conference on Machine Learning (ICML) , 2020

    2020

  15. [23]

    Deepmdp: Learning continuous latent space models for representation learning,

    C. Gelada, S. Kumar, J. Buckman, O. Nachum, and M. G. Bellemare, “Deepmdp: Learning continuous latent space models for representation learning,” in Proceedings of International Conference on Machine Learning (ICML) , 2019

    2019

  16. [24]

    The surprising effectiveness of representation learning for visual imitation,

    J. Pari, N. M. Shafiullah, S. P. Arunachalam, and L. Pinto, “The surprising effectiveness of representation learning for visual imitation,” in Robotics: Science and Systems (RSS) , 2022

    2022

  17. [25]

    Learning state representations with robotic priors,

    R. Jonschkowski and O. Brock, “Learning state representations with robotic priors,” Autonomous Robots , vol. 39, pp. 407–428, 2015

    2015

  18. [26]

    Learning to see before learning to act: Visual pre-training for manipulation,

    L. Yen-Chen, A. Zeng, S. Song, P. Isola, and T.-Y . Lin, “Learning to see before learning to act: Visual pre-training for manipulation,” in IEEE International Conference on Robotics and Automation (ICRA) , 2020

    2020

  19. [27]

    Rrl: Resnet as representation for rein- forcement learning,

    R. Shah and V . Kumar, “Rrl: Resnet as representation for rein- forcement learning,” in Proceedings of International Conference on Machine Learning (ICML) , 2021

    2021

  20. [28]

    The unsurprising effectiveness of pre-trained vision models for control,

    S. Parisi, A. Rajeswaran, S. Purushwalkam, and A. Gupta, “The unsurprising effectiveness of pre-trained vision models for control,” in Proceedings of International Conference on Machine Learning (ICML), 2022

    2022

  21. [29]

    Reinforcement learning with action-free pre-training from videos,

    Y . Seo, K. Lee, S. L. James, and P. Abbeel, “Reinforcement learning with action-free pre-training from videos,” in Proceedings of Interna- tional Conference on Machine Learning (ICML) , 2022

    2022

  22. [30]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proceedings of International Confere...

    2021

  23. [31]

    Imagenet large scale visual recognition challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. S. Bernstein, A. C. Berg, and L. Fei-Fei, “Imagenet large scale visual recognition challenge,” International Journal of Computer Vision (IJCV) , vol. 115, no. 3, pp. 211–252, 2015

    2015

  24. [32]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in Proceedings of International Conference on Learning Representa- tions (ICLR) , 2022

    2022

  25. [33]

    The EPIC-KITCHENS dataset: Collection, challenges and baselines,

    D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W. Price, and M. Wray, “The EPIC-KITCHENS dataset: Collection, challenges and baselines,” Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , vol. 43,...

    2021

  26. [34]

    Reconstructing hands in 3d with transformers,

    G. Pavlakos, D. Shan, I. Radosavovic, A. Kanazawa, D. Fouhey, and J. Malik, “Reconstructing hands in 3d with transformers,” in Pro- ceedings of Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  27. [35]

    Ultralytics, “YOLOv5,” https://github.com/ultralytics/yolov5, 2020

    2020

  28. [36]

    Open-vocabulary panoptic segmentation with text-to-image diffusion models,

    J. Xu, S. Liu, A. Vahdat, W. Byeon, X. Wang, and S. D. Mello, “Open-vocabulary panoptic segmentation with text-to-image diffusion models,” in Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR) , 2023

    2023

  29. [37]

    Towards an end- to-end framework for flow-guided video inpainting,

    Z. Li, C. Lu, J. Qin, C. Guo, and M. Cheng, “Towards an end- to-end framework for flow-guided video inpainting,” in Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR) , 2022

    2022

  30. [38]

    Graspnet-1billion: A large-scale benchmark for general object grasping,

    H.-S. Fang, C. Wang, M. Gou, and C. Lu, “Graspnet-1billion: A large-scale benchmark for general object grasping,” in Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR) , 2020

    2020

  31. [39]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  32. [40]

    Eureka: Human-level reward design via coding large language models,

    Y . J. Ma, W. Liang, G. Wang, D. Huang, O. Bastani, D. Jayaraman, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human-level reward design via coding large language models,” in Proceedings of Interna- tional Conference on Learning Representations (ICLR) , 2024

    2024

  33. [41]

    Spearman, The proof and measurement of association between two things

    C. Spearman, The proof and measurement of association between two things. Appleton-Century-Crofts, 1961

    1961

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.