pith. machine review for the scientific record. sign in

arxiv: 2108.10470 · v2 · submitted 2021-08-24 · 💻 cs.RO · cs.LG

Recognition: 2 theorem links

Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning

Ankur Handa, Arthur Allshire, David Hoeller, Gavriel State, Kier Storey, Lukasz Wawrzyniak, Michelle Lu, Miles Macklin, Nikita Rudin, Viktor Makoviychuk, Yunrong Guo

Pith reviewed 2026-05-12 21:39 UTC · model grok-4.3

classification 💻 cs.RO cs.LG
keywords GPU physics simulationrobot learningreinforcement learningroboticsIsaac GymPyTorchparallel trainingphysics engine
0
0 comments X

The pith

Isaac Gym trains robot policies entirely on one GPU by moving data directly between physics buffers and PyTorch tensors.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents Isaac Gym as a platform that runs both physics simulation of robot environments and neural network policy training on the same GPU. Data passes directly from the simulator's memory buffers to PyTorch tensors, bypassing any CPU involvement. This design targets the data transfer overhead that slows conventional reinforcement learning setups where a CPU simulator feeds a GPU-based learner. A sympathetic reader would care because the change could make training intricate robot behaviors practical on single-GPU hardware rather than requiring large CPU clusters.

Core claim

Isaac Gym offers a high performance learning platform to train policies for a wide variety of robotics tasks directly on GPU. Both physics simulation and the neural network policy training reside on GPU and communicate by directly passing data from physics buffers to PyTorch tensors without ever going through any CPU bottlenecks. This leads to blazing fast training times for complex robotics tasks on a single GPU with 2-3 orders of magnitude improvements compared to conventional RL training that uses a CPU based simulator and GPU for neural networks.

What carries the argument

Direct sharing of GPU memory buffers between the physics simulator and PyTorch tensors, keeping all computation on the GPU.

If this is right

  • Complex robotics tasks become trainable on a single GPU instead of distributed CPU clusters.
  • Reinforcement learning loops for robot control avoid all CPU-to-GPU data copies during each training step.
  • Simulation and learning can run in tight parallel without synchronization delays from host-device transfers.
  • The same framework supports a wide range of robotics tasks through its integrated GPU simulator.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar direct-buffer designs could apply to other simulation-heavy domains such as molecular modeling or fluid dynamics.
  • The speed increase might allow individual labs to explore longer training runs or larger robot fleets without shared compute resources.
  • If policies transfer well, the method could shorten the usual sim-to-real iteration cycle by reducing the time between experiments.

Load-bearing premise

The GPU physics engine must produce accurate and stable results that match real-world behavior closely enough for learned policies to succeed when moved to physical robots.

What would settle it

Train a policy in Isaac Gym for a known robotics task, then deploy it on the corresponding physical robot and measure whether performance matches simulation predictions within acceptable error.

read the original abstract

Isaac Gym offers a high performance learning platform to train policies for wide variety of robotics tasks directly on GPU. Both physics simulation and the neural network policy training reside on GPU and communicate by directly passing data from physics buffers to PyTorch tensors without ever going through any CPU bottlenecks. This leads to blazing fast training times for complex robotics tasks on a single GPU with 2-3 orders of magnitude improvements compared to conventional RL training that uses a CPU based simulator and GPU for neural networks. We host the results and videos at \url{https://sites.google.com/view/isaacgym-nvidia} and isaac gym can be downloaded at \url{https://developer.nvidia.com/isaac-gym}.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The paper presents Isaac Gym, a high-performance GPU-based physics simulation platform for robot learning. Both the physics simulation and neural network policy training run entirely on the GPU, with direct passing of data from physics buffers to PyTorch tensors to eliminate CPU bottlenecks. This architecture is claimed to deliver 2-3 orders of magnitude faster training times for complex robotics tasks on a single GPU compared to conventional RL setups that use CPU-based simulators paired with GPU-based networks. Results, videos, and the software are made publicly available via linked resources.

Significance. If the reported speedups hold under rigorous verification, the work has substantial significance for robotics and reinforcement learning. The seamless GPU integration for both simulation and learning removes a key bottleneck, enabling faster iteration on complex tasks with fewer resources. Explicit credit is due for releasing a downloadable implementation, hosting results and videos, and providing direct PyTorch integration, which supports reproducibility and adoption. This could accelerate research in sim-to-real transfer and large-scale policy training.

major comments (2)
  1. [Abstract and experimental results] Abstract and experimental results: The central claim of 2-3 orders of magnitude speedup over conventional RL training is load-bearing but rests on comparisons whose methodology is not fully specified. No details are given on the baseline simulator (e.g., MuJoCo version or custom code), CPU hardware, number of parallel environments, or exact wall-clock measurement protocol. This prevents determining whether gains derive purely from GPU buffer sharing or from unoptimized baselines.
  2. [System description] System description: The direct GPU-to-PyTorch buffer sharing is presented as introducing no hidden costs, yet there is limited analysis of potential synchronization overheads, numerical artifacts, or precision differences between the GPU physics engine and standard CPU simulators. This is relevant to the claim that policies trained in Isaac Gym transfer reliably.
minor comments (3)
  1. [Abstract] The abstract summarizes results without referencing specific tables, figures, or sections that detail the benchmarks, which would aid quick assessment of the evidence.
  2. [Related work] Related work section could include more citations to prior GPU-accelerated physics engines and vectorized simulators to better contextualize the contribution.
  3. [Figures and tables] Figure captions and table legends should explicitly define the speedup metric (e.g., steps per second or wall-clock time to convergence) and list the exact environment counts used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive review and for recognizing the potential impact of our work along with the value of the public release. We address each major comment below and have revised the manuscript accordingly to improve clarity on experimental methodology and system analysis.

read point-by-point responses

  1. Referee: [Abstract and experimental results] Abstract and experimental results: The central claim of 2-3 orders of magnitude speedup over conventional RL training is load-bearing but rests on comparisons whose methodology is not fully specified. No details are given on the baseline simulator (e.g., MuJoCo version or custom code), CPU hardware, number of parallel environments, or exact wall-clock measurement protocol. This prevents determining whether gains derive purely from GPU buffer sharing or from unoptimized baselines.

    Authors: We agree that the experimental methodology requires more explicit specification. In the revised manuscript we have added a dedicated 'Experimental Setup' subsection that details the baseline as MuJoCo 2.1 accessed via the standard Gym interface, the CPU hardware (dual Intel Xeon Gold 6248R CPUs), the range of parallel environments (1 to 4096), and the wall-clock timing protocol (host-side high-resolution timers combined with CUDA events for GPU operations). The baselines follow standard RL library implementations (e.g., Stable-Baselines3 with default hyperparameters) without custom optimizations. Ablation experiments already present in the paper show that speedup scales with environment count only under the direct GPU buffer-sharing architecture, supporting that the gains are not solely from unoptimized baselines. revision: yes

  2. Referee: [System description] System description: The direct GPU-to-PyTorch buffer sharing is presented as introducing no hidden costs, yet there is limited analysis of potential synchronization overheads, numerical artifacts, or precision differences between the GPU physics engine and standard CPU simulators. This is relevant to the claim that policies trained in Isaac Gym transfer reliably.

    Authors: We concur that additional analysis strengthens the claims. The revised manuscript includes a new subsection titled 'GPU Buffer Sharing Overhead and Numerical Consistency' that reports profiling results from NVIDIA Nsight showing synchronization overhead below 3% of total runtime via asynchronous CUDA streams. All computations use single-precision floating point, consistent with common CPU simulator practice; we added quantitative comparisons confirming no measurable policy performance degradation. We also include new sim-to-real transfer results for a quadruped locomotion task demonstrating comparable success rates between Isaac Gym-trained policies and MuJoCo-trained policies when deployed on hardware. revision: yes

Circularity Check

0 steps flagged

No circularity: implementation paper with external empirical benchmarks

full rationale

The paper describes a GPU-based physics simulation system (Isaac Gym) for robot learning, with no mathematical derivation chain, equations, predictions, or fitted parameters. Claims rest on direct GPU buffer sharing between physics and PyTorch, evaluated via wall-clock comparisons to external conventional CPU-based RL setups rather than any self-referential construction. No self-definitional steps, fitted-input predictions, load-bearing self-citations, or ansatz smuggling appear; the contribution is an implemented artifact whose performance is measured against independent baselines.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper is an engineering systems contribution rather than a theoretical derivation. It relies on standard rigid-body dynamics and GPU programming primitives already established in prior literature.

axioms (1)
  • domain assumption Rigid-body Newtonian dynamics provide a sufficient model for the targeted robotics tasks
    Invoked implicitly by any physics simulator used for robot learning; no new physics is derived.

pith-pipeline@v0.9.0 · 5447 in / 1183 out tokens · 45400 ms · 2026-05-12T21:39:24.424114+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 31 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Self-Supervised On-Policy Reinforcement Learning via Contrastive Proximal Policy Optimisation

    cs.LG 2026-05 unverdicted novelty 7.0

    CPPO is an on-policy contrastive RL method that derives advantages from contrastive Q-values for PPO optimization, outperforming prior CRL baselines in 14/18 tasks and matching or exceeding reward-based PPO in 12/18 tasks.

  2. Coordinated Diffusion: Generating Multi-Agent Behavior Without Multi-Agent Demonstrations

    cs.RO 2026-05 unverdicted novelty 7.0

    CoDi decomposes the multi-agent diffusion score into pre-trained single-agent policies plus a gradient-free cost guidance term to generate coordinated behavior from single-agent data alone.

  3. Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllers

    cs.CV 2026-05 unverdicted novelty 7.0

    A two-stage framework augments HOI data with dynamic priors and blends pre-trained dynamic motion and static interaction agents via a composer network to enable long-term dynamic human-object interactions with higher ...

  4. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 accept novelty 7.0

    3D generation for embodied AI is shifting from visual realism toward interaction readiness, organized into data generation, simulation environments, and sim-to-real bridging roles.

  5. HiPAN: Hierarchical Posture-Adaptive Navigation for Quadruped Robots in Unstructured 3D Environments

    cs.RO 2026-04 unverdicted novelty 7.0

    HiPAN enables quadruped robots to navigate unstructured 3D environments more successfully by combining a high-level posture-adaptive policy with a low-level controller and curriculum learning on depth images.

  6. HANDFUL: Sequential Grasp-Conditioned Dexterous Manipulation with Resource Awareness

    cs.RO 2026-04 unverdicted novelty 7.0

    HANDFUL learns resource-aware grasps using finger contact rewards and curriculum learning to improve success on sequential dexterous tasks in simulation and on a real LEAP hand.

  7. SECOND-Grasp: Semantic Contact-guided Dexterous Grasping

    cs.RO 2026-05 conditional novelty 6.0

    SECOND-Grasp integrates semantic contact proposals from vision-language reasoning with geometric refinement to achieve 98%+ lifting success and improved intent-aware grasping on seen and unseen objects.

  8. NavOL: Navigation Policy with Online Imitation Learning

    cs.RO 2026-05 unverdicted novelty 6.0

    NavOL collects expert trajectory labels online from a global planner during policy rollouts in simulation to train a diffusion navigation policy, mitigating distribution shift and improving performance on visual navig...

  9. Explicit Stair Geometry Conditioning for Robust Humanoid Locomotion

    cs.RO 2026-05 unverdicted novelty 6.0

    Explicit conditioning of a PPO policy on interpretable stair parameters (height, depth, yaw) yields improved generalization to unseen stairs and reliable real-world traversal on the Unitree G1, including 33 consecutiv...

  10. Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching

    cs.RO 2026-05 unverdicted novelty 6.0

    DRIS improves zero-shot sim-to-real transfer for reactive catching by maintaining and acting on sets of randomized dynamics instances instead of single instances per episode.

  11. RigidFormer: Learning Rigid Dynamics using Transformers

    cs.CV 2026-05 unverdicted novelty 6.0

    RigidFormer learns mesh-free rigid dynamics from point clouds using object-centric anchors, Anchor-Vertex Pooling, Anchor-based RoPE, and differentiable Kabsch alignment to enforce rigidity.

  12. ANO: A Principled Approach to Robust Policy Optimization

    cs.AI 2026-05 unverdicted novelty 6.0

    ANO derives a robust policy optimizer from geometric principles that replaces clipping with a smooth redescending gradient, showing better performance and stability than PPO, SPO, and GRPO in MuJoCo, Atari, and RLHF e...

  13. GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning

    cs.RO 2026-04 unverdicted novelty 6.0

    GS-Playground delivers a high-throughput photorealistic simulator for vision-informed robot learning via parallel physics integrated with batch 3D Gaussian Splatting at 10^4 FPS and an automated Real2Sim workflow for ...

  14. dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model

    cs.RO 2026-04 unverdicted novelty 6.0

    A discrete diffusion model tokenizes multimodal robotic data and uses a progress token to predict future states and task completion for scalable policy evaluation.

  15. Learn Weightlessness: Imitate Non-Self-Stabilizing Motions on Humanoid Robot

    cs.RO 2026-04 unverdicted novelty 6.0

    The Weightlessness Mechanism lets humanoid robots imitate non-self-stabilizing motions by dynamically relaxing specific joints to exploit passive environmental contacts, generalizing from single demonstrations to vari...

  16. ETac: A Lightweight and Efficient Tactile Simulation Framework for Learning Dexterous Manipulation

    cs.RO 2026-04 unverdicted novelty 6.0

    ETac is a data-driven tactile simulation framework that matches FEM deformation accuracy at high speed, supporting 4096 parallel environments at 869 FPS and yielding 84.45% success in blind grasping across four object types.

  17. FLASH: Fast Learning via GPU-Accelerated Simulation for High-Fidelity Deformable Manipulation in Minutes

    cs.RO 2026-04 unverdicted novelty 6.0

    A new GPU-accelerated deformable simulation framework trains manipulation policies in minutes using only synthetic data, achieving robust zero-shot transfer to physical robots.

  18. Chain of Uncertain Rewards with Large Language Models for Reinforcement Learning

    cs.LG 2026-04 unverdicted novelty 6.0

    CoUR uses LLMs for efficient RL reward design through uncertainty quantification and similarity selection, achieving better performance and lower evaluation costs on IsaacGym and Bidexterous Manipulation benchmarks.

  19. Trajectory-based actuator identification via differentiable simulation

    cs.RO 2026-04 unverdicted novelty 6.0

    Differentiable simulation enables torque-sensor-free actuator model identification from trajectory data, achieving 1.88x better position tracking than a stand-trained baseline and 46% longer travel in downstream locom...

  20. FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control

    cs.LG 2026-04 unverdicted novelty 6.0

    FlashSAC scales up Soft Actor-Critic with fewer updates, larger models, higher data throughput, and norm bounds to deliver faster, more stable training than PPO on high-dimensional robot control tasks across dozens of...

  21. Veo-Act: How Far Can Frontier Video Models Advance Generalizable Robot Manipulation?

    cs.RO 2026-04 unverdicted novelty 6.0

    Veo-3 video predictions enable approximate task-level robot trajectories in zero-shot settings but require hierarchical integration with low-level VLA policies for reliable manipulation performance.

  22. SimART: A Unified and Open Real-world Multimodal Simulation Platform for 6G Integrated Sensing and Communication

    eess.SP 2026-05 unverdicted novelty 5.0

    SimART is an open platform that unifies robotics, ray tracing, and wireless tools via ROS for reproducible multimodal simulation in 6G integrated sensing and communication.

  23. REAP: Reinforcement-Learning End-to-End Autonomous Parking with Gaussian Splatting Simulator for Real2Sim2Real Transfer

    cs.RO 2026-05 unverdicted novelty 5.0

    REAP trains an end-to-end SAC policy with behavior cloning and collision penalties inside a 3DGS Real2Sim simulator and transfers it to physical vehicles, succeeding in narrow mechanical parking slots.

  24. Finite-Step Invariant Sets for Hybrid Systems with Probabilistic Guarantees

    eess.SY 2026-04 unverdicted novelty 5.0

    A sampling-based optimization framework computes finite-step invariant ellipsoids for hybrid system return maps with user-specified probabilistic guarantees on invariance.

  25. Edge Deep Learning in Computer Vision and Medical Diagnostics: A Comprehensive Survey

    cs.CV 2026-05 unverdicted novelty 4.0

    A comprehensive survey of edge deep learning in computer vision and medical diagnostics that presents a novel categorization of hardware platforms by performance and usage scenarios.

  26. Robotic Affection -- Opportunities of AI-based haptic interactions to improve social robotic touch through a multi-deep-learning approach

    cs.HC 2026-05 unverdicted novelty 4.0

    A position paper proposes decomposing affective robotic touch into multiple specialized deep learning models for better social human-robot interaction.

  27. Quadruped Parkour Learning: Sparsely Gated Mixture of Experts with Visual Input

    cs.RO 2026-04 unverdicted novelty 4.0

    Sparsely gated MoE policies double the success rate of a real Unitree Go2 quadruped on large-obstacle parkour versus matched-active-parameter MLP baselines while cutting inference time compared with a scaled-up MLP.

  28. Micro-Dexterity in Biological Micromanipulation: Embodiment, Perception, and Control

    cs.RO 2026-04 unverdicted novelty 4.0

    The paper introduces micro-dexterity as a framework for biological micromanipulation by reformulating classical primitives in fluidic, surface-dominated micro-environments and comparing contact-based, field-mediated, ...

  29. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 unverdicted novelty 3.0

    The survey organizes 3D generation for embodied AI into data generators for assets, simulation environments for interaction, and sim-to-real bridges, noting a shift toward interaction readiness and listing bottlenecks...

  30. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 unverdicted novelty 2.0

    The paper surveys 3D generation techniques for embodied AI and robotics, categorizing them into data generation, simulation environments, and sim-to-real bridging while identifying bottlenecks in physical validity and...

  31. A Survey of Legged Robotics in Non-Inertial Environments: Past, Present, and Future

    cs.RO 2026-04 unverdicted novelty 2.0

    A literature survey summarizing modeling, state estimation, control methods, applications, and open challenges for legged robots operating in non-inertial environments where the ground moves or accelerates.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages · cited by 29 Pith papers · 1 internal anchor

  1. [1]

    A general reinforcement learning algorithm that masters chess, shogi, and go through self-play

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 2018

    work page 2018

  2. [2]

    A survey of real-time strategy game ai research and competition in starcraft

    Santiago Ontanón, Gabriel Synnaeve, Alberto Uriarte, Florian Richoux, David Churchill, and Mike Preuss. A survey of real-time strategy game ai research and competition in starcraft. IEEE Transactions on Computational Intelligence and AI in games , 5(4):293–311, 2013

    work page 2013

  3. [3]

    Dota 2 with Large Scale Deep Reinforcement Learning

    Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemysław D˛ ebiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680, 2019

    work page internal anchor Pith review arXiv 1912

  4. [4]

    Learning Agile Robotic Locomotion Skills by Imitating Animals

    Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Edward Lee, Jie Tan, and Sergey Levine. Learning Agile Robotic Locomotion Skills by Imitating Animals. In Robotics: Science and Systems, 07 2020. doi: 10.15607/RSS.2020.XVI.064

    work page doi:10.15607/rss.2020.xvi.064 2020

  5. [5]

    Pachocki, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, Jonas Schneider, Szymon Sidor, Josh Tobin, Peter Welinder, Lilian Weng, and Wojciech Zaremba

    OpenAI, Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Józefowicz, Bob McGrew, Jakub W. Pachocki, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, Jonas Schneider, Szymon Sidor, Josh Tobin, Peter Welinder, Lilian Weng, and Wojciech Zaremba. Learning dexterous in-hand manipulation. CoRR, abs/1808.00177, 2018. URL http://ar...

    work page arXiv 2018

  6. [6]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 5026–5033. IEEE, 2012

    work page 2012

  7. [7]

    Pybullet, a python module for physics simulation for games, robotics and machine learning, 2016

    Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning, 2016. URL http://pybullet. org, 2016

    work page 2016

  8. [8]

    Dart: Dynamic animation and robotics toolkit

    Jeongseok Lee, Michael X Grey, Sehoon Ha, Tobias Kunz, Sumit Jain, Yuting Ye, Siddhartha S Srinivasa, Mike Stilman, and C Karen Liu. Dart: Dynamic animation and robotics toolkit. Journal of Open Source Software, 2018

    work page 2018

  9. [9]

    Drake: Model-based design and verification for robotics, 2019

    Russ Tedrake and the Drake Development Team. Drake: Model-based design and verification for robotics, 2019. URL https://drake.mit.edu

    work page 2019

  10. [10]

    V-rep: A versatile and scalable robot simulation framework

    Eric Rohmer, Surya PN Singh, and Marc Freese. V-rep: A versatile and scalable robot simulation framework. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 1321–1326. IEEE, 2013

    work page 2013

  11. [11]

    Solving rubik’s cube with a robot hand,

    Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. Solving Rubik’s Cube with a Robot Hand. arXiv preprint arXiv:1910.07113, 2019

    work page arXiv 1910

  12. [12]

    Gpu-accelerated robotic simulation for distributed reinforcement learning

    Jacky Liang, Viktor Makoviychuk, Ankur Handa, Nuttapong Chentanez, Miles Macklin, and Dieter Fox. Gpu-accelerated robotic simulation for distributed reinforcement learning. In Conference on Robot Learning. PMLR, 2018

    work page 2018

  13. [13]

    Nvidia PhysX, 2020

    NVIDIA. Nvidia PhysX, 2020. URL https://developer.nvidia.com/physx-sdk

    work page 2020

  14. [14]

    Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem

    C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation, 2021. URL http://github.com/google/brax

    work page 2021

  15. [15]

    Domain randomization for transferring deep neural networks from simulation to the real world,

    Joshua Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. CoRR, abs/1703.06907, 2017. URL http://arxiv.org/abs/1703.06907

    work page arXiv 2017

  16. [16]

    Hutter, Christian Gehring, Dominic Jud, Andreas Lauber, Dario Bellicoso, Vassilios Tsounis, Jemin Hwangbo, K

    M. Hutter, Christian Gehring, Dominic Jud, Andreas Lauber, Dario Bellicoso, Vassilios Tsounis, Jemin Hwangbo, K. Bodie, P. Fankhauser, Michael Bloesch, Remo Diethelm, Samuel Bachmann, A. Melzer, and M. Höpflinger. Anymal - a highly mobile and dynamic quadrupedal robot. (IROS), 2016. 22

    work page 2016

  17. [17]

    AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control

    Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control. ACM Trans. Graph., 2021

    work page 2021

  18. [18]

    Small steps in physics simulation

    Miles Macklin, Kier Storey, Michelle Lu, Pierre Terdiman, Nuttapong Chentanez, Stefan Jeschke, and Matthias Müller. Small steps in physics simulation. In Proceedings of the 18th Annual ACM SIGGRAPH/Eurographics Symposium on Computer Animation , SCA ’19, New York, NY , USA, 2019. Association for Computing Machinery. doi: 10.1145/3309486.3340247. URL https:...

    work page doi:10.1145/3309486.3340247 2019

  19. [19]

    Proximal Policy Optimization Algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, 2017

    work page 2017

  20. [20]

    RL Games, 2021

    Denys Makoviichuk and Viktor Makoviychuk. RL Games, 2021. URL https://github. com/Denys88/rl_games/

    work page 2021

  21. [21]

    Asymmetric Actor Critic for Image-Based Robot Learning

    Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, and Pieter Abbeel. Asymmetric actor critic for image-based robot learning. CoRR, 2017. URL http://arxiv. org/abs/1710.06542

    work page Pith review arXiv 2017

  22. [22]

    Learning Agile and Dynamic Motor Skills for Legged Robots

    Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter. Learning Agile and Dynamic Motor Skills for Legged Robots. Science Robotics, Jan 2019

    work page 2019

  23. [23]

    Learning to walk in minutes using massively parallel deep reinforcement learning

    Anonymous. Learning to walk in minutes using massively parallel deep reinforcement learning. In Submitted to 5th Annual Conference on Robot Learning , 2021. URL https://openreview. net/forum?id=wK2fDDJ5VcF. under review

    work page 2021

  24. [24]

    Deepmimic: Example-guided deep re- inforcement learning of physics-based character skills

    Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. Deepmimic: Example- guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph., 37 (4), July 2018. doi: 10.1145/3197517.3201311

    work page doi:10.1145/3197517.3201311 2018

  25. [25]

    O. Khatib. A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE Journal on Robotics and Automation , 3(1):43–53, 1987. doi: 10.1109/JRA.1987.1087068

    work page doi:10.1109/jra.1987.1087068 1987

  26. [26]

    robosuite: A modular simulation framework and benchmark for robot learning, 2020

    Yuke Zhu, Josiah Wong, Ajay Mandlekar, and Roberto Martín-Martín. robosuite: A modular simulation framework and benchmark for robot learning, 2020

    work page 2020

  27. [27]

    Lee, Rachel Gardner, Silvio Savarese, Jeannette Bohg, and Animesh Garg

    Roberto Martín-Martín, Michelle A. Lee, Rachel Gardner, Silvio Savarese, Jeannette Bohg, and Animesh Garg. Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks, 2019

    work page 2019

  28. [28]

    TriFin- ger: An Open-Source Robot for Learning Dexterity

    Manuel Wüthrich, Felix Widmaier, Felix Grimminger, Joel Akpo, Shruti Joshi, Vaibhav Agrawal, Bilal Hammoud, Majid Khadiv, Miroslav Bogdanovic, Vincent Berenz, Julian Viereck, Maximilien Naveau, Ludovic Righetti, Bernhard Schölkopf, and Stefan Bauer. TriFin- ger: An Open-Source Robot for Learning Dexterity. CoRR, abs/2008.03596, 2020. URL https://arxiv.org...

    work page arXiv 2008

  29. [29]

    Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger

    Arthur Allshire, Mayank Mittal, Varun Lodaya, Viktor Makoviychuk, Denys Makoviichuk, Felix Widmaier, Manuel Wuthrich, Stefan Bauer, Ankur Handa, and Animesh Garg. Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger. CoRR, 2021

    work page 2021

  30. [30]

    Schaff, Rishabh Madan, Takuma Yoneda, Julen Urain De Jesus, Joe Watson, Ethan K

    Niklas Funk, Charles B. Schaff, Rishabh Madan, Takuma Yoneda, Julen Urain De Jesus, Joe Watson, Ethan K. Gordon, Felix Widmaier, Stefan Bauer, Siddhartha S. Srinivasa, Tapomayukh Bhattacharjee, Matthew R. Walter, and Jan Peters. Benchmarking structured policies and policy optimization for real-world dexterous object manipulation. CoRR, abs/2105.02087, 202...

    work page arXiv 2021

  31. [31]

    J. D. Hunter. Matplotlib: A 2d graphics environment. Computing in Science & Engineering , pages 90–95, 2007

    work page 2007

  32. [32]

    Guido Van Rossum and Fred L. Drake. Python 3 Reference Manual. CreateSpace, Scotts Valley, CA, 2009. 23

    work page 2009

  33. [33]

    Harris and K

    Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fer- nández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin She...

    work page doi:10.1038/s41586-020-2649-2 2020

  34. [34]

    Pytorch: An imperative style, high- performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- perf...

    work page 2019

  35. [35]

    Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murra...

    work page 2015

  36. [36]

    TensorBoard Aggregator, February 2021

    Sebastian Penhouet. TensorBoard Aggregator, February 2021. URL https://github.com/ Spenhouet/tensorboard-aggregator

    work page 2021

  37. [37]

    Garrett and Hsin-Hsiang Peng

    John D. Garrett and Hsin-Hsiang Peng. garrettj403/SciencePlots, February 2021. URL http: //doi.org/10.5281/zenodo.4106649

    work page doi:10.5281/zenodo.4106649 2021

  38. [38]

    URL https://www.overleaf.com/

    Overleaf, 2012. URL https://www.overleaf.com/

    work page 2012

  39. [39]

    Ahmed, F

    Ossama Ahmed, Frederik Träuble, Anirudh Goyal, Alexander Neitz, Manuel Wüthrich, Yoshua Bengio, Bernhard Schölkopf, and Stefan Bauer. CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning. CoRR, abs/2010.04296, 2020. URL https://arxiv.org/abs/2010.04296. 24 A Appendix A.1 Tendons We simulate tendons as part of the Shadow...

    work page arXiv 2010