Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

DoorGym: A Scalable Door Opening Environment And Baseline Agent

T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DoorGym claims a domain-randomized door-opening simulator can train a policy that transfers to a real door.

desk verdict A useful open-source door-opening benchmark whose baseline results and DR ablation are worth having, but the real-world transfer claim is under-specified and the evaluation reporting needs tightening. read the letter →

arxiv 1908.01887 v4 pith:GTWGN37L submitted 2019-08-05 cs.RO cs.AIcs.LG

Yusuke Urakami , Alec Hodgkinson , Casey Carlin , Randall Leu , Luca Rigazio , Pieter Abbeel This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords DoorGymopeningdomainrandomizationsim-to-realtransferreinforcementlearningrobotmanipulationProximalPolicyOptimizationSoftActor-Critic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces DoorGym, a simulated door-opening environment built for domain randomization, and argues that policies trained across its randomized door worlds become robust enough to open unseen simulated doors and, in one zero-shot experiment, a real wooden door. The authors report a PPO baseline achieving success rates between 0% and 95% depending on knob type and robot configuration, with the hardest cases (round knobs, lever knobs with certain arms) often failing completely. The transfer experiment reports 70% success in simulation and 59% on a real Baxter robot, which the paper presents as evidence that the randomized training recipe carries over to reality. The aim is to give the reinforcement-learning community an accessible benchmark that sits at the intersection of domain transfer, practical manipulator tasks, and realism.

What carries the argument

The load-bearing object is DoorGym's randomized door-world generator, which resamples a new simulated world for every episode: door dimensions, mass, hinge side, opening direction, knob type (pull, lever, or round), knob geometry from a CAD-generated dataset, robot joint damping, lighting, and material appearance all draw from specified uniform ranges. Behind it sits a shaped reward that drives the end-effector toward the knob, aligns fingertip orientation, penalizes control effort, and rewards door and knob rotation, plus a vision network that estimates the knob's 3D position from top and front camera views and feeds a direction vector into a small policy network. PPO and SAC serve as the baseline algorithms, with PPO the stronger exploitative policy in these experiments. The generator is what the transfer claim rests on: it is the mechanism that produces a distribution broad enough to overlap the real door.

What would settle it

Inspect the released DoorGym code for a Baxter arm model and reproduce the Section 5 experiment with the arm used in training; if no Baxter model exists, or if deploying the trained policy requires converting its action space, the sim-to-real transfer claim fails as reported. A simpler check is to rerun the transfer with the same arm inside and outside simulation and compare the 70%/59% gap.

Watch

Extended reading notes

Core claim

The paper's central claim is that domain randomization over door geometry, knob type, physical properties, and visuals is sufficient to train a single reinforcement-learning policy that generalizes across a distribution of doors. With ground-truth knob position, a PPO agent trained on randomized worlds reaches 0.95 success on a pull-knob task with a floating hook, 0.68 on a lever knob with a floating hook, and 0.71 on a pull knob with a gripper, all on 100 unseen test worlds. Replacing ground truth with a learned vision estimator lowers these numbers but keeps some tasks viable, and the same training recipe transfers zero-shot to a real pull-knob door at 0.59 success. The paper also claims that domain randomization is necessary for this transfer: policies and vision networks trained on a single environment collapse on new domains.

Load-bearing premise

The real-world experiment assumes the policy trained in the simulator can be run on the Baxter robot even though the paper does not state which robot model was trained for that deployment; if the simulator arm and the real arm do not share the same action space and kinematics, the 59% transfer number is not interpretable as stated.

Editorial extensions

If this is right

  • If the claims hold, a single PPO policy checkpoint can open a range of randomized doors without resetting or retraining, making DoorGym a ready benchmark for comparing door-opening algorithms.
  • The large gap between ground-truth and vision-estimated success rates implies that improvements in 3D knob perception, not control, are the clearest route to raising overall performance.
  • The ablation showing single-environment training collapses from 46% to 0% vision-based success on randomized worlds indicates that domain randomization is not optional for this task's sim-to-real goals.
  • SAC's near-zero success on lever and round knobs means the baseline results are algorithm-dependent; reporting both on-policy and off-policy baselines lets users choose according to their sample-efficiency needs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's Section 5 uses a Baxter robot without stating whether DoorGym was given a Baxter model; if the deployed policy was trained on the BLUE arm, the 59% result suggests an unstated cross-embodiment transfer that would be worth isolating in a controlled experiment.
  • The reported 'about 3 cm localization error makes opening hard, above 5 cm impossible' threshold suggests a concrete design target: a perception system good to within a few centimeters is a prerequisite for door-open RL, and DoorGym could be used to score that target directly.
  • The same randomized-world generator could be repurposed for other articulated household objects such as cabinets or drawers, since the knob, hinge, and door parameterization generalizes, though the paper does not claim this.
  • A natural next experiment the paper leaves implicit is to combine DoorGym with automatic domain-randomization methods that adapt randomization ranges based on task success; that would test whether the current fixed ranges are near-optimal or merely sufficient.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces DoorGym, an open-source MuJoCo/Gym-based door-opening environment designed for domain randomization (DR), and provides PPO and SAC baselines. The environment includes multiple doorknob types (pull, lever, round), randomized door and robot parameters, and a vision network for estimating doorknob position. The authors report success rates on three tasks (pull knob with floating hook, lever with floating hook, pull knob with gripper), an ablation showing that DR improves policy and vision-network transfer, and a zero-shot sim-to-real transfer experiment on a Baxter robot that achieves a 59% real-world success rate. The central claims are that DoorGym provides a usable DR benchmark and that a policy trained purely in simulation can transfer to a real door.

Significance. If the claims hold, DoorGym fills a useful niche as an open-source, domain-randomized benchmark for a practical manipulation task, complementing existing sim-to-real environments. The paper provides several concrete strengths: the environment code is released, the randomization ranges are tabulated (Table 5), the baseline PPO/SAC implementations are described, the DR ablation supports the qualitative conclusion that DR improves transfer, and the real-world vision-network and policy experiments attempt to validate sim-to-real transfer. The PPO results (up to 95% success on a pull-knob task with ground-truth position) indicate the environment is solvable by a standard algorithm. However, the real-world transfer section does not specify which simulated robot the deployed policy was trained on, which is load-bearing for the paper's strongest claim. In addition, all quantitative results are single-seed point estimates, so the statistical reliability of the reported differences is unknown.

major comments (3)
  1. [Section 5, Table 3] The real-world transfer experiment uses a Baxter robot, but Section 2.2 describes only the Berkeley BLUE Robot arm, and all Section 4 experiments and Tables 8/9 use BLUE arm variants (hook, gripper, floating, mobile). No Baxter model in DoorGym is described, and no action/observation mapping from BLUE to Baxter is given. Because observations are per-joint positions/velocities and actions are per-DoF forces/torques (Section 3), a BLUE-trained policy has different dimensionality and kinematics than a Baxter right arm, so direct deployment requires either a documented Baxter simulation model or an explicit retargeting scheme. Neither is present. Moreover, the simulator success rate of 0.70 in Table 3 does not match any condition in Table 1 (PPO with vision is 0.48–0.57 on the three tasks, and PPO with ground-truth position is 0.71–0.95), so even the simulator baseline for the deployed policy is not identifiable. This gap makes the real-world transfer claim unreproducible and undermines the paper's central contribution of zero-shot sim-to-real transfer.
  2. [Tables 1, 2, 3, 8, 9] All reported success rates and average opening times are point estimates from a single training run, with no variance, confidence intervals, or number of random seeds. For a benchmark intended to support comparisons (PPO vs. SAC, DR vs. no-DR, simulator vs. real), the absence of uncertainty makes it impossible to assess whether differences such as 0.95 vs. 0.68 (Task 1 vs. Task 2 PPO) or 0.71 vs. 0.59 (Table 3) are meaningful or within run-to-run noise. This limits the strength of the quantitative claims and is particularly important for the DR ablation, where the conclusion depends on the gap between single-environment and randomized training.
  3. [Section 4.5, Table 2] The DR ablation compares a policy trained on 'env1' with a policy trained on randomized environments, but env1's parameters are not specified, the test conditions are only 'env1' and 'randomized env', and no statistical uncertainty is reported. The paper concludes that 'DR is required' for both policy and vision-network robustness, yet the evidence is limited to two conditions per comparison and single seeds. While the qualitative direction (single-env policies generalize poorly) is plausible and consistent with prior work, the quantitative support is weaker than the text implies. Please report the randomization parameters of env1, multiple seeds, and standard deviations, or soften the claim to 'DR improves transfer in these experiments.'
minor comments (7)
  1. [Section 4.4, paragraph 1] The text states that 'All experiments are evaluated using the unshaped reward as defined in section 4.1,' but Eq. (1) is a shaped reward with weights a0–a5; 'unshaped' appears to be a typo for 'shaped' or the sentence should be rephrased.
  2. [Table 1] The SAC row for 'Ground Truth Position + N(0, σ)' is filled with dashes, and the text does not explain whether this condition was not run or the results were omitted; please clarify.
  3. [Section 2.2, Figure 3] The text says '6 different arm/gripper combinations' but Figure 3 shows four arm types (8DoF BLUE with grippers, 7DoF BLUE with hook, 7DoF floating gripper, 6DoF floating hook), with mobile variants mentioned in the text; the count and the distinction between fixed and mobile platforms should be made explicit.
  4. [Section 5, paragraph 1] The sentence 'the arm starts from the hanging position' is ambiguous; specify whether this is a known joint configuration and whether it matches the initialization distribution used in simulation.
  5. [References] Reference [2] is incomplete: the venue is listed as 'In None, 2015,' and the title appears truncated; please provide the full citation.
  6. [Figure 1 caption] The caption contains a typo: 'ramdomizable' should be 'randomizable'.
  7. [Section 5, Table 3] The real-world success criterion is the same as simulation (door opening >0.2 rad within 20 s), but the control-loop frequency and the number of real-world evaluations are reported; please also state whether the 100 evaluations used the same door and whether the robot was reset to the same initial configuration each time.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the environment, baselines, and transfer results are empirical evaluations rather than derivations from their own inputs.

full rationale

DoorGym makes no first-principles derivation claim; its central claims are empirical measurements of trained policies in randomized and real environments. The domain randomization training distribution and the 100 test door-worlds are generated from the same randomization ranges specified in Table 5, which is a standard evaluation protocol rather than a circular reduction. The DR ablation in Section 4.5 directly compares policies trained with and without DR on held-out environments, so the claim that DR is needed is supported by a non-DR baseline rather than assumed. The baseline success rates (Tables 1, 8, 9) and the vision-network transfer results (Table 4) are measured on unseen environments and real doorknob images, with no parameter fitted to those test results. The real-world transfer experiment in Section 5 is zero-shot: the policy is trained purely in simulation and deployed without using real-world data, so the 59% success rate is an independent empirical outcome. Self-citations appear only as standard algorithmic references (PPO, SAC, GAE) and do not carry any load-bearing uniqueness or derivation burden. The apparent mismatch between the Baxter robot in Section 5 and the BLUE arms described in Section 2.2 is a reproducibility and under-specification concern, not a circularity concern, because it does not reduce a claimed prediction to an input assumption. Overall, the paper's results are self-contained against its own evaluation protocol and exhibit no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-chosen reward weights, hand-picked randomization ranges, and the fidelity of MuJoCo and Unity modeling. None of these are derived or validated against external benchmarks beyond the single real-world door test.

free parameters (4)
  • Reward shaping weights = a0=1.0, a1=1.0, a2=1.0, a3=1.0, a4=30.0, a5=50.0
    Hand-set weights in Eq. (1); the policy's behavior and the reported success rates depend on this shaping, and no sensitivity analysis is provided.
  • Distance log scale alpha = 0.005
    Set in Eq. (1) to encourage precision near the knob; chosen by hand.
  • Domain randomization ranges = Door mass 22.4-76.5 kg, door height 2000-2500 mm, width 800-1200 mm, knob height 950-1050 mm, friction 0.5-1.0, and…
    Hand-chosen ranges determine the coverage of the randomized training distribution; the claim that DR enables transfer depends on these ranges overlapping real-world conditions.
  • PPO and SAC training hyperparameters = Learning rate 0.001, gamma 0.99, GAE lambda 0.95, etc. (Tables 6 and 7)
    Chosen to make training converge; baseline results are contingent on these settings and on the hardware configuration.
assumptions (4)
  • domain assumption MuJoCo's physics simulation of door, hinge, knob, friction, and contact is accurate enough to support transfer to a real door.
    The entire sim-to-real claim in Section 5 relies on MuJoCo dynamics matching the real door and robot interaction.
  • domain assumption The hand-picked randomization ranges (Table 5) cover the real-world door and environment used in transfer.
    Section 4.4 and Section 5 train only over these ranges; if the real door falls outside them, zero-shot transfer would not be expected.
  • domain assumption Unity rendering and post-processing approximate real-world visual conditions for the vision network.
    Section 7.3 argues Unity improves transfer, but the vision network results still show large out-of-domain errors.
  • ad hoc to paper Opening a door by more than 0.2 rad within 20 seconds is a meaningful proxy for a practical door-opening skill.
    Defined in Eq. (2); it is a convenient threshold but not motivated by any downstream task requirement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DoorGym: A Scalable Door Opening Environment And Baseline Agent." pith.science (2026). https://pith.science/paper/GTWGN37L

@misc{pith2026190801887,
  author       = {Pith},
  title        = {Pith review of: DoorGym: A Scalable Door Opening Environment And Baseline Agent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GTWGN37L}},
  note         = {Machine review of arXiv:1908.01887}
}
read the original abstract

In order to practically implement the door opening task, a policy ought to be robust to a wide distribution of door types and environment settings. Reinforcement Learning (RL) with Domain Randomization (DR) is a promising technique to enforce policy generalization, however, there are only a few accessible training environments that are inherently designed to train agents in domain randomized environments. We introduce DoorGym, an open-source door opening simulation framework designed to utilize domain randomization to train a stable policy. We intend for our environment to lie at the intersection of domain transfer, practical tasks, and realism. We also provide baseline Proximal Policy Optimization and Soft Actor-Critic implementations, which achieves success rates between 0% up to 95% for opening various types of doors in this environment. Moreover, the real-world transfer experiment shows the trained policy is able to work in the real world. Environment kit available here: https://github.com/PSVL/DoorGym/

Figures

Figures reproduced from arXiv: 1908.01887 by the authors.

Figure 1
Figure 1. Various doors sampled with different visual and physical characteristics. For a full list of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Random Samples from the doorknob distribution. 3000 each for knob are included in the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Available arm types Various simulator environments have been released with the aim of creating a common benchmark suite [19–21]. In addition there are frameworks based on these popular benchmark libraries to enable more realistic tasks. One such framework is Fan et al’s 2018 [22] SURREAL environment. The SURREAL framework provides a straightforward framework with practical robotic manipulation tasks, but it is not d… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparison between the domain randomized images from MuJoCo and Unity. Unity can [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Training Pipeline 10−3 for all environments. This PPO training pipeline was structured based on code by Kostrikov, 2018 [30]. In contrast, due to the fact that SAC is an off-policy RL algorithm, it is trained in different way. For each epoch, a single worker performs 1…
Figure 6
Figure 6. Figure 6: Behavior of successful policies in Task 1, Task 2, and Task 3 respectively. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Real world experiment is required to make both the vision network and policy networks robust enough to work in different domains. 5 Real World Transfer In order to check the robustness of the trained policy, we performed zero-shot sim-to-real policy transfer. The exper…
Figure 8
Figure 8. Figure 8: Examples of the post processing that can be apply by Unity. Top row shows the function to [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Sample knobs that used for sim2real transfer for the vision network. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Vision Network Architecture [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Policy Network Architecture 14 [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adaptive Articulated Object Manipulation On The Fly with Foundation Model Reasoning and Part Grounding

    cs.RO 2025-07 conditional novelty 5.0 of 10

    AdaRPG uses GPT-4o, GroundingDINO, and SAM to locate and segment the movable part, a part-affordance model to choose a grasp, and GPT-4o to write the control loop, outperforming prior methods on new articulated objects.

Reference graph

Works this paper leans on

35 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    Klingbeil, A

    E. Klingbeil, A. Saxena, and A. Y . Ng. Learning to open new doors. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 2751–2757, 2010

    work page 2010

  2. [2]

    R. Du, S. Feng, P. Franklin, M. Gennert, J. P. Graff, P. He, A. Jaeger, J. Kim, K. Knoedler, L. Li, C. Y . Liu, X. Long, T. Padir, F. Polido, G. G. Tighe, and X. Xinjilefu. What happened at the darpa robotics challenge , and why ? In None, 2015

    work page 2015

  3. [3]

    Nemec, L

    B. Nemec, L. Žlajpah, and A. Ude. Door opening by joining reinforcement learning and intelligent control. In 2017 18th International Conference on Advanced Robotics (ICAR) , 2017

    work page 2017

  4. [4]

    S. Gu, E. Holly, T. Lillicrap, and S. Levine. Deep reinforcement learning for robotic manipula- tion with asynchronous off-policy updates. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 3389–3396, May 2017. doi: 10.1109/ICRA.2017.7989385

    arXiv 2017

  5. [5]

    open sesame!

    Y . Karayiannidis, C. Smith, F. E. Viña, P. Ogren, and D. Kragic. “open sesame!” adaptive force/velocity control for opening unknown doors. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , 2012

    work page 2012

  6. [6]

    Endres, J

    F. Endres, J. Trinkle, and W. Burgard. Learning the dynamics of doors for robotic manipulation. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 3543– 3549, 2013

    work page 2013

  7. [7]

    Anguelov, D

    D. Anguelov, D. Koller, E. Parker, and S. Thrun. Detecting and modeling doors with mobile robots. In IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA ’04. 2004, volume 4, pages 3777–3784 V ol.4, April 2004. doi: 10.1109/ROBOT.2004.1308857

    arXiv 2004

  8. [8]

    R. B. Rusu, W. Meeussen, S. Chitta, and M. Beetz. Laser-based perception for door and handle identification. In 2009 International Conference on Advanced Robotics , pages 1–8, June 2009

    work page 2009

Show all 35 references
  1. [9]

    Kalakrishnan, L

    M. Kalakrishnan, L. Righetti, P. Pastor, and S. Schaal. Learning force control policies for compliant manipulation. In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4639–4644, Sep. 2011. doi: 10.1109/IROS.2011.6095096

    2011

  2. [10]

    Theodorou, J

    E. Theodorou, J. Buchli, and S. Schaal. A generalized path integral control approach to reinforcement learning. J. Mach. Learn. Res., 11:3137–3181, Dec. 2010. ISSN 1532-4435

    2010

  3. [11]

    Rajeswaran, V

    A. Rajeswaran, V . Kumar, A. Gupta, J. Schulman, E. Todorov, and S. Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. CoRR, abs/1709.10087, 2017. URL http://arxiv.org/abs/1709.10087

    2017 arXiv

  4. [12]

    Rajeswaran, S

    A. Rajeswaran, S. Ghotra, S. Levine, and B. Ravindran. Epopt: Learning robust neural network policies using model ensembles. CoRR, abs/1610.01283, 2016. URL http://arxiv.org/ abs/1610.01283

    2016 arXiv

  5. [13]

    Sadeghi and S

    F. Sadeghi and S. Levine. (cad)$ˆ2$rl: Real single-image flight without a single real image. CoRR, abs/1611.04201, 2016. URL http://arxiv.org/abs/1611.04201

    2016 arXiv

  6. [14]

    Tobin, R

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. CoRR, abs/1703.06907, 2017

    2017 arXiv

  7. [15]

    Sadeghi, A

    F. Sadeghi, A. Toshev, E. Jang, and S. Levine. Sim2real view invariant visual servoing by recurrent control. CoRR, abs/1712.07642, 2017. URL http://arxiv.org/abs/1712.07642

    2017 arXiv

  8. [16]

    X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. CoRR, abs/1710.06537, 2017

    2017 arXiv

  9. [17]

    Andrychowicz, B

    OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Józefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba. Learning dexterous in-hand manipulation. CoRR, 2018. URL http://arxiv.org/ab...

    2018 arXiv

  10. [18]

    Shadowrobot dexterous hand, 2005

    ShadowRobot. Shadowrobot dexterous hand, 2005. URL https://www.shadowrobot.com/ products/dexterous-hand/

    2005

  11. [19]

    Tassa, Y

    Y . Tassa, Y . Doron, A. Muldal, T. Erez, Y . Li, D. de Las Casas, D. Budden, A. Abdolmaleki, J. Merel, A. Lefrancq, T. P. Lillicrap, and M. A. Riedmiller. Deepmind control suite. CoRR, abs/1801.00690, 2018. URL http://arxiv.org/abs/1801.00690

    2018 arXiv

  12. [20]

    Mandlekar, Y

    A. Mandlekar, Y . Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay, S. Savarese, and L. Fei-Fei. Roboturk: A crowdsourcing platform for robotic skill learning through imitation. CoRR, abs/1811.02790, 2018. URL bit.ly/2XJsT9N

    2018 arXiv

  13. [21]

    Brockman, V

    G. Brockman, V . Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. Openai gym, 2016

    2016

  14. [22]

    L. Fan, Y . Zhu, J. Zhu, Z. Liu, O. Zeng, A. Gupta, J. Creus-Costa, S. Savarese, and L. Fei-Fei. Surreal: Open-source reinforcement learning framework and robot manipulation benchmark. In Proceedings of The 2nd Conference on Robot Learning , volume 87 of Proceedings of Machine...

    2018

  15. [23]

    Todorov, T

    E. Todorov, T. Erez, and Y . Tassa. Mujoco: A physics engine for model-based control. In2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 5026–5033, 2012

    2012

  16. [24]

    Mujoco-py

    OpenAI. Mujoco-py. https://github.com/openai/mujoco-py, 2018

    2018

  17. [25]

    Autodesk fusion360, 2013

    AutoDesk, Inc. Autodesk fusion360, 2013. URL https://autode.sk/2XvQgiL

    2013

  18. [26]

    D. V . Gealy, S. McKinley, B. Yi, P. Wu, P. R. Downey, G. Balke, A. Zhao, M. Guo, R. Thomasson, A. Sinclair, P. Cuellar, Z. McCarthy, and P. Abbeel. Quasi-direct drive for low-cost compliant robotic manipulation. CoRR, abs/1904.03815, 2019. URL http: //arxiv.org/abs/1904.03815

    1904 arXiv

  19. [27]

    Levine, N

    S. Levine, N. Wagener, and P. Abbeel. Learning contact-rich manipulation skills with guided policy search. CoRR, abs/1501.05611, 2015. URL http://arxiv.org/abs/1501.05611

    2015 arXiv

  20. [28]

    P. D. A. R. O. K. John Schulman, Filip Wolski. Proximal policy optimization algorithms.CoRR,

  21. [29]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. CoRR, abs/1801.01290, 2018

    2018 arXiv

  22. [30]

    Kostrikov

    I. Kostrikov. Pytorch implementations of reinforcement learning algorithms.https://github. com/ikostrikov/pytorch-a2c-ppo-acktr-gail , 2018

    2018

  23. [31]

    R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3):229–256, 1992. ISSN 1573-0565. doi: 10.1007/BF00992696

    1992 doi

  24. [32]

    Schulman, P

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. In Proceedings of the International Conference on Learning Representations (ICLR), 2016

    2016

  25. [34]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization, 2014. cite arxiv:1412.6980Comment: Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015. 10 7 Appendix 7.1 Proximal Policy Optimization (PPO) Since ...

    2014 arXiv

  26. [2017]

    URL https://arxiv.org/abs/1707.06347

    arXiv

  27. [2018]

    URL http://arxiv.org/abs/1812.05905

    arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.