Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A distributional sim-to-real gap score, optimized with CMA-ES over eight actuator parameters, lets a fully learned policy drive Spot at over 5.2 m/s with a flight-phase gallop, slippery-ground robustness, and disturbance rejection.

desk verdict A credible engineering advance—first learned end-to-end policy on Spot—with real but addressable weaknesses in the quantitative evidence and the noise handling. read the letter →

arxiv 2504.17857 v3 pith:VZPSOQM4 submitted 2025-04-24 cs.LG cs.RO

AJ Miller , Fangzhou Yu , Michael Brauckmann , Farbod Farshidian This is my paper · ORCID
classification cs.LGcs.RO
keywords reinforcementlearningsim-to-realtransferquadrupedlocomotiondistributionalgapmeasureWassersteindistancemaximummeandiscrepancyCMA-ESactuatorparameteroptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a gray-box modeling loop can close the sim-to-real gap for Spot enough to run a fully learned end-to-end control policy at speeds and gaits the stock controller cannot reach. The authors score the difference between hardware and simulation by comparing distributions of joint positions, joint velocities, and policy actions with Wasserstein distance and maximum mean discrepancy, then use CMA-ES to tune eight actuator parameters (joint friction and torque-speed limits) that are hard to measure directly. A single retraining pass with the optimized model yields a policy that sustains 5.2 m/s, more than triple Spot's default 1.6 m/s limit, gallops with a flight phase, and stays upright on soap-wet plexiglass. If correct, this is the first end-to-end RL policy deployed on Spot with public training code, and it offers a reusable recipe for high-performance learned control on a commercial quadruped.

What carries the argument

The load-bearing mechanism is a distributional gap score built from Wasserstein distance and maximum mean discrepancy, computed over scaled joint positions, joint velocities, and policy actions from hardware and simulated rollouts of the same command sequences. The score plays the role of a GAN-style evaluation: the simulated policy is the generator and the hardware data is the target, so the two rollouts do not need to start from matched states. CMA-ES treats that score as the objective and searches an eight-parameter gray-box actuator model (two friction coefficients plus a six-parameter torque-speed curve), and the optimized values are used to retrain the policy.

What would settle it

Train a second policy with a substantially different gait or reward budget using the same eight optimized parameters without retuning, then deploy it; if the distributional gap score shrinks but hardware speed and robustness do not improve, the optimization was overfitting to the original policy rather than recovering the true actuator model.

Watch

Extended reading notes

Core claim

The paper's central claim is that the sim-to-real gap for Spot is best treated as a distribution-matching problem rather than a trajectory-matching one. Hardware rollouts are collected under several scripted command sequences, the same commands are replayed in simulation, and the gap is scored as the Wasserstein and MMD distance between the scaled joint positions, joint velocities, and policy actions of the two data sets. CMA-ES then minimizes that score over eight parameters describing joint friction and torque-speed limits, and a single retraining pass with the optimized simulator produces policies that gallop with a flight phase, reject disturbances, and reach 5.2 m/s sustained speed — more than triple the stock controller's 1.6 m/s maximum and well above the roughly 3.7 m/s baseline policy trained without the optimization.

Load-bearing premise

The method assumes that matching the distribution of joint positions, joint velocities, and actions between simulation and hardware is enough to identify the true actuator mismatches, so the eight optimized parameters are the real cause of the gap rather than an overfit to one policy and its command sequences.

Editorial extensions

If this is right

  • An end-to-end learned controller can push Spot to a sustained 5.2 m/s flying-trot gallop, a capability the stock controller does not offer.
  • The distributional scoring procedure makes simulator tuning possible without joint torque measurements or matched-state resets, relying only on onboard state and action logs.
  • One iteration of optimization plus retraining is enough for a substantial transfer gain, raising sustained speed from roughly 3.7 m/s to 5.2 m/s.
  • The released training and deployment code provides a publicly reproducible baseline for future low-level RL control on Spot.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stress test the paper does not run: keep the eight optimized parameters fixed while training a policy with a different gait or reward budget; if speed and robustness gains persist, the parameters are true actuator properties rather than compensations for one policy's failure modes.
  • The scoring features could be extended beyond joint positions, velocities, and actions to exteroceptive signals such as body height or contact forces, which may expose mismatches the current joint-space score cannot see.
  • The optimization loop couples system identification with policy retraining; a variant that optimizes parameters against a frozen policy and only then retrains would isolate how much of the gain comes from each step.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents a workflow for sim-to-real transfer of a learned locomotion policy on a Boston Dynamics Spot robot. The authors quantify the sim-to-real gap by comparing distributions of joint positions, joint velocities, and policy actions between hardware rollouts and simulation rollouts, using Wasserstein distance and Maximum Mean Discrepancy. These measures serve as the objective for CMA-ES, which optimizes eight actuator-related parameters: hip and knee friction coefficients, torque limits, speed limits, and torque-speed intersection points. After optimization, a PPO policy is retrained in NVIDIA Isaac Lab and deployed through the Spot RL Researcher Development Kit. The paper claims the deployed policy achieves over 5.2 m/s forward speed, more than triple Spot's default controller maximum, and shows robustness to slippery surfaces and a flight phase gait. The authors state that training code is publicly available and that this is the first end-to-end RL policy deployed on Spot with open training code.

Significance. If the quantitative results hold, this would be a notable empirical contribution: a publicly documented end-to-end RL pipeline on a commercial quadruped, with a concrete method for tuning simulation parameters from distributional data. The idea of using Wasserstein/MMD scores as a CMA-ES objective is a reasonable and potentially reusable approach for gray-box actuator identification, and the hardware demonstrations of a flying trot at 5.2 m/s would be impressive. However, the evidence supporting the central claims is currently incomplete: the headline speed results lack trial counts and error bars, the reward weights are omitted, the code link is missing, and the scoring procedure excludes sensor noise from simulation while hardware data necessarily includes it. These gaps make the paper's central methodological and performance claims plausible but not yet fully supported.

major comments (4)
  1. [IV.C] Section IV.C reports that the policy 'reached sustained running speeds of 5.2m/s in a flying trot gait' and compares this to the baseline policy's '3.7m/s sustained maximum,' but the manuscript gives no number of hardware trials, no run-to-run variability, and no measurement protocol for either number. The headline claim of a more-than-tripled maximum speed is load-bearing and currently rests on single-point estimates. Please report repeated trials with mean and standard deviation (or all individual runs) for both the baseline and the CMA-ES-optimized policy under identical command protocols, including how the maximum speed was determined from each trial.
  2. [IV.A] Section IV.A states, 'Importantly, simulated data does not include sensor noise,' while the hardware rollouts used as the optimization target are necessarily corrupted by state-estimation noise. Section III.A describes a Gaussian observation-noise model, but that model is not applied to the simulated data used in scoring. The distributional distance therefore contains a noise-induced component that no actuator parameter can physically eliminate, so CMA-ES may reduce the score by shifting simulated joint distributions to match the noise spread. The optimized friction and torque-speed values in Table I may thereby partly absorb sensor-noise mismatch rather than physical actuator effects. A concrete test is to corrupt the simulated observations with the Section III.A noise model before scoring, or to denoise the hardware estimates, and to report whether the optimized parameters change.
  3. [III.D / IV.A] Section III.D lists reward components but gives no weights, and Sections III.C and IV.A state that 'some fine-tuning of the reward functions may be required' and that 'only minor weight value adjustments were made.' Because the speed comparison is between a baseline policy and a policy retrained after both parameter optimization and reward modifications, the reported improvement from 3.8 m/s to 5.2 m/s cannot be cleanly attributed to the CMA-ES parameter search alone; it could also come from reward fine-tuning or from PPO stochasticity across random seeds. Please report the exact reward weights before and after the optimization, run multiple training seeds, and include an ablation that keeps rewards fixed while changing only the optimized simulation parameters.
  4. [IV.B] Section IV.B explicitly states that 'both policies struggle to produce negative torque during swing leg recirculation... due to power distribution limits not being included in our modeling.' This is an admitted unmodeled physical effect that appears in exactly the high-performance regime where the distributional score is computed. The eight optimized parameters cannot represent this mismatch, so the optimized values and the residual distributional gap may be dominated by the missing power-limit model rather than by the identified actuator parameters. Please either include a power-distribution-limit model in the simulator, or add an experiment that isolates whether the optimized parameters capture the observed hardware-simulation difference beyond this unmodeled effect.
minor comments (6)
  1. [Abstract / Conclusion] The abstract and conclusion promise publicly available training code, but no repository URL appears anywhere in the manuscript; add a link, ideally in a footnote in Section I or III.
  2. [III.B] The scoring procedure is described qualitatively, but the manuscript does not state which Wasserstein variant is used (1D, sliced, or joint product), which MMD kernel and bandwidth are chosen, how the two measures are combined into the reported weighted average, or how many simulated rollouts and samples are used per CMA-ES candidate. An appendix with these details would make the method reproducible.
  3. [III.A] The sentence 'Delaying action application the total delay is identical to delaying both the observations and actions' is grammatically incomplete, and the choice of a 5 ms buffer is not justified against the measured delay distribution in Fig. 2.
  4. [III.D] The reward components are listed by name but no weights are given; please include the full reward table, since the policy and its reward tuning are central to the performance comparisons.
  5. [IV.A vs IV.C] The baseline maximum speed is reported as 3.8 m/s in Section IV.A and 3.7 m/s in Section IV.C; please reconcile these numbers and use one consistent value.
  6. [Figures 2 and 6] Figure 2's axis labels and tick marks are unclear (the x-axis appears to show only 2-6), and the term 'Occurences' is misspelled; Figure 6 lacks units and trial-count annotations. Please improve caption clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the optimized parameters are fitted to external hardware rollouts, and the headline speed is a newly measured outcome under a retrained policy, not a fitted constant.

full rationale

The paper's derivation chain is a standard system-identification loop rather than a self-justifying construction. First, the authors collect hardware rollouts under an initial policy (Section III.C: 'we deploy this policy on hardware and log the real data'). Second, they define a distributional distance between hardware and simulated rollouts in joint position, joint velocity, and action space, and minimize it with CMA-ES over eight actuator parameters (friction and torque-speed limits). The hardware data are external ground truth, not derived from the simulator or from the parameters being optimized. Third, they retrain a new policy with the optimized parameters and measure its hardware speed: 'we found a single iteration of the algorithm sufficient to markedly improve performance and increase maximum linear velocity from 3.8m/s to 5.2m/s on hardware' (Section IV.A). This speed is an outcome of a subsequent hardware deployment, not a quantity fitted by the optimization; nothing in the equations forces the speed to equal the optimized parameters. The reported distributional gap is the optimization objective itself, but using a gap metric as a cost function is inherent to system identification, not a circular prediction. The concern that simulated data exclude sensor noise (Section IV.A: 'Importantly, simulated data does not include sensor noise') is an identifiability or confounding issue: unmodeled noise could be absorbed into the fitted actuator values. That weakens causal interpretation of the optimized parameters, but it is not a definitional reduction, because the score is not constructed from the parameters and the parameters are not defined in terms of the final speed. No load-bearing self-citation appears: the only author-overlapping reference ([11], an MPC paper) is contextual related work, and the procedure does not invoke a uniqueness theorem or an ansatz smuggled in via citation. The comparison between baseline and optimized policies on hardware (Section IV.B) is an external benchmark, and the robustness and speed demonstrations are empirical results with independent content. Therefore no circular step can be exhibited from the paper's own equations or citations.

Assumptions & free parameters 12 free parameters · 8 assumptions · 0 invented entities

The central claim rests on a gray-box system identification loop: eight actuator parameters are fitted to hardware distributional data, plus a delay estimate, observation noise model, and hand-set control gains and reward weights. No new physical entities are introduced. The main burden is whether the simplified parameterization captures the true sim-to-real gap rather than absorbing sensor noise and unmodeled power limits.

free parameters (12)
  • Hip friction coefficient = 0.008 Nm
    Optimized by CMA-ES to minimize distributional sim-to-real gap; Table I.
  • Knee friction coefficient = 0.180 Nm
    Optimized by CMA-ES; remotized lead-screw joint has higher friction; Table I.
  • Torque limit max = 97.00 Nm
    Optimized torque-speed curve parameter; Table I.
  • Torque limit min = -108.79 Nm
    Optimized torque-speed curve parameter; Table I.
  • Speed limit max = 25.03 rad/s
    Optimized torque-speed curve parameter; Table I.
  • Speed limit min = -22.22 rad/s
    Optimized torque-speed curve parameter; Table I.
  • Torque-speed intersect max = 9.48 rad/s
    Optimized parameter for quadrant 1 torque-speed trade-off; Table I.
  • Torque-speed intersect min = -8.32 rad/s
    Optimized parameter for quadrant 3 torque-speed trade-off; Table I.
  • Communication delay buffer = 5 ms
    Estimated from measured gRPC delay histogram (Fig. 2) and baked into simulator as an action buffer; chosen from data rather than optimized.
  • Observation noise sigma per feature = Not reported
    Estimated from an FFT of state estimator data during walking; values used in training but not listed in the paper.
  • Reward weights = Not reported
    Reward terms are listed in Section III.D but the weights are omitted; Section IV.A mentions 'minor weight value adjustments' without specifics.
  • PD gains kp and kd = kp = 60.0 Nm/rad, kd = 1.5 Nm s/rad
    Hand-set control-law constants for all joints; central to the action-to-torque mapping in Eq. 1.
assumptions (8)
  • domain assumption Isaac Sim physics engine with rigid-body contact models is an adequate representation of Spot dynamics for policy training.
    Used throughout Section III.D; no validation of the physics engine against hardware other than the distributional scores.
  • domain assumption Torque-sensing feedback in Spot's motor controllers makes commanded torque accurately achieved, so only friction, delay, and limits need modeling.
    Section III.A states this to justify ignoring gear ratios and unreliable actuators.
  • domain assumption Wasserstein distance and MMD on joint position, velocity, and action features are sufficient measures of sim-to-real gap for parameter optimization.
    Section III.B chooses these features and measures; no ablation or comparison with other feature sets is provided.
  • domain assumption CMA-ES with 100 iterations and population size 10 finds a good minimum of the distributional score.
    Section III.C; no convergence analysis or repeated optimization runs are reported.
  • domain assumption A fixed 5 ms action buffer captures the effect of gRPC communication delay.
    Section III.A and Fig. 2; a measured distribution of delays is collapsed to a constant.
  • domain assumption The FFT-based normalization estimates zero-mean Gaussian observation noise per feature.
    Section III.A; noise is assumed white and Gaussian, with sigma computed from the right half of the frequency spectrum.
  • domain assumption Power distribution limits can be neglected for the main claims; the paper admits they are not modeled.
    Section IV.B states swing-leg negative torque struggles are due to unmodeled power distribution limits.
  • domain assumption The default Spot controller maximum speed is 1.6 m/s as given by the specification sheet.
    Used in Section IV.C as the baseline for the tripling claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures." pith.science (2026). https://pith.science/paper/VZPSOQM4

@misc{pith2026250417857,
  author       = {Pith},
  title        = {Pith review of: High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VZPSOQM4}},
  note         = {Machine review of arXiv:2504.17857}
}
read the original abstract

This work presents an overview of the technical details behind a high performance reinforcement learning policy deployment with the Spot RL Researcher Development Kit for low level motor access on Boston Dynamics Spot. This represents the first public demonstration of an end to end end reinforcement learning policy deployed on Spot hardware with training code publicly available through Nvidia IsaacLab and deployment code available through Boston Dynamics. We utilize Wasserstein Distance and Maximum Mean Discrepancy to quantify the distributional dissimilarity of data collected on hardware and in simulation to measure our sim2real gap. We use these measures as a scoring function for the Covariance Matrix Adaptation Evolution Strategy to optimize simulated parameters that are unknown or difficult to measure from Spot. Our procedure for modeling and training produces high quality reinforcement learning policies capable of multiple gaits, including a flight phase. We deploy policies capable of over 5.2ms locomotion, more than triple Spots default controller maximum speed, robustness to slippery surfaces, disturbance rejection, and overall agility previously unseen on Spot. We detail our method and release our code to support future work on Spot with the low level API.

Figures

Figures reproduced from arXiv: 2504.17857 by the authors.

Figure 1
Figure 1. Spot galloping in flight on a flat track with our policy at 5.2m/s. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Histogram plot of the measured communication delay of Google [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Torque-position (left) and torque-speed (right) limits of the [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Command sequence used to generate test data for evaluation of [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: Visual overview of the training and deployment pipeline. On the left, is an approximation of how the policy is trained in simulation, and on the [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Comparison of knee joint action and desired torque during a [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control

    cs.LG 2025-12 conditional novelty 6.0 of 10

    Logic-gate circuits trained with gradient descent can match neural-network policies on most MuJoCo continuous-control tasks and run on FPGAs in a few clock cycles.

  2. Towards bridging the gap: Systematic sim-to-real transfer for diverse legged robots

    cs.RO 2025-09 conditional novelty 6.0 of 10

    PACE fits a compact set of actuator parameters from brief in-air data and trains energy-aware locomotion policies that transfer zero-shot to real quadrupeds without dynamics randomization.

Reference graph

Works this paper leans on

51 extracted references · 32 canonical work pages · cited by 2 Pith papers

  1. [1]

    Boston Dynamics

    Boston dynamics: Spot. Boston Dynamics. [Online]. Available: https://bostondynamics.com/products/spot/

    work page

  2. [2]

    [Online]

    Hey buddy, can you give me a hand? Boston Dynamics. [Online]. Available: https://www.youtube.com/watch?v=fUyU3lKzoio

    work page

  3. [3]

    [Online]

    What does it take to put spot to work? Boston Dynamics. [Online]. Available: https://www.youtube.com/watch?v= Ux-N-NK2GM

    work page

  4. [4]

    Boston Dynamics

    Uptown spot. Boston Dynamics. [Online]. Available: https://www. youtube.com/watch?v=kHBcVlqpvZ8

    work page

  5. [5]

    Boston Dynamics

    Spot sdk. Boston Dynamics. [Online]. Available: https: //dev.bostondynamics.com/

    work page

  6. [6]

    Domanico

    P. Domanico. Reinforcement learning with spot. Boston Dynamics. [Online]. Available: https://bostondynamics.com/video/ reinforcement-learning-with-spot/

    work page

  7. [7]

    Mit cheetah 3: Design and control of a robust, dynamic quadruped robot,

    G. Bledt, M. J. Powell, B. Katz, J. Di Carlo, P. M. Wensing, and S. Kim, “Mit cheetah 3: Design and control of a robust, dynamic quadruped robot,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 2245–2252

    2018

  8. [8]

    Design of hyq–a hydraulically and electrically actuated quadruped robot,

    C. Semini, N. G. Tsagarakis, E. Guglielmino, M. Focchi, F. Cannella, and D. G. Caldwell, “Design of hyq–a hydraulically and electrically actuated quadruped robot,” Proceedings of the Institution of Mechan- ical Engineers, Part I: Journal of Systems and Control Engineering , vol. 225, no. 6, pp. 831–849, 2011

    work page 2011

Show all 51 references
  1. [9]

    Anymal-a highly mobile and dynamic quadrupedal robot,

    M. Hutter, C. Gehring, D. Jud, A. Lauber, C. D. Bellicoso, V . Tsounis, J. Hwangbo, K. Bodie, P. Fankhauser, M. Bloesch et al. , “Anymal-a highly mobile and dynamic quadrupedal robot,” in 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS) . IEEE, 2...

    2016

  2. [10]

    Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,

    J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim, “Dynamic locomotion in the mit cheetah 3 through convex model-predictive control,” in 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS) . IEEE, 2018, pp. 1–9

    2018

  3. [11]

    Real-time motion planning of legged robots: A model predictive control approach,

    F. Farshidian, E. Jelavic, A. Satapathy, M. Giftthaler, and J. Buchli, “Real-time motion planning of legged robots: A model predictive control approach,” in 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids) . IEEE, 2017, pp. 577–584

    2017

  4. [12]

    Dynamic locomotion and whole-body control for quadrupedal robots,

    C. D. Bellicoso, F. Jenelten, P. Fankhauser, C. Gehring, J. Hwangbo, and M. Hutter, “Dynamic locomotion and whole-body control for quadrupedal robots,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2017, pp. 3359–3365

    2017

  5. [13]

    Learning agile and dynamic motor skills for legged robots,

    J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,” Science Robotics, vol. 4, no. 26, p. eaau5872, 2019

    2019

  6. [14]

    Rapid locomotion via reinforcement learning,

    G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal, “Rapid locomotion via reinforcement learning,” The International Journal of Robotics Research, vol. 43, no. 4, pp. 572–587, 2024

    2024

  7. [15]

    Anymal parkour: Learning agile navigation for quadrupedal robots,

    D. Hoeller, N. Rudin, D. Sako, and M. Hutter, “Anymal parkour: Learning agile navigation for quadrupedal robots,” Science Robotics , vol. 9, no. 88, p. eadi7566, 2024

    2024

  8. [16]

    Learning to walk in minutes using massively parallel deep reinforcement learning,

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” in Conference on Robot Learning . PMLR, 2022, pp. 91–100

    2022

  9. [17]

    Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion,

    G. Ji, J. Mun, H. Kim, and J. Hwangbo, “Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion,” IEEE Robotics and Automation Letters , vol. 7, no. 2, pp. 4630–4637, 2022

    2022

  10. [18]

    Rma: Rapid motor adaptation for legged robots,

    A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor adaptation for legged robots,” arXiv preprint arXiv:2107.04034 , 2021

    2021 arXiv

  11. [19]

    Actuator-constrained reinforcement learning for high-speed quadrupedal locomotion,

    Y .-H. Shin, T.-G. Song, G. Ji, and H.-W. Park, “Actuator-constrained reinforcement learning for high-speed quadrupedal locomotion,” arXiv preprint arXiv:2312.17507, 2023

    2023 arXiv

  12. [20]

    Robot learning from randomized simulations: A review,

    F. Muratore, F. Ramos, G. Turk, W. Yu, M. Gienger, and J. Peters, “Robot learning from randomized simulations: A review,” Frontiers in Robotics and AI , vol. 9, p. 799893, 2022

    2022

  13. [21]

    Feedback control for cassie with deep reinforcement learning,

    Z. Xie, G. Berseth, P. Clary, J. Hurst, and M. Van de Panne, “Feedback control for cassie with deep reinforcement learning,” in2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 1241–1246

    2018

  14. [22]

    Policy transfer via kinematic domain randomization and adaptation,

    I. Exarchos, Y . Jiang, W. Yu, and C. K. Liu, “Policy transfer via kinematic domain randomization and adaptation,” in 2021 IEEE In- ternational Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 45–51

    2021

  15. [23]

    Reinforcement learning for robust parameterized locomotion control of bipedal robots,

    Z. Li, X. Cheng, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Reinforcement learning for robust parameterized locomotion control of bipedal robots,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 2811–2817

    2021

  16. [24]

    Sim2real transfer for reinforcement learning without dynamics randomization,

    M. Kaspar, J. D. M. Osorio, and J. Bock, “Sim2real transfer for reinforcement learning without dynamics randomization,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 4383–4388

    2020

  17. [25]

    Estimation of inertial parameters of rigid body links of manipulators,

    C. H. An, C. G. Atkeson, and J. M. Hollerbach, “Estimation of inertial parameters of rigid body links of manipulators,” in 1985 24th IEEE Conference on Decision and Control . IEEE, 1985, pp. 990–995

    1985

  18. [26]

    Geometric robot dynamic identification: A convex programming approach,

    T. Lee, P. M. Wensing, and F. C. Park, “Geometric robot dynamic identification: A convex programming approach,” IEEE Transactions on Robotics , vol. 36, no. 2, pp. 348–365, 2019

    2019

  19. [27]

    Pros and cons of gan evaluation measures,

    A. Borji, “Pros and cons of gan evaluation measures,” Computer vision and image understanding , vol. 179, pp. 41–65, 2019

    2019

  20. [28]

    The wasserstein distance and approximation the- orems,

    L. R ¨uschendorf, “The wasserstein distance and approximation the- orems,” Probability Theory and Related Fields , vol. 70, no. 1, pp. 117–129, 1985

    1985

  21. [29]

    Training generative neural networks via maximum mean discrepancy optimization,

    G. K. Dziugaite, D. M. Roy, and Z. Ghahramani, “Training generative neural networks via maximum mean discrepancy optimization,” arXiv preprint arXiv:1505.03906, 2015

    2015 arXiv

  22. [30]

    Amp: Adversarial motion priors for stylized physics-based character con- trol,

    X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, “Amp: Adversarial motion priors for stylized physics-based character con- trol,” ACM Transactions on Graphics (ToG) , vol. 40, no. 4, pp. 1–20, 2021

    2021

  23. [31]

    Learning agile skills via adversarial imitation of rough partial demonstrations,

    C. Li, M. Vlastelica, S. Blaes, J. Frey, F. Grimminger, and G. Mar- tius, “Learning agile skills via adversarial imitation of rough partial demonstrations,” in Conference on Robot Learning . PMLR, 2023, pp. 342–352

    2023

  24. [32]

    Knowl- edge transfer across imaging modalities via simultaneous learning of adaptive autoencoders for high-fidelity mobile robot vision,

    M. M. Rahman, T. Rahman, D. Kim, and M. A. U. Alam, “Knowl- edge transfer across imaging modalities via simultaneous learning of adaptive autoencoders for high-fidelity mobile robot vision,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IE...

    2021

  25. [33]

    grpc: A high performance, open source universal rpc framework. Google. [Online]. Available: https://grpc.io/

  26. [34]

    About the nyquist frequency,

    F. Mignard, “About the nyquist frequency,” Observatoire de la C ˆote d’Azur , Dpt. Cassiop´ee. GAIA FM , vol. 22, 2005

    2005

  27. [35]

    Simulation-based design of dynamic controllers for humanoid balancing,

    J. Tan, Z. Xie, B. Boots, and C. K. Liu, “Simulation-based design of dynamic controllers for humanoid balancing,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2016, pp. 2729–2736

    2016

  28. [36]

    Sim-to-real: Learning agile locomotion for quadruped robots,

    J. Tan, T. Zhang, E. Coumans, A. Iscen, Y . Bai, D. Hafner, S. Bo- hez, and V . Vanhoucke, “Sim-to-real: Learning agile locomotion for quadruped robots,” arXiv preprint arXiv:1804.10332 , 2018

    2018 arXiv

  29. [37]

    Sim-to-real via sim- to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks,

    S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis, “Sim-to-real via sim- to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks,” in Proceedings of the IEEE/CVF conference on com...

    2019

  30. [38]

    Learning inertial odometry for dynamic legged robot state estimation,

    R. Buchanan, M. Camurri, F. Dellaert, and M. Fallon, “Learning inertial odometry for dynamic legged robot state estimation,” in Conference on robot learning . PMLR, 2022, pp. 1575–1584

    2022

  31. [39]

    Starleth & co.: Design and control of legged robots with compliant actuation,

    M. Hutter, “Starleth & co.: Design and control of legged robots with compliant actuation,” Ph.D. dissertation, ETH Zurich, 2013

    2013

  32. [40]

    Mini cheetah: A platform for push- ing the limits of dynamic quadruped control,

    B. Katz, J. Di Carlo, and S. Kim, “Mini cheetah: A platform for push- ing the limits of dynamic quadruped control,” in 2019 international conference on robotics and automation (ICRA) . IEEE, 2019, pp. 6295–6301

    2019

  33. [41]

    Tutorial cma-es: evolution strategies and covariance matrix adaptation,

    A. Auger and N. Hansen, “Tutorial cma-es: evolution strategies and covariance matrix adaptation,” in Proceedings of the 14th annual conference companion on Genetic and evolutionary computation , 2012, pp. 827–848

    2012

  34. [42]

    Hardware as policy: Mechanical and computational co-optimization using deep reinforcement learn- ing,

    T. Chen, Z. He, and M. Ciocarlie, “Hardware as policy: Mechanical and computational co-optimization using deep reinforcement learn- ing,” arXiv preprint arXiv:2008.04460 , 2020

    2008 arXiv

  35. [43]

    Simulation aided co-design for robust robot optimization,

    G. Fadini, T. Flayols, A. Del Prete, and P. Sou `eres, “Simulation aided co-design for robust robot optimization,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11 306–11 313, 2022

    2022

  36. [44]

    Multiple task optimization with a mixture of controllers for motion generation,

    N. Dehio, R. F. Reinhart, and J. J. Steil, “Multiple task optimization with a mixture of controllers for motion generation,” in2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2015, pp. 6416–6421

    2015

  37. [45]

    Kicking motion planning of nao robots based on cma-es,

    X. Li, Z. Liang, and H. Feng, “Kicking motion planning of nao robots based on cma-es,” in The 27th Chinese Control and Decision Conference (2015 CCDC) . IEEE, 2015, pp. 6158–6161

    2015

  38. [46]

    Orbit: A unified simulation framework for interactive robot learning environments,

    M. Mittal, C. Yu, Q. Yu, J. Liu, N. Rudin, D. Hoeller, J. L. Yuan, R. Singh, Y . Guo, H. Mazhar et al. , “Orbit: A unified simulation framework for interactive robot learning environments,”IEEE Robotics and Automation Letters , 2023

    2023

  39. [47]

    Nvidia isaacsim. Nvidia. [Online]. Available: https://developer.nvidia. com/isaac/sim

  40. [48]

    Robotic Systems Lab

    Rsl rl. Robotic Systems Lab. [Online]. Available: https://github.com/ leggedrobotics/rsl rl

  41. [49]

    A survey of actor-critic reinforcement learning: Standard and natural policy gradients,

    I. Grondman, L. Busoniu, G. A. Lopes, and R. Babuska, “A survey of actor-critic reinforcement learning: Standard and natural policy gradients,” IEEE Transactions on Systems, Man, and Cybernetics, part C (applications and reviews) , vol. 42, no. 6, pp. 1291–1307, 2012

    2012

  42. [50]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  43. [51]

    Boston Dynamics

    Spot specifications. Boston Dynamics. [Online]. Available: https: //support.bostondynamics.com/s/article/Robot-specifications

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.