Pith. sign in

REVIEW 3 major objections 4 minor 45 references

Poke and Strike: Learning Task-Informed Exploration Policies

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims a robot can learn to explore only the physical properties a one-shot task depends on, then strike successfully with 90.1% simulated success and 8/9 real-world trials.

desk verdict A solid, honest advance in task-informed exploration: the simulation results are strong, the independence assumption in threshold design is the main caveat, and the physical validation is partial but clearly reported. read the letter →

arxiv 2509.00178 v1 pith:WWXB4T3P submitted 2025-08-29 cs.RO

Marina Y. Aoyama , Joao Moura , Juan Del Aguila Ferrandis , Sethu Vijayakumar This is my paper · ORCID
classification cs.RO
keywords InteractivePerceptionManipulationReinforcementLearningSystemIdentificationTask-informedexplorationUncertaintyestimationOne-shottasksSim-to-realtransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a robot facing a one-shot dynamic task—strike an unseen puck, push an unknown box to a table edge—can learn an exploration policy that pokes the object just enough to estimate the physical properties that actually matter, then immediately execute the task with those estimates. The central trick is to generate the exploration reward automatically from a privileged task policy: perturb one estimated property at a time, measure how task success degrades, fit a unimodal curve, and use the curve's threshold to decide when estimates are good enough. This makes exploration task-informed rather than exhaustive—properties like center of mass and dynamic friction get tight thresholds, while static friction and restitution stay loose. The paper reports 90.1% success on the simulated striking task against baselines that reach at most 40%, adaptive exploration times under 1.2 seconds, and 8/9 successful trials with a physical robot arm. If these results hold, the method is a reusable recipe for one-shot manipulation without simulator retraining.

What carries the argument

The load-bearing object is the task-informed reward threshold: for property j, fit g_j(ε) to (perturbation error, success rate) data from the privileged task policy, then solve g_j(ε_threshold,j) ≥ p·max_ε g_j(ε). Tighter thresholds encode higher task relevance and force the exploration policy to estimate that property more accurately. Around this sit two supporting mechanisms: a recurrent property estimator trained on the exploration policy's own rollouts (so the estimate stays in distribution), and an ensemble uncertainty estimator whose covariance is used both as a switching signal and, via percentile thresholds from successful trials, as the deployment-time proxy for estimation error.

What would settle it

In simulation, take the trained task policy and add paired perturbations at levels that each individually leave success above threshold (e.g., CoM-y error plus dynamic-friction error). If joint success drops substantially below the single-perturbation success, the independence assumption fails and the exploration rewards are mis-specified.

Watch

Extended reading notes

Core claim

The paper's central claim is that task-relevant exploration can be learned, not hand-designed, by converting a privileged task policy's sensitivity to property-estimation error into reward thresholds. For each physical property, the authors roll out the privileged task policy with the property's ground-truth value replaced by a perturbed value, record task success as a function of perturbation size, and fit a unimodal function g_j. They then set the estimation-error threshold for that property to the largest error that keeps success above a fixed fraction p of the policy's maximum. These thresholds define the exploration reward: the robot gets positive reward when all property estimates fall

Load-bearing premise

The per-property error thresholds assume the effects of estimation errors on task success are independent; if errors interact, the ranking of which properties to explore will be wrong.

Editorial extensions

If this is right

  • One-shot dynamic manipulation with unknown object properties becomes feasible without simulator queries or retraining at test time.
  • The exploration policy learns qualitatively different motions from the task policy—poking versus striking—so each policy can be optimized for its own objective.
  • Task relevance emerges automatically: thresholds tighten for properties that matter and stay loose for irrelevant ones, matching physics intuition in striking, edge pushing, and CartPole.
  • Adaptive exploration length replaces fixed-duration exploration: the robot switches when uncertainty in task-relevant estimates falls below a threshold calibrated from successful trials.
  • The sensitivity-modelling procedure is task-agnostic: it produced useful thresholds on two manipulation tasks and the classical CartPole, suggesting the recipe extends to other one-shot tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The independence assumption is the main hidden load: if two property errors interact, the per-property thresholds could misrank what to explore; a paired-perturbation sweep would test this directly.
  • The physical-robot failures with shifted center of mass point to model mismatch in contact dynamics, not the exploration loop; combining this method with a small residual real-world correction is a natural extension.
  • Uncertainty thresholds are tied to the fixed privileged task policy; retraining the task policy would require recomputing the thresholds, so the approach is best understood as a pipeline with a frozen task policy.
  • The same reward-from-sensitivity recipe could accept other estimators and sensing modalities (force, tactile, vision) if the estimator can output calibrated uncertainty.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a task-informed exploration framework for one-shot robotic manipulation. A privileged task policy is first trained with ground-truth physical properties; the exploration policy is then trained with rewards that depend on whether estimation errors for all properties fall below task-derived thresholds. These thresholds are computed from univariate sensitivity curves of task success to estimation error per property (Sec. 4.4), and an ensemble-based uncertainty estimate is used to decide when to switch from exploration to task execution (Sec. 4.5). The method is evaluated on simulated striking, edge pushing, and CartPole, and on a physical KUKA iiwa for striking and edge pushing. The headline claims are a 90.1% simulated striking success rate and 8/9 physical success on friction-varying pucks.

Significance. If the claims hold, the paper offers a practically valuable recipe: automatically translate a privileged task policy into exploration rewards that focus estimation effort on task-relevant properties, while enabling immediate one-shot execution without test-time simulator queries. The baseline coverage is broad and the ablation contrasting task-informed with task-agnostic thresholds (Sec. 5.3) is informative. The physical experiments, though partial, are a useful step beyond pure simulation. However, the central claims rest on two calibration steps that are not independently validated as rigorously as they need to be: the per-property independence assumption in the threshold generation and the success-label-based uncertainty thresholds. These points are load-bearing for the reported success rates, so the result is promising but not yet fully established.

major comments (3)
  1. [Sec. 4.5] The threshold-generation procedure perturbs one property at a time while holding all others at their ground-truth values, under an explicit independence assumption (Sec. 4.4). The exploration reward in Eq. (1), however, is a conjunction over per-property thresholds. For the Striking task, the privileged policy consumes six properties jointly, and the two most task-relevant properties (CoM_y and dynamic friction) both influence puck trajectory; their errors plausibly interact, so thresholds that are individually safe may be jointly insufficient. The Limitations section acknowledges that correlated errors are not handled, but the paper does not quantify the risk. A joint-perturbation study (e.g., perturbing CoM_y and dynamic friction together and measuring success against the independent-threshold prediction) is needed to support the 90.1% claim.
  2. [Appx. C.4, Eq. (3)] The uncertainty threshold parameter is described only as the 'p-th percentile' (Sec. 4.5), but p is never given. Moreover, the same symbol p is used for the success-retention proportion in Eq. (3). Without reporting the chosen percentile, the policy-switching mechanism cannot be reproduced, and the sensitivity of the reported success rates to this free parameter is unknown.
  3. [Sec. 4.4, Appx. C.4] The threshold-generation procedure in Eq. (3) also depends on the choice of the uni-modal function family (Gaussian, Beta, Gamma in Appx. C.4) and on the unspecified proportion p. Different families can yield different thresholds, and the paper does not report which family is used for the main results, nor does it show how the success rates vary with p. This is a reproducibility issue: the 'automatic' reward generation still contains free parameters that are not fully disclosed or ablated.
minor comments (4)
  1. [Appx. D.1] In the caption of Fig. 19, 'converge to 0.9' should read '0.09' (the values listed in the text are 0.09, 0.12, 0.15).
  2. [Sec. 4.5] The symbol p is used both for the success-retention proportion in Eq. (3) and for the percentile of uncertainty in Sec. 4.5. Use different symbols to avoid confusion.
  3. [Sec. 5.3] The text in Sec. 5.3 says 'ϵ = max_j ϵ_threshold,j' and 'ϵ = min_j ϵ_threshold,j' but does not clearly define how the uniform threshold is applied. Clarify whether the same scalar threshold is used for all properties in the ablation.
  4. [Table 3] The hyperparameter table reports 'Total Timesteps 24,000' but the training curves in Fig. 16 show x-axes up to 4e7. This discrepancy should be explained or corrected.

Circularity Check

3 steps flagged · score 4.0 of 10

Two 'demonstrations' are partially restatements of how thresholds were computed, but the central 90% striking claim has independent robot validation.

  1. self definitional [Sec. 4.4 (Eq. 3) and Sec. 5.2]
    "From the fitted uni-modal function gj, we compute a set of estimation error thresholds εthreshold,j for each physical property j, such that the task success rate remains above a proportion p of the maximum success rate achieved by the privileged task policy. Specifically, we solve: gj(εthreshold,j) ≥ p · max ε gj(ε) ... Properties that are more relevant to the task result in tighter thresholds, encouraging the exploration policy to estimate those properties with higher accuracy."

    The claim in Sec. 5.2 that task-informed rewards 'capture the relative importance of physical properties' is a restatement of the construction in Eq. (3): thresholds are defined by thresholding the fitted success-vs-error curves gj. Therefore the observed ranking (CoM_y and dynamic friction have tighter thresholds) is not an independent prediction; it is the literal output of fitting gj to task-success data. The only non-circular content would be validating these thresholds against held-out joint perturbations, which the paper's Limitations section concedes is not done.

  2. fitted input called prediction [Sec. 4.5 and Sec. 5.3 / Appx. C.6]
    "Then, we calculate the p-th percentile of the uncertainty values for each property from successful task trials (i.e., q% of successful trials have uncertainty values lower than this threshold). Therefore, the task policy is likely to succeed when the uncertainty values are below these thresholds."

    The uncertainty threshold is chosen as a percentile of successful trials' uncertainties, so 'q% of successful trials fall below the threshold' is true by construction. The supporting sim2sim evidence in Fig. 5b and Fig. 18 is obtained from the same 100 PyBullet rollouts used to compute the thresholds (Appx. C.6), making the observed success/uncertainty separation partly an artifact of the percentile definition. The physical robot trials (8/9 successful on friction-varying pucks) are independent but small, and the shifted-CoM cases fail, so the uncertainty-switching claim is only partially supported outside the fitting procedure.

1 more flagged steps
  1. fitted input called prediction [Sec. 4.3 (Eq. 1) and Sec. 5.1]
    "We define the exploration policy reward as rexp = restimation if ∀j εestimation,j < εthreshold,j / rfailure otherwise ... Our policies achieve 92.3% success in exploration (see Appx. C.1) and 98.7% in task."

    The reported 'exploration success' metric is defined by the same threshold condition that defines the exploration reward in Eq. (1). Achieving 92.3% exploration success means the policy frequently satisfies its own reward's success predicate; this is an optimization outcome, not an independent measure of estimation quality. The genuinely non-circular result is the downstream 90.1% task success and the robot trials, which do not reduce to the reward definition.

full rationale

The paper's central derivation chain is largely self-contained: a privileged task policy is used to generate sensitivity curves, thresholds are derived from those curves via Eq. (3), and the exploration policy is trained against the resulting reward. This is a design procedure rather than a circular prediction, and the headline 90.1% striking success is an independent downstream result with physical-robot validation (8/9 on friction-varying pucks). No load-bearing self-citation chain or imported uniqueness theorem is present; the citations to the authors' own prior work are related-work mentions, not premises of the derivation. However, two of the paper's supporting 'demonstrations' are partially circular by construction: (1) the claim that task-informed rewards capture property relevance is a restatement of how the thresholds in Eq. (3) were computed, and (2) the uncertainty-based switching thresholds are percentiles of successful trials' uncertainties, so showing that successes cluster below the threshold on the same data used to fit it is partly tautological. The paper's own Limitations section concedes that the independence assumption behind Eq. (3) is unvalidated for correlated property errors, which further weakens the sensitivity-ranking claim as an independent finding. Overall, the partial circularity is limited to these validation narratives; the central one-shot task-execution result retains independent content.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The method rests on calibrated thresholds rather than a closed-form derivation. The main free parameters are the per-property error thresholds, the uncertainty switching thresholds, and the user-chosen proportion and percentile values. The paper does not report sensitivity to these choices. No new physical entities are postulated.

free parameters (5)
  • Estimation error thresholds epsilon_threshold,j for each property j = Not tabulated numerically; shown as dashed lines in Fig. 4 and C.4
    Computed by solving Eq. (3) from fitted uni-modal sensitivity curves; these thresholds define the exploration reward in Eq. (1).
  • Uncertainty switching thresholds per property = 0.056 (dynamic friction) and 0.054 (CoM-y) for Striking; 0.018 for Edge Pushing (Fig. 19-21)
    Computed as a percentile of uncertainty values from successful task trials (Sec 4.5); determine when the robot stops exploring and starts executing the task.
  • Proportion p in Eq. (3) = Not reported
    Sets the allowed drop in success rate relative to the privileged policy. No sensitivity analysis is given for this value.
  • Percentile q used for uncertainty thresholds = Not reported
    Defines how conservative the switching criterion is. The paper's p/q notation is inconsistent and no sensitivity analysis is given.
  • Parameters of the fitted uni-modal sensitivity functions g_j = Not reported; Gaussian, Beta, and Gamma forms are shown in C.4
    The thresholds depend on these fits. The paper does not report fitted coefficients or a principled selection rule among the three forms.
assumptions (6)
  • domain assumption Physical properties of the manipulated object are constant within an episode.
    Problem formulation, Sec 4.1. If properties drift, the estimator and switching rule would be invalid.
  • domain assumption The privileged task policy trained with ground truth properties provides a reliable measure of task success sensitivity.
    Used in Sec 4.4 to fit the sensitivity curves g_j. If the privileged policy is not representative, the thresholds are wrong.
  • ad hoc to paper The impact of estimation error in each property on task success is independent of other properties.
    Stated in Sec 4.4 and acknowledged in Limitations as an assumption of uncorrelated effects. The method does not handle correlated errors.
  • ad hoc to paper Task success is a uni-modal function of estimation error for each property.
    Sec 4.4 assumes a uni-modal relationship and tests Gaussian, Beta, and Gamma forms, but gives no justification that all properties behave this way.
  • domain assumption Uncertainty estimates from the ensemble are a valid proxy for estimation error at test time.
    Sec 4.5 and Appx C.6. Only demonstrated in simulation; physical shifted-CoM trials show the proxy can fail under model mismatch.
  • domain assumption Isaac Lab dynamics sufficiently match the physical robot and object interaction.
    Appx D.1 admits significant sim2real mismatch for shifted CoM, so this assumption is load-bearing and only partially validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Poke and Strike: Learning Task-Informed Exploration Policies." pith.science (2026). https://pith.science/paper/WWXB4T3P

@misc{pith2026250900178,
  author       = {Pith},
  title        = {Pith review of: Poke and Strike: Learning Task-Informed Exploration Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WWXB4T3P}},
  note         = {Machine review of arXiv:2509.00178}
}
read the original abstract

In many dynamic robotic tasks, such as striking pucks into a goal outside the reachable workspace, the robot must first identify the relevant physical properties of the object for successful task execution, as it is unable to recover from failure or retry without human intervention. To address this challenge, we propose a task-informed exploration approach, based on reinforcement learning, that trains an exploration policy using rewards automatically generated from the sensitivity of a privileged task policy to errors in estimated properties. We also introduce an uncertainty-based mechanism to determine when to transition from exploration to task execution, ensuring sufficient property estimation accuracy with minimal exploration time. Our method achieves a 90% success rate on the striking task with an average exploration time under 1.2 seconds, significantly outperforming baselines that achieve at most 40% success or require inefficient querying and retraining in a simulator at test time. Additionally, we demonstrate that our task-informed rewards capture the relative importance of physical properties in both the striking task and the classical CartPole example. Finally, we validate our approach by demonstrating its ability to identify object properties and adjust task execution in a physical setup using the KUKA iiwa robot arm.

Figures

Figures reproduced from arXiv: 2509.00178 by the authors.

Figure 1
Figure 1. Task-informed exploration approach en￾ables the robot to autonomously learn how to explore and identify task-relevant properties by modeling task sensitivity to each property. For dynamic tasks such as striking, the robot must first identify the object’s prop￾erties through exploratory motions to achieve the task success, avoiding irreversible failure. We address the problem of performing one￾shot robotic tasks that… view at source ↗
Figure 2
Figure 2. Manipulation tasks. We evaluate our task-informed exploration approach in simulation and on a physical robot. We provide additional implementation details in Appx. B. Tasks. We evaluate our method on three tasks: Striking a puck with unknown physical properties toward an unreachable goal; Edge Pushing a box with unknown contents to the table edge, where incor￾rect property estimates cause it to fall; and the classic… view at source ↗
Figure 3
Figure 3. Performance of different methods on the Striking task, with mean and standard deviation reported across three training seeds. DR baselines. DR alone learns an average motion across properties and achieves only a 25.4% suc￾cess rate. In methods with temporal information, DR+Stack and DR+LSTM, exploratory pushing mo￾tions emerge but attains only 35.4% and 23.3% task success, respectively. These methods rely on de￾laye… view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: Uni-modal functions fitted to the relationship [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: (a) RMSE and uncertainty of dy￾namic friction over time. (b) Uncertainty distribution for successful vs. failed trials. Uncertainty estimates. Fig. 5a shows that both estimation errors and uncertainties decrease as exploration progresses when rolling out the exploratio…
Figure 6
Figure 6. Figure 6: Robot experiments on Striking task. We deploy our method on the robot, with examples provided in the video and on the website. Our ap￾proach addresses sim-to-real challenges by learning to explore and estimate task-relevant physical prop￾erties of objects, and by enabl…
Figure 7
Figure 7. Figure 7: Method outline. The exploration component involves simultaneous training of an explo [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Striking environment (a) in Isaac Lab [40, 41] and (b) top view schematics. The red cylindrical pusher, on the left of the dark blue puck, moves at a given velocity inside of the yellow reachable workspace. The goal of the task is to strike the puck to reach the square…
Figure 9
Figure 9. Figure 9: Edge Pushing environment (a) in Isaac Lab [40, 41] and (b) lateral view schematics. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: CartPole environment (a) in Isaac Lab [40, 41] and (b) horizontal view schematics. The square cart slides on the horizontal slider when pushed with a force τ . The goal of the task is to balance the rectangular pole vertically, i.e., θ = 0. B.2 Simulation setup We dev…
Figure 11
Figure 11. Figure 11: The task-informed exploration approach enables the robot to autonomously learn how to explore and identify the physical properties of objects relevant to a given task. For one-shot tasks such as striking, the robot must first identify the object’s properties through e…
Figure 12
Figure 12. Figure 12: Exploration policy training and estimator loss of LSTM-based estimator. [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Exploration policy training and estimator loss of Transformer-based estimator. [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Performance with different training configurations of the control policy for Edge Pushing [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 16
Figure 16. Figure 16: RMSE of the property parameter estimation at the end of the episode during training [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]
Figure 17
Figure 17. Figure 17: Estimation errors and uncertainties of property parameters during the exploration [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]
Figure 18
Figure 18. Figure 18: Relationship between uncertainties and task outcomes when rolling out the exploration [PITH_FULL_IMAGE:figures/full_fig_p025_18.png]
Figure 19
Figure 19. Figure 19: Estimates of dynamic friction and its associated uncertainty during exploration policy execution on the physical setup. Dynamic friction estimates consistently converge to 0.9, 0.12, and 0.15 for the pucks with ball bearings, nylon, and aluminum, respectively. Shifted…
Figure 20
Figure 20. Figure 20: Estimates of the y component of the center of mass and its associated uncertainty during exploration policy execution on the physical setup. Uncertainty decreases in successful trials, while it remains high (i.e., above the threshold 0.054 obtained from [PITH_FULL_IM…
Figure 21
Figure 21. Figure 21: Estimates of the x component of the center of mass and its associated uncertainty dur￾ing exploration policy execution on the physical setup for the Edge Pushing task. Uncertainty decreases below the computed threshold 0.018, leading to successful trials . 27 [PITH_F…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 26 canonical work pages

  1. [1]

    Memmel, A

    M. Memmel, A. Wagenmaker, C. Zhu, D. Fox, and A. Gupta. ASID: Active exploration for sys- tem identification in robotic manipulation. In The Twelfth International Conference on Learn- ing Representations, 2024. URL https://openreview.net/forum?id=jNR6s6OSBT

    work page 2024

  2. [2]

    M. Y . Aoyama, J. Moura, N. Saito, and S. Vijayakumar. Few-shot learning of force-based motions from demonstration through pre-training of haptic representation. In 2024 IEEE In- ternational Conference on Robotics and Automation (ICRA) , 2024. doi:10.1109/icra57147. 2024.10610502

    work page arXiv 2024

  3. [3]

    Saito, N

    N. Saito, N. B. Dai, T. Ogata, H. Mori, and S. Sugano. Real-time liquid pouring motion gener- ation: End-to-end sensorimotor coordination for unknown liquid dynamics trained with deep neural networks. In IEEE International Conference on Robotics and Biomimetics (ROBIO) ,

    work page

  4. [4]

    Lopez-Guevara, R

    T. Lopez-Guevara, R. Pucci, N. K. Taylor, M. U. Gutmann, S. Ramamoorthy, and K. Subr. Stir to pour: Efficient calibration of liquid properties for pouring actions. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2020. doi:10.1109/IROS45743.2020. 9340852

    arXiv 2020

  5. [5]

    S. J. Lederman and R. L. Klatzky. Hand movements: A window into haptic object recogni- tion. Cognitive Psychology, 19(3):342–368, 1987. doi:https://doi.org/10.1016/0010-0285(87) 90008-9

    doi:10.1016/0010-0285(87 1987

  6. [6]

    Y .-L. Tai, Y . C. Chiu, Y .-W. Chao, and Y .-T. Chen. SCONE: A food scooping robot learning framework with active perception. In 7th Annual Conference on Robot Learning, 2023. URL https://openreview.net/forum?id=yHlUVHWnBN

    work page 2023

  7. [7]

    Antonova, J

    R. Antonova, J. Yang, P. Sundaresan, D. Fox, F. Ramos, and J. Bohg. A bayesian treatment of real-to-sim for deformable object manipulation. IEEE Robotics and Automation Letters, 7(3): 5819–5826, 2022. doi:10.1109/lra.2022.3157377

    work page arXiv 2022

  8. [8]

    Saito, T

    N. Saito, T. Ogata, S. Funabashi, H. Mori, and S. Sugano. How to select and use tools? : Active perception of target objects using multimodal deep learning. IEEE Robotics and Automation Letters, 6(2):2517–2524, 2021. doi:10.1109/LRA.2021.3062004

    arXiv 2021

Show all 45 references
  1. [9]

    Liang, S

    J. Liang, S. Saxena, and O. Kroemer. Learning active task-oriented exploration policies for bridging the sim-to-real gap. In Robotics: Science and Systems XVI , 2020. doi:10.15607/rss. 2020.xvi.085

    2020 doi

  2. [10]

    Tobin, R

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) , 2017. doi:10.1109/iros. 2017.8202133

    2017

  3. [11]

    Akkaya, M

    I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, et al. Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113, 2019. doi:10.48550/arXiv.1910.07113

    1910 doi arXiv

  4. [12]

    Ramos, R

    F. Ramos, R. Possas, and D. Fox. Bayessim: Adaptive domain randomization via probabilistic inference for robotics simulators. In Robotics: Science and Systems XV , 2019. doi:10.15607/ rss.2019.xv.029. 10

    2019

  5. [13]

    X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018. doi:10.1109/icra.2018.8460528

    2018

  6. [14]

    L. Ljung. System Identification: Theory for the User . Prentice Hall information and system sciences series. Prentice Hall PTR, 1999. ISBN 9780136566953. URL https://books. google.co.uk/books?id=nHFoQgAACAAJ

    1999

  7. [15]

    V . Lim, H. Huang, L. Y . Chen, J. Wang, J. Ichnowski, D. Seita, M. Laskey, and K. Goldberg. Real2sim2real: Self-supervised learning of physical single-step dynamic actions for planar robot casting. In 2022 International Conference on Robotics and Automation (ICRA) , 2022. doi...

    2022

  8. [16]

    Kolev and E

    S. Kolev and E. Todorov. Physically consistent state estimation and system identification for contacts. In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Hu- manoids), 2015. doi:10.1109/HUMANOIDS.2015.7363481

    2015

  9. [17]

    A. Z. Ren, H. Dai, B. Burchfiel, and A. Majumdar. Adaptsim: Task-driven simulation adap- tation for sim-to-real transfer. In 7th Annual Conference on Robot Learning , 2023. URL https://openreview.net/forum?id=9GRE34K0SB

    2023

  10. [18]

    C. Chi, B. Burchfiel, E. Cousineau, S. Feng, and S. Song. Iterative residual policy: For goal-conditioned dynamic manipulation of deformable objects. The International Journal of Robotics Research, 43(4):389–404, 2024. doi:10.1177/02783649231201201

    2024 doi

  11. [19]

    K. N. Kumar, I. Essa, S. Ha, and C. K. Liu. Estimating mass distribution of articulated objects using non-prehensile manipulation. Object Representations for Learning and Reasoning Work- shop NeurIPS 2020, 2020. URL https://orlrworkshop.github.io/program/orlr_25. html

    2020

  12. [20]

    W. Zhou, L. Pinto, and A. Gupta. Environment probing interaction policies. In International Conference on Learning Representations , 2019. URL https://openreview.net/forum? id=ryl8-3AcFX

    2019

  13. [21]

    S. Dass, J. Hu, B. Abbatematteo, P. Stone, and R. Mart ´ın-Mart´ın. Learning to look: Seeking information for decision making via policy factorization. In 8th Annual Conference on Robot Learning, 2024. URL https://openreview.net/forum?id=B2X57y37kC

    2024

  14. [22]

    Pechyony and V

    D. Pechyony and V . Vapnik. On the theory of learnining with privileged information. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Ad- vances in Neural Information Processing Systems , volume 23. Curran Associates, Inc.,

  15. [23]

    D. Chen, B. Zhou, V . Koltun, and P. Kr ¨ahenb¨uhl. Learning by cheating. In Conference on Robot Learning, 2020. URL https://proceedings.mlr.press/v100/chen20a.html

    2020

  16. [24]

    Fuchioka, C

    Y . Fuchioka, C. C. Beltran-Hernandez, H. Nguyen, and M. Hamaya. Robotic object inser- tion with a soft wrist through sim-to-real privileged training. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2024. doi:10.1109/iros58592.2024. 10801575

    2024

  17. [25]

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter. Learning quadrupedal lo- comotion over challenging terrain. Science Robotics , 5(47):eabc5986, 2020. doi:10.1126/ scirobotics.abc5986

    2020

  18. [26]

    H. Qi, B. Yi, S. Suresh, M. Lambeta, Y . Ma, R. Calandra, and J. Malik. General in-hand object rotation with vision and touch. In 7th Annual Conference on Robot Learning , 2023. URL https://openreview.net/forum?id=RN00jfIV-X. 11

    2023

  19. [27]

    X. Lin, Y . Wang, Z. Huang, and D. Held. Learning visible connectivity dynamics for cloth smoothing. In Conference on Robot Learning , 2022. URL https://proceedings.mlr. press/v164/lin22a.html

    2022

  20. [28]

    Del Aguila Ferrandis, J

    J. Del Aguila Ferrandis, J. Moura, and S. Vijayakumar. Learning visuotactile estimation and control for non-prehensile manipulation under occlusions. In 8th Annual Conference on Robot Learning, 2024. URL https://openreview.net/forum?id=oSU7M7MK6B

    2024

  21. [29]

    W. Yu, J. Tan, C. Karen Liu, and G. Turk. Preparing for the unknown: Learning a universal policy with online system identification. In Robotics: Science and Systems XIII , 2017. doi: 10.15607/rss.2017.xiii.048

    2017 doi

  22. [30]

    E. S. Hu, J. Springer, O. Rybkin, and D. Jayaraman. Privileged sensing scaffolds reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=EpVe8jAjdx

    2024

  23. [31]

    R ¨ostel, J

    L. R ¨ostel, J. Pitz, L. Sievers, and B. B ¨auml. Estimator-coupled reinforcement learning for robust purely tactile in-hand manipulation. In 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids), 2023. doi:10.1109/humanoids57100.2023.10375194

    2023

  24. [32]

    Pinto, M

    L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel. Asymmetric actor critic for image-based robot learning. Robotics: Science and Systems XIV, Jun 2018. doi:10.15607/ rss.2018.xiv.008

    2018

  25. [33]

    Hochreiter and J

    S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Comput. , 9(8): 1735–1780, 1997. doi:10.1162/neco.1997.9.8.1735

    1997 doi

  26. [34]

    Shyam, W

    P. Shyam, W. Ja ´skowski, and F. Gomez. Model-based active exploration. In International conference on machine learning, pages 5779–5788, 2019

    2019

  27. [35]

    Gawlikowski, C

    J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, P. Jung, R. Roscher, et al. A survey of uncertainty in deep neural networks. Artificial Intelligence Review, 56(Suppl 1):1513–1589, 2023. doi:10.1007/s10462-023-10562-9

    2023 doi

  28. [36]

    Nix and A

    D. Nix and A. Weigend. Estimating the mean and variance of the target probability distribu- tion. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), volume 1, pages 55–60 vol.1, 1994. doi:10.1109/ICNN.1994.374138

    1994

  29. [37]

    Lakshminarayanan, A

    B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems , 30, 2017

    2017

  30. [38]

    R. L. Russell and C. Reale. Multivariate uncertainty in deep learning. IEEE Transactions on Neural Networks and Learning Systems, 33(12):7937–7943, 2021. doi:10.1109/TNNLS.2021. 3086757

    2021 doi

  31. [39]

    Kumar, Z

    A. Kumar, Z. Fu, D. Pathak, and J. Malik. Rma: Rapid motor adaptation for legged robots. In Robotics: Science and Systems XVII, 2021. doi:10.15607/rss.2021.xvii.011

    2021 doi

  32. [40]

    Makoviychuk, L

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State. Isaac gym: High performance gpu-based physics simula- tion for robot learning, 2021. URL https://github.com/isaac-sim/IsaacGymEnvs

    2021

  33. [41]

    Mittal, C

    M. Mittal, C. Yu, Q. Yu, J. Liu, N. Rudin, D. Hoeller, J. L. Yuan, R. Singh, Y . Guo, H. Mazhar, A. Mandlekar, B. Babich, G. State, M. Hutter, and A. Garg. Orbit: A unified simulation framework for interactive robot learning environments.IEEE Robotics and Automation Letters, 8...

    2023

  34. [42]

    Coumans and Y

    E. Coumans and Y . Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016–2021. 12

    2016

  35. [43]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. doi:10.48550/arXiv.1707.06347. 13 Appendix A Method A.1 Task-informed exploration framework Our proposed approach utilizes the task p...

    2017 doi arXiv

  36. [2010]

    URL https://proceedings.neurips.cc/paper_files/paper/2010/file/ c73dfe6c630edb4c1692db67c510f65c-Paper.pdf

    2010

  37. [2019]

    doi:10.1109/ROBIO49542.2019.8961718

    2019

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.