Pith. sign in

REVIEW 3 major objections 5 minor 36 references

In-Context Iterative Policy Improvement for Dynamic Manipulation

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pretrained large language models can act as sample-efficient policy improvement operators for dynamic manipulation, working purely in-context without weight updates.

desk verdict Novel ICL formulation for policy improvement, but a missing train/test split detail undermines the main claim until clarified. read the letter →

arxiv 2508.15021 v1 pith:LMLGVACV submitted 2025-08-20 cs.RO

Mark Van der Merwe , Devesh Jha This is my paper · ORCID
classification cs.RO
keywords in-contextlearninglargelanguagemodelspolicyimprovementdynamicmanipulationfew-shotrobotalgorithmdistillationsample-efficientoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a large pretrained language model can improve a robot's dynamic manipulation policy from a handful of trials, without any gradient training. The authors frame the problem as learning a policy improvement operator: take the current policy parameters and a compact error signal from the last rollout, and return a parameter adjustment. They build a small dataset of about 300 improvement examples on similar tasks, retrieve nearest neighbors in a normalized policy-error space, and place them in the LLM prompt; the model completes the pattern with a new delta. Across simulated puck-sliding and rope-swinging tasks and a physical ball-rolling task, the method outperforms Bayesian optimization, random shooting, piecewise-linear regression, and two LLM-based baselines when data is scarce. If right, this means an off-the-shelf LLM can act as a sample-efficient optimizer for dynamic manipulation problems whose physics are not directly observable.

What carries the argument

The central object is In-Context Policy Improvement (ICPI), a policy improvement operator implemented as an in-context sequence-to-sequence completion. Given the current policy parameters θi and a relative-error encoding ei of the last rollout, the LLM outputs a parameter delta Δθi. Prompt examples are selected by k-nearest-neighbor retrieval over a normalized (θ, e) space from a dataset D of roughly 300 labels, with k=20 examples ordered by decreasing distance. The relative-error encoding is what makes corrections transferable across hidden physical parameters.

What would settle it

Run the same ICPI pipeline on test tasks drawn from a shifted distribution—for example, puck frictions or rope lengths outside the range seen in the improvement dataset. If the final best-policy cost no longer beats the piecewise-linear and nearest-neighbor baselines, or degrades sharply, the apparent skill is memorization of the training-task distribution rather than a transferable correction rule.

Watch

Extended reading notes

Core claim

The paper's central claim is that in-context learning with a pretrained LLM can perform iterative policy improvement for dynamic manipulation without fine-tuning or gradient updates. The method, ICPI, predicts a parameter delta from the current policy and the relative error of the latest execution, using about 300 improvement labels from similar tasks as prompt examples. Across four simulated tasks and one physical robot task, ICPI achieves the lowest final best-policy cost in the low-data regime compared with random shooting, Bayesian optimization, nearest-neighbor averaging, piecewise-linear regression, an earlier sequence-improvement LLM method, and in-weights LLM reasoning.

Load-bearing premise

The load-bearing premise is that the small improvement dataset, built from training-task instances, is representative of the test task distribution in the normalized policy-and-error space, so nearest-neighbor examples transfer corrections; all reported tests draw test tasks from the same distribution used to build that dataset.

Editorial extensions

If this is right

  • An off-the-shelf pretrained LLM can improve parametric policies for dynamic manipulation with no fine-tuning or gradient updates, as long as a small improvement dataset exists.
  • The improvement dataset can be collected once on training tasks via brute-force search or hindsight relabeling, then reused for many test tasks through nearest-neighbor retrieval.
  • Encoding the rollout as relative error to the goal matters: the same information in raw state-plus-goal form performs worse, so how the execution outcome is tokenized changes the quality of the predicted adjustment.
  • Model choice matters: newer, larger LLMs give better in-context policy improvements, indicating the capability tracks model capability rather than being a fixed property of in-context learning.
  • The method transfers from simulation to a physical robot, achieving the best final cost among the methods tested on real ball-rolling tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the operator is fit entirely in the prompt, the retrieval pool could be updated online: adding successful test-time interactions to the dataset would let the method improve from its own rollouts without any weight update.
  • The relative-error recipe—current parameters plus error to goal, next parameters—may transfer to any parametric control task with a vector-valued cost, not just manipulation.
  • The paper evaluates tasks sampled from the same distribution that generated the improvement dataset; whether the operator generalizes across shifted physical parameters, such as new friction ranges or object shapes, remains an open test.
  • Exemplar ordering is fixed by decreasing distance; testing alternative orderings or retrieval criteria could reveal how sensitive the predicted delta is to prompt layout.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes In-Context Policy Improvement (ICPI), a method that uses a pretrained LLM as a few-shot policy improvement operator for dynamic manipulation. Given a parametric policy and the relative error of a rollout, ICPI tokenizes current policy parameters and error, retrieves k similar input-output examples from a small dataset D of (policy, trajectory, delta) tuples, and prompts the LLM to output a policy parameter delta. The dataset D is constructed by computing optimal parameters θ* (via brute force or hindsight relabeling) for task instances from P(τ) and generating labels Δθ = θ* − θ for nearby executions. The method is evaluated on simulated slide and rope-swing tasks (with goal-conditioned variants) and a real ball-rolling task, against random shooting, Bayesian optimization, KNN regression baselines, ICSI, and in-weights reasoning. The paper reports that ICPI achieves the lowest best-policy cost after 20 iterations on most tasks.

Significance. If the results hold, the paper offers a compelling demonstration that pretrained LLMs can act as sample-efficient policy improvement operators for low-dimensional dynamic manipulation policies without task-specific training or fine-tuning. The method is simple, requires no gradient updates, and the authors include useful ablations (model choice, tokenization) and a physical robot experiment. The problem formulation is clean and the writing is generally clear. However, the experimental validation has important gaps that must be addressed before the central claim can be accepted.

major comments (3)
  1. [Sec. 5.1.2 / Sec. 4.4] The paper does not state whether the 100 evaluation tasks used in Sec. 5.1.2 are disjoint from the task instances used to construct the improvement dataset D described in Sec. 4.4. This is critical because ICPI retrieves in-context examples via KNN over D (Sec. 4.3). If a test task τ appears in D, the retrieved labels may include Δθ computed from the exact optimal θ* for that task, effectively providing the answer in the prompt. The central claim is that ICPI generalizes to new task instances; without a held-out task split, the comparison is invalid or at least substantially weakened. Please clarify the split explicitly, and if the current protocol does not hold out the evaluation tasks, rerun the experiments with disjoint task sets and report whether the conclusions change.
  2. [Table 1] No statistical significance tests or confidence intervals are reported. The headline claim that ICPI 'outperforms alternative methods' is not supported by point estimates alone. For example, on rope-swing, ICPI and random shooting both report mean 0.007 with standard deviations 0.012 vs 0.022, respectively, so the reported difference is not clearly meaningful. Given that 100 tasks are evaluated, paired tests or bootstrap confidence intervals across tasks should be reported for all simulated tasks. Without this, the claimed superiority is not established.
  3. [Sec. 5.1.2 / Table 1 (roll-gc-real)] The real-robot experiment compares ICPI to only one baseline (Linear KNN-20) on 10 tasks. The abstract claims superior performance 'on a physical robot,' but with a single baseline, no repeated trials, and large standard deviations (17.1 vs 33.8), the evidence is thin. Please either include additional baselines (e.g., random shooting, Bayes Opt) on the real system or temper the abstract and conclusions to describe the real-robot result as a feasibility demonstration rather than a comparative superiority claim.
minor comments (5)
  1. [Sec. 6.1] Typo: 'In this work, found that some amount of feature selection...' appears to be missing 'we'.
  2. [Sec. 4.3] The normalization of the [θ, e] vector for KNN is described only as 'normalize the policy and error terms to have comparable scale.' Please specify the normalization method (e.g., min-max or z-score) and whether the normalization parameters are computed on D only.
  3. [Sec. 5.1.1] In the Random Shooting baseline description, the context parameter c_i is not defined. Clarify whether it is a fixed constant, a decay schedule, or something task-specific.
  4. [Sec. 5.2] The model ablation compares gpt-3.5-turbo, gpt-4o-mini, and gpt-4o using point estimates only. No discussion of variance due to LLM sampling (e.g., temperature) or repeated runs is provided; please note whether outputs are deterministic or averaged over calls.
  5. [Appendix A.1] The example prompts are helpful, but the exact prompt template for all tasks, especially the fixed 'task-agnostic header' mentioned in Sec. A.1.1, should be explicitly stated so that the method is fully reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ICPI is supervised in-context regression on improvement labels and is compared against external baselines; the central claim does not reduce to its inputs by construction.

full rationale

The paper's core object is a policy improvement operator f(θi, s_i, D) → Δθi, fitted entirely in-context from a dataset D of labels Δθ = θ* − θ (Sec. 4.1, 4.4). These labels are derived by brute-force search over the same cost Eq. 1 that defines task success; this is standard supervised learning / algorithm distillation, not circular reasoning, because the method is then queried on new task instances and compared against baselines that do not share the labels (random shooting, Bayes Opt, KNN, Linear KNN, ICSI, in-weights reasoning; Sec. 5.1.1). The LLM is frozen and no parameter is fitted to the evaluation tasks. No equation in the paper is equivalent to its own input by construction, and no load-bearing claim rests on a self-citation: reference [6] (Mirchandani et al.) supplies the general ICL framing and ICSI baseline, but the paper's contribution is evaluated empirically against that baseline. The only protocol ambiguity worth noting is that Sec. 5.1.2 does not explicitly state that the 100 sampled evaluation tasks are disjoint from the task instances used to construct D in Sec. 4.4; if they were not disjoint, KNN retrieval could return labels computed for the identical task, which would weaken the generalization claim. This is a missing experimental-protocol detail rather than a circularity that can be exhibited from the paper's equations, so it does not affect the circularity score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a supervised-learning-style setup: a dataset of improvement labels D, a smoothness assumption for nearest-neighbor selection, and distributional alignment between training and test tasks. No new physical entities are introduced. The only hand-chosen quantities are method hyperparameters (k, normalization, error time index).

free parameters (3)
  • number of in-context examples k = 20
    Set to 20 without an ablation study; the KNN neighborhood size directly affects the prompt and the interpolation quality.
  • normalization scaling for KNN vector = not specified
    Policy and error terms are normalized to comparable scale (Sec. 4.3), but the exact scaling is not reported, affecting nearest-neighbor selection.
  • relative error time index t = task-specific: t=T for slide/roll, argmin over t for rope
    The choice of which state to use for the error encoding is hand-designed per task (Sec. 4.2) and is a form of feature engineering.
assumptions (4)
  • domain assumption The task distribution P(tau) is available for sampling training and test tasks, and test tasks are drawn from the same distribution used to build D.
    Sec. 4.4 and Sec. 5.1.2: dataset construction and evaluation use the same task distribution; cross-distribution transfer is not tested.
  • domain assumption The policy improvement operator is approximately smooth and locally linear in the normalized [theta, e] space, so KNN selection of in-context examples yields transferable deltas.
    Sec. 4.3: KNN lookup assumes nearest examples are most useful for predicting the delta.
  • domain assumption The hidden physical parameters (friction, mass, rope length) are sufficiently reflected in the relative error e_i to make the mapping learnable; the LLM does not need direct access to the physical parameters.
    Sec. 4.2: tokenization discards raw state and goal, providing only relative error, which the paper itself identifies as critical in Sec. 5.2 and Sec. 6.1.
  • domain assumption A brute-force or hindsight-derived optimal theta* exists and can be used to construct labels delta_theta = theta* - theta.
    Sec. 4.4: dataset construction relies on obtaining optimal or guiding executions via brute force or hindsight relabeling.

how reviews work

0 comments
Cite this review

Pith. "Pith review of In-Context Iterative Policy Improvement for Dynamic Manipulation." pith.science (2026). https://pith.science/paper/LMLGVACV

@misc{pith2026250815021,
  author       = {Pith},
  title        = {Pith review of: In-Context Iterative Policy Improvement for Dynamic Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LMLGVACV}},
  note         = {Machine review of arXiv:2508.15021}
}
read the original abstract

Attention-based architectures trained on internet-scale language data have demonstrated state of the art reasoning ability for various language-based tasks, such as logic problems and textual reasoning. Additionally, these Large Language Models (LLMs) have exhibited the ability to perform few-shot prediction via in-context learning, in which input-output examples provided in the prompt are generalized to new inputs. This ability furthermore extends beyond standard language tasks, enabling few-shot learning for general patterns. In this work, we consider the application of in-context learning with pre-trained language models for dynamic manipulation. Dynamic manipulation introduces several crucial challenges, including increased dimensionality, complex dynamics, and partial observability. To address this, we take an iterative approach, and formulate our in-context learning problem to predict adjustments to a parametric policy based on previous interactions. We show across several tasks in simulation and on a physical robot that utilizing in-context learning outperforms alternative methods in the low data regime. Video summary of this work and experiments can be found https://youtu.be/2inxpdrq74U?si=dAdDYsUEr25nZvRn.

Figures

Figures reproduced from arXiv: 2508.15021 by the authors.

Figure 1
Figure 1. We investigate in-context learning for iteratively improving policy parameters for dynamic manip [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed In-Context Policy Improvement (ICPI) method. We tokenize policy pa [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Task Cost convergence plots for the best policy so far at each step across three of our tasks comparing [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative examples of iterative in-context policy improvement using our proposed ICPI for the [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 25 canonical work pages

  1. [1]

    Z. Yang, L. Li, K. Lin, J. Wang, C.-C. Lin, Z. Liu, and L. Wang. The dawn of lmms: Prelimi- nary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421, 2023

    arXiv 2023

  2. [2]

    Ichter, A

    B. Ichter, A. Brohan, Y . Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian, D. Kalashnikov, S. Levine, Y . Lu, C. Parada, K. Rao, P. Sermanet, A. T. To- shev, V . Vanhoucke, F. Xia, T. Xiao, P. Xu, M. Yan, N. Brown, M. Ahn, O. Cortes, N. Sievers, C. Tan, S. Xu, D. Reyes, J. Rettinghouse, J. Quiambao, P. Pastor, L. Lu...

    work page 2023

  3. [3]

    Liang, W

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE Inter- national Conference on Robotics and Automation (ICRA) , pages 9493–9500, 2023. doi: 10.1109/ICRA48891.2023.10160591

    arXiv 2023

  4. [4]

    Huang, C

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models. In J. Tan, M. Toussaint, and K. Darvish, editors, Proceedings of The 7th Conference on Robot Learning , volume 229 of Proceedings of Machine Learning Research, pages 540–562. PMLR, 06–09 Nov 2023. URL https://proce...

    2023

  5. [5]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei...

    work page 1901

  6. [6]

    Mirchandani, F

    S. Mirchandani, F. Xia, P. Florence, B. Ichter, D. Driess, M. G. Arenas, K. Rao, D. Sadigh, and A. Zeng. Large language models as general pattern machines. In J. Tan, M. Toussaint, and K. Darvish, editors, Proceedings of The 7th Conference on Robot Learning , volume 229 9 of Proceedings of Machine Learning Research , pages 2498–2518. PMLR, 06–09 Nov 2023....

    work page 2023

  7. [7]

    Zhang, J

    H. Zhang, J. Ichnowski, D. Seita, J. Wang, H. Huang, and K. Goldberg. Robots of the lost arc: Self-supervised learning to dynamically manipulate fixed-endpoint cables. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 4560–4567, 2021. doi: 10.1109/ICRA48506.2021.9561630

    arXiv 2021

  8. [8]

    A. Zeng, S. Song, J. Lee, A. Rodriguez, and T. Funkhouser. Tossingbot: Learning to throw arbitrary objects with residual physics. IEEE Transactions on Robotics , 36(4):1307–1319,

    work page

Show all 36 references
  1. [9]

    Ha and S

    H. Ha and S. Song. Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding. In A. Faust, D. Hsu, and G. Neumann, editors, Proceedings of the 5th Con- ference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pages 24–33. PM...

    2022

  2. [10]

    Z. Xu, C. Chi, B. Burchfiel, E. Cousineau, S. Feng, and S. Song. Dextairity: Deformable manipulation can be a breeze. In Proceedings of Robotics: Science and Systems (RSS), 2022

    2022

  3. [11]

    C. Wang, S. Wang, B. Romero, F. Veiga, and E. Adelson. Swingbot: Learning physical fea- tures from in-hand tactile exploration for dynamic swing-up manipulation. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 5633–5640, 2020. doi:10....

    2020

  4. [12]

    Z. Xu, J. Wu, A. Zeng, J. Tenenbaum, and S. Song. Densephysnet: Learning dense physical ob- ject representations via multi-step dynamic interactions. In Proceedings of Robotics: Science and Systems, FreiburgimBreisgau, Germany, June 2019. doi:10.15607/RSS.2019.XV .046

    2019 doi

  5. [13]

    Wang and A

    Z. Wang and A. H. Qureshi. Implicit physics-aware policy for dynamic manipulation of rigid objects via soft body tools. In 2025 IEEE International Conference on Robotics and Automa- tion (ICRA). IEEE, 2025

    2025

  6. [14]

    C. Chi, B. Burchfiel, E. Cousineau, S. Feng, and S. Song. Iterative Residual Policy for Goal- Conditioned Dynamic Manipulation of Deformable Objects. In Proceedings of Robotics: Sci- ence and Systems, New York City, NY , USA, June 2022. doi:10.15607/RSS.2022.XVIII.016

    2022 doi

  7. [15]

    V . Lim, H. Huang, L. Y . Chen, J. Wang, J. Ichnowski, D. Seita, M. Laskey, and K. Goldberg. Real2sim2real: Self-supervised learning of physical single-step dynamic actions for planar robot casting. In 2022 International Conference on Robotics and Automation (ICRA) , pages 828...

    2022

  8. [16]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. V . Luxburg, S. Bengio, H. Wal- lach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Process...

    2017

  9. [17]

    Srivastava, A

    A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research, 2023

    2023

  10. [18]

    Suzgun, N

    M. Suzgun, N. Scales, N. Sch ¨arli, S. Gehrmann, Y . Tay, H. W. Chung, A. Chowdhery, Q. V . Le, E. H. Chi, D. Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. In ACL (Findings), 2023

    2023

  11. [19]

    S. Chan, A. Santoro, A. Lampinen, J. Wang, A. Singh, P. Richemond, J. McClelland, and F. Hill. Data distributional properties drive emergent in-context learning in transformers. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in 10 Neura...

    2022

  12. [20]

    Olsson, N

    C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y . Bai, A. Chen, et al. In-context learning and induction heads. arXiv preprint arXiv:2209.11895, 2022

    2022 arXiv

  13. [21]

    S. C. Chan, I. Dasgupta, J. Kim, D. Kumaran, A. K. Lampinen, and F. Hill. Transform- ers generalize differently from information stored in context vs in weights. arXiv preprint arXiv:2210.05675, 2022

    2022 arXiv

  14. [22]

    S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer. Re- thinking the role of demonstrations: What makes in-context learning work? In EMNLP, 2022

    2022

  15. [23]

    L. Sun, D. K. Jha, C. Hori, S. Jain, R. Corcodel, X. Zhu, M. Tomizuka, and D. Romeres. Inter- active planning using large language models for partially observable robotic tasks. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 14054–14061. IEEE, 2024

    2024

  16. [24]

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.-T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, B. Ichter, T. Xiao, P. Xu, A. Zeng, T. Zhang, N. Heess, D. Sadigh, J. Tan, Y . Tassa, and F. Xia. Language to rewards for robotic skill synthesis. In J. Tan, M....

    2023

  17. [25]

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. Erickson. RL-VLM-f: Rein- forcement learning from vision language foundation model feedback. In Forty-first Interna- tional Conference on Machine Learning , 2024. URL https://openreview.net/forum? id=YSoMmNWZZx

    2024

  18. [26]

    T. Kwon, N. D. Palo, and E. Johns. Language models as zero-shot trajectory generators. IEEE Robotics and Automation Letters, 9(7):6728–6735, 2024. doi:10.1109/LRA.2024.3410155

    2024

  19. [27]

    Di Palo and E

    N. Di Palo and E. Johns. Keypoint action tokens enable in-context imitation learning in robotics. In Proceedings of Robotics: Science and Systems, 2024

    2024

  20. [28]

    M. Yoo, W. K. Kim, and H. Woo. In-context policy adaptation via cross-domain skill diffusion. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 39, pages 22191– 22199, 2025

    2025

  21. [29]

    Zhang, S

    X. Zhang, S. Liu, P. Huang, W. J. Han, Y . Lyu, M. Xu, and D. Zhao. Dynamics as prompts: In- context learning for sim-to-real system identifications. IEEE Robotics and Automation Letters, 2025

    2025

  22. [30]

    Laskin, L

    M. Laskin, L. Wang, J. Oh, E. Parisotto, S. Spencer, R. Steigerwald, D. Strouse, S. Hansen, A. Filos, E. Brooks, et al. In-context reinforcement learning with algorithm distillation. ICLR, 2023

    2023

  23. [31]

    Lynch, M

    C. Lynch, M. Khansari, T. Xiao, V . Kumar, J. Tompson, S. Levine, and P. Sermanet. Learning latent plans from play. In L. P. Kaelbling, D. Kragic, and K. Sugiura, editors, Proceedings of the Conference on Robot Learning, volume 100 ofProceedings of Machine Learning Research, p...

    2020

  24. [32]

    Y . Luo, Y . Wang, K. Dong, Y . Liu, Z. Sun, Q. Zhang, and B. Song. Sirl: Self-imitation rein- forcement learning for single-step hitting tasks. In2023 International Conference on Advanced Robotics and Mechatronics (ICARM), pages 185–190. IEEE, 2023. 11

    2023

  25. [33]

    Nogueira

    F. Nogueira. Bayesian Optimization: Open source constrained global optimiza- tion tool for Python, 2014–. URL https://github.com/bayesian-optimization/ BayesianOptimization

    2014

  26. [34]

    E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, NY , USA,

    2021

  27. [2020]

    doi:10.1109/TRO.2020.2988642

    2020

  28. [2021]

    ISBN 9781450383097

    Association for Computing Machinery. ISBN 9781450383097. doi:10.1145/3442188. 3445922. URL https://doi.org/10.1145/3442188.3445922. 12 A Appendix A.1 Example Prompts We provide example prompts and responses for our method along with the other LLM baselines. The examples provid...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.