REVIEW 3 major objections 4 minor 7 cited by
A Survey of In-Context Reinforcement Learning
T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Survey maps in-context RL: agents learn new tasks by reading their own history, no weight updates needed.
desk verdict A genuinely useful survey of post-Laskin ICRL, held back by a scope rule that is too fuzzy to make the 'first comprehensive survey' claim checkable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The organizing concept is the hypothesis that the forward pass of a pretrained sequence model implements a known or unknown RL algorithm, so that conditioning on context substitutes for gradient-based learning. The survey's structural machinery is its taxonomy: it classifies work by pretraining objective (supervised log-likelihood versus reinforcement objectives), by how context is built at test time, by the benchmark-specific notion of generalization, by which theoretical guarantees are proven, and by which sequence architecture is used. These axes jointly define the field and allow the survey to map progress and open problems.
What would settle it
A concrete check would be to take a pre-Laskin recurrent meta-RL agent (e.g., one trained as in Duan et al. 2016 or Wang et al. 2016) and evaluate it on the same out-of-distribution benchmarks used in recent ICRL papers, such as Dark Room with adversarial reward configurations or held-out Procgen games; if it matches or exceeds transformer-based ICRL agents on in-context improvement, the survey's periodization and its claim that strong out-of-distribution generalization began only after Laskin et al. would be undercut.
Extended reading notes
Core claim
The paper's central claim is that ICRL constitutes an emerging area worth a dedicated survey, defined by a forward pass that implements an RL algorithm: a pretrained network, given a state and accumulated context (often multi-episode histories), outputs actions or value estimates that improve as context grows, without any backward pass. The survey identifies Laskin et al. (2023) as the milestone that coined the term and first demonstrated strong out-of-distribution generalization, and it treats earlier work (dating to Duan et al. 2016) as limited in that regard. It then taxonomizes the field along pretraining method (supervised behavior cloning versus reinforcement pretraining), test-time context construction (prompts, return-to-go estimation, cultural accumulation), demonstrated generalization by benchmark (Dark Room, Watermaze, Mujoco, Procgen, Meta-World), theoretical results (transformers provably implementing LinUCB, Thompson sampling, TD methods, FTRL), and architecture choices (causal transformers, state space models, hierarchical designs).
Load-bearing premise
The survey's load-bearing premise is its scope rule: it excludes return-conditioned and goal-conditioned methods unless they show in-context improvement, and it treats Laskin et al. (2023) as the start of modern ICRL; if that boundary is wrong, the claim of comprehensive coverage and the field's periodization would need revision.
Editorial extensions
If this is right
- If the survey's scope is right, ICRL after Laskin et al. (2023) forms a distinct research program, separate from earlier meta-RL work that only handled near-distribution tasks.
- The survey's taxonomy implies that supervised and reinforcement pretraining are two complementary routes to ICRL, with different trade-offs in sample efficiency, stability, and out-of-distribution robustness.
- The theoretical results it surveys suggest that transformers and related architectures can provably implement specific learning algorithms in-context, giving a foundation for designing agents that learn from context rather than weight updates.
- If the identified open problems are addressed, ICRL could extend to multi-agent settings, robotics with sim-to-real transfer, and large-scale pretraining where the learning algorithm itself emerges.
Reading between the lines
- The survey's periodization—treating Laskin et al. as the start of 'modern' ICRL—implicitly predicts that earlier recurrent architectures were limited by their memory and credit-assignment capacity, a claim that could be tested by re-evaluating those models on today's out-of-distribution benchmarks.
- The emphasis on in-context improvement as the defining property suggests that a practical evaluation protocol for any new ICRL method should measure performance as a function of context length, not just final return.
- The survey's coverage of theory (transformers implementing specific algorithms) hints at a possible unification: supervised pretraining may be understood as distilling a known algorithm, while reinforcement pretraining may discover new ones, and the two could be compared by the regret or sample complexity of the implemented algorithm.
- The architectural findings point to a concrete design heuristic: because transformers scale quadratically with context, state space models and hierarchical token sampling may be necessary for long-horizon ICRL, a trade-off the survey documents but does not resolve.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper surveys the emerging area of in-context reinforcement learning (ICRL), in which pretrained agents adapt to new tasks at test time by conditioning on context—such as action-observation histories—without updating network parameters. The survey organizes the literature along five axes: supervised pretraining, reinforcement pretraining, test-time context construction, test-time generalization and sample efficiency, and theory and architectures. It positions Laskin et al. (2023) as the starting point of modern ICRL, claims to be the first comprehensive survey of ICRL, and closes with open problems in multi-agent RL, robotics, and the emergence of ICRL from reinforcement pretraining.
Significance. If the survey's scope and periodization are accepted, the paper provides a useful taxonomy that complements Beck et al. (2023) by covering recent post-2023 ICRL advances, including theoretical results and architectural innovations. Strengths include the broad citation coverage, the organization into pretraining and testing phases, the discussion of benchmarks and generalization types, and the inclusion of a theory section that summarizes provable in-context implementations of RL algorithms. The survey also names concrete open problems, which is valuable for the community. However, the central contribution—comprehensiveness—rests on an unoperationalized boundary between ICRL and excluded paradigms such as RvS and goal-conditioned RL, and on a historical priority claim about Laskin et al. (2023) that is not substantiated. These issues need to be addressed before the survey's coverage claims are fully reliable.
major comments (3)
- [Section 1 and Section 2] The scope criterion for excluding RvS and goal-conditioned RL—that they 'do not demonstrate key properties of ICRL, such as in-context improvement'—is not operationalized anywhere in the paper. Section 2 defines in-context improvement only qualitatively as performance improving 'with the length of the context Ct', without specifying a threshold, a measurement protocol, a benchmark, or a comparison baseline. As a result, a reader cannot determine whether a particular excluded work (e.g., Chen et al. 2021) should have been included, and the Section 10 claim to be the 'first comprehensive survey of ICRL' is not falsifiable from the text. The authors should either provide a concrete, measurable definition of in-context improvement, or explicitly acknowledge that the boundary is a community judgment and weaken the comprehensiveness claim accordingly.
- [Section 1 and Section 3] The exclusion of Decision-Transformer-style RvS methods is inconsistent with the inclusion of Prompt-DT (Xu et al. 2022) and Hierarchical Prompt Decision Transformer (Wang et al. 2024b), which are RvS-based, condition on the same kind of trajectory context with return-to-go, and are surveyed in Section 3. The stated distinguishing property—demonstrated in-context improvement—is not evidenced for the included works, while the same property is denied to excluded works. To make the taxonomy coherent, the authors should either justify the inclusion of prompt-based RvS methods with reference to specific evidence of in-context improvement, or refine the definition of ICRL so that the boundary is drawn on a checkable feature (e.g., cross-episode context versus single-episode context, or presence of a prompt/curriculum).
- [Section 6] The claim that 'The first remarkable out-of-distribution generalization in ICRL is demonstrated by Laskin et al. [2023]' is a strong historical statement that the manuscript does not support with comparative evidence. Section 4 acknowledges that earlier works (e.g., Wang et al. 2016, Mishra et al. 2018) demonstrated limited out-of-distribution generalization, but 'remarkable' is not defined, and no benchmark comparison or quantitative criterion is provided to distinguish Laskin et al. from prior work. This claim should be qualified (e.g., 'among works that explicitly coin the term and demonstrate adversarial bandit generalization') or supported by a concrete comparison of generalization performance.
minor comments (4)
- [Section 2] Typographical error: 'recevied' should be 'received' in the sentence 'where ri is the reward recevied after executing the action ai'.
- [Throughout] There are numerous spacing artifacts in the text (e.g., 'f rom', 'V arious', 'T ransformer', 'Y u et al.', 'T o') that appear to be rendering issues in the PDF. The authors should run a proofreading pass to ensure proper spacing and capitalization.
- [Section 5] The sentence 'This approach allows the current agent to buil d upon prior experience' contains a stray space in 'buil d'; also the variable 'Y et' in the Meta-World section should be 'Yet'.
- [References] Many references are listed as 'ArXiv preprint' even when they were later published in peer-reviewed venues (e.g., Laskin et al. 2023 in ICLR, Xu et al. 2022 in ICML). Updating these to the published versions would improve the survey's utility.
Circularity Check
No material circularity: the survey organizes external literature, and its two self-citations are minor and non-load-bearing.
full rationale
This is a survey paper, not a derivation chain, so most circularity patterns do not apply. The central deliverable is an organizing taxonomy and a map of the ICRL literature after Laskin et al. (2023). The paper cites two works by its own authors: Beck et al. (2023), a meta-RL survey, is used to delimit scope by referring early ICRL work to that survey; Wang et al. (2024a) is cited as an example of ICRL for policy evaluation via temporal-difference methods implemented in Transformer forward passes. Neither citation is load-bearing in the sense of making the survey's taxonomy or conclusions reduce to those references by construction: the taxonomy is built from the cited external empirical and theoretical literature, and the periodization around Laskin et al. (2023) is a stated editorial scope rule rather than a derived mathematical claim. The boundary excluding many RvS and goal-conditioned methods is justified by appeal to 'in-context improvement,' which is defined only qualitatively as performance improving with context length, making the Section 10 'first comprehensive survey' claim difficult to check externally. That is a scope/operationalization weakness and a falsifiability concern, not a circular reduction: no equation, fitted parameter, or self-citation chain forces the coverage decision. The self-citations are routine and do not carry the organizational argument, so the appropriate finding is no significant circularity, with a minor self-citation note only.
Assumptions & free parameters
assumptions (3)
- domain assumption A context-conditioned forward pass can implement a full RL algorithm, so in-context improvement is evidence of ICRL.
- domain assumption Laskin et al. (2023) is the milestone that separates modern ICRL from earlier meta-RL work.
- domain assumption ICRL is defined broadly enough that imitation learning and TD count as RL algorithms.
Cite this review
Pith. "Pith review of A Survey of In-Context Reinforcement Learning." pith.science (2026). https://pith.science/paper/CVIQCQIY
@misc{pith2026250207978,
author = {Pith},
title = {Pith review of: A Survey of In-Context Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/CVIQCQIY}},
note = {Machine review of arXiv:2502.07978}
}
read the original abstract
Reinforcement learning (RL) agents typically optimize their policies by performing expensive backward passes to update their network parameters. However, some agents can solve new tasks without updating any parameters by simply conditioning on additional context such as their action-observation histories. This paper surveys work on such behavior, known as in-context reinforcement learning.
Figures
Forward citations
Cited by 7 Pith papers
-
LiteOdyssey: A Lightweight Reasoning AI Agent for Interpretable Rare-Disease Diagnosis
A clinician-audited diagnostic policy plus public tools lets a single unmodified LLM reach high phenotype-first rare-disease Recall@1 and modestly beat baselines on real UDN patients.
-
How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
Heavy-tailed pretraining distributions improve in-context task selection under distribution shift but worsen ICL generalization, especially in low-data regimes.
-
Discovering New Theorems via LLMs with In-Context Proof Learning in Lean
LLMs in a conjecturing-proving loop that conditions on their own prior verified Lean proofs discover more hard-to-prove theorems than baselines that generate statements and proofs together.
-
In-Context Reinforcement Learning via Communicative World Models
CORAL trains an information agent as a world model that sends concise messages to a control agent, improving in-context reinforcement learning and zero-shot adaptation.
-
LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra
The LLM Economist framework couples persona-conditioned worker agents with an in-context RL planner to search US-bracket tax schedules, yet its Saez benchmark is derived from the planner's own solution and its headlin...
-
Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks
Adversarially training a Decision-Pretrained Transformer against learned reward-poisoning attackers makes it robust to test-time reward corruption, outperforming robust bandit baselines in experiments.
-
ReBRAC-v2: The Return of the King
A fixed-recipe offline RL method combining normalizing-flow actors, categorical critics, staged training, and test-time refinement beats recent flow-based baselines by 22.5 points averaged over ten OGBench categories.
Reference graph
Works this paper leans on
-
[1]
Pieter Abbeel and Andrew Y. Ng. Exploration and apprenticeship learning in reinforcement learning. In Proceedings of the International Conference on Machine Learning , 2005
work page 2005
-
[2]
In- Context Language Learning : Architectures and Algorithms
Ekin Aky \"u rek, Bailin Wang, Yoon Kim, and Jacob Andreas. In- Context Language Learning : Architectures and Algorithms . ArXiv preprint , 2024
work page 2024
-
[3]
Minimax regret bounds for reinforcement learning
Mohammad Gheshlaghi Azar, Ian Osband, and R \' e mi Munos. Minimax regret bounds for reinforcement learning. In Proceedings of the International Conference on Machine Learning , 2017
work page 2017
-
[4]
Provable self-play algorithms for competitive reinforcement learning
Yu Bai and Chi Jin. Provable self-play algorithms for competitive reinforcement learning. In Proceedings of the International Conference on Machine Learning , 2020
work page 2020
-
[5]
Jakob Bauer, Kate Baumli, Feryal M. P. Behbahani, Avishkar Bhoopchand, Nathalie Bradley - Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks - Thompson, Hannah Openshaw, Jack Parker - Holder, Shreya Pathak, Nicolas Perez Nieves, Nemanja Rakicevic, Tim Rockt \" a ...
work page 2023
-
[6]
A Survey of Meta-Reinforcement Learning
Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, and Shimon Whiteson. A Survey of Meta-Reinforcement Learning . ArXiv preprint , 2023
work page 2023
-
[7]
o ppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, G \
Maximilian Beck, Korbinian P \"o ppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, G \"u nter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xLSTM : Extended Long Short-Term Memory . ArXiv preprint , 2024
work page 2024
-
[8]
David Brandfonbrener, Alberto Bietti, Jacob Buckman, Romain Laroche, and Joan Bruna. When does return-conditioned supervised learning work for offline reinforcement learning? In Advances in Neural Information Processing Systems , 2022
work page 2022
Show all 75 references
-
[9]
Rt-2: Vision-language-action models transfer web knowledge to robotic control
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818 , 2023
2023 arXiv
-
[10]
Learning to cooperate with unseen agent via meta-reinforcement learning
Rujikorn Charakorn, Poramate Manoonpong, and Nat Dilokthanakul. Learning to cooperate with unseen agent via meta-reinforcement learning. arXiv preprint arXiv:2111.03431 , 2021
2021 arXiv
-
[11]
Decision transformer: Reinforcement learning via sequence modeling
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. In Advances in Neural Information Processing Systems , 2021
2021
-
[12]
Contextual bandits with linear payoff functions
Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the International Conference on Artificial Intelligence and Statistics , 2011
2011
-
[13]
Leveraging procedural generation to benchmark reinforcement learning
Karl Cobbe, Christopher Hesse, Jacob Hilton, and John Schulman. Leveraging procedural generation to benchmark reinforcement learning. In Proceedings of the International Conference on Machine Learning , 2020
2020
-
[14]
Leibo, and Jakob Nicolaus Foerster
Jonathan Cook, Chris Lu, Edward Hughes, Joel Z. Leibo, and Jakob Nicolaus Foerster. Artificial Generational Intelligence : Cultural Accumulation in Reinforcement Learning . In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024
2024
-
[15]
In-context Exploration-Exploitation for Reinforcement Learning
Zhenwen Dai, Federico Tomasi, and Sina Ghiassian. In-context Exploration-Exploitation for Reinforcement Learning . ArXiv preprint , 2024
2024
-
[16]
A Survey on In-context Learning
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. A Survey on In-context Learning . ArXiv preprint , 2023
2023
-
[17]
Fang, Zhuoran Yang, and Vahid Tarokh
Juncheng Dong, Moyang Guo, Ethan X. Fang, Zhuoran Yang, and Vahid Tarokh. In- Context Reinforcement Learning Without Optimal Action Labels . In ICML 2024 Workshop on In-Context Learning , 2024
2024
-
[18]
Bartlett, Ilya Sutskever, and Pieter Abbeel
Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, and Pieter Abbeel. RL \ 2\ : Fast Reinforcement Learning via Slow Reinforcement Learning . ArXiv preprint , 2016
2016
-
[19]
ReLIC : A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI
Ahmad Elawady, Gunjan Chhablani, Ram Ramrakhya, Karmesh Yadav, Dhruv Batra, Zsolt Kira, and Andrew Szot. ReLIC : A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI . ArXiv preprint , 2024
2024
-
[20]
Rvs: What is essential for offline RL via supervised learning? In Proceedings of the International Conference on Learning Representations , 2022
Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, and Sergey Levine. Rvs: What is essential for offline RL via supervised learning? In Proceedings of the International Conference on Learning Representations , 2022
2022
-
[21]
Generalized decision transformer for offline hindsight information matching
Hiroki Furuta, Yutaka Matsuo, and Shixiang Shane Gu. Generalized decision transformer for offline hindsight information matching. In Proceedings of the International Conference on Learning Representations , 2022
2022
-
[22]
Meta-rl for multi-agent rl: Learning to adapt to evolving agents
Matthias Gerstgrasser and David C Parkes. Meta-rl for multi-agent rl: Learning to adapt to evolving agents. In Advances in Neural Information Processing Systems , 2022
2022
-
[23]
AMAGO : Scalable In-Context Reinforcement Learning for Adaptive Agents
Jake Grigsby, Linxi Fan, and Yuke Zhu. AMAGO : Scalable In-Context Reinforcement Learning for Adaptive Agents . ArXiv preprint , 2023
2023
-
[24]
AMAGO-2 : Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers
Jake Grigsby, Justin Sasek, Samyak Parajuli, Daniel Adebi, Amy Zhang, and Yuke Zhu. AMAGO-2 : Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers . ArXiv preprint , 2024
2024
-
[25]
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Albert Gu and Tri Dao. Mamba: Linear-Time Sequence Modeling with Selective State Spaces . ArXiv preprint , 2023
2023
-
[26]
Muesli: Combining improvements in policy optimization
Matteo Hessel, Ivo Danihelka, Fabio Viola, Arthur Guez, Simon Schmitt, Laurent Sifre, Theophane Weber, David Silver, and Hado Van Hasselt. Muesli: Combining improvements in policy optimization. In International conference on machine learning , 2021
2021
-
[27]
In- Context Decision Transformer : Reinforcement Learning via Hierarchical Chain-of-Thought
Sili Huang, Jifeng Hu, Hechang Chen, Lichao Sun, and Bo Yang. In- Context Decision Transformer : Reinforcement Learning via Hierarchical Chain-of-Thought . ArXiv preprint , 2024
2024
-
[28]
Decision Mamba : Reinforcement Learning via Hybrid Selective Sequence Modeling
Sili Huang, Jifeng Hu, Zhejian Yang, Liwei Yang, Tao Luo, Hechang Chen, Lichao Sun, and Bo Yang. Decision Mamba : Reinforcement Learning via Hybrid Selective Sequence Modeling . ArXiv preprint , 2024
2024
-
[29]
V-learning—a simple, efficient, decentralized algorithm for multiagent reinforcement learning
Chi Jin, Qinghua Liu, Yuanhao Wang, and Tiancheng Yu. V-learning—a simple, efficient, decentralized algorithm for multiagent reinforcement learning. Mathematics of Operations Research , 2024
2024
-
[30]
Kingma and Jimmy Ba
Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization . ArXiv preprint , 2017
2017
-
[31]
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rockt \"a schel. A Survey of Zero-shot Generalisation in Deep Reinforcement Learning . ArXiv preprint , 2021
2021
-
[32]
Daniel Freeman, Jascha Sohl-Dickstein , and J \"u rgen Schmidhuber
Louis Kirsch, James Harrison, C. Daniel Freeman, Jascha Sohl-Dickstein , and J \"u rgen Schmidhuber. Towards General-Purpose In-Context Learning Agents . In NeurIPS 2023 Foundation Models for Decision Making Workshop , 2023
2023
-
[33]
Foster, Cyril Zhang, and Aleksandrs Slivkins
Akshay Krishnamurthy, Keegan Harris, Dylan J. Foster, Cyril Zhang, and Aleksandrs Slivkins. Can large language models explore in-context? ArXiv preprint , 2024
2024
-
[34]
Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, and Volodymyr Mnih
Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Stenberg Hansen, Angelos Filos, Ethan A. Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, and Volodymyr Mnih. In-context reinforcement learning with algorithm...
2023
-
[35]
Supervised pretraining can learn in-context reinforcement learning
Jonathan Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, and Emma Brunskill. Supervised pretraining can learn in-context reinforcement learning. In Advances in Neural Information Processing Systems , 2023
2023
-
[36]
Lillicrap, Jonathan J
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. ArXiv preprint , 2019
2019
-
[37]
Transformers as Decision Makers : Provable In-Context Reinforcement Learning via Supervised Pretraining
Licong Lin, Yu Bai, and Song Mei. Transformers as Decision Makers : Provable In-Context Reinforcement Learning via Supervised Pretraining . ArXiv preprint , 2023
2023
-
[38]
Emergent agentic transformer from chain of hindsight experience
Hao Liu and Pieter Abbeel. Emergent agentic transformer from chain of hindsight experience. In Proceedings of the International Conference on Machine Learning , 2023
2023
-
[39]
Goal-conditioned reinforcement learning: Problems and solutions
Minghuan Liu, Menghui Zhu, and Weinan Zhang. Goal-conditioned reinforcement learning: Problems and solutions. In Proceedings of the International Joint Conference on Artificial Intelligence , 2022
2022
-
[40]
Foerster, Satinder Singh, and Feryal M
Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob N. Foerster, Satinder Singh, and Feryal M. P. Behbahani. Structured state space models for in-context reinforcement learning. In Advances in Neural Information Processing Systems , 2023
2023
-
[41]
Luckeciano C. Melo. Transformers are meta-reinforcement learners. In Proceedings of the International Conference on Machine Learning , 2022
2022
-
[42]
Offline pre-trained multi-agent decision transformer
Linghui Meng, Muning Wen, Chenyang Le, Xiyun Li, Dengpeng Xing, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, Yaodong Yang, et al. Offline pre-trained multi-agent decision transformer. Machine Intelligence Research , 2023
2023
-
[43]
A simple neural attentive meta-learner
Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta-learner. In Proceedings of the International Conference on Learning Representations , 2018
2018
-
[44]
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature , 2015
2015
-
[45]
First- Explore , then Exploit : Meta-Learning to Solve Hard Exploration-Exploitation Trade-Offs
Ben Norman and Jeff Clune. First- Explore , then Exploit : Meta-Learning to Solve Hard Exploration-Exploitation Trade-Offs . ArXiv preprint , 2023
2023
-
[46]
Do LLM Agents Have Regret ? A Case Study in Online Learning and Games
Chanwoo Park, Xiangyu Liu, Asuman Ozdaglar, and Kaiqing Zhang. Do LLM Agents Have Regret ? A Case Study in Online Learning and Games . ArXiv preprint , 2024
2024
-
[47]
Efficient off-policy meta-reinforcement learning via probabilistic context variables
Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, and Deirdre Quillen. Efficient off-policy meta-reinforcement learning via probabilistic context variables. In Proceedings of the International Conference on Machine Learning , 2019
2019
-
[48]
Generalization to New Sequential Decision Making Tasks with In-Context Learning
Sharath Chandra Raparthy, Eric Hambro, Robert Kirk, Mikael Henaff, and Roberta Raileanu. Generalization to New Sequential Decision Making Tasks with In-Context Learning . ArXiv preprint , 2023
2023
-
[49]
Wang, Zeb Kurth - Nelson, Siddhant M
Samuel Ritter, Jane X. Wang, Zeb Kurth - Nelson, Siddhant M. Jayakumar, Charles Blundell, Razvan Pascanu, and Matthew Botvinick. Been there, done that: Meta-learning with episodic recall. In Proceedings of the International Conference on Machine Learning , 2018
2018
-
[50]
A tutorial on thompson sampling
Daniel J Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen, et al. A tutorial on thompson sampling. Foundations and Trends in Machine Learning , 2018
2018
-
[51]
o ppel, Johannes Brandstetter, G \
Thomas Schmied, Thomas Adler, Vihang Patil, Maximilian Beck, Korbinian P \"o ppel, Johannes Brandstetter, G \"u nter Klambauer, Razvan Pascanu, and Sepp Hochreiter. A Large Recurrent Action Model : xLSTM enables Fast Inference for Robotics Tasks . ArXiv preprint , 2024
2024
-
[52]
Proximal policy optimization algorithms
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. ArXiv preprint , 2017
2017
-
[53]
A primal-dual perspective of online learning algorithms
Shai Shalev-Shwartz and Yoram Singer. A primal-dual perspective of online learning algorithms. Machine Learning , 2007
2007
-
[54]
Cross-episodic curriculum for transformer agents
Lucy Xiaoyang Shi, Yunfan Jiang, Jake Grigsby, Linxi Fan, and Yuke Zhu. Cross-episodic curriculum for transformer agents. In Advances in Neural Information Processing Systems , 2023
2023
-
[55]
Transformers as Game Players : Provable In-context Game-playing Capabilities of Pre-trained Models
Chengshuai Shi, Kun Yang, Jing Yang, and Cong Shen. Transformers as Game Players : Provable In-context Game-playing Capabilities of Pre-trained Models . ArXiv preprint , 2024
2024
-
[56]
In- Context Reinforcement Learning for Variable Action Spaces
Viacheslav Sinii, Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman, and Sergey Kolesnikov. In- Context Reinforcement Learning for Variable Action Spaces . ArXiv preprint , 2023
2023
-
[57]
Jimmy T. H. Smith, Andrew Warrington, and Scott W. Linderman. Simplified state space layers for sequence modeling. In Proceedings of the International Conference on Learning Representations , 2023
2023
-
[58]
Stadie, Ge Yang, Rein Houthooft, Xi Chen, Yan Duan, Yuhuai Wu, Pieter Abbeel, and Ilya Sutskever
Bradly C. Stadie, Ge Yang, Rein Houthooft, Xi Chen, Yan Duan, Yuhuai Wu, Pieter Abbeel, and Ilya Sutskever. Some Considerations on Learning to Explore via Meta-Reinforcement Learning . ArXiv preprint , 2019
2019
-
[59]
Reinforcement Learning: An Introduction (2nd Edition)
Richard S Sutton and Andrew G Barto. Reinforcement Learning: An Introduction (2nd Edition) . MIT press, 2018
2018
-
[60]
Richard S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning , 1988
1988
-
[61]
Mujoco: A physics engine for model-based control
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In Proceedings of the International Conference on Intelligent Robots and Systems , 2012
2012
-
[62]
Tsitsiklis and Benjamin Van Roy
John N. Tsitsiklis and Benjamin Van Roy. Average cost temporal-difference learning. Automatica , 1999
1999
-
[63]
Gomez, Lukasz Kaiser, and Illia Polosukhin
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems , 2017
2017
-
[64]
Wang, Zeb Kurth-Nelson , Dhruva Tirumala, Hubert Soyer, Joel Z
Jane X. Wang, Zeb Kurth-Nelson , Dhruva Tirumala, Hubert Soyer, Joel Z. Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, and Matt Botvinick. Learning to reinforcement learn. ArXiv preprint , 2016
2016
-
[65]
Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning
Jiuqi Wang, Ethan Blaser, Hadi Daneshmand, and Shangtong Zhang. Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning . ArXiv preprint , 2024
2024
-
[66]
Hierarchical Prompt Decision Transformer : Improving Few-Shot Policy Generalization with Global and Adaptive Guidance
Zhe Wang, Haozhu Wang, and Yanjun Qi. Hierarchical Prompt Decision Transformer : Improving Few-Shot Policy Generalization with Global and Adaptive Guidance . ArXiv preprint , 2024
2024
-
[67]
Large Sequence Models for Sequential Decision-Making : A Survey
Muning Wen, Runji Lin, Hanjing Wang, Yaodong Yang, Ying Wen, Luo Mai, Jun Wang, Haifeng Zhang, and Weinan Zhang. Large Sequence Models for Sequential Decision-Making : A Survey . ArXiv preprint , 2023
2023
-
[68]
Tenenbaum, and Chuang Gan
Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Joshua B. Tenenbaum, and Chuang Gan. Prompting decision transformer for few-shot policy generalization. In Proceedings of the International Conference on Machine Learning , 2022
2022
-
[69]
Meta- Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning
Tengye Xu, Zihao Li, and Qinyuan Ren. Meta- Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning . In Proceedings of the International Conference on Machine Learning , 2024
2024
-
[70]
Meta- World : A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta- World : A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning . ArXiv preprint , 2021
2021
-
[71]
A survey on model compression for large language models
Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. A survey on model compression for large language models. Transactions of the Association for Computational Linguistics , 2024
2024
-
[72]
Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, and Shimon Whiteson
Luisa M. Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, and Shimon Whiteson. Varibad: A very good method for bayes-adaptive deep RL via meta-learning. In Proceedings of the International Conference on Learning Representations , 2020
2020
-
[73]
Emergence of In-Context Reinforcement Learning from Noise Distillation
Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin, Viacheslav Sinii, and Sergey Kolesnikov. Emergence of In-Context Reinforcement Learning from Noise Distillation . ArXiv preprint , 2023
2023
-
[74]
N- Gram Induction Heads for In-Context RL : Improving Stability and Reducing Data Needs
Ilya Zisman, Alexander Nikulin, Andrei Polubarov, Nikita Lyubaykin, and Vladislav Kurenkov. N- Gram Induction Heads for In-Context RL : Improving Stability and Reducing Data Needs . ArXiv preprint , 2024
2024
-
[75]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.