Pith. sign in

REVIEW 4 major objections 5 minor 32 references

ReflexGrad: Within-Episode Failure Recovery in LLM Agents via Progress-Gated Dual-Process Routing

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ReflexGrad claims that a progress-gated dual-process router lets an LLM agent recover from its own failed actions within a single episode, reaching 67% zero-shot success on ALFWorld without demonstrations.

desk verdict A genuinely new architecture with a compelling zero-shot question, but the submitted evidence is a curated 9-task sample, the headline 134-task result is missing from the body, and the synergy diagnostic is circular. read the letter →

arxiv 2511.14584 v4 pith:RLABCV43 submitted 2025-11-18 cs.LG cs.AI

Ankush Kadu , Aswanth Krishnan This is my paper
classification cs.LGcs.AI
keywords ReflexGraddual-processroutingwithin-episodefailurerecoveryzero-shotLLMagentsTextReflexionALFWorldprogressgating
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that an LLM agent can recover from its own failures within a single episode by combining two complementary processes: a fast, continuous refinement loop (TextGrad-style) and a slow, causal diagnostic loop (Reflexion-style), gated by measured progress. If true, this would mean zero-shot agents do not need demonstrations or fine-tuning to learn from their mistakes; the architecture itself provides the learning signal. The paper reports 67% success on first exposure (Trial 0) and 78% after one trial on 9 ALFWorld environments, with zero action loops, and claims this approaches few-shot baselines (91-96%) from prior work in a strictly harder zero-shot setting. The abstract additionally reports a larger lift to 75.4% on 134 tasks with a different model, so the exact quantitative claim depends on which setting is being referenced.

What carries the argument

The load-bearing mechanism is the progress-gated dual-process router: a fast TextGrad loop that continuously refines the policy prompt every k=3 steps, and a slow Reflexion loop that performs history-aware causal diagnosis when m=5 consecutive low-progress scores fire. The routing gate converts progress scores into a discrete switch, and a deterministic priority merge integrates the outputs. Supporting machinery includes LLM-based hierarchical TODO decomposition (with dual verification of completion), a three-tier hierarchical memory (working, consolidated, episodic) with a forgetting curve, and LLM-based semantic retrieval for cross-task transfer.

What would settle it

Run ReflexGrad on the full ALFWorld suite (or a random sample with named game IDs, multiple seeds) and compare Trial-0 success to 67%. Also run a compute-matched ablated version with the progress gate removed: if the gate-free version matches or exceeds the gated version, or if the full-suite success falls far below 67%, the central claim is refuted.

Watch

Extended reading notes

Core claim

ReflexGrad's central claim is that the post-failure trajectory contains the information needed to escape a wrong approach, and that no published architecture acts on it within a single episode. The system routes between a fast process (TextGrad-style continuous refinement every k=3 steps) and a slow process (Reflexion-style causal diagnosis when m=5 consecutive low-progress scores trigger a routing gate), with a deterministic priority merge keeping the natural-language policy coherent. Each slow activation produces a reproducible trigger, a causal diagnostic, and a verified fix. The paper argues that the triple coupling of TODO decomposition, reflexion, and TextGrad — not model scale — is th

Load-bearing premise

The evaluation's external validity rests on the assumption that 9 hand-picked ALFWorld environments spanning 3 task types fairly represent the full benchmark; if these tasks are unrepresentative or cherry-picked, the headline success rates and the ablation ordering collapse.

Editorial extensions

If this is right

  • Zero-shot LLM agents can learn from their mistakes within a single episode, without demonstrations or fine-tuning.
  • Failure recovery can be engineered as a routing problem rather than a scaling problem; the gating thresholds (k, m) become tunable hyperparameters.
  • The triple coupling claim implies that removing any one component should degrade performance; the paper's ablations (33-50% success for component-only variants) support this.
  • If the mechanism holds, the same progress-gated router could be ported to other interactive environments beyond ALFWorld.
  • The within-episode learning signal could reduce the need for episodic memory across trials, though the paper shows cross-trial transfer still adds 11pp.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's two quantitative claims — 67%/78% on 9 environments with GPT-5, and 75.4% on 134 tasks with Qwen-3-8B — are not reconciled; a reader should not assume they measure the same benchmark or model configuration.
  • If the routing gate is truly the driver, then sweeping k (refinement interval) and m (low-progress threshold) across domains could yield a practical recipe for zero-shot agent stabilization.
  • The 9-environment benchmark is small and hand-selected; the claim would be much stronger if replicated on the full ALFWorld suite or a randomly sampled subset with named instances.
  • The 100% alignment metric measures process consistency; whether perfect alignment is necessary or sufficient for the success rate is not established by the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes ReflexGrad, a dual-process architecture that couples LLM-based TODO decomposition, Reflexion-style causal reflection, and TextGrad-style textual gradient optimization for zero-shot LLM agents. The stated contribution is that this triple coupling enables within-episode failure recovery, zero-shot first-trial success, and cross-trial transfer without demonstrations. The full-text evaluation reports 67% (6/9) success on Trial 0 and 78% (7/9) on Trial 1 on 9 hand-picked ALFWorld environments, zero action loops, and 100% TextGrad-action alignment, with ablations of 33%, 44%, 50%, 67%, and 78%. The submitted abstract, however, reports a substantially different experiment: 134 ALFWorld tasks, Qwen-3-8B, n=10 seeds, +40.3pp over a baseline, and p-values, none of which appear in the full text. The paper also claims that the 67% zero-shot result is competitive with prior few-shot baselines (91–96%) in a strictly harder setting.

Significance. If the claimed synergistic architecture were supported by rigorous evidence, it would be of interest to the LLM-agent community: combining hierarchical planning, episodic reflection, and textual gradient optimization is a plausible and potentially productive direction. The paper includes algorithmic pseudocode, a detailed execution trace, and an explicit limitations section, and it promises open-source releases. However, the manuscript in its current form does not establish the central claim. The abstract and full text describe different evaluations; the full-text evaluation is only 9 environments with no error bars, no per-seed variance, no game IDs, and hand-picked selection; and the key 'alignment' evidence of synergy is circular by construction. The paper's strengths are therefore outweighed by load-bearing evidential failures.

major comments (4)
  1. [Abstract vs. §6.1, Table 1, §7.3] The abstract reports that on ALFWorld 134 tasks, n=10 seeds, no demonstrations, ReflexGrad lifts Qwen-3-8B from 35.1% to 75.4% (+40.3pp), with p-values against LATS, ToT, and Self-Refine, and a GPT-5 result of 46.3%→88.1%. Nowhere in the full text — including the appendix — are this task count, this model, these seeds, or these p-values reported. The full-text evaluation (§6.1, Table 1) is 9 environments, 2 trials, GPT-5/GPT-4o-mini, and §7.3 explicitly states 'Our experiments use 9 environments over 2 trials.' The abstract's headline empirical claim is therefore unverifiable from the submission and is contradicted by the body. This is the central quantitative claim of the paper; the authors must either add the 134-task experiment to the full text or correct the abstract.
  2. [§6.1, Table 1, §6.3] The 9-task benchmark has no game IDs, no selection protocol, no random sampling, no per-seed variance, and no confidence intervals. The success counts are 6/9 and 7/9 over a single configuration; Table 2's ablations (33%, 44%, 50%, 67%, 78%) are likewise single-point estimates. With n=9 environments, the differences between ReflexGrad (67%) and Sequential Combination (50%) correspond to about 1.5 environments, so the claimed 17–34pp improvement over ablations is not statistically supported. The comparison in Table 1 is also against aggregate few-shot results from prior work on different tasks, models, and settings; the text itself concedes 'direct comparison is not straightforward.' The headline 'competitive with few-shot baselines (91–96%)' is therefore not established by the reported data.
  3. [§4.4, Algorithm 2; §5.4] The 100% TextGrad-action alignment is circular. Algorithm 2 takes the executed action a_t as input and asks the LLM 'how should we improve' given (s_t, a_t, r_t, s_{t+1}, past reflexions); the recommendation is therefore generated after seeing the executed action. Measuring whether the recommendation matches the same executed action is definitionally guaranteed to be high and cannot diagnose inter-component synergy. The 'causal validation' in §5.4 — that removing reflexions drops alignment to 81% and success to 52% — is explicitly described as 'not reported in main results' and is given with no protocol or data. This is load-bearing because 'perfect triple synergy' is a central contribution claim.
  4. [Abstract vs. §3, §4.3, Table 3] The abstract's routing mechanism is absent from the method. The abstract states a fast process runs 'every k=3 steps' and 'm=5 consecutive low-progress scores fire a routing gate.' The full text has no routing gate or score threshold m: §4.3 and Algorithm 1 trigger reflexion 'every 5 steps or on failure,' Table 3 sets the history window k=5, and Figure 1 says the optimizer synthesizes gradients every 3 steps while §6 says gradient computation occurs every step. This is not a notational nitpick; it changes the architecture being described and prevents reproduction of the abstract's claimed mechanism.
minor comments (5)
  1. [§4.4, Eq. (11)] The LLM-Merge operation is never instantiated; no prompt, template, or procedure is provided for merging textual gradients into the policy prompt. Likewise, §5.1 refers to 'gradient magnitudes' and 'large gradients early... small refinements later' without defining how a textual gradient's magnitude is measured.
  2. [§5.2] The quantitative analyses '89% (42/47)' and '91% (49/54)' are based on manual inspection, but no annotation criteria, inter-rater agreement, or raw retrieval logs are provided. As presented, these numbers are not independently checkable.
  3. [References] Reference [7] is labeled 'Tree of Thoughts' but has a mismatched author list; several references ([5], [15], [16], [23]) are not clearly tied to the claims they are cited for, and no URLs or version identifiers are given for baseline implementations.
  4. [§6.1, §A.3] There are internal inconsistencies in experimental accounting: §6.1 says agents run for 2 trials, while §A.3 says 'across 9 envs×4 trials'; the abstract mentions routing gates not in the body; the arXiv title ('Within-Episode Failure Recovery') differs from the full-text title ('Three-Way Synergistic Architecture'). Please harmonize all versions.
  5. [§7.3] The limitations section explicitly states that scaling to hundreds or thousands of tasks 'may require more sophisticated memory management' and that evaluation on 9 environments over 2 trials is the basis of the claims. This is an honest statement, but it directly undercuts the abstract's 134-task claim and the generalization title claims.

Circularity Check

1 steps flagged · score 7.0 of 10

The mechanistic '100% synergy/alignment' evidence is circular: TextGrad's gradient is generated from the executed action, and 'alignment' is then measured against that same action.

  1. self definitional [§4.4 (Algorithm 2); §5.4; §6.1 Metrics; §6.2 Table 1]
    "Algorithm 2: "Input: State s_t, action a_t, outcome (r_t, s_{t+1}), prompt p_θ, past reflexions {ρ_i}_{i<t}" ... "g_t ← LLM(prompt_grad, s_t, a_t, r_t, s_{t+1}, {ρ_i}_{i<t})" — §6.1: "Alignment: Percentage of steps where TextGrad recommendation matches executed action" — §5.4: "The 100% TextGrad-action alignment (23/23 recommendation-action pairs match) provides a diagnostic signal that synergistic coupling is functioning correctly.""

    The TextGrad recommendation is produced by an LLM call whose inputs include the already-executed action a_t. The alignment metric checks whether that recommendation equals a_t. Asking "how should we improve?" while conditioning on the action that was just taken makes the recommendation-action match a tautology (up to LLM compliance), not evidence of information flow between reflexion, TODO, and gradient components. All of the paper's "perfect synergy" claims — no component conflicts, bidirectional information flow, and the causal validation that removing reflexions drops alignment only to 81% — derive from this by-construction identity, since a_t remains an input even in the ablation. The celebrated mechanistic result is therefore forced by the definition of the gradient and the metric.

full rationale

The central circularity is in the paper's own mechanistic evidence: Algorithm 2 feeds the executed action into the function that produces TextGrad's recommendation, and §6.1 defines alignment as agreement between that recommendation and the executed action. The 100% alignment reported in §5.4 and §6.2 is therefore an identity-by-construction, not an empirical discovery about synergistic coupling. This reduces the paper's headline mechanistic claim — 'perfect triple synergy' — to its own input, warranting a score near the 6–8 boundary; I assign 7 because the zero-shot success rates, ablations, and loop counts remain independent empirical content and are not themselves circular. Separately, the submitted abstract's 134-task Qwen-3-8B result with n=10 seeds and p-values is absent from the full text, which reports 9 hand-picked GPT-5 environments over 2 trials and even states in §7.3 'Our experiments use 9 environments over 2 trials.' That is a verifiability/misreporting failure, not a circularity failure, and I do not let it inflate the circularity score beyond the alignment tautology. No load-bearing self-citations or imported uniqueness theorems are present; the cited Reflexion and TextGrad works are external to these authors.

Assumptions & free parameters 7 free parameters · 6 assumptions · 2 invented entities

The central architecture relies on LLM reasoning as an oracle for decomposition, diagnosis, merging, and verification. The measured numbers are produced by hand-picked hyperparameters and a hand-picked 9-task benchmark, and the abstract/full-text mismatch means the contribution is not a single well-defined claim. Nothing here is derived from first principles; the main invented construct is the 'textual gradient,' which is not observable outside the LLM's own text generation.

free parameters (7)
  • History window size k = 5
    Hand-chosen (§A.2); controls reflexion context and working-memory size, and therefore every measured performance number.
  • Reflexion generation frequency = every 5 steps or on failure
    Hand-chosen (§A.2); determines when the slow causal-diagnosis loop fires.
  • Routing gate threshold m (abstract-only) = 5 consecutive low-progress scores
    Central to the metadata-abstract version of ReflexGrad, but not described in the full text.
  • Memory retrieval top-k = 6
    Hand-chosen (§A.2); limits how many memories are considered for cross-task transfer.
  • Consolidated memory strength threshold = 3.0
    Hand-chosen (§A.2); filters memories before LLM retrieval in Algorithm 3.
  • Forgetting decay rate = 0.995/hour
    Hand-chosen (§A.2, Eq. 12); directly controls which memories survive and hence cross-trial transfer.
  • Compression token budgets = 350/150/100 tokens
    Hand-chosen (§A.2); the paper claims tiered compression raises success from 58% to 67%.
assumptions (6)
  • domain assumption LLM semantic reasoning can decompose tasks, diagnose failure causes, merge prompts, and verify TODO completion without demonstrations
    Used throughout §4; if this fails, the whole architecture cannot perform the claimed zero-shot learning.
  • domain assumption The 9 selected ALFWorld environments are representative of ALFWorld
    §6.1 constructs a benchmark of 9 environments from 3 task types without listing game IDs or random selection; 67% is 6/9.
  • domain assumption An LLM-generated textual 'gradient' can be treated as a real optimization gradient in Eq. (3)
    §4.4; there is no mathematical gradient, no loss surface, and no convergence guarantee for LLM-Merge.
  • domain assumption Prior-work baseline success rates (91–96%) are comparable to the paper's 67% Trial-0 zero-shot rate
    Table 1 explicitly notes baselines are aggregate few-shot results from different setups; the comparison is not controlled.
  • ad hoc to paper Alignment between TextGrad recommendation and executed action measures synergy
    §5.4 treats 100% alignment as causal evidence, but Algorithm 2 computes the gradient from the executed action, so alignment is forced.
  • domain assumption The forgetting curve and strength heuristics produce useful memory without harmful loss
    §4.6 introduces decay and strength scoring; no ablation is run on these constants.
invented entities (2)
  • Textual gradient g_t
    purpose: An LLM-generated natural-language token used in Eq. (3) to update the policy prompt as if it were a gradient
    There is no measurable gradient; the 'gradient' is an LLM output conditioned on the same action it is later claimed to align with, and it has no external falsifiable handle.
  • Reflexion strength score
    purpose: Heuristic priority score for storing, retrieving, and decaying memories
    Composed of success weighting, urgency keyword bonuses, and brevity bonuses; no validation against ground-truth memory utility.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReflexGrad: Within-Episode Failure Recovery in LLM Agents via Progress-Gated Dual-Process Routing." pith.science (2026). https://pith.science/paper/RLABCV43

@misc{pith2026251114584,
  author       = {Pith},
  title        = {Pith review of: ReflexGrad: Within-Episode Failure Recovery in LLM Agents via Progress-Gated Dual-Process Routing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RLABCV43}},
  note         = {Machine review of arXiv:2511.14584}
}
abstract

We present ReflexGrad, a dual-process architecture for within-episode failure recovery in LLM agents without demonstrations. When agents commit to a wrong approach early and exhaust the step budget, the post-failure trajectory contains the information to escape -- but no published architecture acts on it within a single episode. ReflexGrad routes between a fast process (TextGrad-style continuous refinement every $k{=}3$ steps) and a slow process (Reflexion-style causal diagnosis when $m{=}5$ consecutive low-progress scores fire a routing gate). A deterministic priority merge keeps the natural-language policy coherent, and each slow activation emits three observable artifacts: a reproducible trigger, a causal diagnostic, and a verified fix. On ALFWorld 134 tasks, $n{=}10$ seeds, no demonstrations, ReflexGrad lifts Qwen-3-8B from $35.1\%$ to $75.4\%$ ($+40.3$pp), beating compute-matched 1-shot LATS by $+2.7$pp ($p{\approx}0.01$), ToT by $+5.7$pp ($p{<}10^{-4}$), and Self-Refine by $+6.7$pp ($p{<}10^{-5}$); on GPT-5 the lift is $46.3{\to}88.1\%$ ($+41.8$pp). The $1.5$pp cross-model difference is within seed noise ($p{\approx}0.13$), suggesting that the routing mechanism, rather than model scale, is the primary source of the gain. Code, prompts, per-seed logs, and sensitivity sweeps are released.

Figures

Figures reproduced from arXiv: 2511.14584 by the authors.

Figure 1
Figure 1. ReflexGrad Dual-Loop Self-Evolution Mechanism. The system operates through continuous execution with two parallel feedback loops around a central episodic memory core. Forward Pass: (1) Action selection receives optimized policy + reflexion insights, (2) Execute action in environment, (3) Compute TextGrad loss. Backward Pass: (4) TextGrad backward propagation with reflexion context, (5) Append gradient to policy wit… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 6 linked inside Pith

  1. [1]

    Reflexion: Language Agents with Verbal Reinforcement Learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language Agents with Verbal Reinforcement Learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2023. 27 A PREPRINT - NOVEMBER 18, 2025

    2023

  2. [2]

    Differentiation

    Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, James Zou, and Carlos Guestrin. TextGrad: Automatic "Differentiation" via Text. Nature, 634:245–252, 2024

    2024

  3. [3]

    ALFWorld: Aligning Text and Em- bodied Environments for Interactive Learning

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALFWorld: Aligning Text and Em- bodied Environments for Interactive Learning. InInternational Conference on Learning Representations (ICLR), 2021

    2021

  4. [4]

    ReAct: Synergizing Reasoning and Acting in Language Models.arXiv preprint arXiv:2210.03629, 2023

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing Reasoning and Acting in Language Models.arXiv preprint arXiv:2210.03629, 2023

    arXiv 2023

  5. [5]

    A Survey of Zero-shot Generalisation in Deep Reinforcement Learning.Journal of Artificial Intelligence Research, 76:201–264, 2023

    Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rocktäschel. A Survey of Zero-shot Generalisation in Deep Reinforcement Learning.Journal of Artificial Intelligence Research, 76:201–264, 2023

    2023

  6. [6]

    MemoryBank: Enhancing Large Language Models with Long-Term Memory

    Wanjun Zhong, Lianghong Guo, Qiqi Gao, and Yanlin Wang. MemoryBank: Enhancing Large Language Models with Long-Term Memory. InAAAI Conference on Artificial Intelligence, 2024

    2024

  7. [7]

    Tree of Thoughts: Delib- erate Problem Solving with Large Language Models

    Shibo Hu, Shunyu Yao, and Karthik Narasimhan. Tree of Thoughts: Delib- erate Problem Solving with Large Language Models. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  8. [8]

    AgentReview: Exploring Peer Review Dynamics with LLM Agents

    Yiqiao Chen, Ryo Kamoi, Shijia Peng, Jiaxin Xin, Frederic Kroeger, Navid Beigi, Yilun Zhou, Bo Pang, and Caiming Xiong. AgentReview: Exploring Peer Review Dynamics with LLM Agents. InConference on Empirical Methods in Natural Language Processing (EMNLP), 2024

    2024

Show all 32 references
  1. [9]

    MetaReflection: Learning Instructions for Language Agents using Past Reflections.arXiv preprint arXiv:2405.13009, 2024

    Seonghyeon Park, Seungone Kang, and Minjoon Choi. MetaReflection: Learning Instructions for Language Agents using Past Reflections.arXiv preprint arXiv:2405.13009, 2024

    2024 arXiv

  2. [10]

    REMO: Retrieval- Enhanced Memory Optimization for Language Agents.arXiv preprint arXiv:2508.14521, 2025

    Chen Wang, Zheng Liu, Yiming Zhang, and Wei Chen. REMO: Retrieval- Enhanced Memory Optimization for Language Agents.arXiv preprint arXiv:2508.14521, 2025

    2025 arXiv

  3. [11]

    Large Language Models Are Human- Level Prompt Engineers

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large Language Models Are Human- Level Prompt Engineers. InInternational Conference on Learning Repre- sentations (ICLR), 2023

    2023

  4. [12]

    Self-Refine: Iterative Refinement with Self-Feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and others. Self-Refine: Iterative Refinement with Self-Feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2023....

    2023

  5. [13]

    AriGraph: Learning Knowl- edge Graph World Models with Episodic Memory

    Jiashuo Ji, Zheyuan Wang, Yiyang Chen, Ran Zhang, Jidong Li, Yuanchen Liu, Weixiong Zhang, Hao Luo, and Yang Liu. AriGraph: Learning Knowl- edge Graph World Models with Episodic Memory. InInternational Joint Conference on Artificial Intelligence (IJCAI), 2025

    2025

  6. [14]

    Reasoning with Language Model is Planning with World Model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with Language Model is Planning with World Model. InConference on Empirical Methods in Natural Language Processing (EMNLP), 2023

    2023

  7. [15]

    RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit

    Jianlyu Dong, Jiachun Xu, Dawei Zhang, Man Luo, Deyi Xiong, and Qingqing Li. RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit. InConference on Empirical Methods in Natural Language Processing (EMNLP), 2023

    2023

  8. [16]

    Meta-Learning for Fast Adaptation of LLM Agents.arXiv preprint arXiv:2402.11791, 2024

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Meta-Learning for Fast Adaptation of LLM Agents.arXiv preprint arXiv:2402.11791, 2024

    2024 arXiv

  9. [17]

    Compositional Generalization through Meta Sequence-to- Sequence Learning.Advances in Neural Information Processing Systems (NeurIPS), 31, 2018

    Brenden M Lake. Compositional Generalization through Meta Sequence-to- Sequence Learning.Advances in Neural Information Processing Systems (NeurIPS), 31, 2018

    2018

  10. [18]

    Human Memory: A Proposed System and Its Control Processes.Psychology of Learning and Motivation, 2:89–195, 1968

    Richard C Atkinson and Richard M Shiffrin. Human Memory: A Proposed System and Its Control Processes.Psychology of Learning and Motivation, 2:89–195, 1968

    1968

  11. [19]

    What Are the Differences between Long-term, Short-term, and Working Memory?Progress in Brain Research, 169:323–338, 2008

    Nelson Cowan. What Are the Differences between Long-term, Short-term, and Working Memory?Progress in Brain Research, 169:323–338, 2008

    2008

  12. [20]

    Neural Episodic Control

    Alexander Pritzel, Benigno Uria, Sriram Srinivasan, Adrià Puigdomènech, Oriol Vinyals, Demis Hassabis, Daan Wierstra, and Charles Blundell. Neural Episodic Control. InInternational Conference on Machine Learning (ICML), 2017

    2017

  13. [21]

    Neural Turing Machines

    Alex Graves, Greg Wayne, and Ivo Danihelka. Neural Turing Machines. arXiv preprint arXiv:1410.5401, 2014

    2014 arXiv

  14. [22]

    Experience Replay for Continual Learning

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience Replay for Continual Learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  15. [23]

    On the Sample Complexity of Reinforcement Learning

    Sham M Kakade. On the Sample Complexity of Reinforcement Learning. PhD thesis, University College London, 2003

    2003

  16. [24]

    Planning in a Hierarchy of Abstraction Spaces.Artificial Intelligence, 5(2):115–135, 1974

    Earl D Sacerdoti. Planning in a Hierarchy of Abstraction Spaces.Artificial Intelligence, 5(2):115–135, 1974

    1974

  17. [25]

    Recent Advances in Hierarchical Reinforcement Learning.Discrete Event Dynamic Systems, 13(1):41–77, 2003

    Andrew G Barto and Sridhar Mahadevan. Recent Advances in Hierarchical Reinforcement Learning.Discrete Event Dynamic Systems, 13(1):41–77, 2003. 29 A PREPRINT - NOVEMBER 18, 2025

    2003

  18. [26]

    Lan- guage Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Lan- guage Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents. InInternational Conference on Machine Learning (ICML), 2022

    2022

  19. [27]

    LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei- Lun Chao, and Yu Su. LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models. InInternational Conference on Computer Vision (ICCV), 2023

    2023

  20. [28]

    Automated Planning: Theory and Practice

    Malik Ghallab, Dana Nau, and Paolo Traverso. Automated Planning: Theory and Practice. Morgan Kaufmann, 2004

    2004

  21. [29]

    REBACT: Episodic Memory with Be- havioral Trajectory Learning for Embodied AI Agents

    Thomas Carta, Clément Romac, Thomas Wolff, Sylvain Lamprier, Olivier Sigaud, and Pierre-Yves Oudeyer. REBACT: Episodic Memory with Be- havioral Trajectory Learning for Embodied AI Agents. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  22. [30]

    ReflAct: Reflection- Augmented Action Learning for Embodied Agents.arXiv preprint arXiv:2403.12281, 2024

    Yongchao Kim, Dongyun Lee, and Chanhee Lee. ReflAct: Reflection- Augmented Action Learning for Embodied Agents.arXiv preprint arXiv:2403.12281, 2024

    2024 arXiv

  23. [31]

    A3T: Alignment with Adversarial Augmentation and Test-Time Training

    Yuqing Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. A3T: Alignment with Adversarial Augmentation and Test-Time Training. InConference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  24. [32]

    Sutton, Doina Precup, and Satinder Singh

    Richard S. Sutton, Doina Precup, and Satinder Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning.Artificial Intelligence, 112(1-2):181–211, 1999. A Experimental Details A.1 ALFWorld Environment Configuration We use ALFWorld versio...

    1999

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.