Pith. sign in

REVIEW 3 major objections 4 minor 66 references

MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that on-policy distillation for compact-memory agents is valid only when the teacher scores each sampled action under the exact invocation state that produced it, and that MemOPD's reconstruction-and-packing framework…

desk verdict A genuine state-alignment fix for on-policy distillation in memory-rewriting agents; the RCE audit is strong, but the 7% matched-control gain needs a named teacher and a significance test. read the letter →

arxiv 2608.07068 v1 pith:W2SS5QBK submitted 2026-08-07 cs.AI

Zhiyuan Liu , Tinghong Ye , Chenghao Liu , Yizhuo Li , Songfang Huang This is my paper · ORCID
classification cs.AI
keywords on-policydistillationmemorystatealignmentcompactlong-horizonagentsproximalpolicyoptimizationknowledgecontextrewritingrolloutequivalence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that on-policy distillation for agents with compact memory is only valid if the teacher scores each student action under the exact state in which the action was sampled. Compact memory rewrites context between model calls, so flattening the interaction into one history changes token positions and causal visibility; an action can remain on-policy by provenance but be scored under a state the student never visited. MemOPD records every invocation, separates sampled responses from later context copies, and packs reconstructed calls with a shared task prefix. The paper verifies with rollout context equivalence that packed training reproduces independent invocation logits, and reports F1 gains over PPO that grow with horizon, up to 416.2% on Q16.

What carries the argument

The central mechanism is memory state alignment. A memory state is the complete tokenized input to one model call together with token positions, causal visibility, and the prediction position for each sampled action token; identical decoded text can correspond to different memory states. The reconstruction compiler records exact token IDs from rollout, restores each call's original positions and visibility, duplicates a retained response once as a sampled action and once as later context, and packs reconstructed calls under one shared stable prefix $q$ with attention blocked between private blocks. The action mask $m^{\mathrm{act}}$ marks which physical positions are policy decisions. Rollout context equivalence (RCE) checks that packed full-vocabulary logits match independent invocation logits within a numerical tolerance $\epsilon_{\mathrm{num}}$.

What would settle it

Train the same 3B student with the same PPO and OPD coefficients but with a 1B teacher, or a teacher with a different tokenizer, on Q2 and evaluate Q16; if the state-alignment advantage over persistent-history scoring disappears or reverses, the identified mechanism is not the cause.

Watch

Extended reading notes

Core claim

The central claim is that standard teacher distillation, which assumes the autoregressive prefix used to generate an action remains unchanged when the action is scored, breaks as soon as memory rewriting changes that prefix. MemOPD treats the realized memory state $\sigma_t = (z_t, \rho_t, \mathcal{V}_t, \gamma_t)$ as the object that must be preserved, and reconstructs it for every invocation before any objective is applied. In an audit of native MEM1 trajectories, persistent-history reconstruction changed the teacher's top prediction at 651 sampled action positions and falsely triggered PPO clipping for 13.29% of actions, while reconstructed packing matched independent invocations at the numerical floor. With the same reward and data, teacher guidance on aligned states adds 7.0% F1 over persistent-history teacher scoring in the matched Q2 control, and the full MemOPD-3B system improves F1 over PPO by up to 416.2%.

Load-bearing premise

The whole gain rests on the assumption that the frozen 7B teacher's dense full-vocabulary supervision on the aligned 3B student states transfers beyond the Q2 training horizon to Q8/Q16 and Wiki-RAG, and this is tested on two benchmarks with one teacher, so a different teacher or tokenizer could weaken or reverse the advantage.

Editorial extensions

If this is right

  • Any agent that rewrites context between invocations must reconstruct the exact invocation state before applying teacher, value, or reference objectives; provenance alone does not make a rollout batch valid.
  • The same packing strategy can be reused across memory topologies: RCE holds for full-response retention, suffix retention, summary replacement, sliding windows, retrieval refresh, and native MEM1, so the interface is not tied to one memory format.
  • Because the teacher supervises every sampled response token with full-vocabulary reverse KL while PPO retains the final task reward, a student can receive dense local guidance without losing the global task objective.
  • Measured efficiency: packing shared computation yields up to a 1.63x speedup in actor computation during training, with no change in the optimized states.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The state-alignment principle should apply to any auxiliary objective evaluated at action positions, not only teacher distillation; the paper's audit metrics give a cheap diagnostic for detecting misaligned training states in other memory-agent pipelines.
  • The reported reductions in dependency and inference time on Wiki-RAG suggest that state-aligned teacher guidance may push the policy toward more economical memory formats, though the paper reports these efficiency gains without analyzing the mechanism.
  • A natural extension is to apply MemOPD to agents whose context update is noisy or non-deterministic, or where the environment itself rewrites observations; the paper tests deterministic memory updaters, and whether RCE still holds under stochastic rewriting remains an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper argues that on-policy distillation for agents with compact memory is valid only if the teacher scores each sampled action under the exact model-invocation state (tokens, positions, causal visibility, prediction position) in which the action was generated, not under a flattened persistent transcript. MemOPD records each invocation, separates the sampled occurrence of a response from later context copies, reconstructs positions and visibility, and packs the invocations under a shared stable prefix for batched training. The training objective is PPO on the sampled-action mask plus a full-vocabulary reverse-KL teacher term. The paper's main evidence is (i) a rollout-context-equivalence (RCE) audit showing packed logits match independent-invocation logits at p99 |Δlog p| ≈ 3.4e-5, whereas persistent history changes 651 top predictions and falsely clips 13.29% of ratios; (ii) a matched Q2 control where persistent-teacher OPD improves F1 over PPO by 5.6% and MemOPD adds 7.0%; (iii) end-to-end gains on Q2/Q8/Q16 (F1 up to 416.2% relative to PPO) and Wiki-RAG (F1 +7.4%), plus up to 1.63× actor speedup from packing.

Significance. If the empirical claims hold, the paper makes a useful conceptual contribution: it distinguishes action provenance from training-state validity and provides a concrete compiler-style reconstruction plus a verification criterion (RCE) that is checked against independent execution. The matched Q2 control and the RCE audit are well designed and give the paper internal credibility. The method is also practically relevant because it makes dense teacher supervision usable under memory rewriting while preserving task-level PPO, and the public code and machine-auditable RCE are strengths. The main uncertainty is not the mechanism but the breadth of the empirical support: the alignment advantage is demonstrated with a single unnamed teacher and without uncertainty quantification, and the transfer to longer horizons rests on the same Q2-trained teacher signal.

major comments (3)
  1. [Section 4.1 and Section 3.4, Eq. (7)] The teacher is only described as "the frozen 7B teacher" in the models and training paragraph, but its architecture and tokenizer are not given, although Eq. (7) requires the teacher and student to share vocabulary V. If the unnamed 7B teacher is not from the Qwen2.5 family, the shared-vocabulary condition is nontrivial and may be violated, which would make the reverse-KL term ill-defined or inconsistent across positions. Please name the teacher, confirm the tokenizer and vocabulary match, and ideally report a sensitivity check with at least one different teacher family or size; this is load-bearing because the portability of the alignment benefit is the main empirical claim.
  2. [Section 4.5, Figure 3a] The matched-control claim that state alignment "improves F1 by 7.0%" is reported as a point improvement with no standard errors, per-seed values, or paired significance test. Since this 7.0% is the cleanest evidence that the alignment mechanism, rather than teacher guidance in general, drives the gains, the paper should report the five-seed distribution and a paired test, or at least confidence intervals; without this, the improvement could be within seed noise. The same applies to the 5.6% persistent-teacher gain over PPO in the same figure.
  3. [Sections 4.2 and 4.3] The transfer claims for Q8/Q16 and Wiki-RAG are presented as evidence that the alignment benefit generalizes, but the teacher is selected on Q2 only and the absolute PPO baseline at Q8/Q16 is near floor (F1 0.655 and 0.549), making the 283.2% and 416.2% relative gains inflated and not directly informative about state alignment. Please provide at least one longer-horizon matched control or a per-horizon teacher-state audit, and report absolute improvements or effect sizes rather than only relative percentages; without such evidence, the central empirical payoff of alignment at longer horizons remains under-supported.
minor comments (4)
  1. [Section 4.1] Report the hyperparameters α, β, λ, entropy coefficient, GAE parameters, and PPO clipping ϵ; the text mentions λ = 0.02 but gives no values for the other coefficients in Eqs. (8) and (11).
  2. [Section 4.4 and Table 3] The persistent-history row reports p99 |Δlog p| = 1.7738 while the introduction states 1.774; please align the precision or explain the difference.
  3. [Figure 3a] Add error bars or per-seed scatter points to the bar chart; the displayed values (0.660, 0.689, 0.758, 0.813, 0.859, 0.919) have no indication of variance, which is especially important for the 7.0% alignment comparison.
  4. [Table 1 and Table 2 captions] The caption text "Peak context is in 102 tokens" should read "10^2 tokens"; similarly, Table 2's "105" should be "10^5" to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the teacher is an external frozen model, RCE is checked against independent invocation logits, and the matched control isolates state alignment.

full rationale

The paper's derivation chain is self-contained. The teacher is a frozen external 7B model; the task reward remains the PPO objective; and the OPD term is a standard reverse-KL loss applied after state reconstruction. The central mechanism claim — that flattening context rewriting misaligns the scoring state — is tested directly in the RCE audit (Section 4.4), which compares packed logits against independently executed invocation logits (an external reference), not against the teacher's own outputs or the training loss. The matched-control experiment in Section 4.5 compares persistent-history teacher scoring with reconstructed-state scoring under the same teacher and the same λ=0.02, so the reported 7.0% F1 difference isolates the state-alignment intervention rather than a fitted parameter. MEM1 is cited only as an external benchmark and memory protocol; the authors of MEM1 are disjoint from the present authors, so no load-bearing self-citation is present. The only mild self-referential element is that λ is selected on the Q2 development set, but this is standard hyperparameter selection: the transfer claims are evaluated on Q8/Q16 and Wiki-RAG, and both arms of the matched control use the same coefficient. None of the paper's predictions reduce by construction to its inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests primarily on the assumed utility of the unnamed 7B teacher and on the ability to reconstruct exact invocation states under the deterministic MEM1 update rule. No new physical or conceptual entities are introduced beyond the training representation itself. The free parameters are standard RLHF hyperparameters, with λ being the only one selected on a development set.

free parameters (2)
  • teacher guidance weight λ = 0.02
    Selected on the Q2 development evaluation (Section 4.1). It controls the balance between PPO and the reverse-KL teacher loss in Eq. 11, so the headline results depend on this choice.
  • reference KL penalty β = not stated in the paper
    Controls reference regularization in Eq. 8. It is a hand-chosen RLHF hyperparameter that shapes token-level rewards and influences the results, but the paper does not report its value.
assumptions (4)
  • domain assumption The teacher and student share a tokenizer and vocabulary V (Section 3.4).
    Full-vocabulary reverse KL in Eq. 7 requires matching vocabularies. The teacher is only described as a 'frozen 7B teacher', not named, so this shared-tokenizer condition is assumed but not verified in the text.
  • domain assumption The compact memory update rule U is deterministic and known at training time (Section 3.2).
    Reconstruction of exact invocation states requires replaying the same context updates that occurred during rollout. The paper's local Wikipedia environment is deterministic, but the method may not extend to stochastic or live environments without modification.
  • domain assumption SFT data generated by gpt-oss-120b and filtered for correct final answers provides a valid initialization for PPO and MemOPD (Section 4.1).
    The quality and distribution of this filtered data is not independently audited; the paper reports only that 20,036 turn-level examples were retained. If the filtering biases the student toward a narrow behavior distribution, the PPO baseline and the reported gains could both be affected.
  • standard math Standard PPO and GAE equations (Eqs. 9-10) hold over the reconstructed action domain.
    The paper uses the standard clipped PPO objective and GAE recursion, but with positions advanced over sampled actions only. This adaptation is argued in the text but not proven; it is a reasonable extension of standard RL math.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents." pith.science (2026). https://pith.science/paper/W2SS5QBK

@misc{pith2026260807068,
  author       = {Pith},
  title        = {Pith review of: MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W2SS5QBK}},
  note         = {Machine review of arXiv:2608.07068}
}
read the original abstract

Long-horizon agents accumulate growing contexts during interaction, impairing performance and stability. Compact memory mitigates this problem by compressing and rewriting the history retained between model invocations. Learning what to retain typically relies on proximal policy optimization (PPO) with final task rewards, but sparse rewards provide little guidance for individual memory updates. This limitation motivates on-policy distillation (OPD), which supplies dense teacher supervision on student rollouts. For such supervision to be valid, the teacher must evaluate each sampled action under the same state in which it was generated. However, the context rewriting performed during memory compression can break this alignment. When sampled responses are retained and re-encoded for later invocations, flattening the interaction into a persistent history may cause the teacher to score the action under a state that the student never visited during rollout. The action therefore remains on-policy by provenance, but not necessarily by state. We therefore propose Memory-Aligned On-Policy Distillation (MemOPD). MemOPD records the inputs and sampled outputs of each model invocation, restores its original token positions and causal visibility, and packs the reconstructed invocations for efficient teacher scoring. The teacher provides full-vocabulary supervision at the sampled action positions, while PPO preserves the final task objective. Experiments verify state alignment across several context updates and show that it improves F1 by 7.0% over persistent-history teacher scoring in a matched control. Overall, MemOPD-3B improves F1 over PPO by up to 416.2%, while packing yields up to a 1.63x speedup in actor computation during training. The code for this work is publicly available at: https://github.com/TPssp/MemOPD.

Figures

Figures reproduced from arXiv: 2608.07068 by the authors.

Figure 1
Figure 1. Motivation for MemOPD. (A) During compact memory interaction, the behavior policy retains selected content from [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of MemOPD. Here, a call is a model invocation. (1) MemOPD reconstructs each call, separates sampled actions from later context occurrences, and packs the aligned computations under a shared task prefix. (2) RCE compares packed and independent full vocabulary logits within numerical tolerance. (3) The aligned batch supports PPO over the sampled action domain and full vocabulary teacher guidance over the teac… view at source ↗
Figure 3
Figure 3. Experiment diagnostics. (a) Q2 results for PPO, persistent-history OPD, and MemOPD. (b) Log probability error, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 32 canonical work pages

  1. [1]

    2024 , journal =

    A Survey on Large Language Model based Autonomous Agents , author =. 2024 , journal =. doi:10.1007/s11704-024-40231-1 , url =

    doi:10.1007/s11704-024-40231-1 2024

  2. [2]

    2023 , booktitle =

    ReAct: Synergizing Reasoning and Acting in Language Models , author =. 2023 , booktitle =

    work page 2023

  3. [3]

    2023 , booktitle =

    Reflexion: Language agents with verbal reinforcement learning , author =. 2023 , booktitle =

    work page 2023

  4. [4]

    2022 , booktitle =

    WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents , author =. 2022 , booktitle =

    work page 2022

  5. [5]

    2023 , booktitle =

    Mind2web: Towards a generalist agent for the web , author =. 2023 , booktitle =

    work page 2023

  6. [6]

    2024 , booktitle =

    SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents , author =. 2024 , booktitle =

    work page 2024

  7. [7]

    2024 , booktitle =

    A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis , author =. 2024 , booktitle =

    work page 2024

  8. [8]

    2024 , booktitle =

    Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents , author =. 2024 , booktitle =

    work page 2024

Show all 66 references
  1. [9]

    2024 , booktitle =

    Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning , author =. 2024 , booktitle =

    2024

  2. [10]

    2022 , eprint =

    WebGPT: Browser-assisted question-answering with human feedback , author =. 2022 , eprint =

    2022

  3. [11]

    2025 , eprint =

    Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning , author =. 2025 , eprint =

    2025

  4. [12]

    2025 , eprint =

    DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments , author =. 2025 , eprint =

    2025

  5. [13]

    2025 , doi =

    ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning , author =. 2025 , doi =. 2503.19470 , archiveprefix =

    2025 arXiv

  6. [14]

    2017 , booktitle =

    Attention Is All You Need , author =. 2017 , booktitle =

    2017

  7. [15]

    2020 , eprint =

    Longformer: The Long-Document Transformer , author =. 2020 , eprint =

    2020

  8. [16]

    2024 , journal =

    Lost in the Middle: How Language Models Use Long Contexts , author =. 2024 , journal =. doi:10.1162/tacl_a_00638 , url =

    2024 doi

  9. [17]

    2023 , booktitle =

    Measuring and Narrowing the Compositionality Gap in Language Models , author =. 2023 , booktitle =

    2023

  10. [18]

    2025 , booktitle =

    Infinite Retrieval: Attention Enhanced LLMs in Long-Context Processing , author =. 2025 , booktitle =

    2025

  11. [19]

    2025 , booktitle =

    LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory , author =. 2025 , booktitle =

    2025

  12. [20]

    2025 , booktitle =

    Why Does the Effective Context Length of LLMs Fall Short? , author =. 2025 , booktitle =

    2025

  13. [21]

    2020 , booktitle =

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , author =. 2020 , booktitle =

    2020

  14. [22]

    2020 , booktitle =

    Dense Passage Retrieval for Open-Domain Question Answering , author =. 2020 , booktitle =

    2020

  15. [23]

    2022 , booktitle =

    Improving Language Models by Retrieving from Trillions of Tokens , author =. 2022 , booktitle =

    2022

  16. [24]

    2023 , doi =

    MemoryBank: Enhancing Large Language Models with Long-Term Memory , author =. 2023 , doi =. 2305.10250 , archiveprefix =

    2023 arXiv

  17. [25]

    2024 , doi =

    MemGPT: Towards LLMs as Operating Systems , author =. 2024 , doi =. 2310.08560 , archiveprefix =

    2024 arXiv

  18. [26]

    2025 , doi =

    Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory , author =. 2025 , doi =. 2504.19413 , archiveprefix =

    2025 arXiv

  19. [27]

    2025 , booktitle =

    A-MEM: Agentic Memory for LLM Agents , author =. 2025 , booktitle =. 2502.12110 , archiveprefix =

    2025 arXiv

  20. [28]

    2024 , booktitle =

    CompAct: Compressing Retrieved Documents Actively for Question Answering , author =. 2024 , booktitle =

    2024

  21. [29]

    2023 , booktitle =

    Compressing Context to Enhance Inference Efficiency of Large Language Models , author =. 2023 , booktitle =

    2023

  22. [30]

    2024 , booktitle =

    Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs , author =. 2024 , booktitle =. doi:10.18653/v1/2024.acl-long.685 , url =

    2024 doi

  23. [31]

    2025 , doi =

    MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents , author =. 2025 , doi =. 2506.15841 , archiveprefix =

    2025 arXiv

  24. [32]

    2015 , eprint =

    Distilling the Knowledge in a Neural Network , author =. 2015 , eprint =

    2015

  25. [33]

    2024 , booktitle =

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author =. 2024 , booktitle =. 2306.13649 , archiveprefix =

    2024 arXiv

  26. [34]

    2016 , booktitle =

    High-Dimensional Continuous Control Using Generalized Advantage Estimation , author =. 2016 , booktitle =. 1506.02438 , archiveprefix =

    2016 arXiv

  27. [35]

    2017 , eprint =

    Proximal Policy Optimization Algorithms , author =. 2017 , eprint =

    2017

  28. [36]

    2018 , booktitle =

    HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , author =. 2018 , booktitle =

    2018

  29. [37]

    2019 , journal =

    Natural Questions: A Benchmark for Question Answering Research , author =. 2019 , journal =. doi:10.1162/tacl_a_00276 , url =

    2019 doi

  30. [38]

    2025 , eprint =

    Qwen2.5 Technical Report , author =. 2025 , eprint =

    2025

  31. [39]

    2024 , booktitle =

    SGLang: Efficient Execution of Structured Language Model Programs , author =. 2024 , booktitle =

    2024

  32. [40]

    2023 , booktitle =

    Efficient Memory Management for Large Language Model Serving with PagedAttention , author =. 2023 , booktitle =

    2023

  33. [41]

    2024 , url =

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author =. 2024 , url =

    2024

  34. [42]

    2024 , booktitle =

    RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation , author =. 2024 , booktitle =. doi:10.18653/v1/2024.emnlp-main.1236 , url =

    2024 doi

  35. [43]

    2023 , eprint =

    Retrieval-Augmented Generation for Large Language Models: A Survey , author =. 2023 , eprint =

    2023

  36. [44]

    2024 , booktitle =

    Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models , author =. 2024 , booktitle =

    2024

  37. [45]

    2024 , doi =

    Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control , author =. 2024 , doi =. 2306.07863 , archiveprefix =

    2024 arXiv

  38. [46]

    2025 , doi =

    Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window , author =. 2025 , doi =. 2510.08276 , archiveprefix =

    2025

  39. [47]

    2023 , doi =

    GAIA: a benchmark for General AI Assistants , author =. 2023 , doi =. 2311.12983 , archiveprefix =

    2023 arXiv

  40. [48]

    2022 , doi =

    MuSiQue: Multihop Questions via Single-hop Question Composition , author =. 2022 , doi =. 2108.00573 , archiveprefix =

    2022 arXiv

  41. [49]

    2020 , doi =

    Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps , author =. 2020 , doi =. 2011.01060 , archiveprefix =

    2020 arXiv

  42. [50]

    2017 , doi =

    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension , author =. 2017 , doi =. 1705.03551 , archiveprefix =

    2017 arXiv

  43. [51]

    2603.00680 , archiveprefix =

    Li, Ruoran and Zhang, Xinghua and Yu, Haiyang and Duan, Shitong and Li, Xiang and Xiang, Wenxin and Liao, Chonghua and Guo, Xudong and Li, Yongbin and Suo, Jinli , year =. 2603.00680 , archiveprefix =

    arXiv

  44. [52]

    Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages =

    Sequence-Level Knowledge Distillation , author =. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages =. 2016 , doi =

    2016

  45. [53]

    2024 , eprint =

    Gu, Yuxian and Dong, Li and Wei, Furu and Huang, Minlie , booktitle =. 2024 , eprint =

    2024

  46. [54]

    Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , series =

    A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , author =. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , series =. 2011 , publisher =

    2011

  47. [55]

    2024 , volume =

    Ko, Jongwoo and Kim, Sungnyun and Chen, Tianyi and Yun, Se-Young , booktitle =. 2024 , volume =. 2402.03898 , archiveprefix =

    2024 arXiv

  48. [56]

    International Journal of Computer Vision , volume =

    Knowledge Distillation: A Survey , author =. International Journal of Computer Vision , volume =. 2021 , doi =. 2006.05525 , archiveprefix =

    2021 arXiv

  49. [57]

    Advances in Neural Information Processing Systems , year =

    Training Language Models to Follow Instructions with Human Feedback , author =. Advances in Neural Information Processing Systems , year =. 2203.02155 , archiveprefix =

    arXiv

  50. [58]

    Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology , year =

    Generative Agents: Interactive Simulacra of Human Behavior , author =. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology , year =. doi:10.1145/3586183.3606763 , eprint =

  51. [59]

    The Twelfth International Conference on Learning Representations , year =

    A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts , author =. The Twelfth International Conference on Learning Representations , year =. 2402.09727 , archiveprefix =

    arXiv

  52. [60]

    2023 , doi =

    Jiang, Huiqiang and Wu, Qianhui and Lin, Chin-Yew and Yang, Yuqing and Qiu, Lili , booktitle =. 2023 , doi =. 2310.05736 , archiveprefix =

    2023 arXiv

  53. [61]

    2024 , doi =

    Jiang, Huiqiang and Wu, Qianhui and Luo, Xufang and Li, Dongsheng and Lin, Chin-Yew and Yang, Yuqing and Qiu, Lili , booktitle =. 2024 , doi =. 2310.06839 , archiveprefix =

    2024 arXiv

  54. [62]

    2025 , eprint =

    Scaling Long-Horizon LLM Agent via Context-Folding , author =. 2025 , eprint =

    2025

  55. [63]

    2026 , eprint =

    Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents , author =. 2026 , eprint =

    2026

  56. [64]

    2026 , eprint =

    InfMem: Learning System-2 Memory Control for Long-Context Agent , author =. 2026 , eprint =

    2026

  57. [65]

    2026 , eprint =

    Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning , author =. 2026 , eprint =

    2026

  58. [66]

    2025 , eprint=

    gpt-oss-120b & gpt-oss-20b Model Card , author=. 2025 , eprint=

    2025

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.