Pith. sign in

REVIEW 2 major objections 5 minor 42 references

Training agents on their own DO/AVOID reflections beats reward-only RL and skill-distillation baselines across three agentic benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 03:15 UTC pith:LCTZHSOZ

load-bearing objection GRSD is a genuinely new, well-specified way to densify agentic RL rewards with the policy's own group-contrasted reflections, but the paper overstates the 'turn-level credit assignment' story: the key signal is, by the authors' own appendix, a hindsight-conditioned compatibility measure, and the empirical case is single-run. the 2 major comments →

arxiv 2607.28076 v2 pith:LCTZHSOZ submitted 2026-07-30 cs.AI cs.LG

Group-Reflective Self-Distillation for Agentic Reinforcement Learning

Binbin Zheng , Zijun Xie , Guanqun Zhao , Enlei Gong , Xing Ma , Xiaoliang Fu , Zeyu Chen This is my paper
classification cs.AI cs.LG
keywords agentic reinforcement learningRLVRself-distillationcredit assignmentGRPOnatural-language guidanceLLM agentsturn-level advantages
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that sparse terminal rewards in multi-turn agent training can be refined into useful turn-level credit without calling in a stronger model or an external skill library. The proposed loop is: the policy itself reflects on each of its verified rollouts, a frozen copy of the same policy contrasts the reflections from successes and failures into a compact DO/AVOID guidance, and this guidance modulates the outcome-based advantage per turn while never flipping its sign. The authors argue this keeps guidance capability-aligned and outcome-discriminative, and their experiments across embodied reasoning, search-based QA, and web shopping show consistent aggregate gains over GRPO and skill-based self-distillation, plus better performance on held-out tasks. A sympathetic reader would care because it suggests an agent can bootstrap its own training signal from verified experience, with only a modest training-time cost and zero inference overhead.

Core claim

GRSD's central claim is that the policy's own, outcome-verified reflections, contrasted across successful and failed rollouts in the same prompt group, produce privileged guidance z_x that is better aligned with the policy's capability than externally retrieved or stronger-model-generated skills. Conditioned on this guidance, a stop-gradient self-teacher re-weights the GRPO advantage for each turn by a clipped factor (1+λ q̂_i,t) that is sign-preserving; Proposition 3 shows the resulting surrogate is exactly a detached, positively reweighted clipped GRPO surrogate. Empirically, this yields the strongest aggregate results across three environments and three model scales, with particularly lar

What carries the argument

The central object is the group-level privileged guidance z_x, built in two detached stages: Stage A samples a policy-native reflection s_i from each verified trajectory under a reflection instruction; Stage B has a stop-gradient snapshot of the same policy contrast the successful and failed reflection sets into a compact DO/AVOID guide. The load-bearing identity is the turn-level modulation: Δ_i,t measures how the frozen policy's log-likelihood of the sampled action changes when z_x is prepended to the context, and after exponential reweighting, normalization, and clipping it multiplies the trajectory advantage A_i without changing its sign. Proposition 3 proves that this construction is ex

Load-bearing premise

The method assumes the frozen policy's log-likelihood difference under guidance-augmented vs plain context (Δ_i,t) is a meaningful proxy for how much that turn contributed to the outcome; the paper concedes it is hindsight-conditioned, not causal, so if the difference mostly reflects text-completion behavior rather than credit, the modulation adds noise and the reported gains would not transfer.

What would settle it

Run GRSD with Δ_i,t replaced by random values drawn from the same sign and magnitude distribution as the real teacher–student gaps while keeping z_x fixed; if the resulting policy matches GRSD's performance, the likelihood-difference signal is not what drives the gains. A complementary check is to compare Δ_i,t with per-turn outcome effects obtained by counterfactual action replacement in a simulator.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If GRSD is right, sparse-reward agent training can be improved with no external skill sources: the policy's own verified rollouts supply the guidance.
  • Turn-level credit assignment can be performed as a bounded, sign-preserving reweighting of the verifier's outcome advantage, so the method inherits the stability of GRPO rather than replacing it.
  • The learned reflection ability in Stage A feeds back into better task behavior over training, implying auxiliary reflection objectives can improve downstream task performance rather than merely producing explanations.
  • Because guidance is used only during training, deployment cost is unchanged; the only price is a reported 17% training-time overhead.
  • The method generalizes to unseen tasks in the ALFWorld split, suggesting contrastive policy-native guidance captures transferable decision principles rather than memorized paths.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the contrastive guidance construction is not tied to GRPO; any outcome-based advantage (e.g., PPO-style advantage with a value baseline) could be modulated the same way, provided the modulation stays sign-preserving and detached.
  • Editorial inference: if Δ_i,t is mostly measuring the policy's compliance with its own injected text rather than causal turn contribution, the method's gains should shrink when guidance is semantically vacuous but stylistically similar; that is testable by ablating guidance content.
  • Editorial inference: the judge's scalar reflection reward is a second channel of supervision; one could test whether the reflection objective alone (α > 0 with a frozen judge) carries most of the benefit, independent of the turn-level modulation.
  • Editorial inference: the reported gains are aggregate across mixed-outcome groups; in settings where almost all groups are all-success or all-failure (e.g., very easy or very hard tasks), GRSD falls back to plain GRPO and should show no benefit, which is a clean boundary condition.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes Group-Reflective Self-Distillation (GRSD), a training-time method for multi-turn LLM agents. For each prompt, the current policy rolls out a group of trajectories, reflects on each verified trajectory (Stage A), and a stop-gradient snapshot contrasts reflections from successful and failed rollouts into a compact DO/AVOID guidance z_x (Stage B). This guidance is used as privileged context for a self-teacher: the frozen policy evaluates the same sampled action tokens under plain and guidance-augmented contexts, forming a teacher–student gap Δ_i,t. After sign alignment, normalization, and clipping, this gap becomes a turn-level modulator that rescales the trajectory-level GRPO advantage A_i into bA_i,t = A_i(1 + λ bq_i,t m_x), with the sign of A_i preserved. A separate reflection loss trains the reflection tokens. The theory shows a bounded positive reweighting of clipped GRPO (Props. 2–3) and an idealized variance-reduction bound for group contrast (Prop. 1). Experiments cover ALFWorld, Search-based QA, and WebShop across Qwen3-1.7B, Qwen2.5-3B, and Qwen2.5-7B, with ablations, training dynamics, hyperparameter sensitivity, a generalization split, and a cost table.

Significance. GRSD is a clean, training-only method: it requires no external skill library, adds no inference-time guidance, and is supported by a useful algebraic decomposition: the task surrogate is exactly a detached, positively reweighted clipped GRPO surrogate (Prop. 3). The empirical scope is broad (three environments × three backbones), and the paper includes ablations, hyperparameter sensitivity, pseudocode, full prompt templates, and a GitHub link. If the headline gains are reproducible, the method is a solid contribution to agentic RL self-distillation. However, the central mechanism—using Δ_i,t as turn-level credit—is explicitly conceded in Appendix A.5 to be a hindsight-conditioned compatibility signal, and no experiment isolates the information content of that signal. In addition, all results are single-run, with no error bars or significance tests, while some aggregate margins are very small. These two issues are load-bearing for the paper's abstract claims and require additional work before acceptance.

major comments (2)
  1. [§3.3 (Eqs. 9–11); Appendix A.5; Table 2] The credit-assignment premise is the load-bearing element of the method, and the manuscript itself concedes its weakness. Appendix A.5 states that because z_x is built from complete trajectories, Δ_i,t is 'a hindsight-conditioned compatibility signal rather than an unbiased or causal estimate of turn-level credit.' Eq. (11) nevertheless uses Δ_i,t as the sole per-turn credit signal, and the abstract claims GRSD 'refines turn-level credit assignment.' Prop. 3 only proves that any nonnegative detached weight reweights the clipped GRPO surrogate; it says nothing about whether the weights carry credit information. The ablation 'w/o group-reflective guidance' (Table 2) replaces z_x with a standard reference solution, changing both the teacher context and the modulation, so it cannot isolate Δ's contribution. Please add a control that breaks the per-turn alignment while keeping a comparable te
  2. [§4.2, Table 1; also Tables 2, Fig. 6] All main results and ablations are single-run, with no error bars or significance tests. Table 1 contains aggregate margins of 0.1 points (Qwen3-1.7B Search-QA: GRSD 43.8 vs GRPO 43.7 and RLSD 43.7), and in the same row GRSD is below GRPO on NQ, PopQA, HotpotQA, and MuSiQue. The abstract's 'consistently outperforms' is not established at this magnitude without variance estimates. Please report at least 3 seeds (or paired bootstrap intervals) for the headline comparisons and propagate the uncertainty to the ablations (Table 2) and the ALFWorld Unseen split (Fig. 6).
minor comments (5)
  1. [Appendix A.1, Prop. 1] Prop. 1 concerns an idealized contrast statistic bd_x that the actual method never computes, and the assumptions (conditionally uncorrelated reflection noise) are acknowledged as idealized. As written, the 'Theoretical Analysis' section may be read as supporting Stage B's outcome discriminability, which it cannot do for the actual natural-language guidance. Please move this to a clearly labeled 'theoretical intuition' subsection or state explicitly in the main text that no formal guarantee applies to the actual z_x.
  2. [§4.3, Fig. 3] A smaller mean teacher–student gap is interpreted as 'more compatible guidance,' but a smaller gap could equally mean the teacher context is less informative. Please justify this interpretation, e.g., by showing that within GRSD, lower-gap runs also have better downstream performance.
  3. [Abstract and §4.2] 'Consistently outperforms' overstates Table 1, where GRSD loses on several individual subsets (e.g., Qwen3-1.7B NQ 41.8 vs GRPO 43.4). Consider softening to 'achieves the best aggregate performance in most settings' unless statistical significance is established.
  4. [Eqs. (10)–(11)] The mixed-outcome mask m_x is defined in Eq. (7) but used later; consider moving the definition immediately before Eq. (10) to reduce reader friction.
  5. [Appendix E, Table 6] The comparison of teacher-forward time (12.0s vs 25.1s) is informative, but the conclusion that a shorter teacher pass partially offsets online guidance construction is descriptive; GRSD still has the highest total per-update time (1.17× GRPO). Please keep the cost discussion neutral.

Circularity Check

0 steps flagged

No significant circularity: the central empirical claim is externally benchmarked, and Prop. 3 is an algebraic identity about the method's own construction, not a disguised prediction. Only minor non-load-bearing self-citations appear.

full rationale

The paper's load-bearing claim is empirical: GRSD 'consistently outperforms competitive baselines and generalizes more effectively to unseen tasks' (Abstract), supported by Table 1 against GRPO, Skill-SD, SDAR, and other baselines under shared backbones, wrappers, rollout budgets, and schedules. No fitted constant is used to manufacture the result; hyperparameters such as lambda=0.5 and alpha=0.01 are chosen by sensitivity analysis, not fitted to the reported test sets. Proposition 3 (Eq. 38-40) shows that the GRSD surrogate is 'exactly a detached, positively reweighted clipped GRPO surrogate.' This is a formal identity about the method's own construction, and the paper does not use this identity as evidence of performance improvement. The turn-level signal Delta_i,t (Eq. 9) is the method's own modulation input, and Appendix A.5 explicitly concedes that 'Delta_i,t is a hindsight-conditioned compatibility signal rather than an unbiased or causal estimate of turn-level credit.' This is an admitted limitation on the credit-assignment interpretation, but it is not a case where a predicted quantity reduces by construction to a fitted input. Several self-citations (e.g., Xie et al. 2026, Zhao et al. 2026a) appear in motivation or related work, but they are not load-bearing: no uniqueness theorem or prior result is invoked to force the architecture. Proposition 1 is an idealized variance-reduction statement with its own stated assumptions and does not smuggle in the conclusion. Overall, the derivation chain is self-contained against external benchmarks; the minor self-citations do not carry the argument.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

The method's added explanatory machinery is mostly internal: the free hyperparameters λ, b_G, and α are chosen by hand/ablation rather than derived; the key unproved premise is that the same-policy teacher/student likelihood gap is a meaningful credit signal; and the new entities are training-time text constructs with no external falsifiable handle.

free parameters (4)
  • λ (self-distillation strength) = 0.5
    Controls the magnitude of turn-level modulation in Eq. (11); selected by ablation on ALFWorld Qwen3-1.7B (Fig. 5b), not derived.
  • b_G (clip bound) = 0.2
    Bounding modulation in Eq. (10) so λ*b_G < 1 preserves advantage signs; fixed design hyperparameter without an ablation.
  • α (reflection-loss weight) = 0.01
    Balances L_ref against L_task in Eq. (12); chosen by ablation on ALFWorld (Fig. 5a).
  • ϵ_adv / ϵ_gap stabilizers = not reported
    Small constants in Eq. (1) and Eq. (10); values are not given and are needed for numerical stability.
axioms (5)
  • domain assumption Verifier provides correct reward R(τ) and binary outcome label b_i
    Used in §3.1 to define group advantages and in Eq. (5) to partition successful and failed reflections; if the verifier is wrong, the guidance contrast is meaningless.
  • domain assumption External judge J's {0,1,2,3} score is a valid training signal for reflection quality
    Eq. (4) and §3.4 use judge scores as A_ref to optimize reflection generation; the judge is a fixed proprietary LLM, not independently validated.
  • domain assumption Teacher–student likelihood gap Δ_i,t reflects action-level credit rather than spurious context effects
    §3.3 Eq. (9)-(11) uses Δ to modulate turn advantages; Appendix A.5 concedes Δ is hindsight-conditioned and not an unbiased or causal credit estimate.
  • ad hoc to paper Idealized reflection representation h(s_i)=μ^b_x+ξ_i with conditionally uncorrelated noise
    Assumed only for Prop. 1 variance-reduction claim; the paper states it is an idealized model and does not guarantee natural-language guidance recovers the contrast.
  • standard math GRPO/PPO clipped surrogate and KL penalty are valid optimization primitives
    Borrowed from Shao et al. 2024 and used throughout L_task and L_ref.
invented entities (2)
  • Group-reflective privileged guidance z_x no independent evidence
    purpose: Privileged teacher context in Eq. (8) and source of Δ_i,t; constructed by a stop-gradient snapshot from success/failure reflections.
    Internal training signal; the only support is in-paper ablations (§4.4) and a worked example (Appendix F); no falsifiable external handle.
  • Policy-native reflection skills s_i no independent evidence
    purpose: Trajectory-level skills sampled by the policy in Stage A; basis for the success/failure contrast.
    Internal construct; judged by the same external judge but not verified outside this paper.

pith-pipeline@v1.3.0-alltime-deepseek · 24405 in / 13912 out tokens · 365891 ms · 2026-08-04T03:15:48.229201+00:00 · methodology

0 comments
read the original abstract

Reinforcement learning with verifiable rewards (RLVR) is effective for training large language model agents. However, terminal rewards provide only coarse trajectory-level supervision, leaving successful behaviors, recurring mistakes, and incidental choices entangled in the same outcome signal. Existing agentic self-distillation methods enrich sparse supervision with natural-language skills, but skills retrieved externally or extracted from a single trajectory by stronger models may mismatch current experience, exceed the policy's capability, or remain path-specific. We propose Group-Reflective Self-Distillation (GRSD), which derives capability-aligned and outcome-discriminative guidance from the policy's own verified rollouts. For each prompt, the policy reflects on each verified trajectory in an on-policy group, and a stop-gradient snapshot contrasts the resulting reflections from successful and failed rollouts to construct group-level privileged guidance. Conditioned on this guidance, a self-teacher refines turn-level credit assignment by modulating outcome-based advantages while preserving the verifier-determined learning direction. Experiments across multiple agentic environments and model scales demonstrate that GRSD consistently outperforms competitive baselines and generalizes more effectively to unseen tasks.

Figures

Figures reproduced from arXiv: 2607.28076 by Binbin Zheng, Enlei Gong, Guanqun Zhao, Xiaoliang Fu, Xing Ma, Zeyu Chen, Zijun Xie.

Figure 1
Figure 1. Figure 1: Performance overview of GRSD across multiple [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of GRSD. Stage A learns policy-native reflection skills from verified trajectories. Stage B contrasts skills [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training dynamics on ALFWorld using Qwen3-1.7B. (a) Validation success rate over training steps. (b) Mean teacher– [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of the complete policy-native reflec [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Hyperparameter sensitivity analysis on ALFWorld [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Training dynamics on ALFWorld. (a) Mean judge [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 7
Figure 7. Figure 7: Validation success over 240 training steps with Qwen2.5-3B-Instruct across three environments. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Validation success over 240 training steps with Qwen2.5-3B-Instruct across three environments. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Validation success over 240 training steps with Qwen2.5-7B-Instruct across three environments. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 23 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2601.18734 , year=

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. arXiv preprint arXiv:2601.18734 , year=

    Pith/arXiv arXiv

  2. [2]

    International Conference on Learning Representations , volume=

    On-policy distillation of language models: Learning from self-generated mistakes , author=. International Conference on Learning Representations , volume=

  3. [3]

    International Conference on Learning Representations , volume=

    Minillm: Knowledge distillation of large language models , author=. International Conference on Learning Representations , volume=

  4. [4]

    arXiv preprint arXiv:2604.10688 , year=

    Scope: Signal-calibrated on-policy distillation enhancement with dual-path adaptive weighting , author=. arXiv preprint arXiv:2604.10688 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2511.10643 , year=

    Black-Box On-Policy Distillation of Large Language Models , author=. arXiv preprint arXiv:2511.10643 , year=

    arXiv

  6. [6]

    arXiv preprint arXiv:2605.07711 , year=

    Simct: Recovering lost supervision for cross-tokenizer on-policy distillation , author=. arXiv preprint arXiv:2605.07711 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2603.25562 , year=

    Revisiting on-policy distillation: Empirical failure modes and simple fixes , author=. arXiv preprint arXiv:2603.25562 , year=

    Pith/arXiv arXiv

  8. [8]

    arXiv preprint arXiv:2604.14084 , year=

    Tip: Token importance in on-policy distillation , author=. arXiv preprint arXiv:2604.14084 , year=

    Pith/arXiv arXiv

  9. [9]

    arXiv preprint arXiv:2604.13016 , year=

    Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe , author=. arXiv preprint arXiv:2604.13016 , year=

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2602.22495 , year=

    Reinforcement-aware knowledge distillation for llm reasoning , author=. arXiv preprint arXiv:2602.22495 , year=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2604.03128 , year=

    Self-distilled rlvr , author=. arXiv preprint arXiv:2604.03128 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2605.15155 , year=

    Self-distilled agentic reinforcement learning , author=. arXiv preprint arXiv:2605.15155 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2604.10674 , year=

    Skill-sd: Skill-conditioned self-distillation for multi-turn llm agents , author=. arXiv preprint arXiv:2604.10674 , year=

    Pith/arXiv arXiv

  14. [14]

    arXiv preprint arXiv:2606.26790 , year=

    OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning , author=. arXiv preprint arXiv:2606.26790 , year=

    Pith/arXiv arXiv

  15. [15]

    arXiv preprint arXiv:2210.03629 , year=

    React: Synergizing reasoning and acting in language models , author=. arXiv preprint arXiv:2210.03629 , year=

    Pith/arXiv arXiv

  16. [16]

    Advances in neural information processing systems , volume=

    Toolformer: Language models can teach themselves to use tools , author=. Advances in neural information processing systems , volume=

  17. [17]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2606.31650 , year=

    ECHO: Prune to act, trace to learn with selective turn memory in agentic RL , author=. arXiv preprint arXiv:2606.31650 , year=

    Pith/arXiv arXiv

  19. [19]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Maspo: Unifying gradient utilization, probability mass, and signal reliability for robust and sample-efficient llm reasoning , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  20. [20]

    Fu, Xiaoliang and Lin, Jiaye and Fang, Yangyi and Hu, Chaowen and Qin, Cong and Shao, Zekai and Zheng, Binbin and Pan, Lu and Zeng, Ke , booktitle=. From

  21. [21]

    Advances in Neural Information Processing Systems , volume=

    Dapo: An open-source llm reinforcement learning system at scale , author=. Advances in Neural Information Processing Systems , volume=

  22. [22]

    arXiv preprint arXiv:2506.13585 , year=

    Minimax-m1: Scaling test-time compute efficiently with lightning attention , author=. arXiv preprint arXiv:2506.13585 , year=

    Pith/arXiv arXiv

  23. [23]

    arXiv preprint arXiv:2510.14545 , year=

    Agentic entropy-balanced policy optimization , author=. arXiv preprint arXiv:2510.14545 , year=

    arXiv

  24. [24]

    Advances in Neural Information Processing Systems , volume=

    Group-in-group policy optimization for llm agent training , author=. Advances in Neural Information Processing Systems , volume=

  25. [25]

    arXiv preprint arXiv:2503.09516 , year=

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning , author=. arXiv preprint arXiv:2503.09516 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2603.08754 , year=

    Hindsight credit assignment for long-horizon llm agents , author=. arXiv preprint arXiv:2603.08754 , year=

    arXiv

  27. [27]

    arXiv preprint arXiv:2606.11709 , year=

    RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation , author=. arXiv preprint arXiv:2606.11709 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2601.20802 , year=

    Reinforcement Learning via Self-Distillation , author=. arXiv preprint arXiv:2601.20802 , year=

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2605.06597 , year=

    UniSD: Towards a Unified Self-Distillation Framework for Large Language Models , author=. arXiv preprint arXiv:2605.06597 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2605.12400 , year=

    OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning , author=. arXiv preprint arXiv:2605.12400 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2010.03768 , year=

    Alfworld: Aligning text and embodied environments for interactive learning , author=. arXiv preprint arXiv:2010.03768 , year=

    Pith/arXiv arXiv 2010

  32. [32]

    Advances in Neural Information Processing Systems , volume=

    Webshop: Towards scalable real-world web interaction with grounded language agents , author=. Advances in Neural Information Processing Systems , volume=

  33. [33]

    Transactions of the Association for Computational Linguistics , volume=

    Natural questions: a benchmark for question answering research , author=. Transactions of the Association for Computational Linguistics , volume=. 2019 , publisher=

    2019

  34. [34]

    Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    HotpotQA: A dataset for diverse, explainable multi-hop question answering , author=. Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    2018

  35. [35]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension , author=. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  36. [36]

    Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

    When not to trust language models: Investigating effectiveness of parametric and non-parametric memories , author=. Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

  37. [37]

    Proceedings of the 28th International Conference on Computational Linguistics , pages=

    Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps , author=. Proceedings of the 28th International Conference on Computational Linguistics , pages=

  38. [38]

    Transactions of the Association for Computational Linguistics , volume=

    MuSiQue: Multihop Questions via Single-hop Question Composition , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=

    2022

  39. [39]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Measuring and narrowing the compositionality gap in language models , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  40. [40]

    arXiv preprint arXiv:2605.07725 , year=

    Sod: Step-wise on-policy distillation for small language model agents , author=. arXiv preprint arXiv:2605.07725 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2212.03533 , year=

    Text embeddings by weakly-supervised contrastive pre-training , author=. arXiv preprint arXiv:2212.03533 , year=

    Pith/arXiv arXiv

  42. [42]

    2026 , eprint=

    Deconstructing Off-Policy Ratios: Entropy-Scaled Trust Regions for Asynchronous Reinforcement Learning , author=. 2026 , eprint=

    2026