Pith. sign in

REVIEW 2 major objections 6 minor 54 references

RLAP: A Reinforcement Learning Enhanced Adaptive Planning Framework for Multi-step NLP Task Solving

T0 review · 2 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RLAP claims a lightweight Q-learning Actor can choose each subtask order from linguistic features, improving LLM accuracy without fine-tuning.

desk verdict A solid extension of the authors' RL4IE line with a strong IE ablation, but the MRC results don't isolate learned order from the multi-step protocol, so the central claim is only partly supported. read the letter →

arxiv 2505.11893 v1 pith:Q6NTPAWT submitted 2025-05-17 cs.CL cs.AI

Zepeng Ding , Dixuan Wang , Ziqin Luo , Guochao Jiang , Deqing Yang , Jiaqing Liang This is my paper · ORCID
classification cs.CLcs.AI
keywords reinforcementlearninglargelanguagemodelsmulti-stepplanningadaptiveQ-learningMarkovdecisionprocessmachinereadingcomprehensioninformationextraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes RLAP, a method for making multi-step LLM problem-solving adaptive to each task instance. It models a decomposable NLP task as an MDP, embeds a frozen LLM as the environment, and trains a lightweight Actor model to estimate Q-values for pairs of the current state and each candidate next subtask. At inference, the Actor picks the subtask with the highest Q-value, so the order in which a context is read, slots are extracted, or sentences are concatenated depends on the linguistic features of that instance rather than on a preset order or the LLM's own planning. The paper reports consistent accuracy gains over fixed-order and LLM-search baselines on machine reading comprehension, information extraction, and sentence-level text completion, with larger gains on complex instances. A sympathetic reader would take the central claim to be that order choice is a learnable, task-independent policy that can be decoupled from the LLM.

What carries the argument

The load-bearing object is the Actor model: a pretrained language model plus a linear projection from a sequence representation to a scalar Q-value, written as $\hat{Q}(S_t,a_t)=W h+b$. Its input is the flattened state dictionary concatenated with a candidate action, wrapped in special tokens. It is trained with double deep Q-learning, using a discounted target with a target network, experience replay, and epsilon-greedy exploration. The action space shrinks by one each step because every subtask is executed exactly once; rewards are either stepwise (matching ground truth at each substep) or episode-level (matching the final answer), with the discount factor set accordingly. This machinery converts linguistic features into a quantitative ordering criterion that does not depend on the LLM's planning ability.

What would settle it

Take a held-out set of small contexts (4-6 sentences or slots), enumerate all possible subtask orders, run the frozen LLM on each order, and compare the accuracy of RLAP's chosen order with the best-order accuracy and with fixed and random orders. If RLAP does not approach the best order, or if random or fixed orders already match it, the claim that learned linguistic-feature Q-values drive the gains is refuted.

Watch

Extended reading notes

Core claim

RLAP's central claim is that the optimal ordering of subtasks in a multi-step NLP task is a function of the instance's linguistic features, and that this function can be learned by a lightweight model without modifying the LLM. The paper models the solving process as an MDP in which states are dictionaries of task definition, original text, intermediate results, and requirements; actions are subtasks that can be executed once; and transitions are determined by the LLM's outputs. The Actor model flattens each state-action pair into a natural-language sequence, encodes it with a pretrained encoder, and applies a linear layer to produce a Q-value, trained by double deep Q-learning with epsilon-greedy exploration and stepwise or episode-level rewards. At inference, the LLM simply executes the subtask with the highest Q-value at each step, and the episode ends when the action space is empty. The experiments on MRC, IE, and STC claim that this adaptive ordering outperforms fixed-order prompting, tree-search self-planning, and the prior RL-based planner on the tested datasets.

Load-bearing premise

The framework assumes that a fixed-size embedding of the flattened state-and-action text captures the linguistic features that determine the best next subtask, and that the Q-values learned on training instances transfer to test instances; for MRC it additionally assumes that reordering sentences changes the final answer in a way the reward signal can learn.

Editorial extensions

If this is right

  • If the learned ordering policy transfers, closed-source LLMs can be steered to better task accuracy by a small external planner, avoiding fine-tuning and catastrophic forgetting.
  • A single trained Actor per task type and language can be reused across datasets with the same state and action structure, because training samples are pooled across datasets of the same type.
  • For tasks whose final answer is a concatenation of substep results, stepwise rewards with discount factor $\gamma=1$ make the Q-values rank entire order chains; for end-task rewards, the discount factor propagates final correctness back through intermediate states.
  • Because the LLM is frozen and only the Actor is trained, the framework can be applied to any decomposable task with a well-defined state and action space without additional LLM supervision.
  • The reported complex-case gains suggest the method is most valuable where instance complexity makes the choice of next subtask harder for a fixed or LLM-driven order.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit: if the Q-value mapping transfers beyond the three task families, the same Actor-plus-frozen-LLM separation could steer any decomposable task, because the LLM never needs retraining.
  • A testable extension: inspect the learned Q-values on MRC; if they rank sentences by lexical overlap with the question, the policy reduces to relevance ranking and a simpler deterministic baseline would replicate RLAP.
  • Another extension: since the Actor is trained on rewards produced by one specific executor LLM, the same Actor may not transfer to a different LLM; retraining per executor is a likely requirement, and the paper does not test cross-executor transfer.
  • The stated future directions of pre-training and multimodal tasks suggest using the Actor's ordering signal as a curriculum selector or modality-attention controller, which could be evaluated directly in those settings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes RLAP, a reinforcement-learning framework for adaptive multi-step planning in LLM-based NLP tasks. The task is modeled as an MDP in which a lightweight Actor network, trained with deep Q-learning, selects the next subtask to execute by estimating Q-values over natural-language state-action sequences, while the LLM acts as the task executor. The framework is instantiated for machine reading comprehension (MRC), information extraction (IE), and sentence-level text completion (STC). Experiments on multiple datasets report accuracy and F1 gains over fixed-order baselines (ChatIE, CoT), LLM-planning baselines (ToT-BFS), and the authors' earlier RL4IE, along with ablations (RLAP-random, RLAP-sequence) for IE.

Significance. If validated, RLAP is a useful contribution: it provides a general recipe for injecting a small, trainable policy into LLM pipelines without fine-tuning the LLM, and the IE ablations in Table 3 give credible evidence that the learned ordering, not just the multi-step protocol, drives performance. The attention to state construction and reward design for three task types is valuable. However, the central claim that the Actor adaptively selects better subtask orders is not fully supported because the MRC experiments lack a fixed-order control under the same multi-step protocol, and because the quantitative results are reported without error bars or significance tests despite many small gains. These issues are addressable and do not undermine the plausibility of the underlying idea.

major comments (2)
  1. [Section 4.1, Table 2] The MRC experiments never run the same sentence-by-sentence protocol with sentences in original or any fixed order. Since the LLM answers only after all sentences have been read, the sole effect of RLAP on the final answer is the sentence permutation. The comparison against IO and ToT-BFS therefore conflates the learned ordering with the multi-step protocol itself (the 'processed/candidate' framing, the final answer prompt, and recency effects from sentence segmentation). Add an RLAP-sequence baseline that uses the identical prompt template and LLM execution but processes sentences in the original document order, and an RLAP-random baseline, as done for IE in Table 3. Without this control, the reported +0.06 to +3.92 percentage-point gains on SQuAD2.0, CMRC18, C3-mix, and RACE-H cannot be attributed to the Actor's adaptive planning.
  2. [Section 4.1-4.3, Tables 2-4 and Figure 5] All quantitative results come from a single training run and a single evaluation pass, with no standard deviations, confidence intervals, or significance tests. Several headline improvements are small (CMRC18 general +0.06 percentage points, RACE-H general +0.22, Mistral-7B on NYT10 +1.4, ACE05 +0.7), and these could easily fall within run-to-run variance of the LLM or the RL training. To support the language 'significantly outperforms' and 'remains stable,' report means and variances over multiple random seeds (and multiple LLM decoding runs if sampling is used), and apply paired bootstrap or similar tests on the test sets. State whether decoding is greedy or sampling-based.
minor comments (6)
  1. [Section 3.3, Algorithm 1] In line 24, the loss is written as L(θ) = (y_j − Q(S_t, a_t; θ))^2, but the sampled transition uses state S_j, so it should be Q(S_j, a_j; θ).
  2. [Section 3.2] The sentence 'treating each subtask as an action that can and can only be executed once..' contains a duplicated period; also, the wording 'can and can only be executed once' is clearer as 'that can be executed at most once' or 'that must be executed exactly once.'
  3. [Section 4, first paragraph] The statement 'we set the input of LLMs to be plain prompts without chain-of-thought (CoT) and in-context examples' is ambiguous because the IO and CoT baselines include three in-context examples. Clarify that this applies to the RLAP executor's prompts, not to the baseline configurations.
  4. [Table 3 caption] The 'Improvement' row reports relative gains over RL4IE (e.g., (77.27−58.83)/58.83 ≈ 31.3%), but the caption does not state whether these are relative or absolute, or against which baseline. Please specify.
  5. [Section 4.1] The task setup says 'we select 2-3 questions for each context, treating a set of question-answer-context as a sample.' For multiple-choice datasets, clarify what 'answer' means in this sample definition and how accuracy is aggregated when a sample contains multiple questions.
  6. [Section 3.3 and Table 1] The claim that a single linear projection of a fixed-size encoded state-action sequence captures the linguistic features needed for ordering rests on an assumption that is only indirectly tested through final accuracy. Adding an analysis of the learned Q-values (e.g., correlation with ground-truth optimal order on a development set, or qualitative examples of selected orderings) would make the mechanism more transparent.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RLAP trains a Q-function against ground-truth rewards and evaluates on held-out test sets; the central claim is not equivalent to its inputs.

full rationale

The paper's derivation chain is self-contained in the relevant sense. The Actor model is trained with deep Q-learning (Algorithm 1) using rewards computed by comparing intermediate or final LLM outputs against ground-truth annotations, and the resulting policy is evaluated on disjoint test sets. None of the reported predictions are defined in terms of the fitted Q-values by construction: the MRC, IE, and STC metrics measure final correctness against held-out labels, not against the training objective itself. The IE ablations (RLAP-random and RLAP-sequence) directly test whether the learned order matters, and the S2P/SFB experiments compare RLAP against CoT and random-action baselines. The absence of a fixed-order MRC ablation is an experimental-control concern, not a circularity concern, since the MRC results still compare against IO and ToT-BFS baselines on held-out data. Citations to the authors' prior work (e.g., RL4IE, reference [9]) are used as a comparison baseline and as related work, but the central claim does not rest on an unverified self-citation, a uniqueness theorem, or an ansatz smuggled in through citation. The framework's Q-value parametrization (Eq. 2) and Bellman updates (Eqs. 1 and 3) are standard RL formulations applied to a defined MDP, and training rewards are not renamed as predictions. Therefore, no circular step is exhibited.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central contribution rests on a learned Q-function over text sequences; the ledger shows the key hand-chosen hyperparameters and domain assumptions. No new physical or conceptual entities are introduced.

free parameters (5)
  • discount factor gamma = 0.5 (Appendix B)
    Hand-chosen; affects how episode-level rewards propagate to intermediate Q-values.
  • exploration schedule = epsilon starts 0.9, decays by 0.95 per 100 steps to 0.02
    Hand-chosen; controls exploration-exploitation trade-off in DQN training.
  • replay buffer size and target update frequency = buffer 5000, target update every 20 steps
    Hand-chosen DQN hyperparameters; no sensitivity analysis is provided.
  • training epochs = 10
    Hand-chosen; no early stopping or sensitivity analysis is provided.
  • foundation model per task = gte-multilingual-base, bert-base-uncased/chinese, qwen2.5-7B
    Architecture choices; no comparison across encoders is reported.
assumptions (4)
  • domain assumption Subtask decomposition meets Markov property, completeness, and homogeneity (Section 3.2 criteria i-iii)
    If subtasks are not Markov or overlap, the MDP formulation and the Q-learning update are invalid.
  • domain assumption LLM output serves as implicit transition probability P(s'|s,a), with its stochasticity treated as environment noise
    Section 3.2 Transition Function; assumes the LLM is a stable environment during both training and inference.
  • ad hoc to paper A single linear projection on top of a pretrained encoder captures linguistically meaningful state-action Q-values
    Section 3.3 Eq. 2; the core generalization assumption, not ablated across encoders.
  • domain assumption Ground-truth labels provide valid reward signals at step or episode level
    Reward design in Section 3.2 and Table 1 requires gold subtask answers for training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RLAP: A Reinforcement Learning Enhanced Adaptive Planning Framework for Multi-step NLP Task Solving." pith.science (2026). https://pith.science/paper/Q6NTPAWT

@misc{pith2026250511893,
  author       = {Pith},
  title        = {Pith review of: RLAP: A Reinforcement Learning Enhanced Adaptive Planning Framework for Multi-step NLP Task Solving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q6NTPAWT}},
  note         = {Machine review of arXiv:2505.11893}
}
read the original abstract

Multi-step planning has been widely employed to enhance the performance of large language models (LLMs) on downstream natural language processing (NLP) tasks, which decomposes the original task into multiple subtasks and guide LLMs to solve them sequentially without additional training. When addressing task instances, existing methods either preset the order of steps or attempt multiple paths at each step. However, these methods overlook instances' linguistic features and rely on the intrinsic planning capabilities of LLMs to evaluate intermediate feedback and then select subtasks, resulting in suboptimal outcomes. To better solve multi-step NLP tasks with LLMs, in this paper we propose a Reinforcement Learning enhanced Adaptive Planning framework (RLAP). In our framework, we model an NLP task as a Markov decision process (MDP) and employ an LLM directly into the environment. In particular, a lightweight Actor model is trained to estimate Q-values for natural language sequences consisting of states and actions through reinforcement learning. Therefore, during sequential planning, the linguistic features of each sequence in the MDP can be taken into account, and the Actor model interacts with the LLM to determine the optimal order of subtasks for each task instance. We apply RLAP on three different types of NLP tasks and conduct extensive experiments on multiple datasets to verify RLAP's effectiveness and robustness.

Figures

Figures reproduced from arXiv: 2505.11893 by the authors.

Figure 1
Figure 1. (a) For the same task instance, different action or [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An illustration of RLAP in MRC tasks. consistent across our method and baselines, and it has no overlap with the training data. 4.1 Machine Reading Comprehension Machine Reading Comprehension (MRC) task is a challenging task and hot topic in NLP, where the goal is to answer the questions regarding a given context. In this paper, we consider two types of MRC tasks: the extractive QA requires extracting a continuous s… view at source ↗
Figure 3
Figure 3. An example of RLAP in IE tasks. Baselines. We apply the ChatIE [39] multi-turn instruction tem￾plate to open-source LLMs, with 5 in-context samples. We also use the previous method that employs an LLM as a reward model and only considers step-wise rewards (denoted RL4IE in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: An illustration of RLAP in STC tasks. known as Sentence Cloze-Style MRC). The S2P task involves con￾catenating the given candidate sentences into a fluent paragraph. The SFB task provides a context with several blanks, and we need to select the most appropriate sentenc…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 24 canonical work pages

  1. [13]

    Wenhao Huang, Jiaqing Liang, Zhixu Li, Yanghua Xiao, and Chuanjun Ji. 2023. Adaptive Ordered Information Extraction with Deep Reinforcement Learning. In Findings of the Association for Computational Linguistics: ACL 2023. 13664–13678

    work page 2023

  2. [1]

    Jonathan D Chang, Kiante Brantley, Rajkumar Ramamurthy, Dipendra Misra, and Wen Sun. 2023. Learning to generate better than your llm. arXiv preprint arXiv:2306.11816 (2023)

    arXiv 2023

  3. [2]

    Qiao Cheng, Juntao Liu, Xiaoye Qu, Jin Zhao, Jiaqing Liang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan, and Yanghua Xiao. 2021. HacRED: A Large- Scale Relation Extraction Dataset Toward Hard Cases in Practical Applications. In Findings of ACL: ACL-IJCNLP 2021 . 2819–2831

    work page 2021

  4. [3]

    Yiming Cui, Ting Liu, Wanxiang Che, Li Xiao, Zhipeng Chen, Wentao Ma, Shijin Wang, and Guoping Hu. 2019. A Span-Extraction Dataset for Chinese Machine Reading Comprehension. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Ass...

    doi:10.18653/v1/d19-1600 2019

  5. [4]

    Yiming Cui, Ting Liu, Ziqing Yang, Zhipeng Chen, Wentao Ma, Wanxiang Che, Shijin Wang, and Guoping Hu. 2020. A Sentence Cloze Dataset for Chinese Machine Reading Comprehension

    work page 2020

  6. [5]

    Gautier Dagan, Frank Keller, and Alex Lascarides. 2023. Dynamic planning with a llm. arXiv preprint arXiv:2308.06391 (2023)

    arXiv 2023

  7. [6]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    arXiv 2018

  8. [7]

    Yan Ding, Xiaohan Zhang, Chris Paxton, and Shiqi Zhang. 2023. Task and motion planning with large language models for object rearrangement. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2086–2092

    work page 2023

Show all 54 references
  1. [8]

    Zepeng Ding, Wenhao Huang, Jiaqing Liang, Yanghua Xiao, and Deqing Yang

  2. [9]

    Zepeng Ding, Ruiyang Ke, Wenhao Huang, Guochao Jiang, Yanda Li, Deqing Yang, and Jiaqing Liang. 2024. Adaptive reinforcement learning planning: Har- nessing large language models for complex information extraction.arXiv preprint arXiv:2406.11455 (2024)

    2024 arXiv

  3. [10]

    Sina Gholamian and Domingo Huh. 2024. Reinforcement Learning Problem Solving with Large Language Models. arXiv preprint arXiv:2404.18638 (2024)

    2024 arXiv

  4. [11]

    Shixiang Gu, Timothy Lillicrap, Zoubin Ghahramani, Richard E Turner, and Sergey Levine. 2016. Q-prop: Sample-efficient policy gradient with an off-policy critic. arXiv preprint arXiv:1611.02247 (2016)

    2016 arXiv

  5. [12]

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. 2023. Reasoning with Language Model is Planning with World Model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bal...

    2023

  6. [14]

    Wenhao Huang, Chenghao Peng, Zhixu Li, Jiaqing Liang, Yanghua Xiao, Liqian Wen, and Zulong Chen. 2024. AutoCrawler: A Progressive Understanding Web Agent for Web Crawler Generation. arXiv preprint arXiv:2404.12753 (2024)

    2024 arXiv

  7. [15]

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al . 2022. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608 (2022)

    2022 arXiv

  8. [16]

    Julia Kreutzer, Joshua Uyheng, and Stefan Riezler. 2018. Reliability and Learn- ability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning. In Proceedings of the 56th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers...

    2018

  9. [17]

    Sawan Kumar and Partha Talukdar. 2021. Reordering Examples Helps during Priming-based Few-Shot Learning. In Findings of the Association for Computa- tional Linguistics: ACL-IJCNLP 2021. 4507–4518

    2021

  10. [18]

    Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. RACE: Large-scale ReAding Comprehension Dataset From Examinations. arXiv preprint arXiv:1704.04683 (2017)

    2017 arXiv

  11. [19]

    Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao

  12. [20]

    Xinyu Li, Fayuan Li, Lu Pan, Yuguang Chen, Weihua Peng, Quan Wang, Yajuan Lyu, and Yong Zhu. 2020. DuEE: a large-scale dataset for Chinese event extraction in real-world scenarios. In Natural Language Processing and Chinese Computing: 9th CCF International Conference, NLPCC 20...

    2020

  13. [21]

    Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. 2023. Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477 (2023). Ding et al

    2023 arXiv

  14. [22]

    Hongxuan Liu, Zhiyao Luo, and Tingting Zhu. 2024. Best of Both Worlds: Har- monizing LLM Capabilities in Decision-Making and Question-Answering for Treatment Regimes. In Advancements In Medical Foundation Models: Explainabil- ity, Robustness, Security, and Beyond

    2024

  15. [23]

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747 (2023)

    2023 arXiv

  16. [24]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)

    2013 arXiv

  17. [25]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. nature 518, 7540 (2015), 529–533

    2015

  18. [26]

    OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]

    2023 arXiv

  19. [27]

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2023. Measuring and Narrowing the Compositionality Gap in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2023 . 5687–5711

    2023

  20. [28]

    Pengda Qin, Weiran Xu, and William Yang Wang. 2018. Robust Distant Supervi- sion Relation Extraction via Deep Reinforcement Learning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2137–2147

    2018

  21. [29]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K...

    2023

  22. [30]

    Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know What You Don’t Know: Unanswerable Questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . 784–789

    2018

  23. [31]

    Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases . Springer, 148–163

    2010

  24. [32]

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. InProceedings of the 40th International Conference on Machine Learning. 31210–31227

    2023

  25. [33]

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in Neural Information Processing Systems 33 (2020), 3008–3021

    2020

  26. [34]

    Kai Sun, Dian Yu, Dong Yu, and Claire Cardie. 2020. Investigating Prior Knowl- edge for Challenging Chinese Machine Reading Comprehension. Transactions of the Association for Computational Linguistics (2020). https://arxiv.org/abs/1904. 09679v3

    2020

  27. [35]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  28. [36]

    Hado Van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double q-learning. InProceedings of the AAAI conference on artificial intelligence, Vol. 30

    2016

  29. [37]

    William Yang Wang, Jiwei Li, and Xiaodong He. 2018. Deep reinforcement learning for NLP. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts. 19–21

    2018

  30. [38]

    Christopher JCH Watkins and Peter Dayan. 1992. Q-learning. Machine learning 8 (1992), 279–292

    1992

  31. [39]

    Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, et al . 2023. Zero-shot information extraction via chatting with chatgpt. arXiv preprint arXiv:2302.10205 (2023)

    2023 arXiv

  32. [40]

    Jiawei Wu, Lei Li, and William Yang Wang. 2018. Reinforced Co-Training. In Proceedings of NAACL-HLT. 1252–1262

    2018

  33. [41]

    Chenhao Xie, Jiaqing Liang, Jingping Liu, Chengsong Huang, Wenhao Huang, and Yanghua Xiao. 2021. Revisiting the Negative Data of Distantly Supervised Relation Extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th In...

    2021

  34. [42]

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024. Travelplanner: A benchmark for real-world planning with language agents. arXiv preprint arXiv:2402.01622 (2024)

    2024 arXiv

  35. [43]

    An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang, Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, Weijia Xu, Wenbiao Yin, Wenyuan Yu, Xiafei Qiu, Xingzhang R...

    2025 arXiv

  36. [44]

    Sherry Yang, Ofir Nachum, Yilun Du, Jason Wei, Pieter Abbeel, and Dale Schuur- mans. 2023. Foundation models for decision making: Problems, methods, and opportunities. arXiv preprint arXiv:2303.04129 (2023)

    2023 arXiv

  37. [45]

    Jia-Yu Yao, Kun-Peng Ning, Zhen-Hui Liu, Mu-Nan Ning, Yu-Yang Liu, and Li Yuan. 2023. Llm lies: Hallucinations are not bugs, but features as adversarial examples. arXiv preprint arXiv:2310.01469 (2023)

    2023 arXiv

  38. [46]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In Advances in Neural Information Processing Systems , A. Oh, T. Naumann, A. Globerson, K. Saenko, ...

    2023

  39. [47]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In The Eleventh International Conference on Learning Representations

    2023

  40. [48]

    Qingyu Yin, Yu Zhang, Weinan Zhang, Ting Liu, and William Yang Wang. 2018. Deep Reinforcement Learning for Chinese Zero Pronoun Resolution. In Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 569–578

    2018

  41. [49]

    Biao Zhang, Zhongtao Liu, Colin Cherry, and Orhan Firat. 2024. When scaling meets llm finetuning: The effect of data, model and finetuning method. arXiv preprint arXiv:2402.17193 (2024)

    2024 arXiv

  42. [50]

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, et al. 2024. mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval. In Proceedings of the 2024 Conference ...

    2024

  43. [51]

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, et al. 2022. Least-to- most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625 (2022)

    2022 arXiv

  44. [52]

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019). RLAP: A Reinforcement Learning Enhanced Adaptive Plann...

    2019 arXiv

  45. [2016]

    In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

    Deep Reinforcement Learning for Dialogue Generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing . 1192–1202

    2016

  46. [2024]

    In Proceedings of the 2024 Joint In- ternational Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)

    Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction. In Proceedings of the 2024 Joint In- ternational Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 8890–8901

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.