Pith. sign in

REVIEW 4 major objections 5 minor 48 references

Think&Cite: Improving Attributed Text Generation with Self-Guided Tree Search and Progress Reward Modeling

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Attributed text generation improves when a language model searches over partial answers, reflects on retrieval before each expansion, and is guided by progress rewards.

desk verdict The first MCTS-for-attributed-generation paper with real gains; solid empirically, but the Section 4.3.1 reward derivation has a genuine formal hole (division by zero) that needs fixing before I'd trust the theory section. read the letter →

arxiv 2412.14860 v2 pith:V3GFFW5V submitted 2024-12-19 cs.CL

Junyi Li , Hwee Tou Ng This is my paper · ORCID
classification cs.CL
keywords attributedtextgenerationMonteCarlotreesearchself-reflectionprogressrewardmodelingdirectpreferenceoptimizationcitationqualityretrieval-augmentedlong-formquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Think&Cite tries to establish that attributed text generation, producing long-form answers with in-text citations, is best solved as a multi-step search problem rather than a single auto-regressive pass. The paper builds a Monte Carlo tree whose nodes are partial answer states, a sentence together with its search query, retrieved passages, and citations, and lets the language model reflect on intermediate retrieval states before expanding the tree. A progress reward scores each partial state from two directions: how much closer the text is to a complete answer, and how well the cited passages support the sentences written so far. On the ASQA, QAMPARI, and ELI5 datasets, the resulting system outperforms prompting-based and fine-tuning baselines on both correctness and citation quality. That claim matters because it identifies search and intermediate reflection, rather than more training data or larger models, as a path to more verifiable long-form generation.

What carries the argument

The engine is Self-Guided Monte Carlo Tree Search (SG-MCTS). A node is a state $s_t = [q_t, \mathcal{D}_t, y_t, \mathcal{C}_t]$ recording the retrieval query, the retrieved passages, the sentence generated, and the passages cited for that sentence. Selection follows the UCT formula $UCT(s_t)=V(s_t)+w\sqrt{\ln N(p)/N(s_t)}$; expansion runs the think-verbalize-cite loop with an added reflection step that critiques and reformulates the initial query based on the question and the retrieved passages. New nodes are scored by Progress Reward Modeling: $R(s_{t+1}) = R_g + R_a$, where $R_g$ is a weighted sum of sentence-level DPO log-ratios, $\sum_{k=0}^t w_k \log(\pi_*(y_{k+1}\mid x,y_{1:k})/\pi_{\mathrm{ref}}(y_{k+1}\mid x,y_{1:k}))$, and $R_a$ is the F1 of NLI-based citation recall and precision. That reward is backpropagated to update each ancestor's value and visit count. The mechanism is what allows the model to abandon weak evidence paths early rather than letting one bad retrieval contaminate every later sentence.

What would settle it

If a faithful re-implementation on ASQA, QAMPARI, and ELI5 fails to beat the best non-search baseline on both correctness and citation metrics, the central claim is falsified; a more surgical check is to ablate the generation-progress reward and see whether the reported citation scores on ASQA survive.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a think-verbalize-cite agent, when embedded in a self-guided tree search, produces answers that are both more factually complete and better supported by citations than standard retrieval-augmented generation. The search tree is built sentence by sentence; before a node is expanded, the model reflects on the proposed retrieval query and the passages it returned, and reformulates the query if the evidence looks weak. Each expanded node is scored by a progress reward that combines a generation-progress term derived from preference-optimized log-ratios with an attribution-progress term based on NLI citation recall and precision. The strongest configuration reaches 50.1 exact-match recall and 89.5 citation recall on ASQA, with gains on QAMPARI and ELI5, and the ablations show that removing the tree search, the reflection step, or either reward component all hurt performance.

Load-bearing premise

The framework's search signal rests on the assumption that a preference-tuned language model's sentence-level scores really measure how much closer the partial answer is to a good final answer; if that proxy fails, the generation-progress reward is an ad hoc heuristic rather than a grounded value estimate.

Editorial extensions

If this is right

  • Base models can gain attribution ability at inference time: with a DPO-aligned reward model and an NLI model, the policy model itself does not need fine-tuning for citation quality.
  • Reflection on intermediate retrieval states should stop error cascades: a bad initial query is corrected before a sentence is written, so downstream sentences do not inherit the bad evidence.
  • Search budget can be traded for quality: increasing MCTS iterations and reflection steps improves correctness and citation scores up to a point, after which over-reflection begins to hurt.
  • Generation and attribution checks are complementary: ablating either reward component degrades results, so both text quality and citation consistency contribute to the final score.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the theoretical derivation of $R_g$ yields a telescoping sum that needs no per-sentence weights; the weights $w_k=1/|y_{1:k}|$ are an extra modeling choice, and comparing weighted versus unweighted variants would test whether the grounding or the heuristic is doing the work.
  • The same reward decomposition could plausibly transfer to other evidence-constrained generation tasks such as summarization, fact-checking, or dialogue grounding, because neither the DPO reward model nor the NLI model is task-specific.
  • The reflection step is currently applied to retrieval queries; a natural extension is to reflect on the drafted sentence before finalizing citations, which would create a second chance to catch unsupported claims before they enter the tree.
  • The paper's own limitations note the computational cost of tree search; a practical extension is to test whether an adaptive stopping rule based on progress rewards can recover most of the quality gain at a fraction of the fixed iteration budget.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Think&Cite, a framework for attributed text generation that combines a think-verbalize-cite language agent with Self-Guided Monte Carlo Tree Search (SG-MCTS) and Progress Reward Modeling (PRM). SG-MCTS uses LLM self-reflection to guide tree expansion, and PRM provides two reward signals: a generation progress reward based on DPO log-ratios and an attribution progress reward based on NLI citation recall/precision F1. The method is evaluated on ASQA, QAMPARI, and ELI5 with LLaMA-3.1-8B and GPT-4o, reporting gains over several prompting-based and fine-tuning baselines. The paper also includes ablation studies, hyperparameter analyses, cost comparisons, and a released code/data repository.

Significance. If the reported results hold, the paper makes a useful contribution: it is among the first to apply tree search to attributed text generation, and the self-reflection-guided expansion is a reasonable way to reduce search cost. The code and data release, the inclusion of multiple datasets, and the component-level ablations are strengths that help the community reproduce and build on the work. The main weakness is the theoretical grounding of the generation progress reward, which contains a derivation gap and an undefined term; this is fixable but should be addressed carefully, as the ablation attributes nontrivial gains to this reward.

major comments (4)
  1. [Section 4.3.1, Eq. (9)] The generation progress reward R_g is defined as a weighted sum of DPO log-ratios with weights w_k = 1/|y_{1:k}|, but the preceding derivation establishes an unweighted telescoping sum: Σ_{k=0}^t β log(π*(a_k|s_k)/π_ref(a_k|s_k)) = V*(s_{t+1}) - V*(s_0). The weights are introduced without any justification and do not follow from the derivation. Moreover, for k=0 the weight w_0 = 1/|y_{1:0}| is undefined because |y_{1:0}| = 0 (the empty prefix); the paper does not state a special case. Since R_g is evaluated at every partial response, a literal implementation of Eq. (9) would involve division by zero. This is load-bearing: the ablation in Table 2 attributes a 3-point EM Recall drop on ASQA to removing the GP reward, and the claimed theoretical grounding of SG-MCTS rests on this reward. The authors should either derive the weights from the MDP/DPO formulation, remove them, or explicitly define w_0 and explain why the reweighting is valid; they should also confirm that the released code implements the corrected formula.
  2. [Section 5.2, Table 1] The statement that 'our approach outperforms all baselines significantly across all three datasets' is not supported by any statistical significance test. The table reports point estimates only; no confidence intervals, p-values, or multiple-comparison corrections are provided. Given that the proposed method is substantially more computationally expensive than the prompting baselines (Appendix B), the claim of significance should be backed by a bootstrap or paired test to rule out that the differences are within the noise of the evaluation sets.
  3. [Section 4.3.2 and Section 5.1] The attribution progress reward R_a is computed as the F1 of citation recall and precision, which are exactly the citation-quality metrics used in the evaluation (Section 5.1). This is not circular reasoning, but it does mean that the reported citation gains arise in part because the search directly optimizes the evaluation objective. The paper should state this overlap explicitly and provide a robustness check, for example by measuring citation quality with a different NLI model or with human judgments, to show that the gains are not an artifact of optimizing the specific NLI model used both as reward and evaluator.
  4. [Section 4.3.1] The derivation identifies the DPO-aligned model's log-ratio with the optimal policy π* for the sentence-level MDP, but the chosen DPO model (Llama-3-8B-SFR-Iterative-DPO-R2) was not trained for attributed text generation and no evidence is given that its log-ratio is a meaningful progress signal for this task. A concrete test would be to compute the correlation between R_g and final answer quality (e.g., EM Recall or citation F1) across partial prefixes; without such evidence, the 'generation progress' interpretation of R_g remains an assumption.
minor comments (5)
  1. [Section 4.3.1] The notation is inconsistent: the final displayed equation for R_g uses π* in the log-ratio, but the text says the reward is computed using a DPO policy. Please clarify that the DPO model serves as a proxy for π* in practice.
  2. [Section 4.3.1] The boundary conditions of the sentence-level MDP are under-specified: the terminal value V* is not defined, and the reward r(y|x) appears without definition. Please state the terminal value and how T (the number of sentences) is determined.
  3. [Section 1 and Section 2] The claim that this is 'the first to apply tree search algorithms to the task of attributed text generation' should be qualified, since Self-RAG, discussed in Section 2, explicitly supports tree-decoding with critique tokens. Please clarify the precise sense in which this is the first application.
  4. [Appendix C, Table 5] The prompt description refers to blue and green text colors, but the tables in the manuscript are monochrome. Either include the colored prompts as supplementary material or remove the color reference.
  5. [Figure 2] The figure shows reward values (e.g., R_g = 0.25, R_a = 1.0) but does not explain how these numbers are computed or how they relate to Eq. (9). Adding a brief notation explanation would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; external rewards and external benchmarks; the Section 4.3.1 weighting gap is a correctness concern, not a circular step.

full rationale

This paper's derivation chain is not circular. The generation progress reward Rg is computed from a pre-trained external DPO model (Llama-3-8B-SFR-Iterative-DPO-R2) and the attribution reward uses an external NLI model, neither of which is fitted to the paper's test outputs or to the ALCE benchmark labels, so no fitted parameter is renamed as a prediction. The attribution progress reward Ra is the F1 of citation recall and precision computed by an NLI model; this is the same family of quantities used in ALCE citation evaluation, so the search directly optimizes the target metric, but this is objective alignment rather than circular reasoning because the metric is an external benchmark objective, not an input that is later reported as a derived prediction. I do flag Section 4.3.1 as an omitted proof: the weights w_k = 1/|y_{1:k}| do not follow from the DPO telescoping-sum derivation, and w_0 = 1/|y_{1:0}| = 1/0 is undefined for the empty prefix; this is a rigor and implementation concern, not a circularity. The only co-authored citation (Ye and Ng 2024) is related-work context and is not load-bearing. No self-citation chain forces the central claim, so the circularity score is 0.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The central contribution is an inference-time search procedure, so most of its moving parts are hyperparameters rather than fitted scientific constants. The theoretical claim about progress rewards rests on the DPO and NLI models from prior work, and the reflection step assumes the LLM can critique its own retrieval.

free parameters (7)
  • UCT exploration weight = 0.2
    Set in Eq. (1) to balance exploration and exploitation; chosen by hand without sensitivity analysis in the main results.
  • Maximum MCTS iterations = 30
    Set in Section 5.1; Table 4 and Figure 3 show iteration count is tuned on the test set, with 30 giving the best citation F1 on ASQA.
  • Maximum reflection steps = 10
    Section 5.1; Figure 3 shows performance peaks around 10 and degrades at 20, suggesting test-set selection.
  • Maximum tree layer = 6
    Section 5.1; limits response length to at most six sentences, chosen for the datasets.
  • Number of expanded child nodes = 3
    Section 5.1; Figure 4 shows more nodes help but with diminishing returns, so 3 is chosen as a cost-quality tradeoff.
  • Number of retrieved passages per query (top-K) = 3
    Section 5.1; Figure 4 shows citation quality degrades with too many passages, so K=3 is selected.
  • Per-sentence weight w_k = 1/|y_{1:k}|
    Introduced without derivation in Eq. (9) of Section 4.3.1; later sentences receive smaller weights because the prefix token count grows. This weighting is an ad hoc choice.
assumptions (6)
  • standard math DPO log-ratio is an implicit token-level reward under max-entropy RL
    From Rafailov et al. 2023, used in Section 4.3.1 to motivate the generation progress reward.
  • domain assumption The DPO-aligned model (Llama-3-8B-SFR-Iterative-DPO-R2) approximates the optimal policy for generation progress
    Section 4.3.1 assumes the pre-trained DPO model can serve as the optimal policy π* for scoring partial responses.
  • domain assumption NLI entailment between cited passages and sentence is a valid proxy for attribution quality
    Section 4.3.2 uses T5-XXL-TRUE-NLI-Mixture to determine whether passages support sentences.
  • domain assumption LLM self-reflection on intermediate search states improves query and citation quality
    Section 4.2 relies on the LLM's ability to critique and refine initial retrieval queries.
  • standard math UCT and MCTS are effective for this discrete search space
    Section 4.2 uses standard MCTS with UCT selection; the paper adopts this without task-specific modification.
  • domain assumption Dividing the corpus into 100-word passages is beneficial for fine-grained retrieval
    Section 3 follows Gao et al. 2023b; this chunking affects retrieval and citation granularity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Think&Cite: Improving Attributed Text Generation with Self-Guided Tree Search and Progress Reward Modeling." pith.science (2026). https://pith.science/paper/V3GFFW5V

@misc{pith2026241214860,
  author       = {Pith},
  title        = {Pith review of: Think&Cite: Improving Attributed Text Generation with Self-Guided Tree Search and Progress Reward Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V3GFFW5V}},
  note         = {Machine review of arXiv:2412.14860}
}
read the original abstract

Despite their outstanding capabilities, large language models (LLMs) are prone to hallucination and producing factually incorrect information. This challenge has spurred efforts in attributed text generation, which prompts LLMs to generate content with supporting evidence. In this paper, we propose a novel framework, called Think&Cite, and formulate attributed text generation as a multi-step reasoning problem integrated with search. Specifically, we propose Self-Guided Monte Carlo Tree Search (SG-MCTS), which capitalizes on the self-reflection capability of LLMs to reason about the intermediate states of MCTS for guiding the tree expansion process. To provide reliable and comprehensive feedback, we introduce Progress Reward Modeling to measure the progress of tree search from the root to the current state from two aspects, i.e., generation and attribution progress. We conduct extensive experiments on three datasets and the results show that our approach significantly outperforms baseline approaches.

Figures

Figures reproduced from arXiv: 2412.14860 by the authors.

Figure 1
Figure 1. Given a question, the model generates texts by [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall framework of our proposed Think&Cite approach. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Results on ASQA w.r.t. the number of itera￾tions (left) or the number of reflection steps (right). w/o GP Reward and w/o AP Reward lead to worse performance, indicating that both generation and citation quality check are critical. Reflection vs MCTS Iteration. In each iteration, SG-MCTS employs four key steps and reflection to improve the quality of intermediate states in the expansion phase by criticizing and refin… view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: A qualitative example from ASQA showing the attributed generation process of Think&Cite. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 27 canonical work pages

  1. [1]

    Samuel Joseph Amouyal, Ohad Rubin, Ori Yoran, Tomer Wolfson, Jonathan Herzig, and Jonathan Berant. 2022. QAMPARI : An open-domain question answering benchmark for questions with many answers from multiple paragraphs. CoRR, abs/2205.12665

    arXiv 2022

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    work page 2024

  3. [3]

    Bradley C. A. Brown, Jordan Juravsky, Ryan Saul Ehrlich, Ronald Clark, Quoc V. Le, Christopher R \' e , and Azalia Mirhoseini. 2024. https://doi.org/10.48550/ARXIV.2407.21787 Large language monkeys: Scaling inference compute with repeated sampling . CoRR, abs/2407.21787

    arXiv doi:10.48550/arxiv.2407.21787 2024

  4. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    2020

  5. [5]

    Lucas, Peter I

    Cameron Browne, Edward Jack Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez Liebana, Spyridon Samothrakis, and Simon Colton. 2012. A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games , 4(1):1--43

    work page 2012

  6. [6]

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. 2023. https://doi.org/10.48550/ARXIV.2310.05915 Fireact: Toward language agent fine-tuning . CoRR, abs/2310.05915

    arXiv doi:10.48550/arxiv.2310.05915 2023

  7. [7]

    Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. ELI5: long form question answering. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers , pages 3558--3567. Association for Computational Linguistics

    work page 2019

  8. [8]

    Constanza Fierro, Reinald Kim Amplayo, Fantine Huot, Nicola De Cao, Joshua Maynez, Shashi Narayan, and Mirella Lapata. 2024. Learning to plan and generate text with citations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024 , pages 11397--11...

    work page 2024

Show all 48 references
  1. [9]

    Zhao, Ni Lao, Hongrae Lee, Da - Cheng Juan, and Kelvin Guu

    Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da - Cheng Juan, and Kelvin Guu. 2023 a . RARR: researching and revising what language models say, using language models. In Proceedings of the 61st ...

    2023

  2. [10]

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023 b . Enabling large language models to generate text with citations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023 , pages 6465--6488...

    2023

  3. [11]

    Hart, Nils J

    Peter E. Hart, Nils J. Nilsson, and Bertram Raphael. 1968. A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cybern. , 4(2):100--107

    1968

  4. [12]

    Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansky, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. TRUE: re-evaluating factual consistency evaluation. In Proceedings of the 2022 Conference of the North America...

    2022

  5. [13]

    Courville, Alessandro Sordoni, and Rishabh Agarwal

    Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron C. Courville, Alessandro Sordoni, and Rishabh Agarwal. 2024. https://doi.org/10.48550/ARXIV.2402.06457 V-STaR : Training verifiers for self-taught reasoners . CoRR, abs/2402.06457

    2024 doi arXiv

  6. [14]

    Chengyu Huang, Zeqiu Wu, Yushi Hu, and Wenya Wang. 2024. Training language models to generate text with citations via fine-grained rewards. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Th...

    2024

  7. [15]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023. https://doi.org/10.48550/ARXIV.2311.05232 A survey on hallucination in large language models: Principles, taxonomy, challen...

    2023 doi arXiv

  8. [16]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Comput. Surv. , 55(12):248:1--248:38

    2023

  9. [17]

    Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi - Yu, Yiming Yang, Jamie Callan, and Graham Neubig

    Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi - Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Si...

    2023

  10. [18]

    Daniel Kahneman. 2011. Thinking, fast and slow. Farrar, Straus and Giroux

    2011

  11. [19]

    Levente Kocsis and Csaba Szepesv \' a ri. 2006. Bandit based Monte-Carlo planning. In Machine Learning: ECML 2006, 17th European Conference on Machine Learning, Berlin, Germany, September 18-22, 2006, Proceedings , volume 4212 of Lecture Notes in Computer Science, pages 282--2...

    2006

  12. [20]

    u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \" u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \" a schel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP task...

    2020

  13. [21]

    Dongfang Li, Zetian Sun, Baotian Hu, Zhenyu Liu, Xinshuo Hu, Xuebo Liu, and Min Zhang. 2024. Improving attributed text generation of large language models via preference learning. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and vir...

    2024

  14. [22]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let's verify step by step. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, M...

    2024

  15. [23]

    Liu, Tianyi Zhang, and Percy Liang

    Nelson F. Liu, Tianyi Zhang, and Percy Liang. 2023. Evaluating verifiability in generative search engines. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023 , pages 7001--7025. Association for Computational Linguistics

    2023

  16. [24]

    Shashi Narayan, Joshua Maynez, Reinald Kim Amplayo, Kuzman Ganchev, Annie Louis, Fantine Huot, Anders Sandholm, Dipanjan Das, and Mirella Lapata. 2023. Conditional generation with a question-answering blueprint. Trans. Assoc. Comput. Linguistics, 11:974--996

    2023

  17. [25]

    Zhao, Yi Luan, Keith B

    Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hern \' a ndez \' A brego, Ji Ma, Vincent Y. Zhao, Yi Luan, Keith B. Hall, Ming - Wei Chang, and Yinfei Yang. 2022. Large dual encoders are generalizable retrievers. In Proceedings of the 2022 Conference on Empirical Methods in ...

    2022

  18. [26]

    Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Dmytro Okhonko, Samuel Broscheit, Gautier Izacard, Patrick S. H. Lewis, Barlas Oguz, Edouard Grave, Wen - tau Yih, and Sebastian Riedel. 2021. http://arxiv.org/abs/2112.09924 The web is your oyster - knowledge-intensive NLP...

    2021 arXiv

  19. [27]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural In...

    2023

  20. [28]

    Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval , 3(4):333--389

    2009

  21. [29]

    Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December ...

    2023

  22. [30]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, Neu...

    2023

  23. [31]

    David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap,...

    2016

  24. [32]

    Aviv Slobodkin, Eran Hirsch, Arie Cattan, Tal Schuster, and Ido Dagan. 2024. Attribute first, then generate: Locally-attributable grounded text generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2...

    2024

  25. [33]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. CoRR, abs/2408.03314

    2024 arXiv

  26. [34]

    Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming - Wei Chang. 2022. ASQA: factoid questions meet long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022 , pag...

    2022

  27. [35]

    Hao Sun, Hengyi Cai, Bo Wang, Yingyan Hou, Xiaochi Wei, Shuaiqiang Wang, Yan Zhang, and Dawei Yin. 2024. Towards verifiable text generation with evolving memory and self-reflection. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNL...

    2024

  28. [36]

    Richard Sutton. 2019. The bitter lesson. Incomplete Ideas (blog), 13(1):38

    2019

  29. [37]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

    2023 arXiv

  30. [38]

    Chaojie Wang, Yanchen Deng, Zhiyi Lv, Zeng Liang, Jujie He, Shuicheng Yan, and Bo An. 2024. https://doi.org/10.48550/ARXIV.2406.14283 Q*: Improving multi-step reasoning for LLMs with deliberative planning . CoRR, abs/2406.14283

    2024 doi arXiv

  31. [39]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629

    2022 arXiv

  32. [40]

    Hai Ye and Hwee Tou Ng. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1206 Preference-guided reflective sampling for aligning language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21646--21668, Miami, Florida, US...

    2024 doi

  33. [41]

    Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao. 2021. Mastering Atari games with limited data. In NeurIPS, pages 25476--25488

    2021

  34. [42]

    Xiao Yu, Baolin Peng, Vineeth Vajipey, Hao Cheng, Michel Galley, Jianfeng Gao, and Zhou Yu. 2024. Improving autonomous AI agents with reflective tree search and self-learning. arXiv preprint arXiv:2410.02052

    2024 arXiv

  35. [43]

    Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, et al. 2024. LLaMA-Berry : Pairwise optimization for O1-like Olympiad-Level mathematical reasoning. arXiv preprint arXiv:2410.02884

    2024 arXiv

  36. [44]

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023. https://doi.org/10.48550/ARXIV.2309.01219 Siren's song in the AI ocean: A survey on hallucin...

    2023 doi arXiv

  37. [45]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian - Yun Nie, and Ji - Ro...

    2023 arXiv

  38. [46]

    Andy Zhou, Kai Yan, Michal Shlapentokh - Rothman, Haohan Wang, and Yu - Xiong Wang. 2024. Language agent tree search unifies reasoning, acting, and planning in language models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27,...

    2024

  39. [47]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.