Pith. sign in

REVIEW 4 major objections 5 minor 93 references

Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding turn-level rewards for using retrieved evidence — not just landing on the right answer — lifts a search agent's reasoning faithfulness and keeps or improves its accuracy.

desk verdict The faithfulness taxonomy is a useful contribution, but the headline gains are measured with the same reward model that generated the training rewards, so the empirical claim is not yet established. read the letter →

arxiv 2510.13272 v3 pith:3Z5HKBSH submitted 2025-10-15 cs.CL

Zhichao Xu , Zongyu Wu , Yun Zhou , Aosong Feng , Kang Zhou , Sangmin Woo , Kiran Ramnath , Yijun Tian
show 4 more authors
Xuan Qi Weikang Qiu Lin Lee Cheong Haibo Ding
This is my paper
classification cs.CL
keywords retrieval-augmentedgenerationagenticsearchreinforcementlearningchain-of-thoughtfaithfulnessprocessrewardsrewardmodelquestionansweringRLVR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Search agents trained to maximize final-answer correctness can get the right answer while ignoring or distorting the evidence they retrieved in the middle of their reasoning. The paper formalizes this failure as three traceability checks — do the thoughts justify the search queries, do the thoughts use the retrieved information, and is the final answer grounded in the thoughts — and shows that a strong outcome-trained agent (Search-R1) fails primarily on the information-to-thought link. It then introduces VERITAS, which turns the two problematic checks into fine-grained turn-level rewards added to the exact-match reward in reinforcement learning. On seven QA benchmarks, VERITAS-R1 improves Information-Think faithfulness by about 15% relative to Search-R1 and Think-Answer faithfulness by about 3%, while matching or slightly improving exact-match accuracy. A sympathetic reader should care because this suggests process supervision can make retrieval-augmented agents more trustworthy without trading away performance.

What carries the argument

The load-bearing object is the turn-level faithfulness reward computed over the agent's structured trajectory — the <think>, <search>, <information>, <answer> blocks. A distilled reward model (a Qwen2.5-14B fine-tuned with LoRA on labels from a stronger judge) scores each transition; in particular, the Information-Think score asks whether the new think block is a genuine synthesis of the just-retrieved information. This score is added as a dense per-turn reward to the sparse exact-match outcome reward with a small weight, creating an optimization signal at the exact point where unfaithfulness occurs. The Think-Search dimension is measured but, because outcome-only models already score high o

What would settle it

Score held-out VERITAS-R1 trajectories with a faithfulness judge that was not used anywhere in training (fresh human annotation, or a strong judge excluded from the reward-model distillation) and compare the Information-Think score against Search-R1; the paper's reported gains should survive this re-scoring. A complementary counterfactual: take a trajectory, replace a retrieved passage with contradictory content, and check whether the following think block and final answer change — if they do not, the model is not actually using the evidence.

Watch

Extended reading notes

Core claim

The central claim is that unfaithful chain-of-thought in RL-trained search agents is not an inevitable byproduct of tool use but a misspecified reward: outcome-only training never asks whether the reasoning steps are true to the retrieved evidence. VERITAS operationalizes that by rewarding, at each turn, (1) whether the thought block actually engages the retrieved information (Information-Think faithfulness) and (2) whether the final answer is entailed by the last thought (Think-Answer faithfulness), with weights 0.9 on exact match and smaller weights on the two faithfulness signals. Using the same PPO pipeline, retriever, and corpus as Search-R1, the resulting policy's average Information-T

Load-bearing premise

The paper's central claim depends on its trained reward model being a truthful judge of whether the agent's thinking uses the retrieved evidence, and the same judge is used for both training and evaluation — if that judge rewards format rather than substance, the reported faithfulness gains could be an artifact.

Editorial extensions

If this is right

  • If the central claim holds, process-based faithfulness rewards are a viable training objective: models can be made to reason from evidence rather than merely to the answer, and the effect shows up on held-out QA datasets without an accuracy penalty.
  • The Information-Think reward does most of the work, so future agentic-search training can focus on the evidence-to-thought link and spend less effort on answer-to-thought alignment.
  • Because Search-R1 already had high Think-Search faithfulness, outcome-only RL appears to teach agents to search deliberately; the gap is in using what they find and grounding the final answer.
  • The evaluation framework (three traceability checks over tagged trajectories) can serve as a general diagnostic for any retrieval or tool-using agent with structured outputs.
  • A small Think-Answer reward improves answer grounding on target datasets, but too large a weight overfits to training domains — so the practical recipe is asymmetric weights across faithfulness dimensions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same evidence-to-thought reward could be applied to any RL agent that alternates observations and reasoning — web automation, code interpreters, database querying — since the Information-Think check does not depend on search syntax, only on whether the next reasoning step uses the most recent observation.
  • Editorial inference: because the reward model used for training is the same one used to measure the results, an independent evaluation — fresh human labels on final VERITAS-R1 trajectories, or a judge model excluded from distillation — is the key untested check on whether the gains reflect genuine evidence use rather than optimization against the training judge.
  • Editorial inference: the regex-based Think-Answer metric likely undercounts faithful paraphrases, so the true answer-grounding improvement could be larger than reported; a semantic-entailment version of the metric would sharpen the measurement.
  • Editorial inference: the asymmetry in reward weights implies a practical rule of thumb — moderate weight on evidence-use, small weight on answer-grounding — which could be tuned dynamically by domain difficulty in future systems.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies faithfulness of RL-trained agentic search systems. It defines three faithfulness dimensions (Think-Search, Information-Think, Think-Answer), evaluates Search-R1, and proposes VERITAS, which adds turn-level faithfulness rewards R_info-think and R_think-answer to the exact-match reward (Eq. 1), using a distilled Qwen2.5-14B reward model. Experiments on seven QA benchmarks report that VERITAS-R1 improves Information-Think faithfulness substantially and Think-Answer faithfulness modestly while maintaining task accuracy. The manuscript includes ablations, hyperparameter sensitivity studies, and reward-model scaling experiments.

Significance. If the empirical claims held, the paper would provide a practical demonstration that process supervision on faithfulness can improve reasoning reliability without sacrificing accuracy, and its trajectory-level faithfulness metrics could become a reusable evaluation framework. Strengths include a clear formalization of the three faithfulness dimensions, reported reward-model agreement statistics, and ablation over reward components and weights. However, the headline faithfulness gains are measured with the same reward model used to train the policy, and the human validation does not cover final policy trajectories; the significance is therefore conditional on independent verification. The inconsistent headline numbers in the abstract versus Table 6 further reduce the current reliability of the reported effects.

major comments (4)
  1. [Section 4, Eq. (1), Figure 3/Table 6] The central faithfulness improvement is evaluated with the same reward model used as the RL reward. Section 4 states 'we use this fine-tuned RM for all subsequent RL training and evaluation.' The RM is distilled from Claude-3.7 labels on Search-R1 trajectories; the 50-sample human check (Table 9) is on the RM test split, again Search-R1 trajectories, not rollouts of the trained VERITAS-R1 policy. Because R_info-think in Eq. (1) directly rewards the RM's judgment, the Table 6 gains (Info-Think 0.607 to 0.849 for EM+Info-Think; 0.607 to 0.700 for full VERITAS) may reflect optimizing surface cues of that judge rather than genuine faithfulness. Please evaluate final VERITAS-R1 rollouts with an independent judge (e.g., Claude-3.7-Sonnet or human annotators) and report agreement and per-trajectory examples.
  2. [Section 3.3, Think-Answer metric and Eq. (1)] The regex metric counts an answer as faithful if its content appears in the preceding <think> block. Since R_think-answer directly rewards exactly this substring match, the reported Think-Answer increase (Table 6: 0.805 to 0.831 for full VERITAS; 0.805 to 0.853 for EM+Think-Ans) may simply reflect the policy copying the answer into the thought block. This is metric gaming, not evidence of entailed reasoning. Please report qualitative examples and/or evaluate Think-Answer with a semantic entailment judge on final trajectories.
  3. [Abstract/Introduction vs Table 6] The paper claims 'improves information-think faithfulness by 15.3% and think-answer faithfulness by 3.2% over Search-R1.' Table 6 reports absolute scores: baseline Info-Think 0.607 vs EM+Info-Think 0.849 (+24.2pp) and full VERITAS 0.700 (+9.3pp); Think-Answer baseline 0.805 vs full 0.831 (+2.6pp) and EM+Think-Ans 0.853 (+4.8pp). The 15.3%/3.2% figures do not match any row or variant. Specify which model variant and whether the percentages are absolute or relative, and align the abstract with the table.
  4. [Section 5.2 and Table 6] The text states 'VERITAS-R1 consistently outperforms the strong Search-R1 baseline across the majority of datasets' and the Conclusion says it 'achieves higher task accuracy.' Table 6 shows EM+Info-Think exceeds baseline on 4/7 datasets (NQ, TriviaQA, HotpotQA, 2wiki) and is lower on PopQA, Musique, Bamboogle; EM+Info-Think+Think-Ans exceeds baseline on only 3/7. Average EM differences are +0.006 and +0.003. No confidence intervals or significance tests are reported. The data support 'maintains accuracy' at best, not 'higher accuracy.' Please revise the claim and add statistical evaluation.
minor comments (5)
  1. [Section 2] Typo: 'consistenting' should be 'consisting'; 'RL to d-based reasoning' should be 'RL-based reasoning.'
  2. [Table 6 note] The note says EM results may differ from Table 1 due to vLLM randomness. If sampling is stochastic, average over multiple inference seeds or fix the seed so that the comparisons are not confounded by randomness.
  3. [Section G, Table 9] The human evaluation is performed by a single author. Please provide annotation instructions, report whether the annotator was blind to the model condition, and ideally use multiple independent annotators with agreement statistics.
  4. [Abstract vs Conclusion] The abstract says 'comparable task performance' while the Conclusion says 'higher task accuracy'; these are inconsistent and should be aligned.
  5. [Section 5.2] The word 'skyrockets' is subjective; replace with the precise numerical change.

Circularity Check

2 steps flagged · score 7.0 of 10

Headline faithfulness gains are measured by the same reward model and regex used as training rewards; no independent judge on final VERITAS-R1 policy, so the gains are partly forced by construction.

  1. fitted input called prediction [Section 4 'Practical implementation via a distilled reward model'; Section 5.2 Figure 3 and Table 6]
    "The final reward function is a weighted sum: R=w EM · REM +w info-think · Rinfo-think +wthink-answer · Rthink-answer (1) ... Given its strong performance and efficiency, we use this fine-tuned RM for all subsequent RL training and evaluation."

    The Info-Think component of Eq. (1) is scored by the fine-tuned Qwen2.5-14B reward model, and the Info-Think faithfulness numbers in Figure 3/Table 6 are scored by that same RM ('use this fine-tuned RM for all subsequent RL training and evaluation'). The policy is therefore trained to maximize the very function that is later reported as the faithfulness improvement; a rise in that number is the training objective itself, not an independent measurement. The 50-sample human check (Table 9) was run on the RM's evaluation split of Search-R1 trajectories, not on final VERITAS-R1 rollouts, so it does not validate RM judgments on the trained policy's distribution. The headline 15.3pp Info-Think gain is thus partly forced by construction.

  2. fitted input called prediction [Section 3.3 'Regex-Based metrics for answer faithfulness'; Eq. (1); Section 5.2 Table 6]
    "We assess whether the content of the <answer></answer> block is present in its preceding thinking content, <think></think>. ... The answer is considered faithful if a match is found, confirming that the final output was directly extracted from the last reasoning process before giving the answer."

    This regex match is used both as Rthink-answer in Eq. (1) and as the reported Think-Answer faithfulness metric (Table 6). The RL policy is directly rewarded to make answer text appear in the preceding think block; copying the answer into the thought block satisfies the metric by construction. Consequently the reported Think-Answer gains (e.g., 0.805 to 0.831 for the combined model, 0.853 for +Think-Ans) reflect optimization of the regex, not independent evidence of entailment or grounding. The paper's own limitation notes the regex 'may lack recall' and cannot recognize paraphrase/inference, so it is not a neutral validator of the training claim.

full rationale

VERITAS's contribution is a training objective; the paper's own derivation (Eq. 1) is explicit. The circularity is not in the math but in the measurement: R_info-think is produced by a distilled RM trained on Claude-3.7 labels of Search-R1 trajectories, and the same RM is used to report Info-Think faithfulness on VERITAS-R1. Since RL maximizes this RM score, the reported improvement is the training signal itself. The human validation (50 samples, Table 9) was on the RM's evaluation split of Search-R1 trajectories, so it does not certify RM behavior on the final policy's outputs. Similarly, Think-Answer faithfulness is a regex match that is also R_think-answer; it can be gamed by copying. Task accuracy (EM) results are independent and not circular, and no load-bearing self-citation or imported uniqueness theorem appears. But the central faithfulness claim, as measured, is not independent of the training reward; score 7 reflects partial circularity rather than a fully tautological derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on one tuned weighted reward function and on the assumption that a distilled judge trained on Claude labels is a valid, transferable measure of faithfulness. The judge itself is the main unexamined load-bearing component.

free parameters (3)
  • w_EM = 0.9
    Weight on exact-match reward in Eq. (1); selected from the hyperparameter sensitivity study in Figure 4.
  • w_info-think = 0.05
    Weight on Information-Think faithfulness reward; reported as the optimal balance after tuning.
  • w_think-ans = 0.02
    Weight on Think-Answer reward; set lower because 0.05 'leads to overfit on the trainset' (Section 5.2).
assumptions (4)
  • domain assumption LLM-as-a-judge (Claude-3.7-Sonnet) faithfulness labels are valid ground truth for Information-Think and Think-Search.
    Used to build the reward-model training set; no independent benchmark, only a 50-sample human agreement check on RM test examples.
  • domain assumption A regex match between the final answer and the preceding thinking block measures Think-Answer faithfulness.
    The paper itself notes this metric lacks recall and fails on paraphrasing (Limitations section).
  • ad hoc to paper The distilled RM's judgments transfer from Search-R1 trajectories to the final VERITAS-R1 policy trajectories.
    The RM was trained and labeled on Search-R1 trajectories but used to evaluate the new policy; no validation on final policy outputs is provided.
  • ad hoc to paper Information-Think faithfulness is the key optimization target, while Think-Search is dropped because it is already high.
    Section 3.4: only Information-Think and Think-Answer rewards are used in training because Think-Search is already high; this post-hoc selection shapes the headline claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation." pith.science (2026). https://pith.science/paper/3Z5HKBSH

@misc{pith2026251013272,
  author       = {Pith},
  title        = {Pith review of: Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3Z5HKBSH}},
  note         = {Machine review of arXiv:2510.13272}
}
read the original abstract

Inspired by the success of reinforcement learning (RL) in Large Language Model (LLM) training for domains like math and code, recent work has begun training LLMs to dynamically plan, query, and reason with search engines as tools -- a paradigm increasingly referred to as agentic search. Although these methods achieve performance improvement across popular short-form QA benchmarks, many prioritize final answer correctness while overlooking the quality of intermediate reasoning steps, which may lead to chain-of-thought unfaithfulness. In this paper, we first introduce a comprehensive evaluation framework for agentic search, covering three distinct faithfulness metrics: Think-Search faithfulness, Information-Think faithfulness, and Think-Answer faithfulness. Our evaluations reveal that canonical agentic search systems trained through Reinforcement Learning from Verifiable Reward (RLVR) using episode-level outcome-based reward -- including Search-R1 and ReSearch -- have significant room for improvement on these faithfulness dimensions. To foster faithful reasoning in agentic search, we introduce VERITAS (Verifying Entailed Reasoning through Intermediate Traceability in Agentic Search), a novel framework that integrates fine-grained turn-level faithfulness rewards into the reinforcement learning process. Our experiments show that models trained with \ours not only significantly improve reasoning faithfulness, but also achieve better task performance compared to baselines trained against episode-level outcome-based reward.

Figures

Figures reproduced from arXiv: 2510.13272 by the authors.

Figure 1
Figure 1. Evaluation of Search-R1. ∗ and † denotes LLMaaJ and trained reward model, respectively. We can see models trained with outcome-based reward, their good task performance does not automatically guarantee fully faithful reasoning chains, which may be suboptimal for practical deployment of such systems. traceability: Information-Think faithfulness (do the agent’s thoughts consider the retrieved infor￾mation?), Think-Sea… view at source ↗
Figure 2
Figure 2. The pipeline of VERITAS-R1 in the RL framework with PPO RL algorithm. Applying process supervision (VERITAS Reward Model) improves the faithfulness of policy model’s intermediate reasoning steps. Outcome in this figure refers to the rollout trajectory used for subsequent reward calculation and advantage estimation. we define their corresponding rewards as Rinfo-think and Rthink-answer. The final reward function is a… view at source ↗
Figure 3
Figure 3. Faithfulness evaluation results comparing two variants of Search-R1 and our methods. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Hyperparameter sensitivity study. models optimized for final answers often produce untrustworthy intermediate steps. We introduced a formal evaluation framework with three faithful￾ness metrics and proposed VERITAS, a training paradigm that integrates these metrics as …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

93 extracted references · 33 linked inside Pith

  1. [1]

    Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 225--237. Association for Computational Linguistics

    2024

  2. [2]

    Anthropic . 2025. Claude 3.7 sonnet and claude code

    2025

  3. [3]

    Iv \'a n Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Conmy. 2025. Chain-of-thought reasoning in the wild is not always faithful. arXiv preprint arXiv:2503.08679

    arXiv 2025

  4. [4]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-RAG : Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511

    arXiv 2023

  5. [5]

    Pepa Atanasova, Oana-Maria Camburu, Christina Lioma, Thomas Lukasiewicz, Jakob Grue Simonsen, and Isabelle Augenstein. 2023. Faithfulness tests for natural language explanations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 283--294

    2023

  6. [6]

    Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. 2025. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. arXiv preprint arXiv:2503.11926

    arXiv 2025

  7. [7]

    Sriram Balasubramanian, Samyadeep Basu, and Soheil Feizi. 2025. A closer look at bias and chain-of-thought faithfulness of large (vision) language models. arXiv preprint arXiv:2505.23945

    arXiv 2025

  8. [8]

    Guangsheng Bao, Hongbo Zhang, Cunxiang Wang, Linyi Yang, and Yue Zhang. 2025. How likely do LLM s with C o T mimic human reasoning? In Proceedings of the 31st International Conference on Computational Linguistics, pages 7831--7850, Abu Dhabi, UAE. Association for Computational Linguistics

    2025

Show all 93 references
  1. [9]

    Oliver Bentham, Nathan Stringham, and Ana Marasovic. 2024. Chain-of-thought unfaithfulness as disguised accuracy. Transactions on Machine Learning Research. Reproducibility Certification

    2024

  2. [10]

    Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, and 1 others. 2024. Context-dpo: Aligning language models for context-faithfulness. arXiv preprint arXiv:2412.15280

    2024 arXiv

  3. [11]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  4. [12]

    Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, and Jie Fu. 2024. RQ - RAG : Learning to refine queries for retrieval augmented generation. In First Conference on Language Modeling

    2024

  5. [13]

    Chia-Yuan Chang, Zhimeng Jiang, Vineeth Rakesh, Menghai Pan, Chin-Chia Michael Yeh, Guanchu Wang, Mingzhi Hu, Zhichao Xu, Yan Zheng, Mahashweta Das, and Na Zou. 2025. MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation. In Proceedings of the 63rd Annual Meeting of t...

    2025

  6. [14]

    Sherman Chann. 2023. Non-determinism in gpt-4 is caused by sparse moe. Accessed on August, 5:2023

    2023

  7. [15]

    Mingyang Chen, Linzhuang Sun, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z Pan, Wen Zhang, Huajun Chen, and 1 others. 2025. Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470

    2025 arXiv

  8. [16]

    Yanda Chen, Ruiqi Zhong, Narutatsu Ri, Chen Zhao, He He, Jacob Steinhardt, Zhou Yu, and Kathleen McKeown. 2023. Do models explain themselves? counterfactual simulatability of natural language explanations. arXiv preprint arXiv:2307.08678

    2023 arXiv

  9. [17]

    Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu, Bing Qin, and Ting Liu. 2024. Navigate through enigmatic labyrinth a survey of chain of thought reasoning: Advances, frontiers and future. In Proceedings of the 62nd Annual Mee...

    2024

  10. [18]

    James Chua and Owain Evans. 2025. Are deepseek r1 and other reasoning models more faithful? arXiv preprint arXiv:2501.08156

    2025 arXiv

  11. [19]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    2021 arXiv

  12. [20]

    Ponti, and Siva Reddy

    Nouha Dziri, Ehsan Kamalloo, Sivan Milton, Osmar Zaiane, Mo Yu, Edoardo M. Ponti, and Siva Reddy. 2022 a . F aith D ial: A faithful benchmark for information-seeking dialogue. Transactions of the Association for Computational Linguistics, 10:1473--1490

    2022

  13. [21]

    Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane, and Siva Reddy. 2022 b . On the origin of hallucinations in conversational models: Is it the datasets or the models? In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguis...

    2022

  14. [22]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining,...

    2024

  15. [23]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997

    2023 arXiv

  16. [24]

    Gemini, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, and 1 others. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530

    2024 arXiv

  17. [25]

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346--361

    2021

  18. [26]

    Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024. A general theoretical paradigm to understand learning from human preferences. In Proceedings of The 27th International Conference on Artificial Int...

    2024

  19. [27]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  20. [28]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  21. [29]

    Horace He and Thinking Machines Lab. 2025. Defeating nondeterminism in llm inference. Thinking Machines Lab: Connectionism. Https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/

    2025

  22. [30]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609--6625, Barcelona, Spain (Onl...

    2020

  23. [31]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst

    2025

  24. [32]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  25. [33]

    Jinhao Jiang, Jiayi Chen, Junyi Li, Ruiyang Ren, Shijie Wang, Xin Zhao, Yang Song, and Tao Zhang. 2025. RAG -star: Enhancing deliberative reasoning with retrieval augmented verification and refinement. In Proceedings of the 2025 Conference of the Nations of the Americas Chapte...

    2025

  26. [34]

    Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969--7992

    2023

  27. [35]

    Bowen Jin, Jinsung Yoon, Priyanka Kargupta, Sercan O Arik, and Jiawei Han. 2025 a . An empirical study on reinforcement learning for reasoning-search interleaved llm agents. arXiv preprint arXiv:2505.15117

    2025 arXiv

  28. [36]

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025 b . Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516

    2025 arXiv

  29. [37]

    Jiajie Jin, Yutao Zhu, Zhicheng Dou, Guanting Dong, Xinyu Yang, Chenghao Zhang, Tong Zhao, Zhao Yang, and Ji-Rong Wen. 2025 c . Flashrag: A modular toolkit for efficient retrieval-augmented generation research. In Companion Proceedings of the ACM on Web Conference 2025, page 7...

    2025

  30. [38]

    Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551

    2017 arXiv

  31. [39]

    Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237--285

    1996

  32. [40]

    Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke H \"u llermeier. 2025. A survey of reinforcement learning from human feedback. Transactions on Machine Learning Research. Survey Certification

    2025

  33. [41]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199--22213

    2022

  34. [42]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019

  35. [43]

    Jack Lanchantin, Angelica Chen, Janice Lan, Xian Li, Swarnadeep Saha, Tianlu Wang, Jing Xu, Ping Yu, Weizhe Yuan, Jason E Weston, and 1 others. 2025. Bridging offline and online reinforcement learning for llms. arXiv preprint arXiv:2506.21495

    2025 arXiv

  36. [44]

    Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson E. Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, John Kernion, Kamil.e Lukovsiut.e, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandl...

    2023 arXiv

  37. [45]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Adva...

    2020

  38. [46]

    Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. H alu E val: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449--6464. Associa...

    2023

  39. [47]

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366

    2025 arXiv

  40. [48]

    Jintao Liang, Gang Su, Huifeng Lin, You Wu, Rui Zhao, and Ziyue Li. 2025. Reasoning rag via system 1 or system 2: A survey on reasoning agentic retrieval-augmented generation for industry challenges. arXiv preprint arXiv:2506.10408

    2025 arXiv

  41. [49]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. In The Twelfth International Conference on Learning Representations

    2023

  42. [50]

    Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303--5315, Singapore. Association for Computationa...

    2023

  43. [51]

    Chaitanya Malaviya, Joseph Chee Chang, Dan Roth, Mohit Iyyer, Mark Yatskar, and Kyle Lo. 2025. Contextualized evaluations: Judging language model responses to underspecified queries. Transactions of the Association for Computational Linguistics, 13:878--900

    2025

  44. [52]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Compu...

    2023

  45. [53]

    Gary Marchionini. 2006. Exploratory search: from finding to understanding. Communications of the ACM, 49(4):41--46

    2006

  46. [54]

    Katie Matton, Robert Ness, John Guttag, and Emre Kiciman. 2025. Walk the talk? measuring the faithfulness of large language model explanations. In The Thirteenth International Conference on Learning Representations

    2025

  47. [55]

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics

    2020

  48. [56]

    Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Sim PO : Simple preference optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  49. [57]

    Yifei Ming, Senthil Purushwalkam, Shrey Pandit, Zixuan Ke, Xuan-Phi Nguyen, Caiming Xiong, and Shafiq Joty. 2025. Faitheval: Can your language model stay faithful to context, even if ''the moon is made of marshmallows''. In The Thirteenth International Conference on Learning R...

    2025

  50. [58]

    Seyed Mahed Mousavi, Simone Alghisi, and Giuseppe Riccardi. 2024. Dyknow: dynamically verifying time-sensitive factual knowledge in llms. arXiv preprint arXiv:2404.08700

    2024 arXiv

  51. [59]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing sys...

    2022

  52. [60]

    Letitia Parcalabescu and Anette Frank. 2024. On measuring faithfulness or self-consistency of natural language explanations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6048--6089

    2024

  53. [61]

    Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings. 2024. Making reasoning matter: Measuring and improving faithfulness of chain-of-thought reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15012--15032

    2024

  54. [62]

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5687--5711, Singapore. Association for Comp...

    2023

  55. [63]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728--53741

    2023

  56. [64]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  57. [65]

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. 2015. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438

    2015 arXiv

  58. [66]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  59. [67]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  60. [68]

    Yaorui Shi, Sihang Li, Chang Wu, Zhiyuan Liu, Junfeng Fang, Hengxing Cai, An Zhang, and Xiang Wang. 2025. Search and refine during think: Autonomous retrieval-augmented reasoning of llms. arXiv preprint arXiv:2505.11277

    2025

  61. [69]

    Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. 2025. Agentic retrieval-augmented generation: A survey on agentic rag. arXiv preprint arXiv:2501.09136

    2025 arXiv

  62. [70]

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592

    2025 arXiv

  63. [71]

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in neural information processing systems, 33:3008--3021

    2020

  64. [72]

    Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, and Jingren Zhou. 2025. Zerosearch: Incentivize the search capability of llms without searching. arXiv preprint arXiv:2505.04588

    2025 arXiv

  65. [73]

    Richard S Sutton, Andrew G Barto, and 1 others. 1998. Reinforcement learning: An introduction. MIT press Cambridge

    1998

  66. [74]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539--554

    2022

  67. [75]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vo...

    2023

  68. [76]

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2023. Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36:74952--74965

    2023

  69. [77]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533

    2022 arXiv

  70. [78]

    Ziliang Wang, Xuhui Zheng, Kang An, Cijun Ouyang, Jialu Cai, Yuhang Wang, and Yichao Wu. 2025. Stepsearch: Igniting llms search ability via step-wise proximal policy optimization. arXiv preprint arXiv:2505.15107

    2025 arXiv

  71. [79]

    Christopher JCH Watkins and Peter Dayan. 1992. Q-learning. Machine learning, 8(3):279--292

    1992

  72. [80]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  73. [81]

    Peng Xu, Wei Ping, Xianchao Wu, Chejian Xu, Zihan Liu, Mohammad Shoeybi, and Bryan Catanzaro. 2025. Chat QA 2: Bridging the gap to proprietary LLM s in long context and RAG capabilities. In The Thirteenth International Conference on Learning Representations

    2025

  74. [82]

    Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, Bei Wang, Jimmy Lin, and Vivek Srikumar. 2025. Distillation versus Contrastive Learning: How to Train Your Rerankers. arXiv preprint arXiv:2507.08336

    2025

  75. [83]

    Zhichao Xu, Aosong Feng, Yijun Tian, Haibo Ding, and Lin Lee Cheong. 2025. CSPLADE: Learned Sparse Retrieval with Causal Language Models. arXiv preprint arXiv:2504.10816

    2025

  76. [84]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024. Qwen2.5 technical repo...

    2024 arXiv

  77. [85]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. H otpot QA : A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Langu...

    2018

  78. [86]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations

    2023

  79. [87]

    Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, and Weiqiang Jia. 2023. Cognitive mirage: A review of hallucinations in large language models. arXiv preprint arXiv:2309.06794

    2023 arXiv

  80. [88]

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301

    2025 arXiv

  81. [89]

    Qingfei Zhao, Ruobing Wang, Dingling Xu, Daren Zha, and Limin Liu. 2025. R-search: Empowering llm reasoning with search via multi-reward reinforcement learning. arXiv preprint arXiv:2506.04185

    2025 arXiv

  82. [90]

    Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. 2024. Dense text retrieval based on pretrained language models: A survey. ACM Trans. Inf. Syst., 42(4)

    2024

  83. [91]

    Chi, Quoc V Le, and Denny Zhou

    Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen, Heng-Tze Cheng, Ed H. Chi, Quoc V Le, and Denny Zhou. 2024. Take a step back: Evoking reasoning via abstraction in large language models. In The Twelfth International Conference on Learning Representations

    2024

  84. [92]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  85. [93]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.