Pith. sign in

REVIEW 4 major objections 4 minor 48 references

Two-way Evidence self-Alignment based Dual-Gated Reasoning Enhancement

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ESA-DGR improves knowledge-intensive multi-step reasoning by aligning evidence selection with the LLM's attention and hidden states, then fusing the evidence with the model's own knowledge through two learned gates.

desk verdict Plausible architecture, but the GRPO reward never checks the gold answer, so the reported gains are unexplained as written. read the letter →

arxiv 2505.16806 v1 pith:R4EKPFHK submitted 2025-05-22 cs.CL cs.IR

Kexin Zhang , Junlan Chen , Daifeng Li , Yuxuan Zhang , Yangyang Feng , Bowen Deng , Weixu Chen This is my paper
classification cs.CLcs.IR
keywords knowledge-intensivemulti-stepreasoningretrieval-augmentedgenerationevidenceextractionself-alignmentdual-gatedfusionGRPOmulti-hopquestionansweringhallucinationreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes ESA-DGR, a training framework for knowledge-intensive multi-step reasoning that combines two ideas: a two-way evidence self-alignment module that links evidence selection to the LLM's attention and hidden states, and a dual-gated reasoning module that fuses selected evidence with knowledge the LLM already has. The authors claim this addresses the two main failure modes of retrieval-augmented reasoning: selecting semantically similar but logically irrelevant evidence, and hallucinating when retrieved evidence is incomplete or noisy. On HotpotQA, 2WikiMultiHopQA, and MuSiQue, they report average improvements of about 4 percentage points in exact match and 5 percentage points in F1 over prior LLM fine-tuning methods, with the largest absolute gains on HotpotQA and 2WikiMultiHopQA. If the result holds, ESA-DGR offers a concrete recipe for making multi-hop question answering both more accurate and more robust to imperfect evidence.

What carries the argument

The load-bearing object is the two-way evidence self-alignment loss in TW-ESA: $L_{\mathrm{align}} = \lambda_3 \, \mathrm{CE}(\alpha_{R,i}, M_i) + \lambda_4 \, \mathrm{JS}(Z_{R,i} \| Z_{U,i}) + \lambda_5 L_s$, where $M_i$ is the binary token-selection mask over the retrieved evidence, $\alpha_{R,i}$ is the LLM's attention over the full evidence, $Z_{R,i}$ and $Z_{U,i}$ are hidden states of the full and selected evidence, and $L_s$ is a sparsity-and-continuity regularizer on the mask. This loss pushes the selector to mimic where the LLM looks, and pushes the full-evidence representation to match the selected-evidence representation, which is the mechanism claimed to teach causal relevance rather than semantic similarity. The dual-gated mechanism then computes two learned gates: the first fuses $Z_{U,i}$ with $Z_{R,i}$, and the second fuses that result with the original LLM hidden state $Z_i$, producing the final state $Z_{\mathrm{final}}$. A GRPO objective with a structure-aware reward supplies the reasoning-quality signal, and a KL term keeps the second gate from drifting too far from first-level reasoning.

What would settle it

Run a sufficiency test on correctly answered examples: erase every token that ESA-DGR did not select from the input and regenerate the answer; if exact match and F1 stay flat, the selection mask is not causally load-bearing. As a control, replace the LLM attention target $\alpha_{R,i}$ in $L_{\mathrm{align}}$ with a random attention vector of matched entropy; if scores do not drop, the two-way alignment is not the source of the reported gains.

Watch

Extended reading notes

Core claim

ESA-DGR's central claim is that the two known weaknesses of retrieval-augmented multi-hop reasoning—semantic-logic mismatch in evidence extraction and uncertainty-aware hallucination—can be treated as alignment problems. The TW-ESA module trains a rationale information extraction module by aligning the token-selection distribution with the LLM's attention over the full evidence, and by aligning the hidden states computed from full evidence with those computed from selected evidence. The DGR module then combines three sources: the strict model over selected evidence, the LLM-enhanced representation over full evidence, and the original LLM's hidden state, with two learned gates that decide how much of each source to trust. The whole framework is trained jointly with a GRPO objective and a KL regularizer. On three multi-hop QA benchmarks the authors report the highest exact match and F1 among compared methods, including +5.5 EM/+6.1 F1 on HotpotQA and +4.3 EM/+5.7 F1 on 2WikiMultiHopQA over the previous best baselines, with additional gains on MuSiQue.

Load-bearing premise

The whole framework rests on the assumption that aligning the evidence selector with the LLM's attention, and aligning the two evidence representations, trains the model to pick causally relevant evidence rather than merely statistically similar text.

Editorial extensions

If this is right

  • On HotpotQA and 2WikiMultiHopQA, ESA-DGR reports the largest gains over prior methods: +5.5 EM/+6.1 F1 and +4.3 EM/+5.7 F1 respectively, with additional gains on MuSiQue.
  • ESA-DGR achieves higher answer quality with a comparable or lower average number of retrieval queries than Search-o1, so the alignment and gating modules improve retrieval efficiency rather than trading accuracy for more search.
  • The extracted rationale tokens separate more cleanly from non-rationale tokens in representation space and receive higher evidence-quality scores than SEER, which supports the claim of better interpretability.
  • Ablation results show that removing either alignment loss, either gate, or the GRPO objective lowers performance, with hidden-state alignment and the second gate among the largest individual contributors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same two-way alignment could be lifted out of the GRPO training loop and used as a plug-in evidence selector for existing retrieval-augmented systems; if the causal-alignment claim is right, it should improve accuracy even without the policy-optimization component.
  • A stronger test of the paper's causal story would compare selected evidence against minimal human-authored rationale sets; the paper reports separation statistics and evidence-quality scores, but not whether the unselected tokens are individually unnecessary for the answer.
  • The dual-gated fusion recipe generalizes beyond multi-hop QA: any task with noisy retrieved evidence and a model with useful parametric knowledge could use the same two-gate structure to decide when to trust the evidence and when to trust the model.
  • Because the reported gains are smallest on MuSiQue, the framework's benefit appears tied to settings where retrieved passages are compact and reasoning chains are long; applying it to tasks with very long or multi-document evidence may require re-tuning the gates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes ESA-DGR, a framework for knowledge-intensive multi-step reasoning (KIMSR) built on a 'claim-select-reason' pipeline. A rationale information extraction (RIE) module selects tokens from retrieved evidence; the two-way evidence self-alignment (TW-ESA) module aligns the selected-token distribution and hidden states with the LLM's attention and representations; and a dual-gated reasoning enhancement (DGR) module fuses the strict selected-evidence representation with the fine-tuned and original LLM hidden states. The model is trained with an alignment loss plus GRPO-based policy optimization. Experiments on HotpotQA, 2WikiMultiHopQA, and MuSiQue with Qwen2.5-7B and LLaMA3.1-8B report improvements over ten baselines, with average gains of about 4 EM and 5 F1. The full text includes ablation, sensitivity, and efficiency analyses, an anonymous code release link, and an explicit Limitations section noting that the underlying mechanism needs further investigation.

Significance. If the reported gains are reproducible, the combination of token-level and hidden-state alignment with a two-level gating mechanism is a plausible and potentially useful recipe for improving retrieval-augmented multi-hop reasoning; the paper also evaluates on three standard benchmarks with an extensive baseline set and provides an anonymous code release, which are strengths. However, the significance is currently bounded by verification problems: the described GRPO reward does not use the golden answer, the tables contain impossible or identical values, and the 'causal' interpretation of the objective is not formally supported. I therefore cannot yet assess whether the central claim holds.

major comments (4)
  1. [Section 4.4 and Algorithm 1] The advantage A_i in Eq. (6) is computed from the reward in Algorithm 1, whose three components (CHECKFORMAT, CHECKORDER, CHECKCONFLICT) never compare the completion with the gold answer. A polished but wrong answer receives the same reward as a correct one, so the policy gradient described cannot be the source of the reported +4 EM / +5 F1 gains. Since L_align in Eq. (3) also uses only LLM attention and hidden states rather than gold labels, the training procedure as written is internally inconsistent with the empirical claim. Please add an answer-correctness term to the reward (or a supervised answer loss) and rerun, or identify the missing answer-sensitive signal.
  2. [Table 3] The Recall value 336.3 for RAG on HotpotQA exceeds 100 and is therefore not a valid percentage, and the entire DSLR row is numerically identical to the DSLR row in Table 1 for all three datasets, which is implausible across two different backbone models. These anomalies strongly suggest tabulation or copy-paste errors and undermine confidence in the reported comparisons; please correct the tables and provide the underlying runs or a reproducibility check.
  3. [Section 6.1 and Tables 1-3] All results appear to be single-run numbers without error bars, confidence intervals, or significance tests, despite the abstract's claim that ESA-DGR 'significantly surpasses' state-of-the-art methods. Please report multiple seeds with mean and standard deviation and a paired test (e.g., bootstrap or signed test) for the headline EM/F1 differences, or temper the significance wording.
  4. [Section 4.4, Eq. (5)] The lower bound I(a;Z_final) >= E[log P(a|Z_final)] - H(a) is just a conditional log-likelihood plus a constant, so maximizing it does not, by itself, enforce a 'causal mapping' from evidence to answer; the paper's causal language in the abstract and contributions is therefore not supported by the derivation. Either supply a formal argument for why the alignment and gating losses enforce causal selection, or soften the causal claims.
minor comments (4)
  1. [Section 6.4 and Table 4] The structural parameters k=8 and maximum claim steps=3 are selected on the development data and then used for the final results; please state the model-selection protocol and whether the test numbers were produced only after fixing this configuration, to rule out selection bias.
  2. [Appendix F and Appendix G] The case studies end with 'Incorrect answer' but the text does not identify which system produced these outputs; please clarify whether these are baseline failures or failures of ESA-DGR, as the current placement is ambiguous.
  3. [Section 4.2 and Section 6.3] Section 4.2 describes lambda_5 as a Lagrange multiplier that 'should be estimated', while Section 6.3 treats it as a tuned loss weight; please clarify the estimation procedure and report the estimated or tuned value.
  4. [Throughout] There are several typos and inconsistencies: 'Figuurnov' should be 'Figurnov', 'MuSiQue' and 'Musique' are used interchangeably, and the example 'the capital of the United States is Washington' in Section 4.3 is confusingly phrased.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are empirical benchmark results, not derived from the method's definitions or from a self-citation chain.

full rationale

The paper's central claim is empirical: ESA-DGR outperforms baselines on HotpotQA, 2WikiMultiHopQA, and MuSiQue (Tables 1 and 3). These are external benchmarks, so the performance numbers are not produced by definition from the loss functions. The method builds on cited prior work (mPLUG-Owl2 alignment, SEER evidence extraction, GRPO), but none of the load-bearing citations is authored by the present authors, and no uniqueness theorem is imported to forbid alternatives. The alignment loss L_align = lambda3 * CE(alpha_R,i, M_i) + lambda4 * JS(Z_R,i || Z_U,i) + lambda5 * L_s (Eq. 3) is a training regularizer, not a restatement of the answer prediction; the dual-gated fusion in Eq. (4) is an architectural choice; and the final objective in Eq. (7) is an optimization target, not an evaluation metric. One internal-consistency concern is not circularity: Algorithm 1's GRPO reward checks format, tag order, and conflict but never compares the completion against the golden answer, so the reported EM/F1 improvements are not explained by the described reward signal. This is a reproducibility and correctness risk that would need a code check, but it does not make any result equivalent to its input by construction. Similarly, the best k=8 and 3-claim-step settings and loss weights appear to be chosen on the development data and then reported as sensitivity analysis; this is mild selection bias, not circular reasoning. The limitations section also acknowledges that the mechanism 'still require further investigation,' consistent with the absence of a derived theoretical claim. Overall, the derivation chain does not reduce to its own inputs.

Assumptions & free parameters 9 free parameters · 5 assumptions · 0 invented entities

The framework relies on several modeling assumptions: attention as a proxy for logical relevance, hidden states as reliable knowledge sources, and GRPO with format-only rewards as a useful reasoning optimizer. Loss weights, the number of retrieved passages, the maximum claim steps, and reward weights are selected during development and count as free parameters. No new external entities are introduced.

free parameters (9)
  • lambda_1 (token selection sparsity) = 0.5 or 1.0 (from sensitivity analysis, exact values not reported)
    Controls penalty on number of selected tokens in L_s.
  • lambda_2 (token selection continuity) = 0.5 or 1.0 (from sensitivity analysis, exact values not reported)
    Controls penalty on changes between adjacent token selections.
  • lambda_3 (token alignment weight) = 0.5 or 1.0 (from sensitivity analysis, exact values not reported)
    Weights cross-entropy between attention and selection mask in L_align.
  • lambda_4 (hidden-state alignment weight) = 0.5 or 1.0 (from sensitivity analysis, exact values not reported)
    Weights JS divergence between hidden states in L_align.
  • lambda_5 (Lagrange multiplier for L_s) = estimated, exact value not reported
    Multiplier for selection regularizer; the paper says it 'should be estimated' but does not state the estimator.
  • top-k retrieved passages = 8
    Best performance at k=8 in Table 4; selected on HotpotQA development.
  • maximum claim steps = 3
    Best performance at 3 steps in Table 4; selected on HotpotQA development.
  • GRPO reward weights alpha_1, alpha_2, alpha_3 = not reported
    Tunable weights in Algorithm 1; values not given in the paper.
  • GRPO hyperparameters (learning rate, batch size, epsilon, beta, mu) = not reported
    Algorithm 2 requires these but no values are given.
assumptions (5)
  • domain assumption Attention scores from an LLM are a valid proxy for the logical relevance of tokens in evidence selection.
    Token-level alignment uses cross entropy between alpha_R,i and the selection mask M_i; if attention does not reflect causal relevance, the mask is trained toward a poor target.
  • domain assumption The hidden states of the original LLM contain accurate factual knowledge that can be fused through gates to reduce uncertainty-aware hallucination.
    DGR's second-level gating relies on the original LLM hidden state Z_i to correct evidence gaps, citing Azaria and Mitchell (2023) and Hu et al. (2024) without verification in this setting.
  • domain assumption GRPO with a rule-based format/order/conflict reward improves reasoning quality on KIMSR tasks.
    Collaborative training uses GRPO; the reward is only format-based, not answer-based, so optimizing it may not align with answer correctness.
  • standard math The mutual information lower bound justifies maximizing P(a|Z_final) as a causal association objective.
    Equation (5) is a standard variational lower bound, but interpreting it as 'causal association' is an added assertion not implied by the bound.
  • domain assumption The three benchmarks (HotpotQA, 2WikiMultiHopQA, MuSiQue) are representative of KIMSR and the official splits are used as reported.
    The empirical generalization rests on these three datasets without external validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Two-way Evidence self-Alignment based Dual-Gated Reasoning Enhancement." pith.science (2026). https://pith.science/paper/R4EKPFHK

@misc{pith2026250516806,
  author       = {Pith},
  title        = {Pith review of: Two-way Evidence self-Alignment based Dual-Gated Reasoning Enhancement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R4EKPFHK}},
  note         = {Machine review of arXiv:2505.16806}
}
read the original abstract

Large language models (LLMs) encounter difficulties in knowledge-intensive multi-step reasoning (KIMSR) tasks. One challenge is how to effectively extract and represent rationale evidence. The current methods often extract semantically relevant but logically irrelevant evidence, resulting in flawed reasoning and inaccurate responses. We propose a two-way evidence self-alignment (TW-ESA) module, which utilizes the mutual alignment between strict reasoning and LLM reasoning to enhance its understanding of the causal logic of evidence, thereby addressing the first challenge. Another challenge is how to utilize the rationale evidence and LLM's intrinsic knowledge for accurate reasoning when the evidence contains uncertainty. We propose a dual-gated reasoning enhancement (DGR) module to gradually fuse useful knowledge of LLM within strict reasoning, which can enable the model to perform accurate reasoning by focusing on causal elements in the evidence and exhibit greater robustness. The two modules are collaboratively trained in a unified framework ESA-DGR. Extensive experiments on three diverse and challenging KIMSR datasets reveal that ESA-DGR significantly surpasses state-of-the-art LLM-based fine-tuning methods, with remarkable average improvements of 4% in exact match (EM) and 5% in F1 score. The implementation code is available at https://anonymous.4open.science/r/ESA-DGR-2BF8.

Figures

Figures reproduced from arXiv: 2505.16806 by the authors.

Figure 1
Figure 1. The proposed ESA-DGR model. αO∈{R,U},i = {αO,i[1], αO,i[2], ...}, where αO,i[j] is the attention score of token ej in either Ei or E ′ i . 4.2 Two-way Self-alignment We define ZU,i, which uses E ′ i for reasoning, as the hidden state of the strict reasoning model. To achieve mutual enhancement between ZR,i and ZU,i, we introduce token-level and hidden-state level alignment. Token level alignment. We use cross entrop… view at source ↗
Figure 2
Figure 2. Visualization of token representations for rationale selection. Blue: correctly predicted rationale tokens; [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. ). To further examine whether ESA-DGR has effec￾tively disentangled the textual input into rationale and non-rationale representations, we visualize the token representations using UMAP in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: Answer quality and query efficiency compari [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 5
Figure 5. Figure 5: Sensitivity analysis of five loss-related hyperparameters ( [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 14 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Wasim Aftab, Zivkos Apostolou, Karim Bouazoune, and Tobias Straub. 2024. Optimizing biomedical information retrieval with a keyword frequency-driven prompt enhancement strategy. BMC bioinformatics, 25(1):281

    work page 2024

  4. [4]

    A Azaria and T Mitchell. 2023. The internal state of an llm knows when it’s lying. In Proceedings of the Association for Computational Linguistics: EMNLP'23, pages 967--976

    work page 2023

  5. [5]

    Jasmijn Bastings, Wilker Aziz, and Ivan Titov. 2019. Interpretable neural predictions with differentiable binary variables. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2963--2977

    work page 2019

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  7. [7]

    Antonia Creswell and Murray Shanahan. 2022. Faithful reasoning using large language models. arXiv preprint arXiv:2208.14271

    arXiv 2022

  8. [8]

    Juan Duque, Milad Aghajohari, Tim Cooijmans, Razvan Ciuca, Tianyu Zhang, Gauthier Gidel, and Aaron Courville. 2025. Advantage alignment algorithms. In ICLR'25

    work page 2025

Show all 48 references
  1. [9]

    Tao Feng, Lizhen Qu, Niket Tandon, Zhuang Li, Xiaoxi Kang, and Gholamreza Haffari. 2024. From pre-training corpora to large language models: What factors influence llm performance in causal discovery tasks? arXiv preprint arXiv:2407.19638

    2024

  2. [10]

    Michael Figuurnov, Shakir Mohamed, and Mnih Andriy. 2018. Implicit reparameterization gradients. In 32nd Conference on Neural Information Processing Systems (NeurIPS'18)

    2018

  3. [11]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020 a . Realm: retrieval-augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org

    2020

  4. [12]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020 b . Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929--3938. PMLR

    2020

  5. [13]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. https://doi.org/10.18653/v1/2020.coling-main.580 Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps . In Proceedings of the 28th International Conference on Computational Li...

    2020 doi

  6. [14]

    Cheng-Yu Hsieh, Chun-Liang Li, Chih-kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. https://doi.org/10.18653/v1/2023.findings-acl.507 Distilling step-by-step! outperforming larger language models with less training d...

    2023 doi

  7. [15]

    Xuming Hu, Junzhe Chen, Xiaochuan Li, Yufei Guo, Lijie Wen, Philip S Yu, and Zhijiang Guo. 2024. Towards understanding factual knowledge of large language models. In The Twelfth International Conference on Learning Representations

    2024

  8. [16]

    Taeho Hwang, Soyeong Jeong, Sukmin Cho, SeungYoon Han, and Jong Park. 2024. https://doi.org/10.18653/v1/2024.knowledgenlp-1.6 DSLR : Document refinement with sentence-level re-ranking and reconstruction to enhance retrieval-augmented generation . In Proceedings of the 3rd Work...

    2024 doi

  9. [17]

    Gautier Izacard and Edouard Grave. 2021. https://doi.org/10.18653/v1/2021.eacl-main.74 Leveraging passage retrieval with generative models for open domain question answering . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Li...

    2021 doi

  10. [18]

    Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong Park. 2024. https://doi.org/10.18653/v1/2024.naacl-long.389 Adaptive- RAG : Learning to adapt retrieval-augmented large language models through question complexity . In Proceedings of the 2024 Conference of the N...

    2024 doi

  11. [19]

    Harsh Jhamtani and Peter Clark. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.10 Learning to explain: Datasets and models for identifying valid reasoning chains in multihop question-answering . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language ...

    2020 doi

  12. [20]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  13. [21]

    Jinhyuk Lee, Mujeen Sung, Jaewoo Kang, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.acl-long.518 Learning dense representations of phrases at scale . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International...

    2021 doi

  14. [22]

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366

    2025 arXiv

  15. [23]

    Runqi Lin, Chaojian Yu, Bo Han, and Tongliang Liu. 2023. On the over-memorization during natural, robust and catastrophic overfitting. arXiv preprint arXiv:2310.08847

    2023 arXiv

  16. [24]

    Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. 2022. https://doi.org/10.18653/v1/2022.acl-long.225 Generated knowledge prompting for commonsense reasoning . In Proceedings of the 60th Annual Meeting of the Asso...

    2022 doi

  17. [25]

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.153 G -eval: NLG evaluation using gpt-4 with better human alignment . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language ...

    2023 doi

  18. [26]

    Yanming Liu, Xinyue Peng, Xuhong Zhang, Weihao Liu, Jianwei Yin, Jiannan Cao, and Tianyu Du. 2024. https://doi.org/10.18653/v1/2024.findings-acl.281 RA - ISF : Learning to answer and understand from retrieval augmentation via iterative self-feedback . In Findings of the Associ...

    2024 doi

  19. [27]

    Sewon Min, Victor Zhong, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019. https://doi.org/10.18653/v1/P19-1613 Multi-hop reading comprehension through question decomposition and rescoring . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguist...

    2019 doi

  20. [28]

    Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henghui Zhu, Xinchi Chen, Peng Xu, Zhiheng Huang, Andrew Arnold, and Dan Roth. 2022. https://doi.org/10.18653/v1/2022.findings-naacl.35 Entailment tree explanations via iterative retrieval-generation reasoner ...

    2022 doi

  21. [29]

    Avinash Patil. 2025. Advancing reasoning in large language models: Promising methods and approaches. arXiv preprint arXiv:2502.03671

    2025 arXiv

  22. [30]

    Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, and Douwe Kiela. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.713 Unsupervised question decomposition for question answering . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing...

    2020 doi

  23. [31]

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.378 Measuring and narrowing the compositionality gap in language models . In Findings of the Association for Computational Linguistics: EMNLP 20...

    2023 doi

  24. [32]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  25. [33]

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Sch\" a rli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org

    2023

  26. [34]

    Ishneet Sukhvinder Singh, Ritvik Aggarwal, Ibrahim Allahverdiyev, Muhammad Taha, Aslihan Akalin, Kevin Zhu, and Sean O'Brien. 2024. Chunkrag: Novel llm-chunk filtering method for rag systems. arXiv preprint arXiv:2410.19572

    2024 arXiv

  27. [35]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. ♫ musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539--554

    2022

  28. [36]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. https://doi.org/10.18653/v1/2023.acl-long.557 Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions . In Proceedings of the 61st Annual Meeting of th...

    2023 doi

  29. [37]

    Jinyuan Wang, Junlong Li, and Hai Zhao. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.179 Self-prompted chain-of-thought on large language models for open-domain multi-hop reasoning . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2717-...

    2023 doi

  30. [38]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  31. [39]

    Tomer Wolfson, Mor Geva, Ankit Gupta, Matt Gardner, Yoav Goldberg, Daniel Deutch, and Jonathan Berant. 2020. https://doi.org/10.1162/tacl_a_00309 Break it down: A question understanding benchmark . Transactions of the Association for Computational Linguistics, 8:183--198

    2020 doi

  32. [40]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  33. [41]

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. 2024. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In Proceedings of the ieee/cvf conference on computer vision and pattern recognit...

    2024

  34. [42]

    Linan Yue, Qi Liu, Yichao Du, Yanqing An, Li Wang, and Enhong Chen. 2022. Dare: disentanglement-augmented rationale extraction. Advances in Neural Information Processing Systems, 35:26603--26617

    2022

  35. [43]

    Automatic chain of thought prompting in large language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arxiv 2022. arXiv preprint arXiv:2210.03493

    2022 arXiv

  36. [44]

    Jiachen Zhao, Wenlong Zhao, Andrew Drozdov, Benjamin Rozonoyer, Arafat Sultan, Jay-Yoon Lee, Mohit Lyyer, and Andrew McCallum. 2024 a . https://aclanthology.org/2024.acl-long.766/ Multistage collaborative knowledge distillation from a large language model for semi-supervised s...

    2024

  37. [45]

    Xinping Zhao, Dongfang Li, Yan Zhong, Boren Hu, Yibin Chen, Baotian Hu, and Min Zhang. 2024 b . https://doi.org/10.18653/v1/2024.emnlp-main.178 SEER : Self-aligned evidence extraction for retrieval-augmented generation . In Proceedings of the 2024 Conference on Empirical Metho...

    2024 doi

  38. [46]

    Zhe Zhao, Pengkun Wang, Haibin Wen, Yudong Zhang, Zhengyang Zhou, and Yang Wang. 2024 c . Atwist for graph classification: Optimizing causal information flow in graph neural networks. In The Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24)

    2024

  39. [47]

    Denny Zhou, Nathanael Sch \"a rli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, et al. 2022. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625

    2022 arXiv

  40. [48]

    Zhi-Hua Zhou and Zhi-Hao Tan. 2024. Learnware: Small models do big. Science China Information Sciences, 67(1):112102

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.