Pith. sign in

REVIEW 3 major objections 5 minor 26 references

Agentic Verification for Ambiguous Query Disambiguation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VERDICT unifies diversification and verification for ambiguous query disambiguation, improving grounding-aware F1 by 23%.

desk verdict Plausible method, circular evaluation: the 23% G-F1 gain is uninterpretable until the grounded gold set is redefined and the tables re-run. read the letter →

arxiv 2502.10352 v1 pith:HSE4GVWZ submitted 2025-02-14 cs.CL

Youngwon Lee , Seung-won Hwang , Ruofan Wu , Feng Yan , Danmei Xu , Moutasem Akkad , Zhewei Yao , Yuxiong He This is my paper · ORCID
classification cs.CL
keywords ambiguousquestionansweringretrieval-augmentedgenerationquerydisambiguationverificationgroundingdiversificationexecutionfeedbackASQAbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the standard 'diversify-then-verify' pipeline for ambiguous questions in retrieval-augmented generation is wasteful: it generates interpretations before looking at the corpus, retrieves for each of them, and only then prunes noise. It proposes VERDICT, which instead retrieves a high-recall universe of passages with a single relaxed query, then asks the generator to produce a disambiguated question-answer pair only for passages that can actually answer such a pair, abstaining otherwise. This unifies diversification and verification and, after a clustering step that consolidates consistent pairs, yields interpretations that are both diverse and grounded. On the ASQA benchmark, VERDICT reports an average 23% improvement in grounding-aware F1 over the strongest baseline across three backbone LLMs.

What carries the argument

The load-bearing mechanism is 'execution feedback' (Eq. 8): for each passage $p_i$ in the retrieved universe $U_q$, the LLM is asked to generate a question-answer pair $(q_i, y_i)$ that the passage can support, or abstain with 'null'. This single step enforces both relevance to the original query and answerability from the passage alone. The resulting pairs are embedded with the retriever encoder and clustered with HDBSCAN; each cluster's medoid is selected as a consolidated interpretation, which deduplicates redundant outputs and removes outliers without extra LLM calls. The relaxed-query retrieval (Eqs. 6–7) with a larger top-$k$ provides the high-recall universe that makes per-passage verification meaningful.

What would settle it

Recompute G-Recall on the ASQA development set using a gold set restricted to human-annotated interpretations (excluding all model-generated interpretations). If the paper's G-Recall drops by more than a few points under this definition, the reported 23% G-F1 improvement is largely an artifact of the self-referential gold set.

Watch

Extended reading notes

Core claim

VERDICT's central claim is that diversification and verification should be interleaved rather than sequential: a single high-recall retrieval (with a relaxed query) provides the universe of passages, and per-passage execution feedback decides which passages can ground a valid interpretation. For each passage, the generator either outputs a disambiguated question and an answer supported by that passage, or replies 'null'. Only surviving pairs are kept, and embedding-based clustering with medoid selection duplicates and filters them. The paper argues this reduces cascading errors from pseudo-interpretations, makes smaller LLMs more reliable by avoiding long-context verification, and produces explicitly citable supporting passages. The headline empirical result is a grounding-aware F1 gain of 23% on average over the best DtV baseline on ASQA.

Load-bearing premise

The grounded recall reported in the paper is only meaningful if the gold set of correct interpretations is defined without counting the model's own accepted interpretations as correct answers.

Editorial extensions

If this is right

  • A single retriever call with a relaxed query replaces per-interpretation retrieval calls, lowering query latency and cost proportionally to the number of interpretations.
  • Per-passage generation keeps each LLM call's context short, so smaller models (e.g., 8B) perform more consistently than in DtV's long-context verification.
  • Every surviving interpretation carries its supporting passage, making citation-quality metrics computable without extra annotation.
  • The clustering parameter tunes the number of output interpretations, giving practitioners a conservative or diverse mode that DtV does not offer.
  • The proposed G-precision/G-recall metrics penalize ungrounded diversity, which is desirable in enterprise RAG where unsupported answers are costly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's G-Recall definition (Eq. 22) places the model's own verified predictions into the gold set, which could inflate recall; a version using human-only gold interpretations would be a sterner test of the 23% gain.
  • The same execution-feedback design should transfer to other generation tasks where outputs must be conditionable on evidence, such as fact-checking or tool-use planning.
  • Clustering could incorporate retriever relevance scores as a prior, potentially stabilizing consolidation when the generator is weak.
  • The claimed improvement is demonstrated on one public benchmark; domain-specific enterprise corpora with evolving terminology remain an untested but motivating use case.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes VERDICT, a method for disambiguating ambiguous queries in RAG by unifying diversification with verification. Instead of the typical Diversify-then-Verify pipeline, VERDICT first retrieves a single high-recall passage universe with a relaxed query, then uses per-passage generator feedback to extract question-answer pairs that are both relevant and answerable, and finally consolidates these pairs via embedding clustering and medoid selection. The authors claim that this joint approach reduces cascading errors and improves efficiency, and they report a 23% average improvement in grounding-aware F1 over the strongest baseline on the ASQA benchmark across three backbone LLMs. The paper also introduces a grounded evaluation protocol intended to measure whether interpretations are supported by retrieved passages.

Significance. The conceptual contribution is valuable: integrating retrieval and generation feedback early in the diversity-generation process, and using clustering to consolidate noisy feedback, addresses a real weakness of post-hoc verification in RAG. If the empirical results were valid, the method would be a practical improvement for enterprise RAG systems. The paper also releases code and an evaluation framework, which aids reproducibility. However, the central quantitative claim is compromised by the circular definition of the grounded gold set used to compute G-Recall, so the headline improvement is not interpretable as reported. The underlying idea may be sound, but the evidence presented does not currently support the paper's main claim.

major comments (3)
  1. [Appendix A, Eq. (22) and §4.3, Eq. (12)] The definition of the grounded gold set in Eq. (22) is self-referential. Qbar is defined as the set of interpretations in (Qhat ∪ Qtilde) for which V(qhat, phat) = 1, and Eq. (12)/Eq. (21) then averages V over that set. For every model prediction admitted into Qbar, V is 1 by construction, so those terms contribute perfect recall. As written, G-Recall should be identically 1 for any method whose verified predictions are included in Qbar. The non-100% values in Table 2 (e.g., 54.77 for LLaMA 8B) therefore imply an unstated, more restrictive definition of Qbar, which is not documented. This makes G-Recall and the reported G-F1 values uninterpretable and seriously undermines the headline claim of a 23% average G-F1 improvement.
  2. [§5.2, Table 2] Because G-F1 is the harmonic mean of G-Precision and G-Recall, and G-Recall is computed from a self-referential gold set, the comparison in Table 2 cannot be used to conclude that VERDICT improves grounding-aware F1 over the baselines. The authors should recompute G-Recall using a model-independent gold set (e.g., only human interpretations Qtilde, or human interpretations plus verified model predictions only when those predictions also match a human interpretation) and report the resulting G-F1 values. Without such a recomputation, the central empirical claim of the paper is unsupported.
  3. [§4.1, Eq. (8) and Algorithm 1] The per-passage execution feedback in Eq. (8) uses a single LLM prompt to decide whether a passage can answer a disambiguated question, and the same type of verification is later used in the evaluation protocol (Eq. (20)). This creates a potential evaluation bias: the model's own generation and verification are performed by the same LLM family, and the judge LLM is also a backbone model. The paper should at least discuss the risk that V(qhat, phat) overestimates grounding for the model's own predictions, and ideally use a held-out judge model or human annotations for the grounded evaluation.
minor comments (5)
  1. [§4.3, Eq. (12)] The notation in Eq. (12) is ambiguous: for a gold interpretation qbar that is a human annotation, there is no predicted passage phat; the text in Appendix A says 'we replace phat with retrieved passages from Uq' but this replacement is not specified precisely. Please clarify the exact procedure for computing V(qbar, phat) for each element of Qbar.
  2. [§5.1.2] The paper says the ASQA validation split has 948 questions, but it does not report the total number of human interpretations or the average number per question in that split. Providing these statistics would help contextualize the recall and diversity numbers.
  3. [Table 4] The 'Conservative' clustering row reports only G-Precision and G-Recall but not G-F1 or the Sufficient%/recall metrics. Adding those would make it easier to assess the precision-recall trade-off when adjusting the clustering parameter.
  4. [§6] The production deployment description in Figure 4 is illustrative but does not include any quantitative evaluation. Adding even a small case study with user-visible metrics would strengthen the claim of practical impact.
  5. [Appendix A, Eq. (22)] There is a minor typographical issue: the condition in Eq. (22) is written as 'V(qhat, phat)' without a predicate such as '= 1'. Adding the explicit equality would make the definition consistent with the surrounding text.

Circularity Check

1 steps flagged · score 8.0 of 10

G-Recall's gold set is built from the model's own verified predictions (Eq. 22), so the reported 23% G-F1 gain is uninterpretable as written.

  1. self definitional [Appendix A, Eqs. 21-22]
    "To increase the recall of this proxy, we add verified model prediction V (ˆq, ˆp) = 1. ¯Q = { ˆq ∈ ( ˆQ ∪˜Q) | V (ˆq, ˆp) }. Eq. 21 shows how ¯Q complements the human interpretations ˜Q."

    Eq. 22 defines the grounded gold set ¯Q to include every model-generated interpretation that passes verification, i.e., V(ˆq, ˆp)=1. Eq. 21 then computes G-Recall as the average of V(¯q, ˆp) over that same ¯Q. For every model prediction admitted to ¯Q, V is 1 by construction, so each such element contributes a guaranteed 1 to the recall average. Literally, a model that emits many verified predictions inflates its own recall, and the metric approaches 1 as verified model predictions dominate ¯Q. The non-100 values in Table 2 (e.g., 54.77 for LLaMA 8B) therefore require an unstated restriction on ¯Q. If the intended gold set is human interpretations only, the published G-Recall numbers need re-reporting; if it is the union including verified model predictions, the metric is circular.

full rationale

The central empirical claim of the paper is the average 23% improvement in grounding-aware F1 (Abstract, Table 2). That claim depends on G-Recall (Eq. 12 / Eq. 21). Appendix A defines the gold set ¯Q in Eq. 22 as all model predictions with V(ˆq, ˆp)=1 unioned with human interpretations, and then evaluates recall by averaging V over exactly that set. The step is circular: for every model-generated element of ¯Q, the recalled value is definitionally 1, making G-Recall an upwardly biased measure that rewards a method for generating many self-verified interpretations. No external, model-independent gold standard is used for the grounded recall computation. The method itself is agentic and efficiency-oriented, and its ungrounded recall and cost comparisons are independent of this flaw, but the headline grounded-quality result is not. The correct score is therefore high, though not maximal: the derivation of VERDICT's predictions is not itself circular, but the evaluation result that supports the main claim reduces by construction to the metric definition.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central G-F1 claim depends on four domain assumptions (LLM judge reliability, clustering semantics, ASQA transferability, single-passage answerability) and two free parameters (retrieval depth k and HDBSCAN settings). The largest hidden burden is the self-referential definition of the grounded recall gold set.

free parameters (2)
  • retrieval depth k = 20
    Empirically tuned in footnote 3 to nearly match G-F1 from top-100 with GPT-4o; a hyperparameter fitted to the evaluation metric.
  • HDBSCAN clustering hyperparameters = not reported
    The consolidation step's number of clusters and outlier behavior depend on HDBSCAN settings, which are not specified; the 'conservative' variant in Table 4 implies a tunable threshold.
assumptions (4)
  • domain assumption LLM judge verification V(q,p) is a reliable binary measure of whether a passage supports a question
    Used for execution feedback (Eq. 8) and as the evaluation function (Eq. 20, prompt IV in Figure 8). No human agreement or calibration is reported.
  • domain assumption Embedding-space HDBSCAN clustering of concatenated question-answer pairs produces meaningful interpretation groups and medoids
    Algorithm 1 and Section 4.2 rely on this to deduplicate and consolidate; no validation that clusters correspond to distinct interpretations.
  • domain assumption ASQA's Wikipedia-based dev set is an adequate proxy for the enterprise RAG scenarios motivating the work
    Motivation in Section 1 and production demo in Section 6 are enterprise-specific, but all experiments use the public ASQA benchmark.
  • domain assumption A disambiguation is only valid if a single retrieved passage is self-contained enough to answer it
    Prompt IE (Figure 6) instructs the LLM to abstain (output null) unless the passage alone can answer; this definition of 'grounded' is adopted without external justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agentic Verification for Ambiguous Query Disambiguation." pith.science (2026). https://pith.science/paper/HSE4GVWZ

@misc{pith2026250210352,
  author       = {Pith},
  title        = {Pith review of: Agentic Verification for Ambiguous Query Disambiguation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HSE4GVWZ}},
  note         = {Machine review of arXiv:2502.10352}
}
read the original abstract

In this work, we tackle the challenge of disambiguating queries in retrieval-augmented generation (RAG) to diverse yet answerable interpretations. State-of-the-arts follow a Diversify-then-Verify (DtV) pipeline, where diverse interpretations are generated by an LLM, later used as search queries to retrieve supporting passages. Such a process may introduce noise in either interpretations or retrieval, particularly in enterprise settings, where LLMs -- trained on static data -- may struggle with domain-specific disambiguations. Thus, a post-hoc verification phase is introduced to prune noises. Our distinction is to unify diversification with verification by incorporating feedback from retriever and generator early on. This joint approach improves both efficiency and robustness by reducing reliance on multiple retrieval and inference steps, which are susceptible to cascading errors. We validate the efficiency and effectiveness of our method, Verified-Diversification with Consolidation (VERDICT), on the widely adopted ASQA benchmark to achieve diverse yet verifiable interpretations. Empirical results show that VERDICT improves grounding-aware F1 score by an average of 23% over the strongest baseline across different backbone LLMs.

Figures

Figures reproduced from arXiv: 2502.10352 by the authors.

Figure 1
Figure 1. Comparison of (a) DtV (Diversify-then-Verify) and (b) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the full pipeline of VERDICT: Verified diversification (Section 4.1) followed by consoli￾dation phase (Section 4.2). On the right, yellow and blue dots represent embeddings of generated interpretations and their answers, embedded together after concatenation, while yellow color denotes medoids, or representatives chosen from each cluster. Algorithm 1 Verified Diversification for ambiguous queries Inp… view at source ↗
Figure 3
Figure 3. Analysis on accuracy of generated qˆ’s and yˆ’s from VERDICT. (i, answer correctness) Models easily derive correct q, ˆ yˆ given an answerable passage. (ii, interpretation error rate) Impact of model scale is more critical in discerning unanswerable passages. do not provide such knobs to control, as deciding the number of resulting interpretations is up to the LLM [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Query clarification in Snowflake’s Cortex Agents API setup with tool access to a series of synthetically [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Comparison of end-to-end workflow of VERDICT and DtV (DIVA; In et al., 2024) for handling ambiguous question answering task. Vertical arrangement denotes sequential dependency, while calls that can run in parallel are placed at the same horizontal level. C LLM Prompts …
Figure 6
Figure 6. Figure 6: Prompt IE for obtaining interpretation qˆ and answer yˆ with execution feedback from the LLM. Evaluation Prompt for Ungrounded Precision/Recall Given a list of generated disambiguated subquestions that clarify the intent of an ambiguous question, compare them with the …
Figure 7
Figure 7. Figure 7: Prompt IM for determining matches between qˆ’s and q˜’s. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Prompt IV for determining a match between qˆ and pˆ. Evaluation Prompt for Deduplication Given a list of subquestions, which are derived disambiguations of an ambiguous query, remove nearly identical duplicates and leave only distinct ones. You should provide a list of…
Figure 9
Figure 9. Figure 9: Prompt ID for removing (near-)duplicates in a list of interpretations for an ambiguous question. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 8 canonical work pages

  1. [1]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. https://openreview.net/forum?id=hSyW5go0v8 Self-rag: Learning to retrieve, generate, and critique through self-reflection . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    2024

  2. [2]

    Chris Buckley, Gerard Salton, James Allan, and Amit Singhal. 1994. http://trec.nist.gov/pubs/trec3/papers/cornell.new.ps.gz Automatic query expansion using SMART: TREC 3 . In Proceedings of The Third Text REtrieval Conference, TREC 1994, Gaithersburg, Maryland, USA, November 2-4, 1994 , volume 500-225 of NIST Special Publication , pages 69--80. National I...

    work page 1994

  3. [3]

    Ricardo J. G. B. Campello, Davoud Moulavi, and J \" o rg Sander. 2013. https://doi.org/10.1007/978-3-642-37456-2\_14 Density-based clustering based on hierarchical density estimates . In Advances in Knowledge Discovery and Data Mining, 17th Pacific-Asia Conference, PAKDD 2013, Gold Coast, Australia, April 14-17, 2013, Proceedings, Part II , volume 7819 of...

    doi:10.1007/978-3-642-37456-2 2013

  4. [4]

    Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian - Guang Lou, and Weizhu Chen. 2023. https://openreview.net/forum?id=ktrw68Cmu9c Codet: Code generation with generated tests . In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net

    work page 2023

  5. [5]

    Jeremy Cole, Michael Zhang, Daniel Gillick, Julian Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.35 Selectively answering ambiguous questions . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 530--543, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.35 2023

  6. [6]

    Efthimiadis and Paul V

    Efthimis N. Efthimiadis and Paul V. Biron. 1993. http://trec.nist.gov/pubs/trec2/papers/ps/ucla.ps Ucla-okapi at TREC-2: query expansion experiments . In Proceedings of The Second Text REtrieval Conference, TREC 1993, Gaithersburg, Maryland, USA, August 31 - September 2, 1993 , volume 500-215 of NIST Special Publication , pages 278--290. National Institut...

    work page 1993

  7. [7]

    Evans and Robert G

    David A. Evans and Robert G. Lefferts. 1993. http://trec.nist.gov/pubs/trec2/papers/ps/clarit.ps Design and evaluation of the CLARIT-TREC-2 system . In Proceedings of The Second Text REtrieval Conference, TREC 1993, Gaithersburg, Maryland, USA, August 31 - September 2, 1993 , volume 500-215 of NIST Special Publication , pages 137--150. National Institute ...

    work page 1993

  8. [8]

    Arnold, and Bing Xiang

    Yifan Gao, Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O. Arnold, and Bing Xiang. 2021. https://doi.org/10.18653/v1/2021.acl-long.253 Answering ambiguous questions through generative evidence fusion and round-trip prediction . In Proceedings of the 59th Annual Meeting of the Associatio...

    doi:10.18653/v1/2021.acl-long.253 2021

Show all 26 references
  1. [9]

    Rossi, Md Mehrab Tanjim, Tong Yu, Ritwik Sinha, and Chanyoung Park

    Yeonjun In, Sungchul Kim, Ryan A. Rossi, Md Mehrab Tanjim, Tong Yu, Ritwik Sinha, and Chanyoung Park. 2024. https://arxiv.org/abs/2409.02361 Diversify-verify-adapt: Efficient and robust retrieval-augmented ambiguous question answering . Preprint, arXiv:2409.02361

    2024 arXiv

  2. [10]

    Gangwoo Kim, Sungdong Kim, Byeongguk Jeon, Joonsuk Park, and Jaewoo Kang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.63 Tree of clarifications: Answering ambiguous questions with retrieval-augmented large language models . In Proceedings of the 2023 Conference on Empiri...

    2023 doi

  3. [11]

    u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \" u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \" a schel, Sebastian Riedel, and Douwe Kiela. 2020. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e...

    2020

  4. [12]

    Dongfang Li, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Baotian Hu, Aiguo Wu, and Min Zhang. 2023 a . https://arxiv.org/abs/2311.03731 A survey of large language models attribution . Preprint, arXiv:2311.03731

    2023 arXiv

  5. [13]

    Xiaonan Li, Changtai Zhu, Linyang Li, Zhangyue Yin, Tianxiang Sun, and Xipeng Qiu. 2024. https://doi.org/10.18653/v1/2024.naacl-long.305 LL atrieval: LLM -verified retrieval for verifiable generation . In Proceedings of the 2024 Conference of the North American Chapter of the ...

    2024 doi

  6. [14]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023 b . Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281

    2023 arXiv

  7. [15]

    Nelson Liu, Tianyi Zhang, and Percy Liang. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.467 Evaluating verifiability in generative search engines . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7001--7025, Singapore. Association for C...

    2023 doi

  8. [16]

    Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.322 Query rewriting in retrieval-augmented large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 53...

    2023 doi

  9. [17]

    Luke Merrick, Danmei Xu, Gaurav Nuti, and Daniel Campos. 2024. https://arxiv.org/abs/2405.05374 Arctic-embed: Scalable, efficient, and accurate text embedding models . Preprint, arXiv:2405.05374

    2024 arXiv

  10. [18]

    Sewon Min, Kenton Lee, Ming-Wei Chang, Kristina Toutanova, and Hannaneh Hajishirzi. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.560 Joint passage ranking for diverse multi-answer retrieval . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language P...

    2021 doi

  11. [19]

    Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.466 A mbig QA : Answering ambiguous open-domain questions . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)...

    2020 doi

  12. [20]

    J. J. Rocchio. 1971. Relevance feedback in information retrieval. In G. Salton, editor, The Smart retrieval system - experiments in automatic document processing, pages 313--323. Englewood Cliffs, NJ: Prentice-Hall

    1971

  13. [21]

    Zhihong Shao and Minlie Huang. 2022. https://doi.org/10.18653/v1/2022.acl-long.128 Answering open-domain multi-answer questions via a recall-then-verify framework . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    2022 doi

  14. [22]

    Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.566 ASQA : Factoid questions meet long-form answers . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8273--8288, Abu ...

    2022 doi

  15. [23]

    Weiwei Sun, Hengyi Cai, Hongshen Chen, Pengjie Ren, Zhumin Chen, Maarten de Rijke, and Zhaochun Ren. 2023. https://doi.org/10.18653/v1/2023.acl-long.424 Answering ambiguous questions via iterative prompting . In Proceedings of the 61st Annual Meeting of the Association for Com...

    2023 doi

  16. [24]

    Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. https://arxiv.org/abs/2401.15884 Corrective retrieval augmented generation . Preprint, arXiv:2401.15884

    2024 arXiv

  17. [25]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  18. [26]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.