Pith. sign in

REVIEW 4 major objections 3 minor 3 cited by

Correctness is not Faithfulness in RAG Attributions

T0 review · 4 major / 3 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A citation can be factually correct yet unfaithful: RAG models often cite documents they did not actually use to answer.

desk verdict A genuinely useful conceptual distinction between citation correctness and citation faithfulness, but the 57% headline is a loose reading of the experiment. read the letter →

arxiv 2412.18004 v1 pith:HUXG33WM submitted 2024-12-23 cs.CL

Jonas Wallat , Maria Heuss , Maarten de Rijke , Avishek Anand This is my paper · ORCID
classification cs.CL
keywords citationfaithfulnesspost-rationalizationretrieval-augmentedgenerationRAGattributionevaluationlargelanguagemodelstrustworthyAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Retrieval-augmented generation systems answer questions with citations to source documents, and standard evaluation measures whether each cited document supports the statement made. This paper argues that support is not enough: a faithful citation must also have causally influenced the model's answer, not merely match its wording. The authors define citation faithfulness in terms of causal dependence and design an adversarial test that inserts the answer's own wording into unrelated documents. In a study of a 104B-parameter RAG model, the model went on to cite those planted documents for the same statement in up to 57% of cases, suggesting post-rationalization: the model answers from memory and then finds a plausible-looking source. If this is right, correctness metrics overstate trustworthiness, and users cannot verify an answer by checking that its citations exist and support the claim.

What carries the argument

The load-bearing mechanism is the post-rationalization test, an adversarial-document experiment. The test starts from an attributed answer, extracts short statements (e.g., 'Emperor penguin'), appends those statements to documents that have no contextual support for them, and re-runs generation with the doctored documents in the context. If the model cites a doctored document for the same statement, the citation cannot reflect a causal reliance on the document's actual content, because the document only contains the statement's wording. This makes post-rationalization observable without requiring access to model internals. The formal definition of citation faithfulness—document in context, support, and causal impact—frames the test as evaluating one necessary condition of faithfulness, following the causality-based definition of faithfulness from the explainability literature.

What would settle it

Replace the planted answer snippet in the adversarial document with a factually contradicting sentence (e.g., 'The capital of Germany is Bonn') while keeping the rest of the document unchanged. If the model continues to answer 'Berlin' and continues to cite that document, the citation is not causally dependent on the cited content, confirming post-rationalization. If the answer or citation changes, the causal-use assumption of the test fails.

Watch

Extended reading notes

Core claim

The central discovery is that citation correctness and citation faithfulness are distinct properties, and current attributed answers frequently satisfy the first while failing the second. The paper defines citation faithfulness as a three-part condition: the cited document is in the retrieved context, the document supports the statement, and the statement is causally impacted by the document. It then operationalizes the causal condition through a post-rationalization test. Short statements (2–4 tokens) from a model's own answers are appended to other documents—random documents, relevant-but-uncited documents, and documents cited for other reasons—and those documents are substituted into the context. When the model cites the adversarial document for the original statement, the citation is taken to be a post-rationalization rather than genuine use. Command-R+, a 104B-parameter RAG-optimized model, cited the planted documents in 12% of random cases, 57% of relevant-but-uncited cases, and 55% of documents cited for other reasons. The authors conclude that correct-but-unfaithful citations are common, so correctness alone is insufficient for trustworthy attribution.

Load-bearing premise

The test assumes that when a document contains the exact wording of the answer but no supporting context, a citation to it must be post-rationalization rather than genuine causal use of the newly inserted content, and that inserting the document does not alter the model's internal answer-generation process; the paper notes that verifying the latter would require investigating internal states.

Editorial extensions

If this is right

  • Evaluating attribution solely by citation correctness will overstate trustworthiness; faithfulness must be measured as a separate dimension.
  • Post-hoc attribution methods that retrieve evidence after the answer is generated are unfaithful by design and should not be used where users rely on citations for verification.
  • Models trained for grounded generation also exhibit post-rationalization, so even direct-prompting RAG pipelines need faithfulness checks.
  • The adversarial-document test provides a cheap, model-agnostic way to detect one form of unfaithful behavior without probing hidden states.
  • High-stakes applications such as legal and medical question answering should treat a correct citation as a necessary but insufficient condition for trusting an answer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to test other RAG models and datasets; if the 57% figure is representative, the problem is systematic rather than specific to one model.
  • The insertion test could be strengthened into an ablation test: replace the planted statement with a contradicted one and check whether the answer and citation change; if neither changes, the citation's causal role is refuted even more directly.
  • The paper's framing connects RAG attribution to the broader chain-of-thought faithfulness literature, implying that the same post-rationalization failure likely affects other self-explanations such as step-by-step reasoning.
  • For users, the practical consequence is that verifying a citation's content is not the same as verifying the answer; interfaces should communicate the difference, or display process-level evidence when available.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper argues that citation correctness in RAG is insufficient for trustworthy attribution, and introduces the notion of citation faithfulness, defined as a causal dependence of the generated statement on the cited document (Section 4.1). The authors propose a set of desiderata and a necessary-condition test, then report an experiment on Cohere's Command-R+ over Natural Questions in which short statements from previously generated answers are appended to other documents. The model cites these adversarial documents in a large fraction of cases where the original statement is recovered, which the authors interpret as post-rationalization. They conclude that current attributed answers often lack citation faithfulness, with 'up to 57% of the citations' affected.

Significance. The conceptual contribution is useful and timely: separating whether a citation is supported by a document from whether the model actually used that document is an important distinction for trustworthy RAG. The paper is also transparent about its limitations, releases code, and proposes a concrete perturbation-based probe that others can build on. However, the headline quantitative claim is not yet solidly established: it rests on an invariance assumption the authors themselves flag, uses a conditional denominator without a reported baseline, tests only a necessary condition, and covers a single model with no statistical uncertainty. If the empirical evidence is strengthened, the paper could be a valuable agenda-setting contribution for evaluation of attribution faithfulness.

major comments (4)
  1. [Section 5, 'Experiments'; Section 6] The test treats 'the model cites a document that contains the previously generated statement' as evidence of post-rationalization, but this inference requires the invariance assumption the authors themselves flag in Section 6, namely that inserting the statement does not change where the model looks or how it generates the answer. Because the input context is changed, citing the adversarial document could be genuine context use rather than post-rationalization. A contradiction control (e.g., appending a conflicting statement and observing whether the answer changes) is needed to distinguish these possibilities; without it, the experiment cannot establish the causal dependence required by the definition in Section 4.1.
  2. [Section 5, 'Results'; Figure 3] The headline 'up to 57 percent of the citations' is computed conditionally on the cases in which the original statement was recovered in the new answer (273/476 = 57%, 290/525 = 55%), not from the full set of adversarial documents (290/829 = 35% in the 'cited for other reason' condition, or 273/702 = 39% in the 'relevant but not cited' condition). The abstract and introduction therefore overstate the finding unless the conditional denominator and the relevant baseline citation rates are reported prominently.
  3. [Section 4.1, condition (2); Section 5, 'Setup'] The experimental manipulation does not instantiate necessary condition (2), because a document containing the exact answer statement (e.g., 'Emperor penguin') is not an 'irrelevant document' in the sense of that condition; it is highly relevant by token match. There is also no control condition in which an irrelevant document without the inserted statement is presented, so the paper does not measure whether the insertion itself changes citation behavior relative to a baseline. As a result, the experiment tests only a heuristic for token-matching behavior, and it cannot by itself establish that current attributed answers 'often lack citation faithfulness'.
  4. [Section 5, 'Setup'; Section 7] All quantitative claims come from a single model (Command-R+), a single dataset (Natural Questions), and a single run, with no confidence intervals or repeated sampling; the paper's own Section 7 acknowledges the limited scope. For the empirical claim to be load-bearing, the authors should add at least one additional model family, report variance across runs, and state whether decoding was greedy or sampled.
minor comments (3)
  1. [Throughout] There are several typos that should be corrected: 'correction' for 'correctness' in Section 1, 'casual' for 'causal' in Section 4, 'Comprehnsiveness' in Table 1, and 'Emperror' in Example 1.
  2. [Figure 3] Figure 3 would be easier to interpret if the bars were labeled with the exact percentages in addition to counts, and if the denominator for each percentage were stated in the caption.
  3. [Section 5, 'Results'] The statement-recovery rate (63–70%) is important for interpreting all downstream percentages; the paper should state the matching criterion (exact token match, substring, or paraphrase) used to decide that an old statement was recovered.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports an external empirical measurement with explicit assumptions, and its self-citations are not load-bearing.

full rationale

The paper's central claim is an empirical finding about Command-R+'s citation behavior on NaturalQuestions, not a derivation that reduces to its own inputs. No parameter is fitted to a subset of data and then reported as a prediction; the reported frequencies (12%, 57%, 55%) are observed counts from an external benchmark. The definition of citation faithfulness in Section 4.1 is stipulative rather than circular: it identifies causal dependence as a necessary condition and then tests one more concrete necessary condition (adding documents to the context should not change the attribution). The inference from the test result to post-rationalization relies on the assumption that inserting a short statement into another document does not change the model's internal reasoning, and the manuscript explicitly flags this in Section 6: 'this test ... implicitly assumes that the model internals ... do not change through the insertion of additional irrelevant documents.' That is an acknowledged limitation of the operational test, not a circular step. The paper's self-citations (Wallat et al. 2023, 2024; Singh & Anand 2020; Idahl et al. 2021) appear only in the discussion of future evaluation strategies and do not carry the argument. The quantitative headline wording ('up to 57% of the citations') overstates what Figure 3 directly shows—the 57% is the share of adversarial documents cited in the 'relevant but not cited' condition among cases where the old statement was recovered—but this is a presentation issue, not circularity. Overall, the derivation chain is self-contained against external data and explicit assumptions, so there is no circularity to correct.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

The paper makes no fitted numeric parameters. Its central claim depends on a stipulative definition of faithfulness and on two behavioral assumptions in the adversarial test: that citing a document with the answer phrase indicates post-rationalization, and that inserting the phrase does not change the model's internal generation process. These assumptions, not any mathematical derivation, carry the empirical claim.

assumptions (4)
  • ad hoc to paper Citing a document that merely contains the statement, such as "Emperor penguin", in a random document indicates post-rationalization.
    Section 5, Results: "We operate under the assumption that citing documents that just randomly contain the statement (“Emperor penguin”) indicates post-rationalization." This operational assumption is needed to interpret citation shifts as unfaithful.
  • domain assumption Inserting irrelevant or adversarial documents does not change the model's internal decision process for generating the answer.
    Section 5 experiment design and Section 6 Discussion: "this test ... implicitly assumes that the model internals ... do not change through the insertion of additional irrelevant documents." The authors note this requires internal-state investigation.
  • ad hoc to paper Faithfulness should be defined as a causal dependency of the generated statement on the cited document.
    Section 4.1, Definition of Citation Faithfulness. This stipulative definition is the paper's conceptual contribution; the experiment only tests one necessary condition derived from it.
  • domain assumption Short statements of 2 to 4 tokens have no factual context and should not be cited when they appear without that context.
    Section 5, Experiments: "Since statements are, on average, 2-4 tokens, they mostly contain short concepts ... which should not be cited when appearing without factual context." This underlies the adversarial setup.
invented entities (1)
  • Post-rationalization independent evidence
    purpose: Explains correct-but-unfaithful citations where the model answers from parametric memory and then attaches supporting documents by shallow token matching.
    The paper's adversarial insertion test is a falsifiable behavioral probe: if post-rationalization occurs, injecting the answer phrase into arbitrary documents should shift citations toward those documents, which was observed for Command-R+. This is a testable prediction that can be run on other models and datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Correctness is not Faithfulness in RAG Attributions." pith.science (2026). https://pith.science/paper/HUXG33WM

@misc{pith2026241218004,
  author       = {Pith},
  title        = {Pith review of: Correctness is not Faithfulness in RAG Attributions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HUXG33WM}},
  note         = {Machine review of arXiv:2412.18004}
}
read the original abstract

Retrieving relevant context is a common approach to reduce hallucinations and enhance answer reliability. Explicitly citing source documents allows users to verify generated responses and increases trust. Prior work largely evaluates citation correctness - whether cited documents support the corresponding statements. But citation correctness alone is insufficient. To establish trust in attributed answers, we must examine both citation correctness and citation faithfulness. In this work, we first disentangle the notions of citation correctness and faithfulness, which have been applied inconsistently in previous studies. Faithfulness ensures that the model's reliance on cited documents is genuine, reflecting actual reference use rather than superficial alignment with prior beliefs, which we call post-rationalization. We design an experiment that reveals the prevalent issue of post-rationalization, which undermines reliable attribution and may result in misplaced trust. Our findings suggest that current attributed answers often lack citation faithfulness (up to 57 percent of the citations), highlighting the need to evaluate correctness and faithfulness for trustworthy attribution in language models.

Figures

Figures reproduced from arXiv: 2412.18004 by the authors.

Figure 1
Figure 1. Different answer scenarios for the query “What is the c [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Different methods of attribution generation regardi [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of the post-rationalization tests. We measu [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A provenance-constrained ledger runtime improves multimodal agent accuracy and trajectory faithfulness by binding claims to tool evidence and restricting repair to typed, non-amplifying operators.

  2. Evaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Citation-faithfulness metrics for AI science agents are verifier-dependent (3–18% on identical outputs), and a split-conformal guard provides a finite-sample catch-rate guarantee anchored on human gold.

  3. Tracing Facts or just Copies? A critical investigation of the Competitions of Mechanisms in Large Language Models

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Attention heads in GPT-2 and Pythia-6.9B that promote factual output act by general copy suppression rather than selective counterfactual suppression, with domain-dependent effects that sharpen in larger models.

Reference graph

Works this paper leans on

59 extracted references · 45 canonical work pages · cited by 3 Pith papers

  1. [1]

    Transactions of the Association for Computatio nal Linguistics 12, 775–793 (2024)

    Adlakha, V., BehnamGhader, P., Lu, X.H., Meade, N., Redd y, S.: Evaluat- ing correctness and faithfulness of instruction-followin g models for question answering. Transactions of the Association for Computatio nal Linguistics 12, 775–793 (2024)

    work page 2024

  2. [2]

    plausibility: On the (un) reliability of explanations from large language models

    Agarwal, C., Tanneru, S.H., Lakkaraju, H.: Faithfulnes s vs. plausibility: On the (un) reliability of explanations from large language models. arXiv preprint arXiv:2402.04614 (2024)

    arXiv 2024

  3. [3]

    arXiv preprint arXiv :2311.01463 (2023)

    Ahmad, M.A., Yaramis, I., Roy, T.D.: Creating trustwort hy llms: Dealing with hallucinations in healthcare ai. arXiv preprint arXiv :2311.01463 (2023)

    arXiv 2023

  4. [4]

    In: The Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 (2024)

    Asai, A., Wu, Z., Wang, Y., Sil, A., Hajishirzi, H.: Self- rag: Learning to retrieve, generate, and critique through self-reflection. In: The Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 (2024)

    work page 2024

  5. [5]

    Will you find these shortcuts?

    Bastings, J., Ebert, S., Zablotskaia, P., Sandholm, A., Filippova, K.: “Will you find these shortcuts?” A protocol for evaluating the fait hfulness of input salience methods for text classification. In: Proceedings o f the 2022 Confer- ence on Empirical Methods in Natural Language Processing, E MNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022...

    work page 2022

  6. [6]

    , Soares, L.B., Ciaramita, M., Eisenstein, J., Ganchev, K., Herzig, J., et a l.: Attributed question answering: Evaluation and modeling for attribute d large language models

    Bohnet, B., Tran, V.Q., Verga, P., Aharoni, R., Andor, D. , Soares, L.B., Ciaramita, M., Eisenstein, J., Ganchev, K., Herzig, J., et a l.: Attributed question answering: Evaluation and modeling for attribute d large language models. arXiv preprint arXiv:2212.08037 (2022)

    arXiv 2022

  7. [7]

    Camburu, O.M., Giunchiglia, E., Foerster, J., Lukasiew icz, T., Blunsom, P.: Can I trust the explainer? Verifying post-hoc explanatory m ethods (2019)

    work page 2019

  8. [8]

    Advances in Neural Information Processing Systems 31 (2018)

    Camburu, O.M., Rocktäschel, T., Lukasiewicz, T., Bluns om, P.: e-SNLI: Natural language inference with natural language explanat ions. Advances in Neural Information Processing Systems 31 (2018)

    work page 2018

Show all 59 references
  1. [9]

    arXiv preprint arXiv:2305.14908 (2023)

    Chen, A., Pasupat, P., Singh, S., Lee, H., Guu, K.: PURR: e fficiently editing language model hallucinations by denoising language model corruptions. arXiv preprint arXiv:2305.14908 (2023)

    2023 arXiv

  2. [10]

    In: Pro- ceedings of the 47th International ACM SIGIR Conference on R esearch and Development in Information Retrieval, SIGIR 2024, Washing ton DC, USA, July 14-18, 2024, pp

    Chen, C., Merullo, J., Eickhoff, C.: Axiomatic causal in terventions for re- verse engineering relevance computation in neural retriev al models. In: Pro- ceedings of the 47th International ACM SIGIR Conference on R esearch and Development in Information Retrieval, SIGIR 2024, ...

    2024

  3. [11]

    In: Forty-first Internatio nal Conference on Machine Learning (2024)

    Chen, Y., Zhong, R., Ri, N., Zhao, C., He, H., Steinhardt , J., Yu, Z., McK- eown, K.: Do models explain themselves? counterfactual sim ulatability of natural language explanations. In: Forty-first Internatio nal Conference on Machine Learning (2024)

    2024

  4. [12]

    In: Proceedings of the 47th Inter national ACM SIGIR Conference on Research and Development in Informatio n Retrieval, pp

    Cuconasu, F., Trappolini, G., Siciliano, F., Filice, S ., Campagnano, C., Maarek, Y., Tonellotto, N., Silvestri, F.: The power of nois e: Redefining Correctness is not Faithfulness in RAG Attributions 15 retrieval for rag systems. In: Proceedings of the 47th Inter national ACM ...

    2024

  5. [13]

    In: Proceedings of t he 33rd ACM International Conference on Information and Knowledge Man agement, pp

    Djeddal, H., Erbacher, P., Toukal, R., Soulier, L., Pin el-Sauvagnat, K., Ka- trenko, S., Tamine, L.: An evaluation framework for attribu ted information retrieval using large language models. In: Proceedings of t he 33rd ACM International Conference on Information and Knowled...

    2024

  6. [14]

    In: Proceedings of the 1 8th Conference of the European Chapter of the Association for Computationa l Linguistics: System Demonstrations, pp

    Es, S., James, J., Anke, L.E., Schockaert, S.: Ragas: Au tomated evaluation of retrieval augmented generation. In: Proceedings of the 1 8th Conference of the European Chapter of the Association for Computationa l Linguistics: System Demonstrations, pp. 150–158 (2024)

    2024

  7. [15]

    Formal, T., Piwowarski, B., Clinchant, S.: Match your w ords! A study of lexical matching in neural information retrieval. In: Adva nces in Informa- tion Retrieval - 44th European Conference on IR Research, EC IR 2022, Stavanger, Norway, April 10-14, 2022, Proceedings, Part II...

    2022

  8. [16]

    ar Xiv preprint arXiv:2009.03457 (2020)

    Gao, J., Peng, B., Li, C., Li, J., Shayandeh, S., Liden, L ., Shum, H.Y.: Robust conversational ai with grounded text generation. ar Xiv preprint arXiv:2009.03457 (2020)

    2020 arXiv

  9. [17]

    In: The 61s t Annual Meeting Of The Association For Computational Linguistics ( 2023)

    Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A.T., Fan, Y., Zhao, V.Y., Lao, N., Lee, H., Juan, D.C., et al.: RARR: Researching and revising what language models say, using language models. In: The 61s t Annual Meeting Of The Association For Computational Linguistics ( 2023)

    2023

  10. [18]

    In: Proceedings of the 2023 Co nference on Empirical Methods in Natural Language Processing, pp

    Gao, T., Yen, H., Yu, J., Chen, D.: Enabling large langua ge models to generate text with citations. In: Proceedings of the 2023 Co nference on Empirical Methods in Natural Language Processing, pp. 6465 –6488 (2023)

    2023

  11. [19]

    arXiv preprint arXiv:2312.10997 (2023)

    Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H.: Retrieval-augmented generation for large langua ge models: A survey. arXiv preprint arXiv:2312.10997 (2023)

    2023 arXiv

  12. [20]

    High-Level Expert Group on AI: Ethics guide- lines for trustworthy AI. Tech. rep., EU (2019), https://ec.europa.eu/digital-single-market/en/news/ ethics-guidelines-trustworthy-ai

    2019

  13. [21]

    In: Proceedin gs of the First Workshop on Trustworthy Natural Language Processing, pp

    Idahl, M., Lyu, L., Gadiraju, U., Anand, A.: Towards ben chmarking the utility of explanations for model debugging. In: Proceedin gs of the First Workshop on Trustworthy Natural Language Processing, pp. 6 8–73 (2021)

    2021

  14. [22]

    4198– 4205 (2020)

    Jacovi, A., Goldberg, Y.: Towards faithfully interpre table nlp systems: How should we define and evaluate faithfulness? In: Proceedings of the 58th Annual Meeting of the Association for Computational Lingui stics, pp. 4198– 4205 (2020)

    2020

  15. [23]

    In: Proceedings of the 2021 ACM Conference on Fairness, Acco untability, and Transparency, p

    Jacovi, A., Marasović, A., Miller, T., Goldberg, Y.: Fo rmalizing trust in artificial intelligence: Prerequisites, causes and goals o f human trust in ai. In: Proceedings of the 2021 ACM Conference on Fairness, Acco untability, and Transparency, p. 624–635, Association for Comp...

    2021

  16. [24]

    : HAGRID: A human-LLM collaborative dataset for generative informati on-seeking with attribution

    Kamalloo, E., Jafari, A., Zhang, X., Thakur, N., Lin, J. : HAGRID: A human-LLM collaborative dataset for generative informati on-seeking with attribution. arXiv preprint arXiv:2307.16883 (2023)

    2023 arXiv

  17. [25]

    Tr ansactions of the Association for Computational Linguistics 7, 453–466 (2019)

    Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M ., Parikh, A., Al- berti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al.: Natural questions: a benchmark for question answering research. Tr ansactions of the Association for Computational Linguistics 7, 453–...

    2019

  18. [26]

    arXiv prepri nt arXiv:2307.13702 (2023)

    Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., D enison, C., Her- nandez, D., Li, D., Durmus, E., Hubinger, E., Kernion, J., et al.: Measuring faithfulness in chain-of-thought reasoning. arXiv prepri nt arXiv:2307.13702 (2023)

    2023 arXiv

  19. [27]

    In : AMIA annual symposium proceedings, vol

    Lee, M., Cimino, J., Zhu, H.R., Sable, C., Shanker, V., E ly, J., Yu, H.: Be- yond information retrieval—medical question answering. In : AMIA annual symposium proceedings, vol. 2006, p. 469, American Medical Informatics Association (2006)

    2006

  20. [28]

    In: Ad- vances in Neural Information Processing Systems, vol

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhi n, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., Riedel, S., Kiela, D.: Retrieval-augmented generation for knowledge-intensive nlp tasks. In: Ad- vances in Neural Information Processing Systems, vol. ...

    2020

  21. [29]

    arXiv preprint ar Xiv:2311.03731 (2023)

    Li, D., Sun, Z., Hu, X., Liu, Z., Chen, Z., Hu, B., Wu, A., Z hang, M.: A sur- vey of large language models attribution. arXiv preprint ar Xiv:2311.03731 (2023)

    2023 arXiv

  22. [30]

    In: Proceedings of the 2016 Conference of t he North American Chapter of the Association for Computational Ling uistics: Hu- man Language Technologies, pp

    Li, J., Chen, X., Hovy, E., Jurafsky, D.: Visualizing an d understanding neu- ral models in nlp. In: Proceedings of the 2016 Conference of t he North American Chapter of the Association for Computational Ling uistics: Hu- man Language Technologies, pp. 681–691, Association for...

    2016

  23. [31]

    In: Findings of the Association for Computational Linguistics: EMNLP 2023, pp

    Liu, N.F., Zhang, T., Liang, P.: Evaluating verifiabili ty in generative search engines. In: Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 7001–7025 (2023)

    2023

  24. [32]

    Computational Linguistics pp

    Lyu, Q., Apidianaki, M., Callison-Burch, C.: Towards f aithful model expla- nation in NLP: A survey. Computational Linguistics pp. 1–67 (2024)

    2024

  25. [33]

    Lyu, Q., Havaldar, S., Stein, A., Zhang, L., Rao, D., Won g, E., Apidianaki, M., Callison-Burch, C.: Faithful chain-of-thought reasoning. In: Proceedings of the 13th International Joint Conference on Natural Langu age Processing and the 3rd Conference of the Asia-Pacific Chapte...

    2023

  26. [34]

    In: Legal Knowledge and Information Systems, pp

    Maxwell, K.T., Schafer, B.: Concept and context in lega l information re- trieval. In: Legal Knowledge and Information Systems, pp. 6 3–72, IOS Press (2008)

    2008

  27. [35]

    Mayfield, J., Yang, E., Lawrie, D.J., MacA vaney, S., McN amee, P., Oard, D.W., Soldaini, L., Soboroff, I., Weller, O., Kayi, E.S., San ders, K., Mason, M., Hibbler, N.: On the evaluation of machine-generated rep orts. In: Pro- ceedings of the 47th International ACM SIGIR Confer...

    2024

  28. [36]

    , et al.: Teaching language models to support answers with verified quotes

    Menick, J., Trebacz, M., Mikulik, V., Aslanides, J., So ng, F., Chadwick, M., Glaese, M., Young, S., Campbell-Gillingham, L., Irving, G. , et al.: Teaching language models to support answers with verified quotes. arX iv preprint arXiv:2203.11147 (2022)

    2022 arXiv

  29. [37]

    ACM Computing Surveys 55(13s), 1–42 (2023)

    Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., Van Keulen, M., Seifert, C.: From anecdota l evidence to quantitative evaluation methods: A systematic review on ev aluating ex- plainable ai. ACM Computing Surveys 55(13s), 1–42 (2023)

    2023

  30. [38]

    In: Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Volu me 1: Long Pa- pers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp

    Neeman, E., Aharoni, R., Honovich, O., Choshen, L., Szp ektor, I., Abend, O.: DisentQA: Disentangling parametric and contextual kno wledge with counterfactual question answering. In: Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Vol...

    2023

  31. [39]

    Petroni, F., Piktus, A., Fan, A., Lewis, P.S.H., Yazdan i, M., Cao, N.D., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., Plach ouras, V., Rock- täschel, T., Riedel, S.: KILT: a benchmark for knowledge int ensive lan- guage tasks. In: Proceedings of the 2021 Conference o...

    2021

  32. [40]

    arXiv preprint arXiv:2406.13663 (2024)

    Qi, J., Sarti, G., Fernández, R., Bisazza, A.: Model int ernals-based answer attribution for trustworthy retrieval-augmented generat ion. arXiv preprint arXiv:2406.13663 (2024)

    2024 arXiv

  33. [41]

    Computational Linguisti cs 49(4), 777–840 (2023)

    Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins , M., Das, D., Petrov, S., Tomar, G.S., Turc, I., Reitter, D.: Measuring at tribution in nat- ural language generation models. Computational Linguisti cs 49(4), 777–840 (2023)

    2023

  34. [42]

    In: ICML 2024 Workshop on Foundation Models in the Wild (2024)

    Roychowdhury, S., Soman, S., Ranjani, H., Gunda, N., Ch habra, V., BALA, S.K.: Evaluation of rag metrics for question answering in th e telecom do- main. In: ICML 2024 Workshop on Foundation Models in the Wild (2024)

    2024

  35. [43]

    In: Pro- ceedings of the 2024 Conference of the North American Chapte r of the Association for Computational Linguistics: Human Languag e Technologies (Volume 1: Long Papers), pp

    Saad-Falcon, J., Khattab, O., Potts, C., Zaharia, M.: A res: An automated evaluation framework for retrieval-augmented generation systems. In: Pro- ceedings of the 2024 Conference of the North American Chapte r of the Association for Computational Linguistics: Human Languag e ...

    2024

  36. [44]

    , Zaharia, M.: Col- BERTv2: Effective and efficient retrieval via lightweight lat e interaction

    Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C. , Zaharia, M.: Col- BERTv2: Effective and efficient retrieval via lightweight lat e interaction. In: Proceedings of the 2022 Conference of the North American Cha pter of the Association for Computational Linguistics: Human La...

    2022

  37. [45]

    Wallat et al

    Sen, P., Ganguly, D., Verma, M., Jones, G.J.F.: The curi ous case of IR explainability: Explaining document scores within and acr oss ranking mod- 18 J. Wallat et al. els. In: Proceedings of the 43rd International ACM SIGIR con ference on research and development in Informatio...

    2020

  38. [46]

    In: Proceedings of the 2020 Conference on Fairne ss, Account- ability, and Transparency, pp

    Singh, J., Anand, A.: Model agnostic interpretability of rankers via intent modelling. In: Proceedings of the 2020 Conference on Fairne ss, Account- ability, and Transparency, pp. 618–628 (2020)

    2020

  39. [47]

    arXiv preprint arXiv:2403.17104 (2024)

    Slobodkin, A., Hirsch, E., Cattan, A., Schuster, T., Da gan, I.: Attribute first, then generate: Locally-attributable grounded text g eneration. arXiv preprint arXiv:2403.17104 (2024)

    2024 arXiv

  40. [48]

    arXiv preprint arXiv:2201 .08239 (2022)

    Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Ku lshreshtha, A., Cheng, H.T., Jin, A., Bos, T., Baker, L., Du, Y., et al.: Lamda : Language models for dialog applications. arXiv preprint arXiv:2201 .08239 (2022)

    2022

  41. [49]

    arXiv preprint arXiv:2401.01313 (2024)

    Tonmoy, S., Zaman, S., Jain, V., Rani, A., Rawte, V., Cha dha, A., Das, A.: A comprehensive survey of hallucination mitigation tec hniques in large language models. arXiv preprint arXiv:2401.01313 (2024)

    2024 arXiv

  42. [50]

    Advances in Neural Information Processing Syst ems 36 (2024)

    Turpin, M., Michael, J., Perez, E., Bowman, S.: Languag e models don’t always say what they think: Unfaithful explanations in chai n-of-thought prompting. Advances in Neural Information Processing Syst ems 36 (2024)

    2024

  43. [51]

    In: European Conference on Information Retriev al, pp

    Wallat, J., Beringer, F., Anand, A., Anand, A.: Probing BERT for ranking abilities. In: European Conference on Information Retriev al, pp. 255–273, Springer Nature Switzerland Cham (2023)

    2023

  44. [52]

    In: Proceedings of the 33rd ACM International Conference on Inf ormation and Knowledge Management, pp

    Wallat, J., Hinrichs, H., Anand, A.: Causal probing for dual encoders. In: Proceedings of the 33rd ACM International Conference on Inf ormation and Knowledge Management, pp. 2292–2303 (2024)

    2024

  45. [53]

    arXiv preprint arXiv:2310.00935 (2023)

    Wang, Y., Feng, S., Wang, H., Shi, W., Balachandran, V., He, T., Tsvetkov, Y.: Resolving knowledge conflicts in large language models. arXiv preprint arXiv:2310.00935 (2023)

    2023 arXiv

  46. [54]

    In: Advances in Neural Information Proces sing systems, vol

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Ch i, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. In: Advances in Neural Information Proces sing systems, vol. 35, pp. 24824–24837 (2022)

    2022

  47. [55]

    according to

    Weller, O., Marone, M., Weir, N., Lawrie, D., Khashabi, D., Van Durme, B.: “according to...”: Prompting language models improves quoting from pre-training data. In: Proceedings of the 18th Conference o f the European Chapter of the Association for Computational Linguistics ( V...

    2024

  48. [56]

    In: Proceedin gs of the 58th Annual Meeting of the Association for Computational Lingui stics (ACL), Association for Computational Linguistics

    Wu, Z., Chen, Y., Kao, C., Liu, F.: Perturbed masking: Pa rameter-free probing for analyzing and interpreting bert. In: Proceedin gs of the 58th Annual Meeting of the Association for Computational Lingui stics (ACL), Association for Computational Linguistics. (2020)

    2020

  49. [57]

    arXiv preprint arXiv:2403.083 19 (2024)

    Xu, R., Qi, Z., Guo, Z., Wang, C., Wang, H., Zhang, Y., Xu, W.: Knowledge conflicts for llms: A survey. arXiv preprint arXiv:2403.083 19 (2024)

    2024

  50. [58]

    Ye, X., Sun, R., Arik, S.Ö., Pfister, T.: Effective large l anguage model adap- tation for improved grounding and citation generation. In: Proceedings of the 2024 Conference of the North American Chapter of the Asso ciation for Computational Linguistics: Human Language Technolog...

    2024

  51. [59]

    h., Kanoulas, E.: Towards fine-grained citation evaluation in generated text : A comparative analysis of faithfulness metrics

    Zhang, W., Aliannejadi, M., Yuan, Y., Pei, J., Huang, J. h., Kanoulas, E.: Towards fine-grained citation evaluation in generated text : A comparative analysis of faithfulness metrics. In: Proceedings of the 17 th International Natural Language Generation Conference, pp. 427–439 (2024 )

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.