Pith. sign in

REVIEW 3 major objections 6 minor 31 references

Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that LLMs can be aligned to cite internal parametric knowledge as transparently as external documents, with calibrated confidence scores, and that its INTRALIGN method outperforms existing citation-generation baselines…

desk verdict Useful new task and method, but the internal/external distinction is unenforced, so the transparency headline overreaches. read the letter →

arxiv 2504.14856 v1 pith:TSUDGEEJ submitted 2025-04-21 cs.CL

Jiajun Shen , Tong Zhou , Yubo Chen , Delai Qiu , Shengping Liu , Kang Liu , Jun Zhao This is my paper · ORCID
classification cs.CL
keywords citationgenerationretrieval-augmentedparametricknowledgeinternalcitationsconfidencecalibrationtrustworthinessLLMalignmentinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that current citation-generation systems treat all cited information as coming from retrieved documents, leaving the model's use of its own memorized knowledge opaque. It proposes a new task, Context-Prior Augmented Citation Generation, in which the model must cite either an external span or an internally generated reference paired with a confidence score, and it introduces INTRALIGN, a data-generation and alignment method that teaches open-source LLMs to do this faithfully. The reported experiments indicate that INTRALIGN improves answer accuracy, citation recall, reference convincingness and conciseness, and confidence calibration relative to six baselines. If the claim holds, LLM answers could transparently separate “this comes from the documents” from “this comes from what I know,” with a number the user can check.

What carries the argument

The load-bearing mechanism is the RAEL (Rational Attribution and Elaboration) generation paradigm combined with INTRALIGN's token-weighted alignment loss. RAEL forces the model to first write a context review and a parameter-knowledge scrutiny, then produce extractive external references and recited internal references with confidence scores, making knowledge use explicit in the output stream. The alignment loss assigns type-dependent weights so that reference tokens, confidence tokens, and citation markers receive more gradient signal than ordinary answer tokens. The paper also uses a self-consistency-inspired golden confidence formula — the fraction of sampled documents that entail the golden answer — to supervise confidence scores, and it computes ECE against FACTSCORE to measure calibration.

What would settle it

Take a test question where the provided documents contain the full answer and the model nevertheless emits an internal citation. If that internal reference is a near-verbatim substring of a provided document (measured, say, by longest-common-subsequence overlap above 80 percent), the claim that internal citations transparentize parametric knowledge would be falsified for that model. A systematic version: measure internal-citation overlap with the provided documents across the GT, PK set and check whether confidence scores still calibrate after removing all such overlapping cases.

Watch

Extended reading notes

Core claim

The central claim is that an LLM can be aligned to cite parameter knowledge as transparently as it cites external evidence, and that the resulting internal citations carry confidence scores that track their factual accuracy. The paper defines the Context-Prior Augmented Citation Generation task, where each cited segment is either an extractive span from the provided documents or a model-generated reference paired with a probability, and a refusal is expected when neither source answers the question. INTRALIGN builds a training set by sampling documents and direct answers from the target model itself, checking them with an NLI model, and generating RAEL-style gold responses with GPT-4o; the alignment loss then up-weights reference, confidence, and citation-marker tokens. Across Llama-3.1-8B, Llama-3.1-70B, and GPT-4o, the method reports higher accuracy, higher internal citation recall, and lower ECE than the baselines, with ablations attributing the gains to the RAEL paradigm and the weighted loss.

Load-bearing premise

The method assumes that when the model writes an “internal” reference it is genuinely drawing on parametric knowledge rather than quietly rewriting the documents it was just shown; the paper's own Section 5.4.3 shows this happens, and it does not penalize it.

Editorial extensions

If this is right

  • If INTRALIGN works as claimed, LLM answers can distinguish “stated in the retrieved documents” from “stated from the model's own knowledge” in the citation markup itself.
  • Confidence scores on internal citations become a checkable quantity: users can spot low-confidence parametric claims instead of treating all claims equally.
  • The task's refusal behavior gives a principled way to abstain when both retrieval and parametric knowledge fail, reducing hallucination in low-retrieval scenarios.
  • The paper's finding that internal citation recall is generally higher than external recall suggests models are more faithful when reciting their own knowledge than when quoting retrieved text, which bears on how RAG systems should mix sources.
  • The higher plagiarism rate of larger models indicates that alignment for convincingness may push models to rewrite external evidence as internal knowledge, a behavior the task currently does not penalize.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stricter operationalization of “internal reference” would require the paraphrase distance between internal references and provided documents to exceed a threshold, or would penalize verbatim overlap; the paper's own plagiarism analysis shows this distinction is currently unenforced.
  • The NLI-checked self-consistency confidence formula could generalize beyond citation: the same method might calibrate chain-of-thought claims or tool-use outputs in other retrieval-augmented settings.
  • The ECE metric as defined checks whether confidence tracks the FACTSCORE of the reference, but not whether the confidence explains why the user should trust the span; a testable extension is to compare two systems with equal ECE but different confidence distributions.
  • The open-source models benefited most from INTRALIGN, suggesting alignment may substitute for scale in citation trustworthiness; a concrete test is to run the same pipeline on a model with different pretraining data and see whether internal-citation calibration shifts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a new task, Context-Prior Augmented Citation Generation, in which LLMs must generate citations that draw on both external retrieved context and internal parametric knowledge, reporting a confidence score for each internal citation. The authors define five metrics covering answer helpfulness, citation faithfulness, and reference trustworthiness, and introduce RAEL, a generation paradigm, and INTRALIGN, an alignment pipeline that combines GPT-4o-based data generation, rejection sampling, and token-type weighted fine-tuning. Experiments on CRAG, FRAMES, and SFE across several open and closed LLMs and six baselines report consistent improvements in accuracy, citation recall, convincingness, conciseness, and internal-reference ECE. The paper also analyzes model behavior under low-quality retrieval, knowledge conflicts, and what it calls dishonest internal reference generation.

Significance. If the central claims hold, the paper makes a useful contribution to interpretable retrieval-augmented generation: it formalizes a citation task that explicitly includes parameter knowledge, proposes a practical alignment recipe, and introduces evaluation metrics for reference quality and confidence calibration. The authors include ablations, human validation of the automatic metrics, shortcut analysis, and an extended study of plagiarism behavior. However, the significance is conditional on two unresolved issues: the evaluation cannot verify that so-called internal references actually originate from parametric knowledge rather than from rewritten context, and the trustworthiness metrics rely on a GPT-4o-mini judge that is also involved in training-data curation, with only moderate agreement against three human raters. These issues bear directly on the paper's headline transparency claim.

major comments (3)
  1. [Section 2 and Section 5.4.3 (Table 2)] The task definition classifies any non-extracted reference as an internal reference, with no mechanism to ensure it comes from parametric knowledge rather than a paraphrase or rewrite of a provided document. Section 5.4.3 reports plagiarism rates between 0.054 and 0.154 (Table 2) and explicitly states that this behavior is not penalized. Consequently, the internal citation recall Rcin and the Internal Reference ECE (Section 2.1.5, Tables 1, 3, and 4) are computed over a mixture of genuine parametric knowledge and plagiarized external content. This undermines the central claim, stated in the Abstract and Introduction, that INTRALIGN transparentizes internal versus external knowledge utilization: the evaluation framework cannot enforce or measure the distinction it requires. The paper needs either a provenance check (e.g., n-gram overlap with the provided context) or a substantially more modest interpretation of the results.
  2. [Section 6 (Table 5) and Section 4.2] The Convincingness and Conciseness metrics are assigned by GPT-4o-mini, which agrees with a single human rater only moderately (PCC 0.53 and 0.66, Table 5) based on just three raters. The same model family was used during data curation to rerank and select training responses (Section 4.2), and the same evaluator is then used to score the final outputs. The reported improvements on these two trustworthiness metrics may therefore reflect alignment to the judge's preferences rather than genuine reference quality. The Introduction's claim of a 'strong correlation' between automatic metrics and human judgments is not supported by these numbers, and the paper should either obtain a larger and more diverse human evaluation or use an independent judge for final evaluation.
  3. [Section 5.4.3 (equation for PR)] The plagiarism rate PR is defined as the proportion of internal references that entail the golden answer among questions in the GT,PK set. Because the GT setting is defined (Section 3, Appendix D) by the absence of any retrieved document that entails the golden answer, an internal reference that entails the answer cannot be a verbatim subsequence of the provided documents. The measure therefore does not directly detect copying from external documents; it largely counts any correct parametric citation as plagiarism. The conclusion that INTRALIGN achieves a 'relatively lower plagiarism rate' is not established by Table 2, and the analysis of dishonest internal reference generation should instead measure textual overlap between internal references and the provided context, or otherwise disentangle genuine parametric knowledge from rewritten context.
minor comments (6)
  1. [Section 4.2] The golden confidence formula uses 'Pk' as a sum over NLI checks, which is easily confused with the confidence variable P_i; please rename it to avoid notation collision.
  2. [Table 1] INTRALIGN results are missing for GPT-4o, DeepSeek-R1, and o1-mini; please explain why the alignment method was not applied to these models, since the cross-scenario claim would be stronger with those results.
  3. [Appendix B] There are typos such as 'Convinvingness' and 'desgin'; the text should be proofread before publication.
  4. [Appendix D] The sentence 'our pipeline first retrieves top-100 passages from a chunked Wikipedia snapshot using a GTR retriever (Wang et al., 2021) and the question' is missing a word, and 'an NLI model (Honovich et al., 2022) is applied' is grammatically awkward; please revise.
  5. [Section 5.3] No statistical significance tests are reported; given the standard deviations shown in Table 1, some differences may overlap across methods, so significance testing or confidence intervals would strengthen the comparison.
  6. [General] The paper does not state whether code and data will be released; please add a reproducibility statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the paper's reported gains are measured on held-out external ground truth, and the acknowledged internal-provenance and evaluator-overlap caveats do not make any prediction equivalent to its inputs by construction.

full rationale

The derivation chain is not circular. Answer Accuracy is checked by an NLI entailment model against golden answers (Sec. 2.1.1), Citation Recall is checked by NLI entailment of cited spans (Sec. 2.1.2), and Internal Reference ECE is calibrated against FACTSCORE factuality (Sec. 2.1.5). Training uses 1K samples and evaluation uses a separate 0.5K test set (Sec. 5.1), so the reported numbers are not produced by the same data points used for filtering or reranking. The use of GPT-4o-mini both to rerank candidate training responses by Convincingness/Conciseness (Sec. 4.2) and to score test references (Sec. 5.1) is a mild evaluator-preference overlap, but the held-out split and the human-evaluation correlation (Sec. 6) prevent this from being a by-construction equivalence. The paper itself acknowledges in Sec. 5.4.3 that internal references can be rewrites of external documents and that this behavior is not penalized, and in Sec. 8 that reranking and selection may introduce bias; these are construct-validity limitations of the internal/external provenance distinction, not circular reductions, because the metrics remain well defined under the paper's operational definitions and the method comparisons do not reduce to the training inputs. The only overlapping-author citation (Jin et al., 2024) is used for background on knowledge conflicts and is not load-bearing for any predicted result. No equation, fitted parameter, or evaluation metric is equivalent by construction to the claimed outcome.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

No fitted physical parameters; the empirical parameters are k, loss weights, LoRA rank, and threshold choices. The main postulates are that LLM-as-judge, NLI, and FACTSCORE measure what the paper claims, and that sampled direct answers reveal parametric knowledge.

free parameters (2)
  • Token-type loss weights W(tau) = Not reported; constraints W(conf)=W(ref), W(mark)=W(answer), W(ref)=W(rs)+W(answer)
    Hand-chosen weights over review, reference, answer, confidence, and citation-marker tokens used in the weighted alignment loss. They steer training toward citation quality and are not optimized against a held-out benchmark.
  • Sampled outputs k for knowledge detection = 5
    The pipeline samples k documents and k direct answers from the target LLM to decide whether the model has parameter knowledge and to compute the golden confidence. This threshold affects the PK/not-PK split and training labels.
assumptions (4)
  • domain assumption GPT-4o-mini's Convincingness and Conciseness scores reflect human trust and verification cost.
    Used as automatic metrics; human validation gives PCC 0.53 and 0.66 with three raters, so this is an assumption rather than a proven equivalence.
  • domain assumption NLI entailment (TRUE model) can decide answer accuracy and citation faithfulness.
    NLI is used for accuracy, citation recall, document annotation, and golden confidence; validated for accuracy on a small human set, not for citation recall.
  • domain assumption FACTSCORE correctly determines whether an internal reference is factually correct for ECE.
    Used as ground truth for calibration; no independent validation of FACTSCORE on internal references is reported.
  • domain assumption A model has parameter knowledge if any of k=5 sampled direct answers entails the golden answer.
    This operationalization splits PK versus not-PK and determines training targets; it equates sampled answer generation with encoded knowledge.
invented entities (2)
  • Internal reference with confidence score (Rin_i; P_i)
    purpose: A new citation format in which the model recites from memory and reports a confidence estimate.
    No external verifiable handle; the confidence is a model-generated number evaluated by internal ECE, and the reference span is unverifiable to users.
  • Context-Prior Augmented Citation Generation task
    purpose: New benchmark/task requiring internal and external citations with confidence, defining evaluation scenarios.
    Benchmark constructed by merging CRAG, FRAMES, and SFE with author-defined splits; no independent test set outside the authors' pipeline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation." pith.science (2026). https://pith.science/paper/TSUDGEEJ

@misc{pith2026250414856,
  author       = {Pith},
  title        = {Pith review of: Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TSUDGEEJ}},
  note         = {Machine review of arXiv:2504.14856}
}
read the original abstract

While hallucinations of large language models could been alleviated through retrieval-augmented generation and citation generation, how the model utilizes internal knowledge is still opaque, and the trustworthiness of its generated answers remains questionable. In this work, we introduce Context-Prior Augmented Citation Generation task, requiring models to generate citations considering both external and internal knowledge while providing trustworthy references, with 5 evaluation metrics focusing on 3 aspects: answer helpfulness, citation faithfulness, and trustworthiness. We introduce RAEL, the paradigm for our task, and also design INTRALIGN, an integrated method containing customary data generation and an alignment algorithm. Our experimental results show that our method achieves a better cross-scenario performance with regard to other baselines. Our extended experiments further reveal that retrieval quality, question types, and model knowledge have considerable influence on the trustworthiness in citation generation.

Figures

Figures reproduced from arXiv: 2504.14856 by the authors.

Figure 1
Figure 1. Compared with Context-Agree Citation Gener [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of our metrics and the auto eval [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Example of different Convincingness and Conciseness Scores 2.1.5 Internal Reference ECE In our task, we ask LLMs to report a confidence score when generating a reference from parameter knowledge. To measure the faithfulness of the con￾fidence score generation, we use Expected Calibra￾tion Error (ECE) to measure the alignment between the confidence of the output reference P and its real factuality. We evaluate the co… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Overview of INTRALIGN. We first conduct multi-scenario trustworthy data sampling to incorporate parameter knowledge and generate a golden response following our RAEL paradigm. The verified high-quality data will be used for subsequent Interpretability-Focused Alignment…
Figure 5
Figure 5. Figure 5: Results on Wikipedia and Reddit dataset. We [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: Citations for questions that the Llama3.1-8B [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Webpage for human evaluations NLI Accuracy is better than Exact Match. Previous citation generation tasks use String Ex￾act Match to compute accuracy. However, it is difficult to exhaust all possible answers, and the response may still mention the golden answer even if…
Figure 11
Figure 11. Figure 11: Dataset Distribution D Datasets The datasets we used focus on various types of QA, multi-document reasoning, and internal/external knowledge fidelity, making them suitable for com￾prehensively evaluating the model’s overall perfor￾mance on our task. They are all factu…
Figure 9
Figure 9. Figure 9: Scatter plot for Convincingness Evaluation. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Scatter plot for Conciseness Evaluation. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 12
Figure 12. Figure 12: Prompt for Convincingness Evaluation [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Prompt for Conciseness Evaluation [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Prompt for Rational Attribution and Elaboration generation [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 28 canonical work pages

  1. [1]

    **Logical Consistency**: Evaluate whether the text avoids logical errors or contradictions

    work page

  2. [2]

    In Proceedings of the 2024 Conference on Empirical Methods in Natu- ral Language Processing, pages 8211–8227, Miami, Florida, USA

    Towards verifiable text generation with evolv- ing memory and self-reflection. In Proceedings of the 2024 Conference on Empirical Methods in Natu- ral Language Processing, pages 8211–8227, Miami, Florida, USA. Association for Computational Lin- guistics. Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. 2023. Recitation-augmented language mod...

    arXiv 2024

  3. [3]

    Unable to answer

    Revealing the parametric knowledge of lan- guage models: A unified framework for attribution methods. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V ol- ume 1: Long Papers) , pages 8173–8186, Bangkok, Thailand. Association for Computational Linguistics. Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan...

    arXiv 2023

  4. [4]

    **Information Density**: Consider whether the text provides sufficient relevant information to substantiate its claims.,→ Please assign a score between 1 and 5 based on the following detailed guidelines:

    work page

  5. [5]

    **Subjectivity**: Assess whether the language is objective and free from excessive bias or personal opinions.,→

    work page

  6. [6]

    **Coherence and Focus**: Determine if the arguments are well-connected and focused rather than scattered or overly parallel.,→

    work page

  7. [7]

    my knowledge,

    When using "my knowledge," you should generate a more credible and less redundant version of the knowledge, use Arabic numerals to mark the spans, and output the provided confidence in the last. ,→ ,→

    work page

  8. [8]

    - Dominated by subjective or emotional language

    **Score: 1 (Very Low Convincingness)** - Contains multiple logical errors or glaring contradictions. - Dominated by subjective or emotional language. - Arguments are highly scattered, with no clear connections between points. - Lacks sufficient information to support its claims

    work page

Show all 31 references
  1. [9]

    - Has a noticeable bias or subjective tone

    **Score: 2 (Low Convincingness)** - Contains some logical inconsistencies or weak reasoning. - Has a noticeable bias or subjective tone. - Arguments are somewhat scattered, with limited connections between points. - Provides insufficient evidence or relies on vague statements

  2. [10]

    - Language is somewhat balanced but may lean towards subjectivity

    **Score: 3 (Moderate Convincingness)** - Mostly logical with minor inconsistencies. - Language is somewhat balanced but may lean towards subjectivity. - Arguments are somewhat connected but may lack focus or clarity. - Contains adequate but not robust information density

  3. [11]

    - Language is objective and neutral

    **Score: 4 (High Convincingness)** - Logically consistent with no major errors. - Language is objective and neutral. - Arguments are mostly coherent and focused. - Provides substantial and relevant evidence for its claims

  4. [12]

    Score: 3

    **Score: 5 (Very High Convincingness)** - Completely free from logical errors or contradictions. - Language is fully objective and professional. - Arguments are tightly connected and maintain a clear focus. - Provides rich, detailed, and highly relevant information to support ...

  5. [13]

    Judge whether it positively contributes to answering the question (positive), partially contributes but feels unnecessary or tangential (neutral), or detracts from the relevance (negative). ,→ ,→

  6. [14]

    Provide a brief explanation for your judgment. After reviewing all sentences, summarize the overall Conciseness of the document and assign a score between 1 and 5, following these guidelines:,→ - **5 (Very High Conciseness):** All sentences are relevant or contribute directly ...

  7. [15]

    Lionel Messi was born on June 24, 1987, in Rosario, Argentina, and is a professional footballer

    "Lionel Messi was born on June 24, 1987, in Rosario, Argentina, and is a professional footballer.",→ - **Positive:** This sentence establishes Messi as the subject, making it clear the document is on topic.,→

    1987

  8. [16]

    His parents are Jorge Messi, a steel factory manager, and Celia Cuccittini, who worked in a magnet manufacturing workshop

    "His parents are Jorge Messi, a steel factory manager, and Celia Cuccittini, who worked in a magnet manufacturing workshop.",→ - **Negative:** This sentence delves into his family background, which feels irrelevant to the question about championships.,→

  9. [17]

    He won his first championship in 2005, leading his team to victory in the U-20 World Cup

    "He won his first championship in 2005, leading his team to victory in the U-20 World Cup." - **Positive:** This sentence is highly relevant, directly addressing Messi’s championship history.,→

    2005

  10. [18]

    His most recent championship was the 2022 FIFA World Cup, where he captained Argentina to victory

    "His most recent championship was the 2022 FIFA World Cup, where he captained Argentina to victory.",→ - **Positive:** This sentence is also highly relevant, discussing a key championship victory

    2022

  11. [19]

    Messi hopes to continue playing at a high level and achieve more milestones in his career

    "Messi hopes to continue playing at a high level and achieve more milestones in his career." - **Neutral (slightly negative):** While unrelated to his past championships, it serves as a closing summary and doesn’t significantly detract from the document.,→ **Overall Assessment...

  12. [20]

    Your answer should contain all the information in the golden answer provided (i.e., the golden answer is a subset of your full answer).,→

  13. [21]

    When multiple sources are available, cite a minimum set

    each statement in your answer should be cited properly, with marks like [1] and [2] to indicate the source of the information. When multiple sources are available, cite a minimum set. ,→ ,→

  14. [22]

    Think step by step to generate the full answer by considering the provided `Documents,` ` my knowledge>`, and the golden answer

    Your answer should be concise and contain supporting evidence from the documents provided. Think step by step to generate the full answer by considering the provided `Documents,` ` my knowledge>`, and the golden answer. Here is a guidance:,→

  15. [23]

    Analyze what kind of knowledge you need to answer the question, and try to find supporting evidence in the documents.,→

  16. [24]

    my knowledge

    Use the provided `Documents` first, and if the information is not enough, use "my knowledge" for a supplement. Scrutinize all the possible "my knowledge" and give an appropriate confidence level according to all the possible "my knowledge." ,→ ,→

  17. [25]

    my knowledge

    Only use `my knowledge ` when provided `Documents` are not sufficient. You don 't need to use "my knowledge" for comfirming the information in the provided documents or other unnecessary situations. ,→ ,→

  18. [26]

    my knowledge

    You pretend to be a Large Language Model with limited knowledge, so you can only use the given documents and "my knowledge" to generate the answer. When using "my knowledge", pretend that you are using the knowledge that you have generated yourself. When thinking about my know...

  19. [27]

    Use Roman numerals to mark the document and use Arabic numerals to mark spans

    When citing the provided documents, you should select a fine-grained span from the documents and ensure the span is credible and less redundant. Use Roman numerals to mark the document and use Arabic numerals to mark spans. Use 'Document I ' to refer to the first document, and...

  20. [28]

    Do not use Roman numerals to cite spans

    Cite spans using Arabic numerals like [1]. Do not use Roman numerals to cite spans

  21. [30]

    my knowledge

    If none of "my knowledge" is available, admit it honestly and say that it is because of your limited capabilities.,→

  22. [31]

    my knowledge

    If none of the documents and "my knowledge" is relevant to the question, you should still output the steps and an empty reference and then generate an abstention response: "I don 't have sufficient knowledge to answer the question, and there is no relevant information in the p...

  23. [2024]

    the moon is made of marshmallows

    Attention with dependency parsing aug- mentation for fine-grained attribution. Preprint, arXiv:2412.11404. Yifan Ding, Matthew Facciani, Amrit Poudel, Ellen Joyce, Salvador Aguinaga, Balaji Veeramani, San- mitra Bhattacharya, and Tim Weninger. 2025. Cita- tions and trust in ll...

    2025 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.