Pith. sign in

REVIEW 3 major objections 6 minor 29 references

The paper claims that recursive turn-by-turn tracing — not a single backward pass over the full context — recovers multi-hop provenance in agentic conversations, lifting source recall from under 20% to 90%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:13 UTC pith:OEPJHI7M

load-bearing objection Useful task framing and solid engineering, but the headline numbers are partly a handshake between the annotation protocol and the method; wait for the released artifacts and human validation. the 3 major comments →

arxiv 2607.22610 v1 pith:OEPJHI7M submitted 2026-06-14 cs.AI cs.CL

Tokengeist: Multi-Turn Attribution Tracing in Agentic Conversations

Jessica Tang , Shraddha Barke , Sharad Agarwal This is my paper
classification cs.AI cs.CL
keywords multi-turn context attributionprovenance graphrecursive attributionagentic conversationstool callingprovenance collapseattribution benchmarklarge language model interpretability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces multi-turn context attribution (MTCA): given a specific span in a model's response, recover not only which earlier turns directly contributed, but the whole chain of intermediate turns through which that information propagated. It argues that existing attribution methods, which score the entire context in a single pass, systematically miss this layered structure — a failure the paper names 'provenance collapse' — and proposes Tokengeist, which applies any base attribution method recursively, walking backward from the target turn through intermediate assistant and tool-result turns until it reaches original user inputs. On a new benchmark of 3,845 target spans across 665 agentic conversations, flat methods recover sources of under 20% of multi-hop dependencies, while Tokengeist recovers over 90%. If the benchmark's annotation standard is accepted, this would mean that verifying or debugging agentic systems requires recursive attribution rather than one-shot context scoring.

Core claim

The central claim is that attribution in multi-turn conversations is inherently multi-hop: the answer to a later turn often depends on an intermediate assistant turn that itself paraphrases, filters, or transforms an earlier tool result or user instruction. Tokengeist formalizes this as building a weighted directed acyclic graph over conversation turns by recursively applying a single-turn attribution scorer, with special handling for tool-result turns (which connect deterministically to their invoking call) and exogenous user/system turns (which act as leaves). Across four open-weight instruction-tuned models and three base scorers, the recursive framework raises source recall from under 20

What carries the argument

The central object is the provenance DAG: a weighted directed acyclic graph whose nodes are conversation turns and whose root is the target span. The machinery is a backward breadth-first traversal that, at each node, concatenates all preceding turns, runs a single-turn attribution scorer to assign each prior turn a relevance score, keeps the top-k (default 3), and dispatches by role: user/system turns terminate the branch; tool-result turns follow a deterministic structural edge to their invoking assistant turn; assistant turns are recursively attributed. A relative-threshold pruning step (alpha=0.85) trims edges based on the local score maximum, and a visited set keeps the graph acyclic wh

Load-bearing premise

The load-bearing premise is that the gold provenance graphs — produced by the annotation model under a strict protocol that encodes the exact structural rule Tokengeist assumes, and human-checked on only 2.6% of targets by a single reviewer — are a valid ground truth for what actually caused each response; if that standard does not match human judgments of causal provenance, the reported gains measure agreement with the annotation protocol rather than true attribution quality

What would settle it

A human study with independent annotators, blind to the annotation protocol, constructing provenance graphs for a random sample of MTCABench targets and comparing them to the gold graphs would settle whether the reported source-recall and edge-F1 numbers reflect true causal provenance. If humans frequently do not reproduce the protocol's strict tool-result→invocation→argument-source chains and the paper's claim of over 90% source recall depends on matching that convention, the headline numbers would not transfer to human judgments.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Flat, single-pass attribution is systematically insufficient for multi-turn contexts: on MTCABench it recovers sources of under 20% of chained dependencies, while recursive tracing exceeds 90% source recall.
  • Recursion, not the choice of base scorer, is the dominant factor: the gains hold across all four models and three attribution backends evaluated, including the weakest one.
  • Reaching original source turns (rather than stopping at a nearby assistant summary) is the bar that makes provenance useful for verification, audit, and hallucination grounding, and is the bar the recursive method clears.
  • For hallucinated spans, recursive tracing markedly improves locating the turn that primed or propagated the erroneous content, helping identify where a wrong value entered the agentic trace.
  • The performance gap is concentrated in the cases MTCA targets: flat methods perform comparably on single-hop 'direct' dependencies but drop below 35% edge F1 on chained dependencies, while recursive tracing reaches 70%.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the central claim is right, then existing single-pass attribution results on long-context questions may understate what context actually drove the answer; a testable extension would be to re-run long-context benchmarks with recursive tracing and compare whether apparent 'lost in the middle' effects shrink when intermediate transformations are traced.
  • The annotation protocol's strict tool-result→invocation→argument-source convention is baked into both the gold graphs and the tracer, so the framework's effectiveness partly measures its match to that convention; a natural stress test would be conversations with asynchronous tool calls or shared mutable state, where the convention may break.
  • The recursion depth and branching factor needed in practice track the provenance-depth distribution of the conversation; one could predict that persistent-memory or multi-agent transcripts will require amortized graph updates rather than per-target backward BFS.
  • The relative-threshold pruning has a flat plateau over alpha in [0.65, 0.85], which suggests the pruning rule is not the bottleneck; a testable improvement would be to condition the threshold on dependency type, since multi-source targets are the hardest category.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces multi-turn context attribution (MTCA), a task of recovering, for a target span in an assistant turn, the full dependency DAG over prior turns. It proposes Tokengeist, a recursive framework that applies a single-turn attribution backend at each frontier turn, follows a deterministic structural edge from tool-result turns to their invocations, and terminates at exogenous user/system turns. The authors construct MTCABench from ConFETTI and TauBench with GPT-5.4-annotated gold provenance graphs, and report that recursive Tokengeist greatly outperforms flat attribution on edge F1 and source recall across four open-weight models and three backends. The central empirical claim is that flat attribution achieves under 20% source recall while Tokengeist reaches about 90%.

Significance. If the empirical claims hold, the paper makes a useful contribution: it identifies a real phenomenon (single-pass attribution missing multi-hop provenance), offers a simple method-agnostic recursive procedure with a clean comparison design (flat is literally d_max=1 in the same tracer), and includes ablations supporting the default hyperparameters. The task definition and benchmark, once released with human-validated gold graphs, could be valuable to the community. However, the current evaluation is not yet convincing as external evidence: the gold annotation protocol hardcodes the same tool-result-to-invocation chain that Tokengeist implements by fiat, the gold standard is produced by a single model with a 2.6% single-author spot-check, and the headline 'under 20%' claim is contradicted by the paper's own tables for some backends. The core idea is defensible, but the quantitative claims need substantial re-grounding.

major comments (3)
  1. [§4.1, §5.2, Appendix C.2 (Listing 2), Table 1] The evaluation has a circularity problem that bears directly on the headline 20%-to-90% source-recall gap. The annotation protocol (Listing 2) instructs GPT-5.4 that 'The correct structure is a strict chain: tool_result --> invocation --> [arg sources]' and forbids connecting a tool result directly to argument sources. Tokengeist, in §4.1, hardcodes exactly this rule as a deterministic structural edge (score 1.0) from a tool-result turn to the nearest preceding invocation. Since Table 1 shows 96.1%/96.2% of targets in ConFETTI/TauBench have tool-call paths, a large part of the recovered structure is guaranteed by construction rather than discovered by attribution. The flat baselines do not use this shortcut, so the reported gap partly measures protocol adherence, not genuine provenance recovery. The Limitations section itself concedes that 'our evaluation therefore measures agreement wit
  2. [Abstract and §7 Conclusion vs Table 2 and Table 3] The abstract and conclusion claim that flat attribution methods 'achiev[e] under 20% source recall' and that Tokengeist raises it 'from under 20% to over 90%.' This is not supported by the paper's own tables. On TauBench, Flat AttnTrace achieves source recall of 28.3% for Llama-3.1-8B and Qwen2.5-7B (Table 2) and 22.7% for Phi-4-Mini (Table 3); on ConFETTI, Flat AttnTrace reaches 54.8% and 55.0%. Even restricting to attention and AT2 backends, the 'under 20%' statement holds only for TauBench, not ConFETTI (where flat source recall is 28-46%). The claim should be rephrased to reflect the actual range, e.g., 'as low as under 20% on the harder tool-call split,' or the claim should be dropped from the abstract. This is not a cosmetic issue, since the magnitude of the improvement is the paper's main takeaway.
  3. [§5.2 and Appendix C.5] The gold-standard validation is too thin to support the quantitative conclusions. Only 100 of 3,845 targets (2.6%) were reviewed, by a single author, using a subjective three-level scale; 11% had 'minor issues' and 2% were 'wrong' before relabeling. The paper states that independent multi-annotator review is planned prior to release, and the Limitations note that no inter-annotator agreement is currently available. Because the gold graphs are produced by GPT-5.4 alone, and because the protocol embeds the exact structural rule the method uses, the current numbers could shift substantially under independent human annotation. At minimum, the paper should either (a) perform and report multi-annotator agreement on a larger stratified sample, or (b) explicitly limit all quantitative claims to 'agreement with the model-assisted annotation standard' throughout, including the abstract and conclus
minor comments (6)
  1. [Table 2] The table header says 'ConFETTI(n=495)' while Section 5 and Table 1 report 688 ConFETTI targets. The discrepancy is explained only in Appendix A.4 (30 held-out conversations for alpha selection). Please state this in the main text near Table 2.
  2. [§5.2] The annotation pipeline uses GPT-5.4 to select targets and then GPT-5.4 to annotate provenance. It would be useful to report the agreement of target selection against an alternative selector or at least to note the potential selection bias.
  3. [§6.2] The definition of 'leaf source' as an exogenous node excludes tool-result turns, but tool-result turns are not exogenous per Section 3. Please clarify whether tool results ever count as leaf sources, since the structural edge makes them non-leaf by construction.
  4. [Appendix A.3] The attention extraction re-projects queries and keys from cached hidden states. Please report whether the resulting attention weights match the true trained attention (e.g., due to numerical precision or rotary-embedding reapplication) on a small validation set.
  5. [General] The name 'MTCABench' is written as 'MTCABench' in the title and 'MTCABENCH' elsewhere; unify.
  6. [Appendix C.2, Listing 2] The prompt says a dependency must be 'causally necessary' but then also instructs a strict structural chain. These two criteria can conflict (a tool result may be causally necessary without the invocation being causally necessary). It would be helpful to note how the annotator resolves such cases.

Circularity Check

1 steps flagged

Headline source-recall gap is partly a handshake: Tokengeist emits by fiat the exact tool_result→invocation edge that the gold annotation protocol enforces, while the flat baseline is definitionally limited to depth 1.

specific steps
  1. self definitional [Section 4.1 (deterministic structural edge) vs. Appendix C.2, Listing 2 (annotation protocol); Table 1 (tool-call path prevalence)]
    "Tool-result turn carries no model-generated attention weights. TOKENGEIST instead follows a deterministic structural edge (score 1.0) to the nearest preceding assistant turn that issued the invocation, then resumes recursive attribution from that invoking turn. ... The correct structure is a strict chain: tool_result --> invocation --> [arg sources]. Never connect a tool result directly to arg sources."

    The gold provenance DAGs are produced by GPT-5.4 under a protocol that hard-codes tool_result→invocation as the only allowed dependency of a tool result. Tokengeist hard-codes the same rule as a structural edge with score 1.0 applied to every tool-result turn it reaches. With 96.1% of ConFETTI and 96.2% of TauBench targets on tool-call paths (Table 1), a large fraction of recovered edges are guaranteed by the annotation rule rather than discovered from model behavior. The flat baseline has no such structural shortcut and cannot produce edges beyond depth 1, so the under-20%-to-over-90% source-recall gap is substantially a handshake between benchmark construction and method, not an independent measurement of causal provenance. The paper's own Limitations confirms: 'our evaluation therefore

full rationale

Tokengeist itself is a real recursive framework, and the base attribution calls do genuine work when selecting assistant/user turns and attributing invocation arguments. The circularity is concentrated in the evaluation design: the gold annotation protocol (Appendix C.2) enforces, as a categorical rule, exactly the tool_result→invocation→arg-sources chain that Tokengeist implements as a deterministic structural edge with weight 1.0 (Section 4.1). On a benchmark where 96% of targets involve tool-call paths and ~88–97% of targets are chained (depth≥2), this guarantees recovery of many edges by construction. The flat baseline is defined as dmax=1, so it cannot emit any deeper edges at all; the headline claim that flat methods 'fail to recover multi-hop dependencies' is therefore partly true by definition and partly by structural handshake. The paper explicitly concedes the gold standard is model-assisted and that ground-truth provenance is a constructed annotation. Human validation of only 100/3,845 targets (2.6%) by a single author is too thin to break the loop. There is no meaningful self-citation circularity; the central issue is benchmark/method co-definition, warranting a partial circularity score of 6.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The framework imports a structural tool-chain rule from the annotation standard, relies on model-generated gold graphs, and assumes base attribution scores transfer between different turns. No new physical or mathematical entities are introduced; MTCA and MTCABench are tasks/benchmarks, not entities.

free parameters (4)
  • branching factor k = 3
    Default top-k selected turns per recursion step; chosen by hand; ablation shows saturation at k=2–3.
  • max recursion depth dmax = 8
    Default depth limit; ablation on TauBench shows saturation around dmax=8.
  • relative pruning threshold α = 0.85
    Selected on a held-out 30-conversation ConFETTI split; ablation reports a flat plateau α∈[0.65,0.85].
  • score threshold θ = 0
    No pre-filtering of low-score candidates before top-k; chosen as default.
axioms (4)
  • ad hoc to paper Tool-result provenance follows a strict chain: tool result depends only on its invocation turn, and invocation depends only on argument-source turns.
    This rule is encoded both in the gold annotation protocol (Appendix C.2, Listing 2) and in Tokengeist's structural edge (Section 4.1), so source-recall gains partly reflect agreement with this assumed structure.
  • domain assumption Gold provenance DAGs produced by GPT-5.4 reflect true causal provenance.
    All 3,845 gold DAGs are model-annotated; human spot-check covers 100 targets (2.6%) by one author with no inter-annotator agreement yet (Limitations, Appendix C.5).
  • domain assumption Attribution scores of a base method on a prior assistant turn indicate that turn's dependence on even earlier context.
    Tokengeist recurses into assistant turns using the same base attributor; if single-turn attribution does not transfer to intermediate summaries, recursive paths fail.
  • domain assumption Conversation provenance can be represented as a DAG over turns with edges from later to earlier turns.
    Problem formulation in Section 3.2 assumes acyclic, turn-level dependencies; this rules out cyclic influences and treats turns as the atomic semantic unit.

pith-pipeline@v1.3.0-alltime-deepseek · 25283 in / 12678 out tokens · 132908 ms · 2026-08-02T11:13:15.135260+00:00 · methodology

0 comments
read the original abstract

When a language model produces a response in a multi-turn conversation, which tokens from prior turns shaped that answer, and how did those dependencies propagate across prior turns? Existing context attribution methods process the full context in a single pass, recovering surface-level dependencies but missing the layered, non-linear structure of real-world dialogues and multi-step reasoning tasks. We introduce multi-turn context attribution (MTCA): given a target span in a model response, the task of tracing attribution backward across turns to identify not only which prior turns were directly relevant, but also how those turns themselves depended on earlier context. We propose Tokengeist, an attribution-method-agnostic and scalable framework that recovers full dependency paths by casting attribution as a recursive traversal of a directed acyclic graph (DAG) over conversation turns. We will release MTCABench, a benchmark of 3,845 target spans across 665 multi-turn conversations, annotated with gold provenance graphs reaching depths of up to 14, across four dependency types. Across four open-weight models, flat attribution methods fail to recover multi-hop dependencies, achieving under 20% source recall, while Tokengeist reaches 90%. Our results reveal systematic failure modes of single-pass attribution -- which we term provenance collapse -- and motivate attribution methods that reason recursively across turns.

Figures

Figures reproduced from arXiv: 2607.22610 by Jessica Tang, Sharad Agarwal, Shraddha Barke.

Figure 1
Figure 1. Figure 1: Recursive provenance tracing example. Right-hand arrows (purple) show TOKENGEIST’s re￾cursive trace, stopping at user-authored turns (T23, T3). Left-hand arrows (gray) show the flat attribution trace. Motivating example [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of TOKENGEIST. Starting from a target span y¯ in a multi-turn conversation, a base attributor A is applied recursively to select the top-k relevant prior turns at each step. Role-aware dispatch determines what happens next: user / system turns become exogenous leaves, tool-result turns follow a deterministic structural edge (dashed) to their invoking tool-call turn, and assistant turns are recurse… view at source ↗
Figure 3
Figure 3. Figure 3: Dependency types in MTCABENCH. Dashed edges omit intermediate hops for clarity. Lifestyle & Local, Telecom & Social, HR Op￾erations, Banking, and Productivity. We cluster all-MiniLM-L6-v2 embeddings with k-means and select k by silhouette score. TauBench covers two domains, airline and retail; airline overlaps with ConFETTI’s travel clusters. Together, MT￾CABENCH covers 11 distinct real-world task fami￾lie… view at source ↗
Figure 4
Figure 4. Figure 4: Edge F1 by dependency type on ConFETTI (left) and TauBench (right). Light bars: flat methods; dark bars: TG (= TOKENGEIST). 6.5 Ablations Appendix B reports ablations over recursion depth dmax, branching factor k, pruning threshold α, and source granularity, supporting the defaults (dmax=8, k=3, α=0.85, sentence-level segmen￾tation) used throughout the main results. 7 Conclusion We introduced multi-turn co… view at source ↗
Figure 5
Figure 5. Figure 5: Recursion depth ablation (Qwen2.5-7B-Instruct; ConFETTI, left; TauBench, right). Edge F1 as a function of maximum provenance depth dmax, measured in DAG hops (including structural tool-result → invocation edges). ConFETTI saturates at dmax=4 (average gold depth 3.4); TauBench saturates at dmax=8 (average gold depth 6.0). Plateau depth tracks the gold depth distribution on each dataset [PITH_FULL_IMAGE:fig… view at source ↗
Figure 6
Figure 6. Figure 6: Pruning threshold ablation (Qwen2.5-7B￾Instruct, held-out 30-conversation ConFETTI split). Edge F1 as a function of the relative-threshold extraction parameter α (Equation 3), which controls how aggres￾sively low-scoring edges are pruned from the predicted provenance graph before evaluation. Dashed line marks the chosen value α = 0.85 with ∆F1 < 0.01 across α ∈ [0.65, 0.85]. ATTNTRACE shows a slightly larg… view at source ↗
Figure 7
Figure 7. Figure 7: Branching factor ablation (Qwen2.5-7B￾Instruct, ConFETTI, α=0.85 relative-threshold pre￾filtering). Edge F1 as a function of per-step branching factor k ∈ {1, 2, 3}. Results saturate at k=2; k=3 adds negligible further gain (∆F1<0.001). B.4 Attribution Signal vs. Provenance Depth We measure the signal coverage of gold nodes (the fraction receiving any non-zero attribution score) as 16 [PITH_FULL_IMAGE:fig… view at source ↗
Figure 8
Figure 8. Figure 8: Signal coverage of gold nodes vs. provenance depth (left) and turn distance (right) (TauBench, Qwen-2.5-7B, n=26,478 gold-node records). Signal coverage = fraction of gold nodes whose turn receives any non-zero attribution score. Flat methods (dashed lines) collapse beyond depth 1 / distance 2; recursive TOKENGEIST variants (solid lines) sustain signal coverage above 75% at all depths and distances tested.… view at source ↗
Figure 9
Figure 9. Figure 9: Source granularity ablation (Llama-3.1-8B-Instruct, 10-conversation subset). Edge F1 (top) and source recall (bottom) for sentence- vs. word-level segmentation across all backends on ConFETTI (left) and TauBench (right). Sentence is better for AVG. ATTENTION and AT2; word is better for ATTNTRACE due to its bootstrap segment-sampling design, but at much higher runtime cost. plained, not only its sources), a… view at source ↗
Figure 10
Figure 10. Figure 10: Dependency-type tag distribution in MT￾CABENCH. Bars show the percentage of targets in each split carrying each tag. were error-free; direct and hallucination each con￾tributed one failure. The hallucination bucket also had the highest rate of minor issues, reflecting the inherent ambiguity of annotating spans that the model should have attended to. Bucket n Pre Post chained 27 100% 100% chained+multi_sou… view at source ↗
Figure 12
Figure 12. Figure 12: Hallucination example from ConFETTI. The gold provenance trace for the target span “I have successfully cancelled your flight ticket” (Turn 9) is shown with arrows indicating directed dependencies. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Tokengeist procedure on a five-turn refund-tool example. Stage 1 lists the inputs (the conversation, the target span y¯, and the base attributor A). Stage 2 unrolls three recursion steps and shows the per-step substeps (concatenate, score, map to turns, top-k) once, in a legend box that applies to every step. Stage 3 shows the constructed provenance DAG and the final trace after relative-threshold extract… view at source ↗
Figure 14
Figure 14. Figure 14: Provenance collapse under flat attribution and recursive recovery. Top: Single-pass (flat) attribution computes attention from the target output span over the entire conversation in a single step. Attention degrades for middle turns (Liu et al., 2024), causing intermediate sources (blue) to be missed or underweighted. The resulting provenance graph has depth 1 and loses the multi-hop structure. Bottom: Re… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 5 canonical work pages

  1. [1]

    Umut Acar, Peter Buneman, James Cheney, Jan Van den Bussche, Natalia Kwasnikowska, and Stijn Vansummeren. 2010. A graph model of data and workflow provenance. In 2nd Workshop on the Theory and Practice of Provenance (TaPP '10)

    2010

  2. [2]

    Sharad Agarwal, Pooria Namyar, Alec Wolman, Rahul Ambavat, Ankur Gupta, and Qizheng Zhang. 2026. https://arxiv.org/abs/2605.07112 Switchcraft: Ai model router for agentic tool calling . arXiv preprint arXiv:2605.07112

    Pith/arXiv arXiv 2026

  3. [3]

    Tamer Alkhouli, Katerina Margatina, James Gung, Raphael Shu, Claudia Zaghi, Monica Sunkara, and Yi Zhang. 2025. https://doi.org/10.18653/v1/2025.acl-long.394 CONFETTI : Conversational function-calling evaluation through turn-level interactions . In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape...

    doi:10.18653/v1/2025.acl-long.394 2025

  4. [4]

    Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, and Wanli Ouyang. 2024. https://doi.org/10.18653/v1/2024.acl-long.401 MT -bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues . In Proceedings of the 62nd Annual Meeting of the Association for Com...

    doi:10.18653/v1/2024.acl-long.401 2024

  5. [5]

    Shuyang Cao and Lu Wang. 2024. https://doi.org/10.18653/v1/2024.findings-acl.920 Verifiable generation with subsentence-level fine-grained citations . In Findings of the Association for Computational Linguistics: ACL 2024, pages 15584--15596, Bangkok, Thailand. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-acl.920 2024

  6. [6]

    Benjamin Cohen-Wang, Yung-Sung Chuang, and Aleksander Madry. 2025. https://arxiv.org/abs/2504.13752 Learning to attribute with attention . arXiv preprint arXiv:2504.13752

    Pith/arXiv arXiv 2025

  7. [7]

    Benjamin Cohen-Wang, Harshay Shah, Kristian Georgiev, and Aleksander Madry. 2024. https://proceedings.neurips.cc/paper_files/paper/2024/hash/adbea136219b64db96a9941e4249a857-Abstract-Conference.html Context C ite: Attributing model generation to context . In Advances in Neural Information Processing Systems, volume 37, pages 95764--95807

    2024

  8. [8]

    Jeanne Ferrante, Karl J Ottenstein, and Joe D Warren. 1987. https://doi.org/10.1145/24039.24041 The program dependence graph and its use in optimization . ACM Transactions on Programming Languages and Systems (TOPLAS), 9(3):319--349

    arXiv 1987

  9. [9]

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.398 Enabling large language models to generate text with citations . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6465--6488, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.398 2023

  10. [10]

    Eran Hirsch, Aviv Slobodkin, David Wan, Elias Stengel-Eskin, Mohit Bansal, and Ido Dagan. 2025. https://doi.org/10.18653/v1/2025.acl-long.746 LAQuer : Localized attribution queries in content-grounded generation . In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15355--15370, Vienna,...

    work page doi:10.18653/v1/2025.acl-long.746 2025

  11. [11]

    Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, and Kam-Fai Wong. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1124 MT -eval: A multi-turn capabilities evaluation benchmark for large language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages...

    doi:10.18653/v1/2024.emnlp-main.1124 2024

  12. [12]

    Philippe Laban, Alexander Fabbri, Caiming Xiong, and Chien-Sheng Wu. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.552 Summary of a haystack: A challenge to long-context LLM s and RAG systems . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9885--9903, Miami, Florida, USA. Association for Computational...

    doi:10.18653/v1/2024.emnlp-main.552 2024

  13. [13]

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2026. https://openreview.net/forum?id=VKGTGGcwl6 LLM s get lost in multi-turn conversation . In The Fourteenth International Conference on Learning Representations

    2026

  14. [14]

    Xinze Li, Yixin Cao, Liangming Pan, Yubo Ma, and Aixin Sun. 2024 a . https://doi.org/10.18653/v1/2024.findings-acl.28 Towards verifiable generation: A benchmark for knowledge-aware language model attribution . In Findings of the Association for Computational Linguistics: ACL 2024, pages 493--516, Bangkok, Thailand. Association for Computational Linguistics

    work page doi:10.18653/v1/2024.findings-acl.28 2024

  15. [15]

    Yifei Li, Xiang Yue, Zeyi Liao, and Huan Sun. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.886 AttributionBench : How hard is automatic attribution evaluation? In Findings of the Association for Computational Linguistics: ACL 2024, pages 14919--14935, Bangkok, Thailand. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-acl.886 2024

  16. [16]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. https://doi.org/10.1162/tacl_a_00638 Lost in the middle: How language models use long contexts . Transactions of the Association for Computational Linguistics, 12:157--173

    doi:10.1162/tacl_a_00638 2024

  17. [17]

    Scott M Lundberg and Su-In Lee. 2017. https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html A unified approach to interpreting model predictions . In Advances in Neural Information Processing Systems, volume 30

    2017

  18. [18]

    Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E Gonzalez. 2025. https://proceedings.mlr.press/v267/patil25a.html The B erkeley function calling leaderboard ( BFCL ): From tool use to agentic evaluation of large language models . In Proceedings of the 42nd International Conference on Machine Learning,...

    2025

  19. [19]

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. https://doi.org/10.1145/2939672.2939778 `` W hy should I trust you?'' E xplaining the predictions of any classifier . In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135--1144. ACM

    arXiv 2016

  20. [20]

    Rossi, Vivek Gupta, and Dinesh Manocha

    Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, Vivek Gupta, and Dinesh Manocha. 2025. https://doi.org/10.18653/v1/2025.emnlp-main.1144 Follow the flow: Fine-grained flowchart attribution with neurosymbolic agents . In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 22485--22508, Suz...

    work page doi:10.18653/v1/2025.emnlp-main.1144 2025

  21. [21]

    Marzieh Tahaei, Aref Jafari, Ahmad Rashid, David Alfonso-Hermelo, Khalil Bibi, Yimeng Wu, Ali Ghodsi, Boxing Chen, and Mehdi Rezagholizadeh. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.277 Efficient citer: Tuning large language models for enhanced answer quality and verification . In Findings of the Association for Computational Linguistics: NAA...

    work page doi:10.18653/v1/2024.findings-naacl.277 2024

  22. [22]

    Hexuan Wang, Jingyu Zhang, Benjamin Van Durme, and Daniel Khashabi. 2026. https://arxiv.org/abs/2604.01432 Are finer citations always better? rethinking granularity for attributed generation . arXiv preprint arXiv:2604.01432

    Pith/arXiv arXiv 2026

  23. [23]

    Yanting Wang, Runpeng Geng, Ying Chen, and Jinyuan Jia. 2025 a . https://arxiv.org/abs/2508.03793 AttnTrace : Contextual attribution of prompt injection and knowledge corruption . arXiv preprint arXiv:2508.03793

    Pith/arXiv arXiv 2025

  24. [24]

    Yanting Wang, Wei Zou, Runpeng Geng, and Jinyuan Jia. 2025 b . https://www.usenix.org/conference/usenixsecurity25/presentation/wang-yanting TracLLM : A generic framework for attributing long context LLM s . In 34th USENIX Security Symposium (USENIX Security 25), pages 3845--3864. USENIX Association

    2025

  25. [25]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. https://openreview.net/forum?id=NG7sS51zVF Efficient streaming language models with attention sinks . In The Twelfth International Conference on Learning Representations

    2024

  26. [26]

    Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2025. https://openreview.net/forum?id=roNSXZpUDN -bench: A benchmark for tool-agent-user interaction in real-world domains . In The Thirteenth International Conference on Learning Representations

    2025

  27. [27]

    Kayo Yin and Graham Neubig. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.14 Interpreting language models with contrastive explanations . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 184--198

    doi:10.18653/v1/2022.emnlp-main.14 2022

  28. [28]

    Yue Yu, Ting Bai, Hengzhi Lan, Li Qian, Li Peng, Jie Wu, Wei Liu, Jian Luan, and Chuan Shi. 2026. https://doi.org/10.1145/3773966.3777933 C 2 - C ite: Contextual-aware citation generation for attributed large language models . In Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining, pages 871--880

    arXiv 2026

  29. [29]

    Zhenyu Zhang, Lei Shen, Yuming Zhao, Meng Chen, and Xiaodong He. 2023. https://doi.org/10.18653/v1/2023.acl-long.564 Dialog-post: Multi-level self-supervised objectives and hierarchical model for dialogue post-training . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10134--10148, ...

    work page doi:10.18653/v1/2023.acl-long.564 2023