Pith. sign in

REVIEW 3 major objections 2 minor 1 cited by

Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer

T0 review · 3 major / 2 minor · reviewed 2026-05-08 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A differentiable alignment transfers node importance scores from smaller language models to larger ones by optimizing a soft faithfulness objective.

desk verdict DFA gives a workable differentiable route for aligning node scores across model sizes, but the transfer story rests on unshown details of how much the source actually constrains the solution. read the letter →

arxiv 2604.24302 v1 submitted 2026-04-27 cs.CL

Shun Shao , Binxu Wang , Shay B. Cohen , Anna Korhonen , Yonatan Belinkov This is my paper · ORCID
classification cs.CL
keywords differentiablefaithfulnessalignmentcross-modelcircuittransfermechanisticinterpretabilitylanguagemodelcircuitsnodeimportanceobjectivediscoveryscaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents Differentiable Faithfulness Alignment as a way to move circuit information between language models without running full circuit discovery on the larger target. It learns a mapping that takes importance scores from a small source model and projects them onto the target, then tunes the mapping so the transferred circuits keep the target model's behavior on tasks like factual retrieval and arithmetic. This avoids the high cost of direct methods on big models and uses the source as a prior. The evaluations show the approach works best when the source and target are close in size and architecture, such as Llama-3 1B to 3B. It beats simple baselines and sometimes matches or exceeds direct attribution in faithfulness.

What carries the argument

Differentiable Faithfulness Alignment (DFA), a learned mapping from source node importance scores to the target model that is optimized by a soft faithfulness loss to preserve task performance.

What would settle it

If DFA-aligned circuits on a new source-target pair produce substantially lower faithfulness scores or task accuracy than circuits found by direct attribution on the target, the transfer method would not succeed.

Watch

Extended reading notes

Core claim

We introduce Differentiable Faithfulness Alignment (DFA), a framework that transfers circuit information from a smaller source model to a larger target model through a learned differentiable alignment. DFA projects source-model node importance scores into the target model and trains this mapping with a soft faithfulness objective, avoiding full circuit discovery on the target model. We evaluate DFA on Llama-3 and Qwen-2.5 across six tasks spanning factual retrieval, multiple-choice reasoning, and arithmetic. The strongest results occur on Llama-3 1B to 3B, where aligned circuits are often competitive with direct node attribution and zero-shot transfer remains effective.

Load-bearing premise

Node importance scores from a smaller source model contain information that can be aligned differentiably to serve as useful mechanistic priors for a larger target without full circuit discovery on the target.

Editorial extensions

If this is right

  • On Llama-3 1B to 3B, DFA circuits are competitive with or stronger than direct node attribution in faithfulness for the tested tasks.
  • Zero-shot transfer of the learned alignment works without additional training in some settings.
  • Transfer performance drops for larger size gaps and is substantially weaker on Qwen-2.5 than on Llama-3.
  • DFA outperforms simple baselines across factual retrieval, reasoning, and arithmetic tasks.
  • In some cases the method recovers target-model circuits whose faithfulness matches or exceeds that of direct attribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method could lower the cost of circuit discovery for very large models by reusing smaller ones as starting points.
  • It suggests mechanistic priors might be chained across successive model scales as training progresses.
  • Architecture differences appear to limit transfer, pointing to a need for alignment methods that account for model family structure.
  • The soft faithfulness objective might extend to transferring other interpretability signals such as activation patterns beyond node importance.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The paper introduces Differentiable Faithfulness Alignment (DFA), a framework that transfers circuit information from a smaller source LM to a larger target LM by projecting source node importance scores into the target via a learned differentiable mapping trained against a soft faithfulness objective. This avoids full circuit discovery on the target. Evaluations on Llama-3 (1B→3B) and Qwen-2.5 models across factual retrieval, multiple-choice reasoning, and arithmetic tasks show DFA often competitive with or exceeding direct node attribution on smaller gaps, outperforming simple baselines, with zero-shot transfer effective in some cases; performance degrades for larger gaps and on Qwen-2.5.

Significance. If the transfer interpretation holds, DFA offers a scalable route to mechanistic interpretability by using smaller models as mechanistic priors for larger ones, reducing the cost of circuit discovery. The empirical results on Llama-3 pairs and the code release are concrete strengths; the observed limits with scale/architectural gaps provide useful boundary conditions for the approach.

major comments (3)
  1. The central transfer claim requires that source node scores remain load-bearing after alignment. The manuscript does not specify the parameterization or capacity of the learned mapping (e.g., whether it is a low-rank or tied function) nor any auxiliary loss that penalizes deviation from source relative importances. Without such constraints, gradient descent on the faithfulness objective alone can reassign importances to maximize the target objective independently of the source, turning DFA into target-only optimization. This must be clarified with an explicit definition of the mapping and a control experiment replacing source scores with random or uniform values.
  2. Recovery is reported as weaker for larger source-target gaps and substantially lower on Qwen-2.5. The manuscript should quantify this with per-task faithfulness scores, statistical significance tests, and an ablation showing whether the drop is due to architectural mismatch or failure of the alignment to preserve source structure (e.g., correlation between source and aligned scores before/after training).
  3. The soft faithfulness objective is central to training but its precise formulation, including any post-hoc choices in evaluation metrics or circuit extraction thresholds, is not detailed enough to assess whether the reported competitiveness with direct attribution is robust or sensitive to those choices.
minor comments (2)
  1. The abstract and introduction would benefit from a clearer statement of the exact node sets being aligned (e.g., attention heads, MLPs, or residual streams) and how the projection handles differing layer counts between source and target.
  2. Table or figure captions should explicitly state the number of runs, random seeds, and error bars for all reported faithfulness numbers to allow assessment of variability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify the transfer mechanism and improve the empirical rigor of the work. We address each major comment below and will incorporate revisions to strengthen the manuscript.

read point-by-point responses

  1. Referee: The central transfer claim requires that source node scores remain load-bearing after alignment. The manuscript does not specify the parameterization or capacity of the learned mapping (e.g., whether it is a low-rank or tied function) nor any auxiliary loss that penalizes deviation from source relative importances. Without such constraints, gradient descent on the faithfulness objective alone can reassign importances to maximize the target objective independently of the source, turning DFA into target-only optimization. This must be clarified with an explicit definition of the mapping and a control experiment replacing source scores with random or uniform values.

    Authors: We agree that an explicit definition of the mapping and a control experiment are necessary to substantiate that source scores remain load-bearing. The current manuscript describes the mapping as a learned differentiable projection of source node importance scores but does not detail its parameterization. In the revision we will add a precise definition (including capacity and any weight-tying) in the Methods section. We will also include the requested control experiment that replaces source scores with random or uniform values and reports the resulting drop in target faithfulness, thereby demonstrating that the alignment depends on source-derived information rather than independent target optimization. revision: yes

  2. Referee: Recovery is reported as weaker for larger source-target gaps and substantially lower on Qwen-2.5. The manuscript should quantify this with per-task faithfulness scores, statistical significance tests, and an ablation showing whether the drop is due to architectural mismatch or failure of the alignment to preserve source structure (e.g., correlation between source and aligned scores before/after training).

    Authors: We will expand the results section to report per-task faithfulness scores for all model pairs and tasks. Where multiple runs are available we will add statistical significance tests. We will further include an ablation that computes the Pearson correlation between source node scores and the aligned scores both before and after training; this will help distinguish whether performance degradation stems from architectural mismatch or from the alignment failing to preserve source structure. revision: yes

  3. Referee: The soft faithfulness objective is central to training but its precise formulation, including any post-hoc choices in evaluation metrics or circuit extraction thresholds, is not detailed enough to assess whether the reported competitiveness with direct attribution is robust or sensitive to those choices.

    Authors: We will provide the complete mathematical formulation of the soft faithfulness objective in the Methods section, together with the exact post-hoc choices used for evaluation metrics and any circuit extraction thresholds. This added detail will allow readers to assess robustness and will be accompanied by a brief sensitivity analysis where feasible. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; derivation relies on external faithfulness objective and empirical comparison.

full rationale

The paper defines DFA as a learned projection of source node importance scores trained via an independent soft faithfulness objective on the target model, then evaluates the resulting circuits against direct attribution baselines. No equations or steps reduce the output to the input by construction, no self-definitional mappings appear, and no load-bearing claims rest on self-citations or renamed known results. The method is presented as an optimization procedure whose success is measured externally rather than tautologically, making the derivation self-contained against the stated benchmarks.

Assumptions & free parameters 1 free parameters · 0 assumptions · 1 invented entities

The central claim rests on the existence of transferable node importance structure across models and the effectiveness of a learned differentiable mapping; no explicit free parameters, axioms, or invented entities are detailed in the abstract beyond the introduced DFA method itself.

free parameters (1)
  • alignment mapping parameters
    Parameters of the learned differentiable projection are fitted during training on the faithfulness objective.
invented entities (1)
  • Differentiable Faithfulness Alignment (DFA)
    purpose: Framework for projecting and aligning source-model node importance into target models
    New method introduced in the paper; no independent evidence provided beyond the reported experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer." pith.science (2026). https://pith.science/paper/2604.24302

@misc{pith2026260424302,
  author       = {Pith},
  title        = {Pith review of: Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2604.24302}},
  note         = {Machine review of arXiv:2604.24302}
}
abstract

Mechanistic interpretability has made it possible to localize circuits underlying specific behaviors in language models, but existing methods are expensive, model-specific, and difficult to scale to larger architectures. We introduce \textbf{Differentiable Faithfulness Alignment (DFA)}, a framework that transfers circuit information from a smaller source model to a larger target model through a learned differentiable alignment. DFA projects source-model node importance scores into the target model and trains this mapping with a soft faithfulness objective, avoiding full circuit discovery on the target model. We evaluate DFA on Llama-3 and Qwen-2.5 across six tasks spanning factual retrieval, multiple-choice reasoning, and arithmetic. The strongest results occur on Llama-3 $1$B$\rightarrow3$B, where aligned circuits are often competitive with direct node attribution and zero-shot transfer remains effective. Recovery weakens for larger source--target gaps and is substantially lower on Qwen-2.5, suggesting that transfer becomes harder as architectural and scaling differences increase. Overall, DFA consistently outperforms simple baselines and, in some settings, recovers target-model circuits with faithfulness comparable to or stronger than direct attribution. These results suggest that smaller models can provide useful mechanistic priors for larger ones, while highlighting both the promise and the limits of node-level cross-model circuit alignment.\footnote{Code is available at https://github.com/jasonshaoshun/dfa-circuits.

Figures

Figures reproduced from arXiv: 2604.24302 by the authors.

Figure 1
Figure 1. Overview of Differentiable Faithfulness Alignment (DFA). Source-model node importance scores are projected into the target model through a learned alignment matrix W and converted into a soft mask mL. This mask interpolates between clean and corrupted target-model activations, producing an intervened output distribution. The alignment matrix is then optimized with a faithfulness loss and an L1 sparsity penalty, allo… view at source ↗
Figure 2
Figure 2. Direct target-model circuits (Gold) and DFA-predicted circuits (Aligned) on ARITHMETIC-SUBTRACTION for three Llama-3 source–target pairs. The x-axis shows the proportion of retained nodes k, and the y-axis shows faithfulness f . Aligned circuits closely track the gold faithfulness curves and often achieve higher CPR. 4.1 Experimental setup Models and tasks. We evaluate on Llama-3 and Qwen-2.5. Our main results focus… view at source ↗
Figure 3
Figure 3. Mean faithfulness recovery ratio across tasks for each attribution method and view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Cross-task transfer matrix for NAP-IG-INPUTS on Llama-3, evaluated with CPR. Rows denote training tasks and columns evaluation tasks. Entries report faithfulness of transferred circuits; strong off-diagonal values indicate cross-task generalization. 4.4 Scaling across …
Figure 5
Figure 5. Figure 5: Best aligned circuits (Aligned) versus direct target-model circuits (Gold) across tasks for NAP-IG-ACTIVATIONS on Llama-3 (1B→ 3B). Across tasks, aligned circuits recover similar faithfulness curves and often achieve comparable or higher CPR. 5 Discussion We studied wh…
Figure 6
Figure 6. Figure 6: Per-task comparison of direct target-model circuits (
Figure 7
Figure 7. Figure 7: Per-task comparison of direct target-model circuits (
Figure 8
Figure 8. Figure 8: Per-task comparison of direct target-model circuits (
Figure 9
Figure 9. Figure 9: Per-task comparison of direct target-model circuits (
Figure 10
Figure 10. Figure 10: Per-task comparison of direct target-model circuits (
Figure 11
Figure 11. Figure 11: Mean faithfulness recovery ratio across tasks for each attribution method and
Figure 12
Figure 12. Figure 12: Mean faithfulness recovery ratio across tasks for each attribution method and
Figure 13
Figure 13. Figure 13: Cross-task transfer matrix for EAP-IG-ACTIVATIONS on LLaMA-3, evaluated using CPR. Each row denotes the task used to train the alignment, and each column denotes the evaluation task. Entries report the faithfulness of the transferred circuit in the target model. Diago…
Figure 14
Figure 14. Figure 14: Cross-task transfer matrix for EAP on LLaMA-3, evaluated using CPR. Each row denotes the task used to train the alignment, and each column denotes the evaluation task. Entries report the faithfulness of the transferred circuit in the target model. Diagonal entries cor…
Figure 15
Figure 15. Figure 15: Cross-task transfer matrix for EAP-IG-INPUTS on Qwen-2.5, evaluated using CPR. Each row denotes the task used to train the alignment, and each column denotes the evaluation task. Entries report the faithfulness of the transferred circuit in the target model. Diagonal …
Figure 16
Figure 16. Figure 16: Cross-task transfer matrix for EAP-IG-ACTIVATIONS on Qwen-2.5, evaluated using CPR. Each row denotes the task used to train the alignment, and each column denotes the evaluation task. Entries report the faithfulness of the transferred circuit in the target model. Diag…
Figure 17
Figure 17. Figure 17: Cross-task transfer matrix for EAP on Qwen-2.5, evaluated using CPR. Each row denotes the task used to train the alignment, and each column denotes the evaluation task. Entries report the faithfulness of the transferred circuit in the target model. Diagonal entries co…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Can Graph Learning Learn Circuits?

    cs.LG 2026-08 conditional novelty 6.0 of 10

    A GNN trained across synthetic transformer-task pairs predicts held-out circuit edges with median AUROC 0.902, close to but below dedicated per-case methods ACDC and EAP-IG.

Reference graph

Works this paper leans on

62 extracted references · 62 canonical work pages · cited by 1 Pith paper

  1. [1]

    RandomW(Lower Bound) 0.25 0.25 0.25 0.25 0.27 0.28

    work page

  2. [2]

    Scrambled inputs0.27 0.25 0.25 0.25 0.25 0.32

    work page

  3. [3]

    PermutedWcolumns 0.29 0.27 0.25 0.25 0.41 0.31

    work page

  4. [4]

    Heuristic Depth Mean 0.25 0.25 0.25 0.25 0.25 0.25 DFA (Zero-shot)0.27 0.46 0.28 0.260.44 0.48 DFA (Best) 0.32 0.49 0.39 0.26 0.46 0.52 NAP-IG-Activations

    work page

  5. [5]

    RandomW(Lower Bound) 0.25 0.25 0.25 0.25 0.26 0.25

    work page

  6. [6]

    Scrambled inputs0.29 0.31 0.33 0.25 0.34 0.37

    work page

  7. [7]

    PermutedWcolumns 0.26 0.26 0.25 0.25 0.26 0.27

    work page

  8. [8]

    We report the zero-shot faithfulness drop under various structural corruptions using CPR

    Heuristic Depth Mean 0.25 0.25 0.25 0.25 0.25 0.25 DFA (Zero-shot) 0.33 0.46 0.820.28 0.34 0.51 DFA (Best) 0.33 0.46 0.82 0.45 0.41 0.51 Table 11: Validation and Ablation Study for target model llama3-3b (source llama3-1b) by NAP and NAP-IG-Activations. We report the zero-shot faithfulness drop under various structural corruptions using CPR. Method Settin...

    work page

Show all 62 references
  1. [10]

    Scrambled inputs0.33 0.26 0.27 0.25 0.34 0.27

  2. [11]

    PermutedWcolumns 0.29 0.27 0.25 0.25 0.26 0.28

  3. [12]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.33 0.51 0.67 0.39 0.47 0.38 DFA (Best) 0.34 0.51 0.67 0.39 0.47 0.38 NAP

  4. [14]

    Scrambled inputs0.390.25 0.26 0.25 0.25 0.30

  5. [15]

    PermutedWcolumns 0.31 0.25 0.25 0.25 0.33 0.31

  6. [16]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.27 0.40 0.400.26 0.30 0.29 DFA (Best)0.33 0.46 0.40 0.33 0.49 0.53 NAP-IG-Activations

  7. [18]

    Scrambled inputs0.560.34 0.31 0.25 0.33 0.26

  8. [19]

    PermutedWcolumns 0.30 0.26 0.25 0.25 0.26 0.27

  9. [20]

    We report the zero-shot faithfulness drop under various structural corruptions using CPR

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.360.400.41 0.28 0.35 0.40 DFA (Best)0.43 0.40 0.41 0.28 0.35 0.40 Table 12: Validation and Ablation Study for target model llama3-8b (source llama3-1b). We report the zero-shot faithfulness drop under various structural corrupt...

  10. [22]

    Scrambled inputs0.28 0.28 0.25 0.27 0.29 0.49

  11. [23]

    PermutedWcolumns 0.25 0.26 0.25 0.25 0.27 0.26

  12. [24]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot) 0.57 0.460.27 0.46 0.27 0.25 DFA (Best) 0.57 0.46 0.45 0.67 0.340.34 NAP

  13. [25]

    RandomW(Lower Bound) 0.25 0.25 0.25 0.25 0.25 0.25

  14. [26]

    Scrambled inputs0.26 0.26 0.25 0.26 0.250.35

  15. [27]

    PermutedWcolumns 0.25 0.26 0.25 0.25 0.26 0.26

  16. [28]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot) 0.36 0.320.27 0.26 0.290.27 DFA (Best) 0.36 0.32 0.40 0.40 0.290.29 NAP-IG-Activations

  17. [29]

    RandomW(Lower Bound) 0.25 0.26 0.25 0.25 0.26 0.26

  18. [30]

    Scrambled inputs0.25 0.31 0.25 0.42 0.25 0.29

  19. [31]

    PermutedWcolumns 0.29 0.26 0.25 0.25 0.26 0.32

  20. [32]

    We report the zero-shot faithfulness drop under various structural corruptions using CPR

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.44 0.50 0.28 0.590.36 0.43 DFA (Best) 0.50 0.54 0.46 0.59 0.56 0.56 Table 13: Validation and Ablation Study for target model llama3-8b (source llama3-3b). We report the zero-shot faithfulness drop under various structural corru...

  21. [34]

    Scrambled inputs0.25 0.27 - 0.25 0.25 0.27

  22. [35]

    PermutedWcolumns 0.250.32- 0.25 0.29 0.29

  23. [36]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.26 0.25 -0.390.27 0.26 DFA (Best) 0.300.30 -0.39 0.29 0.29 NAP

  24. [37]

    RandomW(Lower Bound) 0.25 0.25 - 0.25 0.27 0.27

  25. [38]

    Scrambled inputs0.250.44-0.410.28 0.25

  26. [39]

    PermutedWcolumns 0.24 0.28 - 0.25 0.27 0.26

  27. [40]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.25 0.27 - 0.250.38 0.36 DFA (Best) 0.280.28 - 0.40 0.38 0.36 NAP-IG-Activations

  28. [41]

    RandomW(Lower Bound) 0.25 0.25 - 0.25 0.28 0.30

  29. [42]

    Scrambled inputs0.250.43- 0.25 0.33 0.25

  30. [43]

    PermutedWcolumns 0.25 0.25 - 0.25 0.26 0.25

  31. [44]

    We report the zero-shot faithfulness drop under various structural corruptions using CPR

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.25 0.28 -0.40 0.42 0.44 DFA (Best) 0.290.34 -0.40 0.42 0.44 Table 14: Validation and Ablation Study for target model qwen2.5-1.5b (source qwen2.5- 0.5b). We report the zero-shot faithfulness drop under various structural corrup...

  32. [45]

    RandomW(Lower Bound) 0.25 0.25 - 0.25 0.25 0.25

  33. [46]

    Scrambled inputs0.25 0.36 - 0.25 0.25 0.29

  34. [47]

    PermutedWcolumns 0.27 0.38- 0.250.35 0.34

  35. [48]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot) 0.300.32 - 0.25 0.28 0.28 DFA (Best) 0.300.37 -0.260.35 0.34 NAP

  36. [50]

    Scrambled inputs0.250.57- 0.25 0.29 0.25

  37. [51]

    PermutedWcolumns 0.29 0.30 - 0.25 0.30 0.29

  38. [52]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot) 0.320.27 - 0.24 0.32 0.35 DFA (Best) 0.320.37 -0.26 0.37 0.42 NAP-IG-Activations

  39. [53]

    RandomW(Lower Bound) 0.25 0.27 - 0.25 0.25 0.25

  40. [54]

    Scrambled inputs0.250.60- 0.25 0.24 0.25

  41. [55]

    PermutedWcolumns0.300.30 - 0.25 0.31 0.31

  42. [56]

    We report the zero-shot faithfulness drop under various structural corruptions using CPR

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.25 0.34 - 0.25 0.34 0.48 DFA (Best)0.25 0.36 -0.26 0.34 0.48 Table 15: Validation and Ablation Study for target model qwen2.5-3b (source qwen2.5-0.5b). We report the zero-shot faithfulness drop under various structural corrupti...

  43. [57]

    RandomW(Lower Bound) 0.25 0.28 - 0.25 0.27 0.27

  44. [58]

    Scrambled inputs0.260.38- 0.25 0.280.34

  45. [59]

    PermutedWcolumns 0.29 0.27 -0.280.31 0.30

  46. [60]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot) 0.360.33 - 0.27 0.29 0.29 DFA (Best) 0.360.33 - 0.27 0.310.31 NAP

  47. [61]

    RandomW(Lower Bound) 0.25 0.34 - 0.25 0.29 0.25

  48. [62]

    Scrambled inputs0.25 0.41- 0.26 0.25 0.34

  49. [63]

    PermutedWcolumns 0.24 0.29 - 0.26 0.27 0.28

  50. [64]

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.23 0.28 - 0.260.40 0.38 DFA (Best) 0.280.28 -0.27 0.40 0.38 NAP-IG-Activations

  51. [65]

    RandomW(Lower Bound) 0.250.39- 0.26 0.28 0.25

  52. [66]

    Scrambled inputs0.25 0.34 - 0.27 0.26 0.29

  53. [67]

    PermutedWcolumns 0.23 0.28 -0.270.27 0.27

  54. [68]

    We report the zero-shot faithfulness drop under various structural corruptions using CPR

    Heuristic Depth Mean - - - - - - DFA (Zero-shot)0.23 0.30 - 0.260.35 0.34 DFA (Best) 0.290.30 - 0.27 0.35 0.34 Table 16: Validation and Ablation Study for target model qwen2.5-3b (source qwen2.5-1.5b). We report the zero-shot faithfulness drop under various structural corrupti...

Pith tools

Reviewed May 8, 2026 · model on record in the stance chip above.