Beyond Transfer Accuracy: Faithful Circuits for Controlled Low-Resource Adaptation

Alham Fikri Aji; Ayu Purwarianti; Derry Wijaya; Khumaisa Nur'aini

REVIEW 4 major objections 5 minor 7 references

Beyond Transfer Accuracy: Faithful Circuits for Controlled Low-Resource Adaptation

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Updating only task-relevant attention heads—roughly 0.2–0.7% of parameters—can adapt a model to a low-resource language while preserving its source-language competence.

desk verdict Useful idea—counterfactual-free circuit discovery for low-resource tuning—but the headline numbers rely on oracle scope selection and missing sparse baselines, so treat them as upper bounds until the protocol is fixed. read the letter →

arxiv 2601.08146 v3 pith:6WNSXYFJ submitted 2026-01-13 cs.CL cs.AIcs.LG

Khumaisa Nur'aini , Ayu Purwarianti , Alham Fikri Aji , Derry Wijaya This is my paper

classification cs.CLcs.AIcs.LG

keywords low-resourceadaptationmechanisticinterpretabilitycircuitdiscoveryattentionheadscatastrophicforgettingcross-lingualtransfercontextualdecompositionparameter-efficientfine-tuning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a model can be adapted to a low-resource language by updating only the attention heads that carry the task in a higher-resource proxy language, rather than fine-tuning everything. It adapts circuit-discovery machinery to unstructured text, where clean counterfactuals do not exist, by defining a neutral baseline as a label-balanced mean and scoring heads by how much they move the model toward the correct label. Applied to sentiment transfer from Indonesian to four related languages and to cross-lingual natural-language inference, the method matches or beats continued full fine-tuning in target accuracy while updating under one percent of parameters. Its distinctive claim is that this targeted update prevents catastrophic forgetting: source-language accuracy stays near its pre-transfer level, whereas full fine-tuning degrades it sharply. The paper also reports an editing–preserving trade-off: harder transfers benefit from updating the discovered circuit, while easier transfers benefit from updating near-zero heads.

What carries the argument

The central object is the circuit—a sparse set of attention heads identified by a counterfactual-free adaptation of Contextual Decomposition for Transformers (CD-T). CD-T separates each component activation into an irrelevant baseline (γ) and a relevant deviation (β); the paper replaces templated counterfactual baselines with a label-balanced activation mean, replaces the magnitude ratio with a directional relevance score projecting β onto the task direction (the difference between the correct-label and average-incorrect-label unembedding directions), and selects the top K heads by mean directional relevance at successive depths. Circuit-targeted supervised fine-tuning then applies head-leve

What would settle it

Run mean-ablation or activation-patching on target-language inputs: if removing or ablating the selected circuit heads does not drop target accuracy more than removing the same number of random heads, the circuit is not causally relevant in the target. Alternatively, match random-head selection to the circuit's layer distribution and parameter count and compare target accuracy and source-language retention; if random heads reproduce the gains, sparsity rather than mechanism targeting explains the results.

Watch

Extended reading notes

Core claim

The central claim is that circuits discovered on a source-language checkpoint transfer across languages and can serve as the training locus for low-resource adaptation. Using a 50-example label-balanced mean as the CD-T baseline and a directional relevance score that measures support for the correct label, the authors select a small set of attention heads (6, 12, or 18 in their settings). They then freeze everything except those heads and LayerNorm. Across two tasks, this CT-SFT procedure attains cross-lingual accuracy competitive with or better than continued full fine-tuning, while preserving source-language competence: Indonesian accuracy stays near 0.76 after target tuning, versus drops

Load-bearing premise

The load-bearing premise—flagged by the authors in the Limitations section—is that the circuit discovered on the source checkpoint, using a 50-example label-balanced mean baseline drawn from the same pool as competence tuning, is faithful and transfers to the target language; if that baseline is not neutral or the heads do not carry a transferable mechanism, the accuracy and retention gains cannot be attributed to mechanism targeting.

Editorial extensions

If this is right

Updating only the discovered circuit heads plus LayerNorm should be preferred over continued full fine-tuning for low-resource transfer when source competence exists: it matches or exceeds target accuracy while using 0.2–0.7% of model parameters.
Source-language competence survives adaptation: models tuned with CT-SFT retain near-baseline accuracy in the source language, whereas full fine-tuning can lose 20–40 percentage points.
The update locus should be chosen by transfer difficulty: harder transfers benefit from updating the circuit heads, easier transfers from updating near-zero-relevance heads to preserve the source mechanism.
A checkpoint must first be task-competent; CT-SFT is a post-competence adaptation method, not a substitute for learning the task—a limitation the authors flag explicitly.
The same editing–preserving pattern appears on a second task (natural language inference) once competence is achieved, suggesting the mechanism-transfer view is not sentiment-specific.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is an automated router that measures source-checkpoint accuracy on the target (the paper's A0 baseline) and chooses Circuit versus NearZero heads accordingly; this could make the method deployment-ready.
If the transferred-circuit premise is right, closely related language pairs should need almost no parameter movement and distant pairs the most; sampling a wider typological spread would stress-test the claim.
The paper implicitly defines transfer difficulty as the competence-tuned checkpoint's accuracy on the target; this definition could be validated against independent difficulty metrics such as lexical overlap or syntactic distance.
As the authors themselves caution, the discovered heads are empirical candidates for one model family and task setup, not universal mechanisms; practitioners should run minimal target-language sanity checks before deployment.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Let me give you the short version first. The paper proposes CT-SFT: discover a task-relevant attention-head circuit on a source-language checkpoint using a modified CD-T (label-balanced mean baseline, directional relevance score), then fine-tune only those heads plus LayerNorm on a low-resource target. That is a genuinely useful combination: CD-T normally needs clean counterfactuals, and the authors make it work on unstructured sentiment/NLI text. The within-language sanity check is also honest—it shows continued full fine-tuning is unstable on 25–50 examples and that random sparse updates regularize well.

But the main cross-lingual results are not yet a fair test of the method as specified. The biggest problem is the 'takes the better of Circuit and NearZero' protocol (Tables 2, 3, and Appendix Table 4). No rule is given for when to pick mechanism-editing vs mechanism-preserving scope; if that choice is made after seeing the test results, the reported numbers describe an oracle-augmented variant. That is load-bearing because the headline claim is that CT-SFT beats full fine-tuning.

Second, the cross-lingual comparisons omit random-head and least-relevant baselines, which the paper itself reports within language. Since random sparse updates often beat full fine-tuning, the current tables cannot separate 'targeted circuit' from 'any small sparse update.' The authors say they left the baselines out to reduce clutter; that is not a scientific reason.

Third, there are no error bars or significance tests. Four seeds in this few-shot regime can easily hide high variance, and several numbers are close (e.g., 0.636 vs 0.623). This is fixable but necessary.

The forgetting result is the most interesting piece: CT-SFT keeps Indonesian accuracy near the competence baseline while full fine-tuning drops to 0.34–0.45. That pattern is visible and worth taking seriously, though it also inherits the oracle-scope issue.

The limitations section is candid about single model family and the 50-example mean sensitivity, and the shared-pool D.3 analysis is a reasonable acknowledgment of a real constraint. The abstract's claim about 'model families' is not supported by the experiments, but the limitation text says so.

Who is this for? People working on low-resource adaptation or mechanism-guided fine-tuning. It is a solid empirical exploration with a clear flaw in evaluation protocol, not a paper to ignore. I would send it for peer review with a clear request: specify the scope-selection rule, add random/least-relevant baselines to the cross-lingual tables, and report variance. Then the central claim might hold or collapse, but at least it would be testable.

My bottom line: engage seriously, but ask for the fixed-protocol revision before believing the headline numbers.

Referee Report

4 major / 5 minor

Summary. The paper proposes CT-SFT, a two-stage method: (1) discover a task-relevant circuit of attention heads in a higher-resource 'proxy' language by adapting Contextual Decomposition for Transformers (CD-T) to unstructured text via a label-balanced activation mean (Eq. 3) and a directional relevance score (Eqs. 4-6); (2) adapt to a low-resource target by fine-tuning only the selected heads plus LayerNorm. The authors evaluate on NusaX sentiment and XNLI, claiming that CT-SFT improves cross-lingual accuracy over continued full fine-tuning while preserving source-language performance, and identify an editing-preserving trade-off depending on transfer difficulty.

Significance. If the claims were fully supported, the paper would make a useful practical contribution: a counterfactual-free circuit-discovery procedure that enables mechanism-targeted adaptation for low-resource languages, with a strong forgetting-reduction result. The paper is also commendably transparent, reporting detailed diagnostics (faithfulness, topology, selection-ratio sweep, shared-pool stability) and an explicit limitations section. However, the headline empirical claims currently rest on an oracle scope-selection protocol and on cross-lingual comparisons that lack the controls needed to attribute gains to circuit targeting.

major comments (4)

[§6.2, Table 2; Appendix C.1 Table 4; Table 3] CT-SFT is defined as 'taking the better of the mechanism-editing and mechanism-preserving update scopes' (Table 4; also Table 3). The protocol does not specify when or on what data this better scope is chosen. If it is chosen on the test set, the reported numbers describe an oracle-augmented method rather than a fixed CT-SFT procedure, and the Abstract's claim that 'CT-SFT improves ... while updating only a small subset' is not supported. Please define a deterministic scope-selection rule (e.g., using the competence-tuning baseline A0 or validation faithfulness) and report results for the resulting fixed method, or report Circuit and NearZero separately without selecting the better.
[§6.1, Figure 2; §6.2, Table 2] Section 6.1 reports that random sparse head updates frequently outperform full fine-tuning in the within-language sanity check (Figure 2), and the text states the random-head baseline is omitted from cross-lingual plots 'to reduce visual clutter.' Without a random-head (and least-relevant-head) control in the cross-lingual setting, the gains in Table 2 cannot be attributed to the task-relevance of the selected heads rather than to sparsity/regularization. Please add these controls to the cross-lingual tables/figures, or at minimum report them in an appendix.
[§4.1.1 (Eq. 3); Appendix D.3] Competence tuning, baseline mean µ, circuit-discovery inputs, and faithfulness diagnostics are all drawn from the same 50-example pool. Appendix D.3 shows only that iteration-0 head ranking is more stable under the shared pool; it does not establish that the resulting circuit is a faithful, transferable mechanism for target languages. If the balanced mean is not neutral or the selected heads reflect pool-specific noise, the cross-lingual transfer results in Tables 2 and 4 would not be mechanistically grounded. Please validate the discovery with held-out pools or report cross-lingual faithfulness.
[§5.4; Tables 2-4] All results are averaged over 4 seeds, but no standard deviations, confidence intervals, or significance tests are reported. Given the small tuning sizes (n=25) and small performance differences (e.g., Table 2 bug n=25: 0.374 vs 0.376), the claim that CT-SFT 'consistently improves' over full fine-tuning is not statistically supported. Please include variance estimates or significance tests (e.g., bootstrap or paired tests across seeds).

minor comments (5)

[Eq. (4)] The notation Y and Y_{-other} is used but not explicitly defined at first use; please clarify that Y is the label set and Y_other is all labels except the correct one.
[§5.1] The training recipe (epochs, learning rate, batch size) is deferred to Appendix A; please state it in the main text or add a forward reference.
[Table 2 caption] The underline/bold convention is visually hard to distinguish in the table; consider using symbols (e.g., daggers) or a separate column for the selected scope.
[References] Several references use incomplete author lists (e.g., 'Alan Ansell and 1 others'); use standard et al. or full author lists consistently.
[§6.3] The XNLI summary is very brief; a key table (e.g., the 250-sample English competence setting) should be moved from the appendix to the main text, as the current text relies on a separate figure.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: CT-SFT's headline numbers are an oracle-envelope over two scopes, and the faithfulness diagnostic reuses the same baseline that defines selection; the core empirical comparisons retain independent content.

fitted input called prediction [Appendix C.1, Table 4 note; also §6.2 Table 2 note and Table 3 note]
"CT-SFT (d=2) reports circuit-targeted tuning from the same competence-tuned checkpoint using discovery depth d=2, taking the better of the mechanism-editing and mechanism-preserving update scopes."

The reported 'CT-SFT' performance is defined as the better of two distinct update scopes, Circuit and NearZero, with the better scope selected after seeing the reported test accuracies (Table 2 bolds/underlines whichever wins). Therefore the headline claim that CT-SFT beats continued full fine-tuning is not a prediction of a fixed method; it is an oracle upper envelope. The 'editing-preserving trade-off' is likewise a post-hoc classification of which scope happened to win, rather than a rule derived from the mechanism hypothesis. This reduces the central comparison, in part, to model selection on the outcome rather than to mechanism-targeted updates.
self definitional [§6.4 (Faithfulness) with §3 Eq. 2 and §4.1.1 Eq. 3; Appendix D.1]
"We compute faithfulness using a mean-ablation protocol following CD-T: heads outside the circuit are replaced by their mean activation (the γ / 'irrelevant' baseline) (Hsu et al., 2025), and we measure how well the circuit-only model preserves the full model behavior."

The same mean µ that defines the 'irrelevant' stream γ in the CD-T relevance score used for circuit selection (Eq. 2/3) is the value used to ablate non-circuit heads in the faithfulness test. Thus a head excluded from the circuit is, by construction, one whose activation is measured against that same γ; mean-ablating it to µ partly restates the selection criterion. The faithfulness diagnostic is therefore not fully independent evidence for the task-directional relevance score, but is partially aligned with the selection objective by definition. This circularity affects the causal-validity framing of the method, though it does not by itself determine the cross-lingual accuracy numbers.

full rationale

The core accuracy and forgetting results are not derived from fitted parameters in the usual sense: circuit selection uses source-language activations and labels, and target metrics are computed on a disjoint held-out test split, so the raw numbers for each fixed scope retain independent empirical content. The circularity signal is concentrated in two places. First, the paper's reported 'CT-SFT' is an oracle over the Circuit and NearZero scopes, chosen after the fact, so the headline improvement over full fine-tuning is partly forced by reporting construction rather than by a single specified update rule. Second, the faithfulness diagnostic used to justify the task-directional scoring is self-referential: the baseline µ used to define what is 'relevant' is the same baseline used to mean-ablating non-selected heads, so the metric is partly aligned with the selection objective by construction. These are genuine but partial circularities; the per-scope, per-language accuracy and retention numbers are still informative and are not fully entailed by the discovery equations. Hence a moderate score of 4 rather than a higher one.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper relies on CD-T's decomposition being a faithful model of attention-head contributions, on a 50-sample label-balanced mean being a neutral baseline, on WU being a valid task direction, and on circuits transferring across languages. These are domain assumptions, not proven facts.

free parameters (4)

per-iteration selection ratio p = 2%
Chosen by hand to enforce sparsity; Appendix D.2 sweeps 2/5/10% but headline uses 2%. The choice affects circuit size and faithfulness.
circuit expansion depth max_depth = 2 (headline); 0/1/2 explored
Depth controls how many upstream heads are included; Table 4 and CT-SFT summaries use d=2. The paper reports all depths but selects d=2 for headline transfer/forgetting tables.
discovery pool size / mean pool size = 50 examples
Baseline mean and discovery set use 50 samples due to memory and label-balance constraints; sensitivity is not tested.
number of source competence examples n_src = 50 (NusaX), 50/250 (XNLI)
Competence-tuning data size is varied; results depend on it (XNLI 50 vs 250 show instability).

assumptions (5)

domain assumption CD-T additive decomposition into relevant/irrelevant streams faithfully captures model computation
Paper adopts CD-T (Hsu et al. 2025) as the discovery backbone; all selection is based on this decomposition.
domain assumption Label-balanced mean baseline cancels class-directional signal and approximates a neutral operating point
Section 4.1.1 Eq. 3; the claim that balancing removes class leakage from baseline is asserted, not proven.
domain assumption Unembedding matrix WU provides a valid label-specific task direction for intermediate relevance
Section 4.1.2 Eq. 5-6; assumes residual-stream updates align with readout weights.
domain assumption Mean-ablation faithfulness (replacing non-circuit heads with mean) is a valid measure of circuit sufficiency
Appendix D.1; the same baseline is used for selection and evaluation, so faithfulness may be optimistically biased.
domain assumption Circuits discovered on source-language checkpoint transfer to target languages
The entire CT-SFT transfer rests on this; it is not independently verified beyond task accuracy.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Beyond Transfer Accuracy: Faithful Circuits for Controlled Low-Resource Adaptation." pith.science (2026). https://pith.science/paper/6WNSXYFJ

@misc{pith2026260108146,
  author       = {Pith},
  title        = {Pith review of: Beyond Transfer Accuracy: Faithful Circuits for Controlled Low-Resource Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6WNSXYFJ}},
  note         = {Machine review of arXiv:2601.08146}
}

read the original abstract

Existing circuit discovery methods rely on templated tasks with clean counterfactuals, limiting their use on diverse natural text. We adapt Contextual Decomposition for Transformers (CD-T) for unstructured settings via label-balanced activation means and task-directional relevance scoring, enabling counterfactual-free circuit discovery. We leverage these circuits for Circuit-Targeted Supervised Fine-Tuning (CT-SFT), restricting parameter updates to task-relevant heads and LayerNorm. Experiments on NusaX cross-lingual sentiment transfer show that CT-SFT is highly competitive for low-resource adaptation. While non-circuit sparse updates and full fine-tuning sometimes match target accuracy through capacity recruitment, CT-SFT uniquely minimizes catastrophic forgetting, preserving source-language and related-task performance. Extensions to XNLI confirm these findings hold across broader tasks and model families, demonstrating that circuit-targeted adaptation provides a safer, causally grounded alternative to global fine-tuning.

Figures

Figures reproduced from arXiv: 2601.08146 by the authors.

**Figure 1.** CT-SFT overview. Our method has two phases. Mechanism Discovery (left): starting from a base model, we perform competence tuning on a higher-resource proxy language to obtain a task-competent checkpoint θ1. From a discovery pool, we form (i) a correctly-predicted discovery set X under θ1 and (ii) a label-balanced statistical baseline µ for CD-T. We then run a modified CD-T procedure to score components and select a … view at source ↗

**Figure 2.** Indonesian → Indonesian: test accuracy versus training sample size under different head-selection strategies, shown for circuit expansion depths (max_depth = 0, 1, 2). Target lang n Full FT (100%) Depth 0 (6H, 0.23%) Depth 1 (12H, 0.44%) Depth 2 (18H, 0.66%) Circuit NearZero Circuit NearZero Circuit NearZero ace 25 0.346 0.482 0.47 0.491 0.468 0.486 0.473 bug 25 0.376 0.374 0.373 0.398 0.392 0.394 0.402 jav 25 0.384… view at source ↗

**Figure 3.** Accuracy-based faithfulness across circuit [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Cross-lingual NusaX transfer (Indonesian [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: XNLI cross-lingual transfer results (surgical setting; 250-sample English competence tuning): test [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

**Figure 6.** Figure 6: Faithfulness (margin-based). We report the same comparison as [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

**Figure 8.** Figure 8: Circuit topology per depth under magnitude-based scoring (top) and task-direction projection scoring [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

7 extracted references · 1 linked inside Pith

[5]

InFindings of the Association for Computa- tional Linguistics: EMNLP 2024, pages 4308–4322, Miami, Florida, USA

Mitigating catastrophic forgetting in large lan- guage model tuning with sharpness-aware minimiza- tion. InFindings of the Association for Computa- tional Linguistics: EMNLP 2024, pages 4308–4322, Miami, Florida, USA. Association for Computational Linguistics. Yueyan Li, Wenhao Gao, Caixia Yuan, and Xiaojie Wang. 2025. Fine-tuning is subgraph search: A ne...

arXiv 2024
[7]

entail",

NusaX: Multilingual parallel sentiment dataset for 10 Indonesian local languages. InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 815–834, Dubrovnik, Croatia. Association for Com- putational Linguistics. Haeun Yu, Seogyeong Jeong, Siddhesh Pawar, Jisu Shin, Jiho Jin, Junho Myung, Alice O...

arXiv 2025
[2019]

Association for Computational Linguistics

What does BERT learn about the structure of language? InProceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 3651–3657, Florence, Italy. Association for Computational Linguistics. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity an...

2020
[2020]

InProceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 4411–4421

XTREME: A massively multilingual multi- task benchmark for evaluating cross-lingual gener- alisation. InProceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 4411–4421. PMLR. Also available via ACM DL: https://dl.acm.org/doi/10.5555/3524938.3525348. Ganesh Jawahar, Benoît Sagot,...

arXiv
[2023]

InAdvances in Neural Infor- mation Processing Systems, volume 36

Towards automated circuit discovery for mech- anistic interpretability. InAdvances in Neural Infor- mation Processing Systems, volume 36. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross- lingual sentence representations. InProceedings of the 2018 Conference on...

arXiv 2018
[2024]

In ICML 2024 Workshop on Mechanistic Interpretabil- ity

Have faith in faithfulness: Going beyond cir- cuit overlap when finding model mechanisms. In ICML 2024 Workshop on Mechanistic Interpretabil- ity. Spotlight. Aliyah R. Hsu, Georgia Zhou, Yeshwanth Cherapanam- jeri, Yaxuan Huang, Anobel Odisho, Peter R. Carroll, and Bin Yu. 2025. Efficient automated circuit discov- ery in transformers using contextual deco...

2024
[2025]

InProceed- ings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20465–20497, Suzhou, China

Explainability and interpretability of multilin- gual large language models: A survey. InProceed- ings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20465–20497, Suzhou, China. Association for Computational Lin- guistics. Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeffrey Wu, Lucius Bushnaq, Nicholas Gold...

2025

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [5]

InFindings of the Association for Computa- tional Linguistics: EMNLP 2024, pages 4308–4322, Miami, Florida, USA

Mitigating catastrophic forgetting in large lan- guage model tuning with sharpness-aware minimiza- tion. InFindings of the Association for Computa- tional Linguistics: EMNLP 2024, pages 4308–4322, Miami, Florida, USA. Association for Computational Linguistics. Yueyan Li, Wenhao Gao, Caixia Yuan, and Xiaojie Wang. 2025. Fine-tuning is subgraph search: A ne...

arXiv 2024

[2] [7]

entail",

NusaX: Multilingual parallel sentiment dataset for 10 Indonesian local languages. InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 815–834, Dubrovnik, Croatia. Association for Com- putational Linguistics. Haeun Yu, Seogyeong Jeong, Siddhesh Pawar, Jisu Shin, Jiho Jin, Junho Myung, Alice O...

arXiv 2025

[3] [2019]

Association for Computational Linguistics

What does BERT learn about the structure of language? InProceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 3651–3657, Florence, Italy. Association for Computational Linguistics. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity an...

2020

[4] [2020]

InProceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 4411–4421

XTREME: A massively multilingual multi- task benchmark for evaluating cross-lingual gener- alisation. InProceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 4411–4421. PMLR. Also available via ACM DL: https://dl.acm.org/doi/10.5555/3524938.3525348. Ganesh Jawahar, Benoît Sagot,...

arXiv

[5] [2023]

InAdvances in Neural Infor- mation Processing Systems, volume 36

Towards automated circuit discovery for mech- anistic interpretability. InAdvances in Neural Infor- mation Processing Systems, volume 36. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross- lingual sentence representations. InProceedings of the 2018 Conference on...

arXiv 2018

[6] [2024]

In ICML 2024 Workshop on Mechanistic Interpretabil- ity

Have faith in faithfulness: Going beyond cir- cuit overlap when finding model mechanisms. In ICML 2024 Workshop on Mechanistic Interpretabil- ity. Spotlight. Aliyah R. Hsu, Georgia Zhou, Yeshwanth Cherapanam- jeri, Yaxuan Huang, Anobel Odisho, Peter R. Carroll, and Bin Yu. 2025. Efficient automated circuit discov- ery in transformers using contextual deco...

2024

[7] [2025]

InProceed- ings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20465–20497, Suzhou, China

Explainability and interpretability of multilin- gual large language models: A survey. InProceed- ings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20465–20497, Suzhou, China. Association for Computational Lin- guistics. Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeffrey Wu, Lucius Bushnaq, Nicholas Gold...

2025