Pith. sign in

REVIEW 3 major objections 6 minor 47 references

Demographic bias and general skills sit on separable circuits: zeroing a few dozen MLP neurons changes bias responses while leaving capabilities almost intact.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 11:23 UTC pith:CJEGRICZ

load-bearing objection Solid training-free localization recipe with honest non-mitigation results; the dissociable-circuits claim is suggestive but under-controlled. the 3 major comments →

arxiv 2607.28319 v1 pith:CJEGRICZ submitted 2026-07-30 cs.CL cs.CYcs.LG

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

Pere Martra , Eugenio Mart\'inez C\'amara , Alfonso Ure\~na L\'opez This is my paper
classification cs.CL cs.CYcs.LG
keywords demographic biasfairness pruningdifferential activationsGLU-MLPBiasScoremechanistic interpretabilitywidth pruningactivation-guided intervention
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that demographic bias in small GLU-based language models is not smeared evenly through the network but concentrated in identifiable neurons, especially in the final MLP layers. By feeding minimally contrastive prompt pairs that differ only in one demographic word and measuring activation differences at the down-projection input, the authors score neurons by how much they react to that change. Zeroing the highest-scoring neurons—at most 40 in a 1B model, under 0.031% of MLP width—measurably changes how the model handles the associated demographic attributes, while reasoning and knowledge benchmarks retain about 99.5% of baseline performance on average. The catch is that the score is unsigned, so the selected set mixes neurons that push toward stereotypes with ones that push against them; the net bias effect is therefore bidirectional and non-monotonic rather than clean mitigation. The practical point is causal localization: bias processing and core capabilities are dissociable enough to support surgical structural intervention, and the next step is a signed score that steers rather than blindly zeros.

Core claim

Demographic-attribute processing and general capabilities operate on sufficiently dissociable circuits in GLU-MLP layers that zeroing at most about 40 BiasScore-selected neurons (less than 0.031% of Llama-3.2-1B MLP width) causally alters responses to associated demographic variables while retaining a mean of about 99.49% on the retained capability suite—even though the unsigned score produces bidirectional, non-monotonic bias effects rather than flat mitigation.

What carries the argument

BiasScore: the absolute difference, averaged over contrastive prompt pairs, between per-sequence mean activations of a neuron at the down_proj input when only the demographic attribute changes. It ranks neurons for selective zeroing of the matching gate, up, and down weights without retraining.

Load-bearing premise

That activation differences on explicit, same-length demographic word swaps, scored mainly against multiple-choice stereotype benchmarks, pick out the neurons that actually implement the bias that matters in real use.

What would settle it

If zeroing the top BiasScore neurons left BBQ/EsBBQ category scores and open generations statistically unchanged relative to matched random or importance-matched controls, or if the same tiny zeroing collapsed retained capability benchmarks well below the reported ~99% band, the dissociable-circuit claim would fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Bias can be targeted with structural edits that do not require fine-tuning or full-model rewrites.
  • Unsigned magnitude scores are insufficient for controlled mitigation; signed differential scores are required to separate amplifiers from suppressors.
  • Per-category circuits are mostly distinct, so interventions can be scoped to one demographic axis with limited cross-talk in Llama-scale models.
  • Final-layer down_proj inputs are the highest-yield measurement point for this localization in SwiGLU MLPs.
  • Asymmetric prompt corpora would convert blind zeroing into directional activation steering at the same low cost.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the same dissociability holds at production scale, fairness edits could ship as tiny weight masks rather than alignment retrainings.
  • Tokenization-parity constraints may systematically miss morphologically rich or multi-token identity cues, so implicit-bias pairs are a natural stress test of whether the circuits generalize.
  • Mixing amplifiers and suppressors in one zeroing set resembles knocking out both brakes and accelerators; signed scaling would turn the method into a controllable dial rather than a random shove.
  • Attention-head participation remains an open parallel circuit that could either reinforce or bypass these MLP edits.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Fairness Pruning: locate demographic-sensitive neurons in GLU-MLP layers by measuring unsigned differential activations (BiasScore) at down_proj input on minimally contrastive, token-length-matched prompt pairs, then zero the top-K candidates. On Llama-3.2-1B/3B and Salamandra-2B it reports (i) concentration of the BiasScore signal in final layers with moderate per-category circuit specificity, (ii) that zeroing ≤40 neurons alters BBQ/EsBBQ category scores and open generations in bidirectional, often non-monotonic ways rather than flat mitigation—attributed to mixing stereotype-amplifying and -suppressing units—and (iii) mean ~99.49% retention on a filtered capability suite. The authors frame this as causal localization of dissociable demographic circuits and as a foundation for later signed/directional steering.

Significance. If the localization is causally specific, the work would offer a cheap, training-free alternative to SAE-based feature isolation for demographic attributes, with public EN/ES pair datasets, open code (OptiPFair), and an unusually honest negative result on unsigned zeroing as mitigation. The depth/overlap analyses and the Llama-3B Religion Top-20 sign inversion are interesting mechanistic observations. The practical significance of the current intervention is limited by the authors’ own finding that blind zeroing destabilizes rather than mitigates bias; the main near-term value is methodological (localization pipeline + signed-score roadmap) rather than a deployable fairness fix.

major comments (3)
  1. [§V-C, Tables VII–IX] Central claim of “dissociable demographic circuits” (§V-C, §VI-A, abstract) is under-supported without specificity controls. Capability retention after zeroing ≤0.031% of MLP width (Table IX) is expected for almost any tiny lesion and does not establish demographic specialization. The bias-side evidence (Tables VII–VIII; Llama-3B Religion Top-20 ambig 6.00%→1.50% and disambig sign flip; Table VI generations) lacks (i) random final-layer neuron zeroing matched for K and layer, (ii) high-PPM / low-BiasScore controls, and (iii) scrambled or non-demographic contrastive-pair controls. Final-layer MLP units already shape the output distribution (Geva et al., cited); generic perturbation of weak ambiguous MC stereotype items could produce bidirectional BBQ volatility. Without these ablations, “BiasScore neurons implement demographic processing” remains hard to separate from “small final-layer l
  2. [§V-C2, Tables IV, VII, VIII] BBQ/EsBBQ baseline bias scores are very small (e.g. aggregate ambig ~1.7% on Llama-1B; Religion ambig 5.33%/6.00%), and reported deltas are often sub-percentage-point with no uncertainty (bootstrap/seed/item-level CIs, multiple-comparison control across the experiment grid in Table II). Opposite-direction ambig vs disambig moves in 6/8 Llama-1B runs (Table VII) and non-monotonic Top-K effects could partly be noise. Category-level claims, especially the load-bearing Llama-3B Religion inversion (Table VIII), need error bars and pre-registered primary metrics before they can anchor causal localization.
  3. [Eq. (1), §VI-A, §VI-E] Eq. (1) defines BiasScore as an unsigned absolute mean activation difference on explicitly marked, length-matched pairs, aggregated by per-sequence mean over positions. The paper correctly notes this mixes amplifiers and suppressors (§VI-A, contribution 4), but then still concludes that “demographic bias processing and model capabilities operate on dissociable circuits.” That conclusion overreaches what unsigned magnitude localization plus mixed-sign zeroing can establish. Either (a) add signed/asymmetric-pair results that separate directions, or (b) narrow the claim to “unsigned differential-activation candidates are causally entangled with demographic responses (with mixed sign)” and treat full dissociability as a hypothesis for the signed follow-up. The tokenization-parity filter and BBQ MC format (§III-B, §VI-D) further bound external validity and should constrain the claim language
minor comments (6)
  1. [§III-C, §IV] FairnessPruningScore (Eq. 2, α=0.8) is defined but not used for the reported zeroing grid; state earlier that all main results are pure BiasScore Top-K to avoid implying the composite score was validated.
  2. [§III-E, Table III] Capability suite is filtered post hoc because Llama-1B is near chance on GSM8K/IFEval/etc. (§III-E). Report the excluded tasks at least in an appendix for the intervened models, or justify the filter with a pre-specified margin-above-chance rule.
  3. [§III-B, Table II] Salamandra pair dropout due to tokenizer mismatch (§III-B) and near-chance English MMLU (§V-A) weaken the cross-architecture claims; quantify how many pairs survive per category and avoid over-interpreting Salamandra capability retention (not fully evaluated per Table II).
  4. [§V-B, Figs. 1–3] Figures 1–3 are described clearly in text but depend on color/decile encodings; ensure grayscale-readable scales and define Jaccard threshold choices (Top-0.1/1/5%) in captions.
  5. [§II-B] Related work on Voria et al. [29] is used well to contrast directional vs unsigned attribution; a brief explicit comparison table (method, sign, model family, mitigation vs destabilization) would help readers.
  6. Minor polish: “V oria” spacing artifacts; consistent BiasScore vs bias score (BBQ metric) naming; arXiv-dated 2026 refs are fine if intentional but check metadata.

Circularity Check

0 steps flagged

No significant circularity: BiasScore is an activation contrast, not a BBQ fit; causal claims are checked on external benchmarks and are not forced by construction.

full rationale

The paper’s load-bearing chain is empirical, not definitional. BiasScore (Eq. 1) is the absolute mean activation difference at down_proj input on token-aligned contrastive pairs; it does not use BBQ/EsBBQ labels, stereotype signs, or capability scores. Candidate selection therefore does not bake in the evaluation outcomes. Zeroing is then tested on held-out-style generation prompts and on BBQ/EsBBQ and lm-eval harnesses; the reported bidirectional/non-monotonic bias shifts and ~99.49% capability retention are contingent measurements, not algebraic consequences of Eq. 1. The unsigned nature of BiasScore is used only to interpret why mitigation is not flat—an explanation of observed sign mixing, not a circular prediction. Self-references (OptiPFair, PPM, prior width-pruning dichotomy) supply tooling and motivation; FairnessPruningScore/PPM is explicitly unused in the small-K regime, and no uniqueness theorem or fitted parameter is smuggled in as a forced result. Mild category alignment between pair templates and BBQ axes is ordinary experimental design, not reduction of the claim to its inputs. Score 1 reflects only routine author tooling citation, not a circular derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 3 invented entities

The work is empirical-methodological. Load-bearing commitments are domain choices (explicit contrastive pairs, down_proj_input measurement, unsigned absolute mean difference, BBQ as primary bias read-out, small SwiGLU LLMs) plus a few hand-set intervention knobs (Top-K, α). It does not introduce new physical entities; ‘bias circuits’ are operationally defined by the score and ablations.

free parameters (4)
  • Top-K zeroing sizes (1, 5, 10, 20, 40) = grid in Table II; max 40 on 1B (~0.031% width)
    Hand-chosen intervention scales that define which experiments enter the causal and retention claims; not derived from a principled threshold.
  • FairnessPruningScore α = 0.8
    Mixture weight between BiasScore and (1−Importance); set to 0.8 in exploratory runs though final zeroing uses BiasScore alone.
  • Activation aggregation (per-sequence mean over positions) = mean over sequence positions
    Choice of µ(·) in Eq. 1 affects which neurons rank high; alternatives (max, last-token, integrated gradients) not ablated.
  • Capability benchmark inclusion filter = WikiText, MMLU, ARC-C, HellaSwag EN/ES
    Post-hoc restriction to five tasks where base Llama-1B is sufficiently above chance; changes the retention denominator.
axioms (6)
  • domain assumption Demographic bias relevant to intervention is detectable as differential GLU neuron activations on minimally contrastive, explicitly marked attribute pairs with equal token length.
    Foundation of the entire pipeline (§III-B–C); paper notes implicit-attribute and open-ended generalization are open.
  • domain assumption down_proj input (gate⊙up after SiLU) is the right scalar site to score per-neuron demographic sensitivity in SwiGLU MLPs.
    Motivated by GLU information flow (§III-C) and contrasted with more diffuse gate/up maps; not proven uniquely optimal.
  • domain assumption Zeroing gate/up rows and down column of selected indices is a valid causal intervention on that neuron’s contribution without architectural side effects that invalidate lm_eval comparisons.
    Pruning strategy §III-D; standard structured ablation assumption.
  • domain assumption BBQ/EsBBQ ambiguous-context bias scores are adequate primary metrics of stereotype tendency for validating localization.
    Evaluation framework §III-E; limitations §VI-D acknowledge MC and explicit-attribute limits.
  • standard math Standard transformer/GLU algebra and mean absolute difference as a sensitivity score (Eq. 1).
    Ordinary activation arithmetic; no exotic math.
  • ad hoc to paper Unsigned BiasScore candidate sets mix stereotype-amplifying and -suppressing neurons, so aggregate bias need not fall monotonically with K.
    Explanatory principle for bidirectional results (§V-C, §VI-A); inferred from outcomes rather than independently measured per-neuron signs.
invented entities (3)
  • BiasScore (unsigned differential-activation neuron score) independent evidence
    purpose: Rank neurons for demographic sensitivity without extra training or SAEs.
    Defined in Eq. 1; central selection mechanism. It is a metric, not a latent natural kind; causal status is tested only via group zeroing.
  • FairnessPruningScore no independent evidence
    purpose: Combine bias sensitivity with PPM structural importance for safer large-scale pruning.
    Eq. 2; largely unused in the reported small-K regime, so little empirical support here.
  • Per-category demographic regulator circuits in final MLP layers independent evidence
    purpose: Explain localized, mostly category-specific, depth-concentrated differential responses and high capability retention under ablation.
    Inferred from Top-K overlap, final-layer concentration, and zeroing effects (§V-B–C). Independent handle is partial (ablations + benchmarks), not a unique circuit proof.

pith-pipeline@v1.2.0-daily-grok45 · 26274 in / 3979 out tokens · 77794 ms · 2026-07-31T11:23:31.966604+00:00 · methodology

0 comments
read the original abstract

This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias in large language models (LLMs). As a foundational empirical validation of this method, this work focuses on causal bias localization. Using minimally contrastive prompt pairs and inference-time activation capture, the method identifies neurons that react differentially when processing demographic attributes in GLU architectures, evaluating the signal at the down_proj input. Empirical evaluation was conducted on models of up to 3 billion parameters (Llama-3.2 family and Salamandra-2B), combining standardized benchmark evaluation with qualitative text generation experiments. Results demonstrate that zeroing the identified neurons alters how the model responds to associated demographic variables. However, rather than producing flat mitigation, the intervention causes bidirectional bias destabilization: because BiasScore is unsigned, candidate sets mix neurons that push toward and against the stereotype, and the net effect on aggregate bias depends on which sign dominates. The intervention is extremely surgical: zeroing at most 40 neurons in Llama-3.2-1B (less than 0.031% of total MLP width) achieves a mean retention of 99.49% in reasoning and general knowledge capabilities. These findings empirically confirm that demographic bias processing and model capabilities operate on dissociable circuits, establishing the methodological foundations for transitioning from blind zeroing toward directional behavior modulation.

Figures

Figures reproduced from arXiv: 2607.28319 by Alfonso Ure\~na L\'opez, Eugenio Mart\'inez C\'amara, Pere Martra.

Figure 1
Figure 1. Figure 1: Mean bias signal at down_proj_input across relative model depth, for the three evaluated models in English. Each row represents a depth decile; each column a demographic category. The final layer (100%) consistently concentrates the highest signal across all models and categories. That the intersection is small and restricted to the final layers is consistent with the hypothesis of mostly independent per￾c… view at source ↗
Figure 2
Figure 2. Figure 2: Neuron bias heatmaps at gate_proj, up_proj, and down_proj_input for the Age category in English. Each cell represents the mean BiasScore of a bin of 512 neurons. The SwiGLU filter effect is visible in both models: the signal dispersed across many layers in gate_proj and up_proj concentrates in the final layer at down_proj_input. The contrast between models is marked: Llama-3.2-1B shows gradual accumulation… view at source ↗
Figure 3
Figure 3. Figure 3: Pairwise Jaccard overlap between demographic cate [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 29 linked inside Pith

  1. [1]

    On the dangers of stochastic parrots: Can language models be too big?

    E. M. Bender, T. Gebru, A. McMillan-Major, and M. Shmitchell, “On the dangers of stochastic parrots: Can language models be too big?” inProceedings of the 2021 ACM FAccT Conference on Fairness, Accountability, and Transparency, 2021, pp. 610–623

    2021

  2. [2]

    Gender bias in coreference resolution: Evaluation and debiasing methods,

    J. Zhao, T. Wang, M. Yatskar, V . Ordonez, and K.-W. Chang, “Gender bias in coreference resolution: Evaluation and debiasing methods,” inProceedings of NAACL-HLT 2018, 2018. [Online]. Available: https://arxiv.org/abs/1804.06876

    Pith/arXiv arXiv 2018

  3. [3]

    Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them,

    H. Gonen and Y . Goldberg, “Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them,”arXiv preprint, 2019. [Online]. Available: https://arxiv.org/abs/1903.03862

    Pith/arXiv arXiv 2019

  4. [4]

    Documenting large webtext corpora: A case study on the colossal clean crawled corpus,

    J. Dodge, M. Sap, A. Marasovi ´c, W. Agnew, G. Ilharco, D. Groeneveld, M. Mitchell, and M. Gardner, “Documenting large webtext corpora: A case study on the colossal clean crawled corpus,” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021. [Online]. Available: https://arxiv.org/abs/2104.08758

    Pith/arXiv arXiv 2021

  5. [5]

    Constitutional AI: Harmlessness from AI feedback,

    Y . Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, C. Chen, C. Olsson, C. Olah, D. Hernandez, D. Drain, D. Ganguli, D. Li, E. Tran-Johnson, E. Perez, and et al., “Constitutional AI: Harmlessness from AI feedback,” 2022. [Online]. Available: https://arxiv.org/abs/2212.08073

    Pith/arXiv arXiv 2022

  6. [6]

    Null it out: Guarding protected attributes by iterative nullspace projection,

    S. Ravfogel, Y . Elazar, H. Gonen, M. Twiton, and Y . Goldberg, “Null it out: Guarding protected attributes by iterative nullspace projection,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 7237–7256. [Online]. Available: https://aclanthology.org/2020.acl-main.647/

    2020

  7. [7]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” 2022. [Online]. Available: https://arxiv.org/abs/2203.02155

    Pith/arXiv arXiv 2022

  8. [8]

    Towards monosemanticity: Decomposing language models with dictionary learning,

    T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y . Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, A. Tamkin, K. Nguyen, B. McLean, and et al., “Towards monosemanticity: Decomposing language models with dictionary learning,” Anthropic Transformer Circuits Research, 2023. [Onli...

    2023

  9. [9]

    Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet,

    A. Templeton, T. Conerly, J. Marcus, J. Lindsey, T. Bricken, B. Chen, A. Pearce, C. Citro, E. Durbin, C. McDougall, D. Hernandez, A. Jermyn, C. Lee, A. Jones, N. Shaul, and et al., “Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet,” Anthropic Technical Report, 2024. [Online]. Available: https://transformer-circuits.pub/2024/...

    2024

  10. [10]

    Fragile knowledge, robust instruction-following: The width pruning dichotomy in Llama-3.2,

    P. Martra, “Fragile knowledge, robust instruction-following: The width pruning dichotomy in Llama-3.2,” 2025. [Online]. Available: https://arxiv.org/abs/2512.22671

    Pith/arXiv arXiv 2025

  11. [11]

    Investigating gender bias in language models using causal mediation analysis,

    J. Vig, S. Gehrmann, Y . Belinkov, S. Qian, D. Nevo, Y . Singer, and S. Shieber, “Investigating gender bias in language models using causal mediation analysis,” pp. 12 388–12 401, 2020

    2020

  12. [12]

    Transformer feed- forward layers are key-value memories,

    M. Geva, R. Schuster, J. Berant, and O. Levy, “Transformer feed- forward layers are key-value memories,” 2020. [Online]. Available: https://arxiv.org/abs/2012.14913

    Pith/arXiv arXiv 2020

  13. [13]

    Language (technology) is power: A critical survey of “bias

    S. L. Blodgett, S. Barocas, H. Daum ´e III, and H. Wallach, “Language (technology) is power: A critical survey of “bias” in NLP,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 5454–5476. [Online]. Available: https://arxiv.org/abs/2005.14050

    Pith/arXiv arXiv 2020

  14. [14]

    BBQ: A hand-built bias benchmark for question answering,

    A. Parrish, A. Chen, N. Nangia, V . Padmakumar, J. Phang, J. Thompson, P. M. Htut, and S. R. Bowman, “BBQ: A hand-built bias benchmark for question answering,” inFindings of the Association for Computational Linguistics: ACL 2022, 2022. [Online]. Available: https://arxiv.org/abs/2110.08193

    Pith/arXiv arXiv 2022

  15. [15]

    Evaluating and mitigating social bias for large language models in open-ended settings,

    Z. Liu, T. Xie, and X. Zhang, “Evaluating and mitigating social bias for large language models in open-ended settings,” 2024. [Online]. Available: https://arxiv.org/abs/2412.06134

    arXiv 2024

  16. [16]

    “the dentist is an involved parent, the bartender is not

    A. Wagh and S. Srivastava, ““the dentist is an involved parent, the bartender is not”: Revealing implicit biases in QA with implicit BBQ,”

  17. [17]

    EsBBQ and CaBBQ: The spanish and catalan bias benchmarks for question answering,

    V . Ruiz-Fern ´andez, M. Mina, J. Falc ˜ao, L. Vasquez-Reina, A. Sall ´es, A. Gonzalez-Agirre, and O. Perez-de Vi ˜naspre, “EsBBQ and CaBBQ: The spanish and catalan bias benchmarks for question answering,”

  18. [18]

    Textbooks are all you need,

    S. Gunasekar, Y . Zhang, J. Aneja, C. C. T. Mendes, A. Del Giorno, S. Gopi, M. Javaheripi, P. Kauffmann, G. De Rosa, O. Saarikivi, A. Salim, S. Shah, H. S. Behl, X. Wang, S. Bubeck, R. Eldan, A. T. Kalai, Y . T. Lee, and Y . Li, “Textbooks are all you need,” 2023. [Online]. Available: https://arxiv.org/abs/2306.11644

    Pith/arXiv arXiv 2023

  19. [19]

    Available: https://arxiv.org/abs/2507.11216

    [Online]. Available: https://arxiv.org/abs/2507.11216

    Pith/arXiv arXiv

  20. [20]

    Probing classifiers are unreliable for concept removal and detection,

    A. Kumar, C. Tan, and A. Sharma, “Probing classifiers are unreliable for concept removal and detection,” 2022. [Online]. Available: https://arxiv.org/abs/2207.04153

    Pith/arXiv arXiv 2022

  21. [21]

    An empirical study of catastrophic forgetting in large language models during continual fine-tuning,

    Y . Luo, Z. Yang, F. Meng, Y . Li, J. Zhou, and Y . Zhang, “An empirical study of catastrophic forgetting in large language models during continual fine-tuning,” 2023. [Online]. Available: https://arxiv.org/abs/2308.08747

    Pith/arXiv arXiv 2023

  22. [22]

    Steering language models with activation engineering,

    A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid, “Steering language models with activation engineering,” 2023. [Online]. Available: https://arxiv.org/abs/2308.10248

    Pith/arXiv arXiv 2023

  23. [23]

    CausaLM: Causal model explanation through counterfactual language models,

    A. Feder, N. Oved, U. Shalit, and R. Reichart, “CausaLM: Causal model explanation through counterfactual language models,”Computational Linguistics, vol. 47, no. 2, pp. 333–386, Jun. 2021. [Online]. Available: https://doi.org/10.1162/coli a 00404

    doi:10.1162/coli 2021

  24. [24]

    SparseGPT: Massive language models can be accurately pruned in one shot,

    E. Frantar and D. Alistarh, “SparseGPT: Massive language models can be accurately pruned in one shot,” 2023. [Online]. Available: https://arxiv.org/abs/2301.00774

    Pith/arXiv arXiv 2023

  25. [25]

    Representation engineering: A top-down approach to AI transparency,

    A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, J. Z. Kolter, and D. Hendrycks, “Representation engineering: A top-down approach to AI transparency,” 2023. [Online]. Available: https://arxiv.org/abs/2310.01405

    Pith/arXiv arXiv 2023

  26. [26]

    LLM-Pruner: On the structural pruning of large language models,

    X. Ma, G. Fang, and X. Wang, “LLM-Pruner: On the structural pruning of large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2305.11627

    Pith/arXiv arXiv 2023

  27. [27]

    A simple and effective pruning approach for large language models,

    M. Sun, Z. Liu, A. Bair, and J. Z. Kolter, “A simple and effective pruning approach for large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2306.11695

    Pith/arXiv arXiv 2023

  28. [28]

    A mathematical framework for transformer circuits,

    N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y . Bai, A. Chen, T. Conerly, N. DasSarma, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, and et al., “A mathematical framework for transformer circuits,”Transformer Circuits Thread, 2021. [Online]. Available: https://transforme...

    2021

  29. [29]

    CompactifAI: Extreme compression of large language models using quantum-inspired tensor networks,

    A. Tomut, S. S. Jahromi, A. Sarkar, U. Kurt, S. Singh, F. Ishtiaq, C. Mu ˜noz, P. S. Bajaj, A. Elborady, G. Del Bimbo, M. Alizadeh, D. Montero, P. Martin-Ramiro, M. Ibrahim, O. T. Alaoui, J. Malcolm, S. Mugel, and R. Orus, “CompactifAI: Extreme compression of large language models using quantum-inspired tensor networks,” 2024. [Online]. Available: https:/...

    Pith/arXiv arXiv 2024

  30. [30]

    Knowledge neurons in pretrained transformers,

    D. Dai, L. Dong, Y . Hao, Z. Sui, B. Chang, and F. Wei, “Knowledge neurons in pretrained transformers,” 2021. [Online]. Available: https://arxiv.org/abs/2104.08696

    Pith/arXiv arXiv 2021

  31. [31]

    Tracing stereotypes in pre-trained transformers: From biased neurons to fairer models,

    G. V oria, M. Openja, F. Khomh, G. Catolino, and F. Palomba, “Tracing stereotypes in pre-trained transformers: From biased neurons to fairer models,” 2026. [Online]. Available: https://arxiv.org/abs/2601.05663

    arXiv 2026

  32. [32]

    A framework for few-shot language model evaluation,

    L. Gao, J. Tow, B. Abbasi, S. Biderman, S. Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, A. Le Noac’h, H. Li, K. McDonell, N. Muennighoff, C. Ociepa, J. Phang, L. Reynolds, H. Schoelkopf, A. Skowron, L. Sutawika, E. Tang, A. Thite, B. Wang, K. Wang, and A. Zou, “A framework for few-shot language model evaluation,” 2023. [Online]. Available: https://zen...

    arXiv 2023

  33. [33]

    Optipfair: Structured pruning and knowledge distillation for large language models,

    P. Martra, “Optipfair: Structured pruning and knowledge distillation for large language models,” 2025. [Online]. Available: https://github.com/ peremartra/optipfair

    2025

  34. [34]

    Instruction-following evaluation for large language models,

    J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y . Luan, D. Zhou, and L. Hou, “Instruction-following evaluation for large language models,”

  35. [35]

    Training verifiers to solve math word problems,

    K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman, “Training verifiers to solve math word problems,” 2021. [Online]. Available: https://arxiv.org/abs/2110.14168

    Pith/arXiv arXiv 2021

  36. [36]

    The Belebele benchmark: A parallel reading comprehension dataset in 122 language variants,

    L. Bandarkar, D. Liang, B. Muller, M. Artetxe, S. N. Shukla, D. Husa, N. Goyal, A. Krishnan, L. Zettlemoyer, and M. Khabsa, “The Belebele benchmark: A parallel reading comprehension dataset in 122 language variants,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024. [Online]. Available: https://arxiv.org/abs/...

    Pith/arXiv arXiv 2024

  37. [37]

    TruthfulQA: Measuring how models mimic human falsehoods,

    S. Lin, J. Hilton, and O. Evans, “TruthfulQA: Measuring how models mimic human falsehoods,” 2021. [Online]. Available: https: //arxiv.org/abs/2109.07958

    Pith/arXiv arXiv 2021

  38. [38]

    Think you have solved question answering? try ARC, the AI2 reasoning challenge,

    P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try ARC, the AI2 reasoning challenge,” 2018. [Online]. Available: https://arxiv.org/abs/1803.05457

    Pith/arXiv arXiv 2018

  39. [39]

    Measuring massive multitask language understanding,

    D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,”

  40. [40]

    HellaSwag: Can a machine really finish your sentence?

    R. Zellers, A. Holtzman, Y . Bisk, A. Farhadi, and Y . Choi, “HellaSwag: Can a machine really finish your sentence?” 2019. [Online]. Available: https://arxiv.org/abs/1905.07830

    Pith/arXiv arXiv 2019

  41. [41]

    Pointer sentinel mixture models,

    S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer sentinel mixture models,” 2016. [Online]. Available: https://arxiv.org/abs/1609. 07843

    2016

  42. [42]

    Salamandra technical report,

    A. Gonzalez-Agirre, M. P `amies, J. Llop, I. Baucells, S. Da Dalt, D. Tamayo, J. J. Saiz, F. Espu ˜na, J. Prats, J. Aula-Blasco, M. Mina, I. Pikabea, A. Rubio, A. Shvets, A. Sall ´es, I. Lacunza, J. Palomar, J. Falc ˜ao, L. Tormo, and et al., “Salamandra technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2502.08489

    Pith/arXiv arXiv 2025

  43. [43]

    Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space,

    M. Geva, A. Caciularu, K. R. Wang, and Y . Goldberg, “Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space,” inProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022. [Online]. Available: https://arxiv.org/abs/2203.14680

    Pith/arXiv arXiv 2022

  44. [45]

    Llama 3.2: Revolutionizing edge AI and vision with open, customizable models,

    Meta AI, “Llama 3.2: Revolutionizing edge AI and vision with open, customizable models,” Meta AI Blog, 2024, accessed: 31 May 2026. [Online]. Available: https://ai.meta.com/ blog/llama-3-2-connect-2024-vision-edge-mobile-devices/

    2024

  45. [2020]

    Available: https://arxiv.org/abs/2009.03300

    [Online]. Available: https://arxiv.org/abs/2009.03300

    Pith/arXiv arXiv 2009

  46. [2023]

    Available: https://arxiv.org/abs/2311.07911

    [Online]. Available: https://arxiv.org/abs/2311.07911

    Pith/arXiv arXiv

  47. [2025]

    Available: https://arxiv.org/abs/2512.06732

    [Online]. Available: https://arxiv.org/abs/2512.06732

    arXiv