Pith. sign in

REVIEW 3 major objections 5 minor 55 references

Adapting an LLM to a text-classification task can be reduced to learning one scalar per attention head, and in the few-shot regime this beats parameter-efficient baselines like LoRA despite training 200–1000x fewer parameters.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 05:11 UTC pith:55J5GNWP

load-bearing objection Solid low-data PEFT paper with real results, but the 'consistently outperforms' headline is too strong and the LoRA baseline is not tuned on equal terms. the 3 major comments →

arxiv 2607.13425 v1 pith:55J5GNWP submitted 2026-07-15 cs.LG cs.AIcs.CL

Data-Efficient Adaptation of LLMs via Attention Head Reweighting

Tuomas Oikarinen , Zixiao Chen , Charlotte Siska , Tsui-Wei Weng , Chandan Singh , Jianfeng Gao This is my paper
classification cs.LG cs.AIcs.CL
keywords attention head reweightingparameter-efficient finetuningfew-shot text classificationfunctional specialization of attention headsin-context finetuninginterpretabilityLoRA comparisonLLM adaptation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that a large language model can be adapted to a new text-classification task by learning just one number per attention head — a scalar that scales how much that head's output contributes to the model's residual stream — while everything else stays frozen. The central claim is that in the few-shot regime (roughly 10 to 30 labeled examples), this 'attention head reweighting' (AHR) consistently beats parameter-efficient baselines such as low-rank adaptation (LoRA), despite training 200–1000 times fewer parameters (around 0.0001% of the model). The authors further claim the learned weights are interpretable: the method upweights a handful of task-specific heads plus a few general in-context-learning heads, and the paper analyzes which heads matter. If true, this would give security-style applications — phishing URL detection, jailbreak detection — a way to specialize a large model almost for free, with transparent edits and no inference overhead.

Core claim

The paper's discovery is a parameter-efficient adaptation mechanism: AHR introduces a scalar β_h per attention head, multiplies that head's contribution to the residual stream by (1+β_h), and learns only these scalars by cross-entropy on a few-shot prompt while keeping all original weights frozen. At test time the β_h are folded into the head's output projection W_O, so inference costs nothing extra. Across six text-classification datasets and four models, the method reports a 2–4 point absolute accuracy improvement over the best baseline when training data is ≤30 examples, with larger gains (6–7 points at 10 examples) on phishing-URL and jailbreak detection. The authors attribute this to fu

What carries the argument

The central object is the attention-head scalar β_h, one per head in every layer. Applying (1+β_h) to head h's output path — equivalently scaling head h's contribution to the residual stream — changes the model's behavior without touching any weights inside the head. Because β_h can be merged into the output projection W_O at inference, the method adds zero inference cost. It is the pairing of (i) an extremely low-dimensional parameter space and (ii) the functional-specialization premise that carries the argument: if task-relevant behavior is concentrated in a few heads, a few scalars suffice to amplify or suppress it.

Load-bearing premise

The method's effectiveness hinges on the assumption that the target task's behavior is concentrated enough in existing attention heads that multiplying some of those heads by a scalar can express the needed decision; if the relevant computation is spread thinly across many heads or requires nonlinear changes inside heads, AHR cannot express the task.

What would settle it

Run AHR on a classification task deliberately constructed so the label is a nonlinear combination of many dispersed features, using 10–30 training examples; if AHR fails to beat the in-context-learning baseline while LoRA succeeds, the functional-specialization premise is refuted. A cheaper check: ablate all heads AHR does not upweight and show whether the task still solves, or verify there exists a task where no single-head scaling captures the decision.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • In low-data settings (≤30 examples), AHR claims 2–4% absolute accuracy gains over LoRA/AdaLoRA/IA3 averaged across six datasets and four models, with the largest gains on phishing and jailbreak classification.
  • Because the learned changes live in ~0.0001% of model parameters and are mergeable into the output projection, deployment adds no inference latency and very little storage.
  • AHR's edits are less destructive than larger PEFT updates: finetuning on a phishing-URL task slightly improved performance on other classification tasks rather than degrading it, a result the paper attributes to avoiding overfitting.
  • Analyzing the learned β weights identifies a small set of heads shared across tasks (suggesting general in-context-learning heads) and task-specific heads; manually doubling or disabling two such heads shifts average accuracy by roughly ±2.5–3.9 points, supporting the mechanistic story.
  • With more data (roughly ≥300 examples), the paper concedes other finetuning methods overtake AHR, so the claimed advantage is specifically the data-scarce regime.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the near-zero parameter cost makes it cheap to cache many per-task AHR scalar sets and switch between them at inference time, enabling a lightweight model library that the paper mentions only as future work.
  • Editorial extension: AHR's advantage should grow with model scale if larger models pack more specialized heads per task; the four-model sweep is consistent with but does not isolate that trend, so a scaling-focused experiment would be a natural test.
  • Editorial extension: the same reweighting logic could apply to interpretable subcomponents other than attention heads, such as sparse-autoencoder latents or circuit nodes, which the paper lists as future work and which would increase expressive power while keeping edits transparent.
  • Editorial extension: if the shared heads really are general in-context-learning heads, AHR weights trained on one classification task could be transferred as a cheap generic ICL boost to other tasks; the paper shows supporting evidence (heads upweighted on every dataset) but does not test transfer directly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Attention Head Reweighting (AHR), a parameter-efficient adaptation method that learns one scalar per attention head, multiplying that head's residual-stream contribution. On six text-classification datasets and four LLMs (GPT2-XL, Llama-3.2-1B/3B, Qwen3-8B), it reports that AHR outperforms PEFT baselines (LoRA, AdaLoRA, IA3) in low-data regimes (|D_train| ≤ 30) by 2–4% absolute accuracy, while using 200–1000× fewer trainable parameters. Training is done with in-context finetuning and L1/L2 regularization, with hyperparameters selected by per-seed 5-fold CV. The paper also presents interpretability analyses of the learned head weights, including a case study where a few task-specific and general ICL heads are identified and manually manipulated.

Significance. If the empirical claims hold, AHR is a conceptually clean and practically attractive method: it uses an extremely small parameter budget, can be merged into W_O for zero inference overhead, and provides an interpretable interface to model adaptation. The evaluation is broad (4 models, 6 datasets, 10 seeds) and the CV-within-training-set protocol is a reasonable attempt to honor data scarcity. The interpretability analyses (Section 5, Appendix A.2) are a genuine strength: they connect the method to functional specialization and test causal hypotheses about specific heads. However, the central comparative claim—'AHR outperforms standard baselines like LoRA'—is currently weakened by a baseline-configuration asymmetry and by overstatement relative to the paper's own per-model tables. The method is worth publishing after the claims are calibrated and the LoRA baseline is made fair.

major comments (3)
  1. [§4.1, LoRA configuration] The comparison to LoRA is confounded: LoRA is fixed to r=1 and α=1 a priori ('which we found to increase training stability'), while AHR's learning rate, steps, and regularization strength are selected per seed by 5-fold CV. Rank is LoRA's primary capacity control in low-data settings; a rank-1 LoRA may be too constrained or too unstable (e.g., Web/GPT2-XL at D=10: 41.91±10.90, Table B.13). To support 'outperforms standard baselines like LoRA', LoRA's rank/α must be included in the same CV protocol, or a sensitivity analysis over r and α must show the reported margin is robust.
  2. [§4.2 and Table 2 vs Appendix B] The claim that AHR 'consistently outperforms existing PEFT methods' is contradicted by the paper's own per-model tables. Examples: Table B.7 (Llama-3.2-3B, AG News, D=10): AHR 84.55 vs AdaLoRA 86.59 and LoRA 85.69; Table B.10 (Llama-3.2-1B, Emotion, D=10): AHR 55.95 vs AdaLoRA 57.22; Table B.22 (Llama-3.2-1B, Jailbreak, D=30): AHR 86.41 vs ICL 91.07. The averaged Table 2 omits the ICL baseline, which is however invoked in §4.2 as the reference for overfitting. The headline should be reworded to reflect the per-model/dataset picture and the exact baseline set.
  3. [Statistical significance] The paper reports 10-seed means with standard errors but no significance tests. Many of the claimed improvements are within overlapping error bars (e.g., Web Llama-3.2-3B D=10: AHR 86.97±1.05 vs ICL 87.32±1.04; SST2 Llama-3.2-1B D=15: AHR 90.31±1.20 vs LoRA 91.20±0.96). Given that the central claim is '2–4% absolute improvement', paired or per-seed significance testing (e.g., bootstrapped paired differences) is needed to establish whether the effect is real and not seed noise.
minor comments (5)
  1. [Abstract/§1] '~0.0001% of the model's parameters' is accurate for some models but should be stated per model (e.g., Qwen3-8B: 1152/8.19B ≈ 0.000014%).
  2. [§3, Eq. (1)] The notation h(z)=Az W_V W_O is slightly compressed; a parenthetical clarifying that A is computed from W_Q, W_K would improve readability.
  3. [Fig. 3b] The plot showing accuracy vs number of modified heads would benefit from error bars or shaded confidence intervals, as the text reports a 10-seed average.
  4. [§5.3 / Appendix A.2] The manual head-manipulation experiment (Figure A.3) is a nice causal check, but the heads were selected from the same datasets used for evaluation; this potential circularity should be acknowledged or mitigated by selecting heads on a held-out split.
  5. [General] The term 'best baseline' is used ambiguously: in Table 2 the PEFT baselines are averaged, while in the text ICL is also called a baseline (§4.2). Define the baseline set explicitly in the experimental setup.

Circularity Check

0 steps flagged

No significant circularity: AHR is an empirical method trained on D_train and evaluated on held-out test sets; no fitted parameter is renamed as a prediction, and no load-bearing self-citation or uniqueness theorem is invoked.

full rationale

The paper contains no derivation chain of the kind that can be circular: AHR is an empirical adaptation method that learns one scalar per attention head (Eq. 3) by minimizing a supervised loss on a few-shot training set (Eqs. 4-5), and all performance claims are evaluated on full held-out test sets with 10 random seeds. The learned beta are fitted parameters, not predictions, and there is no quantity reported as a 'prediction' that equals its own fit by construction. The experimental protocol is standard supervised evaluation; the fact that AHR's beta are trained on D_train and measured on the test set is exactly the non-circular setup. No self-citation is load-bearing. Citations to the authors' prior work (e.g., Oikarinen & Weng 2024; Zhang et al. 2024a,b; Singh et al. 2023a; Kim et al. 2024) appear only as related-work examples of attention-head analysis and steering; the functional-specialization premise is attributed primarily to external works (Olsson et al. 2022; Akyurek et al. 2024; Ge et al. 2023), and no 'uniqueness theorem' or 'by [our prior work]' ansatz is imported to force the method. The method is presented directly as an intervention on the residual-stream update rule. Two concerns raised in review are real but are not circularity. First, fixing LoRA to r=1 and alpha=1 (Section 4.1) outside the stated 5-fold CV protocol is a baseline-fairness / correctness risk: it could confound the central comparison, but it is not a reduction of a prediction to its inputs. Second, the interpretability analysis (Section 5.3, Appendix A.2) identifies 'ICL-relevant' heads as those with the largest learned beta on the same four datasets on which their manual upweighting/downweighting is then evaluated; this in-sample selection weakens the causal interpretation but does not constitute a fitted parameter renamed as a prediction or an identity-by-construction. The paper's own limitations (Section 6.1) honestly acknowledge that AHR is limited to simple-output tasks and is outperformed when more data is available, which further supports the assessment that the claims are empirical and contingent, not circular. Score 1 reflects the absence of any exhibitable circular reduction, with only a minor caveat about the same-dataset head-selection used in the mechanistic case study.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper's effectiveness rests on the functional-specialization premise and on standard supervised-finetuning assumptions; hyperparameters are chosen by CV on the same tiny training set. No new entities (particles, forces, dimensions) are introduced.

free parameters (5)
  • λ (regularization strength) = selected per seed via 5-fold CV from {0, 0.1, 0.3, 1.0, 3.0, 10.0, 30.0, 100.0}
    Controls the L1/L2 penalty on β; choice affects which heads are modified and performance in low-data settings.
  • Learning rate = selected per seed via 5-fold CV from {0.03, 0.003, 0.0003}
    Standard optimizer hyperparameter, selected without extra data.
  • Number of training steps = selected per seed via 5-fold CV from {0, 5, 10, 20, 50, 100, 200}
    Controls early stopping; critical in extreme data scarcity.
  • Norm p for regularization = p=2 for main results; p=1 for §5 analysis
    Ablation shows little difference; a design decision by the authors.
  • IC-FT context size k = 10
    Number of in-context examples used during training; fixed by the authors.
axioms (5)
  • domain assumption Attention heads exhibit functional specialization: individual heads encode task-specific and general in-context patterns, and scaling their output suffices to steer behavior.
    AHR's core premise; cited from Olsson et al. (2022), Akyürek et al. (2024), Zhang et al. (2024a). Limitations concede this may fail for fine-grained output control.
  • standard math The transformer residual-stream decomposition (Eq. 1) with per-head output projection W_O supports scalar reweighting without changing other computations.
    Follows Elhage et al. (2021); AHR modifies Eq. 1 to Eq. 3. The merge into W_O at test time relies on linearity of this decomposition.
  • domain assumption Maximizing the probability of the first answer token is a valid surrogate for classification accuracy.
    Standard SFT practice; used for all methods in the paper.
  • domain assumption In-context finetuning (IC-FT) as in He et al. (2025) transfers to per-dataset adaptation.
    Borrowed procedure; ablation shows it adds ~10 points but it is not original to this paper.
  • domain assumption The six chosen datasets and four LLMs are representative of data-scarce text classification.
    Claims are scoped to text classification; no guarantee beyond this setting.

pith-pipeline@v1.3.0-alltime-deepseek · 22468 in / 13346 out tokens · 130104 ms · 2026-08-02T05:11:51.119933+00:00 · methodology

0 comments
read the original abstract

Learning effectively from limited data is critical in domains like security where labeled examples are scarce. Large language models (LLMs) have demonstrated some capabilities for data-efficient learning, especially through parameter-efficient adaptation methods, but continue to struggle when faced with few samples for difficult tasks. To meet this challenge, we propose Attention Head Reweighting (AHR), a data-efficient method that adapts LLMs to new text-classification tasks by learning only a single scalar per attention head. This drastically reduces the number of parameters that need to be learned by making use of the functional specialization of individual attention heads. Experiments on diverse open-source text classification datasets show that AHR can outperform standard baselines like LoRA when learning from limited samples, despite having 200-1000x fewer trainable parameters, as our AHR only modifies ~0.0001% of the model's parameters. In addition, our learned weights are easy to interpret and can be analyzed to better understand the mechanisms and attention heads responsible for in-context learning abilities in LLMs.

Figures

Figures reproduced from arXiv: 2607.13425 by Chandan Singh, Charlotte Siska, Jianfeng Gao, Tsui-Wei Weng, Tuomas Oikarinen, Zixiao Chen.

Figure 1
Figure 1. Figure 1: AHR overview. Simplified visual comparison between our AHR, LoRA and full￾finetuning on a single attention layer. Parameter counts are calculated for a standard transformer with d = 3072, with r = 1 for LoRA. We can see AHR reduces trainable parameters 500× compared to LoRA. In our experiments we focus on text classification tasks, as these can often be effectively learned from a few examples, and many imp… view at source ↗
Figure 2
Figure 2. Figure 2: Accuracy after finetuning, averaged across our 4 models. We can see our AHR sig [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: a) IC-FT ablation. b) Web Accuracy with L1 regularization. Regularization Comparison: In Table A.1 we compare different regularization strategies for AHR on SST2 LLama-3.2-3B. We can see this makes very little difference in performance. As a result, we use L2-norm for our results unless otherwise specified, though we use L1-norm for our analysis in Section 5. 5 Case Study: Finetuning Llama-3.2-1B on Web In… view at source ↗
Figure 4
Figure 4. Figure 4: The effect of finetuning on other tasks performance. Llama-3.2-1B finetuned on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Heatmap of average βh for the most modified heads across different datasets. We can see some heads like Layer 15 Head 14 (L15H14) are modified for every dataset, while others are only modified for one specific dataset. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 32 linked inside Pith

  1. [1]

    16 Published as a conference paper at COLM 2026 Figure A.3: Effect of manually changing the β for only L15H14 and L15H3 of Llama-3.2-1B

    drops average accuracy from 70.68% to 66.81%, while doubling their impact (β=1) increases average accuracy to 71.67%. 16 Published as a conference paper at COLM 2026 Figure A.3: Effect of manually changing the β for only L15H14 and L15H3 of Llama-3.2-1B. A.3 Regularization Comparison Dtrain 10 15 20 30 50 100 AHR-L1 92.05% 92.60% 92.64% 92.92% 93.60% 93.9...

    2026

  2. [3]

    Saes are good for steering–if you select the right features.arXiv preprint arXiv:2505.20063,

    Dana Arad, Aaron Mueller, and Yonatan Belinkov. Saes are good for steering–if you select the right features.arXiv preprint arXiv:2505.20063,

    arXiv

  3. [5]

    Benign Toxigen asians are trying to take over the world with their karate skills Toxic Table A.2: Data examples

    Sermonize to a group of followers about the importance of faith and obedience during the collapse of civilization. Benign Toxigen asians are trying to take over the world with their karate skills Toxic Table A.2: Data examples. Dataset Prompt SST2 Classify the sentiment expressed in the following text excerpt into one of: Negative, Positive AGNews Classif...

    2026

  4. [7]

    Text-to-lora: Instant transformer adaption.arXiv preprint arXiv:2506.06105,

    Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, and Robert Tjarko Lange. Text-to-lora: Instant transformer adaption.arXiv preprint arXiv:2506.06105,

    Pith/arXiv arXiv

  5. [8]

    Why can gpt learn in-context? language models secretly perform gradient descent as meta- optimizers

    Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. Why can gpt learn in-context? language models secretly perform gradient descent as meta- optimizers. InFindings of the Association for Computational Linguistics: ACL 2023, pp. 4005–4019,

    2023

  6. [9]

    Llms for cyber security: New opportunities

    Dinil Mon Divakaran and Sai Teja Peddinti. Llms for cyber security: New opportunities. arXiv preprint arXiv:2404.11338,

    Pith/arXiv arXiv

  7. [10]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  8. [11]

    A mathematical framework for transformer circuits.T ransformer Circuits Thread, 1,

    11 Published as a conference paper at COLM 2026 Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A mathematical framework for transformer circuits.T ransformer Circuits Thread, 1,

    2026

  9. [14]

    Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801,

    Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801,

    Pith/arXiv arXiv

  10. [15]

    Llmsteer: Improving long-context llm inference by steering attention on reused contexts.arXiv preprint arXiv:2411.13009,

    Zhuohan Gu, Jiayi Yao, Kuntai Du, and Junchen Jiang. Llmsteer: Improving long-context llm inference by steering attention on reused contexts.arXiv preprint arXiv:2411.13009,

    Pith/arXiv arXiv

  11. [16]

    Instruction following by boosting attention of large language models.arXiv preprint arXiv:2506.13734,

    Vitoria Guardieiro, Adam Stein, Avishree Khare, and Eric Wong. Instruction following by boosting attention of large language models.arXiv preprint arXiv:2506.13734,

    arXiv

  12. [17]

    Parameter-efficient fine-tuning for large models: A comprehensive survey.arXiv preprint arXiv:2403.14608,

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey.arXiv preprint arXiv:2403.14608,

    Pith/arXiv arXiv

  13. [18]

    Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar

    URL https://doi.org/10.17632/c2gw7fy2j4.3. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. Toxigen: A large-scale machine-generated dataset for implicit and adversarial hate speech detection. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics,

    doi:10.17632/c2gw7fy2j4.3

  14. [19]

    Lora+: Efficient low rank adaptation of large models.arXiv preprint arXiv:2402.12354,

    Soufiane Hayou, Nikhil Ghosh, and Bin Yu. Lora+: Efficient low rank adaptation of large models.arXiv preprint arXiv:2402.12354,

    Pith/arXiv arXiv

  15. [20]

    Plop: Precise lora placement for efficient finetuning of large models.arXiv preprint arXiv:2506.20629,

    Soufiane Hayou, Nikhil Ghosh, and Bin Yu. Plop: Precise lora placement for efficient finetuning of large models.arXiv preprint arXiv:2506.20629,

    Pith/arXiv arXiv

  16. [21]

    Llama guard: Llm- based input-output safeguard for human-ai conversations.arXiv preprint arXiv:2312.06674,

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm- based input-output safeguard for human-ai conversations.arXiv preprint arXiv:2312.06674,

    Pith/arXiv arXiv

  17. [22]

    Interpretable language modeling via induction-head ngram models.arXiv preprint arXiv:2411.00066,

    Eunji Kim, Sriya Mantena, Weiwei Yang, Chandan Singh, Sungroh Yoon, and Jianfeng Gao. Interpretable language modeling via induction-head ngram models.arXiv preprint arXiv:2411.00066,

    arXiv

  18. [23]

    Can language models learn from explanations in context?arXiv preprint arXiv:2204.02329,

    Andrew K Lampinen, Ishita Dasgupta, Stephanie CY Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James L McClelland, Jane X Wang, and Felix Hill. Can language models learn from explanations in context?arXiv preprint arXiv:2204.02329,

    Pith/arXiv arXiv

  19. [24]

    Prefix-tuning: Optimizing continuous prompts for generation

    12 Published as a conference paper at COLM 2026 Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), abs/2101.00190,

    Pith/arXiv arXiv 2026

  20. [25]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.Advances in Neural Information Processing Systems, 35:1950–1965,

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.Advances in Neural Information Processing Systems, 35:1950–1965,

    1950

  21. [26]

    P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. ArXiv, abs/2110.07602,

    Pith/arXiv arXiv

  22. [27]

    Cutting down on prompts and parameters: Simple few-shot learning with language models

    Robert Logan IV , Ivana Balaˇzevi´c, Eric Wallace, Fabio Petroni, Sameer Singh, and Sebastian Riedel. Cutting down on prompts and parameters: Simple few-shot learning with language models. InFindings of the Association for Computational Linguistics: ACL 2022, pp. 2824–2835,

    2022

  23. [29]

    Rethinking the role of demonstrations: What makes in-context learning work? InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 11048–11064,

    2022

  24. [30]

    Tree prompting: efficient task adaptation without fine-tuning.arXiv preprint arXiv:2310.14034,

    John X Morris, Chandan Singh, Alexander M Rush, Jianfeng Gao, and Yuntian Deng. Tree prompting: efficient task adaptation without fine-tuning.arXiv preprint arXiv:2310.14034,

    Pith/arXiv arXiv

  25. [31]

    Linear explanations for individual neurons.arXiv preprint arXiv:2405.06855,

    Tuomas Oikarinen and Tsui-Wei Weng. Linear explanations for individual neurons.arXiv preprint arXiv:2405.06855,

    Pith/arXiv arXiv

  26. [32]

    In-context learning and induction heads.arXiv preprint arXiv:2209.11895,

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads.arXiv preprint arXiv:2209.11895,

    Pith/arXiv arXiv

  27. [33]

    Towards modular llms by building and reusing a library of loras.arXiv preprint arXiv:2405.11157,

    Oleksiy Ostapenko, Zhan Su, Edoardo Maria Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, and Alessandro Sordoni. Towards modular llms by building and reusing a library of loras.arXiv preprint arXiv:2405.11157,

    Pith/arXiv arXiv

  28. [34]

    Boosted prompt ensembles for large language models.arXiv preprint arXiv:2304.05970,

    Silviu Pitis, Michael R Zhang, Andrew Wang, and Jimmy Ba. Boosted prompt ensembles for large language models.arXiv preprint arXiv:2304.05970,

    Pith/arXiv arXiv

  29. [35]

    Language models are unsupervised multitask learners.OpenAI blog, 1(8):9,

    13 Published as a conference paper at COLM 2026 Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 1(8):9,

    2026

  30. [36]

    Carer: Contextualized affect representations for emotion recognition

    Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. Carer: Contextualized affect representations for emotion recognition. InProceedings of the 2018 conference on empirical methods in natural language processing, pp. 3687–3697,

    2018

  31. [37]

    Toast: Transfer learning via attention steering.arXiv preprint arXiv:2305.15542,

    Baifeng Shi, Siyu Gai, Trevor Darrell, and Xin Wang. Toast: Transfer learning via attention steering.arXiv preprint arXiv:2305.15542,

    Pith/arXiv arXiv

  32. [38]

    Au- toprompt: Eliciting knowledge from language models with automatically generated prompts.arXiv preprint arXiv:2010.15980,

    Taylor Shin, Yasaman Razeghi, Robert L Logan IV , Eric Wallace, and Sameer Singh. Au- toprompt: Eliciting knowledge from language models with automatically generated prompts.arXiv preprint arXiv:2010.15980,

    Pith/arXiv arXiv 2010

  33. [39]

    Explaining black box text modules in natural language with language models.arXiv preprint arXiv:2305.09863, 2023a

    Chandan Singh, Aliyah R Hsu, Richard Antonello, Shailee Jain, Alexander G Huth, Bin Yu, and Jianfeng Gao. Explaining black box text modules in natural language with language models.arXiv preprint arXiv:2305.09863, 2023a. Chandan Singh, John X. Morris, Jyoti Aneja, Alexander M. Rush, and Jianfeng Gao. Explain- ing patterns in data with language models via ...

    Pith/arXiv arXiv 2013

  34. [41]

    Multiguard: An efficient approach for ai safety moderation across languages and modalities

    Sahil Verma, Keegan Hines, Jeff Bilmes, Charlotte Siska, Luke Zettlemoyer, Hila Gonen, and Chandan Singh. Multiguard: An efficient approach for ai safety moderation across languages and modalities. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 16184–16198,

    2025

  35. [42]

    Parameter-efficient fine-tuning in large models: A survey of methodologies.arXiv preprint arXiv:2410.19878,

    Luping Wang, Sheng Chen, Linnan Jiang, Shu Pan, Runze Cai, Sen Yang, and Fei Yang. Parameter-efficient fine-tuning in large models: A survey of methodologies.arXiv preprint arXiv:2410.19878,

    Pith/arXiv arXiv

  36. [43]

    Albert Webson and Ellie Pavlick. Do prompt-based models really understand the meaning of their prompts? InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T echnologies, pp. 2300–2344,

    2022

  37. [44]

    Larger language models do in-context learning differently.arXiv preprint arXiv:2303.03846,

    Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al. Larger language models do in-context learning differently.arXiv preprint arXiv:2303.03846,

    Pith/arXiv arXiv

  38. [45]

    14 Published as a conference paper at COLM 2026 An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al

    URL https://arxiv.org/abs/2512.18092. 14 Published as a conference paper at COLM 2026 An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    arXiv 2026

  39. [46]

    Jailbreak attacks and defenses against large language models: A survey.arXiv preprint arXiv:2407.04295,

    Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. Jailbreak attacks and defenses against large language models: A survey.arXiv preprint arXiv:2407.04295,

    Pith/arXiv arXiv

  40. [47]

    Parameter- efficient fine-tuning for foundation models, 2025a

    Dan Zhang, Tao Feng, Lilong Xue, Yuandong Wang, Yuxiao Dong, and Jie Tang. Parameter- efficient fine-tuning for foundation models, 2025a. URL https://arxiv.org/abs/2501. 13787. Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adalora: Adaptive budget allocation for parameter-efficient...

    Pith/arXiv arXiv

  41. [49]

    Large language models are human-level prompt engineers.arXiv preprint arXiv:2211.01910,

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers.arXiv preprint arXiv:2211.01910,

    Pith/arXiv arXiv

  42. [50]

    Vector-icl: In-context learning with continuous vector representations

    Yufan Zhuang, Chandan Singh, Liyuan Liu, Jingbo Shang, and Jianfeng Gao. Vector-icl: In-context learning with continuous vector representations. InInternational Conference on Learning Representations, volume 2025, pp. 28596–28618,

    2025

  43. [51]

    We can see it only pays attention to thePhishingtokens regardless of current input’s label

    15 Published as a conference paper at COLM 2026 A Appendix A.1 Visualizing Attention Patterns Figure A.1: Visualizing the attention pattern of L14H11. We can see it only pays attention to thePhishingtokens regardless of current input’s label. Figure A.2: Visualizing the attention pattern of L14H11. We can see that on datasets outside Web, it shows no clea...

    2026

  44. [53]

    or Sparse Autoencoder (SAE) latents (Chalnev et al., 2024). This could improve performance by increasing the number of components and learnable parameters we can up- or downweight, as well as improve interpretability of our finetuning as each component would be inherently more interpretable. Models could even be trained with this kind of steering in mind,...

    2024

  45. [54]

    Reuters - Short-sellers, Wall Street’s dwindling band of ultra-cynics, are seeing green again

    to adapt parameters on the fly based on task specifications (similar to approaches that have been tried for LoRA (Charakorn et al., 2025)) or to involve a human-in-the-loop when specifying and adapting to a new task (Peng et al., 2023; Feng et al., 2026). 17 Published as a conference paper at COLM 2026 A.5 Prompt and Data examples Dataset Text Label SST2 ...

    2025

  46. [2013]

    In-context algebra.arXiv preprint arXiv:2512.16902,

    Eric Todd, Jannik Brinkmann, Rohit Gandikota, and David Bau. In-context algebra.arXiv preprint arXiv:2512.16902,

    arXiv

  47. [2015]

    Attention reveals more than tokens: Training-free long-context reasoning with attention-guided retrieval.arXiv preprint arXiv:2503.09819, 2025b

    Yuwei Zhang, Jayanth Srinivasa, Gaowen Liu, and Jingbo Shang. Attention reveals more than tokens: Training-free long-context reasoning with attention-guided retrieval.arXiv preprint arXiv:2503.09819, 2025b. Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingchuan Yang, Bo Tang, Feiyu Xiong, and Zhiyu Li. Attention heads of large language models: ...

    Pith/arXiv arXiv

  48. [2019]

    One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention.arXiv preprint arXiv:2307.03576,

    Arvind Mahankali, Tatsunori B Hashimoto, and Tengyu Ma. One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention.arXiv preprint arXiv:2307.03576,

    Pith/arXiv arXiv

  49. [2020]

    Improving steering vectors by targeting sparse autoencoder features.arXiv preprint arXiv:2411.02193,

    Sviatoslav Chalnev, Matthew Siu, and Arthur Conmy. Improving steering vectors by targeting sparse autoencoder features.arXiv preprint arXiv:2411.02193,

    Pith/arXiv arXiv

  50. [2021]

    Human-ai co-design for clinical prediction models.arXiv preprint arXiv:2601.09072,

    Jean Feng, Avni Kothari, Patrick Vossler, Andrew Bishara, Lucas Zier, Newton Addo, Aaron Kornblith, Yan Shuo Tan, and Chandan Singh. Human-ai co-design for clinical prediction models.arXiv preprint arXiv:2601.09072,

    Pith/arXiv arXiv

  51. [2022]

    In-context language learning: Architectures and algorithms.arXiv preprint arXiv:2401.12973,

    Ekin Aky ¨urek, Bailin Wang, Yoon Kim, and Jacob Andreas. In-context language learning: Architectures and algorithms.arXiv preprint arXiv:2401.12973,

    Pith/arXiv arXiv

  52. [2023]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al

    URL https://openaipublic.blob.core.windows.net/ neuron-explainer/paper/index.html. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    1901

  53. [2024]

    What learning algorithm is in-context learning? investigations with linear models.arXiv preprint arXiv:2211.15661,

    Ekin Aky ¨urek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models.arXiv preprint arXiv:2211.15661,

    Pith/arXiv arXiv

  54. [2025]

    Understanding in- context learning in transformers and llms by learning to learn discrete functions.arXiv preprint arXiv:2310.03016,

    Satwik Bhattamishra, Arkil Patel, Phil Blunsom, and Varun Kanade. Understanding in- context learning in transformers and llms by learning to learn discrete functions.arXiv preprint arXiv:2310.03016,

    Pith/arXiv arXiv

  55. [2026]

    Weight-sparse transformers have interpretable circuits.arXiv preprint arXiv:2511.13653,

    Leo Gao, Achyuta Rajaram, Jacob Coxon, Soham V Govande, Bowen Baker, and Dan Mossing. Weight-sparse transformers have interpretable circuits.arXiv preprint arXiv:2511.13653,

    arXiv