Pith. sign in

REVIEW 3 major objections 5 minor 51 references

Command-V: Pasting LLM Behaviors via Activation Profiles

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A behavior trained into one LLM can be pasted into another with no backpropagation or task data, using linear converters between activation spaces.

desk verdict A training-free, plausibly useful method for porting ReFT adapters across models, with a real rank-limitation blind spot that deserves a diagnostic before the paste-behavior claim is taken at face value. read the letter →

arxiv 2506.19140 v1 pith:EZPL3V7T submitted 2025-06-23 cs.LG

Barry Wang , Avi Schwarzschild , Alexander Robey , Ali Payani , Charles Fleming , Mingjie Sun , Daphne Ippolito This is my paper · ORCID
classification cs.LG
keywords activationtransferbehaviorpastingresidualadapterslinearconvertersprofilesrepresentationfinetuningjailbreakchain-of-thought
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Command-V claims that the behavioral effect of a low-rank residual adapter can be copied from one large language model (the donor) to a different model (the recipient) without any gradient training or task-specific data. The transfer works by profiling each model's layer activations on a generic set of 1,030 prompts, fitting linear maps between corresponding layers with a Moore-Penrose pseudoinverse, then applying the donor's activation-space intervention inside the recipient's forward pass. In three case studies—safety-refusal enhancement, jailbreak facilitation, and default chain-of-thought reasoning—the pasted behavior matches or exceeds directly finetuning the recipient, at a fraction of the compute. If the claim holds, trained behaviors become portable assets that can be reused across models and architectures, including on devices that cannot run the larger donor model.

What carries the argument

The load-bearing object is the bidirectional layer converter built from activation profiles: for each matched layer pair, the paper collects last-token activations across the same 1,030 LIMA prompts from both models, then sets $C_{R\to D} = X^\dagger Y$ and $C_{D\to R} = Y^\dagger X$, where $X$ and $Y$ are the activation matrices of recipient and donor. Layers are matched by the linear depth map $l_R = \lfloor \alpha \cdot l_D \rfloor$ with $\alpha = |L_R|/|L_D|$, following the view that layer functions scale roughly linearly with depth. The donor intervention is a DiReFT module $I(h) = h + W_2^{\mathsf{T}}(W_1 h + b)$ of rank 4 to 32, applied only to last tokens. The mechanism converts the recipient's hidden state into donor space, applies that low-rank intervention, converts the delta back, and adds it to the recipient's hidden state.

What would settle it

Recompute the converters for the Llama3.2-3B-to-Llama3.1-8B refusal transfer using 10 randomly sampled LIMA prompts instead of 1,030; if the attack-success-rate reduction vanishes, the profile size is load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that a behavior encoded in a donor model's residual adapter transfers to a recipient as $h_R^{\text{intervened}} = h_R + C_{D\to R}\big(\Delta I_{l_D}(C_{R\to D} h_R)\big)$, Equation (6). Here $\Delta I_{l_D}$ is the low-rank intervention learned by a DiReFT module at donor layer $l_D$, and $C_{R\to D}$, $C_{D\to R}$ are the pseudoinverse-derived linear converters computed from activation profiles. The paper argues that this three-step conversion—recipient to donor space, apply intervention, back to recipient space—preserves the functional effect of the adapter without backpropagation, training data, or weight updates. It supports this with results where refusal rates, jailbreak success rates, and chain-of-thought accuracy on Big Bench Hard after transfer are comparable to or better than direct ReFT finetuning of the recipient, and with converter derivation taking seconds on a CPU.

Load-bearing premise

A linear map fitted on 1,030 generic prompts aligns the residual activation spaces of two models at depth-matched layers well enough that the donor's low-rank intervention produces the intended behavior when pasted; if the alignment is wrong, the pasted behavior degrades or collapses.

Editorial extensions

If this is right

  • Behaviors become portable: an adapter trained once on one model can be reused on any recipient with compatible profiling, so finetuning need not be repeated per architecture.
  • Edge deployment becomes practical: with activation profiles precomputed, a small recipient model can gain an 8B model's adapter behavior without ever loading the 8B weights or running generation on it.
  • The same pipeline transfers jailbreak behavior, so safety alignment can be weakened across model families at low cost; the paper flags this as a broader-impact concern.
  • Task composition becomes plausible: small specialist adapters trained separately could be pasted into one recipient to build a generalist, a direction the paper suggests for future work.
  • Cross-family transfer is uneven: jailbreaking ports across families relatively well, but refusal and reasoning transfers weaken with architectural divergence, so model pairing matters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the linear-converter mechanism is as general as the case studies suggest, then the residual streams of different instruction-tuned models share a geometric structure that can be aligned with a small generic sample; one testable corollary is that converter cycle-consistency MSE should predict porting success across model pairs.
  • The method's reliance on LIMA prompts implies that the 1,030-prompt profile acts as a universal basis for behavior-relevant directions; replacing LIMA with random tokens or code prompts and checking whether jailbreak transfer survives would directly test that basis.
  • Because the donor intervention is applied in the donor's space and only its delta is pasted, the recipient's own representations are otherwise untouched; this suggests Command-V could be combined with other editing methods additively, though the paper does not test composition.
  • The authors note output collapse and formatting failures after transfer; a natural extension is to treat the converter as a learned but frozen projection and study which adapter ranks and layer choices minimize that collapse.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Command-V (⌘V), a backpropagation-free method for transferring a behavior encoded in a donor model's residual activation adapter (DiReFT) to a recipient model. The method profiles last-token activations on 1,030 LIMA prompts, derives linear converters between corresponding layers via Moore-Penrose pseudoinverses, and applies the donor intervention in the recipient's activation space through the round-trip formula in Eq. (6). The authors evaluate the approach on three case studies: safety-refusal enhancement, jailbreak facilitation, and chain-of-thought reasoning, reporting that Command-V matches or exceeds direct finetuning while using orders of magnitude less compute. The paper includes experiments across multiple Llama, Qwen, Gemma, Phi, and OLMo models and releases code and data.

Significance. If the claims hold, Command-V is a useful and inexpensive alternative to distillation or per-recipient finetuning for PEFT-style behavior transfer, and the idea of reusing a donor's residual adapter via linear activation converters is novel. The method is clearly described, the converter derivation is standard least squares, and the paper ships code and data, which are concrete strengths. The empirical scope is substantial, especially the cross-family jailbreak experiments in Section 4.3. However, the central claim is only partially supported: the reverse refusal direction (8B to 3B) does not match direct finetuning, and the rank-limited converter issue identified below means the method's mechanism is not yet fully verified.

major comments (3)
  1. [Section 3.2, Eqs. (4)-(5), with N=1030 profiling prompts from Section 4.1] Each converter C_R→D = X^†Y and C_D→R = Y^†X has rank at most N=1030, which is far below the residual dimensions (3072/4096). Consequently, the transferred intervention in Eq. (6) is confined to the row space of the donor activation-profile matrix Y: any component of ΔI_lD orthogonal to that row space is annihilated by C_D→R. The paper does not test whether the behavior-relevant part of the donor intervention lies in this subspace. Appendix A.3 reports only forward and cycle MSE on the same LIMA activations used to derive the converters, which measures fit on the profiling manifold and cannot detect missing directions. Because the reported failure modes in the Discussion (small-effect adapters, cross-family transfer, degraded or collapsed CoT outputs) are exactly what a rank-limited projection would produce, the central claim that Eq. (6) pastes the donor behavior rather than a rank-limited proxy is unverified. Please add a subspace diagnostic, for example by projecting the donor intervention onto the row space of Y and measuring its effect on held-out prompts, or by reporting the singular-value spectrum of the converters together with the overlap of ΔI_lD with that spectrum.
  2. [Section 4.2, refusal enhancement results] The abstract's claim that Command-V 'matches or exceeds the performance of direct finetuning' is contradicted by the reverse transfer direction: for 8B→3B, Command-V reduces ASR to 16.4%, while direct ReFT training on the 3B recipient reaches a final ASR of 12.20% and an observed intermediate low of 5.65%. The paper also compares Command-V's final transfer against the direct method's final checkpoint while separately reporting the direct method's best intermediate value, making the 'surpasses direct training' statement for 3B→8B ambiguous (20.40% vs. final 28.55%, with ReFT intermediate 21.05%). Please report error bars or multiple seeds and state precisely which comparisons support the headline claim.
  3. [Section 4.3 and the general claim in the abstract] The jailbreak case study shows that Command-V can increase attack success rates substantially on many recipient models, but the paper does not compare against direct finetuning for each donor-recipient pair. Since the headline claim is that Command-V matches or exceeds direct finetuning, the absence of a direct-finetuning baseline in this case study means the central claim is not tested there. Please either add such baselines or explicitly reframe the claim as 'matches or exceeds direct finetuning in selected settings'.
minor comments (5)
  1. [Section 4.3, first paragraph] The phrase 'we next consider refusal enhancement' appears in the refusal-suppression section; it should read 'refusal suppression'.
  2. [Figure 5 caption] The caption contains a typo, 'sqaure', and should clarify that the reported MSE values are computed on the same prompts used to fit the converters, not on a held-out set.
  3. [Equations (4)-(6) and surrounding text] The notation for converters is inconsistent: the text uses C_R→D and C_D→R, while Eq. (6) uses C_lR→lD and C_lD→lR. Please unify the subscripts to avoid confusion.
  4. [Appendix A.2] The appendix states that MSE-guided layer matching 'overwhelmingly' selects early recipient layers and performs poorly downstream, but does not provide a quantitative comparison to the linear mapping in Eq. (3); a small table or figure would make this claim checkable.
  5. [Section 4.2 and Figure 2] The main numerical differences in the refusal experiments are small in some cases (e.g., 20.40% vs. 21.05%), and no variance or number of seeds is reported. Adding error bars or a short seed-sensitivity statement would strengthen the comparisons.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the converters are fitted on generic LIMA prompts, while all target metrics are evaluated out-of-sample on task data and compared against separately trained ReFT baselines.

full rationale

Command-V's converters (Eqs. 4-5) are Moore-Penrose least-squares maps fitted to activation matrices X and Y collected on N=1030 generic LIMA prompts, and the transfer mechanism (Eq. 6) applies a donor intervention through those maps. No success metric is fitted: the WildJailbreak ASR, AdvBench/HarmBench ASR, and Big Bench Hard accuracy numbers are measured on held-out task data disjoint from the profile prompts, and the comparison point is a separately trained DiReFT adapter on the recipient, so the reported gains are not forced by the fitting process. The donor intervention is trained externally with ReFT; the layer-correspondence rule l_R = floor(alpha * l_D) is an explicit heuristic, not a uniqueness claim imported from the authors' prior work, and the self-citations present (e.g., SmoothLLM, Baseline Defenses, TOFU) are background references and never load-bearing for the transfer derivation. Figure 5's forward/cycle MSE is explicitly labeled as training loss on the fitting prompts, so it does not masquerade as an out-of-sample prediction. The stated limitations (small-effect adapters yield negligible transfer, cross-family degradation, output collapse, and exact-token formatting failures) are honest boundary conditions; the rank-1030 subspace concern is a correctness/validity risk about whether the converter preserves the donor intervention, not a circular reduction of the prediction to its inputs. Therefore the paper is self-contained with respect to the circularity failure modes listed.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on the assumption that residual activation spaces of different transformers are approximately linearly relatable at depth-matched layers, which is not proven and is empirically known to fail for some model pairs. Additional domain assumptions cover the representativeness of last-token activations on a general prompt set and the validity of ReFT interventions as behavior encoders. No new theoretical entities are introduced.

free parameters (4)
  • Layer correspondence ratio alpha = |L_R|/|L_D| per model pair
    Chosen by hand in Equation (3); determines which donor layer pairs with which recipient layer. The paper explores alternatives in A.2 but adopts the simple linear rule.
  • Number of profiling prompts N = 1030
    Chosen from the LIMA training split; affects converter quality. The paper shows smaller N in Figure 2 but N=1030 is the main setting.
  • Profiling prompt set = LIMA training split
    Chosen as task-agnostic; the paper argues it avoids task-specific data, but the choice is a design decision.
  • DiReFT adapter rank and layer cadence = rank 8, every other layer
    Training hyperparameters for the donor adapter; they influence the portability of the adapter.
assumptions (5)
  • domain assumption Corresponding layers between models can be matched by l_R = floor(alpha * l_D) with alpha = |L_R|/|L_D|.
    Invoked in Equation (3) and Algorithm 1; no theoretical justification is given, and Section A.2 shows MSE-based matching fails, suggesting the linear depth rule is heuristic.
  • domain assumption A linear map, the pseudoinverse least-squares map, between activations of two models preserves the functional effect of a low-rank affine intervention.
    Embedded in Equation (6); the paper provides empirical evidence but no derivation, and the Discussion concedes transfer works poorly for architecturally dissimilar models.
  • domain assumption Last-token activations on a small set of general prompts capture the directions relevant to the target behaviors.
    Used in Phase 1 of Algorithm 1 and justified by citing Zou et al. (2023a); the paper uses N=1030 LIMA prompts.
  • standard math The Moore-Penrose pseudoinverse solves the least-squares mapping problem.
    Equations (4) and (5); standard linear algebra.
  • domain assumption ReFT and DiReFT interventions, Equation (2), are a valid representation of behaviors in the donor model.
    Taken from prior work by Wu et al. (2024); the paper trains DiReFT adapters on donor models and assumes their effect is portable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Command-V: Pasting LLM Behaviors via Activation Profiles." pith.science (2026). https://pith.science/paper/EZPL3V7T

@misc{pith2026250619140,
  author       = {Pith},
  title        = {Pith review of: Command-V: Pasting LLM Behaviors via Activation Profiles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EZPL3V7T}},
  note         = {Machine review of arXiv:2506.19140}
}
read the original abstract

Retrofitting large language models (LLMs) with new behaviors typically requires full finetuning or distillation-costly steps that must be repeated for every architecture. In this work, we introduce Command-V, a backpropagation-free behavior transfer method that copies an existing residual activation adapter from a donor model and pastes its effect into a recipient model. Command-V profiles layer activations on a small prompt set, derives linear converters between corresponding layers, and applies the donor intervention in the recipient's activation space. This process does not require access to the original training data and needs minimal compute. In three case studies-safety-refusal enhancement, jailbreak facilitation, and automatic chain-of-thought reasoning--Command-V matches or exceeds the performance of direct finetuning while using orders of magnitude less compute. Our code and data are accessible at https://github.com/GithuBarry/Command-V/.

Figures

Figures reproduced from arXiv: 2506.19140 by the authors.

Figure 1
Figure 1. Reusing PEFT weights on an architecturally different model with [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Reduced jailbreak attack success rate (ASR [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Jailbreaking and ⌘V: Porting jailbreakability from one model to another often increases attack success rate (ASR ↑), especially in the same model family (indicated by the white dashed line boxes). All models here are their instruct versions. Diagonal entries reflect validation set performance of the ReFT adapter (not ⌘ V), serve as an upper bound on expected ⌘ V performance. Results As shown by the off diagonals of … view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Trained and ⌘ V performance on Big Bench Hard. When prompted for answers in a JSON format, models with OpenMathReasoning ReFT adapters ported via ⌘ V from a same-family model (e.g. 8B to 3B) are more likely to reason (more output characters), comparable to when the mod…
Figure 5
Figure 5. Figure 5: Comparison of forward and cycle “training” MSE between pseudoinverse residual layer [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Llama 3.1 8B Instruct to Llama 3.1 3B Instruct test loss, averaged across input. Minimizing [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Pseudoinverse Converter Deriving Time per model pair on an M1 Max CPU. Setting up [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 20 canonical work pages

  1. [1]

    Abdin, J

    M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, et al. Phi-4 Technical Report . arXiv preprint arXiv:2412.08905, 2024

    arXiv 2024

  2. [2]

    Bianchi, M

    F. Bianchi, M. Suzgun, G. Attanasio, P. R \"o ttger, D. Jurafsky, T. Hashimoto, and J. Zou. Safety-Tuned LLaMA: Lessons from Improving the Safety of Large Language Models that Follow Instructions . arXiv preprint arXiv:2309.07875, 2023

    arXiv 2023

  3. [3]

    Buciluǎ, R

    C. Buciluǎ, R. Caruana, and A. Niculescu-Mizil. Model Compression . In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD '06, page 535–541, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595933395. doi:10.1145/1150402.1150464. URL https://doi.org/10.1145/1150402.1150464

    arXiv 2006

  4. [4]

    Carlini, M

    N. Carlini, M. Nasr, C. A. Choquette-Choo, M. Jagielski, I. Gao, P. W. W. Koh, D. Ippolito, F. Tramer, and L. Schmidt. Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 36: 0 61478--61500, 2023

    work page 2023

  5. [5]

    P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong. Jailbreaking Black Box Large Language Models in Twenty Queries . arXiv preprint arXiv:2310.08419, 2023

    arXiv 2023

  6. [6]

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep Reinforcement Learning from Human Preferences . Advances in neural information processing systems, 30, 2017

    work page 2017

  7. [7]

    L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, A. Radford, I. Sutskever, J. Leike, and J. Wu. Scaling and evaluating sparse autoencoders . arXiv preprint arXiv:2406.04093, 2024

    arXiv 2024

  8. [8]

    Grattafiori, A

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. The Llama 3 Herd of Models . arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

Show all 51 references
  1. [9]

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning . arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  2. [10]

    Hinton, O

    G. Hinton, O. Vinyals, and J. Dean. Distilling the Knowledge in a Neural Network . arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  3. [11]

    Houlsby, A

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly. Parameter-Efficient Transfer Learning for NLP . In International conference on machine learning , pages 2790--2799. PMLR, 2019

    2019

  4. [12]

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. LoRA : Low-rank adaptation of large language models . In International Conference on Learning Representations , 2022

    2022

  5. [13]

    Z. Hu, L. Wang, Y. Lan, W. Xu, E.-P. Lim, L. Bing, X. Xu, S. Poria, and R. Lee. LLM -adapters: An adapter family for parameter-efficient fine-tuning of large language models. In H. Bouamor, J. Pino, and K. Bali, editors, Proceedings of the 2023 Conference on Empirical Methods ...

    2023 doi

  6. [14]

    Ilharco, M

    G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi. Editing Models with Task Arithmetic . arXiv preprint arXiv:2212.04089, 2022

    2022 arXiv

  7. [15]

    N. Jain, A. Schwarzschild, Y. Wen, G. Somepalli, J. Kirchenbauer, P.-y. Chiang, M. Goldblum, A. Saha, J. Geiping, and T. Goldstein. Baseline Defenses for Adversarial Attacks Against Aligned Language Models . arXiv preprint arXiv:2309.00614, 2023

    2023 arXiv

  8. [16]

    J. Ji, M. Liu, J. Dai, X. Pan, C. Zhang, C. Bian, B. Chen, R. Sun, Y. Wang, and Y. Yang. BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset . Advances in Neural Information Processing Systems, 36: 0 24678--24704, 2023

    2023

  9. [17]

    Jiang, K

    L. Jiang, K. Rao, S. Han, A. Ettinger, F. Brahman, S. Kumar, N. Mireshghallah, X. Lu, M. Sap, Y. Choi, et al. WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models . Advances in Neural Information Processing Systems, 37: 0 47094--47165, 2024

    2024

  10. [18]

    Kojima, S

    T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large Language Models are Zero-Shot Reasoners . Advances in neural information processing systems, 35: 0 22199--22213, 2022

    2022

  11. [19]

    Konen, S

    K. Konen, S. Jentzsch, D. Diallo, P. Sch \"u tt, O. Bensch, R. E. Baff, D. Opitz, and T. Hecking. Style Vectors for Steering Generative Large Language Model . arXiv preprint arXiv:2402.01618, 2024

    2024 arXiv

  12. [20]

    X. L. Li and P. Liang. Prefix-Tuning: Optimizing Continuous Prompts for Generation . arXiv preprint arXiv:2101.00190, 2021

    2021 arXiv

  13. [21]

    J. Luo, T. Ding, K. H. R. Chan, D. Thaker, A. Chattopadhyay, C. Callison-Burch, and R. Vidal. PaCE: Parsimonious Concept Engineering for Large Language Models . In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024

    2024

  14. [22]

    Maini, Z

    P. Maini, Z. Feng, A. Schwarzschild, Z. C. Lipton, and J. Z. Kolter. TOFU: A Task of Fictitious Unlearning for LLMs . arXiv preprint arXiv:2401.06121, 2024

    2024 arXiv

  15. [23]

    M. S. Matena and C. A. Raffel. Merging Models with Fisher-Weighted Averaging . In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 17703--17716. Curran Associates, Inc., 2022. URL h...

    2022

  16. [24]

    Mazeika, L

    M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, et al. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal . arXiv preprint arXiv:2402.04249, 2024

    2024 arXiv

  17. [25]

    K. Meng, D. Bau, A. Andonian, and Y. Belinkov. Locating and Editing Factual Associations in GPT . In Advances in Neural Information Processing Systems , volume 35, pages 17359--17372, 2022 a

    2022

  18. [26]

    K. Meng, A. S. Sharma, A. Andonian, Y. Belinkov, and D. Bau. Mass-Editing Memory in a Transformer . arXiv preprint arXiv:2210.07229, 2022 b

    2022 arXiv

  19. [27]

    Moshkov, D

    I. Moshkov, D. Hanley, I. Sorokin, S. Toshniwal, C. Henkel, B. Schifferer, W. Du, and I. Gitman. AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset . arXiv preprint arXiv:2504.16891, 2025

    2025 arXiv

  20. [28]

    T. OLMo, P. Walsh, L. Soldaini, D. Groeneveld, K. Lo, S. Arora, A. Bhagia, Y. Gu, S. Huang, M. Jordan, et al. 2 OLMo 2 Furious . arXiv preprint arXiv:2501.00656, 2024

    2024 arXiv

  21. [29]

    Learning to Reason with LLMs

    OpenAI. Learning to Reason with LLMs

  22. [30]

    Perez, S

    E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving. Red teaming language models with language models. In Y. Goldberg, Z. Kozareva, and Y. Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language ...

    2022 doi

  23. [31]

    Rimsky, N

    N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. Turner. Steering llama 2 via contrastive activation addition. In L.-W. Ku, A. Martins, and V. Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: L...

    2024 doi

  24. [32]

    Robey, E

    A. Robey, E. Wong, H. Hassani, and G. J. Pappas. SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks . arXiv preprint arXiv:2310.03684, 2023

    2023 arXiv

  25. [33]

    Romero, N

    A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio. FitNets: Hints for Thin Deep Nets . arXiv preprint arXiv:1412.6550, 2014

    2014 arXiv

  26. [34]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models . arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  27. [35]

    Suzgun, N

    M. Suzgun, N. Scales, N. Sch \"a rli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al. Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them . arXiv preprint arXiv:2210.09261, 2022

    arXiv

  28. [36]

    G. Team, M. Riviere, S. Pathak, P. G. Sessa, C. Hardin, S. Bhupatiraju, L. Hussenot, T. Mesnard, B. Shahriari, A. Ram \'e , et al. Gemma 2: Improving Open Language Models at a Practical Size . arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  29. [37]

    Tenney, D

    I. Tenney, D. Das, and E. Pavlick. BERT Rediscovers the classical NLP pipeline . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 4593--4601, 2019 a

    2019

  30. [38]

    Tenney, P

    I. Tenney, P. Xia, B. Chen, A. Wang, A. Poliak, R. T. McCoy, N. Kim, B. Van Durme, S. R. Bowman, D. Das, et al. What Do You Learn from Context? Probing for Sentence Structure in Contextualized Word Representations . arXiv preprint arXiv:1905.06316, 2019 b

    1905 arXiv

  31. [39]

    Trung, X

    L. Trung, X. Zhang, Z. Jie, P. Sun, X. Jin, and H. Li. ReFT: Reasoning with Reinforced Fine-Tuning . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 7601--7614, 2024

    2024

  32. [40]

    A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid. Activation Addition: Steering Language Models Without Optimization . arXiv e-prints, pages arXiv--2308, 2023

    2023

  33. [41]

    Y. Wang, H. Li, X. Han, P. Nakov, and T. Baldwin. Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs . arXiv preprint arXiv:2308.13387, 2023

    2023 arXiv

  34. [42]

    J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le. Finetuned Language Models are Zero-Shot Learners . arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  35. [43]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models . Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  36. [44]

    Wortsman, G

    M. Wortsman, G. Ilharco, S. Y. Gadre, R. Roelofs, R. Gontijo-Lopes, A. S. Morcos, H. Namkoong, A. Farhadi, Y. Carmon, S. Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time . In International confer...

    2022

  37. [45]

    Z. Wu, A. Arora, Z. Wang, A. Geiger, D. Jurafsky, C. D. Manning, and C. Potts. ReFT: Representation Finetuning for Language Models . Advances in Neural Information Processing Systems, 37: 0 63908--63962, 2024

    2024

  38. [46]

    Y. Xia, J. Kim, Y. Chen, H. Ye, S. Kundu, C. C. Hao, and N. Talati. Understanding the Performance and Estimating the Cost of LLM Fine-Tuning . In 2024 IEEE International Symposium on Workload Characterization (IISWC) , pages 210--223. IEEE, 2024

    2024

  39. [47]

    Yadav, D

    P. Yadav, D. Tam, L. Choshen, C. A. Raffel, and M. Bansal. TIES-Merging: Resolving Interference When Merging Models . Advances in Neural Information Processing Systems, 36: 0 7093--7115, 2023

    2023

  40. [48]

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2.5 Technical Report . arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  41. [49]

    C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu, et al. LIMA: Less Is More for Alignment . Advances in Neural Information Processing Systems, 36: 0 55006--55021, 2023

    2023

  42. [50]

    A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski, et al. Representation Engineering: A Top-Down Approach to AI Transparency . arXiv preprint arXiv:2310.01405, 2023 a

    2023 arXiv

  43. [51]

    A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson. Universal and Transferable Adversarial Attacks on Aligned Language Models . arXiv preprint arXiv:2307.15043, 2023 b

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.