Pith. sign in

REVIEW 4 major objections 6 minor 40 references

Behavioral Reprogramming of Open-Weights Models: Cognitive Plasticity and Alignment Bounds

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a terse, question-asking Socratic persona can be installed in an instruction-tuned LLM with fewer than 1,500 curated training pairs, rank-16 LoRA, and 2–3 epochs, and that DPO (β=0.15) then transfers parts of that…

desk verdict A transparent but overclaimed empirical recipe for installing a Socratic persona via LoRA+DPO; the headline zero-shot cross-lingual transfer results are not established by n=10 terminal-? measurements. read the letter →

arxiv 2608.13069 v1 pith:LLACOIRW submitted 2026-08-13 cs.AI

Lucia Malíčková This is my paper · ORCID
classification cs.AI
keywords behavioralreprogrammingSocraticpersonaDirectPreferenceOptimizationLoRAzero-shotcross-lingualtransferparameter-efficientfine-tuningQuestionRatecognitiveplasticity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper attempts to establish that a proactive, question-asking 'Socratic' persona can be installed in an instruction-tuned large language model with very few examples: fewer than 1,500 curated SFT pairs and 440 preference pairs, a LoRA rank of 16, and training capped at 2–3 epochs. Its central claim is that Direct Preference Optimization (DPO) with β=0.15 decouples this assertive behavior from the training language's syntax, so the persona transfers zero-shot to other languages even though the preference data were in Slovak. If correct, this would make behavioral reprogramming a small-data, parameter-efficient operation—no reward model, no large preference corpus—and would suggest that a model's 'personality' can be stored in a low-rank subspace partly independent of surface language. The paper also reports a hard optimal training window and an instruction-tuning prerequisite, framing these as general bounds on low-resource behavioral adaptation.

What carries the argument

The load-bearing object is the LoRA low-rank subspace with rank $r=16$ and scaling $\alpha=32$, combined with the DPO objective (Equation 5) that reparameterizes preference learning as a classification loss over chosen vs. rejected responses. The LoRA subspace defines where behavior can be stored (about 42M trainable parameters, roughly 0.9% of the model); the DPO loss, with KL coefficient $\beta=0.15$, is what decouples the questioning behavior from the surface syntax. The evaluation machinery is the Question Rate (Equation 9), a binary indicator counting whether a response ends with '?', applied per language and per psychological category; the epoch-ablation U-shaped curve fixes the training window at $e \in [2,3]$.

What would settle it

Re-run the same DPO adapter on a held-out set of, say, 100 scenarios per language and compute the terminal-question rate; also ask human raters whether the responses are genuinely assertive. If German, Portuguese, and Slovak sit near 0% while Spanish and English stay high, the decoupling claim survives; if rates swing wildly across scenario draws (as the paper's own German numbers—0% vs 48%—suggest), the measurement is too noisy to support the claimed language stratification.

Watch

Extended reading notes

Core claim

The paper's central discovery is that applying DPO to a rank-16 LoRA subspace ($\alpha=32$, dropout 0.1) after SFT on a multilingual corpus converts an instructed model into a terse Socratic interlocutor: post-DPO responses average 3.22 words, short-response adherence is 100%, and category-specific question rates range from 12% to 32%. The behavioral shift is claimed to be orthogonal to syntax: under a strict zero-shot adversarial matrix, Spanish reaches 60% and English 30% terminal-question rates (Exp. 3) or 48.5% any-question rate (Exp. 6), while morphologically distant German and Portuguese collapse to 0% in the single-turn metric. The paper further reports that this reprogramming fails on base (non-instructed) models, which waste their low-rank capacity learning chat-template syntax, and that the optimal configuration sits at rank 16, learning rate $2 \times 10^{-4}$, and epoch $2$–$3$, beyond which validation loss diverges.

Load-bearing premise

The central claims rest on treating 'ends with a question mark' as the measure of Socratic behavior and on drawing cross-lingual conclusions from only 10 scenarios per language; if that indicator misses real assertiveness, or if 10 samples are too noisy, the decoupling and zero-shot transfer results are not established.

Editorial extensions

If this is right

  • A curated behavioral dataset of under 1,500 SFT pairs and 440 DPO pairs is sufficient to anchor a persona, provided the rank, learning rate, and epoch budget stay in the optimal window.
  • Instruction-tuned checkpoints are a prerequisite; base models diverge and memorize chat-format tokens instead of the persona.
  • Behavioral traits can partly transfer across languages without any target-language preference training, with the strongest transfer to Spanish and English and collapse for morphologically distant languages.
  • Extending training beyond 3 epochs on such small corpora destroys generalization, with validation loss rising even as training loss approaches zero.
  • The 14B model achieves the best perplexity (1.414) but is not deployable in the batch setting due to memory limits, making the 8B model the practical choice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the decoupling is genuine, the same DPO adapter could be re-anchored into a new language with a small native preference set, or perhaps via target-language SFT before DPO, without retraining the full persona.
  • The Question Rate metric counts only terminal '?' marks, so the claimed cross-lingual transfer may be under- or over-stated; a validation against human judgment of assertiveness would test whether the metric tracks the intended construct.
  • The rank-16 threshold suggests that the dimensionality needed for a persona may scale with the morphological complexity of the target languages; testing rank 16 on a morphologically rich language like German with native DPO would separate tokenization effects from representational capacity.
  • The epoch window $[2,3]$ may generalize to other low-resource behavioral edits, but the paper only tests one persona; a direct test would repeat the pipeline for a different persona (e.g., empathetic vs. terse) on the same backbone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript reports a large set of HPC experiments aimed at reprogramming open-weight instruction-tuned LLMs into a terse, Socratic question-asking persona. Using LoRA fine-tuning and DPO on a small curated corpus, the author claims to identify a rank threshold at r=16, an optimal training window of e∈[2,3], and a decoupling of assertive behavior from language-specific syntax, evidenced by zero-shot cross-lingual question rates. The paper also includes dataset-scaling experiments, base-vs-instruct comparisons, epoch ablations, and a 405-job hyperparameter sweep. The central positive claims are that a Socratic persona can be installed with fewer than 1,500 SFT pairs and 440 DPO pairs, and that parts of that persona transfer zero-shot to Spanish and English.

Significance. If the central claims held, the paper would make a useful empirical contribution to low-resource behavioral fine-tuning: it provides a concrete recipe (LoRA rank 16, alpha 32, dropout 0.1, 2–3 epochs, DPO beta 0.15), studies a non-standard persona target, and documents infrastructure-scale compute constraints. The author deserves credit for reporting the hardware budget, publishing an anonymized code/data repository, and including explicit limitation statements about the QR metric and the non-comparability of the two cross-lingual evaluations. However, the evidence as presented does not support the strong claims of 'strict mathematical bounds', 'proofs', or robust zero-shot transfer. The evaluation is built on a binary terminal-'?' metric applied to only 10 scenarios per language, with no confidence intervals, no held-out test set for hyperparameter selection, and internal contradictions between experiments for the same language. The significance of the empirical findings is therefore conditional on substantial additional evidence or a major re-scoping of the claims.

major comments (4)
  1. [Sec. IV-C, Table II, Eq. (9)] The zero-shot transfer claim is not supported by the reported measurement. With n=10 scenarios per language, the 95% Wilson intervals for the reported Spanish (60%) and English (30%) question rates are approximately [31%, 83%] and [11%, 60%], respectively; both intervals overlap the 0% rates reported for German, Portuguese, and the source language Slovak. A 0% rate for Slovak—the language on which SFT/DPO were applied—particularly undermines the claim that the Socratic persona was installed at all. The manuscript should either report per-scenario raw outputs with confidence intervals and demonstrate that the cross-lingual rates differ from each other and from baseline, or substantially weaken the 'decoupling' claim.
  2. [Sec. IV-F/Fig. 4 vs. Sec. IV-C/Table II; appended clarification] The two cross-lingual experiments give incompatible results for German: 0% strict QR in Table II but 48% any-question rate in Fig. 4 (right). The appended clarification states that the two metrics are 'not directly comparable,' which means the manuscript cannot cite both as evidence of the same stable cross-lingual stratification. The 'degradation pathways in morphologically distant targets' in the abstract is therefore not established. Please designate a single primary evaluation protocol (or pre-register both), and report all rates under both protocols for every language so readers can assess the discrepancy.
  3. [Sec. IV-E, Sec. IV-H, Eq. (15), Eq. (20)] The paper repeatedly calls empirical observations 'strict mathematical bounds,' 'proofs,' and 'mathematical sufficiency' (e.g., Sec. IV-E, and the bullet list in the Introduction). No derivation of a bound is provided; the claims are curve readings from validation sets. Moreover, the optimal rank, learning rate, dropout, and epoch window are selected from the same validation curves on which the optimum is then reported (Secs. IV-E and IV-H), with no held-out test set or nested procedure. As a result, the reported 'generalization gap' and 'global optimum' values are in-sample selections. Please either supply a formal derivation or reframe these as empirical observations with out-of-sample evaluation and uncertainty estimates.
  4. [Sec. III-F, Eq. (9), Ethical Considerations] The core behavioral metric QR counts any response whose terminal token is '?' as Socratic. This conflates genuine counter-questions with rhetorical questions, formatting artifacts, and template effects, and the paper itself concedes that QR 'serves strictly as a structural proxy' for inquiry. Given that the central claims (DPO decoupling behavior from syntax, zero-shot persona transfer) rest entirely on this proxy, the manuscript needs independent validation of the metric—for example, human annotation of a sample of responses or a secondary semantic/act-level measure—before those claims can be accepted.
minor comments (6)
  1. [Abstract, Sec. IV-E, Sec. IV-G, Sec. IV-H] The reported 'global' optimum validation loss is inconsistent across sections: 0.919 (Sec. IV-E), 0.7856 (Sec. IV-G), and 0.9277±0.0162 (Sec. IV-H). Please state explicitly which dataset split and evaluation condition each number refers to.
  2. [Sec. III-A and Sec. IV-B] The paper mentions a 405-job sweep (Sec. III-A, IV-H) and a 72-job grid (Sec. IV-B) without explaining how the two relate; please clarify whether the 72-job grid is a subset of the 405-job sweep or a separate experiment.
  3. [Sec. III-E and Table II] Sec. III-E defines the evaluation matrix as 18 scenarios × 7 languages (126 evaluations), but Table II reports n=10 per language; please explain how the 10 scenarios were selected from the 18 and whether the remaining 8 scenarios were also evaluated.
  4. [Eq. (1)] Eq. (1) uses P for token count while PPL is used for perplexity and N for parameter count; the notation should be made consistent to avoid confusion.
  5. [Sec. IV-C and Table II] Table II reports Slovak (source) with 0% QR, but Sec. IV-C says the source of the SFT/DPO data is Slovak; please reconcile this with the claim that the persona was successfully installed in the source language, or explicitly discuss the possibility that the terminal-'?' criterion under-detects the behavior.
  6. [Sec. IV-H] Sec. IV-H lists r∈{4,8,16} for the grid while other experiments include r=32; please justify the omission or include r=32 in the sweep description.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: the claimed 'optimal' LoRA rank, learning rate, epoch window, and dropout are selected as minima of the same validation curves used to report them as 'mathematical bounds'; no held-out test separates fit from prediction. The cross-lingual transfer claim is not circular but is statistically fragile.

  1. fitted input called prediction [Sec. IV-H ('Global Optimum Identification') and Sec. IV-I / Abstract ('precise mathematical bounds for PEFT')]
    "the absolute peak configuration across the entire 405-job optimization grid was achieved by the hyperparameter vector combining r=16, η=2×10−4, e=3, and d_drop=0.10. This specific configuration established a mean evaluation loss of μ_loss = 0.9277±0.0162 and a minimal conditional perplexity of PPL=2.529±0.043. ... Our exhaustive evaluations across the Leonardo supercomputing infrastructure confirm that robust, low-resource behavioral anchoring of foundational language models is achieved under strict structural boundaries."

    The 'global optimum' and the 'strict structural boundaries' are the argmin of the aggregated validation-loss surface defined by Eqs. (20)-(21) and the epoch-ablation validation curves (Fig. 3). There is no held-out test set or temporal split; the hyperparameter vector is selected because it attains the minimum of these exact curves, and the same minimum is then reported as an independently established 'mathematical bound' on generalization. The bound is therefore a restatement of the fitted argmin: the claim that generalization is optimal at e∈[2,3] and r=16 is true by construction of the selection rule, not by an independent prediction. The paper's own clarification that Exp. 3 and Exp. 6 use non-comparable metrics further shows the QR evidence is not a stable independent measure.

full rationale

The LoRA and DPO equations are standard external formulations (Refs. [10], [18]), so there is no equation-level circularity and no self-citation chain. The only reduction of a claimed result to its own inputs is the hyperparameter-sweep 'bounds': the optimal rank, learning rate, epoch, and dropout are chosen as the minima of the validation curves that are then cited as proof of the same bounds. This is a partial fitted-input-as-prediction circularity and warrants score 4. The central zero-shot persona-transfer claim is not circular—it is an empirical claim about held-out languages—but it is statistically fragile: QR (Eq. 9) is a terminal-'?' proxy on n=10 scenarios, the source-language Slovak rate is 0%, and the paper concedes QR 'serves strictly as a structural proxy' and that Exp. 3 vs Exp. 6 are 'not directly comparable.' Those are measurement and validity concerns, not circularity, and are flagged separately.

Assumptions & free parameters 7 free parameters · 8 assumptions · 0 invented entities

No new physical entities are introduced. The paper's claims rest on standard PEFT and DPO equations plus domain assumptions about the QR metric, the adequacy of tiny evaluation sets, and the interpretation of cross-lingual results. Its 'bounds' are empirical hyperparameter settings, not derived quantities.

free parameters (7)
  • LoRA rank r = 16
    Selected as optimal from sweeps over r in {4,8,16,32}; central threshold claim.
  • LoRA alpha = 32
    Set as 2*r in Experiment 1 and in the optimal configuration; scaling factor affects effective learning rate.
  • LoRA dropout = 0.10
    Chosen in the optimal 405-job configuration; reported to mitigate variance at epoch 3.
  • Peak learning rate = 2e-4
    Optimal base learning rate from the sweep over eta in {5e-5, 1e-4, 2e-4}.
  • Epoch budget e = 3 (window [2,3])
    Optimal window from epoch ablation; validation loss rises beyond this window.
  • DPO beta = 0.15
    KL penalty coefficient in the DPO loss, fixed as a strict penalty.
  • DPO learning rate = 5e-5
    One epoch of DPO at learning rate 5e-5.
assumptions (8)
  • standard math LoRA forward pass h = W0 x + (alpha/r) B A x (Eqs 3, 10)
    Adopts the standard low-rank adaptation formulation without modification.
  • standard math DPO loss (Eq 5)
    Uses the standard DPO objective from Rafailov et al., assuming preference pairs and a reference policy.
  • standard math Cross-entropy loss and perplexity definitions (Eqs 8, 13, 18, 19)
    Standard language modeling metrics used for linguistic coherence and convergence.
  • domain assumption Question Rate QR = fraction of responses ending with '?' (Eq 9)
    Assumes a terminal question mark is a valid proxy for Socratic assertive behavior.
  • domain assumption Evaluation matrix Deval = S x L with 18 scenarios x 7 languages (Eq 7, Sec III-E)
    Treats 126 evaluations, and later 10 per language, as a reliable estimate of persona transfer.
  • domain assumption Base vs Instruct comparison isolates instruction tuning (Sec IV-B)
    The authors compare Llama-3-8B base with Llama-3.1-8B-Instruct, and their own Methodological Note acknowledges version-specific architectural changes, so the instruction-tuning prerequisite is not cleanly isolated.
  • ad hoc to paper Tokenization fragmentation explains cross-lingual collapse (Sec IV-C, V-B)
    Attributed without token-level measurements or a quantitative test of the explanation.
  • domain assumption Small curated adversarial prompts are more accurate than large automated benchmarks (Sec III-E)
    Methodological choice asserted in the text without empirical justification against larger benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Behavioral Reprogramming of Open-Weights Models: Cognitive Plasticity and Alignment Bounds." pith.science (2026). https://pith.science/paper/LLACOIRW

@misc{pith2026260813069,
  author       = {Pith},
  title        = {Pith review of: Behavioral Reprogramming of Open-Weights Models: Cognitive Plasticity and Alignment Bounds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LLACOIRW}},
  note         = {Machine review of arXiv:2608.13069}
}
abstract

Large language models (LLMs) are predominantly aligned to function as passive, sycophantic assistants. We challenge this default paradigm by empirically evaluating the cognitive plasticity of open-weight architectures when subjected to rigorous behavioral reprogramming. Our objective is to induce a proactive, Socratic conversational framework, characterized by high-frequency question generation under strictly constrained high-performance computing (HPC) conditions. Through a massively parallelized hyperparameter sweep comprising 405 HPC jobs, we define precise mathematical bounds for parameter-efficient fine-tuning (PEFT). We identify an architectural threshold at LoRA rank $r=16$ and demonstrate via extensive epoch ablation that generalization capacity strictly reaches its optimal convergence within an optimized training window of $e \in [2, 3]$ depending on dataset density (minimum validation loss of 0.919). Furthermore, scaling model capacity to 14B parameters yielded a lower localized evaluation perplexity (1.414). Subsequent Direct Preference Optimization (DPO) successfully decoupled the underlying assertive behavior from localized syntax, while rigorous cross-lingual stress testing reveals both the capabilities and the structural boundaries of zero-shot persona transfer, demonstrating robust alignment in closely related linguistic families alongside identifiable degradation pathways in morphologically distant targets. These findings establish a rigorous empirical framework for compute-efficient, cross-lingual behavioral modification.

Figures

Figures reproduced from arXiv: 2608.13069 by the authors.

Figure 1
Figure 1. Learning curves illustrating the convergence trajectories of Evaluation [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparative boxplot distribution of Evaluation Loss for Base [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Epoch Ablation trajectory illustrating Best Evaluation Loss across [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparative evaluation of Question Response Rate (QR) across [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Empirical evaluation loss distribution (mean [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages

  1. [1]

    Attention is All You Need,

    A. Vaswani et al., “Attention is All You Need,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017

    work page 2017

  2. [2]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,

    J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of NAACL-HLT, pp. 4171–4186, 2019

    work page 2019

  3. [3]

    Language Models are Few-Shot Learners,

    T. B. Brown et al., “Language Models are Few-Shot Learners,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, pp. 1877–1901, 2020

    work page 1901

  4. [4]

    Llama: Open and Efficient Foundation Language Models,

    H. Touvron et al., “Llama: Open and Efficient Foundation Language Models,”arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  5. [5]

    Llama 2: Open Foundation and Fine-Tuned Chat Models,

    H. Touvron et al., “Llama 2: Open Foundation and Fine-Tuned Chat Models,”arXiv preprint arXiv:2307.09288, 2023

    arXiv 2023

  6. [6]

    The Llama 3 Herd of Models,

    A. Dubey et al., “The Llama 3 Herd of Models,”arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  7. [7]

    Qwen3 Technical Report,

    Qwen Team, “Qwen3 Technical Report,”arXiv preprint arXiv:2505.09388, 2025

    arXiv 2025

  8. [8]

    Mistral 7B,

    A. Q. Jiang et al., “Mistral 7B,”arXiv preprint arXiv:2310.06825, 2023

    arXiv 2023

Show all 40 references
  1. [9]

    A Survey of Large Language Models,

    W. X. Zhao et al., “A Survey of Large Language Models,”IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 3, pp. 1152–1173, 2024

    2024

  2. [10]

    LoRA: Low-Rank Adaptation of Large Language Mod- els,

    E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Mod- els,” inInternational Conference on Learning Representations (ICLR), 2022

    2022

  3. [11]

    QLoRA: Efficient Finetuning of Quantized LLMs,

    T. Dettmers, A. Punnakkal, A. Lewis, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” inAdvances in Neural Infor- mation Processing Systems (NeurIPS), vol. 36, 2023

    2023

  4. [12]

    The Power of Scale for Parameter-Efficient Prompt Tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The Power of Scale for Parameter-Efficient Prompt Tuning,” inProceedings of EMNLP, pp. 3045–3059, 2021

    2021

  5. [13]

    Parameter-Efficient Transfer Learning for NLP,

    N. Houlsby et al., “Parameter-Efficient Transfer Learning for NLP,” in International Conference on Machine Learning (ICML), pp. 2790–2799, 2019

    2019

  6. [14]

    Prefix-Tuning: Optimizing Continuous Prompts for Generation,

    X. L. Li and P. Liang, “Prefix-Tuning: Optimizing Continuous Prompts for Generation,” inProceedings of ACL-IJCNLP, pp. 4582–4597, 2021

    2021

  7. [15]

    Good, Better, Adapt: Exploring Parameter-Efficient Fine-Tuning for Cross-Lingual Transfer,

    P. R. Rust et al., “Good, Better, Adapt: Exploring Parameter-Efficient Fine-Tuning for Cross-Lingual Transfer,” inProceedings of EMNLP, pp. 5580–5592, 2021

    2021

  8. [16]

    DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales,

    Y . Shao et al., “DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales,”arXiv preprint arXiv:2308.01336, 2023

    2023 arXiv

  9. [17]

    Training language models to follow instructions with human feedback,

    L. Ouyang et al., “Training language models to follow instructions with human feedback,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 35, pp. 27730–27744, 2022

    2022

  10. [18]

    Direct Preference Optimization: Your Language Model is Se- cretly a Reward Model,

    R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct Preference Optimization: Your Language Model is Se- cretly a Reward Model,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023

    2023

  11. [19]

    Deep reinforcement learning from human preferences,

    P. F. Christiano et al., “Deep reinforcement learning from human preferences,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 30, pp. 4299–4307, 2017

    2017

  12. [20]

    Learning to summarize with human feedback,

    N. Stiennon et al., “Learning to summarize with human feedback,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, pp. 3008–3021, 2020. 13

    2020

  13. [21]

    KTO: Model Alignment as Prospect Theory,

    K. Ethayarajh, W. Xu, N. Muennighoff, D. Jurafsky, and D. Kiela, “KTO: Model Alignment as Prospect Theory,” inInternational Con- ference on Machine Learning (ICML), 2024

    2024

  14. [22]

    What is the State of Neural Network Pruning?

    D. Blalock, J. J. G. Ortiz, D. Frankle, and J. Guttag, “What is the State of Neural Network Pruning?” inProceedings of MLSys, 2020

    2020

  15. [23]

    Scaling Laws for Neural Language Models,

    J. Kaplan et al., “Scaling Laws for Neural Language Models,”arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  16. [24]

    Training Compute-Optimal Large Language Mod- els,

    J. Hoffmann et al., “Training Compute-Optimal Large Language Mod- els,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 35, pp. 30016–30030, 2022

    2022

  17. [25]

    OPT: Open Pre-trained Transformer Language Models,

    S. Zhang et al., “OPT: Open Pre-trained Transformer Language Models,” arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  18. [26]

    Sparks of Artificial General Intelligence: Early experiments with GPT-4,

    S. Bubeck et al., “Sparks of Artificial General Intelligence: Early experiments with GPT-4,”Microsoft Research Technical Report, arXiv:2303.12712, 2023

    2023 arXiv

  19. [27]

    Red Teaming Language Models with Language Models,

    E. Perez et al., “Red Teaming Language Models with Language Models,” inProceedings of EMNLP, pp. 3419–3448, 2022

    2022

  20. [28]

    Red Teaming Language Models to Reduce Harms: Methods, Scaling Roles, and Lessons Learned,

    D. Ganguli et al., “Red Teaming Language Models to Reduce Harms: Methods, Scaling Roles, and Lessons Learned,”arXiv preprint arXiv:2209.07858, 2022

    2022 arXiv

  21. [29]

    A General Language Assistant as a Laboratory for Alignment,

    A. Askell et al., “A General Language Assistant as a Laboratory for Alignment,”arXiv preprint arXiv:2112.00861, 2021

    2021 arXiv

  22. [30]

    Alignment of Language Models Desiderata: Safety, Robustness, and Compliance,

    Z. Kenton et al., “Alignment of Language Models Desiderata: Safety, Robustness, and Compliance,”arXiv preprint arXiv:2103.14659, 2021

    2021 arXiv

  23. [31]

    On the Opportunities and Risks of Foundation Models,

    R. Bommasani et al., “On the Opportunities and Risks of Foundation Models,”Stanford University Whitepaper, arXiv:2108.07258, 2021

    2021 arXiv

  24. [32]

    Measuring Massive Multitask Language Un- derstanding,

    D. Hendrycks et al., “Measuring Massive Multitask Language Un- derstanding,” inInternational Conference on Learning Representations (ICLR), 2021

    2021

  25. [33]

    Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models,

    A. Srivastava et al., “Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models,”Transactions on Machine Learning Research (TMLR), 2023

    2023

  26. [34]

    A Comprehensive Evaluation of Large Language Models on Reasoning Tasks,

    X. Wang et al., “A Comprehensive Evaluation of Large Language Models on Reasoning Tasks,”IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024

    2024

  27. [35]

    InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models,

    L. Chen et al., “InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023

    2023

  28. [36]

    Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing,

    P. Liu et al., “Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing,”ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023

    2023

  29. [37]

    Parameter-Efficient Fine-Tuning for Large Language Models: A Comprehensive Survey,

    Y . Zhao et al., “Parameter-Efficient Fine-Tuning for Large Language Models: A Comprehensive Survey,”IEEE Transactions on Neural Net- works and Learning Systems, 2024

    2024

  30. [38]

    Language Models are Unsupervised Multitask Learners,

    A. Radford et al., “Language Models are Unsupervised Multitask Learners,”OpenAI Technical Report, 2019

    2019

  31. [39]

    GPT-4 Technical Report,

    OpenAI, “GPT-4 Technical Report,”arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  32. [40]

    Constitutional AI: Harmlessness from AI Feedback,

    Y . Bai et al., “Constitutional AI: Harmlessness from AI Feedback,”arXiv preprint arXiv:2212.08073, 2022. DATAAVAILABILITYSTATEMENT To comply with open science standards, the anonymized code, configuration files, and representative behavioral datasets required to reproduce the...

    2022 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.