Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

EnSToM: Enhancing Dialogue Systems with Entropy-Scaled Steering Vectors for Topic Maintenance

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EnSToM scales a refusal steering vector by the model's own generation entropy, lifting topic-adherence accuracy on the CantTalkAboutThis banking domain from 0.610 to 0.802.

desk verdict Entropy-scaled steering is a genuinely new mechanism, but test-set-tuned hyperparameters make the headline gains optimistic; the paper needs a held-out split. read the letter →

arxiv 2505.16526 v1 pith:OH5ARHJQ submitted 2025-05-22 cs.CL

Heejae Suh , Yejin Jeon , Deokhyung Kang , Taehee Park , Yejin Min , Gary Geunbae Lee This is my paper
classification cs.CL
keywords activationengineeringsteeringvectorsentropy-basedscalingtopicmaintenancetask-orienteddialoguesmalllargelanguagemodelsrefusalbehaviorCantTalkAboutThis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EnSToM is a training-free, inference-time method for keeping small language models on topic in task-oriented dialogues. The paper claims that a refusal steering vector should not be applied at constant strength; instead, the model should measure its own uncertainty from the entropy of the first two generated tokens at a middle layer and scale the steering coefficient accordingly. On the CantTalkAboutThis banking domain, this raises overall topic-adherence accuracy from 0.610 with prompting alone to 0.802, driven mostly by distractor refusal improving from 0.282 to 0.709 while on-topic engagement only falls from 0.938 to 0.895. The paper also reports the same recipe improving a second architecture, working across domains with a vector extracted from a different domain, and remaining effective with only ten steering examples.

What carries the argument

The central object is the entropy-scaled steering coefficient $C_H^{(L)} = C_{\max}/(1+e^{-\alpha\delta(H^{(L)}-t)})$, where $H^{(L)}$ is the averaged softmax entropy at layer $L$ over the first two generated tokens, $C_{\max}=1.5$, $\alpha=5$, and $t=7.5$; $\delta$ is $\pm1$ depending on whether distractors have lower or higher entropy than on-topic inputs at that layer. This coefficient multiplies a steering vector $v$ extracted by contrastive activation addition: the averaged, norm-normalized difference between hidden states for desired refusal completions and undesired engaging completions, built from a small Steering QA dataset of paired answers. At inference, the adjusted hidden state is $h^{(l)} + C_H^{(L)} v$, with the entropy-extraction layer $L$ and the steering-injection layer $l$ chosen separately, e.g., $L=16$ with Steer@16. This mechanism is what lets the same vector act strongly on distractors and weakly on on-topic queries without retraining or prompt changes.

What would settle it

Run EnSToM with the reported hyperparameters on a fresh validation split of CantTalkAboutThis, or on a new domain, that was not used to set L=16, t=7.5, and delta=-1. For each input, record the entropy H(16) and the steering coefficient C, then check whether inputs with C>=1.0 are overwhelmingly distractors and inputs with C<0.5 are on-topic. The paper's own Table 6 shows that in the low-coefficient region distractor accuracy falls to 0.417-0.533; if that overlap is large enough on the fresh split to pull overall accuracy below the reported 0.802, the entropy gate is not generalizing.

Watch

Extended reading notes

Core claim

The paper's central claim is that internal generation entropy can act as a per-input gate deciding how hard to apply a refusal steering vector. For a query, EnSToM computes the entropy of the token distribution at a chosen layer during the first two generated tokens, maps that entropy through a sigmoid to a steering coefficient, and adds the coefficient-scaled steering vector to the hidden states before continuing generation. On the CantTalkAboutThis banking domain, EnSToM raises overall topic-adherence accuracy from 0.610 (prompt-only) to 0.802 at L=16, Steer@16, lifting distractor refusal from 0.282 to 0.709 while on-topic accuracy moves from 0.938 to 0.895; vanilla steering, by contrast, reaches 0.80 distractor accuracy but drops on-topic accuracy to 0.70. The paper interprets the entropy signal as a general property of the model's layer-wise attention: middle layers concentrate on distractor tokens and produce lower entropy, while on-topic inputs spread attention and produce higher entropy, with the ordering reversing in deeper layers. It reports the same recipe working on a second architecture, on cross-domain steering vectors, and with as few as ten steering examples.

Load-bearing premise

The method assumes the entropy of the first two generated tokens at a single fixed layer separates off-topic from on-topic inputs on unseen data, with the threshold, sign, and layer chosen from the evaluation set (t=7.5, L=16, delta=-1) continuing to hold; if the overlap region is large or the ordering flips, the coefficient scaling steers many inputs in the wrong direction.

Editorial extensions

If this is right

  • On the banking domain of CantTalkAboutThis, EnSToM's best configuration (L=16, Steer@16, t=7.5) raises overall topic-adherence accuracy from 0.610 with prompt-only prompting to 0.802, with distractor refusal improving from 0.282 to 0.709 and on-topic accuracy decreasing only from 0.938 to 0.895.
  • Vanilla steering with a fixed coefficient reaches 0.80 distractor accuracy but drops on-topic accuracy to 0.70, so entropy scaling recovers most of the on-topic behavior that uniform steering destroys.
  • The entropy separation is not domain-specific: layer 16 shows the same lower-entropy-for-distractors pattern across the ten domains in the dataset, and a steering vector extracted from a different domain still improves banking-domain adherence.
  • The method transfers to another architecture: on the Ministral-8B-Instruct-2410 model, EnSToM raises overall accuracy from 0.62 with prompt-only prompting to 0.76, with distractor accuracy up from 0.25 to 0.65 at steering layer 17 and 0.63 at layer 18.
  • With only 10 steering-vector pairs, EnSToM still yields useful gains (distractor 0.67-0.74 and on-topic 0.85-0.90 at L=16, t=7.5), indicating the approach does not require large training sets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to reuse the two-token entropy measurement as a lightweight out-of-scope detector in production systems, since the method already produces a per-input score without a separate classifier.
  • Because the reported hyperparameters (L=16, t=7.5, delta=-1) were selected from the evaluation set's entropy distributions, a fair generalization test would fit the sigmoid on a validation split and then measure the gap on unseen data.
  • If layer-wise entropy is as stable a signal as the paper suggests, entropy-scaled scaling could be applied to other inference-time controls such as safety refusals, persona steering, or jailbreak defense wherever a contrastive steering direction exists.
  • The overlap region the paper acknowledges (hard negatives with similar entropy in both distributions) bounds the remaining headroom; combining entropy with a second internal signal, such as surprisal at a different layer, could push beyond the reported overall accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes EnSToM, a training-free activation-addition method that steers a small LLM toward topic maintenance in task-oriented dialogue. A steering vector is extracted from contrastive refusal/engaging pairs, and during inference its strength is scaled by a sigmoidal coefficient computed from the layer-wise entropy of the first two generated tokens. Experiments on the banking domain of CantTalkAboutThis with LLaMA-2-7B-Chat report an improvement in overall accuracy from 0.610 (prompt-only) to 0.802, with distractor accuracy rising from 0.282 to 0.709 and on-topic accuracy declining only from 0.938 to 0.895. The paper also reports results on Ministral-8B-Instruct-2410, a cross-domain steering-vector experiment, a jailbreak-defense pilot, and an analysis of per-layer entropy distributions.

Significance. If the reported gains are genuine, EnSToM is a lightweight and practical alternative to fine-tuning for enforcing scenario adherence in small LLMs, and the layer-wise entropy analysis offers a useful empirical characterization of how on-topic and distractor inputs differ internally. The paper provides code, a clear formulation of the scaling mechanism, ablations of sample size and threshold, and a variance analysis of steering vectors. The central weakness is that several key free parameters are selected on the very test set used for evaluation, so the magnitude of the improvement is not yet established.

major comments (3)
  1. [§4.2.2, Table 2, Figure 4] The hyperparameters L, t, δ, and the steering injection layer are selected using the test-set entropy distributions and accuracy results, and EnSToM is then evaluated on the same test set. Section 4.2.2 states that the threshold t is 'empirically set to 7.5', and Figure 4 sweeps t over what appears to be the evaluation data; Table 2 reports the best of the eight (L, Steer@) combinations. The manuscript's own Section 8 acknowledges that L and t require manual selection. This selection procedure makes the reported 0.802 overall accuracy an optimistic, selection-biased estimate of performance on new inputs. Please introduce a held-out validation split (or nested cross-validation) for choosing L, t, δ, and Steer@, and report the test performance of the single selected configuration; this is necessary to support the central claim of improved topic adherence.
  2. [§6.4, Appendix D.2] The cross-domain analysis does not test whether the entropy discriminator generalizes to new domains. Appendix D.2 varies only the domain from which the steering vector is extracted (banking, education, health, insurance), while the entropy extraction layer L=16, the threshold t=7.5, and the evaluation set all remain within the same banking test set. Consequently, these experiments cannot support the statement in Section 6.4 that the entropy pattern 'underscores the robustness of our observations and indicates a generalizable mechanism in the model's internal processing.' Please evaluate the full EnSToM pipeline—including the entropy layer, threshold, sign, and steering layer—on held-out domains, or restrict the generalization claim to what the experiments actually show.
  3. [§5.1, Conclusion] The main evaluation is restricted to the banking domain only, although the abstract and conclusion state that 'Evaluations on the CantTalkAboutThis dataset' demonstrate the improvement and that the method shows generalizability. Section 6.4 itself shows that the magnitude and even the sign of the entropy difference vary across layers and domains (Figure 6), so a single-domain result is insufficient to support the broad claim. Please provide results on at least one additional domain from the dataset (for example, health or insurance), or revise the claims to scope them to the banking domain.
minor comments (4)
  1. [§5.1] The evaluation uses GPT-4o to classify responses, but the paper does not report agreement with human labels or repeated-evaluation variance; given the reported differences between configurations are often small (e.g., 0.789 vs. 0.784 in Table 2), measurement noise could affect the ranking. Please report a few human annotations or an error analysis.
  2. [§6.2 and Table 3] The model name 'Minstral-8B-Instruct-2410' is misspelled; it should be 'Mistral-8B-Instruct-2410' as in the related work and the cited model announcement.
  3. [Figure 2] The layer labels in the figure are inconsistent with the text: the entropy extraction layer is labeled L in the text but the figure shows layers 15–17 with no clear mapping to L=16 and L=19. Please clarify the notation.
  4. [Appendix D.1] The sample-size experiment reports accuracy for banking_10 and banking_30, but the table is missing the vanilla (t='−') row for banking_10 in the Layer 15 column, which makes the comparison incomplete. Please add the missing values or explain the omission.

Circularity Check

2 steps flagged · score 6.0 of 10

Reported EnSToM gains are in-sample maxima: entropy layer L, threshold t, sign delta, and steering layer are selected from the same test set used for the accuracy numbers.

  1. fitted input called prediction [Section 4.2.2, entropy scaling definition and Figure 4 threshold sweep]
    "The slope parameter α, which controls the steepness of the sigmoid function, is set to 5, while the threshold entropy t is empirically set to 7.5. ... In order to adjust the scaling direction based on entropy differences between distractor and on-topic inputs, the parameter δ is set to -1 when the average entropy of distractors is lower than on-topic inputs (Layer 16) and +1 when it is higher (Layer 19)."

    These settings are not derived from a principled rule; they are read off the observed entropy distributions and accuracies on the evaluation data. Section 5.1 describes only a steering-vector split (100 samples) and the test set (550 per class), with no validation split, and Figure 4 sweeps t and reports accuracies on the same data. The entropy sign δ and threshold t are therefore fitted to the very test labels whose accuracy is later reported as EnSToM's result (Table 2), making the headline gain an in-sample fitted quantity rather than a held-out prediction.

  2. fitted input called prediction [Section 8, Limitations; Table 2]
    "Additionally, our current method requires manually selecting the entropy extraction layer L and threshold t. In this study, we empirically identified layers with the most pronounced distribution differences and manually set the coefficient scaling threshold."

    The paper concedes that L is chosen by inspecting the entropy distributions of the evaluation inputs (Figures 3/6), and Table 2 then reports the best of L∈{16,19} × Steer@∈{13,14,15,16} on that same test set. The selected L (16) is the layer with the 'more pronounced' separation, and the reported best accuracy (0.802) is the maximum of this grid. Since the entropy discriminator's layer, threshold, sign, and steering layer are all selected using the target data, the central accuracy claim is an in-sample optimum, not an independent validation of the entropy-scaling mechanism.

full rationale

Most of the paper's machinery is not circular: the steering vector is a standard contrastive activation addition built from a separate 100-sample set, the entropy computation is defined independently of the accuracy labels, and there are no load-bearing self-citations or imported uniqueness claims. The circularity is concentrated in the evaluation protocol. The method's entropy discriminator requires choosing L, t, and δ, and the paper explicitly states that these are set empirically from the entropy distributions. With only a steering-set/test-set split described in Section 5.1 and Figure 4 sweeping t on the evaluation data, these choices are fitted to the test set. Table 2 then reports the best of eight steering-layer/entropy-layer combinations on that same test set as EnSToM's performance, making the headline +0.192 overall gain an in-sample maximum rather than a prediction about unseen inputs. The cross-domain and cross-architecture experiments do not repair this: Appendix D.2 still evaluates on the same banking test set, and Table 3 reports a separately hand-picked L=28, t=3.0. The limitations section acknowledges the manual selection, so the paper itself supports this reading. Score 6 because the central reported numbers are fitted to the evaluation data, though the underlying steering mechanism retains independent content.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the hand-set entropy-scaling hyperparameters (t, alpha, Cmax, delta, L, steer layer) chosen from the test data, and on the domain assumptions that entropy separates input types, that GPT-4o-generated completions are adequate examples, and that GPT-4o evaluation is accurate. No new physical or architectural entities are introduced.

free parameters (7)
  • Entropy threshold t = 7.5 (LLaMA-2-7B-chat); 3.0 (Ministral-8B)
    Set empirically from test-set entropy distributions; Figure 4 sweeps t and selects the value maximizing overall accuracy on the test set (Sections 4.2.2 and 5.2, Figure 4).
  • Sigmoid slope alpha = 5
    Chosen by hand without a stated tuning procedure (Section 4.2.2).
  • Maximum coefficient Cmax = 1.5
    Taken from Rimsky et al. (2024) rather than derived for these models (Section 4.2.2).
  • Delta sign delta = -1 at L=16, +1 at L=19
    Set based on the observed mean entropy difference between distractor and on-topic inputs on the test set (Section 4.2.2).
  • Entropy extraction layer L = 16 and 19 for Llama-2-7B-chat; 28 for Ministral-8B
    Layers selected by inspecting test-set entropy distributions (Figures 3 and 6), i.e., selected on evaluation data (Section 4.2.1).
  • Steering injection layer = 13 to 16 for LLaMA, 17 and 18 for Ministral; best at 16 and 18 respectively
    Chosen from the reported grid based on test-set overall accuracy (Section 5.2, Tables 2 and 3).
  • Number of generated tokens k for entropy = 2
    Design choice for entropy computation; entropy is averaged over two generated tokens (Section 4.2.1).
assumptions (5)
  • domain assumption Layer-wise entropy at selected layers separates distractor from on-topic inputs
    The method assumes the first two generated tokens' layer-wise entropy is a stable discriminator across scenarios and domains. The paper demonstrates this on the test data (Figures 3 and 6) but acknowledges overlapping hard negatives in Limitations (Section 8).
  • domain assumption GPT-4o-generated refusal and engaging completions correctly instantiate desired and undesired behaviors
    The steering QA pairs (Section 4.1) are generated by GPT-4o, and the steering vector inherits any bias or quality limitations in these completions.
  • domain assumption GPT-4o evaluation labels refusals versus engaging responses correctly
    No human agreement or error analysis is reported for the GPT-4o evaluator (Section 5.1, Appendix H.3).
  • domain assumption Steering vector learned from one domain transfers to other domains
    Appendix D.2 reports cross-domain transfer on two target domains, but universal transferability is assumed for practical deployment rather than proven.
  • standard math Standard softmax and entropy definitions apply as used
    Entropy computation in Section 4.2.1 uses standard definitions with epsilon for numerical stability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EnSToM: Enhancing Dialogue Systems with Entropy-Scaled Steering Vectors for Topic Maintenance." pith.science (2026). https://pith.science/paper/OH5ARHJQ

@misc{pith2026250516526,
  author       = {Pith},
  title        = {Pith review of: EnSToM: Enhancing Dialogue Systems with Entropy-Scaled Steering Vectors for Topic Maintenance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OH5ARHJQ}},
  note         = {Machine review of arXiv:2505.16526}
}
read the original abstract

Small large language models (sLLMs) offer the advantage of being lightweight and efficient, which makes them suitable for resource-constrained environments. However, sLLMs often struggle to maintain topic consistency in task-oriented dialogue systems, which is critical for scenarios such as service chatbots. Specifically, it is important to ensure that the model denies off-topic or malicious inputs and adheres to its intended functionality so as to prevent potential misuse and uphold reliability. Towards this, existing activation engineering approaches have been proposed to manipulate internal activations during inference. While these methods are effective in certain scenarios, our preliminary experiments reveal their limitations in ensuring topic adherence. Therefore, to address this, we propose a novel approach termed Entropy-scaled Steering vectors for Topic Maintenance (EnSToM). EnSToM dynamically adjusts the steering intensity based on input uncertainty, which allows the model to handle off-topic distractors effectively while preserving on-topic accuracy. Our experiments demonstrate that EnSToM achieves significant performance gain with a relatively small data size compared to fine-tuning approaches. By improving topic adherence without compromising efficiency, our approach provides a robust solution for enhancing sLLM-based dialogue systems.

Figures

Figures reproduced from arXiv: 2505.16526 by the authors.

Figure 1
Figure 1. The example above illustrates that bots tend [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall process. After extracting steering vectors and applying entropy-based coefficient scaling, responses [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of entropy distribution in different [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Effect of entropy-based scaling at different [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Entropy distribution of on-topic and distractor [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Layer-wise entropy difference (distractor-on-topic) across domains. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Probabilistic Concept-Aware Steering for Trustworthy LLM Inference

    cs.AI 2026-05 reject novelty 4.0 of 10

    PCS improves steering direction accuracy by adaptively sampling the intervention coefficient from a cosine-similarity-conditioned Gaussian, but its evaluation is partly circular because the optimal coefficient is chos...

Reference graph

Works this paper leans on

29 extracted references · 5 canonical work pages · cited by 1 Pith paper

  1. [1]

    Andy Arditi, Oscar Balcells Obeso, Aaquib Syed, Daniel Paleka, Nina Rimsky, Wes Gurnee, and Neel Nanda. 2024. https://openreview.net/forum?id=pH3XAQME6c Refusal in language models is mediated by a single direction . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  2. [2]

    Amos Azaria and Tom Mitchell. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.68 The internal state of an LLM knows when it`s lying . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967--976, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-emnlp.68 2023

  3. [3]

    Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma. 2024. https://doi.org/10.1073/pnas.2311878121 Explaining neural scaling laws . Proceedings of the National Academy of Sciences (PNAS), 121(27):e2311878121

    doi:10.1073/pnas.2311878121 2024

  4. [4]

    Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. 2024. https://openreview.net/forum?id=Zj12nzlQbz INSIDE : LLM s' internal states retain the power of hallucination detection . In The Twelfth International Conference on Learning Representations

    2024

  5. [5]

    Glass, and Pengcheng He

    Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. 2024. https://openreview.net/forum?id=Th6NyL07na Dola: Decoding by contrasting layers improves factuality in large language models . In The Twelfth International Conference on Learning Representations

    2024

  6. [6]

    Ariel Gera, Roni Friedman, Ofir Arviv, Chulaka Gunasekara, Benjamin Sznajder, Noam Slonim, and Eyal Shnarch. 2023. https://doi.org/10.18653/v1/2023.acl-long.580 The benefits of bad advice: Autocontrastive decoding across model layers . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages...

    doi:10.18653/v1/2023.acl-long.580 2023

  7. [7]

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. 2023. https://arxiv.org/abs/2312.06674 Llama guard: Llm-based input-output safeguard for human-ai conversations . Preprint, arXiv:2312.06674

    arXiv 2023

  8. [8]

    Ziwei Ji, Delong Chen, Etsuko Ishii, Samuel Cahyawijaya, Yejin Bang, Bryan Wilie, and Pascale Fung. 2024. https://doi.org/10.18653/v1/2024.blackboxnlp-1.6 LLM internal states reveal hallucination risk faced with a query . In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 88--104, Miami, Florida, US. ...

    doi:10.18653/v1/2024.blackboxnlp-1.6 2024

Show all 29 references
  1. [9]

    Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2024. Propile: probing privacy leakage in large language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems (NeurIPS), NIPS '23, Red Hook, NY, U...

    2024

  2. [10]

    Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar

    Bruce W. Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. 2024. https://arxiv.org/abs/2409.05907 Programming refusal with conditional activation steering . Preprint, arXiv:2409.05907

    2024 arXiv

  3. [11]

    Shen Li, Liuyi Yao, Lan Zhang, and Yaliang Li. 2025. https://openreview.net/forum?id=kUH1yPMAn7 Safety layers in aligned large language models: The key to LLM security . In The Thirteenth International Conference on Learning Representations

    2025

  4. [12]

    Norman Mu, Sarah Chen, Zifan Wang, Sizhe Chen, David Karamardian, Lulwa Aljeraisy, Basel Alomair, Dan Hendrycks, and David Wagner. 2024. https://arxiv.org/abs/2311.04235 Can llms follow simple rules? Preprint, arXiv:2311.04235

    2024 arXiv

  5. [13]

    Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2024. https://arxiv.org/abs/2307.06435 A comprehensive overview of large language models . Preprint, arXiv:2307.06435

    2024 arXiv

  6. [14]

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Pai...

    2024 arXiv

  7. [15]

    Raut, Eyal Zimlichman, Satya Narayana Cheetirala, Girish N

    Dhavalkumar Patel, Ganesh S. Raut, Eyal Zimlichman, Satya Narayana Cheetirala, Girish N. Nadkarni, Benjamin S. Glicksberg, Robert M Freeman, Prem Timsina, and Eyal Klang. 2023. https://api.semanticscholar.org/CorpusID:260731967 The limits of prompt engineering in medical probl...

    2023

  8. [16]

    Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. https://doi.org/10.18653/v1/2024.acl-long.828 Steering llama 2 via contrastive activation addition . In Proceedings of the 62nd Annual Meeting of the Association for Computational L...

    2024 doi

  9. [17]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. https://doi.org/10.1145/3658644.3670388 "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models . In Proceedings of the 2024 on ACM SIGSAC Conference on Co...

    2024

  10. [18]

    Makesh Narsimhan Sreedhar, Traian Rebedea, Shaona Ghosh, Jiaqi Zeng, and Christopher Parisien. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.713 C ant T alk A bout T his: Aligning language models to stay on topic in dialogues . In Findings of the Association for Comput...

    2024 doi

  11. [19]

    Asa Cooper Stickland, Alexander Lyzhov, Jacob Pfau, Salsabila Mahdi, and Samuel R. Bowman. 2024. https://openreview.net/forum?id=tfXIZ8P4ZU Steering without side effects: Improving post-deployment control of language models . In Neurips Safe Generative AI Workshop 2024

    2024

  12. [20]

    Nishant Subramani, Nivedita Suresh, and Matthew Peters. 2022. https://doi.org/10.18653/v1/2022.findings-acl.48 Extracting latent steering vectors from pretrained language models . In Findings of the Association for Computational Linguistics: ACL 2022, pages 566--581, Dublin, I...

    2022 doi

  13. [21]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  14. [22]

    Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. 2023. https://doi.org/10.48550/arXiv.2308.10248 Activation addition: Steering language models without optimization . CoRR, abs/2308.10248

    2023 doi arXiv

  15. [23]

    Haoran Wang and Kai Shu. 2024. https://doi.org/10.1145/3627673.3679821 Trojan activation attack: Red-teaming large language models using steering vectors for safety-alignment . In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIK...

    2024

  16. [24]

    Rossi, Sungchul Kim, and Shuai Li

    Yu Xia, Fang Kong, Tong Yu, Liya Guo, Ryan A. Rossi, Sungchul Kim, and Shuai Li. 2024. https://doi.org/10.1145/3589334.3645420 Which llm to play? convergence-aware online model selection with time-increasing bandits . In Proceedings of the ACM Web Conference 2024, WWW '24, pag...

    2024

  17. [25]

    Yueqi Xie, Minghong Fang, Renjie Pi, and Neil Gong. 2024. https://doi.org/10.18653/v1/2024.acl-long.30 G rad S afe: Detecting jailbreak prompts for LLM s via safety-critical gradient analysis . In Proceedings of the 62nd Annual Meeting of the Association for Computational Ling...

    2024 doi

  18. [26]

    Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. 2024. https://doi.org/10.18653/v1/2024.acl-long.303 S afe D ecoding: Defending against jailbreak attacks via safety-aware decoding . In Proceedings of the 62nd Annual Meeting of the As...

    2024 doi

  19. [27]

    Li-Ming Zhan, Haowen Liang, Bo Liu, Lu Fan, Xiao-Ming Wu, and Albert Y.S. Lam. 2021. https://doi.org/10.18653/v1/2021.acl-long.273 Out-of-scope intent detection with self-supervision and discriminative training . In Proceedings of the 59th Annual Meeting of the Association for...

    2021 doi

  20. [28]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  21. [29]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.