Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Investigating task-specific prompts and sparse autoencoders for activation monitoring

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Prompted probing beats raw activation probing for LLM monitoring when inference-time compute is available, and sparse-autoencoder probing wins when it is not.

desk verdict Useful empirical map of probing methods for LLM monitoring, with a shaky zero-shot baseline and no error bars; the within-probing rankings are plausible. read the letter →

arxiv 2504.20271 v1 pith:MT4WEPQU submitted 2025-04-28 cs.LG

Henk Tillman , Dan Mossing This is my paper · ORCID
classification cs.LG
keywords activationmonitoringlinearprobingsparseautoencoderspromptedzero-shotpromptinghallucinationdetectionsentimentclassificationLLMsafety
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Monitoring a chat-trained language model's outputs is a resource-allocation problem: every method trades labeled data, inference-time compute, and training-time compute. This paper develops and compares prompted probing, raw activation probing, sparse-autoencoder probing, and zero-shot prompting on four tasks, and it argues that no single method dominates. The central ranking is that zero-shot prompting is a reasonable baseline when extra forward passes are cheap; with enough labeled data, activation probes clearly beat it; prompted probing is the most data-efficient probe and generalizes well, so it should be used when inference-time compute is available; and SAE-based probing beats raw activation probing when it is not. The practical takeaway is a compute-budget-dependent decision rule for building output monitors without fine-tuning the model.

What carries the argument

The machinery that carries the argument is prompted probing, defined as a linear probe trained on the last-token residual-stream activation from a forward pass whose prompt explicitly asks the model to judge the target category. The prompt slot is varied with a suffix-only template that shares a single forward pass across categories, which makes the inference-time cost small. The supporting machinery is SAE max-pooled probing: a sparse autoencoder with 512k latents and TopK activation is trained offline, the TopK nonlinearity is swapped for a JumpReLU at probe time so latents can be computed independently, the latent activations are max-pooled across tokens, and a logistic regression probe is trained on the top $Q=1000$ latents picked by a class-mean-difference score. Max-pooling is the crucial step for SAE probes and hurts raw-activation probes, so the privileged SAE basis is what carries the no-prompting regime.

What would settle it

Run the same four task types on a broader set of chat models and a held-out set of monitoring categories, recording AUROC at 50 and 1000 training positives. The central recommendation is refuted if prompted probing is not among the top methods in the low-data regime on some new task, or if SAE-based probing does not beat raw activation probing when prompting is unavailable.

Watch

Extended reading notes

Core claim

The paper reports that monitoring quality is not a matter of picking one universally best method; it depends on which compute budget is available. On four tasks (harassment, violence, factual hallucination, and sentiment) with a single chat-trained model, it finds that zero-shot prompted output is a strong baseline, but all trained probing techniques can beat it once enough labeled examples are available. Prompted probing—prompting the model with the monitoring question and training a logistic-regression probe on the last-token activation—is the most data-efficient technique, and it generalizes best under English-to-non-English and single-message-to-conversation shifts. When prompting is too expensive, max-pooled probing on sparse-autoencoder latents outperforms probing raw activations, although its out-of-distribution generalization is sometimes worse. Combinations of these methods do not stack: adding SAE latents to prompted probing, or prompted model output to prompted probing, adds little beyond the stronger single method.

Load-bearing premise

The ranking of methods holds only if the four evaluation tasks and the single chat model tested are representative of monitoring settings generally; if the ranking is task-specific or model-specific, the compute-budget recommendation does not transfer.

Editorial extensions

If this is right

  • With enough labeled data, any of the trained activation-probing methods beats zero-shot prompted output on the tasks studied.
  • When inference-time compute is available, prompted probing is the best default because it needs fewer labels and generalizes well under distribution shift.
  • When inference-time compute is not available, SAE max-pooled probing is preferable to raw activation probing, subject to its weaker out-of-distribution generalization.
  • Combining prompted probing with SAE latents or with the model's own output does not meaningfully improve on the better single method, so practitioners can skip the stacking complexity.
  • Practical defaults that follow from the experiments: save activations near 75% of model depth, use about 1,000 SAE latents, and max-pool only when probing SAE latents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the strong low-data performance of prompted probing suggests it could be used to build monitors for new categories with only tens of labeled examples, which the paper's data-efficiency curves hint at but do not formally test.
  • Beyond the paper, if a monitoring model and the policy share weights, prompting-based methods may be vulnerable to collusion triggered by the prompt; the paper raises this concern, and an extension would compare un-prompted probes against prompted probes under adversarial optimization pressure.
  • Beyond the paper, the finding that prompted probing concentrates task-relevant signal in the final-token activation could be tested directly with activation-patching experiments to identify which attention heads move that signal.
  • Beyond the paper, the paper's pessimistic zero-shot baseline leaves open that a reasoning model with chain-of-thought might beat all probing methods; a natural extension is to rerun these benchmarks with reasoning tokens before the answer.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper systematically compares monitoring methods for LLM outputs: direct zero-shot prompting, raw last-token activation probing, prompted probing (probing activations after a task-specific prompt), SAE-based probing (with max-pooling or pre-activations), the LAT Scan method of Zou et al., and few-shot prompting. Experiments are run on a single ChatGPT-4o snapshot across four binary tasks: harassment and violence moderation, SimpleQA hallucination detection, and RottenTomatoes sentiment classification. The authors report AUROC as a function of the number of positive training examples, and also examine out-of-distribution generalization on the moderation datasets. Their central claims are: (1) zero-shot prompting is a strong baseline when inference-time compute is cheap; (2) activation probing methods substantially outperform it when training data are sufficient; (3) prompted probing is recommended when inference-time compute is available because of data efficiency and generalization; (4) when inference-time compute is limited, SAE-based probing outperforms raw activation probing. They also report that prompted probing and SAE probing do not stack substantially, and that max-pooling helps SAE probing but hurts raw activation probing.

Significance. If the stated rankings are correct, the paper provides concrete, compute-budget-dependent guidance for practitioners choosing among activation-monitoring methods, a question on which recent literature reports mixed results. The work's strengths include systematic comparison of several methods on multiple tasks, inclusion of the LAT Scan baseline, replication of known findings (max-pooling helps SAEs, hurts raw probes), and an explicit acknowledgment of limitations in Section 5. Because the main recommendations depend on empirical rankings, the absence of a separate validation procedure and the acknowledged pessimism of the zero-shot baseline are consequential; the practical guidance cannot be fully supported until those issues are addressed. The paper is nonetheless a useful contribution to the empirical literature on activation monitoring, and its claims are crisply falsifiable.

major comments (4)
  1. [§3.4, Figures 7 and 10] Hyperparameters such as the model layer depth (75%), Q for SAE latents, and logistic regression regularization C are selected by sweeping AUROC on the same fixed test sets used for the final reported results. The text does not describe any held-out validation set or nested cross-validation. This selection on the evaluation set introduces optimistic bias into the reported AUROCs and can inflate apparent differences between methods, so the quantitative rankings in Figures 2-6 may be distorted. Please rerun the final evaluations on a separate validation split (or use nested cross-validation) and report the corresponding results.
  2. [§5, second limitation paragraph] The authors themselves state that their estimates of zero-shot monitoring performance are likely pessimistic because they did not engineer zero-shot prompts and did not use a reasoning model or test-time compute. Since zero-shot prompting is the direct comparator in Figure 2 and the basis for the abstract's claim that it is a reasonable baseline, a stronger zero-shot baseline (e.g., rubric-based prompting, chain-of-thought, or best-of-N sampling) could plausibly change the ranking, particularly in the low-data regime where prompted probing is said to have superior data efficiency. The paper's central recommendation for prompted probing therefore needs either a stronger zero-shot baseline or substantially qualified conclusions.
  3. [§3.5, §4.3] The zero-shot baseline uses the prefix+suffix template, while the main prompted-probing results use the suffix-only template. The authors note in §4.3 that 'for all experiments in this work we only show results from suffix-only prompting' and that 'adding a prefix in prompted probing would slightly improve on the key results.' This means the comparison between zero-shot prompting and prompted probing confounds method differences with template differences. To support the claim that prompting is important and that prompted probing is data-efficient, the zero-shot baseline should be evaluated with the same suffix-only template (or the main prompted-probing results should be shown for the matching prefix+suffix template).
  4. [§4, Figures 2-6] No error bars, confidence intervals, or repeated-seed runs are reported anywhere in Section 4. Many claims are based on differences between methods that appear modest (e.g., prompted probing vs. SAE max-pooled probing in high-data regimes, and the mixed OOD results in Figure 3). Without uncertainty quantification or at least multiple random training-set subsamples, it is difficult to know whether the stated rankings are reliable, particularly in the small-sample regime (e.g., 5-20 positive examples). Please add repeated runs or confidence intervals, or explicitly report the variability.
minor comments (5)
  1. [§3.2] The prefix-only prompting variant mentioned in §4.3 and Figure 8 is not defined in Section 3.2; please describe it in the methods section for completeness.
  2. [§3.4, LAT Scan description] The text says LAT Scan is unsupervised because 'we do not use labels at any step prior to computing the first principal component,' but the sign of the direction is chosen by maximizing AUROC on the train set. This use of labels is minor but should be acknowledged explicitly to avoid confusion about the method's supervision requirements.
  3. [§3.1] The description of balancing says 'We always balance the positive and negative examples in the training set,' but test sets are not balanced (e.g., 149 positive vs. 373 negative for harassment). Please clarify why this imbalance is acceptable for AUROC reporting and whether any class-weighting was used in the logistic regression.
  4. [Figure 6] The caption and text in §4.2 say a '2-level classifier' combines a probe logit with the model's output logit, but Section 3.7 describes a second-level classifier on the two logits; the figure description is vague about how the two signals are combined (concatenation, weighted sum, etc.). Please be explicit.
  5. [§2, related work] The related-work section mentions several prior comparisons of SAE probing vs. linear probing, including negative results from Kantamneni et al. and Smith et al., but the discussion in Section 5 describing the paper's position relative to those works is brief. A short summary in the related-work section of why results might differ (e.g., model, SAE training data, tasks) would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical comparison on held-out test splits, with no fitted parameter renamed as a prediction and no load-bearing self-citation.

full rationale

This paper does not claim to derive any monitoring ranking from first principles; it reports AUROC comparisons on held-out test splits for moderation, hallucination, and sentiment tasks. Each method is operationally defined in Section 3, trained on the 80% split, and evaluated on the fixed 20% split, so the central comparisons are not defined in terms of their own outputs. Prompted probing is explicitly a simplified variant of LAT from Zou et al. (2023), and the SAE probe follows Bricken et al. (2024); the paper's contribution is comparative measurement, not a derivation that reduces to its inputs. The one self-citation, Gao et al. (2024), is used only to justify the TopK SAE training recipe and hyperparameters in Section 3.3; it is methodological support rather than a load-bearing premise for the headline recommendation. The paper's own limitation statement in Section 5 ('our estimates of zero-shot monitoring performance are likely pessimistic') and the differing templates used for zero-shot versus suffix-only prompted probing are threats to the empirical ranking's validity, but they are not circularity: no quantity is fitted and then renamed as a prediction, and no uniqueness claim is imported. Hyperparameters such as Q and C are described as swept and fixed, which may raise selection-on-test-set concerns, but that is overfitting, not self-reference.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on representativeness of the evaluation suite, reliability of the labels, and transfer of SAEs to the target model. The listed free parameters are the hyperparameters selected by sweeping on the evaluation sets, which is the main source of evaluation bias. No new physical or conceptual entities are introduced.

free parameters (4)
  • Number of top SAE latents Q = 1000
    Selected by jointly sweeping C and Q in Section 3.4 and reported as broadly optimal in Section 4.3. The main SAE comparisons use this value.
  • Logistic regression regularization C for SAE probes = 0.1
    Selected by sweeping in Section 3.4; used for probes on SAE latents and pre-activations.
  • Logistic regression regularization C for raw activation probes = 0.001
    Selected by sweeping in Section 3.4; used for probes on raw last-token activations.
  • Model layer depth for saved activations = 75% of model depth
    Selected by measuring AUROC versus layer in Section 4.3; all other experiments use this layer. The selection appears to use the same test sets reported in the results.
assumptions (4)
  • domain assumption Monitoring-relevant concepts are linearly decodable from LLM activations
    The entire probing approach assumes that information about harassment, violence, hallucinations, and sentiment is linearly represented in residual stream activations. The paper tests this indirectly through probe AUROC but does not prove it. See Sections 3.4 and 5.
  • domain assumption The four evaluation tasks and one model snapshot are representative of monitoring settings
    Section 3.1 fixes the testbed to one previous ChatGPT-4o snapshot and four binary classification tasks; Section 5 generalizes to monitoring tasks without fine-tuning. If the ranking is model- or task-specific, the central recommendation fails.
  • domain assumption The labels used for training and evaluation are trustworthy
    Moderation labels are expert-provided, sentiment labels come from RottenTomatoes, but SimpleQA correctness is auto-labeled by an LLM prompted with the answer key. No validation accuracy for the auto-labeler is reported, and label noise directly affects the hallucination AUROC. See Section 3.1.
  • domain assumption SAEs pre-trained on base GPT-4o and fine-tuned on chat data transfer to the ChatGPT-4o snapshot used here
    Section 3.3 describes pre-training on the base model and fine-tuning on chat-formatted conversations, following Kissane et al. The paper relies on this transfer for all SAE-based probe results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Investigating task-specific prompts and sparse autoencoders for activation monitoring." pith.science (2026). https://pith.science/paper/MT4WEPQU

@misc{pith2026250420271,
  author       = {Pith},
  title        = {Pith review of: Investigating task-specific prompts and sparse autoencoders for activation monitoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MT4WEPQU}},
  note         = {Machine review of arXiv:2504.20271}
}
read the original abstract

Language models can behave in unexpected and unsafe ways, and so it is valuable to monitor their outputs. Internal activations of language models encode additional information that could be useful for this. The baseline approach for activation monitoring is some variation of linear probing on a particular layer: starting from a labeled dataset, train a logistic regression classifier on that layer's activations. Recent work has proposed several approaches which may improve on naive linear probing, by leveraging additional computation. One class of techniques, which we call "prompted probing," leverages test time computation to improve monitoring by (1) prompting the model with a description of the monitoring task, and (2) applying a learned linear probe to resulting activations. Another class of techniques uses computation at train time: training sparse autoencoders offline to identify an interpretable basis for the activations, and e.g. max-pooling activations across tokens using that basis before applying a linear probe. However, one can also prompt the model with a description of the monitoring task and use its output directly. We develop and test novel refinements of these methods and compare them against each other. We find asking the model zero-shot is a reasonable baseline when inference-time compute is not limited; however, activation probing methods can substantially outperform this baseline given sufficient training data. Specifically, we recommend prompted probing when inference-time compute is available, due to its superior data efficiency and good generalization performance. Alternatively, if inference-time compute is limited, we find SAE-based probing methods outperform raw activation probing.

Figures

Figures reproduced from arXiv: 2504.20271 by the authors.

Figure 1
Figure 1. An illustration of our main monitoring methods on an example input. We extract residual [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Prompted probing (orange) and SAE max-pooled probing (blue) help with classification [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. When trained on English data and tested on English, prompted probes generalize relatively [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Last token probes (blue and green) perform best at sentiment classification. SAE pre [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Prompted probing does not substantially stack with SAE-based probing. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: A 2-level classifier which sums a prompted probe output logit with the model’s output [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Probes trained on mid-late layer activations perform best, with and without prompted [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Prompted probes are effective with or without task-specific prefixes. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Max-pooling is helpful only when using an SAE; max-pooling raw activations provides [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Using Q=1000 SAE latents in the classifier is roughly optimal across training dataset [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Adding increasing numbers of few-shot examples hurts classification performance. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Performance of the LAT Scan technique from Zou et al. (2023) saturates at a level lower [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: Classification performance is similar when using SAE latent activations (post-nonlinearity) [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail

    cs.LG 2025-12 conditional novelty 6.0 of 10

    Reliable concept presence in transformers is concentrated in the extreme high-activation tail of in-concept tokens; thresholding that tail improves concept detection and localization.

Reference graph

Works this paper leans on

24 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    Understanding intermediate layers using linear classifier probes, November 2018

    Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes, November 2018. URL http://arxiv.org/abs/1610.01644. arXiv:1610.01644 [stat]

    arXiv 2018

  2. [2]

    The Internal State of an LLM Knows When It 's Lying , October 2023

    Amos Azaria and Tom Mitchell. The Internal State of an LLM Knows When It 's Lying , October 2023. URL http://arxiv.org/abs/2304.13734. arXiv:2304.13734 [cs]

    arXiv 2023

  3. [3]

    Interpretability and Analysis in Neural NLP

    Yonatan Belinkov, Sebastian Gehrmann, and Ellie Pavlick. Interpretability and Analysis in Neural NLP . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics : Tutorial Abstracts , pp.\ 1--5, Online, 2020. Association for Computational Linguistics. doi:10.18653/v1/2020.acl-tutorials.1. URL https://www.aclweb.org/antholo...

    doi:10.18653/v1/2020.acl-tutorials.1 2020

  4. [4]

    Using Dictionary Learning Features as Classifiers , October 2024

    Trenton Bricken, Jonathan Marcus, Siddharth Mishra-Sharma, Meg Tong, Ethan Perez, Mrinank Sharma, Kelley Rivoire, and Thomas Henighan. Using Dictionary Learning Features as Classifiers , October 2024. URL https://transformer-circuits.pub/2024/features-as-classifiers/index.html

    work page 2024

  5. [5]

    Bitterman

    Jack Gallifant, Shan Chen, Kuleen Sasse, Hugo Aerts, Thomas Hartvigsen, and Danielle S. Bitterman. Sparse Autoencoder Features for Classifications and Transferability , February 2025. URL http://arxiv.org/abs/2502.11367. arXiv:2502.11367 [cs]

    arXiv 2025

  6. [6]

    Scaling and evaluating sparse autoencoders, June 2024

    Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, June 2024. URL http://arxiv.org/abs/2406.04093. arXiv:2406.04093 [cs]

    arXiv 2024

  7. [7]

    Detecting Strategic Deception Using Linear Probes , February 2025

    Nicholas Goldowsky-Dill, Bilal Chughtai, Stefan Heimersheim, and Marius Hobbhahn. Detecting Strategic Deception Using Linear Probes , February 2025. URL http://arxiv.org/abs/2502.03407. arXiv:2502.03407 [cs]

    arXiv 2025

  8. [8]

    Estimating Knowledge in Large Language Models Without Generating a Single Token , October 2024

    Daniela Gottesman and Mor Geva. Estimating Knowledge in Large Language Models Without Generating a Single Token , October 2024. URL http://arxiv.org/abs/2406.12673. arXiv:2406.12673 [cs]

    arXiv 2024

Show all 24 references
  1. [9]

    Finding Neurons in a Haystack : Case Studies with Sparse Probing , June 2023

    Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. Finding Neurons in a Haystack : Case Studies with Sparse Probing , June 2023. URL http://arxiv.org/abs/2305.01610. arXiv:2305.01610 [cs]

    2023 arXiv

  2. [10]

    Are Sparse Autoencoders Useful ? A Case Study in Sparse Probing , February 2025

    Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda. Are Sparse Autoencoders Useful ? A Case Study in Sparse Probing , February 2025. URL http://arxiv.org/abs/2502.16681. arXiv:2502.16681 [cs]

    2025 arXiv

  3. [11]

    Saes (usually) transfer between base and chat models

    Connor Kissane, Robert Krzyzanowski, Arthur Conmy, and Neel Nanda. Saes (usually) transfer between base and chat models. In AI Alignment Forum, 2024

    2024

  4. [12]

    Cognitive Dissonance : Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness ?, November 2023

    Kevin Liu, Stephen Casper, Dylan Hadfield-Menell, and Jacob Andreas. Cognitive Dissonance : Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness ?, November 2023. URL http://arxiv.org/abs/2312.03729. arXiv:2312.03729 [cs]

    2023 arXiv

  5. [13]

    LLMs Know More Than They Show : On the Intrinsic Representation of LLM Hallucinations , October 2024

    Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. LLMs Know More Than They Show : On the Intrinsic Representation of LLM Hallucinations , October 2024. URL http://arxiv.org/abs/2410.02707. arXiv:2410.02707 [cs]

    2024 arXiv

  6. [14]

    LatentQA : Teaching LLMs to Decode Activations Into Natural Language , December 2024

    Alexander Pan, Lijie Chen, and Jacob Steinhardt. LatentQA : Teaching LLMs to Decode Activations Into Natural Language , December 2024. URL http://arxiv.org/abs/2412.08686. arXiv:2412.08686 [cs]

    2024

  7. [15]

    Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales

    Bo Pang and Lillian Lee. Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05 , pp.\ 115--124, Ann Arbor, Michigan, 2005. Associ...

    2005

  8. [16]

    Jumping Ahead : Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders , August 2024

    Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping Ahead : Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders , August 2024. URL http://arxiv.org/abs/2407.14435. arXiv:2407.14435 [cs]

    2024 arXiv

  9. [17]

    Negative Results for Sparse Autoencoders On Downstream Tasks and Deprioritising SAE Research ( Mechanistic Interpretability Team Progress Update ), March 2025

    Lewis Smith, Sen Rajamanoharan, Arthur Conmy, Callum McDougall, Janos Kramar, Tom Lieberum, Rohin Shah, and Neel Nanda. Negative Results for Sparse Autoencoders On Downstream Tasks and Deprioritising SAE Research ( Mechanistic Interpretability Team Progress Update ), March 202...

    2025

  10. [18]

    Linear Representations of Sentiment in Large Language Models , October 2023

    Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda. Linear Representations of Sentiment in Large Language Models , October 2023. URL http://arxiv.org/abs/2310.15154. arXiv:2310.15154 [cs]

    2023 arXiv

  11. [19]

    Measuring short-form factuality in large language models, November 2024

    Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. Measuring short-form factuality in large language models, November 2024. URL http://arxiv.org/abs/2411.04368. arXiv:2411.04368 [cs]

    2024 arXiv

  12. [20]

    Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico...

    2023 arXiv

  13. [21]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  14. [22]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  15. [23]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  16. [24]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.