Pith. sign in

REVIEW 4 major objections 6 minor 41 references

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BehaviorBox automatically discovers fine-grained, human-interpretable text features where one language model outperforms another, and shows these features generalize to free generation.

desk verdict BehaviorBox is a promising hypothesis-generation tool for LM comparison, but its reported insights currently rest on single-run SAE features that the appendix admits are seed-sensitive. read the letter →

arxiv 2506.02204 v2 pith:RH2WQF6Q submitted 2025-06-02 cs.CL

Lindia Tjuatja , Graham Neubig This is my paper
classification cs.CL
keywords languagemodelevaluationsparseautoencodersslicefindingperformance-awareembeddingscomparisonperplexityinterpretabilityfeaturediscovery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BehaviorBox is an automated pipeline for comparing two language models by locating fine-grained, human-interpretable text features where one model assigns higher probability than the other. The paper argues that corpus-level perplexity hides these differences, and that features such as conditional 'were' after 'if you', tabs in narrative text, or archaic spellings can be discovered bottom-up without pre-specified domains. The method builds a performance-aware embedding for each word by concatenating a contextual embedding with the two models' word probabilities, trains a sparse autoencoder to decompose these representations, and labels the resulting groups with an LLM annotator. Applied to models differing in size, post-training, and family, BehaviorBox surfaces coherent behavioral contrasts, including differences between models with nearly identical perplexity. A generation experiment confirms that six of eight string hypotheses derived from discovered features appear at significantly different rates in free generation, supporting the claim that the features reflect real behavioral differences.

What carries the argument

The load-bearing mechanism is the performance-aware contextual embedding: a 770-dimensional vector per word made of a 768-dimensional Longformer word embedding concatenated with the word probabilities assigned by the two compared language models. The probabilities are up-weighted so that their components make up 70% of the input magnitude, ensuring the autoencoder actually uses the performance signal. A sparse autoencoder with a batch top-k operation, where only the top k activations per batch survive, learns a 3000-feature dictionary, and each dictionary element is treated as a candidate behavior feature. The paper selects, for each feature, the words with highest activations, filters features by median probability and log-probability difference thresholds, and uses a strong LLM annotator to label coherent groups, with manual correction by the authors.

What would settle it

Run BehaviorBox on the same model pair and corpus multiple times with different random seeds and data-order shuffles, then compare the resulting feature sets. If the coherent features and their probability-difference signs are not largely reproducible across runs, or if a second run produces contradictory labels such as one model outperforming on tabs in one run and the other in another, the central claim that the discovered features capture stable model differences would be refuted.

Watch

Extended reading notes

Core claim

The central discovery is that fine-grained, interpretable differences in language-model behavior can be extracted automatically from a corpus by treating each word as a point in a joint space of context and performance. For every word, BehaviorBox averages the contextual token embeddings from Longformer, multiplies the token probabilities assigned by the two compared models, and concatenates the embedding with the two probabilities into a 770-dimensional vector. A sparse autoencoder with a batch top-k sparsity constraint decomposes these vectors into additive features; the top-activating words for each feature form a slice, and features are kept only if the median probability or log-probability difference between models exceeds fixed thresholds. The result is a list of coherent, labelable slices, such as 'tabs following another tab after sentence-ending punctuation' or 'conditional were in the phrase if you were', that identify where one model is easier or harder to predict than the other. The paper reports that these features generalize beyond the analysis corpus, with six of eight string hypotheses significantly distinguishing free generations of Llama-13B and OLMo-13B.

Load-bearing premise

The load-bearing assumption is that the specific feature sets found by one sparse-autoencoder run represent the true behavioral differences between the two models; the paper itself notes in Appendix A.1 that different random seeds and data orderings can largely change the space of features discovered, so a single run's feature list may be a sample rather than a stable account.

Editorial extensions

If this is right

  • If BehaviorBox is correct, practitioners can replace hand-specified benchmark slices with automatically discovered behavioral contrasts for any pair of language models.
  • Features found in corpus evaluation can be turned into testable hypotheses about open generation, as demonstrated by the significant string-frequency differences for six of eight hypotheses.
  • Perplexity deltas do not predict the number or coherence of salient behavioral differences; near-identical perplexity can coexist with many well-separated features, as in the family comparisons.
  • Post-trained chat models can outperform their base counterparts on conversational phrases and multiword constructions even while having substantially worse perplexity, so aggregate metrics alone mislead about where each model is stronger.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to run multiple sparse-autoencoder initializations and take the union of discovered features, since the paper notes in Appendix A.1 that different random seeds and data orderings can substantially change the feature space; this would test whether the reported contrasts are stable.
  • The method's word-level unit and probability-based thresholds are not specific to English or to the two model families tested, so it could plausibly be applied to other languages, code-heavy corpora, or multimodal sequence models.
  • The generation experiment tests only eight hand-selected strings; a stronger test would automatically map discovered features to generation statistics across additional model pairs to see whether the generalization claim holds broadly.
  • The meta-categories in the qualitative analysis were produced by k-means clustering, LLM labeling, and manual inspection; an automated replication without manual corrections would reveal how much of the reported insight is attributable to the pipeline itself.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes BehaviorBox, an automated pipeline for discovering fine-grained, human-interpretable features that distinguish the next-token prediction performance of two language models. The method builds a per-word representation by concatenating Longformer contextual embeddings with the probabilities assigned by the two evaluated LMs, trains a sparse autoencoder (BatchTopK SAE) on the resulting 770-dimensional vectors, filters features by coherence and by median probability/log-probability difference, and labels surviving features with an LLM annotator. The authors apply the pipeline to comparisons across model size (7B vs. 13B), post-training (base vs. chat/DPO), and model family (Llama-2 vs. OLMo-2), and report qualitative feature categories such as formatting, dialogue, and vernacular text. They also test eight string-frequency hypotheses in free generation for Llama-13B vs. OLMo-13B, finding six significant results. The paper includes full appendices with hyperparameters, annotation prompts, and all feature tables, and releases code.

Significance. If the central claims hold, BehaviorBox would be a useful complement to corpus-level perplexity and benchmark-based evaluation, offering automated hypothesis generation about where one LM systematically differs from another. The manuscript's strengths include a clearly specified pipeline, a thoughtful treatment of tokenization aggregation, a partially automated labeling procedure with validation, and a free-generation experiment that makes falsifiable predictions. The paper is also transparent about its limitations, particularly in Appendix A.1 regarding SAE sensitivity to random seeds and data ordering. However, the empirical support for the core claim that discovered features are stable and generalizable is currently incomplete, and the selective reporting of features and hypotheses makes several conclusions partly tautological or under-supported. The work is timely and of interest to the evaluation and interpretability communities, but the main claims require additional reproducibility evidence.

major comments (4)
  1. [Section 7 and Appendix A.1] Section 6.1 filtering
  2. [Section 6.1] Section 8
  3. [Section 8] Appendix A.1, Table 4
  4. [Appendix A.1, Table 4] The choice of the probability up-weighting fraction (0.7) is based on a single small-scale sweep over a fraction of the data, and the paper does not assess how sensitive the downstream features and conclusions are to this hyperparameter or to other SAE hyperparameters (dictionary size, k, batch order). Given that Appendix A.1 already notes the variability across runs, a sensitivity analysis that varies the probability weight and reports the overlap or stability of the resulting feature sets would materially strengthen the claim that the method is robust and not tuned to produce particular outcomes.
minor comments (6)
  1. [Abstract] The abstract contains a typo: 'datatset' should be 'dataset'.
  2. [Table 2] The table caption uses 'Olmo-13B' with inconsistent capitalization; elsewhere the paper uses 'OLMo-13B'.
  3. [Section 8] The sentence 'Additionally, while did not have a conclusive result for "morrow"' is grammatically incomplete; it should read 'while we did not have a conclusive result'.
  4. [Tables 9 and 10] Several feature labels contain typos: 'puncutation' (Table 9), 'senteses' (Table 9), and 'preceeding' (Table 10) should be corrected. One label in Table 9 also ends with a stray '<SEP>' token, likely a formatting artifact.
  5. [Figures 2-4] The dendrograms are dense and the feature labels are small; a high-resolution version or an interactive supplement would improve readability, especially for the many tab-related labels.
  6. [Section 5.3] The description of probability up-weighting states that the probability components make up 70% of the total magnitude, but it is not clear whether this rescaling is applied per input vector or globally; clarifying this would aid reproducibility.

Circularity Check

1 steps flagged · score 4.0 of 10

BehaviorBox's filtering step makes 'discovered performance differences' true by construction, but semantic coherence and the out-of-sample generation test preserve independent content.

  1. self definitional [Section 6.1 (Processing and Filtering SAE Features)]
    "However, not every feature is indicative of a significant and consistent performance difference between models. To exclude those that are not, we only consider features that either show a median probability difference among the filtered samples greater than 0.1 or a median log-probability difference greater than 1 (i.e. the ratio of probabilities is greater than e)."

    This filter operationally defines what counts as a 'significant performance difference' feature. Every feature reported in Section 7 and Appendix A.4 is, by construction, one whose median probability/log-probability gap exceeds the chosen cutoff, because all other features are discarded. The same two probabilities are also concatenated into the SAE input in Eq. 3, so the extracted dictionary directions can directly encode the probability gap rather than discovering it from embeddings alone. Thus the statement that BehaviorBox 'finds features where one LM outperforms another,' and the directional insights such as 13B models being better at particular contexts, restate the inclusion criterion rather than serving as an independent empirical result.

full rationale

The main circular element is the feature-filtering step: BehaviorBox guarantees that every reported feature has a large median probability difference, so the existence and sign of the performance gap is an artifact of the selection rule. This is a genuine self-definitional step, but the paper's central claim also contains non-circular components. The coherent natural-language labels for the discovered groups are not implied by the probability filter, and the Section 8 free-generation test provides out-of-sample evidence that is not statistically forced: six of eight string hypotheses were significant on newly generated text, and even the 'morrow' result, while based on an uncalibrated 20-vs-0 count, comes from data not used to select the feature. The self-citations in the paper (e.g., Liu et al. 2025 for generation settings, Arora et al. 2022 for interpretability evaluation) are minor and not load-bearing. The single-run SAE instability noted in Appendix A.1 is a robustness threat to the qualitative conclusions, but it is not a circularity argument; it concerns reproducibility rather than equivalence of inputs and outputs. Overall, the 'where one model is better' part is partially tautological, while the coherence and generalization claims retain independent empirical content, supporting a score of 4 rather than 0 or 8.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities. Its free parameters are hyperparameters and filtering thresholds, several of which are fit to the data and directly shape which features are reported. The main burden is on the representational assumptions: word-level aggregation via Longformer plus probability product, and the stability of single-run SAE features.

free parameters (5)
  • Probability feature up-weight fraction = 0.7
    Chosen by a sweep over 0.5-0.9 on an 8M-word subset (Table 4) to balance feature coherence and probability distinctiveness. It directly controls how much the SAE focuses on probability differences.
  • SAE dictionary size and sparsity k = 3000, k=50
    Picked via heuristics (dead latents, number of coherent features) and not systematically varied; different choices change the feature space.
  • Feature filtering thresholds = median prob diff > 0.1 or median log prob diff > 1
    Set post hoc from consistency scatterplots (Figure 5) to ensure features show large probability differences; this selection shapes all reported findings.
  • Activation cutoff for included words = top 75% or >25% of max activation
    Described as "somewhat arbitrary" in Section 6.1; determines which words represent each feature.
  • Top-k words per feature and minimum activations = 50 words, 10 non-zero activations
    Hand-set thresholds that control how many examples define a feature and which features are analyzed.
assumptions (5)
  • domain assumption The 1000-document sample from six Dolma sources (about 80M words) is representative enough to reveal meaningful and generalizable LM differences.
    Used in Section 7; no evidence that the specific sample covers the space of relevant behaviors.
  • domain assumption Longformer's last-layer contextual embeddings capture the semantic and syntactic properties relevant to next-token probabilities for all token types, including whitespace and punctuation.
    The word representations are the backbone of the performance-aware embeddings (Section 4); if embeddings are uninformative for formatting tokens, features like 'tabs' would be found only through probabilities.
  • domain assumption Word-level aggregation (mean of token embeddings, product of token probabilities) preserves enough signal to compare LMs.
    Used in Section 4, equations (1) and (2); the paper acknowledges tokenizer misalignment as a limitation.
  • domain assumption The sparse autoencoder's learned features are stable enough across training runs to support the qualitative conclusions.
    The main results (Section 7) come from single SAE runs, while Appendix A.1 reports variation across seeds. This is the paper's own caution.
  • domain assumption Claude 3.5 annotations after manual verification provide labels that are sufficiently accurate for the reported insights.
    Used in Section 6.2; no inter-annotator agreement or systematic evaluation of label accuracy is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models." pith.science (2026). https://pith.science/paper/RH2WQF6Q

@misc{pith2026250602204,
  author       = {Pith},
  title        = {Pith review of: BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RH2WQF6Q}},
  note         = {Machine review of arXiv:2506.02204}
}
read the original abstract

Language model evaluation is a daunting task: prompts are brittle, corpus-level perplexities are vague, and the choice of benchmarks are endless. Finding examples that show meaningful, generalizable differences between two LMs is crucial to understanding where one model succeeds and another fails. Can this process be done automatically? In this work, we propose methodology for automated comparison of language models that uses performance-aware contextual embeddings to find fine-grained features of text where one LM outperforms another. Our method, which we name BehaviorBox, extracts coherent features that demonstrate differences with respect to the ease of generation between two LMs. Specifically, BehaviorBox finds features that describe groups of words in fine-grained contexts, such as "conditional 'were' in the phrase 'if you were'" and "exclamation marks after emotional statements", where one model outperforms another within a particular datatset. We apply BehaviorBox to compare models that vary in size, model family, and post-training, and enumerate insights into specific contexts that illustrate meaningful differences in performance which cannot be found by measures such as corpus-level perplexity alone.

Figures

Figures reproduced from arXiv: 2506.02204 by the authors.

Figure 1
Figure 1. BEHAVIORBOX is a three-part automatic behavior comparison pipeline that discovers fine-grained features where one LM differs from another. These features are extracted from a corpus of performance-aware embeddings, which take into account the semantics and usage of the text, along with measures of performance via probability under the evaluated LMs. LMs’ performance on that sample (via the prob￾abilities the models … view at source ↗
Figure 2
Figure 2. Representative features between 7B models (blue) and their 13B counterparts (orange). [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Representative features between base models (blue) and their post-trained counterparts (orange). [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Representative features between Llama (blue) and OLMo models (orange) of the same sizes. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Scatterplots showing median probability (top) [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 7 canonical work pages

  1. [1]

    Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977

    arXiv 2020

  2. [2]

    Abien Fred Agarap. 2018. https://api.semanticscholar.org/CorpusID:4090379 Deep learning using rectified linear units (relu) . ArXiv, abs/1803.08375

    arXiv 2018

  3. [3]

    Anthropic. 2024. https://api.semanticscholar.org/CorpusID:268232499 The claude 3 model family: Opus, sonnet, haiku

    2024

  4. [4]

    Siddhant Arora, Danish Pruthi, Norman Sadeh, William W Cohen, Zachary C Lipton, and Graham Neubig. 2022. Explain, edit, and understand: Rethinking user study design for evaluating model explanations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 5277--5285

    work page 2022

  5. [5]

    Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150

    arXiv 2020

  6. [6]

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. 2023. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2

    2023

  7. [7]

    Bart Bussmann, Patrick Leask, and Neel Nanda. 2024. https://openreview.net/forum?id=d4dpOCqybL Batchtopk sparse autoencoders . In NeurIPS 2024 Workshop on Scientific Methods for Understanding Deep Learning

    2024

  8. [8]

    Joel Chan, Steven Dang, and Steven P Dow. 2016. Comparing different sensemaking approaches for large-scale ideation. In Proceedings of the 2016 CHI conference on human factors in computing systems, pages 2717--2728

    work page 2016

Show all 41 references
  1. [9]

    Yeounoh Chung, Tim Kraska, Neoklis Polyzotis, and Steven Euijong Whang. 2018. https://api.semanticscholar.org/CorpusID:49870204 Slice finder: Automated data slicing for model validation . 2019 IEEE 35th International Conference on Data Engineering (ICDE), pages 1550--1553

    2018

  2. [10]

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. https://api.semanticscholar.org/CorpusID:261934663 Sparse autoencoders find highly interpretable features in language models . ArXiv, abs/2309.08600

    2023 arXiv

  3. [11]

    Wright, and Kevin Leyton-Brown

    Greg d'Eon, Jason d'Eon, James R. Wright, and Kevin Leyton-Brown. 2022. https://doi.org/10.1145/3531146.3533240 The spotlight: A general method for discovering systematic errors in deep learning models . In Proceedings of the 2022 ACM Conference on Fairness, Accountability, an...

    2022

  4. [12]

    Sabri Eyuboglu, Maya Varma, Khaled Kamal Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Re. 2022. Domino: Discovering systematic errors with cross-modal embeddings. In International Conference on Learning Representations

    2022

  5. [13]

    Lizhe Fang, Yifei Wang, Zhaoyang Liu, Chenheng Zhang, Stefanie Jegelka, Jinyang Gao, Bolin Ding, and Yisen Wang. 2024. What is wrong with perplexity for long-context language modeling? arXiv preprint arXiv:2410.23771

    2024 arXiv

  6. [14]

    Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, and Noah A. Smith. 2015. https://doi.org/10.3115/v1/P15-1144 Sparse overcomplete word vector representations . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Inte...

    2015 doi

  7. [15]

    Thomas Fel, Ekdeep Singh Lubana, Jacob S Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. 2025. Archetypal sae: Adaptive and stable dictionary learning for concept extraction in large vision models. arXiv pr...

    2025 arXiv

  8. [16]

    Leo Gao, Tom Dupr'e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024. https://api.semanticscholar.org/CorpusID:270286001 Scaling and evaluating sparse autoencoders . ArXiv, abs/2406.04093

    2024 arXiv

  9. [17]

    Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, and Sanmi Koyejo. 2024. Scaling laws for downstream task performance of large language models. arXiv preprint arXiv:2402.04177

    2024

  10. [18]

    Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. 1977. Perplexity—a measure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62(S1):S63--S63

    1977

  11. [19]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

    2020 arXiv

  12. [20]

    Margaret Li, Weijia Shi, Artidoro Pagnoni, Peter West, and Ari Holtzman. 2024. https://api.semanticscholar.org/CorpusID:270878711 Predicting vs. acting: A trade-off between world modeling & agent modeling . ArXiv, abs/2407.02446

    2024 arXiv

  13. [21]

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, J'anos Kram'ar, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. https://api.semanticscholar.org/CorpusID:271843380 Gemma scope: Open sparse autoencoders everywhere all at once on...

    2024 arXiv

  14. [22]

    Zachary C Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31--57

    2018

  15. [23]

    Emmy Liu, Amanda Bertsch, Lintang Sutawika, Lindia Tjuatja, Patrick Fernandes, Lara Marinov, Michael Chen, Shreya Singhal, Carolin Lawrence, Aditi Raghunathan, Kiril Gashteovski, and Graham Neubig. 2025. https://arxiv.org/abs/2503.03862 Not-just-scaling laws: Towards a better ...

    2025

  16. [24]

    Ilya Loshchilov and Frank Hutter. 2017. https://api.semanticscholar.org/CorpusID:53592270 Decoupled weight decay regularization . In International Conference on Learning Representations

    2017

  17. [25]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 4768–4777, Red Hook, NY, USA. Curran Associates Inc

    2017

  18. [26]

    James MacQueen. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics, volume 5, pages 281--298. University of California press

    1967

  19. [27]

    Jha, Oyvind Tafjord, Dustin Schwenk, Pete Walsh, Yanai Elazar, Kyle Lo, Dirk Groeneveld, Iz Beltagy, Hanna Hajishirzi, Noah A

    Ian Magnusson, Akshita Bhagia, Valentin Hofmann, Luca Soldaini, A. Jha, Oyvind Tafjord, Dustin Schwenk, Pete Walsh, Yanai Elazar, Kyle Lo, Dirk Groeneveld, Iz Beltagy, Hanna Hajishirzi, Noah A. Smith, Kyle Richardson, and Jesse Dodge. 2023. https://api.semanticscholar.org/Corp...

    2023 arXiv

  20. [28]

    Alireza Makhzani and Brendan J. Frey. 2013. https://api.semanticscholar.org/CorpusID:14850799 k-sparse autoencoders . CoRR, abs/1312.5663

    2013 arXiv

  21. [29]

    Henry B Mann and Donald R Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, pages 50--60

    1947

  22. [30]

    Rajiv Movva, Kenny Peng, Nikhil Garg, Jon Kleinberg, and Emma Pierson. 2025. Sparse autoencoders for hypothesis generation. arXiv preprint arXiv:2502.04382

    2025 arXiv

  23. [31]

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, et al. 2024. 2 olmo 2 furious. arXiv preprint arXiv:2501.00656

    2024 arXiv

  24. [32]

    why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. https://doi.org/10.1145/2939672.2939778 "why should i trust you?": Explaining the predictions of any classifier . In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining...

    2016

  25. [33]

    Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. https://doi.org/10.18653/v1/2020.acl-main.442 Beyond accuracy: Behavioral testing of NLP models with C heck L ist . In Proceedings of the 58th Annual Meeting of the Association for Computational Lingu...

    2020 doi

  26. [34]

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations

    2023

  27. [35]

    Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher R \'e . 2020. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. Advances in Neural Information Processing Systems, 33:19339--19352

    2020

  28. [36]

    Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A

    Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Mu...

    2024 arXiv

  29. [37]

    Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M

    Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cant \'o n Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes,...

    2023 arXiv

  30. [38]

    Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. 2019. https://doi.org/10.18653/v1/P19-1073 E rrudite: Scalable, reproducible, and testable error analysis . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 747-...

    2019 doi

  31. [39]

    Mengzhou Xia, Mikel Artetxe, Chunting Zhou, Xi Victoria Lin, Ramakanth Pasunuru, Danqi Chen, Luke Zettlemoyer, and Veselin Stoyanov. 2023. https://doi.org/10.18653/v1/2023.acl-long.767 Training trajectories of language models across scales . In Proceedings of the 61st Annual M...

    2023 doi

  32. [40]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  33. [41]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.