Pith. sign in

REVIEW 3 major objections 8 minor 35 references

Training-free Truthfulness Detection via Sparse MLP Value Vectors

T0 review · 3 major / 8 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free method that votes across sparse MLP value vectors detects truthfulness in LLMs better than attention-head and likelihood baselines.

desk verdict TruthV is a sensible incremental extension of NoVo to MLP value vectors with real held-out gains, but the 30-example selection step is fragile and the paper peeks at the test set to set p. read the letter →

arxiv 2509.17932 v2 pith:EGTA4JNR submitted 2025-09-22 cs.CL

Runheng Liu , Heyan Huang , Xingchen Xiao , Yanghao Zhou , Zhijing Wu This is my paper · ORCID
classification cs.CL
keywords truthfulnessdetectionMLPvaluevectorstraining-freehallucinationmultiple-choiceQAinternalrepresentationsmajorityvotingsparseselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that truthfulness-related information in large language models is concentrated in a sparse set of MLP value vectors—columns of the feed-forward down-projection matrix—whose activations rise or fall when the model sees a correct answer. It proposes TruthV, a training-free detector that ranks every value vector by how often its highest or lowest activation lands on the correct answer in just 30 labeled examples, keeps the top 0.1%, and aggregates their votes by majority. On the NoVo benchmark across four LLMs, TruthV outperforms the attention-head-based NoVo method and log-likelihood scoring, often by a wide margin. The paper presents this as evidence that MLP modules, previously neglected in training-free approaches, carry structured and exploitable truthfulness signals.

What carries the argument

The key machinery is the interpretation of each MLP value vector as an individual truthfulness predictor. Under the key-value memory view of feed-forward layers, a scalar key activation is computed for each candidate answer; a vector is called argmax-aligned if the correct answer yields the highest activation, and argmin-aligned if it yields the lowest. The method ranks every value vector by its argmax (or argmin) accuracy on a 30-example support set, keeps the top 0.1%, and aggregates their per-question votes by majority. This turns a sparse set of internal memory columns into a training-free ensemble classifier.

What would settle it

Run TruthV with the 30 support examples replaced by 30 randomly chosen incorrect answers labeled as 'correct.' If TruthV still scores above random on the real test set, the value-vector ranking is not causal; if accuracy collapses to chance, the selection step is load-bearing. A complementary check: measure the overlap between the top-0.1% vectors selected from the 30 samples and the top-0.1% selected from the full training set; near-zero overlap would indicate the 30-sample ranking is noise.

Watch

Extended reading notes

Core claim

The central claim is that a small fraction of MLP value vectors encode truthfulness in a directionally consistent way: for a given question, the correct answer tends to make the corresponding key activation either the maximum or the minimum among candidates. The paper demonstrates this by ranking all value vectors by argmax/argmin accuracy and showing a thin tail of high-accuracy vectors that transfer to held-out data, then builds TruthV on top of that ranking. TruthV selects the top 0.1% of vectors using 30 labeled samples and lets them vote; across four instruction-tuned LLMs and ten datasets it exceeds NoVo and log-likelihood baselines (e.g., 70.33% vs 61.67% average on Gemma-2-2B-it).

Load-bearing premise

The method assumes that the 30 labeled examples are enough to identify value vectors whose per-question preferences keep holding on unseen questions; if the ranking overfits those 30 examples, the voting accuracy collapses.

Editorial extensions

If this is right

  • Truthfulness detection can be done from MLP internal signals without any trained classifier, using only 30 labeled examples and no extra parameters.
  • The MLP module, not just attention heads, contains statistically reliable truthfulness signals, so future training-free detectors should not restrict themselves to attention.
  • The sparse top-0.1% subset suffices for strong accuracy, implying that the truthfulness signal is concentrated in a minority of value vectors rather than distributed uniformly.
  • Using the full training set usually improves over the 30-sample selection, indicating that the ranking is imperfect but already near-saturated; more data mainly refines the list.
  • Value vectors selected on one dataset transfer to other datasets above random chance, so they carry a mixture of task-general and task-specific truthfulness information.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same value-vector voting recipe could be applied to other binary content properties (e.g., toxicity, sentiment, style) whenever a small labeled support set for that property is available, since the mechanism does not rely on truthfulness-specific training.
  • The paper shows that the activation baseline is set by the question prefix and the answer only shifts it; per-question or per-prefix normalization would likely sharpen the signal, a natural next step not run here.
  • Because the method uses the final-token key activation on candidate answers, it is inherently a multiple-choice scorer; applying it to open-ended generation would require enumerating candidate continuations or a different aggregation, a nontrivial gap.
  • The small gain from combining argmax and argmin votes, together with the observed symmetry between the two patterns, suggests using their disagreement as an uncertainty signal for abstention or confidence calibration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper proposes TruthV, a training-free method for detecting the truthfulness of LLM outputs by exploiting individual MLP value vectors. Under the key-value memory interpretation of MLPs, TruthV ranks all value vectors by their argmax/argmin accuracy on 30 labeled multiple-choice examples, selects the top p=0.1% of vectors, and aggregates their per-question predictions by majority voting. The method is evaluated on the NoVo benchmark across four instruction-tuned LLMs (2B-13B) and ten datasets, reporting consistent improvements over NoVo and log-likelihood baselines (e.g., 70.33% vs. 61.67% average accuracy on Gemma-2-2B-it). The paper also presents analyses of the layer-wise distribution, interpretability, and cross-dataset generalization of the selected value vectors, concluding that truthfulness-related variation is captured in a sparse and structured manner in MLP value vectors.

Significance. If the reported results hold, this is a useful contribution: it extends the training-free truthfulness-detection paradigm beyond attention heads to MLP modules, offers a simple and interpretable mechanism (individual value-vector voting), and provides evidence that MLP value vectors encode content-level properties beyond token-level concepts. The method is conceptually clean and does not require classifier training. The main evaluation is broad (4 models, 10 datasets) and the held-out test accuracy is consistently higher than strong baselines. However, the significance is contingent on two load-bearing methodological points: the honest selection of the hyperparameter p, and the stability of vector selection given only 30 labeled examples. Currently both points are insufficiently supported, which tempers confidence in the headline numbers.

major comments (3)
  1. [§5.3, Figure 5] The hyperparameter p=0.001 is chosen by peaking accuracy curves that appear to be computed on the same evaluation sets used for Table 1. The text states 'we compute the average accuracy across all four models and find that the performance peaks around p=0.001' without specifying that a separate validation set was used. If the curves in Figure 5 are test-set accuracies, then the hyperparameter is tuned on the test data, and the reported improvements are optimistic. This is load-bearing because all headline results in Table 1 use p=0.001. The authors should either use a validation split for selecting p, or report results across a range of p without claiming the peak as the method's performance, or justify that the choice is stable and not overfit.
  2. [§5.1, §5.4, Table 2] The 30-example support set is used to rank up to ~352k value vectors (d'×L), yet the paper provides no stability analysis. Table 2 shows that using the full training set can improve results dramatically: for Llama-3.2-3B-Instruct, QASC accuracy jumps from 50.11 to 71.60 and CICv1 from 37.20 to 53.23. These gaps indicate that the 30-example ranking is far from converged. Since the full-training-set column in Table 2 uses test-set p tuning, the comparison is not apples-to-apples, but the magnitude of the gap still suggests that the fixed 30 examples may be an unrepresentative draw. Without variance estimates, multiple random support sets, or a comparison to random vector selection, the reported gains (e.g., 70.33 vs. 61.67 on Gemma-2-2B-it) could be artifacts of the specific support set. This is a core limitation of the method's claimed low-resource capability.
  3. [§4.1, §4.2, Figure 2] The 'existence of truthfulness-related value vectors' analysis is performed on the full training set, as stated in Section 4.1 ('We use the full training set of each dataset except for TQA during this experiment'). Ranking value vectors by training-set accuracy and observing a high-accuracy subset is not evidence of generalization and may reflect overfitting, especially with up to hundreds of thousands of candidate vectors. To support the claim that these vectors genuinely encode truthfulness, the analysis should be repeated on held-out data, or at least the paper should acknowledge that the existence claim is only about training-set correlation. Table 3 partially addresses generalization, but only for the argmax pattern and under full-training selection; the main existence claim and Figure 2 should be re-evaluated or reframed.
minor comments (8)
  1. [Figure 3 caption vs. §4.2] The caption of Figure 3 says 'top 0.01%' but the body text in Section 4.2 says 'top 0.1%'. This inconsistency should be corrected.
  2. [§4.1] Typo: 'the the ARC-Easy training set' should be 'the ARC-Easy training set'.
  3. [General] The baseline NoVo's configuration (e.g., its p value or number of selected heads) is not described in Section 5.1. To ensure reproducibility and fair comparison, the authors should state how the NoVo baseline is instantiated (e.g., which hyperparameters are used and whether they follow the original paper).
  4. [§4.2, Figure 2] The text mentions 'the red dash line' to indicate random guess accuracy, but the figure caption does not explain this. Please add a caption note or ensure the line is visible/described.
  5. [§4.2] Minor typo: 'opposite tend' should be 'opposite trend'.
  6. [§4.2] The paper mentions t-SNE visualizations ('t-SNE visualizations of these selected value vectors reveal no clear clustering'), but no such figure is included. Either add the figure or remove the claim.
  7. [Table 2] Table 2 omits TQA because it has no training set; this should be explicitly noted in the caption or text for clarity.
  8. [§5.3] The 'combined' activation pattern is shown only in Figure 5, not in Table 1. If combining argmax and argmin is claimed to slightly improve accuracy, a quantitative comparison in the main table would strengthen the claim.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline TruthV-vs-NoVo result is a held-out comparison and not circular, but the paper's central representational 'discovery' is a self-selection tautology, and the ensemble-size hyperparameter p is chosen on the same benchmark accuracy curves used for the reported results.

  1. self definitional [Section 4.1 'Scoring and ranking value vectors' / Section 4.2 'Existence of truthfulness-related value vectors']
    "We rank all value vectors by this score and select the top p as the most strongly aligned with the argmax activation pattern. ... We compute accuracy rankings of value vectors and consistently identify a small subset of them that are strongly associated with truthfulness. ... While most value vectors perform near the random baseline, a small subset achieves significantly higher accuracy under the argmax pattern—suggesting a reliable connection to truthfulness."

    The 'discovery' that top-ranked value vectors have high accuracy is a tautology: Eq. (3) computes accuracy on the training set D, the paper ranks by that same score, and then Figure 2 plots that same training accuracy against rank. The high accuracy of the selected top vectors is therefore entailed by the selection operation, not independently evidenced. No held-out evaluation or null-model comparison is used in this section, so the claim that a sparse subset 'exhibits stable and directionally consistent correlations with content truthfulness' is, at this point in the derivation, the selection criterion restated as a finding. It motivates TruthV but does not by itself establish generalization.

  2. fitted input called prediction [Section 5.3 'Effect of the number of selected value vectors']
    "As shown in Figure 5, performance improves rapidly as p increases, reaching a peak at a relatively small p... To identify a unified setting, we compute the average accuracy across all four models and find that the performance peaks around p= 0.001. Accordingly, we adopt p= 0.001 for all models in our experiments."

    The hyperparameter p, which controls the number of value vectors in the ensemble, is selected by peaking the benchmark accuracy curves in Figure 5 (caption: 'the accuracy is averaged over all datasets'), and then the reported Table 1 results are produced with that same p. In addition, the full-training-set analysis in Table 2 states 'we search p from 0.01% to 1% and report the best result for each dataset.' Thus the reported accuracy is not a fully held-out prediction for the ensemble-size hyperparameter: it is the outcome of a selection procedure that used the evaluation labels. This is a fitted input called a prediction, though the value-vector selection itself remains based on the 30 support examples, so the main held-out comparison is only partially compromised.

full rationale

The central TruthV result is not itself circular: value vectors are selected using 30 labeled support examples and evaluated on separate NoVo test questions, so the headline improvement over NoVo and log-likelihood is a genuine held-out comparison with independent content. No self-citation chain or imported uniqueness theorem is load-bearing. However, the paper's abstract-level claim that 'a sparse subset exhibits stable and directionally consistent correlations with content truthfulness' rests on Section 4.2, where vectors are ranked by training accuracy and then the same training accuracy is reported as evidence of the phenomenon. That is a self-definitional reduction. Additionally, the ensemble-size p is chosen by inspecting the benchmark accuracy curves on which the final numbers are reported, and Table 2 explicitly searches p for best per-dataset results. These are real but partial circularities: they affect the representational-discovery claim and the statistical validity of the headline comparison's hyperparameter, but they do not make the main method's predictions equivalent to its inputs. The score is therefore 6 rather than 8 or 10, because a held-out predictor with substantial margins remains the core contribution.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new entities (particles, forces, dimensions) are introduced. The 'truthfulness-related value vectors' are a selected subset of existing model parameters, not newly postulated entities.

free parameters (1)
  • p (fraction of value vectors selected for voting) = 0.001 (chosen by peak of average test accuracy in Figure 5)
    The ensemble size p is tuned on the evaluation curves; Section 5.3 says 'the performance peaks around p=0.001... we adopt p=0.001 for all models'.
assumptions (3)
  • domain assumption MLP modules act as key-value memories where each value vector v_i and key activation k_i are meaningful units (Geva et al. 2021)
    Used throughout Section 3-4 to define the activation patterns that TruthV relies on.
  • domain assumption The final token's key activation for each candidate answer is the correct and sufficient signal for truthfulness
    Section 4.1: 'we focus on the final token of each input sequence', following prior probing work.
  • ad hoc to paper A 30-example support set per dataset is representative enough that vectors ranked on it transfer to the held-out test set
    Section 5.1 adopts the 30 provided examples for selection; this is load-bearing for the method and is only partially supported by Table 2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Training-free Truthfulness Detection via Sparse MLP Value Vectors." pith.science (2026). https://pith.science/paper/EGTA4JNR

@misc{pith2026250917932,
  author       = {Pith},
  title        = {Pith review of: Training-free Truthfulness Detection via Sparse MLP Value Vectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EGTA4JNR}},
  note         = {Machine review of arXiv:2509.17932}
}
read the original abstract

Large language models (LLMs) are prone to generating factually incorrect content, motivating methods for assessing truthfulness from internal model signals. While supervised probing approaches can be effective, they require labeled data and classifier training. Recent training-free methods avoid parameter optimization but rely on coarse activation statistics that provide limited insight into how truthfulness-related signals arise within the model. We present a training-free approach that operates at the level of individual multi-layer perceptron (MLP) value vectors. Through a systematic analysis, we find that although most value vectors show no meaningful signal, a sparse subset exhibits stable and directionally consistent correlations with content truthfulness. Leveraging this observation, we propose \textbf{TruthV}, a simple inference method that aggregates preferences expressed by these value vectors. TruthV requires only a small support set to identify relevant vectors and introduces no additional model parameters or classifier weights. We evaluate TruthV across model scales from 2B to 13B and multiple benchmarks, including question answering, natural language understanding, and hallucination evaluation. TruthV consistently outperforms existing training-free baselines, demonstrating that truthfulness-related variation in LLMs is captured in a sparse and structured manner at the level of MLP value vectors.

Figures

Figures reproduced from arXiv: 2509.17932 by the authors.

Figure 1
Figure 1. (a) illustrates the data flow through the MLP [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Accuracy of value vectors under both argmax and argmin activation patterns, ranked by argmax accuracy. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Layer-wise distribution of the top 0.01% most accurate value vectors across different LLMs. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Key activation distributions of the most truthfulness-predictive value vector on the PIQA dataset. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Performance of TruthV across varying p, where the accuracy is averaged over all datasets. ments. Specifically, we aggregate the predictions from both activation patterns and perform majority voting over the combined sets. As shown in [PITH_FULL_IMAGE:figures/full_fig_…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 2 canonical work pages

  1. [1]

    Guillaume Alain and Yoshua Bengio. 2017. https://openreview.net/forum?id=ryF7rTqgl Understanding intermediate layers using linear classifier probes

    2017

  2. [2]

    Amos Azaria and Tom Mitchell. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.68 The internal state of an LLM knows when it`s lying . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967--976, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-emnlp.68 2023

  3. [3]

    Yonatan Bisk, Rowan Zellers, Ronan Le bras, Jianfeng Gao, and Yejin Choi. 2020. https://doi.org/10.1609/aaai.v34i05.6239 Piqa: Reasoning about physical commonsense in natural language . Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7432--7439

    doi:10.1609/aaai.v34i05.6239 2020

  4. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. https://arxiv.org/abs/2005.14165 Lan...

    arXiv 2020

  5. [5]

    Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2023. https://openreview.net/forum?id=ETKGuby0hcs Discovering latent knowledge in language models without supervision . In The Eleventh International Conference on Learning Representations

    2023

  6. [6]

    Sky CH-Wang, Benjamin Van Durme, Jason Eisner, and Chris Kedzie. 2024. https://doi.org/10.18653/v1/2024.findings-acl.260 Do androids know they`re only dreaming of electric sheep? In Findings of the Association for Computational Linguistics: ACL 2024, pages 4401--4420, Bangkok, Thailand. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-acl.260 2024

  7. [7]

    Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, Ranjay Krishna, Yoon Kim, and James R. Glass. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.84 Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pag...

    doi:10.18653/v1/2024.emnlp-main.84 2024

  8. [8]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. https://arxiv.org/abs/1803.05457 Think you have solved question answering? try arc, the ai2 reasoning challenge . Preprint, arXiv:1803.05457

    arXiv 2018

Show all 35 references
  1. [9]

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. https://doi.org/10.18653/v1/2022.acl-long.581 Knowledge neurons in pretrained transformers . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    2022 doi

  2. [10]

    Xuefeng Du, Chaowei Xiao, and Yixuan Li. 2024. https://openreview.net/forum?id=nfK0ZXFFSn Haloscope: Harnessing unlabeled LLM generations for hallucination detection . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  3. [11]

    Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.3 Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space . In Proceedings of the 2022 Conference on Empirical Methods in Natural L...

    2022 doi

  4. [12]

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.446 Transformer feed-forward layers are key-value memories . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484--5495, ...

    2021 doi

  5. [13]

    Deepanway Ghosal, Siqi Shen, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2022. https://doi.org/10.18653/v1/2022.acl-long.344 CICERO : A dataset for contextualized commonsense inference in dialogues . In Proceedings of the 60th Annual Meeting of the Association for Com...

    2022 doi

  6. [14]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  7. [15]

    Anas Himmi, Guillaume Staerman, Marine Picot, Pierre Colombo, and Nuno M Guerreiro. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1033 Enhanced hallucination detection in neural machine translation through simple detector aggregation . In Proceedings of the 2024 Conference...

    2024 doi

  8. [16]

    Zheng Yi Ho, Siyuan Liang, Sen Zhang, Yibing Zhan, and Dacheng Tao. 2025. https://openreview.net/forum?id=yaOe2xBcLC Novo: Norm voting off hallucinations with attention heads in large language models . In The Thirteenth International Conference on Learning Representations

    2025

  9. [17]

    Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. https://doi.org/10.18653/v1/D19-1243 Cosmos QA : Machine reading comprehension with contextual commonsense reasoning . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processin...

    2019 doi

  10. [18]

    Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. 2020. https://doi.org/10.1609/aaai.v34i05.6319 Qasc: A dataset for question answering via sentence composition . Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8082--8090

    2020 doi

  11. [19]

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. 2023. https://openreview.net/forum?id=aLLuYpn83y Inference-time intervention: Eliciting truthful answers from a language model . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  12. [20]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. https://doi.org/10.18653/v1/2022.acl-long.229 T ruthful QA : Measuring how models mimic human falsehoods . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa...

    2022 doi

  13. [21]

    Junteng Liu, Shiqi Chen, Yu Cheng, and Junxian He. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1012 On the universal truthfulness hyperplane inside LLM s . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18199--18224, Miam...

    2024 doi

  14. [22]

    Samuel Marks and Max Tegmark. 2024. https://openreview.net/forum?id=aajyHYjjsk The geometry of truth: Emergent linear structure in large language model representations of true/false datasets . In First Conference on Language Modeling

    2024

  15. [23]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and 1 others. 2024. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2024 arXiv

  16. [24]

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. https://doi.org/10.18653/v1/D19-1454 Social IQ a: Commonsense reasoning about social interactions . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9...

    2019 doi

  17. [25]

    Noam Shazeer. 2020. https://arxiv.org/abs/2002.05202 Glu variants improve transformer . Preprint, arXiv:2002.05202

    2020 arXiv

  18. [26]

    Siqi Shen, Deepanway Ghosal, Navonil Majumder, Henry Lim, Rada Mihalcea, and Soujanya Poria. 2022. https://arxiv.org/abs/2210.02890 Multiview contextual commonsense inference: A new dataset and task . Preprint, arXiv:2210.02890

    2022 arXiv

  19. [27]

    Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg, Yejin Choi, and Jonathan Berant. 2022. https://arxiv.org/abs/2201.05320 Commonsenseqa 2.0: Exposing the limits of ai through gamification . Preprint, arXiv:2201.05320

    2022 arXiv

  20. [28]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024. https://arxiv.org/abs/2408.00118 Gemma 2: Improving open language models at a practical si...

    2024 arXiv

  21. [29]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. https://arxiv.org/abs/2307.09288 Llama 2: Open foundation and fine-tuned chat models . Preprint, arXi...

    2023 arXiv

  22. [30]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. https://arxiv.org/abs/2505.09388 Qwen3 technical report . Preprint, arXiv:2505.09388

    2025 arXiv

  23. [31]

    Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, and Besmira Nushi. 2024. https://openreview.net/forum?id=gfFVATffPd Attention satisfies: A constraint-satisfaction lens on factual errors of language models . In The T...

    2024

  24. [32]

    Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. https://doi.org/10.18653/v1/D18-1009 SWAG : A large-scale adversarial dataset for grounded commonsense inference . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages ...

    2018 doi

  25. [33]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. https://doi.org/10.18653/v1/P19-1472 H ella S wag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791--4...

    2019 doi

  26. [34]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.