Pith. sign in

REVIEW 3 major objections 5 minor 68 references

Final-prompt-token activations carry a graded entity-familiarity readout, and a single familiarity direction moves Gemma-4 refusal from 0.24 to 1.00 on known entities and from 0.73 to 0.00 on unknown ones.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 04:46 UTC pith:WRAQODNZ

load-bearing objection Careful, honest empirical study of a pre-generation familiarity probe; the main results survive scrutiny, but the surface-form confound and single-judge labels need addressing. the 3 major comments →

arxiv 2607.13568 v1 pith:WRAQODNZ submitted 2026-07-15 cs.CL cs.LG

Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering

Grzegorz Brzezinka This is my paper
classification cs.CL cs.LG
keywords entity familiarityprompt-point probingactivation steeringrefusal behaviorhallucination detectionPolish language modelspopularity gradationselective answering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a language model estimates how familiar an entity is before it starts answering: the hidden state at the final question token contains a graded familiarity signal, not just a binary known/unknown bit. Using 1,440 Polish entities stratified by Wikipedia popularity plus fabricated controls, a supervised probe on that token separates real from fabricated entities at AUROC 0.859–0.934 in all four tested model families. The signal is monotone in popularity for Polish-adapted models (Bielik and PLLuM) but nearly flat in Gemma-4 and Qwen3, and the difference tracks Polish continual pretraining more than model size. In Gemma-4-12B, adding a single familiarity direction at layer 30 moves refusal rates monotonically in both directions, showing the readout is causally potent rather than merely correlational. If true, this gives a one-forward-pass pre-generation gate for routing or abstention, while separating what the model represents from the policy that decides whether to say 'I don't know.'

Core claim

The central discovery is that entity familiarity is present, graded, and locally linear in the residual stream before any answer is generated. A logistic-regression probe trained on hidden states at the final prompt token discriminates real from fabricated entities in every family (0.859–0.934 AUROC), rising with log pageviews in Polish-adapted models (mean Spearman rho 0.28–0.57) and nearly flat in Gemma-4 and Qwen3 (at most 0.11). In a paired experiment, keeping entity names unchanged and switching only the question stem from Polish to English retains 96–101% of within-language AUROC; entity-disjoint transfer is strong for Bielik (98–100%) but weaker for Gemma-4 (74–93%). A rank-one famili

What carries the argument

Final-token hidden states at the 'prompt point' — the last question token before any generated token — measured in one forward pass. The main readout is a supervised logistic-regression probe on the residual-stream hidden state, with the layer chosen by cross-validation; unsupervised activation-dispersion metrics and first-token entropy serve as secondary signals. The dataset is 1,440 Polish entities across four domains, stratified into ten log-pageview deciles, plus 240 token-length-matched fabricated names as unfamiliar controls. The causal tool is rank-one activation steering: a difference-of-means familiarity direction at layer 30 and a refusal direction at layer 44, added to the residua

Load-bearing premise

The load-bearing premise is that fabricated names are a faithful operational stand-in for 'entities the model has never met,' yet they are matched to real names only in token length, not in character n-gram statistics, name-component frequency, or morphological naturalness; behavioral labels also come from a single LLM judge with no human audit, so if either gives way, the probe's edge over the 0.786 lexical ceiling and the behavioral mirror both shrink.

What would settle it

Train the same probe on fabricated names matched to real names on character n-grams, name-component frequency, and morphological likelihood, and on real entities created after the model's training cutoff; if AUROC drops to the ~0.786 lexical-ceiling level in both cases, the 'familiarity' readout is mostly surface-form detection rather than exposure. For the causal claim, rebuild the steering direction from a held-out subset of entities and measure refusal on unseen entities only; if the dose–response disappears, the layer-30 effect is an in-sample artifact.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A familiarity probe can act as a pre-generation gate, routing long-tail queries to retrieval before any answer token is produced, at the cost of one forward pass.
  • In Polish-adapted models the readout is graded with popularity, so a calibrated gate could abstain or retrieve proportionally to long-tail risk rather than at a single threshold.
  • Because the layer-30 familiarity direction flips Gemma-4-12B refusal in both directions, representational familiarity and abstention policy are separable: a model can represent familiarity without acting on it, and acting on it can be steered.
  • The Polish/English stem-swap result implies that, in the paired setting, the readout is not primarily a Polish-surface-form artifact, narrowing where language effects could enter.
  • The probe beats adapted post-generation detectors on real-vs-fabricated discrimination but not on predicting behavioral error on average, so the two targets measure different things.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit: apply a familiarity-derived steering direction to a never-refusing Polish model (Bielik or PLLuM) to see whether abstention can be installed; if it can, the missing ingredient in those families is policy rather than representation.
  • If the probe's edge over the character-n-gram ceiling (0.859–0.934 vs. 0.786) survives matching fabricated names on character n-grams, name-component frequency, and morphological likelihood, the readout is a genuine exposure signal; if not, the gap shrinks toward a surface-form artifact — the paper itself flags this as needed follow-up.
  • The dose–response steering result raises a safety extension the author acknowledges: the same familiarity direction that induces refusal can suppress it, so any deployed abstention gate built on this signal would be vulnerable to activation-level manipulation.
  • The entity-disjoint transfer gap between Bielik (98–100%) and Gemma-4 (74–93%) suggests cross-lingual robustness of the familiarity readout may be tied to Polish-adapted training rather than being a general multilingual feature — testable by adding more model families and languages.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies activations at the final prompt token in twelve instruction-tuned models from the Bielik, PLLuM, Gemma-4, and Qwen3 families, using a new Polish-entity dataset with Wikipedia-pageview deciles and fabricated controls. It introduces a supervised logistic-regression familiarity probe and an unsupervised dispersion metric, and reports four main results: (i) probes separate real from fabricated entities in all families and show graded popularity correlation in the Polish-adapted families; (ii) probe transfer across a Polish/English stem substitution retains 96–101% of within-language AUROC in the paired setting; (iii) a rank-one familiarity direction in Gemma-4-12B steers refusal rates monotonically in both directions; and (iv) the calibrated one-pass probe is the best pre-generation gate, while post-generation detectors have better average behavioral-error prediction. The paper is unusually transparent: it reports out-of-fold probe scores, bootstrap CIs, exact permutation tests, before/after continual-pretraining controls, and an explicit Limitations section that flags the main threats to construct validity.

Significance. If the core claims hold, the paper provides a valuable one-forward-pass, pre-generation entity-familiarity readout for Polish and a concrete method for routing long-tail queries to retrieval. Methodologically, the paper is strong: probe scores are out-of-fold, CIs are bootstrap-based, the permutation test has exact null resolution, and the PLLuM base-vs-adapted comparisons are a useful quasi-experimental control. The paper also ships code, dataset, and provenance artifacts. However, the central construct—entity familiarity—is operationalized through a real-vs-fabricated contrast whose surface-statistics confound the paper itself concedes cannot be fully excluded, and the behavioral labels come from a single LLM judge with no human audit. These are not presentation issues; they bear directly on whether the abstract's 'separation between representational familiarity and policy' is established.

major comments (3)
  1. The fabricated-anchor contrast is the training/selection contrast for the probe, the dispersion metric, and the L30 steering direction, yet the fabricated names are matched only on token length. The paper concedes in §10 that 'a lexical-naturalness shortcut cannot be fully excluded.' The character-n-gram AUROC ceiling of 0.786 bounds the real-vs-fabricated classification, but it does not bound the popularity gradation (Table 1, Fig. 1) or the steering axis (Fig. 6). A probe trained on top-deciles-vs-fabricated could exploit surface cues that correlate both with top-decile real names and with fabricated status; the per-decile gradient would then be partly a name-surface gradient rather than a familiarity gradient. Please add direct controls: (a) train the probe on top-decile vs bottom-decile real entities only (no fabricated examples) and report the popularity gradient; (b) rematch fabric
  2. [§7, Fig. 6/Table 2] The familiarity direction is estimated as mean(known) − mean(unknown-real ∪ fabricated) on the same 42 athletes per condition that are then steered and evaluated. The held-out check in §7 is correlational (AUROC on saved activations), not an intervention on held-out entities. Thus the dose–response could partly reflect test-set-specific direction estimation rather than a general familiarity axis. Please report steering refusal rates on held-out entities using a direction built from a disjoint training subset, across the same alpha grid, with explicit CIs. The random-direction control is coarse, as the paper notes: it shows that arbitrary directions are degenerate at high norm, not that non-familiarity semantic directions are inert. An additional control direction built from an unrelated contrast (e.g., cities vs people) at matched norm is needed to support the causal specificity claim.
  3. [§8/§10] The behavioral-prediction results (Table 3b, Fig. 8, and the abstract's 'post-generation detectors better predict behavioral error on average') inherit a target definition in which the strict LLM judge scores explicit refusals as correct ~88% of the time. For Gemma-4-12B, the only abstaining model, this inflates the apparent success of post-generation detectors. The §8 answered-only reanalysis shows that once refusals are treated as abstentions, no gate separates on Gemma-4 (AURC 0.83–0.93 against a 0.90 answered base error). Since the paper itself identifies this as 'the principal evaluation revision,' the answered-only analysis should be the primary behavioral target, or the abstract and Table 3b should be explicitly qualified. As written, the headline behavioral comparison is not robust to the paper's own preferred label treatment.
minor comments (5)
  1. [§6] The abstract's '96–101% within-language AUROC' should be accompanied by the entity-disjoint transfer numbers (98–100% for Bielik but 74–93% for Gemma-4). The current phrasing understates the family difference in generalization to unseen entities.
  2. [§5] The trichotomy of 'rising/flat/falling' AUROC curves is described as coarse; consider moving it to supplementary material and keeping the primary evidence as the per-cell Spearman correlations with CIs, which are currently only in released artifacts.
  3. [§7] In Figure 6, the 'random: 100% degenerate' annotation is mostly relevant at the amplitudes where familiarity steering saturates; at lower amplitudes random controls are 39–100% degenerate. Please state this in the caption to avoid implying all doses are degenerate.
  4. [§5] The behavioral mirror uses a single LLM judge with no human audit. The second-judge agreement (κ=0.65–0.69) is substantial but not high, particularly for Gemma-4 (κ=0.469). Please state explicitly in the main text that all behavioral conclusions are conditional on this judge reliability, rather than only in §10.
  5. [§8, Table 3 footnote] The supervision asymmetry note is good, but the 'unknown sign' of the MIND fidelity gap is easy to miss. Consider moving this to the main text of §8, since it directly affects how the probe-vs-MIND comparison in Table 3(b) should be read.

Circularity Check

1 steps flagged

Partial circularity: the popularity-gradation claim is coupled to the probe's top-decile training contrast 'by construction' (paper's own §10 concession); other central claims are independent.

specific steps
  1. fitted input called prediction [§4/Table 1 (probe definition), §5/Figure 1 & Abstract (gradation claim), §10 Limitations (concession)]
    "probe = P(real) from a top-3-deciles-vs-fabricated logistic probe (Table 1); 'the probe’s positive training class (top popularity deciles) couples its score to the popularity axis by construction' (§10)."

    The headline gradation result ('familiarity-probe scores ... track entity popularity') is measured with a probe whose positive training class was selected from the top Wikipedia-pageview deciles. High P(real) at the top deciles is therefore partly guaranteed by the training contrast, not discovered from activations; the reported Spearman rho over the full decile range includes those training-positive deciles. The paper explicitly concedes this coupling in Limitations, so the effect size is a partially fitted quantity rather than a fully external prediction. The lower-decile held-out trend and the behavioral mirror retain independent value, which is why the circularity is only partial.

full rationale

The derivation is largely self-contained. Cross-language transfer (§6) uses a probe trained in one language and scored zero-shot in another on identical entities; the steering result (§7) is a behavioral intervention with random-direction controls; the gating comparison (§8) uses disjoint eval rows and external behavioral labels; and the Polish-vs-non-Polish adaptation contrast is not explained by the probe's training setup alone. The self-citation to Brzezinka (2026) supplies the measurement recipe and the v1 dataset but is not load-bearing for the paper's new claims, and no uniqueness theorem is imported. The one real circular step is the popularity-gradation headline: the probe is fitted on a top-decile-versus-fabricated contrast, and the paper itself concedes that this couples the score to the pageview axis 'by construction.' The lower-decile generalization and the behavioral mirror provide independent evidence, so the central 'graded familiarity' claim is only partially affected. The real-vs-fabricated AUROC is an out-of-fold probe of its own training target; while standard practice, it should not be read as an independent prediction of familiarity. Overall, partial, disclosed circularity, not a collapse of the derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The paper introduces no new latent particles or abstract constructs, but it does select several measurement knobs on validation data (layer, metric, dose grid), and its interpretation rests on the assumption that pageviews and the fabricated-name screen capture exposure. The familiarity direction itself is an empirical construct with behavioral evidence, not an unfalsifiable addition.

free parameters (4)
  • Best (metric, layer) cell for dispersion = best layer per model (e.g., Bielik-11B L30-ish; Gemma-4-12B mid-depth)
    Dispersion signal is measured at a layer selected by anchor AUROC (top-decile-vs-fabricated). The paper notes this couples cell selection to rho (Spearman 0.795 across 48 cells) and may optimistically bias gradation estimates.
  • Layer for logistic probe = selected by 5-fold CV per model
    Probe layer is chosen per model using CV. Fine for AUROC estimates, but layer choice is an extra degree of freedom; although out-of-fold scores mitigate leakage, the layer search itself can overfit mildly to the anchor contrast.
  • Steering alpha dose grid = 13-point grid, e.g., alpha=-24, +8, +/-12 reported
    The steering results are dose-response curves over a hand-picked grid. The specific saturation points depend on the grid step; random directions at comparable norm degenerate, so the qualitative claim is robust, but the exact numbers (0.24->1.00 and 0.73->0.00) correspond to chosen doses.
  • SwiGLU MLP activation vector and participation ratio definition = inverse participation ratio and Shannon entropy
    The dispersion metric is a particular choice of two scalar summaries of the MLP activation vector; there are many possible dispersion metrics, and the best one is selected on the anchor contrast. This is a mild set of free choices.
axioms (4)
  • domain assumption Wikipedia pageviews are a valid proxy for pretraining exposure to an entity.
    The paper explicitly states pageviews proxy pretraining exposure imperfectly: 12-month window is contemporary while training cutoffs differ, and pageviews correlate with article length and recency. The bottom decile has zero-pageview stubs with no internal gradient. This is load-bearing for the gradation claim.
  • domain assumption The fabricated-name screening (exact page lookup + full-text search) is sufficient to guarantee fabricated entities were unseen during training.
    The paper screens names by non-existence in Polish Wikipedia and full-text search. This does not guarantee the name never appears in other corpora, though it is a plausible filter. The lexical-control analysis helps but cannot fully exclude subtle name-form leakage.
  • domain assumption The refusal marker list and strict-LLM-judge labels capture the behaviors they intend.
    Refusal is detected via a Polish marker list validated by two audits; the strict judge labels explicit refusals as factually correct ~88% of the time, which the paper re-analyzes with a three-way judge. The initial behavioral numbers are affected by this labeling choice.
  • domain assumption The residual stream at the final prompt token can support a linear familiarity direction.
    The steering and probing results assume the familiarity representation is approximately linear in the residual stream. The empirical results support this, but it is a representational assumption, not a proven fact.
invented entities (2)
  • Familiarity direction at layer 30 (Gemma-4-12B) independent evidence
    purpose: A unit-norm direction computed as difference of condition means; added to residual stream at every generation position to steer refusal.
    The paper shows a dose-response effect on refusal (a behavioral handle) and a held-out-entity AUROC 0.800 vs 0.802 in-sample, and cosine 0.999 between train-fold and full-set directions. Those checks provide evidence outside the exact test entities, though still on the same model/domain.
  • Late refusal direction at layer 44 (Gemma-4-12B) independent evidence
    purpose: A distinct direction (cosine -0.14 with familiarity direction) that suppresses refusal when subtracted.
    Shown to lower refusal from 0.73-0.81 to around 0.31-0.40, with sampled confirmations. Behavior is the falsifiable handle.

pith-pipeline@v1.3.0-alltime-deepseek · 192 in / 4831 out tokens · 104608 ms · 2026-08-02T04:46:36.641484+00:00 · methodology

0 comments
read the original abstract

Can a language model estimate its familiarity with an entity before generating an answer? We study activations at the final prompt token in twelve instruction-tuned models from the Bielik, PLLuM, Gemma-4, and Qwen3 families, using a new dataset of 1,440 Polish entities spanning four domains and ten Wikipedia-pageview deciles, plus fabricated controls. Familiarity-probe scores separate real from fabricated entities in every family; in the Polish-adapted Bielik and PLLuM families they additionally track entity popularity (model-mean Spearman $\rho$ 0.28-0.57, versus at most 0.11 in Gemma-4 and Qwen3), a pattern more strongly associated with Polish adaptation than with parameter count in this model sample. In a paired experiment on two families, probes retain 96-101% of within-language AUROC when the Polish question stem is replaced with an English one around unchanged entity names, showing robustness to prompt language in this setting. In Gemma-4-12B, the only model that natively refuses, adding a one-dimensional familiarity direction at a single layer moves refusal rates monotonically in both directions (0.24 to 1.00 on well-known entities; 0.73 to 0.00 on unknown ones). Finally, a calibrated familiarity probe is competitive among pre-generation abstention gates, although post-generation detectors better predict behavioral error on average. These results support a graded pre-generation entity-familiarity readout, and a separation between representational familiarity and the policy that converts it into abstention.

Figures

Figures reproduced from arXiv: 2607.13568 by Grzegorz Brzezinka.

Figure 1
Figure 1. Figure 1: Per-decile familiarity curves on dataset v2 (largest model per family; probe [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Gradation strength (Spearman ρ of familiarity score vs. log pageviews among real entities, mean over four domains) by model size and family. Qwen3 stays near zero from 1.7B to 14B, while the Polish-adapted models (Bielik, PLLuM) show stronger gradation at every matched size. ROCs vs. decile index: rising ρ ≥ 0.5, falling ρ ≤ −0.5, else flat), 27 of 48 are rising and none crosses the falling threshold. This… view at source ↗
Figure 3
Figure 3. Figure 3: Behavioral mirror of the internal gradation: [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Cross-language probe transfer (v1 entities, six models, four domains). A probe trained on Polish ques [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Correlational prelude (Gemma-4-12B, v1 athletes): within well-known entities, refusal is readable from [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Causal dose–response (Gemma-4-12B, rank-one activation addition, single layer, greedy decoding). [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Risk–coverage curves for selective answering on the Polish long tail (target: entity answered incorrectly [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Gate quality by inference category (we report [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 19 linked inside Pith

  1. [1]

    Ferrando, Javier and Obeso, Oscar and Rajamanoharan, Senthooran and Nanda, Neel , booktitle =. Do. 2025 , eprint =

    2025

  2. [2]

    The Internal State of an

    Azaria, Amos and Mitchell, Tom , booktitle =. The Internal State of an. 2023 , publisher =. doi:10.18653/v1/2023.findings-emnlp.68 , eprint =

    doi:10.18653/v1/2023.findings-emnlp.68 2023

  3. [3]

    2024 , eprint =

    Chen, Chao and Liu, Kai and Chen, Ze and Gu, Yi and Wu, Yue and Tao, Mingyuan and Fu, Zhihang and Ye, Jieping , booktitle =. 2024 , eprint =

    2024

  4. [4]

    2024 , eprint =

    Du, Xuefeng and Xiao, Chaowei and Li, Yixuan , booktitle =. 2024 , eprint =

    2024

  5. [5]

    Semantic Entropy Probes: Robust and Cheap Hallucination Detection in

    Kossen, Jannik and Han, Jiatong and Razzak, Muhammed and Schut, Lisa and Malik, Shreshth and Gal, Yarin , year =. Semantic Entropy Probes: Robust and Cheap Hallucination Detection in. 2406.15927 , archivePrefix =

    Pith/arXiv arXiv

  6. [6]

    2024 , url =

    Sriramanan, Gaurang and Bharti, Siddhant and Sadasivan, Vinu Sankar and Saha, Shoumik and Kattakinda, Priyatham and Feizi, Soheil , booktitle =. 2024 , url =

    2024

  7. [7]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =

    Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps , author =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =. 2024 , publisher =. 2407.07071 , archivePrefix =

    Pith/arXiv arXiv 2024

  8. [8]

    Park, Seongheon and Du, Xuefeng and Yeh, Min-Hsuan and Wang, Haobo and Li, Yixuan , year =. Steer. 2503.01917 , archivePrefix =

    Pith/arXiv arXiv

  9. [9]

    Hallucination Detection in

    Binkowski, Jakub and Janiak, Denis and Sawczyn, Albert and Gabrys, Bogdan and Kajdanowicz, Tomasz , booktitle =. Hallucination Detection in. 2025 , eprint =

    2025

  10. [10]

    International Conference on Learning Representations (ICLR) , year =

    Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation , author =. International Conference on Learning Representations (ICLR) , year =. 2302.09664 , archivePrefix =

    Pith/arXiv arXiv

  11. [11]

    Nature , volume =

    Detecting Hallucinations in Large Language Models Using Semantic Entropy , author =. Nature , volume =. 2024 , publisher =

    2024

  12. [12]

    Manakul, Potsawee and Liusie, Adian and Gales, Mark J. F. , booktitle =. 2023 , publisher =. 2303.08896 , archivePrefix =

    Pith/arXiv arXiv 2023

  13. [13]

    2022 , eprint =

    Language Models (Mostly) Know What They Know , author =. 2022 , eprint =

    2022

  14. [14]

    2025 , howpublished =

    On the Biology of a Large Language Model , author =. 2025 , howpublished =

    2025

  15. [15]

    Deja Vu: Contextual Sparsity for Efficient

    Liu, Zichang and Wang, Jue and Dao, Tri and Zhou, Tianyi and Yuan, Binhang and Song, Zhao and Shrivastava, Anshumali and Zhang, Ce and Tian, Yuandong and R. Deja Vu: Contextual Sparsity for Efficient. Proceedings of the 40th International Conference on Machine Learning (ICML) , series =. 2023 , eprint =

    2023

  16. [16]

    International Conference on Learning Representations (ICLR) , year =

    The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers , author =. International Conference on Learning Representations (ICLR) , year =. 2210.06313 , archivePrefix =

    Pith/arXiv arXiv

  17. [17]

    2024 , journal =

    Massive Activations in Large Language Models , author =. 2024 , journal =. 2402.17762 , archivePrefix =

    Pith/arXiv arXiv 2024

  18. [18]

    Physical Review , volume =

    Absence of Diffusion in Certain Random Lattices , author =. Physical Review , volume =. 1958 , publisher =

    1958

  19. [19]

    The Effective Rank:

    Roy, Olivier and Vetterli, Martin , booktitle =. The Effective Rank:. 2007 , address =

    2007

  20. [20]

    2025 , eprint =

    Bielik v3 Small: Technical Report , author =. 2025 , eprint =

    2025

  21. [21]

    2025 , eprint =

    Bielik 11B v2 Technical Report , author =. 2025 , eprint =

    2025

  22. [22]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories , author =. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2023 , publisher =. doi:10.18653/v1/2023.acl-long.546 , eprint =

    doi:10.18653/v1/2023.acl-long.546 2023

  23. [23]

    Proceedings of the 40th International Conference on Machine Learning (ICML) , series =

    Large Language Models Struggle to Learn Long-Tail Knowledge , author =. Proceedings of the 40th International Conference on Machine Learning (ICML) , series =. 2023 , eprint =

    2023

  24. [24]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages =

    Do Large Language Models Know What They Don't Know? , author =. Findings of the Association for Computational Linguistics: ACL 2023 , pages =. 2023 , publisher =. doi:10.18653/v1/2023.findings-acl.551 , eprint =

    doi:10.18653/v1/2023.findings-acl.551 2023

  25. [25]

    interpreting

    nostalgebraist , year =. interpreting

  26. [26]

    Advances in Neural Information Processing Systems (NeurIPS) , volume =

    Compact Language Models via Pruning and Knowledge Distillation , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =. 2024 , eprint =

    2024

  27. [27]

    1979 , address =

    Adams, Douglas , title =. 1979 , address =

    1979

  28. [28]

    2509.11569 , archivePrefix =

    Ding, Yue and Zhu, Xiaofang and Xia, Tianze and Wu, Junfei and Chen, Xinlong and Liu, Qiang and Wang, Liang , year =. 2509.11569 , archivePrefix =

    arXiv

  29. [29]

    2509.15735 , archivePrefix =

    Ettori, Davide and Darabi, Nastaran and Tayebati, Sina and Krishnan, Ranganath and Subedar, Mahesh and Tickoo, Omesh and Trivedi, Amit Ranjan , year =. 2509.15735 , archivePrefix =

    arXiv

  30. [30]

    2024 , publisher =

    Su, Weihang and Wang, Changyue and Ai, Qingyao and Hu, Yiran and Wu, Zhijing and Zhou, Yujia and Liu, Yiqun , booktitle =. 2024 , publisher =. 2403.06448 , archivePrefix =

    Pith/arXiv arXiv 2024

  31. [31]

    2025 , eprint =

    Real-Time Detection of Hallucinated Entities in Long-Form Generation , author =. 2025 , eprint =

    2025

  32. [32]

    No Answer Needed: Predicting

    Moreno Cencerrado, Iv. No Answer Needed: Predicting. 2025 , eprint =

    2025

  33. [33]

    2025 , eprint =

    Orgad, Hadas and Toker, Michael and Gekhman, Zorik and Reichart, Roi and Szpektor, Idan and Kotek, Hadas and Belinkov, Yonatan , booktitle =. 2025 , eprint =

    2025

  34. [34]

    Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , year =

    V. Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , year =. 2504.11975 , archivePrefix =

    Pith/arXiv arXiv 2025

  35. [35]

    Faiyaz Abdullah , booktitle =

    Alvi, Riasad and Sayeedi, Nurul Labib and Sayeedi, Md. Faiyaz Abdullah , booktitle =. 2026 , publisher =

    2026

  36. [36]

    2503.07833 , archivePrefix =

    Abdaljalil, Samir and Kurban, Hasan and Serpedin, Erchin , year =. 2503.07833 , archivePrefix =

    Pith/arXiv arXiv

  37. [37]

    International Conference on Learning Representations (ICLR) , year =

    Retrieval Head Mechanistically Explains Long-Context Factuality , author =. International Conference on Learning Representations (ICLR) , year =. 2404.15574 , archivePrefix =

    Pith/arXiv arXiv

  38. [38]

    2026 , eprint =

    Brzezinka, Grzegorz , title =. 2026 , eprint =

    2026

  39. [39]

    2025 , eprint =

    Yang, An and others , title =. 2025 , eprint =

    2025

  40. [40]

    2025 , eprint =

    Koco. 2025 , eprint =

    2025

  41. [41]

    Gemma 4 Model Card , year =

  42. [42]

    2024 , eprint =

    Grattafiori, Aaron and others , title =. 2024 , eprint =

    2024

  43. [43]

    2024 , howpublished =

    Mistral. 2024 , howpublished =

    2024

  44. [44]

    Advances in Neural Information Processing Systems 37 (NeurIPS 2024) , year =

    Arditi, Andy and Obeso, Oscar and Syed, Aaquib and Paleka, Daniel and Panickssery, Nina and Gurnee, Wes and Nanda, Neel , title =. Advances in Neural Information Processing Systems 37 (NeurIPS 2024) , year =

    2024

  45. [45]

    and Mini, Ulisse and MacDiarmid, Monte , title =

    Turner, Alexander Matt and Thiergart, Lisa and Leech, Gavin and Udell, David and Vazquez, Juan J. and Mini, Ulisse and MacDiarmid, Monte , title =. 2023 , eprint =

    2023

  46. [46]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    Rimsky, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander Matt , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , year =

  47. [47]

    , title =

    Guo, Chuan and Pleiss, Geoff and Sun, Yu and Weinberger, Kilian Q. , title =. Proceedings of the 34th International Conference on Machine Learning (ICML) , year =

  48. [48]

    , title =

    Platt, John C. , title =. Advances in Large Margin Classifiers , publisher =

  49. [49]

    Journal of Machine Learning Research , volume =

    El-Yaniv, Ran and Wiener, Yair , title =. Journal of Machine Learning Research , volume =

  50. [50]

    Advances in Neural Information Processing Systems 30 (NIPS 2017) , year =

    Geifman, Yonatan and El-Yaniv, Ran , title =. Advances in Neural Information Processing Systems 30 (NIPS 2017) , year =

    2017

  51. [51]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    Estimating Knowledge in Large Language Models Without Generating a Single Token , author =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =. 2024 , publisher =. doi:10.18653/v1/2024.emnlp-main.232 , eprint =

    doi:10.18653/v1/2024.emnlp-main.232 2024

  52. [52]

    International Conference on Learning Representations (ICLR) , year =

    On Linear Representations and Pretraining Data Frequency in Language Models , author =. International Conference on Learning Representations (ICLR) , year =. 2504.12459 , archivePrefix =

    Pith/arXiv arXiv

  53. [53]

    Cheang, Chi Seng and Chan, Hou Pong and Zhang, Wenxuan and Deng, Yang , year =. Do. 2510.09033 , archivePrefix =

    Pith/arXiv arXiv

  54. [54]

    Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) , year =

    Pretraining Exposure Explains Popularity Judgments in Large Language Models , author =. Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) , year =. doi:10.1145/3805712.3809958 , eprint =

    arXiv

  55. [55]

    International Conference on Learning Representations (ICLR) , year =

    Programming Refusal with Conditional Activation Steering , author =. International Conference on Learning Representations (ICLR) , year =. 2409.05907 , archivePrefix =

    Pith/arXiv arXiv

  56. [56]

    Hallucination Reduction with

    Yang, Wannan and Qiu, Xinchi and Yu, Lei and Zhang, Yuchen and Yang, Aobo and Kokhlikyan, Narine and Cancedda, Nicola and Garcia-Olano, Diego , year =. Hallucination Reduction with. 2510.02324 , archivePrefix =

    arXiv

  57. [57]

    2026 , eprint =

    Bielik-Minitron-7B: Compressing Large Language Models via Structured Pruning and Knowledge Distillation for the Polish Language , author =. 2026 , eprint =

    2026

  58. [58]

    Advances in Neural Information Processing Systems 36 (NeurIPS) , year =

    Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , author =. Advances in Neural Information Processing Systems 36 (NeurIPS) , year =. 2306.03341 , archivePrefix =

    Pith/arXiv arXiv

  59. [59]

    The Eleventh International Conference on Learning Representations (ICLR) , year =

    Discovering Latent Knowledge in Language Models Without Supervision , author =. The Eleventh International Conference on Learning Representations (ICLR) , year =. 2212.03827 , archivePrefix =

    Pith/arXiv arXiv

  60. [60]

    First Conference on Language Modeling (COLM) , year =

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author =. First Conference on Language Modeling (COLM) , year =. 2310.06824 , archivePrefix =

    Pith/arXiv arXiv

  61. [61]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , year =

    The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , year =

    2023

  62. [62]

    Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) , year =

    On Early Detection of Hallucinations in Factual Question Answering , author =. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) , year =. 2312.14183 , archivePrefix =

    Pith/arXiv arXiv

  63. [63]

    Do Llamas Work in

    Wendler, Chris and Veselovsky, Veniamin and Monea, Giovanni and West, Robert , booktitle =. Do Llamas Work in. 2024 , doi =

    2024

  64. [64]

    and Lian, Qing and Wang, Xingyao and Chen, Yangyi and Ji, Heng and Zhang, Tong , booktitle =

    Zhang, Hanning and Diao, Shizhe and Lin, Yong and Fung, Yi R. and Lian, Qing and Wang, Xingyao and Chen, Yangyi and Ji, Heng and Zhang, Tong , booktitle =. 2024 , doi =

    2024

  65. [65]

    Cheng, Qinyuan and Sun, Tianxiang and Liu, Xiangyang and Zhang, Wenwei and Yin, Zhangyue and Li, Shimin and Li, Linyang and He, Zhengfu and Chen, Kai and Qiu, Xipeng , booktitle =. Can. 2024 , eprint =

    2024

  66. [66]

    Transactions on Machine Learning Research , year =

    Teaching Models to Express Their Uncertainty in Words , author =. Transactions on Machine Learning Research , year =. 2205.14334 , archivePrefix =

    Pith/arXiv arXiv

  67. [67]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , year =

    Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , year =

    2023

  68. [68]

    Asai, Akari and Wu, Zeqiu and Wang, Yizhong and Sil, Avirup and Hajishirzi, Hannaneh , booktitle =. Self-. 2024 , eprint =

    2024