pith. sign in

arxiv: 2606.21195 · v1 · pith:BEMIJATOnew · submitted 2026-06-19 · 💻 cs.CL · cs.AI

Beyond Hooking Onto the World: Referential Profiles and the Numerical Structure of LLM Grounding

Joo Yull Rhee This is my paper

Pith reviewed 2026-06-26 14:42 UTC · model grok-4.3

classification 💻 cs.CL cs.AI
keywords LLM groundingreferential profilesvector groundingsymbol groundingmechanistic interpretabilitynumerical realizationdiscourse reference
0
0 comments X

The pith

LLMs realize reference as distributed, profile-based numerical structures in vector weights and activations rather than fixed links to objects.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the grounding problem for LLMs must move beyond thin, fixed-link accounts of reference to profile-based, context-sensitive, discourse-level, affectively shaped, and norm-governed forms. LLMs acquire these through optimization that parameterizes linguistic traces of human practice inside finite vector systems, where profiles are distributed, may be superposed, and are recovered via context-sensitive computation in weights, activations, attention, and inner-product alignments. Mechanistic interpretability results on entity-like features, knowledge neurons, and emotion-related directions supply indirect evidence for this limited, derivative form of reference in models.

Core claim

LLMs do not acquire reference through perception or embodiment but parameterize inherited linguistic relations inside a finite vector system; referential profiles are distributed and recovered through context-sensitive computation, with weights, activations, attention-mediated hidden states, softmax-trained contrasts, and inner-product alignments serving as the mathematical sites at which these relations become stable and causally active.

What carries the argument

Profile-based reference realized as numerically structured, distributed, and superposable patterns recovered through context-sensitive vector computation.

Load-bearing premise

Mechanistic interpretability findings supply indirect support for the existence of numerically structured referential profiles inside LLMs.

What would settle it

A controlled test in which LLMs fail to maintain stable, context-sensitive referential distinctions across extended discourse when only vector-internal mechanisms are available and no additional human-like perceptual or embodied inputs are supplied.

read the original abstract

This paper revisits the grounding problem for large language models in light of recent vector-grounding accounts. I accept the shift from classical symbol grounding to vector grounding, but argue that the current debate remains incomplete in two respects. First, reference is often treated too thinly, as if it were a fixed link between an isolated expression and an object. I argue instead that reference is profile-based, context-sensitive, discourse-level, affectively shaped, and norm-governed. Even in the human case, reference is publicly stabilized through patterns of use, correction, distinction, inference, and continuation rather than through identical private representations. Second, vector grounding requires an account of numerical realization. LLMs do not acquire reference through human perception, memory, intention, embodiment, or understanding. Rather, through optimization, they parameterize linguistic traces of human world-directed practice. In a finite vector system, referential profiles must be distributed, may be superposed, and are recovered through context-sensitive computation. Weights, activations, attention-mediated hidden states, softmax-trained contrasts, and inner-product alignments are the mathematical sites at which inherited linguistic relations become stable and causally active. Mechanistic interpretability findings, including entity-like features, knowledge neurons, and emotion-related activation directions, provide indirect support for this view. They do not show that LLMs possess human reference. They support a more limited thesis: LLMs may possess derivative, language-mediated, profile-based, and numerically structured forms of reference.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper revisits the grounding problem for large language models, accepting vector-grounding accounts but arguing that reference should be reconceived as profile-based, context-sensitive, discourse-level, affectively shaped, and norm-governed rather than a thin fixed link. It claims that LLMs realize such referential profiles numerically through optimization over linguistic traces, with weights, activations, attention, and inner-product alignments serving as the mathematical sites of stabilization; mechanistic interpretability results (entity-like features, knowledge neurons, emotion-related directions) are invoked only as indirect, non-deductive support for the limited thesis that LLMs may possess derivative, language-mediated, profile-based, and numerically structured forms of reference.

Significance. If the limited modal thesis holds, the paper supplies a philosophically richer framework for interpreting reference in finite vector systems, emphasizing distributed superposition and context-sensitive recovery over isolated hooks. This could usefully inform how existing interpretability observations are read, without claiming human-like reference or advancing new empirical predictions. The absence of formal derivations, new data, or falsifiable tests keeps the contribution conceptual and interpretive rather than transformative for the core cs.CL literature.

minor comments (2)
  1. [Abstract] Abstract, final paragraph: the claim that interpretability findings 'provide indirect support' for numerically structured profiles would be clearer if the manuscript explicitly mapped at least one cited phenomenon (e.g., knowledge neurons) to a concrete aspect of profile-based reference rather than leaving the link suggestive.
  2. [Abstract] The repeated contrast between 'human perception, memory, intention, embodiment' and LLM optimization is effective, but a short parenthetical example of how a specific linguistic trace (e.g., a discourse continuation pattern) becomes numerically active would help readers see the numerical-realization step.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the accurate summary of the manuscript and for recommending minor revision. The referee correctly identifies the paper as a conceptual and interpretive contribution that accepts vector-grounding accounts while advancing a profile-based reconception of reference and a numerical-realization thesis, supported only indirectly by existing mechanistic interpretability results. We agree that the work supplies no formal derivations, new data, or falsifiable predictions and is not positioned as transformative for core empirical cs.CL research.

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper advances a modal philosophical proposal that LLMs may possess derivative, language-mediated, profile-based, and numerically structured forms of reference. It accepts vector grounding as given and treats mechanistic interpretability observations only as indirect, non-deductive support rather than as premises from which the framework is derived. No equations, fitted parameters, predictions, self-citations, or uniqueness theorems appear in the provided text. The central claim does not reduce to its inputs by construction and remains consistent with multiple readings of the cited findings.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The paper depends on several untested philosophical assumptions about the nature of reference and how it transfers to optimized vector systems, with no free parameters or invented entities but multiple domain assumptions drawn from philosophy of language.

axioms (3)
  • domain assumption Reference is profile-based, context-sensitive, discourse-level, affectively shaped, and norm-governed rather than a fixed link between expression and object.
    Presented as the first respect in which current vector-grounding accounts remain incomplete.
  • domain assumption LLMs acquire reference by parameterizing linguistic traces of human world-directed practice through optimization rather than perception or embodiment.
    Stated as the mechanism by which referential profiles become stable in finite vector systems.
  • domain assumption Mechanistic interpretability findings supply indirect support for distributed, superposed referential profiles realized in weights, activations, and attention.
    Invoked to back the limited thesis that LLMs possess derivative numerically structured reference.

pith-pipeline@v0.9.1-grok · 5794 in / 1566 out tokens · 39328 ms · 2026-06-26T14:42:47.158071+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

15 extracted references · 10 canonical work pages

  1. [1]

    Bender EM, Koller A (2020) Climbing towards NLU : On meaning, form, and understanding in the age of data. Association for Computational Linguistics, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 5185--5198, doi:10.18653/v1/2020.acl-main.463, ://aclanthology.org/2020.acl-main.463

    work page doi:10.18653/v1/2020.acl-main.463 2020

  2. [2]

    Bender EM, Gebru T, McMillan-Major A, et al (2021) On the dangers of stochastic parrots: Can language models be too big? doi:10.1145/3442188.3445922

    work page doi:10.1145/3442188.3445922 2021

  3. [3]

    Midwest Studies In Philosophy 4(1):73--121

    Burge T (1979) Individualism and the mental. Midwest Studies In Philosophy 4(1):73--121. doi:10.1111/j.1475-4975.1979.tb00374.x, ://onlinelibrary.wiley.com/doi/abs/10.1111/j.1475-4975.1979.tb00374.x

    work page doi:10.1111/j.1475-4975.1979.tb00374.x 1979

  4. [4]

    Philosophy and the Mind Sciences 7(1)

    Coelho Mollo D, Millière R (2026) The vector grounding problem. Philosophy and the Mind Sciences 7(1). doi:10.33735/phimisci.2026.12307, ://philosophymindscience.org/index.php/phimisci/article/view/12307

    work page doi:10.33735/phimisci.2026.12307 2026

  5. [5]

    Dai D, Dong L, Hao Y, et al (2022) Knowledge neurons in pretrained transformers. Association for Computational Linguistics, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 8493--8502, doi:10.18653/v1/2022.acl-long.581, ://aclanthology.org/2022.acl-long.581

    work page doi:10.18653/v1/2022.acl-long.581 2022

  6. [6]

    Physica D: Nonlinear Phenomena 42(1--3):335--346

    Harnad S (1990) The symbol grounding problem. Physica D: Nonlinear Phenomena 42(1--3):335--346. doi:10.1016/0167-2789(90)90087-6

    work page doi:10.1016/0167-2789(90)90087-6 1990

  7. [7]

    Philosophy of AI 1:19--33

    Koch S (2025) Babbling stochastic parrots? a K ripkean argument for reference in large language models. Philosophy of AI 1:19--33. doi:10.18716/ojs/phai/2025.2325

    work page doi:10.18716/ojs/phai/2025.2325 2025

  8. [8]

    Harvard University Press, Cambridge, MA

    Kripke SA (1980) Naming and Necessity. Harvard University Press, Cambridge, MA

    1980

  9. [9]

    doi:10.1162/coli_a_00522, ://aclanthology.org/2024.cl-3.12/

    Mandelkern M, Linzen T (2024) Do language models' words refer? Computational Linguistics 50:1191--1200. doi:10.1162/coli_a_00522, ://aclanthology.org/2024.cl-3.12/

    work page doi:10.1162/coli_a_00522 2024

  10. [10]

    Communications of the ACM 19(3):113--126

    Newell A, Simon HA (1976) Computer science as empirical inquiry: Symbols and search. Communications of the ACM 19(3):113--126. doi:10.1145/360018.360022

    work page doi:10.1145/360018.360022 1976

  11. [11]

    Putnam H (1975) The Meaning of `Meaning', Cambridge University Press, pp 215--271

    1975

  12. [12]

    Behavioral and Brain Sciences 3(3):417--424

    Searle JR (1980) Minds, brains, and programs. Behavioral and Brain Sciences 3(3):417--424. doi:10.1017/S0140525X00005756

    work page doi:10.1017/s0140525x00005756 1980

  13. [13]

    ://arxiv.org/abs/2604.07729

    Sofroniew N, Templeton A, Jermyn A, et al (2026) Emotion concepts and their function in a large language model. ://arxiv.org/abs/2604.07729

    Pith/arXiv arXiv 2026

  14. [14]

    Transformer Circuits Thread ://transformer-circuits.pub/2024/scaling-monosemanticity/

    Templeton A, Conerly T, Marcus J, et al (2024) Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread ://transformer-circuits.pub/2024/scaling-monosemanticity/

    2024

  15. [15]

    Blackwell, Oxford

    Wittgenstein L (1953) Philosophical Investigations. Blackwell, Oxford

    1953