Pith. sign in

REVIEW 5 major objections 5 minor 28 references

Path to Intelligence: Measuring Similarity between Human Brain and Large Language Model Beyond Language Task

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read LLM hidden states can be linearly mapped onto human brain activity in a non-linguistic sensory-motor task, the authors report.

desk verdict A fresh but unproven application of brain-LLM alignment to a non-language task; explicit timestamps in the prompts and a projection fitted on the target data leave the central claim underdetermined. read the letter →

arxiv 2509.08831 v1 pith:KD2B77MF submitted 2025-08-26 q-bio.NC

classification q-bio.NC
keywords LLM-brainalignmentintracranialEEGhigh-frequencyactivityrepresentationalsimilarityCKAin-contextlearninganticipatorybehaviorsensory-motortask
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether an LLM trained only on text shares dynamical structure with the human brain when both perform a task with no linguistic content: watching a fixation point, waiting through a short or long delay for a color change, and pressing a button. The authors claim that LLM internal states, obtained by translating each 10 ms time bin into a labeled token position in a text prompt, can be linearly projected onto human intracranial high-frequency activity recorded during the same task. They test this with response-time distributions, centered kernel alignment, and a discriminability measure separating short- and long-delay trials, and they show that electrode-level features derived from the projection highlight known sensory-motor and attention regions. If correct, the result would mean language-trained models can be used as scalable, fine-tuning-free probes for anticipatory and visuomotor neural processes.

What carries the argument

The load-bearing device is text-based temporal alignment: the experiment's 5-second window is quantized into 10 ms bins, each bin becomes a token position, and the token label at that position encodes the trial state ('start', 'cue', 'button', or 'na'), so the prompt itself is a time series. In-context learning with n-shot examples supplies the task structure without fine-tuning. A shared-then-individual projection maps the LLM's hidden state matrix to each participant's electrode matrix, with W_shared intended to capture cross-subject common structure and W_individual to absorb electrode-specific variation. Alignment is quantified with linear CKA on Gram matrices, and trial-type discriminab

What would settle it

Recompute the global CKA and the d-prime-squared peak-location analysis after randomly permuting the time-index labels assigned to the 'na' tokens (or removing the time indices entirely) while keeping the event keywords at their correct positions. If human-LLM similarity stays near CKA = 0.39 and peaks still align, the reported correspondence does not require the claimed temporal alignment; if it falls to noise, the finding is an artifact of the token-time mapping and would disappear under any change in prompt formatting.

Watch

Extended reading notes

Core claim

The central claim is that LLM-derived representations contain task structure, not just lexical content, and that structure maps onto human neural recordings once the task is expressed as a temporally annotated text sequence. The authors encode each trial as a sequence of 10 ms time bins, labeling the fixation onset 'start', the cue 'cue', the response 'button', and everything else 'na', and they give the LLM a handful of example trials in-context before asking it to respond to a new trial. A two-stage linear projection Y = XW_sharedW_individual trained with MSE loss maps LLM hidden states to each patient's electrode signals. On this basis, projected HFA tracks true HFA dynamics, global CKA b

Load-bearing premise

Everything depends on treating one token position in the LLM prompt as one 10 ms bin of experimental time, so that the model's token at position 50 is assumed to correspond to the brain's moment 500 ms after fixation.

Editorial extensions

If this is right

  • LLM hidden states could serve as a surrogate signal for human high-frequency activity in temporally structured tasks, without retraining or task-specific fine-tuning.
  • With six in-context examples, the model predicts human response times with about 6 ms RMSE, so prompt-based conditioning may be enough to capture individual behavioral timing.
  • The shared projection yields electrode-level feature vectors that can be compared across patients despite differing electrode locations, enabling cross-subject studies that patient-specific iEEG alone makes difficult.
  • Above-random cosine similarity among electrode features in regions such as SLF WM, prefrontal cortex, arcuate/uncinate fasciculus, insula, and ILF-MLF WM points to anatomically meaningful structure in the LLM-derived representation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The one-token-per-10-ms mapping is the linchpin, and it is testable: shuffle or delete the time-index labels while keeping the same event keywords, and see whether CKA and d-prime-squared alignment survive; if they do, the similarity is driven by event identities rather than time.
  • The paper frames W_shared/W_individual as a tool, but a natural extension is to treat the learned projection as a hypothesis generator: regions whose electrode vectors cluster under the LLM projection are candidates for task-relevant nodes in other visuomotor experiments.
  • Because the LLM is never trained on neural data, any genuine alignment would suggest that linguistic descriptions of temporal events already encode some of the relational structure that organized neural anticipatory activity uses—an indirect argument about what language training captures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper reports a study comparing LLM internal states with human intracranial EEG high-frequency activity (HFA) in a visual stimulus-detection task with variable foreperiod delays. Each trial is translated into a token sequence with explicit integer time stamps and event keywords (start/cue/button/na), and n-shot prompting is used to elicit response times. The authors train a two-stage projection Y = X Wshared Windividual from LLM hidden states to electrode HFA, then quantify alignment via CKA, d'2 discriminability, and ROI cosine similarity. They report CKA = 0.3926, temporal alignment of d'2 peaks, and several ROIs with low p-values, concluding that LLM-derived reactions can be linearly mapped onto human neural activity beyond language-based tasks.

Significance. If the central claim were established, this would be a valuable extension of brain–LLM alignment to non-linguistic sensorimotor anticipation, with potential tools for cross-patient ROI analysis. The use of real iEEG data, the choice of in-context learning instead of fine-tuning, and the application of CKA/d'2 are appropriate strengths. However, the current evidence does not yet support the claim: the prompt injects a clock/event grid, the projection is fitted to the same neural target used in evaluation, a key n-shot selection is internally inconsistent, and the reported peak alignments are weak without a null comparison. The paper is promising but needs substantial methodological controls and held-out evaluation before the conclusion can be accepted.

major comments (5)
  1. [Text-based Temporal Alignment for LLM] The prompt explicitly encodes every 10 ms bin with an integer time index and an event keyword (0:start, 50:cue, 84:button, etc.). Human HFA is strongly event-locked at exactly these points. The reported CKA/d'2 alignment may therefore reflect shared time-index/event structure rather than shared anticipatory dynamics. The Gaussian noise control in Table 3 does not address this because noise lacks temporal autocorrelation and event-locked transients. Please add a control using time-index-only prompts (no event words), or shuffled time labels, and report CKA/d'2 for that control. This is load-bearing for the central claim.
  2. [LLM's Hidden States Projection to HFA; Results, Projection Model] The projection model Y = X Wshared Windividual is trained with MSE on the same iEEG HFA signals that are subsequently used for CKA, d'2, and ROI analyses. No train/test split or cross-validation is described; the text only states that loss was reduced on 'training and evaluation sets.' Consequently, the reported alignment may reflect overfitting to the target electrodes/trials. Please provide a clear held-out evaluation protocol, report CKA/d'2 on data not used for fitting the projection, and state whether the projection is refit per cross-validation fold.
  3. [Table 1 and Results (n-shot selection)] Table 1 reports RMSE = 6.11 for n = 4 and RMSE = 6.38 for n = 6, but the text states 'We chose n with the lowest RMSE n = 6 for the rest of the paper.' This directly contradicts the table and is load-bearing because all subsequent analyses use n = 6. Please correct the table/reporting or justify the selection with additional detail (e.g., variability across runs, a different metric, or per-patient results).
  4. [Table 2 and Figure 4 (d'2 peak alignment)] The reported peak alignment is weak: at the 50 ms threshold, the proportions of aligned peaks are only 0.29, 0.14, and 0.07 in the three intervals, and at 150 ms they are 0.71, 0.21, and 0.14. Without a null distribution or chance level for these proportions, the claim of 'overall alignment' is not supported. In addition, the Results text says 'S1 begins at 1000 ms' and S2 at 1500 ms, while the prompt alignment in Methods uses 0:start and 50:cue, implying S1 at 0 ms and S2 at 500 ms. This timeline inconsistency affects the interpretation of all temporal comparisons and should be resolved.
  5. [Understanding Human Brain via LLMs; Table 4] The electrode features are derived from Windividual, which was itself fitted to the HFA electrode signals. Grouping these fitted weights by ROI and measuring within-ROI cosine similarity measures structure inherited from the regression target, not an independent LLM-derived representation. Additionally, with 12 ROIs tested against a Gaussian null, several p-values below 0.05 are expected by chance; no multiple-comparison correction is reported. Please clarify the provenance of the features and apply a correction or permutation test across ROIs.
minor comments (5)
  1. [Central Kernel Alignment Analysis] The linear CKA formula should specify that the Gram matrices are centered and define all norms. The current presentation is ambiguous and could mislead readers about normalization.
  2. [Figure 3] The claim that projected HFA 'closely follows' the true signal is qualitative. Please include quantitative fit metrics (e.g., R^2 or correlation on held-out data).
  3. [Results, LLM and Human Behavior Pattern Similarity] The text says 'We have N = 200 for all n' for RMSE, but it is unclear whether this is total trials or per patient and how trials are split across delay conditions. Please specify.
  4. [References] Many references have incomplete author lists (e.g., 'A, K.; AE, F.; ...'). These should be standardized to the journal's reference format.
  5. [Data and code availability] No data or code availability statement is provided. Given the complexity of the pipeline, making the analysis code available would substantially aid reproducibility.

Circularity Check

3 steps flagged · score 6.0 of 10

Neural-alignment evidence partly reduces to fitted projections and prompt-supplied event times; raw CKA lacks an event/time-index control.

  1. fitted input called prediction [Results: Short vs. Long Delay Trial Discriminative Analysis; Methodology: LLM's Hidden States Projection to HFA]
    "Y = XWsharedWindividual ... The model is trained using Adam optimizer ... with a Mean Squared Error (MSE) loss objective for 50 epochs. ... We also calculate the same d′2 on projected LLM hidden states (XWshared) at each time point to compare human and projected LLM d′2 temporal patterns."

    The projected LLM states XWshared are the fitted output of a regression trained to minimize MSE to the HFA matrix Y. Computing d′2 on these fitted values and comparing them to d′2 on Y is an evaluation of the model on its own training target. The reported alignment of d′2 peaks—including the 'anticipation' peaks before S2—is therefore inherited from the regression's ability to reproduce HFA, not an independent confirmation of LLM–human similarity. The peak locations are statistically forced by the fit insofar as the projection captures the short-vs-long HFA separation.

  2. fitted input called prediction [Understanding Human Brain via LLMs; Methodology: LLM's Hidden States Projection to HFA]
    "As the projection model provides Windividual that encodes a k-dimensional vector for each of the electrodes, we naturally have a feature for each of the electrodes across all patients. We measured the average cosine similarity among electrodes grouped by the different areas of interests (ROI)."

    Windividual is a participant-specific linear map trained to predict each electrode's HFA from the shared LLM-derived representation. The 'electrode features' used in the ROI cosine-similarity analysis are therefore the fitted regression weights for those same electrodes. Their within-ROI structure reflects the geometry of the regression solution and the HFA training data, not an independent LLM-only prediction of brain organization. Calling these 'LLM derived representations' is misleading because the projection was optimized against the electrode signals whose anatomical grouping is then rediscovered.

1 more flagged steps
  1. other [Text-based Temporal Alignment for LLM; Results: Global Alignment]
    "At each time step, a keyword token represents the trial’s true state, paired with its time index (starting from 0). The token mappings are as follows: start for S1, cue for S2, button for RT, and na for all other time steps. ... We computed CKA scores using the full time series of LLM hidden states and iEEG features ... LLM and human iEEG data exhibit moderate alignment (CKA = 0.3926)"

    The LLM input is constructed by labeling every 10 ms bin with the true experimental event (start/cue/button) and its absolute time index, while HFA is time-locked to those same S1/S2/RT events. CKA between LLM hidden states and HFA can therefore be high because both carry the same event/time grid—and for the LLM that grid is inserted by construction into the prompt. The Gaussian-noise control does not remove this confound, since noise lacks event-locked transients. Thus the 'moderate alignment' reflects, at least in part, the prompt's explicit event labels rather than an emergent shared dynamic between LLM internals and neural activity.

full rationale

The paper's behavioral result (Table 1, RMSE of LLM response times) is a genuine LLM output and does not use the fitted projection, so it is independent evidence of behavioral mimicry. Raw CKA is also not computed on the fitted projection and is therefore not statistically forced by W. However, the two central neural-alignment analyses that do use the projection—d′2 on XWshared and ROI cosine similarity of Windividual—are evaluating the regression's own fitted outputs against its HFA target, so those claims are partially circular: the fitted mapping can inject HFA structure into the projected LLM states and into the electrode features. In addition, the prompt's explicit event/time tokens mean even raw CKA can reflect a shared input-constructed event grid rather than discovered shared dynamics. The self-citations (e.g., Ramayya et al. 2025) provide background and preprocessing but are not the load-bearing justification for the similarity claims, so they do not independently raise the score. Overall, the derivation chain contains real independent components but the headline neural-alignment evidence is substantially reduced by construction, yielding a score of 6.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim relies on a fitted projection, an assumed token-to-time equivalence, and several domain assumptions about LLM and brain similarity. No new physical entities are introduced; the projection matrices are learned parameters, not inventions.

free parameters (3)
  • Projection matrices Wshared and Windividual = Learned via Adam, MSE loss, 50 epochs, hidden dimensions [512,128,32,16]; numerical values not reported
    Fit to iEEG HFA; used for projected HFA figures, d'2 analyses, and electrode ROI features, so any result derived from them is conditional on this fitting.
  • n-shot context size = n=6 selected, though Table 1 shows n=4 RMSE 6.11 lower than n=6 RMSE 6.38
    Number of in-context examples changes LLM response-time predictions; the stated selection rule is not consistent with the table.
  • Time discretization step = 10 ms
    The 5 s trial is divided into 10 ms bins, each mapped to one prompt token; this choice sets the temporal resolution for all alignment analyses.
assumptions (6)
  • domain assumption Token-position equivalence
    Assumes the LLM hidden state at token position t corresponds to the neural state at time t (Methodology, Text-based Temporal Alignment for LLM).
  • domain assumption Linear readout can bridge LLM and brain geometry
    Projection Y = X Wshared Windividual assumes a linear structure suffices; no identifiability analysis or non-linear baseline is provided.
  • domain assumption In-context learning approximates human anticipatory learning
    Assumes giving n text examples is analogous to a human participant learning the task on the fly; this is stated but not validated.
  • domain assumption High-frequency activity (HFA) reflects task-relevant neural computation
    HFA preprocessing is taken from Ramayya et al. 2025 and not independently validated in this paper.
  • domain assumption Gaussian null for ROI cosine similarity
    Assumes iid Gaussian random vectors are a fair null for electrode features that come from a fitted projection; ignores correlations among electrodes and multiple testing (Results, Table 4).
  • standard math CKA and d'2 are valid similarity and discriminability measures
    Both are standard definitions from cited work; no new mathematics is introduced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Path to Intelligence: Measuring Similarity between Human Brain and Large Language Model Beyond Language Task." pith.science (2026). https://pith.science/paper/KD2B77MF

@misc{pith2026250908831,
  author       = {Pith},
  title        = {Pith review of: Path to Intelligence: Measuring Similarity between Human Brain and Large Language Model Beyond Language Task},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KD2B77MF}},
  note         = {Machine review of arXiv:2509.08831}
}
read the original abstract

Large language models (LLMs) have demonstrated human-like abilities in language-based tasks. While language is a defining feature of human intelligence, it emerges from more fundamental neurophysical processes rather than constituting the basis of intelligence itself. In this work, we study the similarity between LLM internal states and human brain activity in a sensory-motor task rooted in anticipatory and visuospatial behavior. These abilities are essential for cognitive performance that constitute human intelligence. We translate the sensory-motor task into natural language in order to replicate the process for LLMs. We extract hidden states from pre-trained LLMs at key time steps and compare them to human intracranial EEG signals. Our results reveal that LLM-derived reactions can be linearly mapped onto human neural activity. These findings suggest that LLMs, with a simple natural language translation to make them understand temporal-relevant tasks, can approximate human neurophysical behavior in experiments involving sensory stimulants. In all, our contribution is two-fold: (1) We demonstrate similarity between LLM and human brain activity beyond language-based tasks. (2) We demonstrate that with such similarity, LLMs could help us understand human brains by enabling us to study topics in neuroscience that are otherwise challenging to tackle.

Figures

Figures reproduced from arXiv: 2509.08831 by the authors.

Figure 1
Figure 1. Experimental design and analysis pipeline for comparing human iEEG and LLM representations in a sensory-motor [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Distribution of predicted response times (RT) com [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of true HFA power (blue) and pro [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: RT distribution vs. LLM-Human CKA Score over [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: LLM-Human Mean CKA over 15 time bins. Understanding Human Brain via LLMs As the projection model provides Windividual that encodes a k￾dimensional vector for each of the electrodes, we naturally have a feature for each of the electrodes across all patients. We measured…
Figure 7
Figure 7. Figure 7: Average Cosine Similarity of Electrodes by ROIs [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 24 canonical work pages

  1. [1]

    A, K.; AE, F.; J, B.; JV, H.; and KM, H. 2014. Tracing superior longitudinal fasciculus connectivity in the human brain using high resolution diffusion tensor tractography. Brain Struct Funct

  2. [2]

    Caucheteux C, K. J. 2023. Brains and algorithms partially converge in natural language processing. Commun Biol

  3. [3]

    Chen, J.; Leong, Y.; Honey, C.; et al. 2017. Shared memories reveal shared structure in neural activity across individuals. Nature Neuroscience, 20(1): 115--125

  4. [4]

    F, D.; de Schotten M, T.; F, T.; and P, B. 2008. White matter (dis)connections and gray matter (dys)functions in visual neglect: gaining insights into the brain networks of spatial awareness

  5. [5]

    P.; and Dixon, S

    Flanagan, D. P.; and Dixon, S. G. 2014. The Cattell-Horn-Carroll Theory of Cognitive Abilities. John Wiley & Sons, Ltd. ISBN 9781118660584

  6. [6]

    Goldstein, A.; Grinstein-Dabush, A.; and Schain, M. e. a. 2024. Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns. Nat Commun 15

  7. [7]

    Goldstein, A.; Zada, Z.; and Buchnik, E. e. a. 2022. Shared computational principles for language processing in humans and deep language models. Nat Neurosci

  8. [8]

    Hathaway WR, N. B. 2025. Neuroanatomy, Prefrontal Cortex

Show all 28 references
  1. [9]

    J.; and Koehler, A

    Hyndman, R. J.; and Koehler, A. B. 2006. Another look at measures of forecast accuracy. International journal of forecasting, 22(4): 679--688

  2. [10]

    J, S.; J, R.; R, C.; P, J.; D, K.; C, G.; P, R.-N.; and E, K. 2019. Inferior Longitudinal Fasciculus' Role in Visual Processing and Language Comprehension: A Combined MEG-DTI Study. Front Neurosci

  3. [11]

    J, T.; A, L.; S, J.; and AG, H. 2023. Semantic reconstruction of continuous language from non-invasive brain recordings. Nat Neurosci

  4. [12]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2017. Adam: A Method for Stochastic Optimization. arXiv:1412.6980

  5. [13]

    Kornblith, S.; Norouzi, M.; Lee, H.; and Hinton, G. 2019. Similarity of Neural Network Representations Revisited. Journal of Machine Learning Research

  6. [14]

    Kriegeskorte, N. 2015. Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing. Annual Review of Vision Science, 1(Volume 1, 2015): 417--446

  7. [15]

    Mattson, M. P. 2014. Superior pattern processing is the essence of the evolved human brain. Frontiers in neuroscience, 8: 265. Place: Switzerland

  8. [16]

    Mishkin M., M. K. A., Ungerleider L. G. 1983. Object vision and spatial vision: two cortical pathways. Trends Neurosci

  9. [17]

    P, F.; SA, B.; J, P.; and D, R. 2020. An Anatomic, Imaging, and Clinical Review of the Medial Longitudinal Fasciculus. J Clin Imaging Sci

  10. [18]

    PJ, U.; G, P.; B, L.; L, M.; S, N.; D, N.; and Singer. 2009. Neural synchrony in cortical networks: history, concept and current status. Front Integr Neurosci

  11. [19]

    G.; Buch, V.; Richardson, A.; et al

    Ramayya, A. G.; Buch, V.; Richardson, A.; et al. 2025. Human Response Times Are Governed by Dual Anticipatory Processes with Distinct Neural Signatures. Communications Biology, 8: 124

  12. [20]

    I.; Lecoq, J

    Rumyantsev, O. I.; Lecoq, J. A.; Hernandez, O.; Zhang, Y.; Savall, J.; Chrapkiewicz, R.; Li, J.; Zeng, H.; Ganguli, S.; and Schnitzer, M. J. 2020. Fundamental bounds on the fidelity of sensory cortical coding. Nature, 580(7801): 100--105

  13. [21]

    S, E.; J, L.; O, R.; T, T.; Y, Z.; C, I.; J, L.; S, G.; and MJ, S. 2022. Emergent reliability in sensory cortical coding and inter-area communication. Nature

  14. [22]

    Shin, M.; Kim, D.; and Ryu, J.-K. 2024. CAUS: A Dataset for Question Generation based on Human Cognition Leveraging Large Language Models. arXiv preprint arXiv:2404.11835

  15. [23]

    Brain Hodology - Revisiting disconnection approaches to disorders of cognitive function

    Thiebaut de Schotten , M.; Kinkingnéhun, S.; Delmaire, C.; Lehéricy, S.; Duffau, H.; Thivard, L.; Volle, E.; Levy, R.; Dubois, B.; and Bartolomeo, P. 2008. Visualization of disconnection syndromes in humans. Cortex, 44(8): 1097--1103. Special Issue on "Brain Hodology - Revisit...

  16. [24]

    Wang, Y.; Zhong, W.; Li, L.; Mi, F.; Zeng, X.; Huang, W.; Shang, L.; Jiang, X.; and Liu, Q. 2023. Aligning Large Language Models with Human: A Survey. arXiv:2307.12966

  17. [25]

    Xu, B.; Yang, A.; Lin, J.; Wang, Q.; Zhou, C.; Zhang, Y.; and Mao, Z. 2023. Expertprompting: Instructing large language models to be distinguished experts

  18. [26]

    J.; and Wehbe, L

    Zhou, Y.; Liu, E.; Neubig, G.; Tarr, M. J.; and Wehbe, L. 2025. Divergences between Language Models and Human Brains. arXiv:2311.09308

  19. [27]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  20. [28]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.