Pith. sign in

REVIEW 4 major objections 5 minor 59 references

Aligning Brain Activity with Advanced Transformer Models: Exploring the Role of Punctuation in Semantic Processing

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that RoBERTa's internal representations align more closely with fMRI brain activity during narrative reading than BERT's, and that removing punctuation improves BERT's alignment in later layers, suggesting the brain…

desk verdict Useful four-model brain-encoding benchmark, but the punctuation finding is confounded and the headline claims outrun the statistics. read the letter →

arxiv 2501.06278 v3 pith:S36NYKL3 submitted 2025-01-10 cs.CL cs.LG

Zenon Lamprou , Frank Polick , Yashar Moshfeghi This is my paper · ORCID
classification cs.CLcs.LG
keywords fMRItransformermodelsbrainalignmentpunctuationsemanticprocessingRoBERTaBERTsearchlightclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Using fMRI recordings of eight people reading a story chapter, this paper tests whether newer transformer models represent language in a way that tracks human brain activity, and whether punctuation contributes to that alignment. The authors train ridge-regression maps from each model layer's features to brain responses and score them with searchlight classification, following an established alignment procedure, and compare four BERT-derived models against BERT itself. They find that RoBERTa aligns most closely with brain activity, with DistilBERT also ahead of BERT, while ELECTRA and ALBERT do not beat the baseline. In four punctuation-removal scenarios, replacing punctuation with padding or unknown tokens raises BERT's alignment accuracy in layers 7 to 12, with the largest gain near 1.5%, and weakens the usual accuracy drop at longer sequence lengths. The authors take this as evidence that the brain makes limited semantic use of punctuation and needs it less as context grows.

What carries the argument

The mechanism is a two-part alignment pipeline. A ridge regression learns a linear mapping from PCA-reduced transformer features, extracted per layer and per sequence length (4, 5, 10, 15, 20, 25, 30, 35, 40 words), to delayed fMRI BOLD responses, and then searchlight classification scores each voxel's neighbourhood by whether predicted brain images place a correct 20-TR chunk closer than a random incorrect chunk over 1000 trials. The punctuation manipulation is the second load-bearing part: four scenarios replace the fixation symbol or punctuation marks with [UNK] or [PAD] tokens before feature extraction, and the comparison of layer-wise accuracy across these scenarios is what carries the claim about punctuation's semantic role.

What would settle it

Run a control condition that removes or replaces the same number of non-punctuation tokens, or that swaps punctuation for random punctuation tokens, and check whether the roughly 1.5% later-layer accuracy gain in BERT still appears; if it does, the punctuation-specific interpretation is falsified, and if it disappears, the interpretation is supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that, under the brain-alignment procedure, RoBERTa produces layer-wise representations that are more predictive of fMRI responses during natural story reading than BERT's, and DistilBERT also outperforms the BERT baseline, whereas ELECTRA and ALBERT do not. The paper further claims that when punctuation symbols are replaced with [PAD] or [UNK] tokens before feature extraction, BERT's alignment improves specifically in the later layers, 7 through 12, with a peak gain of almost 1.5%, and the accuracy loss at longer sequence lengths is reduced. The authors interpret this pattern as evidence that the brain makes limited semantic use of punctuation and relies on it less as contextual length grows, and they note that layer 6 appears to divide earlier from later layers, consistent with earlier findings that the first six BERT layers are less brain-aligned than the last six.

Load-bearing premise

The argument assumes that replacing punctuation with [PAD] or [UNK] tokens isolates the semantic role of punctuation, but those replacements also change token identities, attention masks, and effective sequence length, so the measured alignment gain could come from mechanical tokenization effects rather than from how the brain processes punctuation.

Editorial extensions

If this is right

  • If RoBERTa's training choices yield more brain-aligned representations, then pretraining decisions such as masked-language-modelling objectives and longer training are candidate levers for making models neurally plausible.
  • If BERT's later layers, 7 to 12, become more brain-aligned when punctuation is masked, then the semantic contribution of punctuation is concentrated in those layers and is small in magnitude.
  • If the accuracy drop with longer context is reduced when punctuation is removed, then punctuation contributes less to comprehension as more context accumulates.
  • If DistilBERT keeps pace with BERT despite being smaller, then model compression does not destroy the brain-relevant semantic information captured in layer representations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a control condition that replaces punctuation with ordinary low-information tokens, rather than [PAD] or [UNK], could separate the semantic effect of punctuation from mechanical tokenization and attention-mask effects; the paper does not include such a control.
  • Beyond the paper: the layer-7-to-12 locus makes a testable prediction that ablating or freezing later BERT layers should remove the punctuation benefit, a manipulation the paper does not perform.
  • Beyond the paper: the alignment score could be turned into a model-selection screen that ranks pretrained models by neural fit before downstream fine-tuning, which would extend the comparison beyond the four models tested.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper applies the Toneva and Wehbe fMRI-alignment pipeline to compare four transformer variants (RoBERTa, DistiliBERT, ALBERT, ELECTRA) against a BERT baseline, using publicly available fMRI data from subjects reading a chapter of Harry Potter. It then tests four punctuation-removal scenarios in which punctuation or fixation symbols are replaced with [UNK] or [PAD] tokens, re-extracts model features, and measures searchlight classification accuracy. The main claims are that RoBERTa aligns most closely with neural activity, surpassing BERT, and that BERT's alignment accuracy improves when punctuation is replaced with [PAD], especially in layers 7–12, which the authors interpret as evidence that the brain makes limited semantic use of punctuation.

Significance. If the central claims held, the paper would provide a useful extension of brain-alignment evaluations to newer transformer architectures and would demonstrate a model-driven way to generate hypotheses about the neural processing of punctuation. The work has real strengths: it uses publicly available data and code, follows a published pipeline, and is transparent about the failure to reproduce the original BERT results exactly, including a qualitative comparison in Figure 2. The alignment scores are computed against external fMRI data with no fitted constants, and the paper's layer-by-layer analyses follow an established methodology. However, the load-bearing conclusions about punctuation rest on a manipulation that is confounded with mechanical tokenization changes, and the reported effects lack statistical validation; these issues affect the central claims rather than just the presentation.

major comments (4)
  1. [§3.3, Removing punctuation] The punctuation-removal manipulation, scenarios 3 and 4, replaces punctuation tokens with [PAD]. This does not isolate the semantic role of punctuation: [PAD] has its own learned embedding, changes the attention mask so that those positions are ignored, and changes the effective token sequence seen by the model. Any of these mechanical effects could alter ridge-regression encoding accuracy without reflecting how the human brain processes punctuation. The manuscript provides no control condition that replaces the same tokens with a semantically neutral but attention-preserving alternative (e.g., [UNK]) or that otherwise varies only the semantic status of the removed tokens. The conclusion in §5 that 'the brain might have limited use for punctuation symbols to understand semantically a sentence' is therefore not uniquely supported by the reported experiments.
  2. [§4, Results with Removing Punctuation] The reported improvement from punctuation removal is described as occurring 'only on layers 7-12' with a maximum boost of almost 1.5%, but no significance tests, error bars, or multiple-comparison corrections are provided. The layer range appears to be selected post hoc, and the claim that layer 6 acts as a 'divisor' is asserted without statistical support. Because the underlying searchlight accuracies vary across subjects, folds, and layers, the authors should report across-subject standard errors or confidence intervals and perform a permutation or bootstrap test comparing the punctuation-removed condition against the baseline, with correction for the number of layers and sequence lengths tested.
  3. [§4, BERT baseline reproduction] The authors state that they were not able to exactly reproduce the original BERT results from [53], and they use their own reproduced results as the baseline for all model comparisons. This is disclosed honestly, but it means the reported differences between RoBERTa or DistiliBERT and the baseline could be within the range of the reproduction discrepancy rather than reflecting genuine model differences. The manuscript should quantify the reproducibility of its own baseline (e.g., across random seeds or PCA initializations) and show that the model comparisons are robust to this variability, or compare all models against the original published numbers on a common metric.
  4. [§5, Conclusion] The conclusion that the brain has limited semantic use of punctuation goes beyond what the encoding-alignment results can show. The experiments demonstrate that replacing punctuation with [PAD] can change the alignment of one transformer model (BERT) with fMRI data; they do not directly measure the brain's use of punctuation. Even if the mechanical confound were addressed, the inference would require additional evidence, such as behavioral reading measures or brain-region-specific analyses that link punctuation processing to semantic processing. The conclusion should be substantially tempered or supplemented with converging evidence.
minor comments (5)
  1. [§3.2] The model names contain typographical artifacts such as 'RoBER T a' and 'DistiliBER T'; these should be corrected to RoBERTa and DistiliBERT throughout the text.
  2. [Throughout] There are several typographical errors in author and reference names, for example 'Toneva and Whebe' should be 'Toneva and Wehbe'; a careful proofreading pass is needed.
  3. [§3.3 and §4] The fixation-replacement scenarios (scenarios 1 and 2) are grouped together with punctuation-removal scenarios in the phrase 'in all four punctuation-modification scenarios,' although the fixation symbol is not punctuation. This conflation makes the summary of results harder to interpret; the two types of manipulations should be reported separately.
  4. [§4, Figure 2] The two panels of Figure 2 use different y-axis ranges, which makes the quantitative comparison between original and reproduced BERT results visually misleading; the authors note this in the caption, but it would be clearer to use a common scale or to overlay the two curves.
  5. [Footnotes] The footnotes referring to the data and code appear as bare placeholders ('the data and the original code can be found at this link', '4') without URL or repository identifiers; these should be completed with working links or DOIs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the results are empirical comparisons against external fMRI data using a published, independent alignment method.

full rationale

The paper's derivation chain is an empirical application, not a derivation from fitted constants. Model features are extracted from pre-trained checkpoints, ridge-regression mappings are trained on training folds and evaluated on held-out fMRI runs in 4-fold cross-validation, and searchlight classification uses pre-computed neighbourhoods from the original external dataset. The comparisons among RoBERTa, DistilBERT, ALBERT, ELECTRA and BERT are therefore computed from data, not constructed by definition. The punctuation-removal results are also empirical: the authors replace punctuation tokens with [PAD] or [UNK] and re-run the same alignment pipeline, and the reported accuracy changes are measured outcomes on test folds. There is no equation-level reduction and no fitted parameter is renamed as a prediction. The only self-citation is reference [31] (Lamprou, Pollick & Moshfeghi 2022) in the related-work section, which is contextual and not load-bearing for any result in the paper; the load-bearing methodological citation is Toneva and Wehbe [53], and the fMRI data come from Wehbe et al. [54], both external to the authors. The reader's confound concern that [PAD] substitution changes token identity, attention masks, and effective context is a validity threat to the punctuation interpretation, but it is not a circularity: the alignment numbers are not forced by construction. Thus the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper makes no theoretical derivation; its central results depend on the validity of the inherited encoding-model pipeline, the interpretability of its evaluation metric, and the assumption that punctuation token substitutions isolate semantic effects.

free parameters (4)
  • PCA dimension = 10
    All model features are reduced to 10 dimensions before ridge regression (§3.3); this truncation is chosen by hand and could affect models with different hidden sizes differently.
  • Ridge regularization lambda = per voxel from 10^x, x = -9 to 9
    For each voxel the lambda with lowest error is selected (§3.3); this is a tuning parameter selected on the data.
  • Searchlight chunk size = 20 TRs
    Prediction evaluation compares chunks of 20 TRs (§3.3); this window size is arbitrary and affects the accuracy metric.
  • Searchlight iterations = 1000
    Each voxel's accuracy is averaged over 1000 random chunk comparisons (§3.3).
assumptions (5)
  • domain assumption A linear ridge regression mapping from PCA-reduced transformer features to fMRI BOLD responses is a valid model of brain-language alignment.
    Adopted from Toneva and Wehbe 2019 (§3.3) without independent validation in this paper.
  • domain assumption Searchlight classification accuracy on 20-TR chunks reflects semantic representational similarity rather than low-level lexical or temporal confounds.
    Evaluation method inherited from Toneva and Wehbe (§3.3); no non-semantic control condition is run.
  • domain assumption Replacing punctuation tokens with [PAD] or [UNK] creates a meaningful counterfactual for studying the semantic role of punctuation.
    The manipulation changes token identity and attention masks, but the paper assumes any alignment change is due to punctuation semantics (§3.3 'Removing punctuation').
  • domain assumption The preprocessed fMRI data of Wehbe et al. (2014) are reliable and suitable for cross-model comparisons.
    Data are taken as given from the public release (§3.1).
  • standard math Ridge regression and PCA are standard linear methods whose mathematical properties are assumed.
    Used throughout §3.3 without proof; the paper claims no new mathematics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligning Brain Activity with Advanced Transformer Models: Exploring the Role of Punctuation in Semantic Processing." pith.science (2026). https://pith.science/paper/S36NYKL3

@misc{pith2026250106278,
  author       = {Pith},
  title        = {Pith review of: Aligning Brain Activity with Advanced Transformer Models: Exploring the Role of Punctuation in Semantic Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S36NYKL3}},
  note         = {Machine review of arXiv:2501.06278}
}
read the original abstract

This research examines the congruence between neural activity and advanced transformer models, emphasizing the semantic significance of punctuation in text understanding. Utilizing an innovative approach originally proposed by Toneva and Wehbe, we evaluate four advanced transformer models RoBERTa, DistiliBERT, ALBERT, and ELECTRA against neural activity data. Our findings indicate that RoBERTa exhibits the closest alignment with neural activity, surpassing BERT in accuracy. Furthermore, we investigate the impact of punctuation removal on model performance and neural alignment, revealing that BERT's accuracy enhances in the absence of punctuation. This study contributes to the comprehension of how neural networks represent language and the influence of punctuation on semantic processing within the human brain.

Figures

Figures reproduced from arXiv: 2501.06278 by the authors.

Figure 1
Figure 1. BERT Following the experimental procedure described in Section 3.3, and using the same data, we were not able to exactly reproduce the same original results as [53]. Thus, our reproduced baseline, used to compare with the other models was obtained by our own code and is presented in [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 1
Figure 1. This figure shows the overall accuracy across all subjects for the four different models we investigated. ALBERT The first model tested was ALBERT. The interesting characteristic to assess on ALBERT was that all the layers shared the same representations. By looking closer it can be seen that there are not any major differences in the accuracy. This would suggest that the changes made on ALBERT to distin￾guish it fr… view at source ↗
Figure 2
Figure 2. On the left hand side are the original results reported in [53]. On the right hand side are the reproduced results we obtained when running our code for BERT. Note that the y-axis has a different minimum and maximum value for the two panels, original and reproduced DistiliBERT Moving on to examine the results with DistiliBERT shows that even though DistiliBERT has fewer layer, and is smaller in size than BERT the pe… view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: The figures presents the results of the 4 different punctuation scenarios. Taking into account the observations mentioned above and the observation of [53] that some layers of BERT are brain aligned, one can hypothesise that the brain might have limited use of punctuat…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 33 canonical work pages

  1. [53]

    In: Advances in Neural Information Processing Systems

    Toneva, M., Wehbe, L.: Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain). In: Advances in Neural Information Processing Systems. vol. 32. Curran Associates, Inc. (2019)

    work page 2019

  2. [1]

    NeuroImage 251, 119005 (May 2022)

    Acunzo, D.J., Low, D.M., Fairhall, S.L.: Deep neural networks reveal topic-level representations of sentences in medial prefrontal cortex, lateral anterior tempo- ral lobe, precuneus, and angular gyrus. NeuroImage 251, 119005 (May 2022). https://doi.org/10.1016/j.neuroimage.2022.119005

    work page arXiv 2022

  3. [2]

    In: Proceed- ings of the 38th International ACM SIGIR Conference on Research and De- velopment in Information Retrieval

    Allegretti, M., Moshfeghi, Y., Hadjigeorgieva, M., Pollick, F.E., Jose, J.M., Pasi, G.: When relevance judgement is happening? an EEG-based study. In: Proceed- ings of the 38th International ACM SIGIR Conference on Research and De- velopment in Information Retrieval. pp. 719–722. SIGIR ’15, Association for Computing Machinery (2015). https://doi.org/10.11...

    arXiv 2015

  4. [3]

    Beltagy, I., Lo, K., Cohan, A.: SciBERT: A Pretrained Language Model for Scientific Text. In: Proceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th International Joint Con- ference on Natural Language Processing (EMNLP-IJCNLP). pp. 3615–3620. Association for Computational Linguistics, Hong Kong, China (Nov 20...

    doi:10.18653/v1/d19-1371 2019

  5. [4]

    The Journal of Neuroscience: The Official Journal of the Society for Neuroscience 37(41), 9999–10011 (Oct 2017)

    Blank, I.A., Fedorenko, E.: Domain-General Brain Regions Do Not Track Linguistic Input as Closely as Language-Selective Regions. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience 37(41), 9999–10011 (Oct 2017). https://doi.org/10.1523/JNEUROSCI.3642-16.2017

    doi:10.1523/jneurosci.3642-16.2017 2017

  6. [5]

    Brain and Language 120(2), 163–173 (Feb 2012)

    Brennan, J., Nir, Y., Hasson, U., Malach, R., Heeger, D.J., Pylkk¨ anen, L.: Syntactic structure building in the anterior temporal lobe during nat- ural story listening. Brain and Language 120(2), 163–173 (Feb 2012). https://doi.org/10.1016/j.bandl.2010.04.002

    work page doi:10.1016/j.bandl.2010.04.002 2012

  7. [6]

    Caucheteux, C., King, J.R.: Language processing in brains and deep neural net- works: computational convergence and its limits. Tech. rep., bioRxiv (Jan 2021). https://doi.org/10.1101/2020.07.03.186288, section: New Results Type: article

    work page doi:10.1101/2020.07.03.186288 2021

  8. [7]

    John, R., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., Strope, B., Kurzweil, R.: Universal sentence encoder for English

    Cer, D., Yang, Y., Kong, S.y., Hua, N., Limtiaco, N., St. John, R., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., Strope, B., Kurzweil, R.: Universal sentence encoder for English. In: Proceedings of the 2018 Conference on Empir- ical Methods in Natural Language Processing: System Demonstrations. pp. 169– 14 Lamprou et al

    work page 2018

Show all 59 references
  1. [8]

    In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, Volume 1 (Long Papers)

    Chen, Y., Gilroy, S., Maletti, A., May, J., Knight, K.: Recurrent neural net- works as weighted language recognizers. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, Volume 1 ...

    2018 doi

  2. [9]

    In: Proceedings of the 4th International Plagiarism Conference (IPC-2010) (2010)

    Chong, M., Specia, L., Mitkov, R.: Using natural language processing for auto- matic detection of plagiarism. In: Proceedings of the 4th International Plagiarism Conference (IPC-2010) (2010)

    2010

  3. [10]

    Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: ELECTRA: Pre-training text encoders as discriminators rather than generators

  4. [11]

    In: Proceedings of the 56th Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers)

    Conneau, A., Kruszewski, G., Lample, G., Barrault, L., Baroni, M.: What you can cram into a single $&!#* vector: Probing sentence embeddings for lin- guistic properties. In: Proceedings of the 56th Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Lo...

    2018 doi

  5. [12]

    arXiv:1901.02860 [cs, stat] (Jun 2019), arXiv: 1901.02860

    Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q.V., Salakhutdinov, R.: Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. arXiv:1901.02860 [cs, stat] (Jun 2019), arXiv: 1901.02860

    1901 arXiv

  6. [13]

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologi...

    2019 doi

  7. [14]

    In: Proceedings of the Probability and Meaning Conference (PaM 2020)

    Ek, A., Bernardy, J.P., Chatzikyriakidis, S.: How does Punctuation Affect Neu- ral Models in Natural Language Inference. In: Proceedings of the Probability and Meaning Conference (PaM 2020). pp. 109–116. Association for Computational Lin- guistics, Gothenburg (Jun 2020)

    2020

  8. [15]

    Procedia Computer Science 113, 273–279 (Jan 2017)

    Etaiwi, W., Naymat, G.: The Impact of applying Different Preprocessing Steps on Review Spam Detection. Procedia Computer Science 113, 273–279 (Jan 2017). https://doi.org/10.1016/j.procs.2017.08.368

    2017 doi

  9. [16]

    Brain and Language140, 1–11 (Jan 2015)

    Frank, S.L., Otten, L.J., Galli, G., Vigliocco, G.: The ERP response to the amount of information conveyed by words in sentences. Brain and Language140, 1–11 (Jan 2015). https://doi.org/10.1016/j.bandl.2014.10.006

    2015 doi

  10. [17]

    Physiological Reviews 91(4), 1357–1392 (Oct 2011)

    Friederici, A.D.: The brain basis of language processing: from struc- ture to function. Physiological Reviews 91(4), 1357–1392 (Oct 2011). https://doi.org/10.1152/physrev.00006.2011

    2011

  11. [18]

    In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Fyshe, A., Talukdar, P.P., Murphy, B., Mitchell, T.M.: Interpretable semantic vec- tors from a joint model of brain- and text- based meaning. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 489–499. Assoc...

    2014 doi

  12. [19]

    Journal of the Association for Information Science and Technology 70, 911–916 (2019)

    Gwizdka, J., Moshfeghi, Y., Wilson, M.L.: Introduction to the special issue on neuro-information science. Journal of the Association for Information Science and Technology 70, 911–916 (2019). https://doi.org/10.1002/asi.24263 Title Suppressed Due to Excessive Length 15

    2019 doi

  13. [20]

    arXiv:1806.04127 [cs] (Jun 2018), arXiv: 1806.04127

    Hale, J., Dyer, C., Kuncoro, A., Brennan, J.R.: Finding Syntax in Human Encephalography with Beam Search. arXiv:1806.04127 [cs] (Jun 2018), arXiv: 1806.04127

    2018 arXiv

  14. [21]

    In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval

    Hong, W., Ji, K., Liu, J., Wang, J., Chen, J., Chu, W.: GilBERT: Generative Vision-Language Pre-Training for Image-Text Retrieval. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval. pp. 1379–1388. SIGIR ’21, A...

    2021

  15. [23]

    Nature 532(7600), 453–458 (Apr 2016)

    Huth, A.G., de Heer, W.A., Griffiths, T.L., Theunissen, F.E., Gallant, J.L.: Nat- ural speech reveals the semantic maps that tile human cerebral cortex. Nature 532(7600), 453–458 (Apr 2016). https://doi.org/10.1038/nature17637

    2016 doi

  16. [24]

    Journal of the Association for Information Science and Technology 70, 917 – 930 (2019), https://api.semanticscholar.org/CorpusID:54041174

    Jacucci, G., Barral, O., Daee, P., Wenzel, M.A., Serim, B., Ruotsalo, T., Pluchino, P., Freeman, J., Gamberini, L., Kaski, S., Blankertz, B.: Integrating neurophysi- ologic relevance feedback in intent modeling for information retrieval. Journal of the Association for Informat...

    2019

  17. [25]

    Jain, S., Huth, A.G.: Incorporating Context into Language Encoding Models for fMRI. Tech. rep., bioRxiv (Nov 2018). https://doi.org/10.1101/327601, section: New Results Type: article

    2018 doi

  18. [26]

    In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Informa- tion Retrieval

    Jia, Q., Li, J., Zhang, Q., He, X., Zhu, J.: RMBERT: News Recommendation via Recurrent Reasoning Memory Network over BERT. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Informa- tion Retrieval. pp. 1773–1777. SIGIR ’21, Associati...

    2021

  19. [27]

    NeuroImage 112, 288–298 (2015)

    Kauppi, J.P., Kandemir, M., Saarinen, V.M., Hirvenkari, L., Parkkonen, L., Klami, A., Hari, R., Kaski, S.: Towards brain-activity-controlled information retrieval: Decoding image relevance from meg signals. NeuroImage 112, 288–298 (2015)

    2015

  20. [28]

    Kingphai, K., Moshfeghi, Y.: Mental workload prediction level from eeg signals using deep learning models (Sep 2021), the 3rd Neuroergonomics Conference 2021, NEC21 ; Conference date: 11-09-2021 Through 16-09-2021

    2021

  21. [29]

    In: Longo, L., Leva, M.C

    Kingphai, K., Moshfeghi, Y.: On EEG preprocessing role in deep learning ef- fectiveness for mental workload classification. In: Longo, L., Leva, M.C. (eds.) Human Mental Workload: Models and Applications. pp. 81–98. Communications in Computer and Information Science, Springer ...

    2021 doi

  22. [30]

    Kingphai, K., Moshfeghi, Y.: On time series cross-validation for deep learn- ing classification model of mental workload levels based on eeg signals. In: Machine Learning, Optimization, and Data Science: 8th International Confer- ence, LOD 2022, Certosa Di Pontignano, Italy, S...

    2023 doi

  23. [31]

    In: Advanced Online & Onsite Course & Symposium on Artificial Intelligence & Neuroscience (2022), https: //strathprints.strath.ac.uk/81945/, num Pages: 15 16 Lamprou et al

    Lamprou, Z., Pollick, F.E., Moshfeghi, Y.: Role of punctuation in semantic map- ping between brain and transformer models. In: Advanced Online & Onsite Course & Symposium on Artificial Intelligence & Neuroscience (2022), https: //strathprints.strath.ac.uk/81945/, num Pages: 15...

    2022

  24. [32]

    Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: A Lite BERT for Self-supervised Learning of Language Representations (Apr 2020)

    2020

  25. [33]

    Journal of Neuro- science 31(8), 2906–2915 (Feb 2011)

    Lerner, Y., Honey, C.J., Silbert, L.J., Hasson, U.: Topographic Mapping of a Hier- archy of Temporal Receptive Windows Using a Narrated Story. Journal of Neuro- science 31(8), 2906–2915 (Feb 2011). https://doi.org/10.1523/JNEUROSCI.3684- 10.2011, publisher: Society for Neurosc...

    2011 doi

  26. [34]

    arXiv:1611.01368 [cs] (Nov 2016), arXiv: 1611.01368

    Linzen, T., Dupoux, E., Goldberg, Y.: Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies. arXiv:1611.01368 [cs] (Nov 2016), arXiv: 1611.01368

    2016 arXiv

  27. [35]

    Mart ´ ınez-Casta˜ no, R., Htait, A., Azzopardi, L., Moshfeghi, Y.: Early Risk Detec- tion of Self-Harm Using BERT-Based Transformers, pp. 183–206. Springer Inter- national Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-04431-1 8, https://doi.org/10.1007/978-3-031-...

    2022 doi

  28. [36]

    ACM Trans

    Michalkova, D., Rodriguez, M.P., Moshfeghi, Y.: Understanding feeling-of-knowing in information search: An eeg study. ACM Trans. Inf. Syst. 42(3) (jan 2024). https://doi.org/10.1145/3611384

    2024 doi

  29. [37]

    Functional Linguistics 3(1), 6 (May 2016)

    Moore, N.: What’s the point? The role of punctuation in realising informa- tion structure in written English. Functional Linguistics 3(1), 6 (May 2016). https://doi.org/10.1186/s40554-016-0029-x

    2016 doi

  30. [38]

    Moshfeghi, Y.: Neurasearch: Neuroscience and information retrieval. CEUR Work- shop Proceedings 2950, 193–194 (Sep 2021), presented at: DESIRES 2021, Design of Experimental Search & Information REtrieval Systems; Proceedings of the Sec- ond International Conference on Design o...

    2021

  31. [39]

    In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Informa- tion Retrieval

    Moshfeghi, Y., Jose, J.M.: An effective implicit relevance feedback technique us- ing affective, physiological and behavioural features. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Informa- tion Retrieval. p. 133–142. SIGIR ’13...

    2013

  32. [40]

    In: The World Wide Web Conference

    Moshfeghi, Y., Triantafillou, P., Pollick, F.: Towards predicting a realisation of an information need based on brain signals. In: The World Wide Web Conference. p. 1300–1309. WWW ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/33085...

    2019

  33. [41]

    In: Proceedings of the 39th International ACM SI- GIR conference on Research and Development in Information Retrieval

    Moshfeghi, Y., Triantafillou, P., Pollick, F.E.: Understanding information need: An fMRI study. In: Proceedings of the 39th International ACM SI- GIR conference on Research and Development in Information Retrieval. pp. 335–344. SIGIR ’16, Association for Computing Machinery (2...

    2016

  34. [42]

    BMC neuroscience 9, 1–21 (2008)

    Mueller, V., Brehmer, Y., Von Oertzen, T., Li, S.C., Lindenberger, U.: Electrophys- iological correlates of selective attention: a lifespan comparison. BMC neuroscience 9, 1–21 (2008)

    2008

  35. [43]

    Communications of the Association for Information Systems 37(1), 46 (2015)

    M¨ uller-Putz, G.R., Riedl, R., C Wriessnegger, S., et al.: Electroencephalography (eeg) as a research tool in the information systems discipline: Foundations, mea- surement, and applications. Communications of the Association for Information Systems 37(1), 46 (2015)

    2015

  36. [44]

    Current Biology 21(19), 1641–1646 (Oct 2011)

    Nishimoto, S., Vu, A.T., Naselaris, T., Benjamini, Y., Yu, B., Gallant, J.L.: Recon- structing visual experiences from brain activity evoked by natural movies. Current Biology 21(19), 1641–1646 (Oct 2011). https://doi.org/10.1016/j.cub.2011.08.031 Title Suppressed Due to Exces...

    2011 doi

  37. [45]

    arXiv:1808.09357 [cs] (Aug 2018), arXiv: 1808.09357

    Peng, H., Schwartz, R., Thomson, S., Smith, N.A.: Rational Recurrences. arXiv:1808.09357 [cs] (Aug 2018), arXiv: 1808.09357

    2018 arXiv

  38. [46]

    In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)

    Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettle- moyer, L.: Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie...

    2018

  39. [47]

    Reddy, A.J., Wehbe, L.: Can fMRI reveal the representation of syntactic structure in the brain? In: Advances in Neural Information Processing Systems. vol. 34, pp. 9843–9856. Curran Associates, Inc. (2021)

    2021

  40. [48]

    Rowling, J.K.: Harry Potter and the Philosopher’s Stone, vol. 1. Bloomsbury Pub- lishing, London, 1 edn. (June 1997)

    1997

  41. [49]

    Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

  42. [50]

    preprint, Neuroscience (Nov 2021)

    Shain, C., Kean, H., Lipkin, B., Affourtit, J., Siegelman, M., Mollica, F., Fedorenko, E.: ‘Constituent length’ effects in fMRI do not provide evi- dence for abstract syntactic processing. preprint, Neuroscience (Nov 2021). https://doi.org/10.1101/2021.11.12.467812

    2021 doi

  43. [51]

    In: Pro- ceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP

    Søgaard, A.: Evaluating word embeddings with fMRI and eye-tracking. In: Pro- ceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP. pp. 116–121. Association for Computational Linguistics, Berlin, Germany (Aug 2016). https://doi.org/10.18653/v1/W16-2521

    2016 doi

  44. [52]

    NeuroImage 62(1), 451–463 (Aug 2012)

    Sudre, G., Pomerleau, D., Palatucci, M., Wehbe, L., Fyshe, A., Salmelin, R., Mitchell, T.: Tracking neural coding of perceptual and semantic features of concrete nouns. NeuroImage 62(1), 451–463 (Aug 2012). https://doi.org/10.1016/j.neuroimage.2012.04.048

    2012 doi

  45. [54]

    PloS One 9(11), e112575 (2014)

    Wehbe, L., Murphy, B., Talukdar, P., Fyshe, A., Ramdas, A., Mitchell, T.: Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses. PloS One 9(11), e112575 (2014). https://doi.org/10.1371/journal.pone.0112575

    2014 doi

  46. [55]

    In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    Wehbe, L., Vaswani, A., Knight, K., Mitchell, T.: Aligning context-based statisti- cal models of language with brain activity during reading. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 233–243. Association for Comput...

    2014 doi

  47. [56]

    In: Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

    Weiss, G., Goldberg, Y., Yahav, E.: On the Practical Computational Power of Finite Precision RNNs for Language Recognition. In: Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 740–745. Association for Comp...

    2018 doi

  48. [57]

    In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

    Zhu, X., Li, T., de Melo, G.: Exploring semantic properties of sentence embeddings. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 632–637. Association for Computational Linguistics, Melbourne, Australi...

    2018 doi

  49. [58]

    In: Proceedings of the 20th Chinese National Confer- ence on Computational Linguistics

    Zhuang, L., Wayne, L., Ya, S., Jun, Z.: A robustly optimized BERT pre-training approach with post-training. In: Proceedings of the 20th Chinese National Confer- ence on Computational Linguistics. pp. 1218–1227. Chinese Information Processing Society of China, Huhhot, China (Aug 2021)

    2021

  50. [174]

    https://doi.org/10.18653/v1/D18-2029

    Association for Computational Linguistics, Brussels, Belgium (Nov 2018). https://doi.org/10.18653/v1/D18-2029

    2018 doi

  51. [2237]

    https://doi.org/10.18653/v1/N18-1202

    Association for Computational Linguistics, New Orleans, Louisiana (Jun 2018). https://doi.org/10.18653/v1/N18-1202

    2018 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.