REVIEW 4 major objections 5 minor 40 references
Higher-order Comparisons of Sentence Encoder Representations
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Hard-to-process sentences show more divergent representations inside language encoders, the paper argues.
desk verdict A useful RSA-based interpretability demonstration with a striking correlation, but the central claim needs length and baseline controls before it lands. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the representational dissimilarity matrix (RDM), a matrix whose entry for a pair of sentences is the correlation distance between their mean-pooled layer representations. For each pair of layers, the paper builds one RDM per layer and then computes a row-wise Kendall's tau between corresponding rows, yielding a per-sentence vector of layer correlations; subtracting from 1 gives per-sentence 'layer disagreement.' These vectors are then correlated with eye-tracking fixation durations and linguistic features using Spearman's rho, a 'third-order' step that connects geometry to behavior.
What would settle it
Recompute the Spearman correlations between layer disagreement and total fixation duration after splitting the Dundee sentences into narrow sentence-length bins (or after partialling out sentence length). If the negative correlations largely disappear or reverse within bins, the claim that processing difficulty specifically drives layer disagreement would be refuted; if they survive, the length-confounding worry is addressed.
Extended reading notes
Core claim
The paper's central discovery is a previously unknown correspondence: sentences with high human processing difficulty, measured by eye-tracking fixation durations, tend to have higher layer disagreement in BERT and ELMo, and also higher disagreement between the two encoders. Layer disagreement for a sentence is defined as 1 minus the row-wise Kendall correlation between two layers' representational dissimilarity matrices, restricted to that sentence's row. Spearman correlations with total fixation duration reach -0.88 for adjacent BERT layers 11-12 and -0.87 for BERT layers 12-13, with similar but weaker negative correlations between ELMo and BERT layers (e.g. -0.66 for E2-B23). The same pattern appears for first-pass reading times. The authors interpret this as evidence that difficult sentences produce degraded or unstable encoder representations, and they report that mid-layer disagreements correlate most with both fixation time and Yngve syntactic complexity, which they read as modest support for the view that middle layers encode syntax.
Load-bearing premise
The load-bearing assumption is that a sentence's layer-disagreement score reflects genuine representation degradation rather than being driven by sentence length or low-level lexical properties, and the paper reports no length-control analysis to rule that out.
Editorial extensions
If this is right
- RSA can serve as an interpretability tool that compares model representations with human behavioral data without training diagnostic classifiers, avoiding overfitting and ceiling effects.
- Layer disagreement, not just individual layer quality, is a meaningful summary statistic of encoder behavior on a sentence.
- The negative correlations predict that similar effects should appear for other pretrained encoders and other eye-tracking corpora, and that middle layers are the most syntax-sensitive.
- Cross-encoder divergence between ELMo and BERT can act as a second, independent signal of processing difficulty.
Reading between the lines
- If the correlation is causal rather than merely correlational, layer disagreement could serve as a cheap proxy for annotating reading difficulty in languages and domains where eye-tracking data do not exist; this is an extension the paper does not test.
- Because the fixation vectors are normalized by sentence length but no length-matched control is reported, the strongest test of the claim is to recompute the correlations within length bins; the paper's central interpretation would be weakened if the effect disappears.
- The method could be turned around: instead of predicting human difficulty from model divergence, one could use fixation data to select sentences that expose representational failure, and then use those sentences as evaluation sets for encoders.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Representational Similarity Analysis (RSA) as a method for comparing sentence encoder representations across layers and across models, and applies it to BERT and ELMo. The authors construct RDM-based per-sentence layer-disagreement vectors, defined as one minus the row-wise Kendall correlation between two layers' RDMs, and correlate these vectors with Dundee eye-tracking fixation durations and three linguistic predictors (log frequency, number of senses, and Yngve score). They report strong negative Spearman correlations between layer disagreement and total fixation duration (e.g., -0.88 for BERT layers 11 and 12), and interpret this as evidence that sentences difficult for humans to process have more divergent encoder representations both within and across models.
Significance. If the central claim holds, the paper offers a conceptually clean, parameter-free way to relate neural language model representations to human processing difficulty, with clear advantages over probing classifiers: it avoids training auxiliary models, makes representational geometry directly comparable, and produces falsifiable predictions about layer pairs. The analysis is self-contained against an external eye-tracking benchmark, no parameters are fitted to the fixation data, and the correlations are reported as statistically significant. The main contributions are methodological (RSA for NLP interpretability) and empirical (a new correspondence between layer disagreement and processing difficulty), and the findings would be of interest to the interpretability and psycholinguistics communities.
major comments (4)
- [Section 3] The paper does not control for sentence length in the layer-disagreement measure. The fixation vectors are explicitly length-normalized ('dividing by its length'), but the disagreement vectors VCorr Li-Lj receive no analogous adjustment: sentence representations are mean-pooled, and mean-pooled vectors from longer sentences can differ in variance, isotropy, and neighborhood structure, which can directly affect row-wise Kendall taus. Because sentence length is plausibly correlated both with fixation duration and with row-wise correlation, the headline negative correlations (e.g., -0.88 for B11-B12) could be partly or wholly artifactual. The authors should report partial Spearman correlations controlling for sentence length, or show that the correlation persists within length-matched bins.
- [Table 1] The middle section of Table 1 reports only the 'top three BERT layer pairs Li and Lj which maximize the correlation scores,' selected from the 276 possible BERT layer pairs. This selection procedure inflates the apparent strength of the relationship and makes the reported -0.88 an extreme-order statistic rather than a pre-specified effect. The authors should report the full distribution of correlations across all layer pairs, or, if selection is used, apply a proper multiple-comparison correction and describe the selection rule before reporting p-values.
- [Section 3 and Section 4] The central claim is not tested against simple baselines. No comparison is made to bag-of-words representations, random or untrained encoders, or representations derived from non-linguistic features. If such baselines show similar negative correlations with fixation duration, then the observed effect would reflect a general property of high-dimensional sentence representations rather than the trained encoders' processing of difficult sentences. Adding these baseline conditions is necessary to support the claim that the correspondence is specific to the language-model representations.
- [Section 4 and Figure 2] The reported correlations between layer disagreement and fixation duration are not accompanied by partial correlations controlling for the paper's own linguistic predictors. Yngve score correlates 0.66 with total fixation duration, and Figure 2 shows that layer disagreement also correlates with Yngve. Because the linguistic features are known processing-difficulty predictors, the authors should report partial Spearman correlations between layer disagreement and fixation duration after controlling for Yngve, log frequency, and word sense, or a mediation analysis. Without this, the 'tripartite affinity' described in the Discussion does not establish that layer disagreement carries information about processing difficulty beyond syntactic complexity and other low-level confounds.
minor comments (5)
- [Section 2 and Figure 1] The correlation distance formula contains a typo in the norm notation ('‖(ni− ¯ni‖2' should be '‖ni− ¯ni‖2'), and the figure caption spells 'similarity' as 'similairty'. These should be corrected.
- [Section 2] The name 'Malahanobis distance' is misspelled; it should be 'Mahalanobis distance'.
- [Section 3] The notation VCorr Li−Lj mixes a superscript and subscript in a way that is hard to parse; a clearer notation, such as V_Corr(Li,Lj), would improve readability.
- [Table 1] The asterisk convention is ambiguous: the text says 'All correlations significant with p<0.0001 after Bonferroni correction unless marked with *,' but it would be clearer to state explicitly that the starred entries are not significant and to report their p-values.
- [Section 5] The word 'complimentary' in the Conclusion should be 'complementary' to match the intended meaning.
Circularity Check
No circularity: the RSA layer-disagreement measure and the Dundee eye-tracking benchmark are independently derived, with no fitted parameters or self-citation chain bearing on the central claim.
full rationale
The derivation chain is self-contained against an external benchmark. Layer disagreement is computed from encoder representations alone: per-sentence row-wise Kendall correlations between two layer RDMs are collected into VCorr Li-Lj, and disagreement is defined as 1 - VCorr Li-Lj. The dependent variables are human gaze fixations from the Dundee corpus, aggregated independently of the encoders by averaging over readers and dividing by sentence length. The final step is a Spearman correlation between these independently measured vectors. No parameter is fitted to the eye-tracking data, and the reported correlations are not used to adjust or re-derive the encoder representations. The central result is therefore not equivalent to its inputs by construction. The only self-citation involving an author of this paper (Matthies and Søgaard 2013) is used as a contrast to note that the tested encoders are not trained with reading behavior in mind; it is not load-bearing evidence for the claimed correspondence. The reviewer's concern that the layer-disagreement correlation might be confounded by sentence length or by syntactic complexity is a validity threat requiring control analyses, but it is not circularity: the paper never defines processing difficulty in terms of encoder disagreement, nor defines layer disagreement in terms of fixation durations. Under the stated rules, no circular step can be quoted and exhibited, so the honest finding is no significant circularity with score 0.
Assumptions & free parameters
assumptions (3)
- domain assumption Total fixation duration and first-pass duration, aggregated per sentence, are valid measures of human processing difficulty.
- domain assumption Layer disagreement, defined as 1 minus the row-wise Kendall correlation between two layer RDMs, reflects representation degradation for a sentence.
- domain assumption Mean-pooling hidden states over word time-steps produces a sentence representation suitable for geometric comparison.
Cite this review
Pith. "Pith review of Higher-order Comparisons of Sentence Encoder Representations." pith.science (2026). https://pith.science/paper/ZB7PU6VW
@misc{pith2026190900303,
author = {Pith},
title = {Pith review of: Higher-order Comparisons of Sentence Encoder Representations},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZB7PU6VW}},
note = {Machine review of arXiv:1909.00303}
}
read the original abstract
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e.g., fMRI, electrophysiology, behavior). As a framework, RSA has several advantages over existing approaches to interpretation of language encoders based on probing or diagnostic classification: namely, it does not require large training samples, is not prone to overfitting, and it enables a more transparent comparison between the representational geometries of different models and modalities. We demonstrate the utility of RSA by establishing a previously unknown correspondence between widely-employed pretrained language encoders and human processing difficulty via eye-tracking data, showcasing its potential in the interpretability toolbox for neural models
Figures
Reference graph
Works this paper leans on
-
[1]
URL: " 'urlintro :=
ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Abnar, S., Beinborn, L., Choenni, R., and Zuidema, W. (2019). Blackbox meets blackbox: Representational similarity & stability analysis of neural language models and brains. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP , pages 191--203, Florence, Italy. Association for Computational Linguistics
work page 2019
-
[4]
Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., and Goldberg, Y. (2016). Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. arXiv preprint arXiv:1608.04207
arXiv 2016
-
[5]
Ashby, J., Rayner, K., and Clifton, C. (2005). Eye movements of highly skilled and average readers: Differential effects of frequency and predictability. The Quarterly Journal of Experimental Psychology Section A , 58(6):1065--1086
work page 2005
-
[6]
Belinkov, Y., M \`a rquez, L., Sajjad, H., Durrani, N., Dalvi, F., and Glass, J. (2017). Evaluating layers of representation in neural machine translation on part-of-speech and semantic tagging tasks. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , volume 1, pages 1--10
work page 2017
-
[7]
Blevins, T., Levy, O., and Zettlemoyer, L. (2018). Deep rnns encode soft hierarchical syntax. arXiv preprint arXiv:1805.04218
arXiv 2018
-
[8]
Bouchacourt, D. and Baroni, M. (2018). How agents see things: On visual representations in an emergent language game. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 981--985, Brussels, Belgium. Association for Computational Linguistics
work page 2018
Show all 40 references
-
[9]
and Alishahi, A
Chrupa a, G. and Alishahi, A. (2019). Correlating neural and symbolic representations of language. arXiv preprint arXiv:1905.06401
2019 arXiv
-
[10]
Chrupa a, G., Gelderloos, L., and Alishahi, A. (2017). Representations of language in a model of visually grounded speech signal. arXiv preprint arXiv:1702.01991
2017 arXiv
-
[11]
M., Khosla, A., Pantazis, D., Torralba, A., and Oliva, A
Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., and Oliva, A. (2016). Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific reports , 6:27755
2016
-
[13]
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., and Bordes, A. (2017b). Supervised learning of universal sentence representations from natural language inference data. arXiv preprint arXiv:1705.02364
2017 arXiv
-
[14]
J., and Goodman, N
Dasgupta, I., Guo, D., Stuhlm \"u ller, A., Gershman, S. J., and Goodman, N. D. (2018). Evaluating compositionality in sentence embeddings. arXiv preprint arXiv:1802.04302
2018 arXiv
-
[15]
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
2018 arXiv
-
[16]
A., Morris, R
Duffy, S. A., Morris, R. K., and Rayner, K. (1988). Lexical ambiguity and fixation times in reading. Journal of memory and language , 27(4):429--446
1988
-
[17]
Gulordava, K., Bojanowski, P., Grave, E., Linzen, T., and Baroni, M. (2018). Colorless green recurrent networks dream hierarchically. arXiv preprint arXiv:1803.11138
2018 arXiv
-
[18]
and Keller, F
Hahn, M. and Keller, F. (2016). Modeling human reading with neural attention. arXiv preprint arXiv:1608.05604
2016 arXiv
-
[19]
Hupkes, D., Veldhoen, S., and Zuidema, W. (2017). Visualisation and'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure. arXiv preprint arXiv:1711.10203
2017 arXiv
-
[20]
Kennedy, A., Hill, R., and Pynte, J. (2003). The dundee corpus. In Proceedings of the 12th European conference on eye movement
2003
-
[21]
Kriegeskorte, N., Goebel, R., and Bandettini, P. (2006). Information-based functional brain mapping. Proceedings of the National Academy of Sciences , 103(10):3863--3868
2006
-
[22]
Kriegeskorte, N., Mur, M., and Bandettini, P. A. (2008). Representational similarity analysis-connecting the branches of systems neuroscience. Frontiers in systems neuroscience , 2:4
2008
-
[23]
Leech, G. N. (1992). 100 million words of english: the british national corpus (bnc)
1992
-
[24]
Levy, R. (2008). Expectation-based syntactic comprehension. Cognition , 106(3):1126--1177
2008
-
[25]
Linzen, T., Dupoux, E., and Goldberg, Y. (2016). Assessing the ability of lstms to learn syntax-sensitive dependencies. arXiv preprint arXiv:1611.01368
2016 arXiv
-
[26]
Loula, J., Baroni, M., and Lake, B. M. (2018). Rearranging the familiar: Testing compositional generalization in recurrent networks. arXiv preprint arXiv:1807.07545
2018 arXiv
-
[27]
and Linzen, T
Marvin, R. and Linzen, T. (2018). Targeted syntactic evaluation of language models. arXiv preprint arXiv:1808.09031
2018 arXiv
-
[28]
and S gaard, A
Matthies, F. and S gaard, A. (2013). With blinkers on: Robust prediction of eye movements across readers. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing , pages 803--807
2013
-
[29]
S., Xiong, C., and Socher, R
McCann, B., Keskar, N. S., Xiong, C., and Socher, R. (2018). The natural language decathlon: Multitask learning as question answering. arXiv preprint arXiv:1806.08730
2018 arXiv
-
[30]
Miller, G. A. (1995). Wordnet: a lexical database for english. Communications of the ACM , 38(11):39--41
1995
-
[31]
Nili, H., Wingfield, C., Walther, A., Su, L., Marslen-Wilson, W., and Kriegeskorte, N. (2014). A toolbox for representational similarity analysis. PLoS computational biology , 10(4):e1003553
2014
-
[32]
E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L
Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv preprint arXiv:1802.05365
2018 arXiv
-
[33]
E., Campbell, S
Raney, G. E., Campbell, S. J., and Bovee, J. C. (2014). Using eye movements to evaluate the cognitive processes involved in text comprehension. Journal of visualized experiments: JoVE , (83)
2014
-
[34]
and Duffy, S
Rayner, K. and Duffy, S. A. (1986). Lexical complexity and fixation times in reading: Effects of word frequency, verb complexity, and lexical ambiguity. Memory & cognition , 14(3):191--201
1986
-
[35]
Rubner, Y., Tomasi, C., and Guibas, L. (2000). The earth mover's distance as a metric for image retrieval. In IJCV
2000
-
[36]
Shepard, R. N. and Chipman, S. (1970). Second-order isomorphism of internal representations: Shapes of states. Cognitive psychology , 1(1):1--17
1970
-
[37]
Tenney, I., Das, D., and Pavlick, E. (2019). Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950
2019 arXiv
-
[38]
T., Kim, N., Van Durme, B., Bowman, S., Das, D., et al
Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., McCoy, R. T., Kim, N., Van Durme, B., Bowman, S., Das, D., et al. (2018). What do you learn from context? probing for sentence structure in contextualized word representations
2018
-
[39]
Van Gompel, R. P. (2007). Eye movements: A window on mind and brain . Elsevier
2007
-
[40]
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. R. (2018). Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461
2018 arXiv
-
[41]
Yngve, V. H. (1960). A model and an hypothesis for language structure. Proceedings of the American philosophical society , 104(5):444--466
1960
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.