{"id":"572bf603-a9c5-46b3-a9d5-c53ff6e35b4f","arxiv_id":"2411.08384","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A linear projection of Word2Vec and GloVe embeddings onto eight part-of-speech axes yields compact interpretable vectors, and combining them with the original vectors gives small gains on a few downstream tasks.","lead":"This paper turns ordinary word vectors into compact, eight-coordinate vectors, one per part of speech, by projecting pretrained embeddings onto averages of WordNet word lists. The derived hierarchical vectors slightly improve a few text classification benchmarks, but the claimed advantage is inconsistent across variants.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Interpretability claim is not established: the POS subspace is fit on WordNet and tested on the same WordNet labels, so reported classification accuracy is expected by construction.","rationale":"The reader's weakest_assumption identifies the same unvalidated modeling assumption: that POS labels are linearly recoverable from pretrained embeddings via WordNet averages, never tested on held-out or non-WordNet data. My stress-test sharpens this into a specific circularity: §7.1 uses the same WordNet resource both to construct the subspace (§3) and as the oracle for classification. The table's accuracy therefore cannot be read as evidence for generalizable interpretability. This is the load-bearing point because the abstract's central claim couples interpretability with downstream gains; if the interpretability evidence is circular, the unique contribution of the paper is unverified. The downstream benchmark story is also weaker than claimed (only WOA and GOL beat the base average, and significance is shown only for selected tasks), but that is secondary to the interpretability finding. A held-out split with independent POS annotations would settle the matter. Thus my assessment does not change the reader's REJECT verdict.","tokens_in":11623,"tokens_out":3829,"duration_ms":39318,"concrete_test":"Build the transition matrix C from a random 50% split of WordNet words, using Eq. 1-3, and evaluate §7.1 classification on the held-out 50% (words never used to compute C). Repeat for both Word2Vec and GloVe. If held-out accuracy is near chance or substantially below Table 4, the reported interpretability is an artifact of training and testing on the same WordNet words. As a second control, build C from frequency-matched random word lists and report the same held-out accuracy; if random directions match the POS-built directions, the method is not capturing syntactic regularities.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that syntactic representations are interpretable rests on the assumption that the eight POS averages from WordNet form a subspace in which a word's largest coordinate reliably marks its part of speech (§3, Eqs. 1-3). The paper's only quantitative evidence for this is the word classification experiment in §7.1, which uses WordNet as both the source of the subspace and the oracle for labels. Words in V∩W are projected onto directions built from their own POS averages, normalized via Eqs. 5-6, and then classified by selecting the coordinate whose value is 1. This is circular: the projection is fit to the very labels being predicted. High accuracy on the construction vocabulary is expected even if pretrained embeddings contain no generalizable POS signal, because each word is projected onto its own class centroid. The result therefore does not show that the coordinates are meaningful for unseen words or for POS labels from an independent annotation source. Since this unvalidated geometry is the foundation for both the interpretability claim and the hierarchical vectors that weight and concatenate these coordinates, the central claim is not supported by the reported experiments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a post-processing method that projects pretrained word embeddings (Word2Vec and GloVe) onto an 8-dimensional 'syntactic subspace' whose axes are derived from average vectors of WordNet words for each part of speech. The resulting Syntactic Representations are then combined with the original vectors, either by Kronecker product (overcomplete) or by weighting, to produce Hierarchical Vectors. The paper evaluates these vectors on seven downstream tasks and word-similarity benchmarks, reports accuracy for POS classification using WordNet labels, gives qualitative top-word examples, and argues that the representations are interpretable and that the hierarchical vectors perform at least as well as, and sometimes better than, the base embeddings.","tokens_in":11803,"tokens_out":7669,"duration_ms":76855,"significance":"A reliable method for converting dense embeddings into low-dimensional, interpretable syntactic axes, with no loss on downstream tasks, would be a useful contribution. The paper includes a broad benchmark suite, a statistical significance test, and qualitative word lists that are suggestive. However, the significance is substantially undercut by three issues: the primary interpretability experiment is circular, the abstract's blanket performance claim is not supported by the per-variant results in Table 1, and the core equations contain notation and normalization errors. These problems affect the central claims of the paper, not just its presentation.","major_comments":[{"comment":"The word-classification experiment is circular as a validation of the interpretability claim. The syntactic subspace is built in Section 3 from averages of WordNet words for each POS, and the oracle for the classification experiment in Section 7.1 is the same WordNet resource. A test word's projection is computed from directions that include that word's own POS class average, so high accuracy on this construction is expected even if the embeddings contain no POS signal beyond the class centroids themselves. The paper needs an evaluation on held-out words (for example, a split of WordNet synsets or leave-one-out) or, preferably, on POS labels from an independent annotation source such as a tagged corpus. The Google Text Normalization experiment in Appendix G is not a substitute: it reports only a confusion matrix and an accuracy of 58.78%, with no description of how the 36 tags were conflated, no baseline, and no held-out split.","section":"Section 7.1 and Section 3"},{"comment":"The abstract's claim that 'subsequent hierarchical vectors outperform the original vectors in benchmark tests' is contradicted by Table 1. Only WOA (81.79 vs 80.49 for Word2Vec) and GOL (83.37 vs 83.34 for GloVe) have higher averages than their base vectors; the other ten variants all have lower averages. Even WOA and GOL do not improve uniformly across tasks: WOA loses on NP bracketing and discriminative attributes, and GOL's average margin is only 0.03 percentage points. The statistical significance table (Table 2) does not state which hierarchical variant enters the comparison, and for NP bracketing and sentiment the reported p-values (0.75 and 0.42/0.97) give no support. The paper should replace the blanket claim with a per-variant claim restricted to the tasks and variants for which the improvement is reliable, and it should address the substantial drops on word similarity in Table 7 (for example, WOA on SimLex-999 falls from 44.20 to 36.89, and GOA from 40.83 to 29.81) rather than dismissing them via Faruqui et al.","section":"Abstract and Section 6.2, Table 1"},{"comment":"The stated normalization does not produce values in [0.5, 1]. With I_intermediate = (S_i - min S)/(max S_i - min S), the maximum of I_intermediate is 1, so Eq. (5) gives I_i = 1.5 at that coordinate. This contradicts both the description 'scales them to a range between 0.5 and 1' and the claim in Section 7.1 that the predicted label is the dimension with value 1. The formula should presumably be I_i = 0.5 * I_intermediate + 0.5, or the prediction rule should be stated as selecting the argmax coordinate. This needs to be corrected because the Interpretable Syntactic Representation is the basis for the hierarchical variants and for the classification experiment.","section":"Appendix A.2, Eqs. (5)-(6)"},{"comment":"The mathematical construction of the syntactic subspace is ambiguous. With C in R^{(V∩W)×8}, Eq. (2) treats C^T as an 8×(V∩W) matrix acting on S_{v,:}, but Eq. (3) then uses S_{v,:} as an 8-vector. The subscript in Eq. (1) sums over V while the set of nouns is N, and the notation C_{1,:} is used for what appears to be a coordinate of a single POS. The transition matrix construction and the projection in Eqs. (2)-(3) must be rewritten with consistent dimensions so that the method is reproducible.","section":"Section 3, Eqs. (1)-(3)"},{"comment":"The definition of the Weighted Hierarchical Vector is incomplete. The index j in 'j ∈ [a, b]' is undefined, and the expression V^s_{i:} × V^r_{j:} is not a standard product with the stated dimensions. Since these vectors are evaluated in Table 1, the operation must be specified precisely. As written, Eq. (9) does not define a reproducible construction.","section":"Appendix B.2, Eq. (9)"}],"minor_comments":[{"comment":"There are several citation and punctuation errors in the Related Work section, including 'Faruqui et al. (2015)Subramanian et al. (2018)Panigrahi et al. (2019)' and '(Mathew et al., 2020)(Engler et al., 2023)', which should be corrected.","section":"Section 2"},{"comment":"In the Weighted Word2Vec block of Table 7, the row labels repeat WOA/WOI/WOL instead of WWA/WWI/WWL, so the table as printed cannot be interpreted reliably.","section":"Table 7"},{"comment":"The condition number is reported as '>100000' without the corresponding effective smallest singular value; please state the exact value and the threshold used to judge it problematic.","section":"Table 3 and Section 6.5"},{"comment":"The word-list-size experiments are presented without error bars or statistical tests, so the conclusion that the change is 'not consistent' is not quantified.","section":"Appendix E and Figure 4"},{"comment":"The paper does not state whether code or data will be released; a reproducibility statement would be helpful.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper has no code or data release statement, and the manuscript would benefit from careful proofreading of equations and tables. The circularity of the interpretability experiment and the overbroad performance claim are the main barriers to acceptance; I do not think they are unfixable, because a held-out or independent-label POS evaluation and a per-variant, task-specific performance claim could be added in a revision. If the authors are unwilling or unable to provide such evidence, rejection would be warranted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is genuinely fresh: project pretrained word vectors onto eight WordNet-derived part-of-speech centroids, then combine those 8-dimensional projections with the original vectors via Kronecker products or weighted averages. It is cheap, simple, and the qualitative top-word lists show recognizable syntactic groupings (the pronoun and interjection dimensions are particularly telling). The author also deserves credit for an honest limitations section, including the polysemy issue and the degraded word-similarity scores, and for attempting statistical significance tests on the downstream results.\n\nBut the central claims are not established. The interpretability validation is circular: the POS subspace is built by averaging WordNet words per POS, and then the same WordNet words are projected onto that subspace and 'classified' by the maximal coordinate. High accuracy is expected by construction, even if the pretrained embeddings carry no generalizable POS signal. The independent eight-class experiment in Appendix G gives 58.78%, which is closer to what you'd expect from an honest held-out evaluation, but there is no indication that those words were excluded from the subspace construction.\n\nBeyond circularity, the math is sloppy in load-bearing places. Equations 2-3 have mismatched dimensions: C is defined as (V∩W)×8, so C^T·S cannot equal a row of X, and the projection formula as written cannot produce a 1×8 syntactic representation. Equations 5-6 normalize to [0.5, 1.5], yet the text says the range is [0.5, 1] and the classification rule relies on the value 1 marking the class. These are not minor typos; they affect how the method is supposed to work. Also, Table 1 shows only WOA (and GOL by 0.03 points) beating the base on average, so the abstract's 'hierarchical vectors outperform' is a clear overstatement. No code or data is provided, which makes it hard to separate presentation error from substantive confusion.\n\nWho is this for? A researcher working on interpretable embedding post-processing might find a seed of an idea, but they would need to redo the evaluation and fix the math. The paper is not ready as is, but it deserves serious referee time rather than a desk reject because the idea is original and the flaws are fixable. I would advise a major-revision outcome: use a held-out POS tagger or non-WordNet labels for the interpretability test, correct the equations and normalization, release code, and temper the performance claims.","headline":"Original post-processing idea with a circular WordNet-based interpretability test and overclaimed performance gains; worth a look but needs major revision.","tokens_in":12340,"tokens_out":4567,"would_cite":false,"duration_ms":47816,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that dense word vectors can be projected onto an 8-dimensional syntactic subspace whose coordinates correspond to the eight parts of speech, producing compact, human-readable vectors that, when combined with the…","keywords":["syntactic representation","word embeddings","interpretability","hierarchical vectors","part-of-speech","WordNet","post-processing","incremental learning"],"falsifier":"Take a random half of the WordNet words per part of speech, build the eight average vectors from them, project the held-out half, and compare the largest coordinate's label against WordNet's label; if accuracy for held-out words drops far below the in-sample accuracy reported in Table 4, the subspace is overfit to the particular word list rather than capturing general syntactic regularities. A weaker but also decisive test is to run the interpretable representation on a balanced corpus labeled by a statistical tagger and measure agreement on words that are absent from WordNet.","tokens_in":11380,"feed_emoji":"","tokens_out":2350,"duration_ms":26324,"temperature":0.7,"pith_summary":"The paper tries to establish that word embeddings, which are normally dense and opaque, can be rewritten as short 8-number vectors where each number measures how strongly a word behaves like one of the eight parts of speech. It derives these numbers by averaging all WordNet word vectors per part of speech and projecting every word onto that subspace. It then concatenates or weights the original embedding with this compact representation to form hierarchical vectors. The paper claims these hierarchical vectors beat the plain pretrained vectors on average across several downstream tasks, while the 8-dimensional representations themselves align with human judgments of a word's part of speech. The point is that interpretability does not have to come at the cost of task performance.","feed_headline":"Eight POS coordinates make word vectors interpretable","feed_subtitle":"Projecting embeddings onto a WordNet-derived 8-axis subspace keeps or improves classifier accuracy.","key_machinery":"The load-bearing object is the syntactic subspace defined by the transition matrix C, whose eight rows are the mean pretrained vectors of all WordNet words belonging to each part of speech. The projection S_v = (C^T)^+ x_v, using the Moore–Penrose inverse, maps a full word vector to an 8-dimensional coordinate vector whose relative values are claimed to encode the word's part of speech. The hierarchical vectors are composite representations: overcomplete ones formed by the Kronecker product of the syntactic and original vectors, and weighted ones formed by an element-wise weighted average. The subspace construction, the projection step, and the weighting rule together carry the argument.","core_discovery":"The central claim is that the syntactic regularities hidden inside ordinary dense word vectors can be isolated as an 8-dimensional subspace whose axes are the eight parts of speech, and that these reduced coordinates are both meaningful to humans and useful for learning. The paper constructs a transition matrix C whose rows are the average word vectors of nouns, verbs, adjectives, adverbs, pronouns, prepositions, conjunctions, and interjections, taken from WordNet words that also appear in the pretrained vocabulary. Any word vector is then projected onto this subspace using the Moore–Penrose inverse, yielding an 8-dimensional syntactic representation. Three normalizations of this representation are defined: absolute, interpretable (scaled to 0.5–1 with 1 marking the dominant part of speech), and L2-normalized. These representations are combined with the original vector either by Kronecker product, giving an overcomplete hierarchical vector, or by weighting the original vector's coordinates with the syntactic scores, giving a weighted hierarchical vector of the same dimension as the input. On news classification, question classification, noun-phrase bracketing, sentiment analysis, and attribute-discrimination tasks, the hierarchical vectors match or exceed the base Word2Vec and GloVe vectors on average, although statistical significance is demonstrated for only a subset of tasks.","pith_inferences":["The same subspace idea could be tested on contextual embeddings from masked language models, where the projection might reveal whether POS information is as linearly recoverable as it appears in static vectors; the paper does not attempt this.","Because the subspace is built from WordNet averages, a natural stress test is to replace WordNet with a POS-tagged corpus and see whether the eight axes change; this would separate the method from the particular lexical resource.","The reported loss on word-similarity benchmarks, combined with gains on classification tasks, suggests the syntactic projection removes semantic nuance; a practical extension would be to blend the syntactic and original vectors with a tunable weight rather than fix the composition rule.","The extremely high condition number of the overcomplete vectors hints that the Kronecker product amplifies rare-axis noise; applying a spectral truncation before concatenation might recover the similarity scores without hurting classification."],"forward_implications":["If the claim holds, any pretrained static embedding can be post-processed into an 8-dimensional syntactic description without retraining on raw text, making large vocabularies cheap to inspect and compare.","The hierarchical vectors give a direct way to inject syntactic information into downstream classifiers: a classifier can start from a small, interpretable feature set and then enlarge it to the full vector, matching human-like incremental learning.","The interpretable representations can be used to label a word's dominant part of speech by reading off the coordinate closest to 1, yielding a lightweight POS tagger for vocabulary not covered by an external tagger.","The weighted hierarchical vectors preserve the original dimension, so they can be dropped into existing models that expect pretrained embeddings without changing architecture or training pipelines.","The overcomplete variant, despite its very large condition number, still matches base performance on several tasks, suggesting that the added syntactic channels carry signal rather than pure noise."],"supporting_citations":[{"why":"Supplies the WordNet lexical database whose per-part-of-speech word lists define the rows of the transition matrix C.","marker":"(Miller, 1995)"},{"why":"Provides the Word2Vec pretrained vectors that are one of the two base representations transformed by the method.","marker":"(Mikolov et al., 2013)"},{"why":"Provides the GloVe pretrained vectors that are the second base representation tested.","marker":"(Pennington et al., 2014)"},{"why":"Gives the Moore–Penrose inverse used in the projection step that maps a word vector onto the syntactic subspace.","marker":"(Ben-Israel & Greville, 2003)"},{"why":"Defines the sparse overcomplete interpretability baseline and the qualitative top-words evaluation procedure that the paper adapts for syntactic grouping.","marker":"(Faruqui et al., 2015)"},{"why":"Provides the corrected paired t-test used to assess whether the hierarchical vectors outperform the base vectors beyond chance.","marker":"(Nadeau & Bengio, 1999)"}],"fun_headline_variants":["8 POS axes make word vectors readable and stronger","Syntactic 8-axis projection yields interpretable word vectors","POS subspace turns dense embeddings into hierarchical vectors","Word vectors become transparent via eight syntactic coordinates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that averaging all WordNet word vectors for each part of speech yields a subspace in which any word's projection has coordinates whose relative sizes reliably indicate its part of speech, and it never checks this premise against words outside WordNet or against an independent POS oracle.","fun_headline_variants_meta":{"raw":{"variants":["8 POS axes make word vectors readable and stronger","Syntactic 8-axis projection yields interpretable word vectors","POS subspace turns dense embeddings into hierarchical vectors","Word vectors become transparent via eight syntactic coordinates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1382,"prompt_tokens":911,"completion_tokens":471,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":411}},"tokens_in":527,"tokens_out":471,"duration_ms":5352,"temperature":1.0,"reasoning_tokens":411,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:38:47.243519+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random half of the WordNet words per part of speech, build the eight average vectors from them, project the held-out half, and compare the largest coordinate's label against WordNet's label; if accuracy for held-out words drops far below the in-sample accuracy reported in Table 4, the subspace is overfit to the particular word list rather than capturing general syntactic regularities. A weaker but also decisive test is to run the interpretable representation on a balanced corpus labeled by a statistical tagger and measure agreement on words that are absent from WordNet.","supporting_citations":[],"review_version":1}