{"id":"1956a460-8299-4153-bbb1-3dee71cbf453","arxiv_id":"2411.08290","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"RESOLVE combines vector symbolic computing with an attention mechanism to improve few-shot accuracy on relational reasoning tasks such as sorting and math problem solving.","lead":"This paper introduces RESOLVE, a neural network architecture that uses high-dimensional vector operations (bundling and binding) to combine object features with learned relational symbols. The authors report accuracy gains over transformer and relational bottleneck baselines on small reasoning benchmarks such as sorting and arithmetic word problems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The HD-attention score in Eq. 1 lacks demonstrated connection to asymmetric relations; without an independent probe of the attention matrix, the claimed relational mechanism is unverified.","rationale":"The central claim has two parts: the architecture achieves higher accuracy and better generalizability, and it does so by superposing object-level and relational information via the novel HD-attention mechanism. The reader's weakest assumption targets the second part, specifically the functional form of the attention score in Eq. 1. I agree that this is the least secure link. The paper provides no theoretical or empirical isolation showing that the attention matrix encodes the intended relation, and the default behavior for random inputs is a constant score of 1/2, which suggests the form is not inherently relational. The proposed concrete test would settle this by probing the learned attention matrix directly. If the mechanism fails the test, the explanation for any accuracy gains would need to be revised, though the architecture might still work for other reasons. The empirical support is mixed, but that is a separate concern already noted by the reader; the mechanism is the more fundamental issue for the paper's contribution. Since the reader's verdict is CONDITIONAL and my concern reinforces that condition, no change to the verdict is needed.","tokens_in":13313,"tokens_out":9561,"duration_ms":93993,"concrete_test":"On a trained RESOLVE model for the pairwise-ordering task (Section 7.1), extract the relation tensor R for a held-out batch of pairs. Compute the fraction of pairs with oi ≺ oj for which Rij > Rji, and the Spearman correlation between (Rij - Rji) and the true order distance. Also compare R to a control where object order is randomly permuted. If the attention scores are not systematically asymmetric in the correct direction (or if they are indistinguishable from a symmetric similarity), the mechanism is not doing the claimed relational work and the accuracy must be attributed to other components.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 defines Rij = cos(δ(hOi), δ(hOi ⊕ hOj)) and asserts without proof that this captures the relation between Oi and Oj, including asymmetric ordering and subtraction. Because the binarized sum sign(hOi+hOj) is dominated by the element with larger magnitude, the score measures how much Oj changes the sign pattern of Oi; for random Gaussian inputs the expected score is 1/2 (derivable from P(sign(X)=sign(X+Y))=3/4 for i.i.d. symmetric variables). A learned encoder can in principle shape this score, but nothing in the paper shows that the trained attention matrix actually encodes the intended relation rather than a generic pairwise similarity. The paper only evaluates the full model's final accuracy, so if the model solves the tasks via object-level features or the learned symbols, the central claim that HD-Attention provides the relational inductive bias is unsupported. This is load-bearing because the novelty (contribution 2) and the explanation for the accuracy gains both rest on this specific functional form.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RESOLVE, a neuro-vector symbolic architecture for relational reasoning. Objects are encoded into high-dimensional bipolar vectors via a learnable HD encoder; a novel HD-attention mechanism computes relation scores using cosine similarity between binarized object vectors and binarized bundles of pairs; the resulting mixed object representation is bound (Hadamard product) with learnable symbol vectors to superpose object-level and relational information. The architecture is evaluated on pairwise ordering, SET classification, MNIST-Math, object sorting, and math problem-solving, comparing against transformers, the Relational Abstractor, CoRelNet, PrediNet, MLP, and LEN. The paper claims higher accuracy, better generalizability, and lower computational overhead than state-of-the-art methods.","tokens_in":13519,"tokens_out":6746,"duration_ms":64232,"significance":"If the claims are substantiated, RESOLVE would be a useful contribution to relational reasoning, particularly for partially relational tasks where object and relational information must coexist. The release of code and detailed hyperparameters in the appendix is a strength. However, the evidence for the central claims is incomplete: the novel HD-attention mechanism is not analyzed or independently validated, and the empirical gains over baselines are mostly confined to the low-data regime with small margins that lack statistical support. The computational overhead comparison also uses mismatched dimensions, weakening the efficiency claim.","major_comments":[{"comment":"The derivation of the HD-attention score is incorrect. The paper states that because the HD objects are bipolar, their L2 norm is sqrt(D), allowing the cosine similarity to be replaced by ⟨δ(hOi), δ(hOi ⊕ hOj)⟩/D. However, the bundle hOi ⊕ hOj is the real-valued sum of two bipolar vectors; after applying δ, entries where hOi and hOj disagree become zero, so δ(hOi ⊕ hOj) is ternary, not bipolar, and its L2 norm is not sqrt(D) in general. More fundamentally, the paper provides no evidence that this score captures asymmetric relations such as ordering or subtraction: for random i.i.d. inputs the expected score is 1/2, and the paper does not show that training shapes the attention matrix into a meaningful relation. Since contribution 2 and the explanation for the accuracy gains rest on this mechanism, the authors should correct the derivation, provide a theoretical characterization, or analyze the learned attention matrix independently of end-task accuracy.","section":"§5, Eq. (1)"},{"comment":"The claim of 'better generalizability' and 'higher accuracy' is only partially supported. At the largest training size (10,000 samples), RESOLVE is behind Rel-Abstractor on all three tasks (Comparison 52.36 vs 52.46, Closest 66.84 vs 69.19, Place Value 98.68 vs 99.43). The reported averages (e.g., 41.86 vs 41.01 overall) are dominated by low-data performance, and no error bars or statistical significance tests are reported for these numbers. The conclusion that RESOLVE 'outperforms the state of the art' should be qualified to the small-data regime, or supported with variance estimates and significance tests.","section":"§7.4, Table 1"},{"comment":"The computational efficiency comparison is not apples-to-apples: HD-Attention operates in D=1024 dimensions while the self-attention baseline uses embedding sizes 32 and 64. The roofline metrics (π and β) therefore reflect different problem sizes, not the claimed advantage. The table also reports no direct latency or energy measurements. To support contribution 3, the comparison should match dimensionality or report end-to-end runtime and memory for the full models.","section":"§7.5, Table 2"}],"minor_comments":[{"comment":"The manuscript contains numerous typos and grammatical errors, e.g., 'an high-dimensional', 'have generated have generated', 'intereference', 'mechansim', and 'descriminative'; a careful proofread is needed.","section":"Throughout"},{"comment":"Section 7 lists LEN as a baseline, but no LEN results are reported in any experiment; Appendix A.1 also refers to 'LARS-VSA' without defining it.","section":"§7, Appendix A.1"},{"comment":"The figures do not show error bars or confidence intervals even though the appendix reports multiple runs (5 trials for pairwise order, 10 runs for sorting); without these, the accuracy differences are difficult to assess.","section":"Figures 8, 10, 12"},{"comment":"The HD-encoder equation hOi[j] = Σ_k Oi[k]·Bi[j-k] is not fully specified: the dimensions of Bi (RN×(D-F+1)) and the relationship between the convolution output length and D should be clarified.","section":"§5, Step 1"},{"comment":"The sentence 'It captures the dominant or relevant features of an object pair' is informal and should be replaced with a precise definition of what bundling is intended to encode and how it relates to the attention score.","section":"§5, bundling"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting problem and includes a commendable reproducibility package, but the central mechanism (HD-Attention) is not validated beyond end-task accuracy, and the headline claims are weaker than the text suggests once the largest training sizes are examined. I would encourage the editor to send the manuscript back for a revision that either provides a theoretical or probing-based analysis of the attention score, or tempers the claims to the low-data regime and adds significance testing. The missing LEN results and the dimension mismatch in the overhead comparison should also be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper is a serious attempt to bring vector symbolic processing into sequence-to-sequence relational reasoning, and the authors deserve credit for shipping code and evaluating on a reasonable spread of tasks. But the load-bearing novelty—the HD-attention score in Eq. 1—is never actually shown to capture relations, and the empirical advantage over the Relational Abstractor mostly evaporates at larger training sizes.\n\nWhat's new: the combination of learned symbols with high-dimensional bundling and binding inside an encoder-decoder is, as far as I know, not in the literature. Using a bipolar attention score computed over bundled hypervectors is also new, even if the ingredients (Hadamard binding, cosine similarity) are standard. The paper includes code and detailed hyperparameters, which is more than many submissions do.\n\nThe experiments are a real attempt to separate pure and partial relational tasks. The low-data regime results are consistently in RESOLVE's favor: it hits above 80% on SET with 600 samples, and the math task gains over the Abstractor are largest at 100 training examples. That's a useful signal.\n\nNow the soft spots, in order of weight.\n\nFirst, Eq. 1 is the core of the claim but it gets no independent analysis. For iid symmetric inputs, the expected score is 1/2 regardless of relation; the paper never shows that after training the attention matrix actually reflects ordering or subtraction rather than some generic pairwise affinity. This isn't a fatal flaw—the encoder can in principle shape the scores—but the paper's own explanation of why this works is hand-waving. A proper ablation or probe of the attention matrix would settle it.\n\nSecond, the scaling story is weaker than advertised. In Table 1, at 10,000 training samples RESOLVE is behind Rel-Abstractor on all three math tasks. The 'better generalizability' claim is really a small-sample advantage. That's worth stating honestly.\n\nThird, the efficiency claim rests on a roofline comparison that shows almost no difference in the reported numbers, and it compares only the attention module, not the full model. With D=1024 and low-dimensional baselines, the complexity argument is not obvious. Actual latency numbers would help.\n\nNone of this sinks the paper. It's an architectural proposal with an unverified mechanism, and the empirical evidence is mixed. I'd send it to reviewers, expecting them to push for a real analysis of the attention matrix and a better efficiency evaluation.","headline":"A creative VSA twist on relational reasoning whose central attention mechanism is unvalidated and whose efficiency gains don't show up in the data.","tokens_in":14021,"tokens_out":4351,"would_cite":false,"duration_ms":46458,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RESOLVE claims that a vector-symbolic layer superposing object and relational codes in high-dimensional bipolar space solves both pure and partial relational reasoning tasks more accurately and more generally than transformer and…","keywords":["vector symbolic architectures","hyperdimensional computing","relational reasoning","bipolar hypervectors","attention mechanism","sequence-to-sequence learning","abstract reasoning","mathematical problem solving"],"falsifier":"Ablate the learned symbols by replacing the learned symbolic hypervectors with random fixed bipolar vectors and retraining only the surrounding encoder and decoder. If accuracy on sorting or ordering stays high, the relational content is not coming from the learned symbols; if accuracy collapses, the symbols are load-bearing. A companion check is to verify on the pairwise-order task that the raw attention scores are order-consistent, i.e., $R_{ij} > R_{ji}$ whenever $o_i \\prec o_j$, on held-out pairs.","tokens_in":13123,"feed_emoji":"🧠","tokens_out":10741,"duration_ms":97255,"temperature":0.7,"pith_summary":"RESOLVE is a proposed neural architecture that aims to fix a known weakness of transformer-based reasoning models: object-level features and relational information are mixed in the same low-dimensional representation and interfere with each other. The paper's central claim is that if both kinds of information are stored as high-dimensional bipolar hypervectors and combined with bundling and binding (elementwise sum and Hadamard product), they can coexist with little interference, so one model handles purely relational tasks (sorting, pairwise ordering) and partially relational tasks (MNIST digit subtraction, math problem-solving) alike. The paper reports higher accuracy and better generalization than the Abstractor, transformer, CorelNet, and PrediNet baselines, and it reports that its bipolar attention-score computation is cheaper in time and memory than standard self-attention. If the claim holds, vector symbolic processing becomes a practical alternative to separating relational from object-level information in sequence-to-sequence reasoning models.","feed_headline":"Superposed high-dim codes beat transformers on relational tasks","feed_subtitle":"Vector-symbolic layer handles sorting, ordering, and math by keeping object and relation info in one hypervector.","key_machinery":"The load-bearing object is the HD-Attention score $R_{ij} = \\cos\\bigl(\\delta(h_{O_i}), \\delta(h_{O_i} \\oplus h_{O_j})\\bigr)$ computed in a bipolar high-dimensional space, together with bundling $\\oplus$ (coordinate-wise summation) and binding $\\otimes$ (Hadamard product). The score measures how much the coarse direction of object $i$'s hypervector changes when object $j$ is bundled into it; this directional change is what the paper treats as the relation between the two objects. The binding of the attention-mixed object hypervector with learned symbolic hypervectors is what lets object-level and relational content occupy the same vector with low interference, and the bipolar representation is what makes the score computable with fast sign and addition operations.","core_discovery":"The central discovery claimed is that a vector-symbolic layer can learn abstract relations while preserving object identity, instead of routing relational information through a separate abstractor module. The layer maps each input object to a high-dimensional bipolar vector (entries in $\\{-1,+1\\}$, dimension $D \\sim 10^3$) with a learnable convolution encoder, then computes attention scores $$R_{ij} = \\cos\\bigl(\\delta(h_{O_i}), \\delta(h_{O_i} \\oplus h_{O_j})\\bigr),$$ where $\\oplus$ is elementwise bundling and $\\delta$ binarizes each coordinate. These scores are softmax-normalized, used to mix the object hypervectors, and the mixed vector is bound (Hadamard product) with learned symbolic hypervectors. The paper's argument is that high dimensionality keeps the superposed object-level and relational codes quasi-orthogonal, so they do not destructively interfere, and that the bipolar form makes attention cheap. On this basis it claims better accuracy and few-sample generalization than the Abstractor and transformer baselines on sorting, pairwise ordering, SET classification, MNIST subtraction, and three math-problem datasets.","pith_inferences":["The attention score can be read as a directional-change measure: bundling object $j$ into object $i$ moves $i$'s bipolar hypervector, and the cosine drop is a proxy for how much $j$ alters $i$. This reading suggests the same layer could be tested on other asymmetric relations, such as spatial containment or causal precedence, without redesigning the score.","The quasi-orthogonality argument predicts graceful degradation as sequence length grows, because longer sequences place more bundled objects in superposition; this is testable by running RESOLVE on sorting inputs longer than the six-object training range.","Because the decoder receives object and relational information in one vector, the module may transfer to multimodal reasoning where input objects come from different modalities; the paper mentions multimodal learning only as future work, so this remains an extension, not a claim of the paper."],"forward_implications":["A single RESOLVE module can replace the abstractor in an encoder-decoder, because object and relational codes coexist in one high-dimensional vector rather than being separated into different pathways.","Bipolar attention scores lower the compute and memory cost of self-attention, as shown by the reported roofline bandwidth and peak-performance numbers.","Few-sample generalization improves: the paper reports above 80% accuracy on pairwise ordering with 210 training samples and on low-processed SET classification with 600 samples.","On the math-problem datasets, RESOLVE outperforms both the Abstractor and the transformer on average accuracy and is the strongest model in the 100-sample training regime."],"supporting_citations":[{"why":"Defines the Abstractor relational-bottleneck module and reports the pure-relational results that RESOLVE extends and compares against.","marker":"(Altabaa et al., 2023)"},{"why":"Supplies the transformer encoder-decoder backbone and self-attention baseline that RESOLVE modifies and outperforms.","marker":"(Vaswani et al., 2017)"},{"why":"Introduces the neuro-vector-symbolic approach to Raven's Progressive Matrices with bundling and binding, the immediate predecessor that required pre-engineered relations.","marker":"(Hersche et al., 2023)"},{"why":"Articulates the relational bottleneck as an inductive bias and the interference claim that RESOLVE uses high-dimensional superposition to avoid.","marker":"(Webb et al., 2024b)"},{"why":"Establishes hyperdimensional computing primitives: bundling, binding, and the holistic property of high-dimensional random vectors.","marker":"(Kanerva, 2009)"},{"why":"Provides the single-dimension convolution-based hyperdimensional encoder that RESOLVE uses as $\\phi_{HD}$.","marker":"(Mejri et al., 2024a)"},{"why":"Contributes the CoRelNet similarity-matrix model used as a baseline in the relational experiments.","marker":"(Kerg et al., 2022)"},{"why":"Supports the quasi-orthogonality and superposition-in-high-dimensions property relied on by the HD-Attention design.","marker":"(Menet et al., 2024)"},{"why":"Supplies the roofline model used to estimate memory-bandwidth and compute efficiency of HD-Attention.","marker":"(Ofenbeck et al., 2014)"},{"why":"Provides the math problem-solving dataset used for the sequence-to-sequence partially relational evaluation.","marker":"(Saxton et al., 2019)"}],"fun_headline_variants":["Hypervector fusion nails relational tasks, beats transformers","Object and relation codes coexist in one hypervector","Vector-symbolic attention does relational math with low latency","RESOLVE superposes object and relational features in bipolar space"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method depends on one unproven assumption: that comparing an object with the summed representation of itself and another object, after binarizing, is a trustworthy measure of the relation between the two objects. The paper does not verify this score on its own, only as part of the full model.","fun_headline_variants_meta":{"raw":{"variants":["Hypervector fusion nails relational tasks, beats transformers","Object and relation codes coexist in one hypervector","Vector-symbolic attention does relational math with low latency","RESOLVE superposes object and relational features in bipolar space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00047,"raw_usage":{"total_tokens":2364,"prompt_tokens":997,"completion_tokens":1367,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":1303}},"tokens_in":613,"tokens_out":1367,"duration_ms":10065,"temperature":1.0,"reasoning_tokens":1303,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:46:08.228641+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Ablate the learned symbols by replacing the learned symbolic hypervectors with random fixed bipolar vectors and retraining only the surrounding encoder and decoder. If accuracy on sorting or ordering stays high, the relational content is not coming from the learned symbols; if accuracy collapses, the symbols are load-bearing. A companion check is to verify on the pairwise-order task that the raw attention scores are order-consistent, i.e., $R_{ij} > R_{ji}$ whenever $o_i \\prec o_j$, on held-out pairs.","supporting_citations":[],"review_version":1}