{"id":"c1dd9b89-6d8c-477e-abb1-c9eb677028e8","arxiv_id":"2608.10416","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A proposed 'Riemann GeoResolver' framework for inverse-distance attention in hyperbolic and spherical geometry, whose central PL inequality rests on an invalid derivative computation.","lead":"A theory paper claims inverse-distance attention beats softmax with exponential optimization advantages, then extends the idea into hyperbolic and spherical spaces with ten modules. The core Euclidean optimization theorem contains a false derivative and the non-Euclidean theorems are only sketched, so the framework does not hold as stated.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2.6's global PL inequality is false: the 1D two-key IDA loss has stationary points with positive loss where the gradient vanishes, and the proof's W'_1(0)=0 expansion is incorrect.","rationale":"The reader's verdict rejects the paper, and I agree. The single load-bearing issue is Theorem 2.6, because the abstract and introduction advertise the exponential PL ratio as the paper's main theoretical result, and the non-Euclidean theorems explicitly reuse the same expansion. The reader's weakest_assumption correctly identifies W'_1(0)=0 as the false step in the proof; my independent differentiation confirms the nonzero value. I also find a stronger refutation: the 1D two-key loss has stationary points at the roots of q^2-Δq-ε=0, and at these points the gradient of the loss is zero while the loss exceeds its infimum for generic target values. A global PL inequality with positive constant cannot hold at such points, so Theorem 2.6 is false as stated, not just mis-derived. I do not rely on the 'absence of spurious local minima' part to make this point; those stationary points are local maxima rather than local minima, and the PL-violation argument does not require them to be minima. The proof error is internal inconsistency, not a disagreement with prior consensus: the series for B(q) itself contains the linear term 2Δq/C^2, so the later assertion W'_1(0)=0 is an algebraic mistake. A machine-checked or symbolic re-derivation of the derivative would settle the issue immediately. Since the central claim fails and the non-Euclidean theorems inherit the same expansion, the REJECT verdict stands; no change to the reader's verdict is needed.","tokens_in":22231,"tokens_out":16639,"duration_ms":145051,"concrete_test":"Set Δ=1, ε=0.01, v1=1, v2=0, y=0.5 in the 1D two-key loss of Section 2.6. Compute q_±=(1±√(1.04))/2. At each q_±, evaluate W1(q_±), L(q_±)=(W1(q_±)-0.5)^2, and L'(q_±)=2(W1(q_±)-0.5)W1'(q_±). Verify W1'(q_±)=0, L(q_±)>0, and that the infimum of L over q is 0 (achieved where W1=0.5). This contradicts any global PL inequality with µ>0. As a second check, compute W1'(0) by direct symbolic differentiation and compare with the paper's asserted W'_1(0)=0; the nonzero value confirms the expansion error in Section 2.6.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorem 2.6, the paper's central claim, asserts a global PL inequality for the 1D two-key IDA loss with µ_IDA = Θ(ε^2/Δ^4), which is then used to claim an exponential optimization advantage over softmax and linear convergence. The proof is invalid in two linked ways. First, the local expansion of W1(q) around q=0 in Section 2.6 drops the linear term of B(q). Direct differentiation gives W'_1(0) = -A(0)B'(0)/(A(0)+B(0))^2 = -2Δ/(ε(Δ^2+ε)^2(1/ε+1/(Δ^2+ε))^2), which is nonzero for Δ>0 because B'(0)=2Δ/(Δ^2+ε)^2. Second, and more decisively, W'_1(q) vanishes at the two roots of q^2-Δq-ε=0. At these points L'(q)=0. For a generic instance such as v1=1, v2=0, y=1/2, the loss L(q)=(W1(q)-1/2)^2 satisfies L(q)>0 at both roots while inf_q L(q)=0, so the global PL inequality 2µ(L-L*) ≤ |L'|^2 fails at these stationary points for every µ>0. Hence the theorem's conclusion is false, not merely its constant. The non-Euclidean analogues (Theorems 5.3 and 8.1) reuse the same expansion, so the hyperbolic and spherical PL claims are unsupported as well.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes inverse-distance attention (IDA) as an alternative to softmax attention and claims three Euclidean theorems: circuit separation with O(1) versus Ω((log n)^2) resources, a Polyak–Łojasiewicz inequality with an exponentially larger constant than softmax, and a width-independent effective rank bound preventing noise memorization. It then extends the framework to hyperbolic and spherical geometries, packaging the results into a ten-module architecture (HIDA, HCC, HyperGate, SIDA, DMG, GSR). The Euclidean part is claimed to be proved in full; the non-Euclidean results are stated as following by analogous arguments.","tokens_in":22655,"tokens_out":7876,"duration_ms":68256,"significance":"If correct, the claimed exponential gap in PL constants and the width-independent capacity bounds would be significant for attention optimization and generalization. The manuscript contains some sound elementary observations: the exact-retrieval limit in Lemma 2.1 is straightforward, the softmax lower bound in Theorem 2.2 is a valid and simple calculation for the constructed orthonormal instance, and the complexity counts for FP-HIDA and L-HIDA are checkable. However, the central optimization theorem is false as stated, and the effective-rank claim is internally inconsistent; these are load-bearing results on which the abstract, the summary table, and the non-Euclidean extensions all rest. The paper therefore does not currently establish its main claims.","major_comments":[{"comment":"The proof of the IDA PL inequality is invalid. The expansion asserts W'_1(0)=0, but direct differentiation of W_1(q)=A(q)/(A(q)+B(q)) with A(q)=(q^2+ε)^{-1} and B(q)=((q-Δ)^2+ε)^{-1} gives W'_1(0)=-A(0)B'(0)/(A(0)+B(0))^2 = -2Δ/(ε(Δ^2+ε)^2(1/ε+1/(Δ^2+ε))^2), which is nonzero for Δ>0. The local quadratic behavior L(q)-L(0)≍q^2 therefore does not hold; the leading term is linear. More decisively, W'_1(q)=0 at the two roots of q^2-Δq-ε=0, and for the generic instance v1=1, v2=0, y=1/2 the loss L(q)=(W_1(q)-1/2)^2 is positive at both stationary points while inf_q L(q)=0. Hence the global PL inequality 2μ(L-L*)≤|L'|^2 fails at those stationary points for every μ>0. This invalidates the stated constant μ_IDA=Θ(ε^2/Δ^4), the ratio μ_IDA/μ_soft, the linear-convergence claim, and Corollary 2.7's absence-of-spurious-minima statement. The softmax PL constant is also not a global PL constant, since for the same 1D two-key loss the gradient tends to zero as q→±∞ while L-L* tends to 1/4.","section":"Section 2.6, Theorem 2.6"},{"comment":"The lemma is internally contradictory. The statement claims ∂²L/∂δ²|₀=Θ(1) for IDA, independent of n, but the proof computes W''₁₁(0)≈-2(n-1)/ρ² and then concludes L''(0)=Θ(W''₁₁(0)), which is Θ(n) in the stated regime. The subsequent sentence asserts Θ(1) Hessian spread, but that does not repair the contradiction in the second-derivative magnitude. Since Corollary 2.7 explicitly invokes Lemma 2 for the Θ(1) Hessian spread and saddle-escape claim, this part of the optimization story is unsupported.","section":"Section 2.5, Lemma 2.5"},{"comment":"The refined effective-rank bound is arithmetically inconsistent with the proof's own computation. The proof derives eff-rank(K)≤n/(1+ε²(n-1)/d_min⁴), which for small ε is close to n, while the theorem claims eff-rank(K)≤1+nε²/d_min⁴. For a concrete example with n=100, ε=0.01, d_min=1, the stated bound is roughly 1.01, whereas the displayed formula gives roughly 99. The refined bound cannot follow from the same effective-rank expression, so the width-independent capacity claim, and with it the noise-memorization contrast in Theorem 2.10, lacks a valid proof.","section":"Section 2.7, Theorem 2.8"},{"comment":"The non-Euclidean PL claims are not independent results: both proofs reuse the same expansion W'_1(0)=0 from Theorem 2.6. In the spherical case the weight W_1(t) has exactly the same functional form with Δ=θ, so the nonzero derivative at t=0 applies verbatim and the claimed PL constant Θ(ε²/θ⁴) is unsupported. In the hyperbolic case the same flawed step is imported through the 'same expansion as Theorem 2' argument. The paper's stated limitation that non-Euclidean theorems are proved by analogous arguments is therefore not a harmless presentation choice; it propagates the central error to the entire Part II.","section":"Theorems 5.3 and 8.1"}],"minor_comments":[{"comment":"Reference [13] (Ge, 2026, 'Under review') is listed but never cited in the text. Since it appears to be the author's own companion manuscript, it should either be cited where the Euclidean results are used or removed.","section":"References"},{"comment":"The notation for the smoothing constant is inconsistent: the text uses both ε and ϵ, sometimes within the same theorem statement, as in Theorem 2.8. Please standardize.","section":"Section 2.7 and throughout"},{"comment":"Step 4 of the proof states that 'the number of non-negligible distance scales is O(log n)' without a precise definition or proof; either supply the missing argument or state the clustering assumption in a way that makes this step checkable.","section":"Section 2.4, Lemma 2.4"},{"comment":"Several cross-references are wrong: Section 5 refers to 'Lemma 0' and Section 11.4 refers to 'Proposition 2', but no Lemma 0 or Proposition 2 exists in the paper. These should be corrected to the actual lemma and proposition numbers.","section":"Sections 5 and 11.4"},{"comment":"The supermartingale argument for the adaptive threshold regret bound is only sketched; the claim that the trigger condition creates negative drift and yields E[S_T]=O(log T) needs a formal proof, especially regarding the dependence of σ_t on the data.","section":"Section 9, Lemma 9.1"}],"recommendation":"reject","confidential_remarks":"The central Euclidean PL theorem is false, the effective-rank refinement is inconsistent with its own proof, and the non-Euclidean extensions inherit the same error. The paper also has no experiments, and several non-Euclidean 'theorems' are presented as sketches or by analogy. I do not see a localized fix within the current scope; the main claims would need to be substantially reworked. In addition, the uncited companion reference [13] should be checked for novelty disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take on 2608.10416. The headline is: the Euclidean core is broken. Theorem 2.6's PL inequality does not hold. The proof claims W1'(0)=0 for the IDA weight W1(q)=A/(A+B), but direct differentiation gives a nonzero value (the linear term in B(q) is dropped). More decisively, the 1D two-key loss has stationary points with positive loss (e.g., v1=1, v2=0, y=1/2, roots of q^2 - Δq - ε = 0), so the global PL inequality fails for every μ>0. The exponential PL advantage and the no-spurious-minima corollary fall with it. The non-Euclidean PL claims (Theorems 5.3, 8.1) reuse the same expansion, so they are unsupported as well.\n\nWhat is good: the paper is clearly written, honest about being theoretical, and has a sensible limitations section. The ten-module framework is new relative to the cited prior work, and some of the complexity claims (FP-HIDA, L-HIDA, C-HIDA) are straightforward and likely correct. The relationship to Shepard's method and McCarter's inverse-distance attention is acknowledged, though McCarter already proposed the kernel.\n\nSoft spots, in proportion: Lemma 2.5 states Θ(1) curvature but its own proof derives Θ(n), and the table in Section 3 misreports this. The refined effective rank bound in Theorem 2.8 does not follow from the calculation given and looks wrong for n=2. The non-Euclidean theorems are all 'analogous arguments' to the Euclidean theorems, which are themselves false, so the extension is unsupported. The paper also cites its own companion [13] for the Euclidean base; until that is public, there is a circularity concern.\n\nThis is a genuinely false main theorem, not a minor gap. A serious referee would catch it quickly, but I don't think the paper in this state deserves peer-review time. Desk reject, with an invitation to resubmit after fixing the derivative computation and re-examining the PL claims. If the author corrects that, the framework might be worth a second look. Would I cite it? No. Reading group? No.","headline":"The paper's central PL theorem is false due to a bad derivative computation, so the exponential optimization advantage and the non-Euclidean extensions built on it are unsupported; desk reject unless substantially revised.","tokens_in":23137,"tokens_out":4820,"would_cite":false,"duration_ms":38630,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that inverse-distance attention beats softmax in expressiveness, optimization, and generalization, and extends the claim to hyperbolic and spherical geometries.","keywords":["inverse-distance attention","Polyak-Lojasiewicz inequality","softmax attention","effective rank","hyperbolic geometry","spherical routing","circuit complexity","attention optimization"],"falsifier":"For two keys at $0$ and $\\Delta$, differentiate $W_1(q) = \\frac{(q^2+\\varepsilon)^{-1}}{(q^2+\\varepsilon)^{-1} + ((q-\\Delta)^2+\\varepsilon)^{-1}}$ at $q=0$. Direct computation gives $W'_1(0) = -\\frac{2\\Delta}{\\varepsilon(\\Delta^2+\\varepsilon)^2(\\varepsilon^{-1}+(\\Delta^2+\\varepsilon)^{-1})^2}$, which is nonzero for $\\Delta>0$. Finding this nonzero value refutes the expansion after the definitions of $A(q)$ and $B(q)$ in Section 2.6 and, with it, the stated PL constant and the no-spurious-minima corollary.","tokens_in":1971,"feed_emoji":"📐","tokens_out":4641,"duration_ms":97402,"temperature":0.7,"pith_summary":"This paper tries to prove that inverse-distance attention—which weights tokens by the inverse of squared distance rather than by softmax over inner products—is a fundamentally stronger attention kernel than softmax. In Euclidean space, it claims three advantages: exact retrieval with only $O(1)$ width where softmax needs $\\Omega((\\log n)^2)$; a Polyak–Lojasiewicz constant exponentially larger than softmax's for separated keys; and a width-independent effective rank that prevents memorizing arbitrary noisy labels. The second half extends the same inverse-distance principle to hyperbolic geodesic distance for storage and spherical geodesic distance for routing, yielding a ten-module framework with complexity, regret, and routing-quality bounds. If the Euclidean theorems hold, the choice of kernel—not the surrounding architecture—is the source of these guarantees.","feed_headline":"Inverse-distance attention promises provable wins over softmax","feed_subtitle":"Three theorems promise exact retrieval, fast convergence, and noise resistance; the proof hinges on a derivative.","key_machinery":"The object carrying the argument is the inverse-distance attention kernel $W_{ij} = (d(q_i,k_j)^2 + \\varepsilon)^{-1} / \\sum_m (d(q_i,k_m)^2 + \\varepsilon)^{-1}$, where $d$ is Euclidean, hyperbolic geodesic, or spherical geodesic distance. In the one-dimensional two-key loss, the proof expands the target weight $W_1(q)$ around $q=0$ at the exact-match key; the assumption $W'_1(0)=0$ produces the quadratic leading term from which the PL constant $\\Theta(\\varepsilon^2/\\Delta^4)$ is extracted. The same expansion, with $\\Delta$ replaced by a hyperbolic distance $\\Delta_H$ or a spherical angle $\\theta$, is reused for the non-Euclidean PL theorems.","core_discovery":"The paper's central claim is that replacing the softmax kernel with an inverse-distance kernel turns attention into an exact retrieval mechanism with favorable optimization geometry. In the Euclidean prototype, Resolver, the three core theorems assert: (1) circuit separation—IDA retrieves an exact match with $O(1)$ resources while any softmax architecture needs $\\Omega((\\log n)^2)$ width; (2) a Polyak–Lojasiewicz inequality for the one-dimensional two-key loss with $\\mu_{\\mathrm{IDA}} = \\Theta(\\varepsilon^2/\\Delta^4)$ versus $\\mu_{\\mathrm{soft}} = \\Theta(e^{-\\Delta^2/\\sqrt{d}}\\varepsilon^2/\\Delta^2)$, implying linear convergence, $\\Theta(1)$ Hessian spread, and no spurious local minima; and (3) a width-independent effective-rank bound $\\mathrm{eff\\text{-}rank}(K) \\le 1 + n\\varepsilon^2/d_{\\min}^4$, so IDA limits test error to $O(\\eta^2)$ under symmetric label noise while softmax memorizes arbitrary labels once the hidden dimension reaches $n$. The non-Euclidean part transfers these results to hyperbolic and spherical geodesic distances using the same inverse-distance kernel and analogous expansions, and adds modules for compression, gating, dynamic memory, and sparse routing.","pith_inferences":["Editorial extension: The expressiveness and effective-rank results in Theorems 1 and 3 do not rely on the $W'_1(0)=0$ expansion; if the PL claim fails, those two advantages may still stand on their own.","Because the hyperbolic and spherical PL theorems are proved by analogous two-key expansions, the correctness of the non-Euclidean optimization claims is coupled to the Euclidean expansion; a fix or refutation of the Euclidean proof automatically carries over to the non-Euclidean settings.","A testable extension is a synthetic two-key optimization with small $\\varepsilon$ and separated keys: the paper predicts linear convergence with constant $\\Theta(\\varepsilon^2/\\Delta^4)$, which is directly measurable without building a full attention architecture."],"forward_implications":["Exact-match retrieval would require only $O(1)$ width for IDA, whereas softmax needs $\\Omega((\\log n)^2)$ width to approximate the same behavior.","Under the two-key loss, gradient descent on IDA would converge linearly with rate constant $\\Theta(\\varepsilon^2/\\Delta^4)$, exponentially larger than softmax's constant for well-separated keys, and every stationary point would be either a global minimum or a strict saddle.","IDA's effective rank would stay bounded independent of hidden width, so under symmetric label noise the test error would be $O(\\eta^2)$ instead of softmax's memorization catastrophe at $d_h \\ge n$.","Replacing Euclidean distance with hyperbolic distance for storage and spherical distance for routing would preserve the exact-retrieval and PL-style guarantees, with per-token complexity ranging from $\\Theta(n^2)$ to $\\Theta(1)$ depending on the HIDA variant.","Dynamic prototype allocation would achieve $O(\\log T)$ regret and sparse spherical routing would have communication cost independent of batch size."],"supporting_citations":[{"why":"Defines the softmax attention baseline whose expressiveness, optimization, and memorization behavior the paper contrasts with IDA.","marker":"[1]"},{"why":"Prior inverse-distance weighting attention; the paper positions its QKV-theoretic analysis and non-Euclidean extension against it.","marker":"[12]"},{"why":"Supplies the Polyak--Lojasiewicz framework used to derive linear convergence for the two-key IDA loss.","marker":"[57]"},{"why":"Introduces Poincaré-ball hyperbolic embeddings used as the storage geometry for the HIDA operators.","marker":"[20]"},{"why":"Proposes hyperbolic attention networks, the baseline the paper's HIDA family extends with inverse-distance kernels.","marker":"[24]"},{"why":"Provides the Nyström approximation analysis used in the L-HIDA linear-complexity error bound.","marker":"[63]"},{"why":"Defines sparsely-gated mixture-of-experts routing, the baseline for GSR's communication complexity comparison.","marker":"[40]"}],"fun_headline_variants":["Inverse-distance attention: exact retrieval, proven convergence","Riemann GeoResolver turns attention into exact retrieval","O(1) retrieval: inverse-distance attention beats softmax","From Euclidean to non-Euclidean: attention with guarantees"],"cache_read_input_tokens":25088,"weakest_assumption_plain":"Theorem 2.6 rests on the claim that the inverse-distance weight $W_1(q)$ has zero first derivative at the exact-match query $q=0$; direct differentiation of the two-key weight gives a nonzero value for separated keys, and without $W'_1(0)=0$ the quadratic expansion and the $\\Theta(\\varepsilon^2/\\Delta^4)$ PL constant do not follow.","fun_headline_variants_meta":{"raw":{"variants":["Inverse-distance attention: exact retrieval, proven convergence","Riemann GeoResolver turns attention into exact retrieval","O(1) retrieval: inverse-distance attention beats softmax","From Euclidean to non-Euclidean: attention with guarantees"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000818,"raw_usage":{"total_tokens":3692,"prompt_tokens":1162,"completion_tokens":2530,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":778,"completion_tokens_details":{"reasoning_tokens":2465}},"tokens_in":778,"tokens_out":2530,"duration_ms":16824,"temperature":1.0,"reasoning_tokens":2465,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:22:56.429458+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For two keys at $0$ and $\\Delta$, differentiate $W_1(q) = \\frac{(q^2+\\varepsilon)^{-1}}{(q^2+\\varepsilon)^{-1} + ((q-\\Delta)^2+\\varepsilon)^{-1}}$ at $q=0$. Direct computation gives $W'_1(0) = -\\frac{2\\Delta}{\\varepsilon(\\Delta^2+\\varepsilon)^2(\\varepsilon^{-1}+(\\Delta^2+\\varepsilon)^{-1})^2}$, which is nonzero for $\\Delta>0$. Finding this nonzero value refutes the expansion after the definitions of $A(q)$ and $B(q)$ in Section 2.6 and, with it, the stated PL constant and the no-spurious-minima corollary.","supporting_citations":[{"cited_title":"N., Kaiser, L., and Polosukhin, I","cited_arxiv_id":null,"evidence_quote":"Defines the softmax attention baseline whose expressiveness, optimization, and memorization behavior the paper contrasts with IDA."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Polyak--Lojasiewicz framework used to derive linear convergence for the two-key IDA loss."},{"cited_title":"and Kiela, D","cited_arxiv_id":null,"evidence_quote":"Introduces Poincaré-ball hyperbolic embeddings used as the storage geometry for the HIDA operators."},{"cited_title":"M., Battaglia, P., Bapst, V., Raposo, D., Santoro, A., and de Freitas, N","cited_arxiv_id":null,"evidence_quote":"Proposes hyperbolic attention networks, the baseline the paper's HIDA family extends with inverse-distance kernels."},{"cited_title":"and Mahoney, M","cited_arxiv_id":null,"evidence_quote":"Provides the Nyström approximation analysis used in the L-HIDA linear-complexity error bound."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines sparsely-gated mixture-of-experts routing, the baseline for GSR's communication complexity comparison."}],"review_version":1}