{"id":"48ac4e0b-5c8b-40a0-976c-992d2ba20ebf","arxiv_id":"2508.01219","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"Reparameterizing layer weights in a learned orthonormal eigenbasis is claimed to improve ImageNet classification, cross-modal retrieval, and enable a faster backpropagation-free variant that surpasses standard backpropagation.","lead":"The paper introduces the Eigen Neural Network (ENN), which reparameterizes neural network weights in a shared learned orthonormal eigenbasis to enforce decorrelated weight dynamics. It claims state-of-the-art ImageNet accuracy and a backpropagation-free variant that is faster and more accurate than end-to-end backpropagation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Shared-eigenbasis reparameterization collapses weight-space dimension from O(Ln²) to O(n²+Ln); without an approximation theorem or a full weight-space basis, the ImageNet and BP-free claims rest on an unstated expressivity assumption.","rationale":"The reader's weakest assumption is exactly the expressivity of the shared eigenbasis, so I agree. I sharpen it from a vague risk to a concrete dimension count: with shared U,V, the hypothesis space dimension is O(n²+Ln), not O(Ln²). This is an internal capacity fact, not a disagreement with empirical consensus, so it is the right first thing to check. I do not move to REJECT because the corrupted full text prevents me from confirming that the paper actually uses the literal shared-SVD decomposition; it might use a full weight-space basis or a per-layer basis with a different 'shared' meaning. CONDITIONAL captures this: the paper should be accepted only if the missing expressivity/parameter-count analysis is supplied and passes the projection test. If the projection test shows large residual, the verdict should become REJECT. The BP-free-superiority claim is also extraordinary, but it is downstream: if the architecture cannot even represent standard networks, the comparison is moot. No ad hominem is intended; the issue is the stated architecture, not the authors.","tokens_in":23833,"tokens_out":10447,"duration_ms":139938,"concrete_test":"Locate the exact decomposition in the paper (likely §3). If W_l = U diag(s_l)V^T with U,V shared, compute the manifold dimension and compare with the unconstrained parameter count; then take a pretrained ResNet-50 and solve min_{U,V,{s_l}} Σ_l ||W_l − U diag(s_l)V^T||_F² (pad/truncate layers as needed). If the relative Frobenius residual exceeds 1% or fine-tuning the projected model loses more than 1% top-1 accuracy, the architecture cannot realize the functions the empirical claim requires. If the paper instead uses a full weight-space basis, count the basis parameters and check whether the 2× speedup claim survives.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing premise is that the layer-shared orthonormal eigenbasis does not shrink the function class. Taken literally, W_l = U diag(s_l) V^T with one learned U,V used by every layer forces all layers to share the same left and right singular vectors. For an L-layer MLP of width n the representable L-tuples form a manifold of dimension n(n−1)+Ln (two orthogonal-group directions plus L singular-value vectors), versus Ln² for unconstrained weights; at ImageNet-scale widths this is roughly a 50-fold parameter reduction, and layers of different shapes make the shared U,V ill-defined without padding. The abstract offers no theorem that ImageNet-trained weight matrices lie near this low-dimensional family, nor any truncation/approximation analysis. If the basis is instead a full orthonormal basis of the matrix space, then the learned basis itself carries ~(mn)² parameters and the claimed efficiency gain collapses. Either reading leaves the central capacity question unanswered, and all downstream claims (decorrelated dynamics, SOTA, BP-free superiority) depend on it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes the Eigen Neural Network (ENN), which reparameterizes each layer's weight matrix in a shared learned orthonormal eigenbasis, i.e., weights of the form U diag(s_l) V^T with common U and V across layers. It claims that this structure axiomatically enforces decorrelated, well-aligned weight dynamics and yields three headline results: consistent state-of-the-art accuracy on ImageNet, a new state-of-the-art on a cross-modal image-text retrieval benchmark, and a backpropagation-free variant (ENN-ell) that trains more than 2x faster than standard backpropagation while surpassing its final accuracy. The submitted full text is heavily corrupted and largely illegible; the readable portions contain no experimental tables, architecture equations, hyperparameters, or derivations that can be independently checked.","tokens_in":24021,"tokens_out":3446,"duration_ms":44292,"significance":"If the claims held, the paper would be significant: it would show that a purely architectural parameterization can replace regularization, that a low-dimensional shared eigenbasis retains enough expressivity for ImageNet-scale models, and that a local learning rule can beat end-to-end backpropagation in both speed and accuracy. The core idea is clearly stated and falsifiable in principle, and the abstract promises concrete, testable comparisons. However, the submitted evidence is far below the bar for such claims: no reproducible protocol, code, numerical results, or verifiable derivations are available in the reviewed text, so the significance cannot be assessed at this stage.","major_comments":[{"comment":"The abstract states that ENN 'consistently outperforms state-of-the-art methods on large-scale image classification benchmarks, including ImageNet' and sets 'a new benchmark in cross-modal image-text retrieval,' but the submitted text contains no results tables, model sizes, training schedules, baselines, or dataset names. Without these details, the central empirical claims are unverifiable.","section":"Abstract"},{"comment":"The parameterization W_l = U diag(s_l) V^T with a single learned pair U,V shared across all layers restricts the set of representable weight matrices to a low-dimensional family (about O(n^2 + Ln) parameters rather than O(Ln^2), and it is ill-defined for layers of different widths without padding). The manuscript provides no expressivity theorem or approximation bound showing that this family can realize ImageNet-competitive functions, so the capacity premise underlying the claimed gains is unsupported.","section":"Abstract"},{"comment":"The claim that ENN-ell surpasses end-to-end backpropagation requires a precise specification of the local update rule, the per-layer loss or target information, and the parallelization scheme. None of these ingredients is readable in the submitted text, so the 2x speedup and accuracy comparisons cannot be checked or reproduced.","section":"Abstract"},{"comment":"Most of the body is corrupted or illegible, including the equations and tables. Because the core derivations and experimental evidence cannot be inspected, I cannot verify any of the paper's formal or empirical statements; this is a blocking issue independent of the scientific content.","section":"Full text (as received)"}],"minor_comments":[{"comment":"The phrase 'Deep Neural Networks(DNN)' should be 'Deep Neural Networks (DNN)' with a space; please run a formatting pass over the text.","section":"Abstract"},{"comment":"The abstract uses 'we introduced' where 'we introduce' would be more consistent with the submission's timeframe.","section":"Abstract"},{"comment":"The term 'new benchmark' is ambiguous; specify whether it is a new dataset, a new evaluation protocol, or simply a new state-of-the-art result on an existing benchmark.","section":"Abstract"},{"comment":"Many section headers and table captions appear garbled; please regenerate the manuscript from the LaTeX source so that all symbols and tables render correctly.","section":"Full text (as received)"}],"recommendation":"reject","confidential_remarks":"I see no evidence of misconduct; the issue is purely that the submitted text does not support the claims as written. If a readable manuscript with full experimental details and an expressivity analysis is provided, I would be willing to review it again."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — quick take on arXiv:2508.01219. The full text we received is a corrupted extraction, so I can only judge the abstract. What's genuinely new: reparameterizing every layer's weights in a shared learned orthonormal eigenbasis, enforcing decorrelation axiomatically rather than through a regularizer, and a local BP-free variant with parallel training. That's a clean idea, and if the ImageNet and retrieval numbers hold, it would be a real advance.\n\nBut the abstract makes claims it doesn't back with a single number, and the architecture as stated has a load-bearing expressivity problem. If W_l = U diag(s_l) V^T with one U,V shared across layers, the representable weight tuples live on a manifold of dimension roughly n^2 + Ln, versus Ln^2 for free weights. For ImageNet-scale widths that's a huge restriction, and nothing in the abstract says why trained ImageNet weights should sit near that family. If instead the \"eigenbasis\" spans the full matrix space, then the basis itself carries ~(mn)^2 parameters and the efficiency story collapses. Either way the central capacity question is unanswered, and both the SOTA claim and the BP-free-beats-BP claim depend on it. The stress-test note gets this right.\n\nWhat's missing from what we can see: any experimental details, hyperparameters, comparisons, or a theorem about expressivity. Maybe the full paper supplies all of that; the corruption prevents me from checking. The authors should be given a chance to show it.\n\nI'd send this to reviewers. The idea is novel and important enough to spend referee time on, even though my prior on the empirical claims is low. A serious referee should first ask: does the shared basis shrink the function class, and is there any approximation or truncation analysis? Then: are the ImageNet numbers reproducible with standard training budgets? If the paper answers the first question cleanly, it's a strong contribution.\n\nFor the reading group: maybe, as a discussion piece about weight reparameterization. I wouldn't cite it until I can see the full text and the expressivity analysis.","headline":"A bold architectural idea with strong empirical claims; the abstract alone cannot carry them, and the shared-basis expressivity question is the first thing a referee must press.","tokens_in":24553,"tokens_out":1886,"would_cite":false,"duration_ms":23260,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","15A18"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that rewriting every layer's weights in a single learned orthonormal basis forces the weights to be decorrelated and aligned by construction, improving large-scale image classification, cross-modal retrieval, and…","keywords":["Eigen Neural Network","orthonormal eigenbasis","backpropagation-free learning","local learning","weight decorrelation","ImageNet classification","cross-modal retrieval","representation learning"],"falsifier":"Run ENN and a matched standard network of the same width and depth on a task engineered so that the optimal first-layer and last-layer weight matrices have incompatible singular vectors; if ENN's shared basis cannot match the standard network's accuracy, the expressiveness assumption fails.","tokens_in":23638,"feed_emoji":"🧠","tokens_out":5286,"duration_ms":64644,"temperature":0.7,"pith_summary":"Deep neural networks trained by backpropagation tend to develop disordered, correlated weight matrices, and the paper attributes part of their training difficulty and feature noise to that disorder. The Eigen Neural Network (ENN) addresses this by writing every layer's weight matrix in one layer-shared, learned orthonormal eigenbasis, so decorrelation and alignment of weights hold by construction rather than by regularization. The paper reports that ENN with standard backpropagation outperforms existing methods on large-scale image classification, including ImageNet, and that its representations set a new state-of-the-art result in cross-modal image-text retrieval. It also introduces ENN-$\\ell$, a backpropagation-free local learning variant, which it reports trains over twice as fast through parallelism while exceeding the accuracy of end-to-end backpropagation. If these results hold, architectural structure alone can replace weight regularization and make local learning competitive with global training.","feed_headline":"One shared eigenbasis beats standard training in accuracy and speed","feed_subtitle":"ENN writes every layer's weights in one learned orthonormal basis, makes decorrelation automatic, and improves both accuracy and training…","key_machinery":"The central object is the layer-shared, learned orthonormal eigenbasis. An orthonormal basis is a set of mutually perpendicular unit directions; calling it an eigenbasis signals that each weight matrix is diagonal in this basis. All layers use the same learned basis, while each layer has its own coefficients, so the network's inductive bias is that useful weight directions align across layers. This object carries the argument because it transforms decorrelation from a regularized soft constraint into a hard structural fact: the basis is orthogonal by construction, and weights expressed in it cannot be arbitrarily aligned. The basis is learned during training, so the network can adapt the shared directions to the data.","core_discovery":"On the paper's own terms, the central discovery is that the disordered weight structure produced by gradient-based optimization is not an unavoidable byproduct of training but a representational choice that can be redesigned. By reparameterizing each layer's weights in a shared orthonormal basis, ENN guarantees that weight directions are mutually orthogonal and aligned, and only layer-specific coefficients vary. This structural guarantee produces more discriminative features, which the paper demonstrates by improved ImageNet accuracy and a new state-of-the-art cross-modal retrieval result. The same structure removes the need for end-to-end gradient flow: in ENN-$\\ell$, each layer can be trained locally in parallel, giving over 2x training speedup and, the paper reports, accuracy above standard backpropagation. The intended lesson is that the geometry of the parameter space, not just the optimization algorithm, determines how well and how fast deep networks learn.","pith_inferences":["A direct extension the paper leaves untested: if the shared basis is the active ingredient, the learned basis itself should transfer across datasets as a fixed representation, so one could test whether the basis improves generalization even when the classifier head is retrained.","The same reparameterization could in principle be applied to attention or recurrent layers, where weight correlation also affects training, though the paper does not report such experiments.","The reported speedup suggests that local layer-wise training can be competitive when weights are preconditioned by structure, so ENN-$\\ell$ could be explored as a pretraining strategy for very deep networks.","One could check whether the structural decorrelation guarantee flattens the loss landscape enough to reduce sensitivity to initialization and learning rate, which would be a separate claim from the accuracy results."],"forward_implications":["Standard backpropagation on ENN outperforms comparable baselines on large-scale image classification, including ImageNet.","ENN representations transfer to cross-modal image-text retrieval and set a new state-of-the-art result on that task.","The backpropagation-free local learning variant ENN-$\\ell$ trains more than twice as fast through layer parallelism.","ENN-$\\ell$ reaches higher accuracy than end-to-end backpropagation on the tested tasks.","Weight decorrelation and alignment, normally encouraged by regularization, become structural properties in ENN rather than training targets."],"supporting_citations":[],"fun_headline_variants":["Eigenbasis network: accuracy up, training 2x faster","Shared orthonormal basis boosts accuracy and slashes training time","One eigenbasis replaces disordered weights: ENN wins on both fronts","Decorrelation by design: ENN beats standard training in speed and accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one learned orthonormal basis shared by every layer is expressive enough to represent the useful weight matrices of all layers; if different layers need fundamentally different directions, the architecture cannot match an unconstrained network.","fun_headline_variants_meta":{"raw":{"variants":["Eigenbasis network: accuracy up, training 2x faster","Shared orthonormal basis boosts accuracy and slashes training time","One eigenbasis replaces disordered weights: ENN wins on both fronts","Decorrelation by design: ENN beats standard training in speed and accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000741,"raw_usage":{"total_tokens":3314,"prompt_tokens":960,"completion_tokens":2354,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":2279}},"tokens_in":576,"tokens_out":2354,"duration_ms":20614,"temperature":1.0,"reasoning_tokens":2279,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:43:23.203625+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ENN and a matched standard network of the same width and depth on a task engineered so that the optimal first-layer and last-layer weight matrices have incompatible singular vectors; if ENN's shared basis cannot match the standard network's accuracy, the expressiveness assumption fails.","supporting_citations":[],"review_version":1}