{"id":"1611d612-7869-4423-8c3b-ff2f4006c50f","arxiv_id":"2511.00044","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ReLaX-Net cycles a small set of fixed weight matrices to deepen physical neural networks, but controlled experiments show a single repeated large layer is the best use of a fixed parameter budget.","lead":"This paper proposes a way to make physical neural networks act deeper by rapidly switching among a few fixed weight layers instead of changing the weights themselves. The idea could help compact optical or spintronic hardware get more computation per parameter, though the paper's own tests show the gain mostly comes from repeating one large layer.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equal-parameter scaling claim is contradicted by the paper's own fixed-budget experiment: at fixed total weights, the wide LW=1 RNN wins (§4.3, Fig. 8).","rationale":"I focused on the fixed-parameter contradiction rather than the hardware switching overhead because it is internal to the paper's evidence: no assumption about DMD/fiber switch latency is needed to see the tension. The reader's formal weakest_assumption was the hardware premise, but the reader's rationale already flags the Fig. 8/Table 1 problem, so this is a partial agreement. The issue is fixable by restating the central claim as a layer-width-cap benefit, adding error bars/seeds, and releasing code; it is not a reason to reject the architecture entirely. I also noticed the separate abstract passage near the top refers to TIDAL-Net rather than ReLaX-Net; that is a manuscript-hygiene problem and does not change the technical verdict. Because the paper already deserves conditional acceptance subject to these revisions, the reader's verdict stands.","tokens_in":26172,"tokens_out":7686,"duration_ms":87861,"concrete_test":"Re-run the §4.3 fixed-budget protocol on SVHN with exact total-parameter matching (adjust H to make hidden weights+biases equal to, say, 33,000 for every LW) and at least 5 seeds per cell, sweeping LW=1..8 and LT∈{1,4,8,12}; report mean±std test error for every cell, plus a same-budget feedforward DNN baseline with unrolled LT full-width layers. If LW=1 remains best at every budget, the equal-parameter scaling claim should be removed or qualified. If some LW>1 configuration wins, that configuration is the evidence needed to retain the claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assertion is that ReLaX-Net 'exceed[s] the performance of equivalent traditional RNNs or DNNs with the same number of parameters' (abstract). The controlled test designed for exactly this question is §4.3: total hidden parameters are held near 33k while per-layer width is traded against LW (Table 1, Fig. 8). The reported result is that LW=1, LT>1 — the standard single-weight RNN limit — has the best test error. Thus at equal parameter count, the wide stateless RNN beats every switched multi-layer configuration in the tested sweep. The favorable-scaling claim is therefore not supported by the paper's own controlled experiment; Figs. 5–6 let the parameter count grow with LW, so they conflate additional parameters with the architectural benefit. There is also no same-parameter DNN baseline anywhere. The defensible, narrower claim is that under a fixed per-layer width cap, a few periodically switched distinct layers improve over a same-width RNN, and that repetition at fixed LW helps (Fig. 7). That may still be useful for PNNs, but it is not the equal-parameter scaling advertised.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReLaX-Net, a physical-neural-network architecture in which a small set of L_W fixed trainable weight matrices is periodically switched over L_T time steps, interpolating between a stateless RNN (L_W=1) and a fully unrolled DNN (L_W=L_T). The claimed advantage is that only fast switches need to operate at forward-pass speed, while all weight parameters can be reconfigured slowly. The authors validate the idea numerically on SVHN image classification and Shakespeare character-level next-token prediction, with supporting experiments on oscillator-based activations and MNIST in the appendices. The central advertised claim is favorable scaling: ReLaX-Nets 'exceed the performance of equivalent traditional RNNs or DNNs with the same number of parameters' (abstract).","tokens_in":26444,"tokens_out":3612,"duration_ms":40025,"significance":"If the central claim were supported, ReLaX-Net would be a useful intermediate design point for PNNs, allowing extra effective depth without fast weight reconfiguration. The paper has clear strengths: it reports direct training results on two benchmark classes, includes ablations on repetition (L_T) and parameter budget (L_W), and is unusually honest in Section 4.3 where the fixed-budget experiment actually contradicts the abstract's equal-parameter claim. The architecture is simple, plausible, and the empirical setup is largely reproducible from the description. The value of the paper, however, depends on which claim is taken as the contribution: the narrow claim (under a fixed per-layer width cap, a few switched distinct layers improve over a same-width RNN) is supported by Figs. 5–7; the broad equal-parameter scaling claim is not.","major_comments":[{"comment":"The abstract's central claim that ReLaX-Nets 'exceed the performance of equivalent traditional RNNs or DNNs with the same number of parameters' is not supported by the paper's own controlled test. In §4.3, the total hidden parameter budget is held fixed (roughly 33k) while H is traded against L_W (Table 1). Fig. 8a reports the best test error at L_W=1, L_T>1 — the standard single-weight RNN limit. Thus, at equal parameter count, the wide stateless RNN outperforms all switched multi-layer configurations. Figures 5–6 increase parameters with L_W, so they do not provide an equal-parameter comparison, and no same-parameter DNN baseline appears anywhere. Please replace the equal-parameter claim with the supported claim (fixed per-layer width cap: a few switched distinct layers outperform a same-width RNN; repeated use at fixed L_W helps), or supply a proper same-parameter baseline that overtu","section":"Abstract; §4.3, Table 1, Fig. 8"},{"comment":"Main performance figures lack error bars and seed counts. Fig. 7 shows differences of roughly 1–2% in test loss between configurations; without repeated runs or confidence intervals the improvement from repetition is not statistically established. Supplementary Fig. C1 reports five trials for the NLP experiment, so multi-seed evaluation is feasible. Please add error bars or at least state the number of seeds for all main quantitative comparisons.","section":"§4.1–4.2, Figs. 5–7"},{"comment":"The hardware premise is load-bearing: the advantage over simply duplicating layers evaporates if the fast switches are slow, lossy, or have overhead that scales with hidden-state dimension. The text mentions DMD, fiber, and current switches but provides no system-level model of latency, insertion loss, crosstalk, or control power. I ask for either an explicit analysis (even a simple cost model) or a softened statement that practical gains are conditional on switch overhead being negligible.","section":"§3, Appendix B"}],"minor_comments":[{"comment":"The abstract refers to 'Time-Indexed Deep Alternating Layers Network (TIDAL-Net)', while the main text and title use 'ReLaX-Net'. Please unify the nomenclature throughout.","section":"Title/Abstract"},{"comment":"References to 'Supplemental Information Section 3.8/3.9' in §4 correspond to Appendix C.8/C.9; the numbering is inconsistent and should be fixed.","section":"Cross-references"},{"comment":"The relation between W_hh[t] in Eq. (3) and θ[t] in Eq. (7) could be made explicit; currently the periodic switching rule appears only in Eq. (7), which may confuse readers about whether Eq. (3) allows arbitrary time variation.","section":"Eq. (3), Eq. (6)–(7)"},{"comment":"The label 'stateless RNN' for Eq. (5) is unconventional; a fixed-weight RNN with recurrent connections is stateful. Consider renaming to 'single-weight RNN' or 'fixed-weight RNN'.","section":"Terminology"}],"recommendation":"major_revision","confidential_remarks":"The central issue is that the abstract promises an equal-parameter scaling result that Section 4.3 explicitly refutes. This is fixable by reframing the claim, but the current version is internally inconsistent. The TIDAL-Net/ReLaX-Net naming mismatch also suggests the paper may have been hastily assembled; the authors should check the consistency of the entire manuscript before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look for the PNN community, but read the abstract skeptically. The core idea—periodically switching among a few fixed weight matrices to emulate depth when weights can't be reconfigured quickly—is a genuine hardware-motivated contribution. It's clearly written, and the experiments show that at fixed per-layer width, using a few distinct weight sets (1 < LW < LT) beats a same-width stateless RNN. That's a useful result for PNN designers facing width caps.\n\nThe main problem is the abstract's claim that ReLaX-Net \"exceed[s] the performance of equivalent traditional RNNs or DNNs with the same number of parameters.\" The paper's own controlled fixed-budget experiment (§4.3, Fig. 8) directly contradicts this: at roughly 33k total hidden parameters, the widest LW=1 RNN has the best error, and every switched multi-layer configuration does worse. The authors acknowledge this in the Discussion (\"dividing a large physical trainable layer into multiple smaller ones is not worth the trade-off\"), but the abstract overstates the result. The figures showing LW>1 improving performance (Figs. 5–6) let parameter count grow with LW, so they conflate extra parameters with the architectural benefit. There is also no same-parameter DNN baseline anywhere.\n\nThat said, the paper is careful in other ways. The math is a special case of time-varying RNNs, and Appendix C.10 openly concedes that when LW divides LT it's a repeated DNN—that's honest. The oscillator-network appendix adds some generality, and the NLP learning curves include 5-trial error bars (Supp. Fig. C1). Main figures lack error bars, and no code or data are released, which is disappointing but not fatal for a concept paper. The hardware premise—fast switches routing entire hidden-state vectors without significant loss or latency—is asserted rather than analyzed, but that's a reasonable simplification for a numerical feasibility study.\n\nIf I were refereeing, I'd ask for a restated abstract advertising the narrower, defensible claim: under a per-layer width cap, periodically switched distinct layers improve over a same-width RNN. Fix that and this is a solid contribution for PNN hardware researchers. As is, the overclaim weakens it, but the underlying idea is genuinely useful and the paper deserves serious refereeing.","headline":"The hardware-motivated idea of periodic layer reuse is real and clearly presented, but the abstract's equal-parameter scaling claim is contradicted by the paper's own fixed-budget experiment.","tokens_in":26972,"tokens_out":2842,"would_cite":true,"duration_ms":29535,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Physical neural networks gain depth by switching fixed weight banks.","keywords":["physical neural networks","time multiplexing","weight tying","recurrent neural networks","parameter efficiency","photonic computing","spintronics","neuromorphic hardware"],"falsifier":"A hardware measurement: run the same ReLaX-Net schedule on a photonic or spintronic testbed with real switches, and compare end-to-end accuracy and throughput against a stateless RNN and an untied DNN with equal parameters. If switch reconfiguration time or insertion loss makes L_T forward steps slower than a single repeated layer, the claimed scaling fails.","tokens_in":26061,"feed_emoji":"🔀","tokens_out":8373,"duration_ms":83018,"temperature":0.7,"pith_summary":"Physical neural networks have two very different speeds: a fast forward pass and weights that are slow to retune. The paper's claim is that a PNN can gain the benefits of a deeper network without fast weight reconfiguration by switching periodically among a small bank of fixed, trainable weight matrices—an architecture it calls ReLaX-Net. In numerical experiments on digit images and character-level next-token prediction, ReLaX-Net with two or more reused weight sets beats a recurrent network that repeats a single fixed layer, and repeating layers improves accuracy even when no parameters are added. A fixed-budget comparison also shows that one wide layer repeated several times outperforms splitting the same parameters into many smaller distinct layers. If the hardware premise holds, this gives a practical, low-modification route to larger effective physical computers built on photonic, spintronic, or other slow-tuning substrates.","feed_headline":"Physical neural nets gain depth by switching fixed weights","feed_subtitle":"ReLaX-Net adds fast switches to reuse a few slow-tuned layers, beating same-size RNNs on image and text tasks.","key_machinery":"The central mechanism is the periodic switching schedule θ[t] = θ_{t mod L_W} inside the update h[t+1] = αh[t] + f(W_xh x[t] + W_hh[t] h[t] + b_h[t]). It turns one physical nonlinear element and L_W sets of slow-tunable weights into an L_T-layer network by time multiplexing, keeping all components except switches on slow timescales. The work it does is to generate effective depth from temporal repetition: reused layers form a function between a stateless RNN and a fully untied DNN, and because the same layer appears at several times, gradients from the different occurrences add together during training, which the paper credits for its stability against vanishing or exploding gradients.","core_discovery":"The load-bearing discovery is a middle architecture between a recurrent and a feedforward network. Instead of reconfiguring weights at every time step or keeping one fixed matrix, ReLaX-Net defines a small number L_W of distinct hidden-layer parameter sets and cycles through them periodically over L_T time steps, with a residual connection and optional input at each step. The nonlinear activation is shared; only the weight and bias elements are duplicated, and fast switches route the hidden state between the fixed parameter banks. The paper's numerical findings are that, for 1 < L_W < L_T, this scheme beats a pure stateless RNN at equal hidden-parameter count on an image classification task","pith_inferences":["This suggests a testable corollary: an aperiodic or learned switching schedule should outperform the fixed periodic schedule used in the paper, since the periodic choice is presented as the simplest option rather than the optimal one.","The fixed-budget result points to a design rule for intermediate-scale PNNs: keep each trainable layer as wide as fabrication allows and use repetition for depth, rather than splitting width into more distinct layers.","The same layer-reuse principle may transfer to other weight-constrained analog systems and to training methods that do not use backpropagation-through-time, because only the forward-path routing has to be fast; the paper's gradient analysis is one training route, not a requirement."],"forward_implications":["Adding fast switches to an existing PNN with slow weight tuning yields a deeper effective network without fast weight reconfiguration; the only new hardware elements are the switches.","On the tested benchmarks, ReLaX-Net with more than one weight set exceeds a same-size stateless RNN, and repeating layers improves performance even when the total parameter count stays fixed.","Under a fixed parameter budget, the best observed configuration is a single wide layer repeated several times, not many narrow distinct layers, so ReLaX-Net is most useful when physical constraints cap the width of each trainable layer.","The scheme is compatible with photonic and spintronic weight banks and with at least two different nonlinearities, suggesting it is a general way to scale intermediate-size physical neural networks."],"fun_headline_variants":["Time-multiplexed layers deepen physical neural nets","Reuse slow weights with fast switches for deeper PNNs","Switching fixed layers cycles depth into physical AI","Few slow layers, fast switching: deeper neural nets"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole advantage rests on fast switches being able to route the hidden state between fixed weight banks at forward-pass speed without appreciable latency, loss, or added noise; if switching is slow or lossy, the gain over simply duplicating layers disappears.","fun_headline_variants_meta":{"raw":{"variants":["Time-multiplexed layers deepen physical neural nets","Reuse slow weights with fast switches for deeper PNNs","Switching fixed layers cycles depth into physical AI","Few slow layers, fast switching: deeper neural nets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"total_tokens":1243,"prompt_tokens":710,"completion_tokens":533,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":454,"completion_tokens_details":{"reasoning_tokens":468}},"tokens_in":454,"tokens_out":533,"duration_ms":5998,"temperature":1.0,"reasoning_tokens":468,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T07:50:07.969335+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A hardware measurement: run the same ReLaX-Net schedule on a photonic or spintronic testbed with real switches, and compare end-to-end accuracy and throughput against a stateless RNN and an untied DNN with equal parameters. If switch reconfiguration time or insertion loss makes L_T forward steps slower than a single repeated layer, the claimed scaling fails.","supporting_citations":[],"review_version":1}