{"id":"f3899230-8d3e-48d5-9150-d1828c67eb4b","arxiv_id":"2507.18700","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Growing the hidden dimension of recurrent neural network quantum states during training cuts wall-clock time by 3-4x and sometimes improves variational accuracy compared to fixed-size training.","lead":"This paper shows that training a quantum wavefunction model by gradually increasing its internal memory size can be roughly three to four times faster than training the full-size model from scratch, while giving similar or better energies. The scheme is a practical trick for making neural-network quantum simulations cheaper on GPUs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Accuracy gains are not cleanly separable from unequal learning-rate schedules; a Static RNN under the Adaptive LR schedule is needed before crediting the growth scheme.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: unequal learning-rate schedules between Static and Adaptive runs leave the accuracy/stability advantage unproven. I agree with that reading. The wall-clock speedup is the most solid contribution: it is reproduced across four benchmarks, is machine-dependent but explicitly reported with A100 GPUs, and the asymptotic 25.6% estimate follows from the measured scaling fits. Open code availability also helps reproducibility. The vulnerable part is the broader claim that Adaptive training 'improves the quality' of variational calculations; that claim needs a controlled Static baseline with the Adaptive LR schedule. Without it, the observed better final variances and lower training fluctuations could simply be a learning-rate effect. This is why the appropriate verdict remains CONDITIONAL, not REJECT: the speedup claim stands, and the quality claim may survive the control experiment, but it is not yet established by the reported data. No ad hominem is intended; the concern is purely about experimental design and attribution.","tokens_in":17349,"tokens_out":3993,"duration_ms":46596,"concrete_test":"Train a Static dh=256 pRNN on the 1D TFIM with N=100 using exactly the Adaptive learning-rate schedule from Table III: LR 5e-3 for training steps 1-25,000 and 5e-4 for steps 25,001-50,000, with the same batch size, Adam hyperparameters, and total steps as the Adaptive run. Repeat for at least five independent seeds and compare the final energy, variance per spin, and the variance-vs-step curve against (i) the published Adaptive RNN and (ii) the Static fixed-LR 5e-4 result. If Static with the Adaptive LR matches the Adaptive variance within seed spread, then the claimed accuracy and stability improvements are not caused by the growth scheme. A complementary check on the 2D Heisenberg benchmark with the Adaptive decay schedule would confirm whether the conclusion transfers to two dimensions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two parts: a wall-clock speedup and matching-or-improved accuracy. The speedup is robust, since it follows directly from training smaller models early and is reported consistently across benchmarks. The accuracy/stability claim, however, is confounded. In three of the four benchmarks (1D TFIM, 2D Heisenberg, Cluster state) the Static and Adaptive runs use different learning-rate schedules, as summarized in Table III, and no Static control is run with the Adaptive schedule. For the 1D TFIM, Adaptive uses 5e-3 for the first 25,000 steps and 5e-4 afterward, while Static uses a fixed 5e-4; for the 2D Heisenberg, the two decay schedules differ; for the Cluster state, Static uses 1e-4 and Adaptive uses 1e-3. Because Adam momentum is carried across growth steps, the confound is not just the final LR but the whole optimization trajectory. The long-range TFIM does use the same LR (1e-3) for both methods and shows an Adaptive energy advantage, but it is a single short run with no seed averaging and comparable total runtimes, so it cannot independently settle the question. In addition, all reported error bars are Monte Carlo sampling errors from the final 1,000,000 samples, not seed-to-seed training variability, so claims of 'reducing training fluctuations' rest on single trajectories. If a fixed dh=256 Static RNN trained under the Adaptive LR schedule reproduces the Adaptive final variance, the quality and stability benefits would be attributable to LR tuning rather than to the dimension-growing scheme.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an Adaptive training scheme for recurrent neural network (RNN) quantum states, in which the hidden-state dimension is doubled at fixed intervals and the parameters of the smaller model, along with the Adam optimizer state, are padded with small random values to initialize the larger model. The method is benchmarked against fixed-size (Static) RNNs on four spin Hamiltonians: the 1D transverse-field Ising model (TFIM), the 2D square-lattice Heisenberg model, the long-range TFIM, and the 1D cluster state. The authors report that Adaptive RNNs reach comparable or better variational energies and energy variances while training in a fraction of the wall-clock time in the 1D TFIM and 2D Heisenberg benchmarks, with an asymptotic time ratio of about 25.6% for large 1D systems. They also report improved stability of the training trajectory and argue that the scheme helps avoid local minima, particularly for the cluster state and long-range TFIM.","tokens_in":17711,"tokens_out":4524,"duration_ms":46819,"significance":"The speed advantage of the Adaptive scheme is well supported: the wall-clock measurements across four benchmarks consistently show that training small models early and reusing them as initializations saves substantial time, and the asymptotic scaling analysis in Appendix C is a useful quantitative contribution. The paper also ships open-source code, which is a valuable strength for reproducibility. If the accuracy and stability improvements are confirmed, the method would be a practical and broadly applicable improvement for NQS optimization, analogous to growing the bond dimension in DMRG. However, the current evidence for improved accuracy and stability is not cleanly separated from unequal learning-rate schedules between Static and Adaptive runs, and the stability claim rests on single training trajectories. These issues are load-bearing for the paper's central claims and require additional controlled experiments.","major_comments":[{"comment":"The accuracy and stability comparison is confounded by different learning-rate schedules. In the 1D TFIM, Static uses a fixed 5e-4 while Adaptive uses 5e-3 for the first 25,000 steps and 5e-4 afterwards; in the 2D Heisenberg model the two schedules differ (Static decay from 5e-4, Adaptive fixed 5e-4 followed by a decay); and in the cluster state Static uses 1e-4 while Adaptive uses 1e-3. No Static control run with the Adaptive learning-rate schedule is reported. Because the Adam momentum is carried across growth steps, the entire optimization trajectory differs, not just the final learning rate. A direct control—for example, a Static dh=256 run under exactly the Adaptive learning-rate schedule, or matched schedules for both methods—is required before the energy/variance improvements can be attributed to the growing hidden-dimension scheme rather than to learning-rate tuning.","section":"§III.A, §III.B, §III.D and Table III"},{"comment":"The claim that Adaptive training 'reduces training fluctuations' is not supported by the data presented. Figures 3(a) and 7 show one Static and one Adaptive trajectory per system size; the error bars in Tables I and II are Monte Carlo sampling errors on the final 1,000,000-sample estimates, not seed-to-seed training variability. A single trajectory cannot establish a reduction in training fluctuations. The authors should either provide multiple independent training runs with error bars on the training curves and final observables, or explicitly restrict the stability claim to the particular trajectories shown.","section":"§III.A and Appendix D"},{"comment":"There are internal inconsistencies in the reported hyperparameters that affect reproducibility. The text in §III.A and Table III state the Static 1D TFIM learning rate is 5e-4, but the Fig. 5 caption in Appendix B says '10^-4 is identified as the optimal rate'. Also, Table III lists the 2D Heisenberg Adaptive model as 'dh doubling every 25,000 steps', while the text in §III.B states 'double it every 50,000 steps'. These inconsistencies should be reconciled, since the empirical comparison depends on the precise training schedule.","section":"Appendix B, Fig. 5 caption, and Table III"}],"minor_comments":[{"comment":"The caption contains a typo: 'Atatic' should be 'Static'.","section":"Fig. 6 caption"},{"comment":"The mapping for the GLU layer writes 'b1, b2 ∈∈ R^{dmodel}'; the double '∈' is a typo and should be corrected.","section":"Appendix A"},{"comment":"The sentence 'The latter is a good indicator of the quality of a variational calculation' uses 'latter' to refer to the energy variance; consider clarifying the antecedent in the surrounding discussion.","section":"§III.D"},{"comment":"The interpretation that Adaptive RNNs start with 'a low entanglement structure' is plausible but presented without direct entanglement measurements; a brief caveat or reference to the entanglement properties of small-dh RNNs would strengthen the discussion.","section":"§III.C"}],"recommendation":"major_revision","confidential_remarks":"The central speed claim is solid and the paper is a good fit for the journal. The main barrier to acceptance is experimental control: the accuracy and stability claims need matched learning-rate schedules and multiple seeds. The internal hyperparameter inconsistencies in Appendix B and Table III should also be fixed before the paper can be considered reproducible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a genuinely useful engineering contribution to neural quantum states, but the headline about improved accuracy is not yet fully supported. The wall-clock speedup is real and reproducible; the quality gain is confounded by unequal learning-rate schedules.\n\nWhat's new: the authors apply Net2Net-style parameter padding to recurrent neural network wave functions, growing the hidden dimension during variational Monte Carlo. That specific application—as opposed to RBM transfer learning or fixed-dimension tensorized RNN pretraining—is not in the cited literature. The paper runs four benchmarks, reports wall-clock times, and provides code. That counts for a lot.\n\nThe strengths: the speedup story is credible. Starting from a tiny hidden state and doubling it at fixed intervals reduces training time by roughly 3-4x on the 1D TFIM, and the scaling analysis suggesting an asymptotic ~25.6% ratio is a useful practical guide. The authors are transparent about hyperparameters; Table III states the exact schedules. The long-range TFIM benchmark uses the same learning rate for both methods, making it the cleanest comparison, and the Adaptive run does get a lower variational energy.\n\nNow the soft spots, and they are real. In three of the four benchmarks (1D TFIM, 2D Heisenberg, Cluster state) the Static and Adaptive runs use different learning-rate schedules. Adam momentum is carried across growth steps, so the whole optimization trajectory differs, not just the final LR. Without a Static run at the identical Adaptive schedule, you cannot attribute the accuracy or variance improvements to the growth scheme. The stress-test note is spot-on here. The reported error bars are Monte Carlo sampling errors from final samples, not seed-to-seed training variability, so 'reducing training fluctuations' rests on single trajectories. That is a minor-to-moderate issue, but it matters for the stability claim. Also, the Cluster and long-range benchmarks show comparable total runtimes, so the speedup is not universal; the big wins come when the growth schedule lets you skip most of the high-dimensional training early.\n\nWho this is for: practitioners using RNN or other autoregressive NQS who care about GPU budgets. It deserves a serious referee—the idea is simple, plausibly useful, and the confound is fixable. I would send it to peer review with a request for Static control runs under the Adaptive LR schedules and at least a few different seeds for the main benchmarks.","headline":"Useful and honest NQS training-speedup paper, but the accuracy claims need control runs under matched learning-rate schedules before they can be trusted.","tokens_in":18259,"tokens_out":2975,"would_cite":true,"duration_ms":28996,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adaptive RNNs, which grow their hidden dimension during training, reach the same or better variational accuracy for quantum ground states in roughly a quarter of the wall-clock time of fixed-size RNNs, and the speedup grows with system…","keywords":["neural quantum states","recurrent neural networks","variational Monte Carlo","adaptive training","hidden-state growth","parameter padding","quantum many-body ground states","GPU efficiency"],"falsifier":"Run a Static RNN with the final hidden dimension using exactly the Adaptive learning-rate schedule, such as 5e-3 for the first half of training and 5e-4 for the second half, and compare wall-clock time, final energy, and variance; if the Static model matches the Adaptive results, the benefit would come from the schedule rather than from growing the hidden dimension.","tokens_in":17153,"feed_emoji":"⚛️","tokens_out":7839,"duration_ms":73858,"temperature":0.7,"pith_summary":"The paper claims that variational calculations of quantum ground states with recurrent-neural-network wave functions become much cheaper if the network's hidden memory grows during training instead of staying fixed. The proposed Adaptive scheme trains a small RNN first, then pads its weights with small random numbers to initialize a larger RNN, repeating until the target size is reached. On four spin-model benchmarks the scheme finishes in a fraction of the wall-clock time, such as 34% on the 100-spin 1D transverse-field Ising model with an asymptotic ratio near 25.6%, and about 6.5 hours versus 14.8 hours on the 6x6 2D Heisenberg model, while matching or improving variational energies and variances. The reason to care is that if true, larger and more accurate neural quantum states become accessible within existing GPU budgets.","feed_headline":"Small-to-large training cuts quantum ground-state runs to ~25 percent","feed_subtitle":"Growing-memory wave functions match fixed-size models in a quarter of the time.","key_machinery":"The load-bearing mechanism is parameter padding at each capacity increase: when the hidden dimension doubles, existing weight matrices and biases are copied into the larger tensors and the remaining entries are filled with small random numbers, so the larger RNN begins as a perturbed continuation of the smaller one rather than a fresh random initialization. Training is organized as a sequence of GRU-based RNN quantum states, each pretrained by the previous one; the hidden-state memory controls expressivity, the chain rule gives autoregressive sampling, and the optimizer's momentum is carried across transfers. This is what converts training a big network into training a small network and then growing it.","core_discovery":"The central claim is that a capacity-growing training scheme makes RNN wave functions cheaper and more stable without sacrificing accuracy. Starting from a small hidden dimension, the network is trained for a fixed number of steps, then its hidden dimension is doubled and its weights and biases are padded with small random numbers to initialize the larger network; the optimizer's momentum is carried over, and all parameters remain trainable. Applied to ground-state searches via variational Monte Carlo, this Adaptive RNN reaches the same or better energies and variances than a Static RNN of the final size trained for the same number of steps, while using a fraction of the wall-clock time. On the long-range TFIM and cluster-state problems it reaches lower final energies than Static RNNs, which the authors interpret as better avoidance of local minima, and a mid-sequence model with half the final hidden dimension matches the Static model's accuracy.","pith_inferences":["If the benefit is a curriculum effect, where the network first learns simple low-entanglement structure before adding capacity, then the optimal growth schedule should track the entanglement structure of the target state; a testable extension would vary the doubling interval and compare across different phases.","The padding trick is architecture-agnostic in principle, so the same small-to-large idea could be tried on other neural quantum states such as transformers or convolutional ansatze, but the paper only demonstrates it for GRU-based RNNs, leaving transfer to other architectures an open question.","Because the penultimate half-size model already matches the full-size Static model's accuracy, the scheme may also serve as a parameter-reduction strategy, lowering GPU memory pressure on larger lattices even when wall-clock time is not the main constraint."],"forward_implications":["On the 1D TFIM with 100 spins, the Adaptive RNN finishes in 34% of the Static RNN's wall-clock time and reaches comparable or lower energy variance, with the fitted time ratio approaching about 25.6% as system size grows.","On the 6x6 2D Heisenberg model, the Adaptive RNN reaches the best energy, error, and variance in 6 hours 33 minutes versus 14 hours 48 minutes for the Static RNN, and the early-stopping variant also beats the Static model while taking 13 hours 49 minutes.","On the long-range TFIM and the 1D cluster state, the Adaptive RNN reaches lower final variational energies than the Static RNN, suggesting it avoids excited-state local minima that trap fixed-size training.","Models earlier in the Adaptive sequence with half the final hidden dimension achieve energies comparable to the Static model with the full dimension, indicating better trainability with fewer parameters."],"supporting_citations":[{"why":"Supplies the RNN wave function ansatz, the pRNN/cRNN construction, and the variational training setup that the Adaptive scheme modifies.","marker":"[25]"},{"why":"Net2Net's parameter-transfer idea is the direct precursor for padding weights with small random numbers when widening the network.","marker":"[42]"},{"why":"Prior transfer-learning approach for RBM neural quantum states that the paper positions its hidden-dimension growth against.","marker":"[43]"},{"why":"Extends RBM transfer learning to larger, broader, and deeper NQS and serves as a comparison point for scalability claims.","marker":"[44]"},{"why":"Defines the cluster-state benchmark at theta equals zero as the hardest point by conditional mutual information, the test used for the complex RNN with sign structure.","marker":"[72]"},{"why":"Adam is the optimizer used throughout, and its momentum state is carried over between models in the Adaptive sequence.","marker":"[57]"},{"why":"Provides the variational Monte Carlo framework and the energy-variance quality measure used to compare Static and Adaptive results.","marker":"[19]"},{"why":"DMRG reference energies define the relative errors reported for the 1D TFIM benchmarks.","marker":"[15]"}],"fun_headline_variants":["Adaptive RNNs cut quantum ground-state cost","Train small, scale up: faster quantum neural states","Recycled weights stabilize quantum variational training","Grow your network, shrink quantum simulation time","Capacity-growing RNNs speed up quantum ground states"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the gains come from the capacity-growing scheme itself and not from the different learning-rate schedules used for the Static and Adaptive runs, since in three of four benchmarks the schedules differ and no Static control with the Adaptive schedule is reported.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive RNNs cut quantum ground-state cost","Train small, scale up: faster quantum neural states","Recycled weights stabilize quantum variational training","Grow your network, shrink quantum simulation time","Capacity-growing RNNs speed up quantum ground states"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1420,"prompt_tokens":841,"completion_tokens":579,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":506}},"tokens_in":457,"tokens_out":579,"duration_ms":6149,"temperature":1.0,"reasoning_tokens":506,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:08:58.270725+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a Static RNN with the final hidden dimension using exactly the Adaptive learning-rate schedule, such as 5e-3 for the first half of training and 5e-4 for the second half, and compare wall-clock time, final energy, and variance; if the Static model matches the Adaptive results, the benefit would come from the schedule rather than from growing the hidden dimension.","supporting_citations":[{"cited_title":"Zen and S","cited_arxiv_id":null,"evidence_quote":"Extends RBM transfer learning to larger, broader, and deeper NQS and serves as a comparison point for scalability claims."},{"cited_title":"Becca and S","cited_arxiv_id":null,"evidence_quote":"Provides the variational Monte Carlo framework and the energy-variance quality measure used to compare Static and Adaptive results."}],"review_version":2}