{"id":"631d6625-d06e-415c-be2e-f2cfb8191936","arxiv_id":"2505.12470","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"NeuroGen trains an LLM to emit neural-network weights from task descriptions, and the generated weights classify MNIST, SVHN, CIFAR-10, SST-2, SNLI, and AG News with usable accuracy, usually below but sometimes above conventionally trained small models.","lead":"NeuroGen fine-tunes a large language model to write the parameters of a small neural network from a text description of the task and data. The authors report that the generated weights produce usable classifiers on standard image and text benchmarks, though usually below conventionally trained models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The data-efficiency claim in §4.4 depends on whether the Stage-1 larger-CNN reference checkpoints were trained on the same full SVHN/CIFAR-10 data, which the paper never states.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern I find: Section 4.4 and Figure 3 claim a data-efficient generalization advantage, but the paper never states what data the larger-CNN Stage-1 checkpoints were trained on. Under the natural reading of Section 3.3.1, those checkpoints were trained on the full target dataset, which would let the LLM indirectly see the full training set before the limited-data Stage 2 comparison. That would make the Figure 3 comparison unfair and would weaken the paper's central narrative about data-efficient, instruction-guided parameter generation. The core feasibility result in Table 1 is less threatened by this specific issue, since even a generator that has seen full-data checkpoints could still demonstrate that LLMs can produce usable parameters; however, the generalization and low-data claims, which are explicitly highlighted as contributions, would be substantially undermined. The 'without gradient-based optimization' phrasing is also in tension with Equation (2), but this is partly a framing issue because the generated target weights themselves are not directly trained. No code or error bars are provided, but those are reproducibility concerns rather than the central logical weak point. I therefore keep the reader's CONDITIONAL verdict: the concern is concrete and testable, and a controlled rerun would either strengthen or require revision of the paper's main claims.","tokens_in":10530,"tokens_out":3865,"duration_ms":45533,"concrete_test":"Re-run the Figure 3 limited-data protocol with a controlled Stage-1 pretraining: first, disclose or inspect whether the larger-CNN checkpoints for SVHN and CIFAR-10 were trained on the full target training sets; second, repeat the 10,000-sample experiment with Stage 1 pretraining performed on an unrelated image dataset (e.g., MNIST or a disjoint class set) while keeping Stage 2 unchanged. If NeuroGen's advantage over the classical 10k-sample model disappears or reverses, the data-efficiency claim is unsupported; if the advantage persists under the controlled pretraining, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 4.4 and Figure 3 claim that NeuroGen, after only Stage 2, can generate a smaller CNN that outperforms a classical model in the 10,000-sample limited-data regime. The comparison is only fair if the LLM's Stage-1 pretraining did not already expose it to label-dependent information from the same full datasets. But Section 3.3.1 states that reference checkpoints are obtained by training the target model on the full dataset D with distinct random seeds, and the paper never specifies what data the larger CNN's Stage-1 checkpoints were trained on for Figure 3. If those checkpoints used the full SVHN or CIFAR-10 training sets, then the LLM saw the target classes before Stage 2, so the comparison against a classical model trained only on 10,000 samples is not a valid test of data-efficient generation. This is load-bearing because the paper's second contribution and the 'edge deployment' motivation rest on exactly this limited-data advantage. A second, related weakness is that the headline claim 'without gradient-based optimization' is contradicted by Equation (2), where {P, phi, theta} are explicitly updated by minimizing the task loss; the target weights w_g are not directly trained, but the generator is, so the novelty framing is weaker than stated. The Fig. 3 information-leakage issue is the more direct threat to the central empirical claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NeuroGen, a two-stage framework in which an LLM is fine-tuned to generate the parameters of a lightweight target neural network directly, given a task description and training data. In Stage 1, the LLM is aligned with reference checkpoints of the target architecture through learnable special tokens, LoRA, and a projection MLP, using a parameter-space similarity loss. In Stage 2, the model is instruction-tuned with a task loss evaluated through the generated weights on a subset of training data. Experiments cover image classification (MNIST, SVHN, CIFAR-10) with CNN and LeNet targets and text classification (SST-2, SNLI, AG News) with MLP and RNN targets. The main empirical claim is that NeuroGen produces usable neural network parameters, competitive with or sometimes better than standard gradient-based training, and that after Stage-1 pretraining on a larger architecture, Stage-2-only tuning can generate smaller models that outperform classical training in low-data regimes.","tokens_in":10831,"tokens_out":2988,"duration_ms":32456,"significance":"If the central claim holds, the paper opens a distinct research direction: treating neural network weight acquisition as a conditional generation problem solved by LLMs. The framework is clearly described, explicitly limited to small classification models, and accompanied by an honest limitation section. The strengths include a transparent two-stage training scheme, the use of non-autoregressive parallel parameter generation, and a direct comparison against classical training on standard benchmarks. However, the empirical support is currently thin: the headline results are single runs with no variance, no code is released, the connection to the diffusion-based weight generation literature is asserted rather than experimentally contrasted, and the low-data generalization experiment in Section 4.4 is vulnerable to information leakage because the Stage-1 checkpoint source is unspecified. The claimed 'without gradient-based optimization' framing is also stronger than what Equation (2) actually implements, since the generator parameters are optimized by gradient descent.","major_comments":[{"comment":"The limited-data comparison in Section 4.4 is load-bearing for the paper's data-efficiency claim, but the paper never states what data was used to obtain the Stage-1 reference checkpoints of the larger CNN. If those checkpoints were trained on the full SVHN or CIFAR-10 training sets, then the LLM was pre-exposed to label-dependent information from the full data before Stage 2 was run on only 10,000 samples, making the comparison against a classical model trained only on 10,000 samples unfair. The authors must specify the training data for the Stage-1 larger-CNN checkpoints, and ideally rerun the comparison with Stage-1 pretraining also restricted to the 10,000-sample subset, to establish that the observed advantage comes from generative generalization rather than from memorization of the full dataset.","section":"§4.4, Figure 3"},{"comment":"The contribution 'without gradient-based optimization' is contradicted by Equation (2), where the parameters {P, phi, theta} are explicitly updated by minimizing the task loss L2 via gradient-based optimization, and by Algorithm 1's update steps. What the paper actually demonstrates is that the target weights w_g are not directly trained, while the generator that produces them is trained with gradients. The claim should be rephrased as 'without direct gradient-based optimization of the target network parameters' to be accurate, and the discussion in Sections 1 and 5 should be adjusted accordingly.","section":"§2.1, Eq. (2), Contribution 1"},{"comment":"All six accuracy numbers in Table 1 are reported as single values with no error bars, no number of seeds, and no statistical significance tests. Because the differences between NeuroGen and the classical baseline are small in several cases (e.g., SST-2 RNN: 76.03 vs. 77.63, AG News RNN: 85.14 vs. 84.72), the central empirical claim that LLM-generated parameters are 'usable' and comparable to classical training is not yet established with confidence. The authors should report means and standard deviations over at least three to five seeds, and ideally provide per-seed results.","section":"Table 1, §4.2"},{"comment":"Several details needed to reproduce NeuroGen are missing: the size of the Stage-2 data subset |D_sub|, the LoRA rank, the projection MLP architecture and hidden dimension, the number of special tokens d1, the exact learning rate schedule for each stage, and the optimizer details beyond 'SGD optimizer'. Additionally, footnote 2 states that code will be released only after acceptance, which further impedes verification. These omissions are significant because the paper's contribution is described as an 'easy-to-implement' framework, and the reported results cannot be checked without them.","section":"§3.3–§3.4, §4.1"},{"comment":"The paper dismisses hypernetwork and diffusion-based weight-generation baselines as 'not directly applicable or comparable' in Section 2.3, but does not provide any experimental comparison with them. Since Neural Network Diffusion, Conditional LoRA Parameter Generation, and hypernetworks are the most closely related prior methods, the significance of NeuroGen relative to those approaches is unclear. At minimum, the authors should include a small-scale comparison on one image and one text task, or provide a concrete argument with evidence for why such baselines cannot be adapted to the instruction-conditioned setting.","section":"§2.3, §4.2"}],"minor_comments":[{"comment":"The Implementation Details paragraph contains a leftover reference to 'the proposed FedType', which is not defined in this paper and appears to be a remnant of another project; it should be replaced with 'NeuroGen'.","section":"§4.1"},{"comment":"The manuscript contains numerous typos and misspellings, including 'approachaes', 'paramaters', 'nerual', 'mechnism', 'priliminary', and 'netwrok'. A thorough proofreading pass is needed.","section":"Throughout"},{"comment":"The figure caption refers to 'Phase 1 and Phase' without specifying the second phase; it should read 'Phase 1 and Phase 2'.","section":"Figure 2"},{"comment":"Equation (1) defines the Stage-1 objective with a generic similarity function sim(·,·), while Algorithm 1 uses L_mse; these should be made consistent, and the exact similarity function used in the experiments should be stated.","section":"Algorithm 1, Eq. (1)"},{"comment":"The table layout is confusing: the header 'NN Image NN Text' merges two separate sub-tables, and the row labels 'Classical CNN' and 'NeuroGen' are repeated for different architectures. Separating the image and text results into two tables or using clearer multi-level headers would improve readability.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and interesting question, but the main empirical claims are currently under-supported. The information-leakage issue in Section 4.4 is the most serious concern and should be resolved before the paper can be considered for publication. I would also encourage the editor to require the authors to release code and data-generation scripts upon acceptance, given that the entire evaluation rests on the reproducibility of the two-stage pipeline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a real demonstration of something new: an LLM finetuned with LoRA can emit a flat weight vector that, after slicing into a small CNN/RNN, classifies MNIST at 97.7% and SST-2 at 76%. That is non-obvious and worth a look. Second, the paper's more exciting claim—that NeuroGen is better than classical training in low-data regimes—is probably contaminated by information leakage. Figure 3 uses an LLM whose Stage 1 pretraining was done on checkpoints of a larger CNN on the same datasets (SVHN, CIFAR-10), almost certainly trained on the full training sets. Those checkpoints carry label-dependent information about those datasets. So comparing against a classical model trained only on 10,000 samples is not a fair test of data-efficient generation. The paper never states what data the larger CNN checkpoints used, and the reader's stress-test gets this right.\n\nWhat's genuinely good: the two-stage recipe is simple and reproducible in principle; the main results show the generated weights are functional, not random; the ablation shows Stage 1 matters; and Section 4.6 honestly lists scalability and modality limitations. The authors also cite the relevant hypernetwork and diffusion checkpoint-generation literature.\n\nSoft spots beyond the leak: no code, no seeds, no error bars, so every reported number is a single draw. No baseline against a hypernetwork (Ha et al.) or a diffusion weight generator, which is the natural comparison given the related work. The headline 'without gradient-based optimization' is misleading—Equation (2) and Algorithm 1 show the generator parameters {P, phi, theta} are updated by backprop through the task loss. What's true is the emitted weights themselves are not directly optimized; that's a weaker, still interesting claim. Writing is rough: typos, 'FedType' leftover from another paper, inconsistent Stage/Phase terminology.\n\nWould I cite it? Probably not in my own work yet. Who should read it: people working on model compression and hypernetworks will find the demonstration intriguing; the data-efficiency claim should be treated with skepticism until the leak is resolved. It deserves peer review—the core idea is new and the evidence, while thin, is enough to warrant a proper referee process requiring code, seeds, and a fair low-data comparison. Recommend: send to review, but expect major revision.","headline":"A genuinely new proof-of-concept for LLM-generated weights, but the low-data advantage likely leaks full-dataset information and the 'no gradient' framing is overstated.","tokens_in":11357,"tokens_out":2555,"would_cite":false,"duration_ms":25993,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An LLM can generate functional neural network parameters directly from task and data descriptions, with no gradient-based optimization of the target network.","keywords":["neural network parameter generation","large language models","parameter reference knowledge injection","context-enhanced instruction tuning","prompt-driven weight generation","non-autoregressive parameter synthesis","classification benchmarks"],"falsifier":"Re-run the Figure 3 generalization experiment with Stage-1 checkpoints trained only on datasets disjoint from SVHN and CIFAR-10 (or on synthetic images), then apply Stage 2 on 10,000 samples of SVHN and CIFAR-10; if NeuroGen still outperforms a classical model trained on those 10,000 samples the data-efficiency claim survives, and if accuracy collapses the limited-data advantage came from prior exposure to full-data checkpoints of the same datasets.","tokens_in":10296,"feed_emoji":"🧠","tokens_out":11272,"duration_ms":98983,"temperature":0.7,"pith_summary":"NeuroGen tries to establish a new route to neural network weights: instead of fitting parameters to data with backpropagation, a large language model produces the whole parameter vector in one non-autoregressive pass, conditioned on a description of the task, the data, and the target architecture. The method is two-stage: first the LLM is aligned to a distribution of conventionally trained checkpoints through a similarity loss, then it is instruction-tuned so that the weights it emits minimize the task loss on a sample of real training data. The experiments show generated classifiers reaching 97.71% accuracy on MNIST and 76.03% on SST-2, close to or better than classically trained counterparts on some benchmarks. The paper interprets this as evidence that LLMs can internalize a latent mapping from training context to weights, opening the possibility of prompt-driven, data-efficient model generation.","feed_headline":"Generates usable neural networks from an LLM without backprop training","feed_subtitle":"The two-stage NeuroGen prompt method reaches 97.71% on MNIST and 76.03% on SST-2 with generated weights.","key_machinery":"The load-bearing object is a learnable special-token matrix $P\\in\\mathbb{R}^{d_1\\times d_2}$ that is concatenated with the instruction embedding, passes through the LLM, and is mapped by a projection MLP to a flat parameter vector, which is then sliced into layer-wise weights of the target network $f$. Two objectives drive training: Stage 1 aligns generated weights $w_g$ to reference checkpoints $w_i$ via $L_1(P,\\phi,\\theta)=\\mathbb{E}_{w_i\\sim W}[\\operatorname{sim}(w_i,w_g)]$, for example negative mean squared error; Stage 2 minimizes $L_2(P,\\phi,\\theta)=\\sum_{(x_j,y_j)\\in D_{\\mathrm{sub}}^t} \\mathcal{L}_{\\mathrm{task}}(f(x_j\\mid w_g^t),y_j)$. Gradients update the special token $P$, the LoRA parameters $\\phi$, and the projection parameters $\\theta$; the target network's own weights $w_g^t$ are outputs, not optimized variables.","core_discovery":"On the paper's own terms, the discovery is that an LLM can be turned into a conditional generator of usable neural network parameters. The target network's weights are never optimized directly; instead, the LLM's learnable special token, LoRA adapters, and a projection MLP are optimized so that the emitted weight vector performs well on the task. Stage 1 supplies the needed prior by regressing generated weights toward checkpoints obtained by ordinary gradient-based training, and Stage 2 refines that prior by backpropagating the target task's cross-entropy through the instantiated generated network into the LLM's parameters. The reported accuracies on standard classification benchmarks are the evidence that the generated parameters are functionally effective, not merely distributionally plausible.","pith_inferences":["A natural extension not tested here: hold out entire classes or datasets from Stage-1 pretraining and ask whether generated weights on those held-out classes still beat classical training; this would separate memorization of a weight distribution from compositional generation.","Another untested design choice: replacing the Stage-1 MSE alignment with a functional loss (such as agreement of logits on unlabeled data) might improve generated weights because it optimizes behavior rather than raw parameter coordinates.","If the mapping from task/data descriptions to weights is genuinely learnable, then inspecting the generated weights becomes a form of interpretability—the LLM's output is its answer to what a classifier for the described task should look like.","Since weights are produced in one forward pass, changing the instruction or data subset changes the weights, suggesting a testable route to model editing by prompt rather than by retraining."],"forward_implications":["A single LLM can act as a reusable weight generator: Stage-1 parameter-reference knowledge transfers across tasks, and only Stage 2 has to be repeated for a new task or dataset.","In small-data settings, generated weights can beat weights obtained by classical training on the same sample, which would make LLM priors a practical substitute for training data.","Deployment could become prompt-driven: rather than shipping fitted weights, a system could ship the architecture description and generate task-specific weights at inference time on the device.","The non-autoregressive, full-weight generation strategy limits target-network size, so scaling to large models would require generating only adapters, prompts, or low-rank updates rather than the full parameter vector."],"supporting_citations":[{"why":"Defines the backpropagation paradigm that NeuroGen bypasses and that its generated weights are compared against.","marker":"[1]"},{"why":"The diffusion-based parameter generation work whose distribution-alignment objective Stage 1 adapts and whose limitations NeuroGen contrasts with.","marker":"[7]"},{"why":"The conditional parameter-generation approach whose distribution-alignment objective Stage 1 also follows.","marker":"[8]"},{"why":"Supplies the checkpoint-dataset and generative-model-of-checkpoints idea that Stage 1 builds on.","marker":"[9]"},{"why":"The hypernetwork baseline that also generates target-network parameters, used to frame NeuroGen's prompt-driven difference.","marker":"[17]"},{"why":"MNIST, the image benchmark where NeuroGen's generated CNN reaches 97.71% accuracy.","marker":"[33]"},{"why":"SST-2, the text benchmark on which NeuroGen's generated RNN reaches 76.03% accuracy.","marker":"[36]"},{"why":"The vision LLM used as the base model for generating image-classifier parameters.","marker":"[39]"},{"why":"The text-only LLM used as the base model for generating text-classifier parameters.","marker":"[40]"},{"why":"The classic convolutional architecture used as one of the target image-classifier models.","marker":"[41]"}],"fun_headline_variants":["LLM generates neural network weights that actually work","No backprop: LLM writes a 97.7% MNIST classifier","NeuroGen: LLM turns prompts into usable network weights","From text to weights: LLM crafts functional neural nets","LLM-generated parameters beat training on small tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that NeuroGen is data-efficient assumes the Stage-1 checkpoints were not trained on the same full datasets used to judge limited-data generalization, because the paper does not report what data those larger-CNN checkpoints were trained on.","fun_headline_variants_meta":{"raw":{"variants":["LLM generates neural network weights that actually work","No backprop: LLM writes a 97.7% MNIST classifier","NeuroGen: LLM turns prompts into usable network weights","From text to weights: LLM crafts functional neural nets","LLM-generated parameters beat training on small tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000166,"raw_usage":{"total_tokens":1218,"prompt_tokens":871,"completion_tokens":347,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":265}},"tokens_in":487,"tokens_out":347,"duration_ms":3902,"temperature":1.0,"reasoning_tokens":265,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:33:32.624284+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Figure 3 generalization experiment with Stage-1 checkpoints trained only on datasets disjoint from SVHN and CIFAR-10 (or on synthetic images), then apply Stage 2 on 10,000 samples of SVHN and CIFAR-10; if NeuroGen still outperforms a classical model trained on those 10,000 samples the data-efficiency claim survives, and if accuracy collapses the limited-data advantage came from prior exposure to full-data checkpoints of the same datasets.","supporting_citations":[{"cited_title":"The mnist database of handwritten digit images for machine learning research.IEEE Signal Processing Magazine, 2012","cited_arxiv_id":null,"evidence_quote":"MNIST, the image benchmark where NeuroGen's generated CNN reaches 97.71% accuracy."},{"cited_title":"Manning, Andrew Y","cited_arxiv_id":null,"evidence_quote":"SST-2, the text benchmark on which NeuroGen's generated RNN reaches 76.03% accuracy."},{"cited_title":"Lecun, L","cited_arxiv_id":null,"evidence_quote":"The classic convolutional architecture used as one of the target image-classifier models."}],"review_version":1}