Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Recurrent Diffusion for Large-Scale Parameter Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a recurrent-conditioned diffusion model can synthesize a network's full parameter set—up to about 200 million weights—in a single pass on one commodity GPU, matching fully trained accuracy across vision and language…

desk verdict RPG is a credible scaling breakthrough for weight generation, but the missing inverse of the layer-wise normalization makes the central claim under-specified as written. read the letter →

arxiv 2501.11587 v2 pith:4TNGNSSO submitted 2025-01-20 cs.LG cs.AI

Kai Wang , Dongwen Tang , Wangbo Zhao , Konstantin Schürholt , Zhangyang Wang , Yang You This is my paper · ORCID
classification cs.LGcs.AI
keywords parametergenerationdiffusionmodelsrecurrentneuralnetworksweightspacelearninglarge-scaleLoRAnetworkweightsAIgenerating
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to generate the complete weight set of a neural network—up to roughly 200 million parameters—with a single diffusion pass on one commodity GPU. It claims this is the first method to reach that scale, and that the generated networks perform on par with normally trained networks on ImageNet-1K classification, ADE20K segmentation, COCO detection, and commonsense reasoning with LLaMA-based LoRA adapters. The approach splits weights into per-layer tokens, uses a recurrent model to build prototypes that summarize cross-token relationships, and lets a 1D diffusion model denoise those tokens conditioned on the prototypes. If correct, it makes 'AI generating AI' practical: entire working networks could be synthesized quickly, generalized to new tasks, and even initialized from text-style prompts.

What carries the argument

The key machinery is parameter tokenization combined with recurrent-conditioned diffusion: layer-wise normalization and uniform chunking turn a network into a sequence of tokens; a recurrent model (Mamba followed by an MLP) produces per-token prototypes encoding global inter-token dependencies; and a 1D convolutional diffusion model denoises random noise into token values conditioned on those prototypes. The recurrent pass captures cross-token correlation while the diffusion pass refines each token, and this two-step decoupling is what keeps memory usage growing mildly with parameter count rather than quadratically.

What would settle it

Generate a network for an unseen task while withholding all training-checkpoint statistics and using only the model's own denoised outputs to invert normalization; if accuracy collapses or the inverse transform is undefined, the single-pass generation claim is not reproducible. Concretely, compute per-layer mean and standard deviation from one held-out checkpoint and reuse them across different generated samples; if performance depends on which checkpoint's statistics are used, the pipeline is not truly synthesizing complete weights from noise.

Watch

Extended reading notes

Core claim

Full-network parameter generation is feasible at practical scale: RPG partitions parameters layer by layer into normalized, uniformly sized tokens, adds a one-hot permutation state to handle weight symmetry, and feeds the token sequence through a recurrent model whose per-token outputs condition a 1D diffusion denoiser. The paper reports that for ConvNeXt-Large (197.8M parameters) and LLaMA-7B DoRA adapters (113.1M parameters) it synthesizes complete parameter sets within minutes on a single H100 with roughly 20GB of memory, matching original accuracy; across seven ImageNet-1K architectures it stays within about 0.5 percent of the trained models; and on 1,022 CIFAR-10 binary tasks it generates valid parameters for unseen tasks without seeing their checkpoints.

Load-bearing premise

The load-bearing assumption is that the per-layer normalization used to create tokens can be inverted at generation time, so denoised normalized tokens can be turned back into real network weights; the paper does not state where the per-layer mean and standard deviation come from when generating for an unseen task.

Editorial extensions

If this is right

  • RPG can synthesize full parameter sets of ConvNeXt-Large and ViT-Base in about one minute on a single H100 with roughly 20GB of memory, moving weight generation from academic demonstrations toward practical deployment.
  • Generated networks match original accuracy on ImageNet-1K, ADE20K, COCO, and seven commonsense reasoning tasks with DoRA/LoRA on LLaMA-7B, so a single trained generator can replace many trained checkpoints.
  • The generator generalizes to unseen CIFAR-10 binary tasks when conditioned on a 10-bit task embedding, meaning a single trained RPG can produce valid models for tasks not in its training set.
  • RPG-initialized parameters speed up downstream training: on an unseen task, an RPG-initialized ViT-Tiny reaches 94.4% accuracy at epoch 0 versus 50.0% for training from scratch.
  • Because the diffusion model is shared across tokens and tokens can be processed sequentially, generation memory can drop below 8GB, allowing deployment on smaller GPUs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the method holds up, the natural next target is cross-architecture generalization: the paper only generates parameters for architectures seen during training, and it explicitly notes that limitation, so conditioning on architecture descriptions rather than fixed token layouts would test whether the learned structure is truly about weights as such.
  • The recurrent-conditioned diffusion recipe may transfer to other high-entropy structured tensors—such as pruning masks, quantization schedules, or optimizer states—where global correlation matters but memory is the bottleneck.
  • One can test the 'unseen task' claim more sharply by conditioning on randomized embeddings far outside the convex hull of the training distribution and measuring whether accuracy degrades gracefully; the current evaluation uses held-out embeddings from the same 1,022-task set.
  • The missing inverse-normalization detail is a reproducibility stake: an independent implementation must decide how per-layer mean and standard deviation are supplied at inference, and if those statistics come from training checkpoints, the claim of generating from pure noise for novel tasks is weakened.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces RPG, a recurrent diffusion framework for generating full neural-network parameter sets. Parameters are layer-wise normalized and tokenized; a recurrent model (Mamba) produces per-token prototypes that condition a 1D diffusion model. The authors report on-par or better accuracy relative to trained models on ImageNet, ADE20K, COCO, and commonsense reasoning with DoRA/LoRA on LLaMA-7B, and demonstrate generalization to unseen binary classification tasks on CIFAR-10. The main scalability claim is generation of up to roughly 200M parameters in a single pass on a single GPU.

Significance. If the claims hold, RPG would be a meaningful step forward in parameter generation, raising the scale by orders of magnitude and offering a practical single-GPU inference pipeline. The combination of recurrent prototypes with token-level diffusion is a sound architectural idea, and the paper includes a broad set of experiments, including held-out task generalization and ablations of the recurrent model, tokenization, and position embeddings. The main limitation is that the paper does not specify the inverse of the layer-wise normalization, without which the synthesized tensors are not demonstrably usable network weights.

major comments (4)
  1. [Sec. 2.2, Eq. (1); Sec. 2.3 Inference] The layer-wise normalization in Eq. (1) maps weights w to (w-μ)/σ, and the diffusion loss in Eq. (5) is computed on these normalized tokens K. The paper never specifies the inverse transform that maps denoised normalized tokens back to raw weights, nor the per-layer μ and σ to be used at synthesis time. For the unseen-task setting in Sec. 4, no training checkpoint exists from which those statistics could be taken. Without an explicit inverse (e.g., multiplying by σ and adding μ, and stripping padding), the generated output is a set of standardized numbers, not the parameters of a usable network; this directly affects the paper's central claim. Please add the inverse transformation and state for each experiment where the μ/σ values come from.
  2. [Sec. 3.1 vs. Sec. B.3, Table 13] The main text states 'The length of parameter tokens, permutation states, position embeddings, and prototypes is set to 8192' (Sec. 3.1), but Appendix B.3 states that ViT-Base, ConvNeXt-Large, ADE20K, COCO, and DoRA rank 64 used RPG-Large with token size 16384. These statements are inconsistent, making the configuration of the headline large-model results ambiguous. Please state the exact token size used for each reported model and correct the default description.
  3. [Sec. 2.2, Eq. (2); Sec. 2.3; Sec. 4] The paper does not explain how padded regions are located and removed after generation, nor how the permutation state S is constructed for unseen tasks. In Sec. 4, binary embeddings are used as conditioning inputs, but the relationship between the binary embedding and S is not defined (e.g., are they concatenated, or does the embedding replace S?). A precise de-tokenization protocol and a definition of S for the unseen-task experiments are needed for the inference procedure to be reproducible.
  4. [Sec. 3.1, Sec. 3.2, Table 1] It is unclear whether the 'original' accuracies in Table 1 refer to the pretrained timm models or to the fine-tuned checkpoints used to create the training set. If the latter, the on-par result is partly a reconstruction of training data; if the former, that should be stated explicitly. The paper should clarify the baseline and, if possible, add an evaluation on a truly held-out architecture checkpoint to strengthen the generalization claim.
minor comments (6)
  1. [General] The PDF contains many garbled 'uni' character sequences (e.g., on pages 1, 3, and 17), indicating an encoding problem; please regenerate the source to a clean text version.
  2. [Eq. (1)] The notation '−µ, /σ' is non-standard; rewrite the normalization as (w[i] − μ_i) / σ_i.
  3. [Table 8] Table 8 is difficult to read: the columns and subscripts for the four baseline methods are misaligned, and the meaning of the subscript numbers is not explained in the caption.
  4. [Fig. 5] The caption refers to 'identical seen binary embeddings' and 'unseen binary embeddings' but does not define the color scheme or the number of parameters visualized; please add explicit axes and legends.
  5. [Appendix C.2, Table 15] Table 15 reports accuracy at epoch 0 for 'RPG initialization + finetune' as 94.4, which seems to correspond to the generated model's zero-shot accuracy; for clarity, distinguish the generated model's zero-shot accuracy from the accuracy after fine-tuning.
  6. [Table 13] The 'parameter counts' of the recurrent model for RPG-Large (3076M) exceeds the generated model size (up to 200M); this is not a flaw but should be highlighted in the efficiency discussion, since the memory and compute of the generator are not negligible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: held-out unseen-task experiments provide genuine prediction; in-distribution main evaluations and an omitted normalization inverse are completeness limitations, not circular reductions.

full rationale

RPG's derivation chain is self-contained: the tokenization/normalization (Eq. 1-2), recurrent prototype conditioning (Eq. 4), and diffusion objective (Eq. 5) define a generative model over parameter tokens, and no predicted quantity is algebraically identical to a fitted input. The generalization claim is supported by an independent held-out experiment in Sec. 4: 1002 seen binary embeddings train RPG, and the remaining unseen embeddings are evaluated without ever accessing their checkpoints during training, so those accuracies are genuine predictions. The ImageNet, ADE20K, COCO, and commonsense-reasoning tables compare generated weights with checkpoints from the same distribution used to collect training data (Sec. 3.1: 'finetune the full parameters ... save 50 checkpoints as the training data'); this is an in-distribution fidelity check rather than an out-of-distribution prediction, and it is not the paper's principal generalization evidence. Self-citations to p-diff (Wang et al., 2024) and COND P-DIFF (Jin et al., 2024) are used as baselines or architectural precedents, not as load-bearing unverified premises. The main genuine gap, that Sec. 2.3 never states the inverse of the layer-wise normalization in Eq. (1) needed to map generated normalized tokens back to raw weights, is an omitted implementation detail and a correctness concern, not a circular reduction. We therefore find no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 2 invented entities

The central claim rests on the unstated inverse-normalization step, the checkpoint collection protocol, and the assumption that token-level diffusion conditioned on recurrent prototypes preserves cross-layer correlations. Token size is the key tuned hyperparameter: for ViT-Base, token size 8192 fails at 45.3% and only 16384 reaches 84.4%, meaning the main results depend on choosing the largest feasible token size. Additionally, the recurrent generator is larger than the generated model, RPG-Large has about 3.3B parameters for a 197.8M target, so single-GPU scaling applies at inference, not training.

free parameters (3)
  • token size k = 8192 default; 16384 for ViT-Base, ConvNeXt-Large and DoRA rank 64 experiments
    The headline results depend on choosing the largest feasible token size. Table 6 shows ViT-Base accuracy collapses from 84.4% at token size 16384 to 45.3% at 8192, making token size a performance-critical tuned hyperparameter.
  • layer-wise normalization statistics mu, sigma = unspecified
    The paper normalizes each layer by its mean and standard deviation in Eq. (1), but the inverse transform that converts generated normalized tokens back to usable weights is never defined, so the effective mapping depends on these statistics.
  • sampling steps for diffusion = 60 for RPG-Large, 1000 for smaller settings
    The number of DDIM or DDPM steps is varied from 20 to 200 in Table 7 and is chosen per model; accuracy is stable across steps, so this is a minor hyperparameter.
assumptions (4)
  • domain assumption The inverse of the layer-wise normalization in Eq. (1) is applied at inference, even though no such step is described.
    Without it, generated normalized tokens are not the actual network weights; the paper never defines how mu and sigma are obtained for generated or unseen models.
  • domain assumption Fine-tuning a pretrained model for one epoch and saving 50 checkpoints produces a training set representative of high-performing weights for that architecture.
    The checkpoint collection in Appendix C.6 assumes these partially fine-tuned states span the useful weight distribution.
  • ad hoc to paper The diffusion training objective in Eq. (5) with random token subsets and a shared 1D CNN is sufficient to learn token-level denoising conditioned on recurrent prototypes.
    The architecture is validated only empirically; no proof is given that the conditioning preserves global correlations across tokens.
  • standard math Standard diffusion theory and the Mamba state-space model are correct.
    The method relies on these off-the-shelf components without modification.
invented entities (2)
  • Prototypes P_j^i
    purpose: Hidden conditioning vectors produced by the recurrent model, one per parameter token, that guide the diffusion denoiser.
    Internal latent representations with no direct falsifiable handle outside the method; their efficacy is shown only indirectly through generated model accuracy.
  • Permutation state S
    purpose: A one-hot vector assigned to each checkpoint to break neural symmetry ambiguities during training.
    A bookkeeping construct; no independent evidence exists beyond the ablation in Appendix C.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Recurrent Diffusion for Large-Scale Parameter Generation." pith.science (2026). https://pith.science/paper/4TNGNSSO

@misc{pith2026250111587,
  author       = {Pith},
  title        = {Pith review of: Recurrent Diffusion for Large-Scale Parameter Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4TNGNSSO}},
  note         = {Machine review of arXiv:2501.11587}
}
read the original abstract

Parameter generation has long struggled to match the scale of today large vision and language models, curbing its broader utility. In this paper, we introduce Recurrent Diffusion for Large Scale Parameter Generation (RPG), a novel framework that generates full neural network parameters up to hundreds of millions on a single GPU. Our approach first partitions a networks parameters into non-overlapping tokens, each corresponding to a distinct portion of the model. A recurrent mechanism then learns the inter token relationships, producing prototypes which serve as conditions for a diffusion process that ultimately synthesizes the full parameters. Across a spectrum of architectures and tasks including ResNets, ConvNeXts and ViTs on ImageNet 1K and COCO, and even LoRA based LLMs RPG achieves performance on par with fully trained networks while avoiding excessive memory overhead. Notably, it generalizes beyond its training set to generate valid parameters for previously unseen tasks, highlighting its flexibility in dynamic and open ended scenarios. By overcoming the longstanding memory and scalability barriers, RPG serves as a critical advance in AI generating AI, potentially enabling efficient weight generation at scales previously deemed infeasible.

Figures

Figures reproduced from arXiv: 2501.11587 by the authors.

Figure 1
Figure 1. Partial roadmap of vision, language, and parameter gen￾eration models. Parameter number in vision or language models is at least 103 times larger than that of generated parameters. as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of parameter processing (left) and inference of recurrent diffusion (right). The recurrent model integrates permutation states and position embeddings, generating prototypes that condition the diffusion model to synthesize the full parameters. where w[i] and wˆ[i] denote the original and normalized parameter blocks of the i-th layer, respectively, and µ, σ are the mean and standard deviation of that lay… view at source ↗
Figure 3
Figure 3. The figure shows the trade-off between accuracy and similarity with ViT-Tiny on ImageNet-1K. The shaded area in￾cludes the approximate range of noise-added checkpoints. This plot demonstrates the strong trade-off between accuracy and simi￾larity and highlights our advantages over trivial interpolation. methods, only our approach consistently achieves the high￾est results and comparable results to original models acr… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: An illustration of our binary embedding strategy and dataset construction. Left: binary embeddings (1022 in total) encode different CIFAR-10 classification tasks, where 1s indicate classes to be classified together (e.g., ‘ship’ and ‘truck’ in the first example). Right…
Figure 5
Figure 5. Figure 5: Illustration of the parameters of original and generated models in seen and unseen embeddings. We select 100 parameters of the classification head and visualize its normalized values. Visualizations of model parameters. We visualize the original and generated models fo…
Figure 6
Figure 6. Figure 6: Principal Component Analysis (PCA) visualization of the classification head. The figures demonstrate the presence of an inherent structure in the parameter space and highlight our method’s effectiveness in capturing this structure for unseen tasks. 16 [PITH_FULL_IMAGE…
Figure 7
Figure 7. Figure 7: Illustration of RPG-generated models guided by binary embeddings from a large language model (Qwen2.5-3B (Yang et al., 2024)), demonstrating neural network parameter generation conditioned by natural language. prompt expected embedding acc. (%) Give me a model to selec…
Figure 8
Figure 8. Figure 8: Training memory cost analysis. Left: GPU memory v.s. token size. Rihgt: GPU memory v.s. parameter counts. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On the Expressive Power of Permutation-Equivariant Weight-Space Networks

    cs.LG 2026-02 conditional novelty 7.0 of 10

    Permutation-equivariant weight-space networks are all equally expressive, and universality holds when hidden-layer biases are pairwise distinct.

  2. Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A prompt-conditioned hyper-network generates LoRA fine-tuning weights for unseen tasks in a single forward pass, without training on the target dataset.

Reference graph

Works this paper leans on

69 extracted references · 34 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    xlstm: Extended long short-term memory

    Beck, M., P \"o ppel, K., Spanring, M., Auer, A., Prudnikova, O., Kopp, M., Klambauer, G., Brandstetter, J., and Hochreiter, S. xlstm: Extended long short-term memory. arXiv preprint arXiv:2405.04517, 2024

    arXiv 2024

  3. [3]

    Piqa: Reasoning about physical commonsense in natural language

    Bisk, Y., Zellers, R., Gao, J., Choi, Y., et al. Piqa: Reasoning about physical commonsense in natural language. In AAAI, volume 34, pp.\ 7432--7439, 2020

    work page 2020

  4. [4]

    Bottou, L. et al. Stochastic gradient learning in neural networks. Proceedings of Neuro-N mes , 91 0 (8), 1991

    work page 1991

  5. [5]

    SMASH : One-shot model architecture search through hypernetworks

    Brock, A., Lim, T., Ritchie, J., and Weston, N. SMASH : One-shot model architecture search through hypernetworks. In ICLR, 2018. URL https://openreview.net/forum?id=rydeCEhs-

    work page 2018

  6. [6]

    Rethinking attention with performers

    Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. Rethinking attention with performers. arXiv preprint arXiv:2009.14794, 2020

    arXiv 2009

  7. [7]

    Diffusion-sdf: Conditional generative modeling of signed distance functions

    Chou, G., Bahat, Y., and Heide, F. Diffusion-sdf: Conditional generative modeling of signed distance functions. In ICCV, pp.\ 2262--2272, 2023

    work page 2023

  8. [8]

    Boolq: Exploring the surprising difficulty of natural yes/no questions

    Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019

    arXiv 1905

Show all 69 references
  1. [9]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  2. [10]

    and Gu, A

    Dao, T. and Gu, A. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060, 2024

    2024 arXiv

  3. [11]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, pp.\ 248--255. Ieee, 2009

    2009

  4. [12]

    and Nichol, A

    Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. In NeurIPS, volume 34, pp.\ 8780--8794, 2021

    2021

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021. URL https://ope...

    2021

  6. [14]

    Hyperdiffusion: Generating implicit neural fields with weight-space diffusion

    Erko c , Z., Ma, F., Shan, Q., Nie ner, M., and Dai, A. Hyperdiffusion: Generating implicit neural fields with weight-space diffusion. In ICCV, 2023

    2023

  7. [15]

    Structural pruning for diffusion models

    Fang, G., Ma, X., and Wang, X. Structural pruning for diffusion models. In NeurIPS, 2023. URL https://openreview.net/forum?id=d4f40zJJIS

    2023

  8. [16]

    and Ghahramani, Z

    Gal, Y. and Ghahramani, Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In ICML. PMLR, 2016

    2016

  9. [17]

    Practical variational inference for neural networks

    Graves, A. Practical variational inference for neural networks. NeurIPS, 24, 2011

    2011

  10. [19]

    and Dao, T

    Gu, A. and Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2024

    2024 arXiv

  11. [20]

    M., and Le, Q

    Ha, D., Dai, A. M., and Le, Q. V. Hypernetworks. In ICLR, 2017. URL https://openreview.net/forum?id=rkpACe1lx

    2017

  12. [21]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, pp.\ 770--778, 2016

    2016

  13. [22]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. In NeurIPS, volume 33, pp.\ 6840--6851, 2020

    2020

  14. [23]

    Untersuchungen zu dynamischen neuronalen netzen

    Hochreiter, S. Untersuchungen zu dynamischen neuronalen netzen. Diploma, Technische Universit \"a t M \"u nchen , 91 0 (1): 0 31, 1991

    1991

  15. [24]

    Long short-term memory

    Hochreiter, S. Long short-term memory. Neural Computation MIT-Press, 1997

    1997

  16. [25]

    Long short-term memory

    Hochreiter, S., Schmidhuber J\" u rgen, Hochreiter, S., and Schmidhuber. Long short-term memory. Neural Comput., 9 0 (8): 0 1735–1780, nov 1997. ISSN 0899-7667. doi:10.1162/neco.1997.9.8.1735. URL https://doi.org/10.1162/neco.1997.9.8.1735

    1997 doi

  17. [26]

    J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lo RA : Low-rank adaptation of large language models. In ICLR, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  18. [27]

    Conditional lora parameter generation

    Jin, X., Wang, K., Tang, D., Zhao, W., Zhou, Y., Tang, J., and You, Y. Conditional lora parameter generation. arXiv preprint arXiv:2408.01415, 2024

    2024 arXiv

  19. [28]

    Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  20. [29]

    and Hinton, G

    Krizhevsky, A. and Hinton, G. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    2009

  21. [30]

    Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. In NeurIPS, volume 25, 2012

    2012

  22. [31]

    L., and Tanaka, H

    Kunin, D., Sagastuy-Brena, J., Ganguli, S., Yamins, D. L., and Tanaka, H. Neural mechanics: Symmetry and broken conservation laws in deep learning dynamics. In ICLR, 2021. URL https://openreview.net/forum?id=q8qLAbQBupm

    2021

  23. [32]

    Autoregressive image generation without vector quantization

    Li, T., Tian, Y., Li, H., Deng, M., and He, K. Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406.11838, 2024 a

    2024 arXiv

  24. [33]

    Exploring plain vision transformer backbones for object detection

    Li, Y., Mao, H., Girshick, R., and He, K. Exploring plain vision transformer backbones for object detection. In ECCV, pp.\ 280--296. Springer, 2022

    2022

  25. [34]

    Text-to-model: Text-conditioned neural network diffusion for train-once-for-all personalization

    Li, Z., Gao, L., and Wu, C. Text-to-model: Text-conditioned neural network diffusion for train-once-for-all personalization. arXiv preprint arXiv:2405.14132, 2024 b

    2024 arXiv

  26. [35]

    Unleash graph neural networks from heavy tuning

    Lin, L., Shi, D., Han, A., Wang, Z., and Gao, J. Unleash graph neural networks from heavy tuning. arXiv preprint arXiv:2405.12521, 2024

    2024 arXiv

  27. [36]

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp....

    2014

  28. [37]

    F., Cheng, K.-T., and Chen, M.-H

    Liu, S.-Y., Wang, C.-Y., Yin, H., Molchanov, P., Wang, Y.-C. F., Cheng, K.-T., and Chen, M.-H. Dora: Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09353, 2024

    2024 arXiv

  29. [38]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., and Zhu, J. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In NeurIPS, volume 35, pp.\ 5775--5787, 2022

    2022

  30. [39]

    Deepcache: Accelerating diffusion models for free

    Ma, X., Fang, G., and Wang, X. Deepcache: Accelerating diffusion models for free. In CVPR, pp.\ 15762--15772, 2024

    2024

  31. [40]

    Introducing meta llama 3: The most capable openly available llm to date

    Meta, A. Introducing meta llama 3: The most capable openly available llm to date. Meta AI, 2024

    2024

  32. [41]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. Can a suit of armor conduct electricity? a new dataset for open book question answering. In EMNLP, 2018

    2018

  33. [42]

    Neal, R. M. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012

    2012

  34. [43]

    Nichol, A. Q. and Dhariwal, P. Improved denoising diffusion probabilistic models. In ICML, pp.\ 8162--8171. PMLR, 2021

    2021

  35. [44]

    T-stitch: Accelerating sampling in pre-trained diffusion models with trajectory stitching

    Pan, Z., Zhuang, B., Huang, D.-A., Nie, W., Yu, Z., Xiao, C., Cai, J., and Anandkumar, A. T-stitch: Accelerating sampling in pre-trained diffusion models with trajectory stitching. arXiv preprint arXiv:2402.14167, 2024

    2024 arXiv

  36. [45]

    and Xie, S

    Peebles, W. and Xie, S. Scalable diffusion models with transformers. In ICCV, pp.\ 4195--4205, 2023

    2023

  37. [46]

    A., and Malik, J

    Peebles, W., Radosavovic, I., Brooks, T., Efros, A. A., and Malik, J. Learning to learn with generative models of neural network checkpoints. arXiv preprint arXiv:2209.12892, 2022

    2022 arXiv

  38. [47]

    Rwkv: Reinventing rnns for the transformer era

    Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Biderman, S., Cao, H., Cheng, X., Chung, M., Grella, M., et al. Rwkv: Reinventing rnns for the transformer era. arXiv preprint arXiv:2305.13048, 2023

    2023 arXiv

  39. [48]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, pp.\ 10684--10695, 2022

    2022

  40. [49]

    Socialiqa: Commonsense reasoning about social interactions

    Sap, M., Rashkin, H., Chen, D., LeBras, R., and Choi, Y. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint arXiv:1904.09728, 2019

    1904 arXiv

  41. [50]

    Hyper-representations as generative models: Sampling unseen neural network weights

    Sch \"u rholt, K., Knyazev, B., Gir \'o -i Nieto, X., and Borth, D. Hyper-representations as generative models: Sampling unseen neural network weights. In NeurIPS, volume 35, pp.\ 27906--27920, 2022 a

    2022

  42. [51]

    Hyper-representations for pre-training and transfer learning

    Sch \"u rholt, K., Knyazev, B., Gir \'o -i Nieto, X., and Borth, D. Hyper-representations for pre-training and transfer learning. arXiv preprint arXiv:2207.10951, 2022 b

    2022 arXiv

  43. [52]

    Model zoos: A dataset of diverse populations of neural network models

    Sch\" u rholt, K., Taskiran, D., Knyazev, B., Gir\' o -i Nieto, X., and Borth, D. Model zoos: A dataset of diverse populations of neural network models. In NeurIPS, 2022

    2022

  44. [53]

    W., and Borth, D

    Sch \"u rholt, K., Mahoney, M. W., and Borth, D. Towards scalable and versatile weight space learning. arXiv preprint arXiv:2406.09997, 2024

    2024 arXiv

  45. [54]

    Temporal dynamic quantization for diffusion models

    So, J., Lee, J., Ahn, D., Kim, H., and Park, E. Temporal dynamic quantization for diffusion models. In NeurIPS, volume 36, 2024

    2024

  46. [55]

    Chaos in random neural networks

    Sompolinsky, H., Crisanti, A., and Sommers, H.-J. Chaos in random neural networks. Physical review letters, 61 0 (3), 1988

    1988

  47. [56]

    Denoising diffusion implicit models

    Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  48. [57]

    Consistency models

    Song, Y., Dhariwal, P., Chen, M., and Sutskever, I. Consistency models. arXiv preprint arXiv:2303.01469, 2023

    2023 arXiv

  49. [58]

    Soro, B., Andreis, B., Lee, H., Chong, S., Hutter, F., and Hwang, S. J. Diffusion-based neural network weights generation. arXiv preprint arXiv:2402.18153, 2024

    2024 arXiv

  50. [59]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  51. [60]

    Attention is all you need

    Vaswani, A. Attention is all you need. In NeurIPS, 2017

    2017

  52. [61]

    Neural network diffusion

    Wang, K., Xu, Z., Zhou, Y., Zang, Z., Darrell, T., Liu, Z., and You, Y. Neural network diffusion. arXiv preprint arXiv:2402.13144, 2024

    2024 arXiv

  53. [62]

    Z., Khabsa, M., Fang, H., and Ma, H

    Wang, S., Li, B. Z., Khabsa, M., Fang, H., and Ma, H. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768, 2020

    2006 arXiv

  54. [63]

    GitHub repository: Pytorch image models

    Wightman, R. GitHub repository: Pytorch image models. GitHub repository, 2019

    2019

  55. [64]

    Unified perceptual parsing for scene understanding

    Xiao, T., Liu, Y., Zhou, B., Jiang, Y., and Sun, J. Unified perceptual parsing for scene understanding. In ECCV, pp.\ 418--434, 2018

    2018

  56. [65]

    Qwen2 technical report

    Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang, J., Yang, J., Tu, J., Zhang, J., Ma, J., Xu, J., Zhou, J., Bai, J., He, J., Lin, J., Dang, K., Lu, K., Chen, K., Yang, K., Li, M., Xue, M.,...

    2024 arXiv

  57. [66]

    Diffusion probabilistic model made slim

    Yang, X., Zhou, D., Feng, J., and Wang, X. Diffusion probabilistic model made slim. In CVPR, pp.\ 22552--22562, 2023

    2023

  58. [67]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  59. [68]

    Dynamic tuning towards parameter and inference efficiency for vit adaptation

    Zhao, W., Tang, J., Han, Y., Song, Y., Wang, K., Huang, G., Wang, F., and You, Y. Dynamic tuning towards parameter and inference efficiency for vit adaptation. arXiv preprint arXiv:2403.11808, 2024

    2024 arXiv

  60. [69]

    Scene parsing through ade20k dataset

    Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., and Torralba, A. Scene parsing through ade20k dataset. In CVPR, 2017

    2017

  61. [70]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.