Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-phase alignment schedule that turns off the teacher loss at a trigger matches vanilla SiT-XL/2 on ImageNet 256x256 in 50 epochs instead of 1400, a 28x reduction in optimization steps.

desk verdict A genuinely useful diagnostic and a plausible two-phase recipe, but the headline speedup rests on a stopping point chosen on the test benchmark, with no validated transferable trigger. read the letter →

arxiv 2505.16792 v1 pith:X2PB6AGQ submitted 2025-05-22 cs.CV cs.AI

Ziqiao Wang , Wangbo Zhao , Yuhao Zhou , Zekai Li , Zhiyuan Liang , Mingjia Shi , Xuanlei Zhao , Pengfei Zhou
show 4 more authors
Kaipeng Zhang Zhangyang Wang Kai Wang Yang You
This is my paper · ORCID
classification cs.CVcs.AI
keywords diffusiontransformersrepresentationalignmentattentiondistillationtrainingaccelerationFIDImageNet256x256text-to-imagegenerationHASTE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that representation alignment (REPA) is a stage-dependent accelerator: it sharply speeds up diffusion transformer training in early epochs, but once the student begins modeling the joint data distribution, the teacher's lower-dimensional embeddings and attention patterns become a straitjacket. To keep the help and drop the hindrance, the authors propose HASTE, a two-phase schedule that first distills both attention maps (relational routing priors) and projected features (semantic anchors) from a frozen DINOv2 teacher into mid-level DiT blocks, then disables the alignment loss entirely at a fixed iteration or gradient-angle trigger. On ImageNet 256x256, HASTE reaches the vanilla SiT-XL/2 baseline FID in 50 epochs and matches REPA's best FID in 500 epochs, a claimed 28x reduction in optimization steps. The practical stake is that external-representation guidance is a cheap, architecture-agnostic way to cut diffusion training cost, and this paper identifies when to use it and when to turn it off.

What carries the argument

The load-bearing mechanism is the gradient-angle cosine $\rho_t = \cos(\nabla_\theta \mathcal{L}_{\text{diff}}, \nabla_\theta \mathcal{L}_{\text{REPA}})$, computed on the 8th block of SiT-XL/2 over 960 ImageNet images; its acute-to-orthogonal-to-obtuse trajectory is both the evidence for the capacity-mismatch diagnosis and the reference for setting the termination trigger. The training recipe is the stage-wise switch of Equation (2): a holistic alignment loss $L_R = \lambda_R L_{\text{REPA}} + \lambda_A L_{\text{ATTA}}$ runs while $n < \tau$ and is dropped afterward, with $L_{\text{ATTA}}$ transferring DINOv2 attention patterns into mid-level student layers and $L_{\text{REPA}}$ transferring projected feature semantics. This lets the student absorb relational routing and semantic anchors early while leaving the shallow noise-processing blocks and final denoising blocks unregularized.

What would settle it

Train SiT-XL/2+HASTE and vanilla SiT-XL/2 on ImageNet 256x256 with at least three independent seeds each and report FID as mean plus standard deviation; the 28x claim holds only if HASTE at 50 epochs is within one standard deviation of vanilla at 1400 epochs. A second check is to hold the termination iteration fixed at 250K and switch to a held-out dataset; if the optimum trigger shifts so much that the recipe breaks, then the fixed-iteration version of HASTE is not the transferable principle.

Watch

Extended reading notes

Core claim

The central discovery is that REPA's gradient relative to the denoising objective passes through three regimes — acute alignment early, orthogonality at a plateau, and obtuse conflict late — and that the late conflict is a capacity mismatch: a frozen non-generative encoder trained for invariant recognition discards the high-frequency detail a mature generative student needs, so continued alignment erases detail. HASTE operationalizes this with a two-phase objective $\mathcal{L} = \mathcal{L}_{\text{diff}} + \lambda_R \mathcal{L}_{\text{REPA}} + \lambda_A \mathcal{L}_{\text{ATTA}}$ for iterations $n < \tau$, then $\mathcal{L} = \mathcal{L}_{\text{diff}}$ afterward, where $\mathcal{L}_{\text{ATTA}}$ is a token-wise cross-entropy between mid-level student attention maps and teacher attention and $\mathcal{L}_{\text{REPA}}$ is token-wise cosine similarity on projected features. Under this schedule, SiT-XL/2+HASTE reaches FID 8.39 at 50 epochs without classifier-free guidance, matching vanilla SiT's FID 8.61 at 1400 epochs, and reaches FID 1.42 with CFG at 500 epochs, matching REPA's best FID of 1.42.

Load-bearing premise

The load-bearing premise is that the reported single-run FID differences are larger than run-to-run noise, and that the termination iteration selected using ImageNet's own FID and gradient measurements transfers to other datasets and model sizes.

Editorial extensions

If this is right

  • A practitioner can match a 1400-epoch vanilla SiT-XL/2 baseline on ImageNet 256x256 with 50 epochs of HASTE, a 28x reduction in optimization steps without classifier-free guidance.
  • Leaving the alignment loss on for the full run is actively harmful, not merely useless: the gradient-angle data predicts that late-stage REPA updates oppose the denoising gradient and erase fine detail.
  • Attention alignment into mid-level blocks transfers relational priors without disturbing the shallow layers that process noisy latents or the final layers that refine the image, so the speed-up needs no architecture change.
  • The same holistic alignment improves MM-DiT text-to-image FID on MS-COCO in the early training stage, suggesting the recipe generalizes beyond class-conditional generation.
  • The gradient-angle trigger offers a dataset-robust alternative to a fixed iteration for choosing when to terminate, since it directly measures when the two objectives start to fight.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the capacity-mismatch explanation is right, a higher-capacity or detail-preserving teacher should push the obtuse-conflict stage later; replacing DINOv2-B with a larger feature-pyramid teacher is a testable extension.
  • The stage-wise termination logic should apply to other auxiliary losses that inject external semantic knowledge, such as U-REPA's manifold alignment or VAE-side distillation, since the same denoising-vs-alignment conflict is likely to arise.
  • The headline 28x figure compares single runs, so a multi-seed replication with confidence intervals is the natural next step before treating the speedup as a hard number.
  • Because HASTE is a schedule rather than an architecture change, combining it with masked or linear-attention DiTs could compound the savings, though the paper does not test that combination.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies REPA, a recently proposed auxiliary loss that aligns DiT hidden features to a frozen DINOv2 teacher. It documents that REPA accelerates early training but becomes harmful later, attributes this to a capacity mismatch between teacher and generative student, and proposes HASTE: a two-phase schedule in which a holistic alignment loss (feature alignment plus attention-map distillation) is applied until a termination iteration tau, after which training continues with only the vanilla denoising objective. On ImageNet 256x256, SiT-XL/2+HASTE is reported to reach the vanilla SiT FID in 50 epochs (8.39 vs 8.61 at 1400 epochs) and to match REPA's best FID in 500 epochs (1.42 vs 1.42), a claimed 28x reduction in optimization steps; additional results are given for DiT-XL and for MM-DiT on MS-COCO.

Significance. The paper has a useful and simple empirical message: representation alignment is beneficial early and should be switched off later, and combining feature and attention alignment gives an additional early boost. The diagnosis via gradient cosine similarity is original, and the ablations in Tables 3 and 4 directly support the two key claims (holistic alignment helps early; termination restores late-stage performance). The code is released, which is a strength. However, as written the method is not a complete recipe: the termination point is chosen post hoc on the same benchmark, the proposed gradient-angle trigger is never tested, and the cross-dataset experiment omits termination entirely. All FID numbers are single runs, so the small margins behind the headline 'matches' claims are not statistically grounded.

major comments (4)
  1. [Section 2.2, Eq. (2), Section 3.2, Table 11] The termination iteration tau is selected post hoc from FID and gradient-observation curves on the same ImageNet benchmark, and the gradient-angle trigger proposed in Section 2.2 is not used in any reported experiment. The text in Section 2.2 states that the gradient rule 'adds robustness across datasets,' but no experiment demonstrates this, and Table 11 fixes tau=100K (SiT-B/2) and tau=250K (SiT-L/2, SiT-XL/2, DiT-XL/2) by inspection of ImageNet FID/gradient behavior. Consequently, the headline 28x claim of Table 1 is not a complete recipe: a user must know tau in advance, which either requires training past the optimum or a validated automatic trigger that the paper does not supply. This is load-bearing because stage-wise termination is the defining component of HASTE.
  2. [Tables 1, 9, 10] All ImageNet FID values are reported as single runs with no error bars or confidence intervals. The central equivalence claims rest on small margins: FID 8.39 vs 8.61 (delta=0.22) for the 50-epoch vanilla-SiT comparison, and FID 1.42 vs 1.42 for the 500-epoch REPA comparison. Given typical run-to-run FID noise on ImageNet 256x256, these margins are within the range that could be explained by seed variation. Please provide multiple seeds with mean and standard deviation for the headline comparisons, or otherwise quantify the FID noise floor.
  3. [Section 3.3, Table 2] The MS-COCO text-to-image experiment explicitly does not apply alignment termination ('we do not apply alignment termination because of limited iteration number'). Table 2 therefore validates only the holistic-alignment component, not HASTE as defined by Eq. (2) and (4), and cannot support the abstract's claim that 'HASTE also improves text-to-image DiTs on MS-COCO.' The claim should be reworded, or termination should be tested in a setting where it can be applied.
  4. [Table 4] The reported benefit of termination for SiT-XL/2 depends on the evaluation checkpoint. At 400K iterations the no-termination run is better (FID 5.5 vs 7.3 for tau=250K), while at 500K iterations the tau=250K run is better (5.3 vs 8.1). The no-termination FID also degrades sharply from 5.5 to 8.1 between 400K and 500K, a large non-monotonic effect that the paper attributes to training oscillation. This makes the choice of tau and the evaluation checkpoint jointly load-bearing and strengthens the concern that the schedule is tuned post hoc rather than following a stable, transferable rule.
minor comments (5)
  1. [Section 5] 'We prove that HASTE can significantly accelerate...' is too strong for an empirical study; use 'show' or 'demonstrate' instead.
  2. [Table 4 caption] The caption should clarify that the 'iteration' column is the evaluation checkpoint and 'tau' is the termination point; as printed, the two columns are easy to conflate.
  3. [Acknowledgments] There is a typo: 'Ackonwledgement' should be 'Acknowledgments'.
  4. [References] Reference [20] is cited as the source of the 'inference pathways' notion for self-attention, but the cited paper ('Do compressed LLMs forget knowledge?') does not appear to discuss attention routing in vision transformers; a directly relevant reference is needed.
  5. [Table 1] The entries for DiT+REPA are missing sFID, IS, Precision, and Recall; please state why these metrics are omitted.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the paper is an empirical recipe paper whose central claims are measured, not derived, and its self-citations are not load-bearing.

full rationale

The paper makes no formal derivation claim that reduces to its inputs. The alignment losses (Eq. 1, Eq. 3) and the termination schedule (Eq. 2) are stated as training objectives; the reported FIDs are evaluated after training on held-out samples, so there is no constructional equivalence between objective and evaluation. The gradient-angle analysis (Sec. 2.2 and Appendix A.1) is a diagnostic that motivates the termination point, not a derivation of the final FID. The self-citations (e.g., Refs. [47], [48], [49], which include current authors) appear in related-work and background contexts and are not used to justify the central claim; no uniqueness theorem or ansatz is imported from the authors' prior work. The main legitimate concern is that the termination iteration tau and hyperparameters (lambda_A, aligned layers) are selected after inspecting ImageNet FID curves and gradient similarity on the same benchmark (Secs. 3.2, 3.4, Tables 5-6), so the headline 28x comparison is an in-sample, post-hoc checkpoint selection rather than a validated transferable prediction from the proposed gradient trigger. This is a reproducibility and statistical-selection concern, not a definitional circularity: the qualitative benefit of termination is supported by multiple tau values in Table 4, and the FID numbers are genuine measurements of trained models.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central result rests on several tuned hyperparameters (termination point, attention weight, layer choices) and on the premise that DINOv2 guidance is beneficial early but harmful late. No new physical entities are introduced.

free parameters (6)
  • Termination iteration tau for large models (SiT-L/2, SiT-XL/2, DiT-XL/2) = 250K iterations
    Ablations in Table 4 show tau=400K gives better FID at 400K but worse at 500K; tau=250K is chosen from observed FID and gradient-angle trends on ImageNet.
  • Termination iteration tau for SiT-B/2 = 100K iterations
    Set per model size; the smaller model terminates earlier because its training dynamics differ.
  • Attention alignment weight lambda_A = 0.5
    Tuned in Table 5 on SiT-L/2; larger weights degrade FID.
  • Feature alignment weight lambda_R = 0.5
    Inherited from REPA and kept fixed for fair comparison.
  • Student-teacher attention layer pairs = student blocks 4-7, teacher DINOv2-B layers 8-11 for large models
    Selected via ablation in Table 6; fewer or shallower pairs reduce FID.
  • Feature alignment depth = block 8 for large models, block 5 for SiT-B/2
    Follows REPA's default alignment depth; not independently tuned here.
assumptions (4)
  • domain assumption Frozen DINOv2 features and attention maps are a useful supervision signal for mid-level DiT blocks during early training.
    Phase I's entire design rests on this; experiments support it on ImageNet but no independent evidence is given across architectures or tasks.
  • domain assumption Gradient cosine similarity measured at the 8th block over 960 images and a few timesteps represents whether alignment helps or hurts the whole model.
    Section 2.2 and Appendix A.1 use this signal to justify termination; it is a small-sample proxy, not a proven diagnostic.
  • ad hoc to paper The teacher's capacity is lower than the student's once the student models the joint data distribution, so alignment becomes a constraint.
    The capacity-mismatch conjecture in Section 2.2 is post hoc and not derived; it is plausible but not independently tested.
  • domain assumption Attention maps from DINOv2-B layers 8-11 map meaningfully onto SiT layers 4-7.
    Layer pairing is chosen empirically in Table 6; the correspondence is assumed for the holistic loss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training." pith.science (2026). https://pith.science/paper/X2PB6AGQ

@misc{pith2026250516792,
  author       = {Pith},
  title        = {Pith review of: REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X2PB6AGQ}},
  note         = {Machine review of arXiv:2505.16792}
}
read the original abstract

Diffusion Transformers (DiTs) deliver state-of-the-art image quality, yet their training remains notoriously slow. A recent remedy -- representation alignment (REPA) that matches DiT hidden features to those of a non-generative teacher (e.g. DINO) -- dramatically accelerates the early epochs but plateaus or even degrades performance later. We trace this failure to a capacity mismatch: once the generative student begins modelling the joint data distribution, the teacher's lower-dimensional embeddings and attention patterns become a straitjacket rather than a guide. We then introduce HASTE (Holistic Alignment with Stage-wise Termination for Efficient training), a two-phase schedule that keeps the help and drops the hindrance. Phase I applies a holistic alignment loss that simultaneously distills attention maps (relational priors) and feature projections (semantic anchors) from the teacher into mid-level layers of the DiT, yielding rapid convergence. Phase II then performs one-shot termination that deactivates the alignment loss, once a simple trigger such as a fixed iteration is hit, freeing the DiT to focus on denoising and exploit its generative capacity. HASTE speeds up training of diverse DiTs without architecture changes. On ImageNet 256X256, it reaches the vanilla SiT-XL/2 baseline FID in 50 epochs and matches REPA's best FID in 500 epochs, amounting to a 28X reduction in optimization steps. HASTE also improves text-to-image DiTs on MS-COCO, demonstrating to be a simple yet principled recipe for efficient diffusion training across various tasks. Our code is available at https://github.com/NUS-HPC-AI-Lab/HASTE .

Figures

Figures reproduced from arXiv: 2505.16792 by the authors.

Figure 1
Figure 1. Training SiT-XL/2 on ImageNet 256×256. Adding REPA slashes FID early on, but its benefit fades and ultimately re￾verses; dropping the alignment loss mid￾training restores progress. Our Conjecture: Capacity mismatch incurrs the hidden turning point. Diffusion models eventu￾ally model the joint data distribution, a harder ob￾jective than the marginal/conditional targets implicit in a frozen, non-generative encoder. Co… view at source ↗
Figure 2
Figure 2. Overview of our framework. Phase I (left) distills both feature embeddings and attention maps from a frozen, non-generative teacher (DINOv2) into mid-level layers of the student DiT. When a simple trigger τ fires, the alignment loss is disabled; Phase II (right) then continues training with pure denoising. Our framework, HASTE, couples two ingredients (see [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Cosine similarity between REPA and de￾noising gradients. Acute → orthogonal → obtuse: the auxiliary signal turns from booster to brake. The cross–over coincides with the iteration where [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (24 more)
Figure 4
Figure 4. Figure 4: Gradient similarity as function of diffu [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: Replacing teacher inputs with low-pass images leaves REPA’s early gain intact: evidence that the auxiliary loss transmits mainly global structure. We train SiT-L/2 for 200K iterations. Take-away. REPA supplies valuable global context but obstructs local detail once the…
Figure 6
Figure 6. Figure 6: Evaluating cross-effects between feature and attention alignment. (a) Attention map [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]
Figure 8
Figure 8. Figure 8: HASTE improves visual scaling. We compare images generated by SiT-XL/2+REPA and [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 7
Figure 7. Figure 7: Comparison of different termination point τ on SiT-XL/2. We observe the training os￾cillation after 250K iteration. Using τ = 250 K leads to better performance at 400K iteration. Different Attention Alignment loss weight λA. We evaluate the sensitivity of model to the …
Figure 9
Figure 9. Figure 9: , we separately compute gradients of the feature alignment and the denoising objective for SiT-XL/2 [34] and compare the cosine similarity of their directions at different training iterations. Specifically, we randomly sample 960 images from the training dataset of Ima…
Figure 10
Figure 10. Figure 10: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p020_18.png]
Figure 19
Figure 19. Figure 19: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p020_19.png]
Figure 20
Figure 20. Figure 20: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p021_20.png]
Figure 21
Figure 21. Figure 21: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p021_21.png]
Figure 22
Figure 22. Figure 22: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p022_22.png]
Figure 23
Figure 23. Figure 23: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p022_23.png]
Figure 24
Figure 24. Figure 24: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p023_24.png]
Figure 25
Figure 25. Figure 25: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p023_25.png]
Figure 26
Figure 26. Figure 26: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p024_26.png]
Figure 27
Figure 27. Figure 27: Uncurated generation results of SiT-XL/2+HASTE. We use classifier-free guidance [PITH_FULL_IMAGE:figures/full_fig_p024_27.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Under One Sun: Multi-Object Generative Perception of Materials and Illumination

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Factorizing video editing into semantic-token anchoring and motion-restoration pre-training produces strong zero-shot and SOTA open-source instruction-guided video edits without heavy external structural priors.

  2. Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?

    cs.CV 2026-01 conditional novelty 6.0 of 10

    A self-supervised two-stage training method—VAE-latent feature alignment then feature-level classifier-free guidance—lets DiT models match or beat DINO-guided REPA training without any external feature extractor.

  3. Align & Invert: Solving Inverse Problems with Diffusion and Flow-based Models via Representation Alignment

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Adding DINOv2 representation alignment to diffusion/flow inverse-problem solvers, using corrupted measurements as proxies, improves LPIPS/FID and cuts required sampling steps.

Reference graph

Works this paper leans on

57 extracted references · 20 canonical work pages · cited by 3 Pith papers

  1. [1]

    All are worth words: A vit backbone for diffusion models

    Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. InCVPR, 2023

    2023

  2. [2]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. https://openai.com/research/video-generation-models-as-world-simulators, 2024

    work page 2024

  3. [3]

    Rethinking attention with performers

    Krzysztof Marcin Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Quincy Davis, Afroz Mohiuddin, Lukasz Kaiser, David Benjamin Belanger, Lucy J Colwell, and Adrian Weller. Rethinking attention with performers. InICLR, 2021

    2021

  4. [4]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. InCVPR, 2009

    2009

  5. [5]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. InNeurIPS, 2021

    2021

  6. [6]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InICLR, 2021

    2021

  7. [7]

    Sigmoid-weighted linear units for neural network function approximation in reinforcement learning.Neural Networks, 107:3–11, 2018

    Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning.Neural Networks, 107:3–11, 2018. ISSN 0893-6080. doi: https:// doi.org/10.1016/j.neunet.2017.12.012. URL https://www.sciencedirect.com/science/article/ pii/S0893608017302976. Special issue on deep reinforcement learning

    doi:10.1016/j.neunet.2017.12.012 2018

  8. [8]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. InICML, 2024

    2024

Show all 57 references
  1. [9]

    Dit4edit: Diffusion transformer for image editing

    Kunyu Feng, Yue Ma, Bingyuan Wang, Chenyang Qi, Haozhe Chen, Qifeng Chen, and Zeyu Wang. Dit4edit: Diffusion transformer for image editing. InAAAI, 2025

    2025

  2. [10]

    Masked diffusion transformer is a strong image synthesizer

    Shanghua Gao, Pan Zhou, Mingg-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer. InICCV, 2023

    2023

  3. [11]

    Mdtv2: Masked diffusion transformer is a strong image synthesizer.arXiv preprint arXiv:2303.14389, 2024

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Mdtv2: Masked diffusion transformer is a strong image synthesizer.arXiv preprint arXiv:2303.14389, 2024. URL https://arxiv.org/abs/ 2303.14389

    2024 arXiv

  4. [12]

    Efficient diffusion training via min-snr weighting strategy

    Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Baining Guo. Efficient diffusion training via min-snr weighting strategy. InICCV, 2023

    2023

  5. [13]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. InCVPR, 2020

    2020

  6. [15]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. InCVPR, 2022

    2022

  7. [16]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. InNeurIPS, 2017

    2017

  8. [17]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. InNeurIPS Workshop, 2021

    2021

  9. [18]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InNeurIPS, 2020. 10

    2020

  10. [19]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. InNeurIPS, 2022

    2022

  11. [20]

    Do compressed llms forget knowledge? an experimental study with practical implications

    Scott Hoang, Minsik Cho, Thomas Merth, Atlas Wang, Mohammad Rastegari, and Devang Naik. Do compressed llms forget knowledge? an experimental study with practical implications. InNeurIPS Workshop, 2024

    2024

  12. [21]

    Analyzing and improving the training dynamics of diffusion models

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. InCVPR, 2024

    2024

  13. [22]

    Transformers are rnns: fast autoregressive transformers with linear attention

    Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are rnns: fast autoregressive transformers with linear attention. InICML, 2020

    2020

  14. [23]

    Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation

    Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and Il-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. InICML, 2022

    2022

  15. [24]

    Understanding diffusion objectives as the elbo with simple data augmentation

    Diederik Kingma and Ruiqi Gao. Understanding diffusion objectives as the elbo with simple data augmentation. InNeurIPS, 2023

    2023

  16. [25]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InICLR, 2015

    2015

  17. [26]

    Tread: Token routing for efficient architecture-agnostic diffusion training.arXiv preprint arXiv:2501.04765, 2025

    Felix Krause, Timy Phan, Vincent Tao Hu, and Björn Ommer. Tread: Token routing for efficient architecture-agnostic diffusion training.arXiv preprint arXiv:2501.04765, 2025

    2025

  18. [27]

    Improved precision and recall metric for assessing generative models

    Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. InNeurIPS, 2019

    2019

  19. [28]

    Applying guidance in a limited interval improves sample and distribution quality in diffusion models

    Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. In NeurIPS, 2024

    2024

  20. [29]

    Flux.https://github.com/black-forest-labs/flux, 2024

    Black Forest Labs. Flux.https://github.com/black-forest-labs/flux, 2024

    2024

  21. [31]

    On the surprising effectiveness of attention transfer for vision transformers

    Alexander Cong Li, Yuandong Tian, Beidi Chen, Deepak Pathak, and Xinlei Chen. On the surprising effectiveness of attention transfer for vision transformers. InNeurIPS, 2024

    2024

  22. [32]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. InECCV, 2014

    2014

  23. [33]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InICLR, 2017

    2017

  24. [34]

    Albergo, Nicholas M

    Nanye Ma, Mark Goldstein, Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740, 2024. URLhttps://arxiv.org/abs/2401.08740

    2024 arXiv

  25. [35]

    Generating images with sparse representations

    Charlie Nash, Jacob Menick, Sander Dieleman, and Peter Battaglia. Generating images with sparse representations. InICML, 2021

    2021

  26. [36]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rab...

    2024

  27. [37]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InICCV, 2023

    2023

  28. [38]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InICML, 2021

    2021

  29. [39]

    Tinymim: An empirical study of distilling mim pre-trained models

    Sucheng Ren, Fangyun Wei, Zheng Zhang, and Han Hu. Tinymim: An empirical study of distilling mim pre-trained models. InCVPR, 2023. 11

    2023

  30. [40]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InCVPR, 2022

    2022

  31. [41]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMICCAI, 2015

    2015

  32. [42]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. InNeurIPS, 2016

    2016

  33. [43]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. InICML, 2015

    2015

  34. [44]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. InICLR, 2021

    2021

  35. [45]

    U-repa: Aligning diffusion u-nets to vits.arXiv preprint arXiv:2503.18414, 2025

    Yuchuan Tian, Hanting Chen, Mengyu Zheng, Yuchen Liang, Chao Xu, and Yunhe Wang. U-repa: Aligning diffusion u-nets to vits.arXiv preprint arXiv:2503.18414, 2025. URL https://arxiv.org/abs/2503. 18414

    2025

  36. [46]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNeurIPS, 2017

    2017

  37. [47]

    Lit: Delving into a simplified linear diffusion transformer for image generation.arXiv preprint arXiv:2501.12976, 2025

    Jiahao Wang, Ning Kang, Lewei Yao, Mengzhao Chen, Chengyue Wu, Songyang Zhang, Shuchen Xue, Yong Liu, Taiqiang Wu, Xihui Liu, Kaipeng Zhang, Shifeng Zhang, Wenqi Shao, Zhenguo Li, and Ping Luo. Lit: Delving into a simplified linear diffusion transformer for image generation.ar...

    2025

  38. [48]

    Attention distillation: self-supervised vision transformer students need more guidance

    Kai Wang, Fei Yang 0004, and Joost van de Weijer 0001. Attention distillation: self-supervised vision transformer students need more guidance. InBMVC, 2022

    2022

  39. [49]

    A closer look at time steps is worthy of triple speed-up for diffusion model training

    Kai Wang, Mingjia Shi, Yukun Zhou, Zekai Li, Zhihang Yuan, Yuzhang Shang, Xiaojiang Peng, Hanwang Zhang, and Yang You. A closer look at time steps is worthy of triple speed-up for diffusion model training. arXiv preprint arXiv:2405.17403, 2024

    2024 arXiv

  40. [50]

    Linformer: Self-attention with linear complexity.arXiv preprint arXiv:2006.04768, 2020

    Sinong Wang, Belinda Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-attention with linear complexity.arXiv preprint arXiv:2006.04768, 2020

    2006 arXiv

  41. [51]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformer.arXiv preprint arXiv:2410.10629, 2024

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. Sana: Efficient high-resolution image synthesis with linear diffusion transformer.arXiv preprint arXiv:2410.10629, 2024. URLhttps://arxiv.org/abs/2410.10629

    arXiv

  42. [53]

    Reconstruction vs

    Jingfeng Yao and Xinggang Wang. Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models. InCVPR, 2025

    2025

  43. [54]

    URLhttps://arxiv.org/abs/2501.18427

    arXiv

  44. [55]

    Representation alignment for generation: Training diffusion transformers is easier than you think

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. In ICLR, 2025

    2025

  45. [56]

    Fasterdit: Towards faster diffusion transformers training without architecture modification

    Jingfeng Yao, Cheng Wang, Wenyu Liu, and Xinggang Wang. Fasterdit: Towards faster diffusion transformers training without architecture modification. InNeurIPS, 2024

    2024

  46. [57]

    Attention distillation: A unified approach to visual characteristics transfer.arXiv preprint arXiv:2502.20235, 2025

    Yang Zhou, Xu Gao, Zichong Chen, and Hui Huang. Attention distillation: A unified approach to visual characteristics transfer.arXiv preprint arXiv:2502.20235, 2025. URL https://arxiv.org/abs/2502. 20235

    2025 arXiv

  47. [58]

    Fast training of diffusion models with masked transformers.TMLR, 2024

    Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers.TMLR, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id= vTBjBtGioE

    2024

  48. [2024]

    loggerhead sea turtle

    URLhttps://arxiv.org/abs/2405.18428. 12 A Additional Results A.1 Gradient Angle We provide detailed results of cosine similarity between REPA [ 55] and denoising gradients. In Figure 9, we separately compute gradients of the feature alignment and the denoising objective for Si...

    1928 arXiv

  49. [2025]

    URLhttps://arxiv.org/abs/2504.10483

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.