Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

DanceOPD composes conflicting image-generation skills into one flow model by matching one expert velocity field on one student-visited low-noise state.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 11:44 UTC pith:DRT274QK

load-bearing objection Solid multi-capability OPD recipe with real ablations; the Table 2 wins need a same-init re-run before you trust the ranking. the 3 major comments →

arxiv 2606.27377 v2 pith:DRT274QK submitted 2026-06-25 cs.CV cs.CLcs.LG

DanceOPD: On-Policy Generative Field Distillation

Wei Zhou , Xiongwei Zhu , Zelin Xu , Bo Dong , Lixue Gong , Yongyuan Liang , Meng Chu , Leigang Qu
show 3 more authors
Lingdong Kong Wei Liu Tat-Seng Chua
This is my paper
classification cs.CV cs.CLcs.LG
keywords on-policy distillationflow matchingvelocity fieldsmulti-capability compositionimage editingtext-to-imageclassifier-free guidancegenerative field distillation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern image generators are asked to do text-to-image, local editing, and global editing at once, but those skills pull the model in different directions and usually degrade one another. DanceOPD treats each frozen expert as a velocity field on a shared flow state space and trains a single student by hard-routing every sample to exactly one field, querying that field at one low-noise state from the student's own rollout, and regressing student velocity to teacher velocity with plain mean-squared error. Because supervision sits on the states the student actually visits, the student is not left under-trained on its own generation path; because only one field and one state supervise each sample, the method avoids mixed targets and correlated trajectory gradients. The same recipe can absorb operator-defined fields such as classifier-free guidance. Across T2I-plus-edit, local-plus-global edit, realism absorption, and CFG absorption, the paper reports stronger target skills with preserved base generation quality, at lower per-step cost than denser on-policy baselines.

Core claim

Multi-capability image generation can be cast as on-policy generative field distillation: hard-routed sample-wise matching of frozen velocity fields on one semantic-side, stop-gradient student rollout state, optimized with plain velocity MSE, composes and absorbs heterogeneous capabilities into a single flow-matching student while preserving the anchor generation skill better than joint training, weight merging, off-policy distillation, or denser on-policy alternatives.

What carries the argument

On-policy generative field distillation (DanceOPD): each sample is hard-routed to one frozen capability velocity field, that field is queried at a single low-noise student-induced state, and the student is updated by velocity mean-squared error.

Load-bearing premise

All expert sources must already speak the same language: compatible velocity fields over one shared generative state space, backbone family, latent, scheduler, and velocity parameterization.

What would settle it

Compose the same T2I and edit teachers under the paper's protocol but replace hard single-field routing plus one low-noise student-state query with soft multi-teacher mixing or dense same-rollout queries; if GEditBench and GenEval no longer improve over the reported baselines while anchor T2I is preserved, the claimed query design fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DanceOPD, an on-policy generative field distillation method for flow-matching image models that composes heterogeneous capabilities (T2I, local/global editing, realism, CFG) into one student. Each frozen source is treated as a velocity field on a shared state space; training hard-routes each sample to one field, queries a single stop-gradient low-noise state from the current student rollout, and matches velocities with plain MSE (Eq. 6; Alg. 1). The authors argue this addresses target-field ambiguity, state-distribution mismatch, and trajectory-query correlation, and report gains over joint training, weight merging, off-policy distillation, DiffusionOPD, and Flow-OPD on GEditBench and GenEval (Table 2), plus realism and CFG absorption (Fig. 6, Table 6), with ablations on routing, query location, density, objective, and initialization (Tables 7–9).

Significance. If the causal claim holds, this is a useful and practical contribution for multi-capability post-training of flow models: a simple, deployable recipe that internalizes composition rather than relying on data mixing, parameter merging, or inference-time score composition. Strengths include a clear three-choice design space, multi-setting evaluation (composition and field absorption), extensive ablations, and appendix analyses (KL–MSE reduction, hard-route bias, dense-query correlation). The shared-field assumption is stated honestly in Sec. 6. The main significance risk is attribution: gains may partly reflect initialization and reimplementation details rather than the field-query recipe alone.

major comments (3)
  1. Sec. 4.3 and Table 8 show initialization is load-bearing: at 2k steps, local-edit init improves GEditBench avg by 37.2% over merged, 112.8% over global-edit, and 204.4% over T2I. Sec. 8.5/Table 5 default DanceOPD from the attribute/local-edit SFT LoRA, while Sec. 8.3 states Flow-OPD uses merge init for both composition blocks, and Table 2 does not state that DiffusionOPD, off-policy, joint training, or weight-merge rows share the same local-edit start. Without a matched-init control, the Table 2 claim that hard-routed single-query MSE is what drives the composition gains is not fully isolated.
  2. Sec. 8.2: DiffusionOPD is a paper-faithful reimplementation (code unavailable), and Sec. 8.3 disables Flow-OPD’s MAR component for lack of a Z-Image teacher. Table 1 and Table 2 present these as representative OPD baselines. The paper should either (i) re-run Table 2 with official/matched recipes and identical init/rollout budgets, or (ii) clearly label them as reimplementations and report sensitivity to the missing pieces, so the multi-capability superiority claim is not overstated relative to the original methods.
  3. Sec. 4.1.C / Fig. 6(a) / Sec. 8.5: realism absorption is measured with a proprietary reward on 200 held-out samples. The reported 9.9% gain and 85.3% gap closure are central to the field-absorption claim, but cannot be independently verified. Please add at least one public realism/preference metric (or release the reward protocol) and report variance over seeds/samples so the absorption result is reproducible.
minor comments (4)
  1. Fig. 1 left panel is hard to parse (dense metric grids without a clear legend mapping); a compact table or clearer color/key would help.
  2. Notation for semantic coordinate s vs physical t (Sec. 3.4, Table 3) is careful in the appendix but could be introduced earlier in the main text to avoid confusion with standard diffusion time.
  3. Sec. 6 limitations are appropriate; a short forward-looking note on verifier-based routing for multi-capability prompts would strengthen the discussion.
  4. Minor consistency: abstract claims “slightly exceeding the T2I source on GenEval” while Table 2 T2I source overall is 0.832 vs DanceOPD 0.849—state the absolute numbers once in the abstract for precision.

Circularity Check

0 steps flagged

No significant circularity: empirical field-distillation design with standard Gaussian KL o MSE reduction and external-benchmark evaluation; results are not forced by definition or self-citation.

full rationale

DanceOPD is an empirical methods paper whose central claim is that hard-routed, on-policy, single low-noise velocity-MSE distillation composes frozen flow-matching capability fields better than joint training, merging, off-policy distillation, DiffusionOPD, and Flow-OPD. That claim is tested on external benchmarks (GEditBench-EN, GenEval) and held-out realism samples, not predicted from fitted parameters renamed as forecasts. The only formal reduction in the appendix (Sec. 7.1, Eqs. 8–10) is the standard equal-covariance Gaussian fact that local transition KL reduces to weighted velocity MSE; the paper then chooses unweighted MSE by ablation, not by tautology. Hard routing, on-policy stop-gradient queries, and single-query correlation arguments (Secs. 7.2–7.4, 7.8) are design motivations with algebraic bias/variance forms, not uniqueness theorems imported from the authors. The operational definition of composition (improve target metrics while preserving anchor metrics) is an evaluation target, not a derivation that forces the Table 2 gains. Self-citations are baseline/related-work citations, not load-bearing uniqueness results. Initialization confounding (local-edit init vs merge init) is an experimental-design risk, not circularity of the derivation chain. Score 0.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central claim rests on standard flow-matching machinery plus several design choices treated as defaults: shared velocity support, hard routing, one low-noise student query, and plain MSE. Free parameters are training/query knobs rather than fitted physical constants; invented entities are methodological constructs, not new physical objects.

free parameters (5)
  • semantic-side Beta query (alpha_sem, beta_sem)
    Default Beta(5,2) low-t bias is a hand-chosen query distribution; ablations show strong sensitivity to low/median/high-t choices.
  • training rollout length N
    Default 16-step Euler ODE rollout is selected by ablation; not forced by theory.
  • route probabilities pi
    Uniform 1:1 or 1:1:1 routing over active buckets is a design choice that controls capability balance.
  • LoRA rank / learning rate / optimizer schedule
    Rank 128, AdamW 2e-4, accumulation 4, and related training knobs are free engineering parameters of the reported student.
  • CFG absorption scales alpha, beta
    Train-time absorbed guidance and eval-time external CFG are chosen scales whose product approximately sets effective guidance.
axioms (5)
  • domain assumption Capability sources define compatible velocity fields over a shared flow state space.
    Stated in Sec. 3.1 and Sec. 6; required for field querying and MSE matching.
  • standard math Local Gaussian transition kernels with shared covariance make KL velocity matching reduce to weighted MSE.
    Appendix Sec. 7.1 derives the local KL-MSE equivalence under isotropic Gaussian kernels.
  • domain assumption Student-visited states remain close enough to the flow marginal for frozen fields to remain locally meaningful.
    Sec. 7.3 Lipschitz/smoothness bound justifies on-policy querying only in a neighborhood regime.
  • ad hoc to paper Hard sample-wise routing preserves semantic identity better than soft multi-teacher mixing under conflicting capabilities.
    Core design claim of Sec. 3.2; supported empirically but not derived as a general theorem.
  • ad hoc to paper One low-noise semantic-side query avoids harmful trajectory-query correlation better than dense same-rollout supervision.
    Sec. 3.4 and Sec. 7.8/7.9 motivate K=1; dense-query failures are diagnosed rather than universally proved.
invented entities (2)
  • DanceOPD training query (hard route + stop-gradient student state + single low-t query + velocity MSE) no independent evidence
    purpose: Package the three design choices into one on-policy generative field distillation procedure.
    Methodological construct; independent evidence is empirical comparison against baselines, not an external physical measurement.
  • Operator-defined capability field (e.g., CFG-guided velocity field) independent evidence
    purpose: Treat guidance operators as absorbable fields under the same MSE objective.
    CFG itself is prior art; casting it as a distillable capability field is a paper-specific framing.

pith-pipeline@v1.1.0-grok45 · 40915 in / 3223 out tokens · 29118 ms · 2026-07-12T11:44:05.700457+00:00 · methodology

0 comments
read the original abstract

Modern image generation demands a single model that unifies diverse capabilities, including text-to-image (T2I), local editing, and global editing. However, these capabilities are rarely naturally aligned and often conflict. For instance, editing tends to degrade T2I performance, while global and local editing interfere with each other. Consequently, effectively composing these capabilities has become a central challenge for image generation model training. To tackle this, we introduce DanceOPD, an on-policy generative field distillation framework for flow-matching models that routes each sample to one capability field, queries one low-noise student-induced state, and trains with a simple velocity MSE objective. With each capability source defined as a velocity field over the shared flow state space, the student learns from fields queried on its own rollout states to compose expert capabilities. This formulation also absorbs operator-defined fields such as classifier-free guidance. Comprehensive experiments on T2I, editing, realism-field absorption, and CFG absorption show that our approach improves multi-capability composition, strengthening target capabilities while preserving anchor generation quality. We believe this work establishes a practical route for generative field distillation in flow-matching models.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

    cs.CV 2026-07 conditional novelty 6.0

    CFG-composed on-policy distillation is branch-underidentified; privileged negative conditioning causes Negative Branch Asymmetry, which Positive–Direction Matching fixes.

  2. FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models

    cs.LG 2026-07 conditional novelty 5.5

    Trajectory-derived, temporally weighted velocity matching from shared student states outperforms KL-based on-policy distillation for multi-reference flow model post-training.

Reference graph

Works this paper leans on

111 extracted references · 41 linked inside Pith · cited by 2 Pith papers

  1. [1]

    On-policy distillation of language models: Learning from self-generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In International Conference on Learning Representations, volume 2024, pages 21246–21263, 2024

    2024

  2. [2]

    Variational information distillation for knowledge transfer

    Sungsoo Ahn, Shell Xu Hu, Andreas Damianou, Neil D Lawrence, and Zhenwen Dai. Variational information distillation for knowledge transfer. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9163–9171, 2019

    2019

  3. [3]

    Git re-basin: Merging models modulo permutation symmetries

    Samuel K Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models modulo permutation symmetries. arXiv preprint arXiv:2209.04836, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Stochastic interpolants: A unifying framework for flows and diffusions.Journal of Machine Learning Research, 26(209):1–80, 2025

    Michael Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions.Journal of Machine Learning Research, 26(209):1–80, 2025

    2025

  5. [5]

    HumanEdit: A high-quality human-rewarded dataset for instruction-based image editing.arXiv preprint arXiv:2412.04280, 2024

    Jinbin Bai, Wei Chow, Ling Yang, Xiangtai Li, Juncheng Li, Hanwang Zhang, and Shuicheng Yan. HumanEdit: A high-quality human-rewarded dataset for instruction-based image editing.arXiv preprint arXiv:2412.04280, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    MultiDiffusion: Fusing diffusion paths for controlled image generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. MultiDiffusion: Fusing diffusion paths for controlled image generation. InInternational Conference on Machine Learning, pages 1737–1752. PMLR, 2023

    2023

  7. [7]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. InInternational Conference on Learning Representations, volume 2024, 2024

    2024

  8. [8]

    InstructPix2Pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. InstructPix2Pix: Learning to follow image editing instructions. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023

    2023

  9. [9]

    Z-Image: An efficient image generation foundation model with single-stream diffusion transformer

    Huanqia Cai, Sihan Cao, Ruoyi Du, Peng Gao, Steven Hoi, Zhaohui Hou, Shijie Huang, Dengyang Jiang, Xin Jin, Liangchen Li, et al. Z-Image: An efficient image generation foundation model with single-stream diffusion transformer. arXiv preprint arXiv:2511.22699, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models.ACM Transactionson Graphics, 42(4):1–10, 2023

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models.ACM Transactionson Graphics, 42(4):1–10, 2023

    2023

  11. [11]

    TINO-Edit: Timestep and noise optimization for robust diffusion-based image editing

    Sherry X Chen, Yaron Vaxman, Elad Ben Baruch, David Asulin, Aviad Moreshet, Kuo-Chin Lien, Misha Sra, and Pradeep Sen. TINO-Edit: Timestep and noise optimization for robust diffusion-based image editing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6337–6346, 2024

    2024

  12. [12]

    An empirical study of GPT-4o image generation capabilities.arXiv preprint arXiv:2504.05979, 2025

    Sixiang Chen, Jinbin Bai, Zhuoran Zhao, Tian Ye, Qingyu Shi, Donghao Zhou, Wenhao Chai, Xin Lin, Jianzong Wu, Chao Tang, et al. An empirical study of GPT-4o image generation capabilities.arXiv preprint arXiv:2504.05979, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks

    Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks. InInternational Conference on Machine Learning, pages 794–803. PMLR, 2018

    2018

  14. [14]

    Just pick a sign: Optimizing deep multitask models with gradient sign dropout.Advances in Neural Information Processing Systems, 33:2039–2050, 2020

    Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gradient sign dropout.Advances in Neural Information Processing Systems, 33:2039–2050, 2020

    2039

  15. [15]

    PhysBench: Bench- marking and enhancing vision-language models for physical world understanding

    Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Campagnolo Guizilini, and Yue Wang. PhysBench: Bench- marking and enhancing vision-language models for physical world understanding. InInternational Conference on Learning Representations, 2025

    2025

  16. [16]

    EditMGT: Unleashing potentials of masked generative transformers in image editing

    Wei Chow, Linfeng Li, Lingdong Kong, Zefeng Li, Qi Xu, Hang Song, Tian Ye, Xian Wang, Jinbin Bai, Shilin Xu, Xiangtai Li, Junting Pan, Shaoteng Liu, Ran Zhou, Tianshu Yang, and Songhua Liu. EditMGT: Unleashing potentials of masked generative transformers in image editing. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 38038–38048, 2026

    2026

  17. [17]

    Masked generative transformer is what you need for image editing.arXiv preprint arXiv:2605.10859, 2026

    Wei Chow, Linfeng Li, Xian Sun, Lingdong Kong, Zefeng Li, Qi Xu, Hang Song, Tian Ye, Xian Wang, Jinbin Bai, Shilin Xu, Xiangtai Li, Junting Pan, Shaoteng Liu, Ran Zhou, Tianshu Yang, and Songhua Liu. Masked generative transformer is what you need for image editing.arXiv preprint arXiv:2605.10859, 2026. 34

    Pith/arXiv arXiv 2026

  18. [18]

    Flow matching in latent space

    Quan Dao, Hao Phung, Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint arXiv:2307.08698, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. InAdvancesin Neural Information Processing Systems, volume 34, pages 8780–8794, 2021

    2021

  20. [20]

    Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and MCMC

    Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl- Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and MCMC. InInternational Conference on Machine Learning, pages 8489–8510. PMLR, 2023

    2023

  21. [21]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In International Conference on Machine Learning, pages 12606–12633. PMLR, 2024

    2024

  22. [22]

    Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advancesin Neural Information Processing Systems, 36:79858–79885, 2023

    2023

  23. [23]

    Rubric-based on-policy distillation.arXiv preprint arXiv:2605.07396, 2026

    Junfeng Fang, Zhepei Hong, Mao Zheng, Mingyang Song, Gengsheng Li, Houcheng Jiang, Dan Zhang, Haiyun Guo, Xiang Wang, and Tat-Seng Chua. Rubric-based on-policy distillation.arXiv preprint arXiv:2605.07396, 2026

    Pith/arXiv arXiv 2026

  24. [24]

    Flow-OPD: On-policy distillation for flow matching models.arXiv preprint arXiv:2605.08063, 2026

    Zhen Fang, Wenxuan Huang, Yu Zeng, Yiming Zhao, Shuang Chen, Kaituo Feng, Yunlong Lin, Lin Chen, Zehui Chen, Shaosheng Cao, et al. Flow-OPD: On-policy distillation for flow matching models.arXiv preprint arXiv:2605.08063, 2026

    Pith/arXiv arXiv 2026

  25. [25]

    Training-free structured diffusion guidance for compositional text-to-image synthesis

    Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for compositional text-to-image synthesis. arXiv preprint arXiv:2212.05032, 2022

    Pith/arXiv arXiv 2022

  26. [26]

    Dream- Sim: Learning new dimensions of human visual similarity using synthetic data.arXiv preprint arXiv:2306.09344, 2023

    StephanieFu, NetanelTamir, ShobhitaSundaram, LucyChai, RichardZhang, TaliDekel, andPhillipIsola. Dream- Sim: Learning new dimensions of human visual similarity using synthetic data.arXiv preprint arXiv:2306.09344, 2023

    Pith/arXiv arXiv 2023

  27. [27]

    Efficient knowledge distillation from an ensemble of teachers

    Takashi Fukuda, Masayuki Suzuki, Gakuto Kurata, Samuel Thomas, Jia Cui, and Bhuvana Ramabhadran. Efficient knowledge distillation from an ensemble of teachers. InInterspeech, pages 3697–3701, 2017

    2017

  28. [28]

    An image is worth one word: Personalizing text-to-image generation using textual inversion.arXiv preprint arXiv:2208.01618, 2022

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion.arXiv preprint arXiv:2208.01618, 2022

    Pith/arXiv arXiv 2022

  29. [29]

    GenEval: An object-focused framework for evaluating text-to-image alignment.Advancesin Neural Information Processing Systems, 36:52132–52152, 2023

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. GenEval: An object-focused framework for evaluating text-to-image alignment.Advancesin Neural Information Processing Systems, 36:52132–52152, 2023

    2023

  30. [30]

    MiniLLM: Knowledge distillation of large language models

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. MiniLLM: Knowledge distillation of large language models. In International Conference on Learning Representations, volume 2024, pages 32694–32717, 2024

    2024

  31. [31]

    Efficient diffusion training via min-SNR weighting strategy

    Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Baining Guo. Efficient diffusion training via min-SNR weighting strategy. InIEEE/CVF International Conference on Computer Vision, pages 7441–7451, 2023

    2023

  32. [32]

    A comprehensive overhaul of feature distillation

    Byeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park, Nojun Kwak, and Jin Young Choi. A comprehensive overhaul of feature distillation. InIEEE/CVF International Conference on Computer Vision, pages 1921–1930, 2019

    1921

  33. [33]

    Prompt-to-prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Pith/arXiv arXiv 2022

  34. [34]

    CLIPScore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning. InConference on Empirical Methods in Natural Language Processing, pages 7514–7528, 2021

    2021

  35. [35]

    Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015. 35

    Pith/arXiv arXiv 2015

  36. [36]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Pith/arXiv arXiv 2022

  37. [37]

    Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems, 33:6840–6851, 2020

    2020

  38. [38]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations, 2022

    2022

  39. [39]

    Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. InIEEE/CVF International Conference on Computer Vision, pages 20406–20417, 2023

    2023

  40. [40]

    Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2I-CompBench++: An enhanced and comprehensive benchmark for compositional text-to-image generation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5):3563–3579, 2025

    2025

  41. [41]

    Composer: Creative and controllable image synthesis with composable conditions.arXiv preprint arXiv:2302.09778, 2023

    Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: Creative and controllable image synthesis with composable conditions.arXiv preprint arXiv:2302.09778, 2023

    Pith/arXiv arXiv 2023

  42. [42]

    Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

    Pith/arXiv arXiv 2022

  43. [43]

    Adaptive mixtures of local experts

    Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts. Neural computation, 3(1):79–87, 1991

    1991

  44. [44]

    Rotograd: Gradient homogenization in multitask learning.arXiv preprint arXiv:2103.02631, 2021

    Adrián Javaloy and Isabel Valera. Rotograd: Gradient homogenization in multitask learning.arXiv preprint arXiv:2103.02631, 2021

    Pith/arXiv arXiv 2021

  45. [45]

    Asymmetric on-policy distillation: Bridging exploitation and imitation at the token level.arXiv preprint arXiv:2605.06387, 2026

    Nan Jia, Haojin Yang, Xing Ma, Jiesong Lian, Shuailiang Zhang, Weipeng Zhang, Ke Zeng, Xunliang Cai, and Zequn Sun. Asymmetric on-policy distillation: Bridging exploitation and imitation at the token level.arXiv preprint arXiv:2605.06387, 2026

    Pith/arXiv arXiv 2026

  46. [46]

    D-OPSD: On-policy self-distillation for continuously tuning step-distilled diffusion models

    Dengyang Jiang, Xin Jin, Dongyang Liu, Zanyi Wang, Mingzhe Zheng, Ruoyi Du, Xiangpeng Yang, Qilong Wu, Zhen Li, Peng Gao, et al. D-OPSD: On-policy self-distillation for continuously tuning step-distilled diffusion models. arXiv preprint arXiv:2605.05204, 2026

    Pith/arXiv arXiv 2026

  47. [47]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advancesin Neural Information Processing Systems, 35:26565–26577, 2022

    2022

  48. [48]

    Consistency trajectory models: Learning probability flow ODE trajectory of diffusion

    Dongjun Kim, Chieh-Hsin Lai, WeiHsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ODE trajectory of diffusion. InInternational Conference on Learning Representations, 2024

    2024

  49. [49]

    Pick-A-Pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-A-Pic: An open dataset of user preferences for text-to-image generation. InAdvances in Neural Information Processing Systems, volume 36, pages 36652–36663, 2023

    2023

  50. [50]

    AI for auto-research: Roadmap & user guide.arXiv preprint arXiv:2605.18661, 2026

    Lingdong Kong, Xian Sun, Wei Chow, Linfeng Li, Kevin Qinghong Lin, Xuan Billy Zhang, Song Wang, Rong Li, Qing Wu, Wei Gao, et al. AI for auto-research: Roadmap & user guide.arXiv preprint arXiv:2605.18661, 2026

    Pith/arXiv arXiv 2026

  51. [51]

    VieScore: Towards explainable metrics for conditional image synthesis evaluation

    Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. VieScore: Towards explainable metrics for conditional image synthesis evaluation. InAnnual Meeting of the Association for Computational Linguistics, pages 12268–12290, 2024

    2024

  52. [52]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1931–1941, 2023

    1931

  53. [53]

    DiffusionOPD: A unified perspective of on-policy distillation in diffusion models.arXiv preprint arXiv:2605.15055, 2026

    Quanhao Li, Junqiu Yu, Kaixun Jiang, Yujie Wei, Zhen Xing, Pandeng Li, Ruihang Chu, Shiwei Zhang, Yu Liu, and Zuxuan Wu. DiffusionOPD: A unified perspective of on-policy distillation in diffusion models.arXiv preprint arXiv:2605.15055, 2026

    Pith/arXiv arXiv 2026

  54. [54]

    Schedule your edit: A simple yet effective diffusion noise schedule for image editing

    Haonan Lin, Yan Chen, Jiahao Wang, Wenbin An, Mengmeng Wang, Feng Tian, Yong Liu, Guang Dai, Jingdong Wang, and Qianying Wang. Schedule your edit: A simple yet effective diffusion noise schedule for image editing. Advancesin Neural Information Processing Systems, 37:115712–115756, 2024. 36

    2024

  55. [55]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    Pith/arXiv arXiv 2022

  56. [56]

    Conflict-averse gradient descent for multi-task learning

    Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. Advancesin Neural Information Processing Systems, 34:18878–18890, 2021

    2021

  57. [57]

    Flow-GRPO: Training flow matching models via online RL.Advances in Neural Information Processing Systems, 38:40783–40818, 2026

    Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-GRPO: Training flow matching models via online RL.Advances in Neural Information Processing Systems, 38:40783–40818, 2026

    2026

  58. [58]

    Towards impartial multi-task learning

    Liyang Liu, Yi Li, Zhanghui Kuang, Jing-Hao Xue, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang. Towards impartial multi-task learning. InInternational Conference on Learning Representations, 2021

    2021

  59. [59]

    Compositional visual generation with composable diffusion models

    Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. InEuropean Conference on Computer Vision, pages 423–439. Springer, 2022

    2022

  60. [60]

    Step1x-Edit: A practical framework for general image editing

    Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Step1x-Edit: A practical framework for general image editing. arXiv preprint arXiv:2504.17761, 2025

    Pith/arXiv arXiv 2025

  61. [61]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    Pith/arXiv arXiv 2022

  62. [62]

    DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    2022

  63. [63]

    DPM-Solver++: Fast solver for guided sampling of diffusion probabilistic models.Machine Intelligence Research, 22(4):730–751, 2025

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver++: Fast solver for guided sampling of diffusion probabilistic models.Machine Intelligence Research, 22(4):730–751, 2025

    2025

  64. [64]

    Demystifying OPD: Length inflation and stabilization strategies for large language models.arXiv preprint arXiv:2604.08527, 2026

    Feng Luo, Yu-Neng Chuang, Guanchu Wang, Zicheng Xu, Xiaotian Han, Tianyi Zhang, and Vladimir Braverman. Demystifying OPD: Length inflation and stabilization strategies for large language models.arXiv preprint arXiv:2604.08527, 2026

    Pith/arXiv arXiv 2026

  65. [65]

    Latent consistency models: Synthesizing high-resolution images with few-step inference.arXiv preprint arXiv:2310.04378, 2023

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference.arXiv preprint arXiv:2310.04378, 2023

    Pith/arXiv arXiv 2023

  66. [66]

    Modeling task relationships in multi- task learning with multi-gate mixture-of-experts

    Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. Modeling task relationships in multi- task learning with multi-gate mixture-of-experts. InACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1930–1939, 2018

    1930

  67. [67]

    Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks

    Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, and James Henderson. Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks. InAnnual Meeting of the Association for Computational Linguistics, pages 565–576, 2021

    2021

  68. [68]

    Merging models with fisher-weighted averaging

    Michael S Matena and Colin Raffel. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35:17703–17716, 2022

    2022

  69. [69]

    SDEdit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073, 2021

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073, 2021

    Pith/arXiv arXiv 2021

  70. [70]

    Multi-task learning as a bargaining game.arXiv preprint arXiv:2202.01017, 2022

    Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi-task learning as a bargaining game.arXiv preprint arXiv:2202.01017, 2022

    Pith/arXiv arXiv 2022

  71. [71]

    Improveddenoisingdiffusionprobabilisticmodels

    AlexanderQuinnNicholandPrafullaDhariwal. Improveddenoisingdiffusionprobabilisticmodels. In International Conference on Machine Learning, pages 8162–8171. PMLR, 2021

    2021

  72. [72]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InIEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023

    2023

  73. [73]

    Adapterfusion: Non-destructive task composition for transfer learning

    Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. InConference of the European Chapter of the Association for Computational Linguistics, pages 487–503, 2021

    2021

  74. [74]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. InInternational Conference on Learning Representations, 2024. 37

    2024

  75. [75]

    Dreamfusion: Text-to-3D using 2D diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3D using 2D diffusion. arXiv preprint arXiv:2209.14988, 2022

    Pith/arXiv arXiv 2022

  76. [76]

    Aligning text-to-image diffusion models with reward backpropagation.arXiv preprint arXiv:2310.03739, 2023

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation.arXiv preprint arXiv:2310.03739, 2023

    Pith/arXiv arXiv 2023

  77. [77]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022

    2022

  78. [78]

    FitNets: Hints for thin deep nets.arXiv preprint arXiv:1412.6550, 2014

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. FitNets: Hints for thin deep nets.arXiv preprint arXiv:1412.6550, 2014

    Pith/arXiv arXiv 2014

  79. [79]

    A reduction of imitation learning and structured prediction to no-regret online learning

    Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. InInternational Conference on Artificial Intelligence and Statistics, pages 627–635. JMLR Workshop and Conference Proceedings, 2011

    2011

  80. [80]

    DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500–22510, 2023

    2023

Showing first 80 references.