Pith. sign in

REVIEW 4 major objections 3 minor 2 cited by

Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization

T0 review · 4 major / 3 minor · reviewed 2026-08-08 · deepseek-v4-flash

Pith's one-line read The paper claims that online reward-weighted conditional flow matching makes the model's data distribution converge to a Dirac delta at the maximum-reward sample, and that adding a Wasserstein-2 distance penalty converts this collapse…

desk verdict A useful likelihood-free RL fine-tuning recipe with a correct W2 bound, but the induced-distribution theorems rest on an assumed exponential tilt and a bogus proportionality step. read the letter →

arxiv 2502.06061 v1 pith:BEBOFXDU submitted 2025-02-09 cs.LG cs.AIcs.CVstat.ML

classification cs.LGcs.AIcs.CVstat.ML
keywords flowmatchingreward-weightedregressionreinforcementlearningfine-tuningWasserstein-2regularizationpolicycollapsereward-diversitytrade-offconditionaltext-to-imagealignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish a simple recipe for aligning continuous flow-matching generative models to arbitrary reward functions: reweight the conditional flow-matching loss by the reward, sample new training data from the model as it is tuned, and add a Wasserstein-2 penalty to keep the tuned model near a reference model. It proves that the reward-weighted loss alone makes the model learn the distribution $p_{\mathrm{new}}(x_1)=w(x_1)q(x_1)/Z$, and that iterating this online drives the distribution to a point mass at the highest-reward sample unless the penalty is present. The Wasserstein-2 penalty is justified by a tractable upper bound expressed directly through the model vector fields, which avoids likelihood computations that are expensive in continuous-time flows. If the results hold, users could fine-tune large flow models such as text-to-image generators with scalar rewards—classifier scores, compression rates, or text-image similarity scores—with one knob controlling how much diversity is sacrificed.

What carries the argument

The engine is the observation that a weighted regression loss is a change of measure: putting $w(x_1)$ inside the CFM expectation replaces the data distribution $q$ with $p_{\mathrm{new}}\propto w q$, so reward maximization becomes iterative reweighting. The brake is the Wasserstein-2 upper bound $W_2^2(p_1^{\theta_1},p_1^{\theta_2})\le e^{2L}\int_0^1\mathbb{E}_{x\sim p_s^{\theta_1}}[\|v_{\theta_1}(s,x)-v_{\theta_2}(s,x)\|^2]\,ds$, obtained by coupling the two flows through the same initial noise and using a differential inequality on the trajectory discrepancy; replacing the integral with a Monte Carlo average turns the penalty into a per-sample loss term $\alpha\|v_{\theta_{\mathrm{ft}}}(t,x)-v_{\theta_{\mathrm{ref}}}(t,x)\|^2$. Composing the reweighting and the penalty across epochs produces the closed-form recursion $q^n_\theta\propto w q^{n-1}_\theta\exp(-\beta D^{n-1})$, from which collapse and trade-off follow.

What would settle it

Run one epoch of reward-weighted conditional flow matching on a toy Gaussian flow with a known reward, then compare the empirical distribution of generated $x_1$ with $w(x_1)q(x_1)/Z$ while recording per-sample fitting losses; if the discrepancy tracks variation in $L_{\mathrm{cfm}}(x_1)$, the constancy assumption fails and Theorem 1's distributional claim would need a correction term.

Watch

Extended reading notes

Core claim

Under a reward-weighting function $w(x_1)\propto r(x_1)$, the loss $L_{\mathrm{RW\text{-}CFM}}(\theta)=\mathbb{E}[w(x_1)\|v_\theta(t,x)-u_t(x|x_1)\|^2]$ is equivalent to running standard conditional flow matching on the reweighted distribution $p_{\mathrm{new}}(x_1)=w(x_1)q(x_1)/Z$ (Theorem 1). If each epoch samples from the current model, the induced distribution after $N$ epochs is $q^N_\theta(x_1)=w(x_1)^N q(x_1)/Z_N$, so with strictly positive weights the limit is the Dirac delta $\delta(x_1-x^*_1)$ at the maximizer of $w$ (Theorem 2 and Lemma 1). Adding the Wasserstein-2 regularizer with coefficient $\alpha$ changes the recursion to $q^n_\theta(x_1)\propto w(x_1)q^{n-1}_\theta(x_1)\exp(-\beta D^{n-1}(x_1))$, where $D^{n-1}$ is the mean squared difference between the fine-tuned and reference vector fields along the flow; with the exponential weight $w(x_1)=\exp(\tau r(x_1))$ this yields $q^N_\theta(x_1)\propto \exp(\tau N r(x_1)-\beta\sum_{n=1}^N D^{n-1}(x_1))q(x_1)$ (Theorems 4 and 5). The paper also proves the squared Wasserstein-2 distance between two flow models is bounded by $e^{2L}\int_0^1 \mathbb{E}_{x\sim p_s^{\theta_1}}\|v_{\theta_1}(s,x)-v_{\theta_2}(s,x)\|^2\,ds$, making the regularizer computable by Monte Carlo (Theorem 3).

Load-bearing premise

The argument stands on the assumption that once the flow-matching model fits its training data well, per-sample fitting errors are roughly equal, so the only forces reweighting the learned distribution are the reward multiplier and the Wasserstein penalty; the exponential-tilt formula $q\exp(-\gamma\tilde L)$ is the template that makes every theorem go through.

Editorial extensions

If this is right

  • Unregularized online reward-weighted fine-tuning of a flow-matching model converges to a delta distribution at the maximum-reward sample, so reward alone is not enough for stable alignment (Lemma 1).
  • With the W2 penalty, the induced distribution after $N$ epochs is proportional to $w(x_1)^N q(x_1)\exp(-\beta\sum_{n=1}^N D^{n-1}(x_1))$, giving an explicit reward-distance trade-off controlled by $\alpha$ and $\tau$ (Theorems 4 and 5).
  • Tuning the regularization strength recovers both extremes: $\alpha\to 0$ reproduces collapse, while $\alpha\to\infty$ keeps the model essentially at the reference distribution (Corollaries 11 and 12).
  • The method needs no likelihoods, no filtered preference datasets, and no differentiable reward, because the regularizer is a Monte Carlo estimate from vector fields (Theorem 3 and Corollary 10).
  • The update rule matches the form of advantage-weighted regression and KL-regularized policy optimization, so standard RL intuition about exploration-exploitation applies to flow matching fine-tuning (Appendix C.9).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the core mechanism is a property of iterative reweighting, not of flow matching specifically—any generative model trained by minimizing a per-sample regression loss should show the same collapse-to-delta behavior when samples are drawn from the improving model, so the W2-style bound may transfer to score-based diffusion or autoregressive models.
  • Beyond the paper: the W2 upper bound contains the factor $e^{2L}$, and for high-dimensional image models the Lipschitz constant $L$ of the reference vector field is likely large, so the practical value of $\alpha$ that prevents collapse may correspond to a much smaller true Wasserstein radius than the theory suggests.
  • Beyond the paper: a direct testable extension is to measure the per-sample fitting loss $L_{\mathrm{cfm}}(x_1)$ during fine-tuning; if high-reward samples are systematically harder to fit, the factorization $q^N\propto w^N q\exp(-\beta\sum D^{n-1})$ should be corrected by a reward-dependent fitting error term.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes ORW-CFM-W2, an online reward-weighted conditional flow matching method with Wasserstein-2 regularization for fine-tuning flow-based generative models toward arbitrary rewards. The central theoretical claims are: (i) unregularized online reward weighting drives the learned distribution to q_N ∝ w^N q, which collapses to a delta at the maximum-reward point (Theorems 1-2, Lemma 1); (ii) adding W2 regularization yields an induced distribution of the form q_N ∝ exp(τ N r − β Σ D_n) q, giving a controllable reward-diversity trade-off (Theorems 4-5); and (iii) the W2 distance between two flow-matching models is bounded by an integral of vector-field deviations (Theorem 3). Experiments on MNIST, CIFAR-10 compression, and Stable Diffusion 3 text-image alignment are presented as empirical support.

Significance. If the theoretical claims were established, this would be a valuable contribution: a likelihood-free, reward-weighted RL method for continuous flow matching with a tractable divergence penalty and an explicit reward-diversity trade-off. The W2 bound in Theorem 3 (App. C.4) appears correct and is a useful standalone result; the proof via flow-map coupling and Gronwall's inequality is sound, and the paper deserves credit for that part. The empirical results, especially the SD3 experiments, suggest that the algorithmic recipe can improve CLIP alignment while retaining diversity. However, the core theoretical results on induced distributions are not supported: Theorem 6 assumes the exponential-tilt form it claims to derive, and Theorem 7's factorization to w exp(−βD) is algebraically invalid under its own Assumption 4. Because the paper's headline claim is to be "theoretically sound," and because the reward-diversity trade-off theorems depend directly on these steps, the main theoretical contribution is not currently established.

major comments (4)
  1. [App. C.1, Theorem 6, Eq. (27)] Theorem 6 is asserted rather than proved. The claim that minimizing L_total(θ) = E_{x1∼q}[\tilde L(x1)] induces p_new(x1) ∝ q(x1) exp(−γ \tilde L(x1)) for any per-sample loss is not derived from the loss-minimization objective. The 'RL perspective' proof simply postulates the tilted distribution p_new ∝ q exp(τ r) as the target; the energy perspective defines an EBM with that Boltzmann form; and the Bayesian perspective requires \tilde L to be a negative log-likelihood. None of these establishes the general theorem. Since Theorem 7, Theorem 4, and Theorem 5 all inherit this step, their conclusions are conditional on an unproved assumption, not consequences of the stated losses.
  2. [App. C.5, Assumption 4 and Eqs. (114)-(115)] The factorization p_new(x1) ∝ q(x1) w(x1) exp(−βD(x1)) does not follow from Assumption 4. Starting from Eq. (114), p_new ∝ q exp(−γ[w(x1)L_cfm(x1) + αD(x1)]), if L_cfm ≈ c is a constant, the factor is exp(−γ c w(x1)), which is not proportional to w(x1) for a generic weighting function. If L_cfm is negligible, then exp(−γ w L_cfm) ≈ 1 and the result is q exp(−βD) with no w factor at all. The remark that 'w(x1)L_cfm(x1) becomes proportional to w(x1)' confuses smallness of the product with proportionality; algebraically, exp(−γ c w) is not w. This invalidates Theorem 7 and, through recursion, Theorem 4 and Theorem 5.
  3. [App. C.6, Theorem 4, Eqs. (129)-(130)] The recursive step q_n ∝ q_{n−1} exp(−γ[w L_cfm + αD_{n−1}]) → q_n ∝ w q_{n−1} exp(−βD_{n−1}) repeats the same unsupported simplification. Even if Theorem 6 were accepted, the replacement of exp(−γ w L_cfm) by w requires Assumption 4, and Assumption 4 does not deliver that replacement, as noted above. Consequently Theorem 5 and the corollaries describing the exponential-weighting case (App. C.7) are not proven. The claimed reward-diversity trade-off is thus a conjecture about the assumed tilted form, not a derived property of the ORW-CFM-W2 loss.
  4. [App. C.5, Assumption 4] Assumption 4 is load-bearing and unmeasured. The paper offers no experiment or estimate showing that w(x1)L_cfm(x1) is approximately constant or negligible relative to αD(x1) during fine-tuning, even though this assumption is the bridge to all closed-form induced-distribution results. Without empirical evidence or a valid derivation, the closed-form expressions in Theorems 4, 5, and 7 remain unsupported for the actual training regime.
minor comments (3)
  1. [Sec. 5 and App. A.2] The experimental plots report single runs without error bars or confidence intervals; Table 1 is computed from 64 samples without repeated seeds, which makes the quantitative trade-off claims harder to assess.
  2. [App. C.6, Theorem 4] The notation for the divergence term is ambiguous: Eq. (10) evaluates the W2 regularizer at the current parameters θft, while D^{n−1}(x1) in Theorem 4 is defined using θ_{n−1}. The relationship between these quantities should be clarified.
  3. [App. G.2] The cross-reference 'Fig. G.2' should be 'Fig. 12'; several other internal references to figures in the appendix are similarly imprecise.

Circularity Check

3 steps flagged · score 8.0 of 10

The induced-distribution theorems are assumed, not derived: Theorem 6 posits the exponential-tilt form and Assumption 4 inserts the w-factor that Theorems 4 and 5 need.

  1. self definitional [Appendix C.1, Theorem 6, 'Proof of Theorem 6, from RL Perspective', Eqs. (28)-(30)]
    "In fact, most previous RLHF ... have found that assume a reward model r(x) that captures human preferences, with the goal of modifying the base generative model q(x) such that it generates the following tilted distribution: pnew(x1) = p∗(x1) ∝ q(x1) exp(τ ∗ r(x1)) ... just set the reward model as the negative of per-sample loss ˜L(x1), namely: r(x1) = −˜L(x1) ... and we can obtain: pnew(x1) = p∗(x1) ∝ q(x1) exp(−γ ˜L(x1))"

    Theorem 6 claims that any per-sample loss L̃ induces pnew ∝ q exp(−γL̃). The proof does not derive this from the flow-matching or optimization dynamics; instead it begins by assuming the target is the tilted distribution q exp(τ r), then substitutes r = −L̃. The conclusion is therefore the starting assumption with L̃ renamed as −r. The energy and Bayesian proofs similarly posit the Boltzmann form or identify L̃ with a negative log-likelihood. Since Theorem 6 is the load-bearing premise for Theorems 7, 4, and 5, the later 'predictions' of delta collapse and reward-diversity trade-off inherit this assumed exponential-tilt form rather than deriving it from the loss.

  2. other [Appendix C.5, Theorem 7, Assumption 4 and Eq. (115)]
    "Assumption 4. Assuming Lcfm(x1) is approximately constant or negligible compared to D(x1), namely: • When the model fits the data well, Lcfm(x1) is small. (i.e., w(x1) dominates) • The term w(x1)Lcfm(x1) becomes proportional to w(x1). ... Thus, we can write: pnew(x1) ∝ q(x1)w(x1) exp(−βD(x1))"

    Starting from pnew ∝ q exp(−γ[w Lcfm + αD]), the only algebraic route to q w exp(−βD) is to require exp(−γ w Lcfm) ∝ w. Assumption 4 asserts exactly that 'w Lcfm becomes proportional to w', but this does not follow from Lcfm being small or constant. If Lcfm ≈ 0, the exponential factor is ≈ 1 and no w factor appears; if Lcfm ≈ c, the factor is exp(−γ c w), which is not proportional to w for a generic weighting function. The desired factorization is thus inserted by assumption, and the reward-diversity trade-off theorems built on it are forced by this assumption.

1 more flagged steps
  1. other [Appendix C.6, Proof of Theorem 4, Eqs. (127)-(130)]
    "Similarly, we can assume that the probability of x1 being selected for training at epoch n is proportional to exp(−γ ˜Ln(x1)). Therefore: qn θ(x1) ∝ qn−1 θ(x1) exp(−γ ˜Ln(x1)) ... Assuming Ln cfm(x1) is approximately constant or negligible compared to αDn−1(x1) (i.e., well learned), we simplify: qn θ(x1) ∝ w(x1)qn−1 θ(x1) exp(−βDn−1(x1))"

    The recursive update that defines the whole W2-regularized online evolution is introduced by 'we can assume', not derived from the ORW-CFM-W2 loss or from flow-matching dynamics. The update is the same exponential-tilt ansatz from Theorem 6, and the simplification to a w(x1) factor repeats the invalid Assumption 4 step. Theorems 4 and 5, which constitute the paper's central controllable-collapse and reward-diversity trade-off claims, therefore reduce to this assumed update rule rather than following from the proposed objective.

full rationale

The paper's main theoretical contribution is the induced-distribution analysis: unregularized online reward weighting collapses to a delta (Theorems 1-2, Lemma 1), and W2 regularization yields the trade-off q_N ∝ exp(τ N r − β Σ D) q (Theorems 4-5). The load-bearing step is Theorem 6, which asserts that any per-sample loss L̃ induces p_new ∝ q exp(−γ L̃). Its three proofs either assume the tilted distribution as the RL target, posit a Boltzmann/EBM form, or identify L̃ with a negative log-likelihood; none derive the exponential tilt from the actual optimization objective. Theorem 7 then needs to factor q exp(−γ[w L_cfm + αD]) into q w exp(−βD); Assumption 4 asserts the needed proportionality, but algebraically a negligible or constant L_cfm does not produce a multiplicative w factor. Theorem 4 repeats the assumed exponential-tilt update and the same simplification, and Theorem 5 inherits it. Thus the central 'predictions' of delta collapse and controllable reward-diversity trade-off are forced by the assumed exponential-tilt form, not by the flow matching dynamics. The paper does contain one genuinely self-contained result, Theorem 3's W2 bound, which follows from a Gronwall argument with Lipschitz continuity, and the experiments compare against external baselines; these provide independent support for the algorithm as a practical method. However, the theoretical derivation chain for the induced distributions—the paper's central claim—reduces by definition and by tailored assumption. No load-bearing self-citation chain is present, so the score reflects definitional circularity rather than citation-based circularity.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests less on standard math than on an energy-model assumption that the authors state as a theorem. The only fully derived piece is the W2 bound (Theorem 3). The distributional results (Theorems 1, 2, 4, 5) reuse the same exponential-tilt input, so the ledger is dominated by that one concept repeated.

free parameters (3)
  • τ (temperature) = user-chosen (e.g., 0, 0.01, 1, 10)
    Controls the sharpness of the reward weight w(x1)=exp(τ r(x1)) and the collapse speed; treated as a free knob in Theorem 5 and experiments.
  • α (W2 regularization coefficient) = user-chosen (e.g., 0, 0.3, 0.8, 1, 10)
    Controls distance to the reference model; the main trade-off dial; not determined by theory.
  • β (regularization strength in induced distribution) = β=γα, with γ unspecified
    Introduced as a conversion constant in Theorem 7 and Theorem 4; the mapping between α in the loss and β in the induced distribution is never measured.
assumptions (6)
  • ad hoc to paper p_new ∝ q exp(-γ L̃) for any per-sample loss (Theorem 6)
    Theorems 1, 2, 4, and 5 all inherit this; the proofs assume the equivalence between loss and negative log-likelihood or start from the tilted target distribution.
  • ad hoc to paper The CFM fitting loss w(x1)L_cfm(x1) is approximately constant or negligible vs αD(x1) (Assumption 4)
    Required to derive p_new ∝ w q exp(-βD) in App. C.5; never validated empirically or theoretically.
  • domain assumption At each epoch the model perfectly learns the distribution implied by the loss
    Used in Theorem 2 and Theorem 4 proofs to replace the learned vector field with an exact distribution q_n; ignores optimization and finite-sample error.
  • domain assumption The reference vector field is Lipschitz in x with constant L, and the two models share the same initial noise x0
    Assumptions 1 and 2 in Theorem 3's proof; L is never estimated, and the e^{2L} constant is dropped in the practical regularizer.
  • domain assumption w(x1) must be positive and proportional to reward, with exponential form w=exp(τ r)
    Required for Lemma 1 and Theorem 5; negative or zero rewards need shifting.
  • standard math Standard analytic tools (Grönwall, Jensen, Wasserstein coupling)
    Used in Theorem 3 and Lemma 5; these parts are fine.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization." pith.science (2026). https://pith.science/paper/BEBOFXDU

@misc{pith2026250206061,
  author       = {Pith},
  title        = {Pith review of: Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BEBOFXDU}},
  note         = {Machine review of arXiv:2502.06061}
}
read the original abstract

Recent advancements in reinforcement learning (RL) have achieved great success in fine-tuning diffusion-based generative models. However, fine-tuning continuous flow-based generative models to align with arbitrary user-defined reward functions remains challenging, particularly due to issues such as policy collapse from overoptimization and the prohibitively high computational cost of likelihoods in continuous-time flows. In this paper, we propose an easy-to-use and theoretically sound RL fine-tuning method, which we term Online Reward-Weighted Conditional Flow Matching with Wasserstein-2 Regularization (ORW-CFM-W2). Our method integrates RL into the flow matching framework to fine-tune generative models with arbitrary reward functions, without relying on gradients of rewards or filtered datasets. By introducing an online reward-weighting mechanism, our approach guides the model to prioritize high-reward regions in the data manifold. To prevent policy collapse and maintain diversity, we incorporate Wasserstein-2 (W2) distance regularization into our method and derive a tractable upper bound for it in flow matching, effectively balancing exploration and exploitation of policy optimization. We provide theoretical analyses to demonstrate the convergence properties and induced data distributions of our method, establishing connections with traditional RL algorithms featuring Kullback-Leibler (KL) regularization and offering a more comprehensive understanding of the underlying mechanisms and learning behavior of our approach. Extensive experiments on tasks including target image generation, image compression, and text-image alignment demonstrate the effectiveness of our method, where our method achieves optimal policy convergence while allowing controllable trade-offs between reward maximization and diversity preservation.

Figures

Figures reproduced from arXiv: 2502.06061 by the authors.

Figure 1
Figure 1. A General Architecture of Our Method. In the meanwhile, many existing reward-weighted RL fine-tuning methods require train on offline datasets manually collected by hu￾mans (Huguet et al., 2024) or derived from pre-trained models (Lee et al., 2023). While this approach ensures stable learning, it restricts the model’s ability to explore optimally in an online setting, inducing what is known as the online-offline gap… view at source ↗
Figure 2
Figure 2. Learning curve and generated images in target image generation task with different [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Learning curve and generated images in target image generation task with different [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Reward-Diversity Trade-off and W2 Distance Control via [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: General Comparison of Different Fine-tuning Methods on SD3 via CLIP Rewards. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Ablation Studies on Text-Image Alignment Tasks of SD3. All use CLIP rewards. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: General comparison of positional relationship understanding between different fine-tuning [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Comparison of different fine-tuning methods on fine-tuning SD3 for the prompt “a cat in the [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Comparison of fine-tuning methods for the prompt “a train on top of a surfboard”, demon [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Additional evaluation of ORW-CFM-W2’s semantic understanding and compositional [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: ). 46 [PITH_FULL_IMAGE:figures/full_fig_p046_11.png]
Figure 11
Figure 11. Figure 11: ) Corollary 17 (Optimal β for Exploration-Exploitation Trade-off). An optimal β ∗ exists that balances the influence of the reward term and the W2 distance bound, satisfying: β ∗ = τ r¯ D¯ (143) where r¯ is the average reward and D¯ is the average W2 distance bound. P…
Figure 11
Figure 11. Figure 11: Diversity Control with different α while τ = 1 and reward is Text-Image Similarity Probability Score (e.g., pclip (x, "an image of cat") for a cat image). W2 distance is estimated by its upper bound (see App. C.4). In practice, we will use a difference form in probabi…
Figure 12
Figure 12. Figure 12: Odd Number Generation in MNIST. (a) α = 0 (b) α = 1 [PITH_FULL_IMAGE:figures/full_fig_p058_12.png]
Figure 13
Figure 13. Figure 13: Diversity Control with different α while τ = 1 and reward is Text-Image Similarity Probability Score (e.g., pclip (x, "an image of dog") for a dog image). In practice, we will use a difference form in probability to promote training like r(x, "dog") = pclip (x, "an im…
Figure 14
Figure 14. Figure 14: Diversity Control with different α while τ = 1 and reward is Text-Image Similarity Probability Score (e.g., pclip (x, "an image of bird") for a bird image). In practice, we will use a difference form in probability to promote training like r(x, "bird") = pclip (x, "an…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FM-IRL: Flow-Matching for Reward Modeling and Policy Regularization in Reinforcement Learning

    cs.LG 2025-10 conditional novelty 6.0 of 10

    An online imitation-learning method uses a flow-matching teacher's class-conditional loss as a reward and a regularizer to train a simple MLP policy, beating cloning and adversarial-imitation baselines on five of six tasks.

  2. Inference-Time Alignment Control for Diffusion Models with Reinforcement Learning Guidance

    cs.LG 2025-08 conditional novelty 4.0 of 10

    Blending a base diffusion model with its RL-finetuned version at sampling time lets users dial alignment strength, with the blend weight corresponding to the KL-regularization coefficient beta/w.

Reference graph

Works this paper leans on

22 extracted references · 20 canonical work pages · cited by 2 Pith papers

  1. [1]

    left of”, “on top of

    Spatial Semantic Coherence/Alignment: Our method shows superior understanding of spatial relationships (“left of”, “on top of”, “underneath”) while maintaining high image quality. This validates our theoretical analysis that ORW-CFM-W2 can effectively optimize arbitrary reward functions without compromising generation quality

  2. [2]

    To address this challenge, we introduce W2 regularization, which effectively prevents over-optimization and policy collapse (Lemma 1)

    Controlled Optimization: As shown in Figure 3 and Figure 7, online RL methods trained on self-generated data (Dong et al., 2023; Huguet et al., 2024; Black et al., 2024; Shumailov et al., 2024) normally face great challenges of policy collapse, where models converge to limited, homogeneous outputs that maximize rewards but lack diversity - a phenomenon th...

  3. [3]

    left of”, “on top of

    without requiring manually collected (Ouyang et al., 2022a) or filtered datasets (Rafailov et al., 2023), while effectively preventing policy collapse through our theoretically-derived W2 regularization (See App. C.4). A.1 T EXT-I MAGE ALIGNMENT WITH SPATIAL UNDERSTANDING Figure 7: General comparison of positional relationship understanding between differ...

  4. [6]

    a cat in the sky

    Easy-to-Use and Stable Fine-tuning Method: Our method achieves these improvements without requiring filtered datasets (Rafailov et al., 2023), likelihood calculations (Black et al., 2024) or differentiable rewards (Domingo-Enrich et al., 2024), demonstrating the practical advantages of our reward-weighted flow matching framework. This aligns with our theo...

  5. [7]

    a train on top of a surfboard,

    illustrates how our theoretical framework enables stable fine-tuning that maximizes rewards while maintaining the rich generative capabilities of flow-based models (Lipman et al., 2023; Esser et al., 2024). Table 1: Performance and Diversity comparison of different fine-tuning methods on text-image alignment using SD3. ’CLIP Score’ measures alignment with...

  6. [8]

    3) Bottom row: Alpha Clip reward showcases consis- tent performance even with different text-image alignment rewards, validating the reward-agnostic nature/property of our approach

    Middle row: Pick Score reward further demonstrates robust spatial understanding across varying train models and surfboard configurations. 3) Bottom row: Alpha Clip reward showcases consis- tent performance even with different text-image alignment rewards, validating the reward-agnostic nature/property of our approach. This comprehensive evaluation across ...

  7. [9]

    HPS-V2: Achieves precise locomotive placement while capturing realistic wave interac- tions

  8. [10]

    Pick Score: Maintains consistent spatial relationships across varying train models and surfboard configurations

Show all 22 references
  1. [11]

    on top",

    Alpha Clip: Shows stable performance even with a different similarity metric framework. Diversity Preservation: Consistent with our theoretical analysis in Theorem 5, the generated images maintain natural variations in train appearances, surfboard designs, and ocean conditions...

  2. [14]

    Given that w (x1) > 0 for all x1 ∈ Xand attains its maximum at x∗ 1, we define: ϵ (x1) = w (x1) w (x∗

    (71) Proof of Lemma 1. Given that w (x1) > 0 for all x1 ∈ Xand attains its maximum at x∗ 1, we define: ϵ (x1) = w (x1) w (x∗

  3. [15]

    (72) which means, ϵ (x∗

  4. [16]

    Then, we can rewrite qN θ (x1) using ϵ(x1) as: qN θ (x1) = [w (x∗ 1)]N ϵ (x1)N q (x1) ZN (73) Then for x1 ̸= x∗ 1, we can have: ϵ(x1)N → 0 as N → ∞since ϵ(x1) < 1

    = 1 , and 0 ≤ ϵ (x1) < 1 for x1 ̸= x∗ 1, since w (x1) < w(x∗ 1). Then, we can rewrite qN θ (x1) using ϵ(x1) as: qN θ (x1) = [w (x∗ 1)]N ϵ (x1)N q (x1) ZN (73) Then for x1 ̸= x∗ 1, we can have: ϵ(x1)N → 0 as N → ∞since ϵ(x1) < 1. Thus qN θ (x1) → 0, ∀x1 ̸= x∗ 1. And for x1 = x∗...

  5. [17]

    And we can have the normalization constant as follows: ZN = Z X w (x1)N q (x1) dx1 = [w (x∗ 1)]N Z X ϵ (x1)N q (x1) dx1

    = [w(x∗ 1 )]N q(x1) ZN . And we can have the normalization constant as follows: ZN = Z X w (x1)N q (x1) dx1 = [w (x∗ 1)]N Z X ϵ (x1)N q (x1) dx1. (74) Similarly, we can obtain ZN ≈ [w (x∗ 1)]N q (x∗

  6. [18]

    Then, we can have the limit behavior: For x1 ̸= x∗ 1 : qN θ (x1) = [w (x∗ 1)]N ϵ (x1)N q (x1) [w (x∗ 1)]N q (x∗ 1) = ϵ (x1)N q (x1) q (x∗

    as N → ∞. Then, we can have the limit behavior: For x1 ̸= x∗ 1 : qN θ (x1) = [w (x∗ 1)]N ϵ (x1)N q (x1) [w (x∗ 1)]N q (x∗ 1) = ϵ (x1)N q (x1) q (x∗

  7. [19]

    (75) For x1 = x∗ 1 : qN θ (x∗

    → 0. (75) For x1 = x∗ 1 : qN θ (x∗

  8. [20]

    = [w (x∗ 1)]N q (x∗ 1) [w (x∗ 1)]N q (x∗ 1) = 1 (76) Then, we can have: lim N →∞ qN θ (x1) = δ (x1 − x∗

  9. [21]

    cat") = pclip (x,

    (77) According to Lemma 1, iteratively utilize the ORW-CFM loss to fine-tune the flow matching model without bounding the distance between reference model and fine-tuned model may lead us to a greedy policy over x1, which may induce the overoptimization problem (Black et al., ...

  10. [22]

    normally rely on a well-collected or filtered offline dataset, requiring massive manual assistance and high costs, which is far from automated fine-tuning. However, traditional online fine-tuning methods are usually too complicated, with too many hyper-parameters, and it is di...

  11. [2014]

    In this paper, we introduce two methods to handle the Overoptimization and ease the mode collapse risk in online RW-CFM algorithms

    or overoptimization in fine-tuning diffusion models (Black et al., 2024). In this paper, we introduce two methods to handle the Overoptimization and ease the mode collapse risk in online RW-CFM algorithms. The first is to introduce a W2 distance bound between fine- tuned model...

  12. [2019]

    James Kirkpatrick, Razvan Pascanu, Neil C

    URL https://openreview.net/forum?id=r1lyTjAqYX. James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, a...

  13. [2023]

    while maintaining coherent generations provides strong empirical support for our theoretical contributions while demonstrating the practical utility of our approach in real-world applications. 22 Published as a conference paper at ICLR 2025 B B ACKGROUND B.1 R EINFORCEMENT LEA...

  14. [2024]

    URL https://doi.org/10.48550/arXiv

    doi: 10.48550/ARXIV .2405.08448. URL https://doi.org/10.48550/arXiv. 2405.08448. Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector- Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with mini...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.