{"id":"168e4f87-ad5a-49d8-abff-ce8c2187fff3","arxiv_id":"2505.01169","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A three-term loss (initial instantaneous, initial average, and terminal velocity matching) distills flow matching models into one-evaluation two-timed flow models, outperforming LFMD, EFMD, and PID baselines on most tested datasets.","lead":"The authors introduce a new loss function, ITVM, for compressing a slow flow-matching generative model into a fast one that generates samples in one or a few steps. The loss matches velocities early in each time segment and enforces self-consistency near the end, and the paper reports better few-step sample quality than three baselines on 2D, tabular, and image datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains may reflect test-set selection of the EMA hyperparameter; the comparison is not controlled.","rationale":"The formal contribution of the paper is internally consistent: Lemmas 4.1 and 4.2 and Theorem 4.3 give a coherent ideal-training characterization, and I do not see a flaw in the proof that L_ITVM=0 implies grid-wise equivalence to the teacher's one-step solver. The load-bearing part is therefore the empirical claim. Its weakest point is the controlledness of the comparison: Tables 1-3 display, per dataset, the ITVM run with the best fused rank computed on the same four test metrics that are being reported, while baselines get no comparable hyperparameter selection. The four EMA variants differ by large margins, and no validation-based model selection is performed, so the headline advantage could be a test-set selection effect. I considered the KL/diffeomorphism issue identified by the reader; it is real and worth checking, but it affects only the 2D and tabular metrics, whereas the selection issue bears on every dataset in the headline claim. A fixed-hyperparameter re-tabulation, or validation-based hyperparameter selection, would settle the concern directly. Under this read the paper still merits a conditional acceptance pending that re-analysis, so the reader's verdict is unchanged.","tokens_in":29057,"tokens_out":11583,"duration_ms":133997,"concrete_test":"Recompute Tables 1-3 with mu fixed a priori to 0.99 (the value recommended in Section 5.2) for every dataset, reporting all four EMA variants or selecting mu on a held-out validation split, and count wins against the single-run baselines. If ITVM still wins the same majority of datasets, the selection concern is not the explanation; if the win count drops materially, the headline claim should be weakened to 'competitive after per-dataset tuning' instead of 'better over baselines.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim that ITVM yields better few-step generation than EFMD, LFMD, and PID rests on Tables 1-3, but those tables show only the best of up to four ITVM runs per dataset. For each dataset, the authors train ITVM with EMA decay mu in {0, 0.9, 0.99, 0.999} and then use reciprocal rank fusion over the four test metrics (NFE=1, 2, 4, 8) to select the model displayed in the main tables (Section 5.1 and Appendix C). The baselines are single runs with no analogous selection procedure. Because mu materially changes results (e.g., WORD at NFE=1: 0.5638 for mu=0 versus 0.2475 for mu=0.9; MINIBOONE: no ITVM run beats LFMD/PID), this procedure is equivalent to choosing hyperparameters on the test set. Without a validation split, multiple seeds, or error bars, the advantage cannot be cleanly attributed to the ITVM loss itself rather than to selection. The CIFAR-10 comparison is additionally uncontrolled: EFMD and LFMD were trained with batch size 56 while ITVM and PID used 80 (Section 5.1), so that baseline comparison is confounded.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes initial/terminal velocity matching (ITVM), a loss for distilling a pretrained flow-matching teacher into a two-timed flow model (TTFM) parameterized as phi_{s,t}(x)=x+(t-s)v_theta(s,t,x). The loss combines three terms: IIVM, which matches the student's instantaneous velocity to the teacher at time s; IAVM, which matches the one-step average velocity to the teacher's one-step solver output; and TVM, which enforces a terminal consistency condition using an EMA copy of the student itself. The authors prove Lemma 4.1/4.2 and Theorem 4.3 (Appendix A), showing that if training reaches zero total loss with the EMA at steady state and tau=1/N, the student exactly reproduces the teacher's one-step ODE solver on the grid {m tau}. Empirically, the method is compared with EFMD, LFMD, and PID on four 2D datasets, four tabular datasets, MNIST, and CIFAR-10, using KL divergence or FID, with additional ablations on the EMA decay rate, the choice of intermediate time in TVM, and the step size tau. The paper reports consistent gains except on MINIBOONE, where ITVM ranks third, and notes that its CIFAR-10 results are not state of the art.","tokens_in":29258,"tokens_out":11399,"duration_ms":121226,"significance":"If the empirical claims held, this would be a useful contribution to diffusion/flow distillation: a simple modification of Lagrangian flow-map distillation that avoids derivative computation in the terminal term, anchors teacher supervision at in-distribution inputs, and comes with a clean zero-loss characterization. The theoretical part is a genuine strength: Appendix A's proofs are careful, and Theorem A.3 gives a non-vacuous sufficient condition under explicit assumptions. The authors also provide full per-mu tables and ablations, which is unusually transparent. However, the main empirical claim is currently not established because of test-set hyperparameter selection, an unchecked invertibility assumption in the KL metric, and an uncontrolled batch-size difference on CIFAR-10. The method itself remains plausible, so the paper merits revision rather than rejection.","major_comments":[{"comment":"The headline comparison is confounded by test-set selection of the EMA decay rate. For each dataset, the ITVM entry in the main tables is the run whose four test metrics (NFE=1,2,4,8) have the best RRF-fused rank among up to four values of mu, whereas each baseline is a single run with no analogous selection procedure. This is equivalent to choosing mu on the test set, and mu materially changes results: for WORD at NFE=1, mu=0 gives KL 0.5638 while mu=0.9 gives 0.2475, and on MINIBOONE no ITVM run beats LFMD or PID. The abstract's superiority claim therefore cannot be cleanly attributed to the ITVM loss itself. Please either select mu on a validation split and report test results for that single choice, or report all mu runs with means and standard deviations over multiple seeds; the main tables should not show only the best-fused run.","section":"Section 5.1, Tables 1-3; Appendix C"},{"comment":"The KL numbers for 2D and tabular datasets rely on evaluating log p_theta^{(K)}(y) with the change-of-variables formula q(y)=p(phi^{-1}(y))/|det grad phi(phi^{-1}(y))|. This formula is valid only if the composed student map phi = phi_theta_{(K-1)/K,1} composed with ... composed with phi_theta_{0,1/K} is a diffeomorphism for every trained student. The student is parameterized as x+(t-s)v_theta(s,t,x), which imposes no injectivity, no orientation constraint, and no nonsingular-Jacobian constraint, and the paper never checks these conditions for the trained models. If trajectories cross, the expression in Eq. (7) is not the density of the student's samples, and the reported KL values do not measure sample quality as claimed. Please add a validity check, such as monitoring the sign and magnitude of det grad phi over sampled trajectories and verifying that the induced density integrates to one, or replace/augment the KL metric with one that does not require invertibility, such as an energy distance or a classifier two-sample test.","section":"Appendix B, Eq. (7) and Section B.1.4"},{"comment":"The CIFAR-10 comparison is not controlled. The paper states that EFMD and LFMD were trained with batch size 56 because of GPU memory constraints, while ITVM and PID used batch size 80. Batch size is a training hyperparameter that affects sample quality in diffusion-style models, so the CIFAR-10 rows of Table 3 do not support the claim that 'when other settings are controlled, ITVM performs better than baselines.' Either retrain all methods with the same batch size or other equalized compute, or explicitly report the comparison as uncontrolled and exclude CIFAR-10 from the headline empirical claim.","section":"Section 5.1, Table 3; Appendix C.3.2"}],"minor_comments":[{"comment":"The hypothesis list in Theorem A.3 contains two items labeled (b); the second should be relabeled (c) for clarity.","section":"Appendix A, Theorem A.3"},{"comment":"The theorem assumes both x_theta_y = theta and L_ITVM = 0; since the EMA necessarily lags the online parameters during finite training, this is an idealized steady-state characterization rather than a convergence guarantee. Please state this limitation directly next to the theorem.","section":"Section 4, Theorem 4.3"},{"comment":"The conclusions about the intermediate-time choice and step size tau are based on training curves; adding the final metric values and, where applicable, fused ranks would make the claims easier to verify quantitatively.","section":"Section 5.2, Figures 2 and 3"},{"comment":"There are several typographical and spacing artifacts, including 'V elocities' in the title and inconsistent spacing in 'LIA VM' and 'IA VM'; these should be cleaned up before publication.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The theoretical contribution is sound and the empirical idea is plausible, but the experimental protocol needs substantial revision: test-set selection over mu must be replaced by a validation-based or seed-averaged protocol, the KL metric's diffeomorphism assumption must be checked, and the CIFAR-10 comparison must be equalized or de-emphasized. If these issues are addressed, the paper could be a solid contribution to the distillation literature. I do not see a novelty or scope problem."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the useful things. The ITVM loss is a clean extension of LFMD/PID: it adds two initial-velocity terms, replaces the terminal derivative with a finite difference, and uses an EMA self-target. The self-referential bootstrap (EMA of the student as target) is a legitimate worry, but the IAVM/IIVM terms keep the teacher in the loop, and Theorem 4.3 gives an external sufficient condition: zero loss on a fixed tau grid implies the student simulates the teacher's one-step solver on that grid. That is crisp and non-trivial, and the proof in Appendix A looks correct under the stated continuity/full-support assumptions. The ablations on the intermediate-time choice and on tau are genuinely informative, and the authors are upfront about not reaching SOTA on CIFAR-10 and about the weaker teacher.\n\nThe soft spots are in the empirical evaluation. The headline comparisons in Tables 1-3 show only the best of four ITVM runs per dataset, chosen by reciprocal rank fusion over the four test metrics (NFE=1, 2, 4, 8). The baselines are single runs with no analogous selection. Since mu changes results a lot (WORD NFE=1: 0.56 for mu=0 vs 0.25 for mu=0.9; MINIBOONE: no ITVM run beats LFMD/PID), this is effectively test-set hyperparameter selection. Without a validation split, multiple seeds, or error bars, the claimed advantage cannot be cleanly attributed to the loss. The CIFAR-10 comparison is additionally confounded by the batch size mismatch (56 for EFMD/LFMD, 80 for ITVM/PID). Also, the KL metric for 2D and tabular datasets relies on a change-of-variables formula that assumes the student's flow map is a diffeomorphism for each (s,t); the authors never check invertibility or Jacobian positivity for the trained students. If trajectories cross, the reported KL numbers are not the student's sample distribution.\n\nThese are fixable issues. The theory and the method itself are worth engaging with; the empirical claims just need to be made on a fairer playing field. I'd send this to peer review and ask for a major revision that addresses the selection protocol, adds error bars, and checks the diffeomorphism assumption. It's not a breakthrough, but it's a useful incremental contribution for people working on flow and diffusion distillation.","headline":"A solid incremental distillation loss with a nice theoretical characterization; the empirical claims are undermined by test-set selection of the EMA hyperparameter and a batch-size confound.","tokens_in":29875,"tokens_out":3943,"would_cite":false,"duration_ms":34426,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new distillation loss trains a two-timed flow model by matching initial and terminal velocities separately, improving few-step generation over existing losses.","keywords":["flow matching distillation","two-timed flow models","velocity matching loss","few-step generation","consistency","Lagrangian PDE","exponential moving average","generative models"],"falsifier":"Take a trained ITVM student on a 2D dataset, sample a batch of $x_0$ from $p_0$, and compute the Jacobian determinant of $\\phi^\\theta_{0,1}(x)$ at those points; if any determinant is non-positive or the map fails to be invertible on the sample trajectories, the KL-based numbers in Table 1 do not measure the student's actual sample distribution, and the claimed gains would need re-evaluation.","tokens_in":28825,"feed_emoji":"⚡","tokens_out":6748,"duration_ms":64214,"temperature":0.7,"pith_summary":"The paper proposes a new loss, initial/terminal velocity matching (ITVM), for distilling a pre-trained flow matching model into a two-timed flow model—a network that maps noise to data in a single function evaluation. Instead of forcing the student to obey the full Lagrangian PDE of the teacher's flow, ITVM matches the teacher only at the starting time of each interval and enforces a self-consistency condition at the terminal time using an exponentially averaged copy of the student. The authors prove that if the loss reaches zero with a fixed step size $\\tau$, the distilled model exactly reproduces the teacher's one-step ODE solver on the $\\tau$-grid, so its error is $O(\\tau^p)$. Controlled experiments on 2D, tabular, and image datasets show ITVM generally beats the LFMD, PID, and EFMD baselines at 1, 2, 4, and 8 function evaluations.","feed_headline":"Matching start and end velocities improves few-step flow distillation","feed_subtitle":"Teacher guides only the interval's start; the student polishes its own end, beating LFMD and PID baselines","key_machinery":"The central object is the average velocity parametrization of a TTFM, $\\phi^\\theta_{s,t}(x) = x + (t-s)v^\\theta_{s,t}(x)$, which makes the initial condition automatic. The argument runs through two lemmas showing that short-interval agreement with the teacher plus self-consistency implies the full Lagrangian PDE. The loss machinery is the three-term ITVM loss: two initial-velocity matching terms (IIVM and IAVM) that keep the teacher's inputs on its training distribution, and a terminal velocity matching term (TVM) that replaces the teacher with the EMA student and replaces a derivative with a finite difference, avoiding second-order automatic differentiation. Together they convert the PDE constraint into local, in-distribution supervision.","core_discovery":"A two-timed flow model $\\phi^\\theta_{s,t}(x)$ trained with ITVM learns to satisfy the Lagrangian PDE without ever being trained against it directly. The key is Lemma 4.1 and Lemma 4.2: if the student agrees with the teacher's implicit flow on short intervals (or at the instantaneous initial velocity) and is consistent with itself for intervals shorter than some $\\tau^*$, then it equals the teacher's flow everywhere. ITVM turns this into a loss: IIVM matches the student's initial instantaneous velocity to the teacher's; IAVM matches the student's average velocity over a short interval to the teacher's one-step solver output; TVM enforces consistency at the terminal time $u = t-\\tau$ by matching a finite-difference velocity against the EMA student's velocity, leaving the teacher out of the terminal term. Theorem 4.3 states that at zero loss on the $\\tau$-grid, $\\phi^\\theta_{m\\tau,n\\tau}(x)$ equals the teacher's one-step solver result $S^{\\eta,\\tau}_{m\\tau,n\\tau}(x)$, so a successful student simulates the solver with one evaluation per step.","pith_inferences":["The authors note that IIVM and IAVM could be retargeted at the conditional velocity estimate $v_t(x|x_{\\text{data}})$ instead of the teacher's output; if that works, TTFMs could be trained directly from data without a separately trained teacher.","The KL evaluation in the paper presumes the student's flow map is a diffeomorphism; checking Jacobian positivity on sampled trajectories would confirm the reported numbers measure the true sample distribution.","The same pattern of local initial matching plus terminal self-consistency might transfer to other distillation settings, such as consistency models with different parameterizations, where the intermediate time $u$ is currently chosen differently."],"forward_implications":["Training TTFMs no longer requires differentiating through the model (as LFMD does), since TVM uses a finite difference; this simplifies implementation and speeds each training step.","The teacher is queried only at the initial time $s$ of each interval, where its input $x_s$ is sampled from its training distribution $p_s$, so distillation remains reliable even if the teacher behaves poorly on out-of-distribution inputs.","If training drives the loss to zero for $\\tau = 1/N$, the student reproduces the teacher's one-step ODE solver on the grid, making the number of function evaluations $N$ and the solver's order $p$ direct handles on the final error.","On the tested 2D, tabular, MNIST, and CIFAR-10 benchmarks, the ITVM student beats the LFMD, PID, and EFMD baselines at most NFE settings, with gains largest at 1-2 evaluations."],"supporting_citations":[{"why":"Supplies the LFMD and EFMD losses that ITVM extends and the baseline it is compared against.","marker":"Boffi et al. (2024)"},{"why":"Defines the conditional flow matching setup and the implicit flow that serves as the teacher.","marker":"Lipman et al. (2023)"},{"why":"Introduces consistency matching and the EMA stabilization that the TVM term adapts.","marker":"Song et al. (2023)"},{"why":"Provides soft consistency matching and the consistency trajectory model as an alternative consistency-based distillation baseline.","marker":"Kim et al. (2023)"},{"why":"Introduces PID, the numerical-differentiation baseline compared in the experiments.","marker":"Tee et al. (2024)"},{"why":"Gives the expected-velocity identity used to define the marginal vector field in the paper's setup.","marker":"Liu et al. (2023)"}],"fun_headline_variants":["Split velocity matching sharpens few-step flow models","Start and end velocity match lifts distillation quality","Two-point velocity matching distills flows faster","ITVM loss: better few-step flow generation","Match both ends to distill flow models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For the 2D and tabular results, the paper's KL metric assumes the student's map never folds space (its Jacobian stays positive), yet training never enforces or checks this.","fun_headline_variants_meta":{"raw":{"variants":["Split velocity matching sharpens few-step flow models","Start and end velocity match lifts distillation quality","Two-point velocity matching distills flows faster","ITVM loss: better few-step flow generation","Match both ends to distill flow models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000677,"raw_usage":{"total_tokens":3105,"prompt_tokens":1001,"completion_tokens":2104,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":2037}},"tokens_in":617,"tokens_out":2104,"duration_ms":16347,"temperature":1.0,"reasoning_tokens":2037,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:24:36.825392+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained ITVM student on a 2D dataset, sample a batch of $x_0$ from $p_0$, and compute the Jacobian determinant of $\\phi^\\theta_{0,1}(x)$ at those points; if any determinant is non-positive or the map fails to be invertible on the sample trajectories, the KL-based numbers in Table 1 do not measure the student's actual sample distribution, and the claimed gains would need re-evaluation.","supporting_citations":[],"review_version":1}