{"id":"c5dbb33a-9766-410c-b154-3c8a8da5bcee","arxiv_id":"2509.21925","paper_version":3,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"For finite training sets, stochastic interpolation models reproduce training samples under deterministic generation and output noise-perturbed copies under stochastic generation.","lead":"This paper asks what a generative model actually outputs when it is trained on a finite set of samples. It claims the model copies the training points, with deterministic generation reproducing them exactly and stochastic generation adding Gaussian noise around them.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2 is false as stated: for n=1 and Gaussian base, the exact reverse SDE terminates at the training point, not at training point plus independent Gaussian; the A.3.4 proof illegally decouples the Itô integral from the path-dependent drift.","rationale":"The Reader's weakest_assumption—the decoupling of Brownian noise in A.3.4—is exactly the step that breaks, and I agree that Theorem 2's proof is invalid. My stress-test goes further: the theorem's conclusion is not just unestablished but contradicted by the defining exact-marginal property of score-based reverse SDEs and by an explicit one-sample Gaussian example. In that example the reverse SDE has terminal law δ_0, not N(0,σ²), because the singular drift z/(2t) and score −z/t conspire to remove all variance at t=0. This explains why the A.3.4 decomposition is hopeless: treating ∫√(2ζ)dW as terminal additive noise ignores the path-dependent cancellation performed by the drift. The deterministic memorization results (Propositions 1–3, Theorem 1, Theorem 3, Corollary 1) may survive, and the experiments are suggestive but are run with estimated models and do not test the oracle stochastic claim. Because the headline contribution—'stochastic generation manifests as training samples with added Gaussian noise'—is false in the idealized setting the paper explicitly analyzes, the REJECT verdict is appropriate. The paper would need to substantially reframe what Eq. 5 actually generates (e.g., a non-exact discretization or a different ζ-schedule not satisfying the marginal property) before the claim could be considered. No ad hominem is intended; the issue is a mathematical error in a central theorem.","tokens_in":18442,"tokens_out":19955,"duration_ms":166888,"concrete_test":"Instantiate the oracle with one training point X1=0, α=1−t, β=t, γ=√(t(1−t)), ρ1=N(0,1), and ζ(t)=t(1−t). From Propositions 1–2, b*(z,t)=z/(2t) and s*(z,t)=−z/t. Simulate Eq. 5 backward from t=1 to t=0 with Euler–Maruyama using 10^5 steps and 10^5 trajectories, and record the empirical variance of the terminal states. The exact-marginal property (and the analytic solution v(t)=t) predicts variance →0 as the step size shrinks, i.e. a point mass at 0. Theorem 2 predicts variance 2∫_0^1 ζ(t)dt = 1/3. If the numerical terminal variance is ≈1/3, the exact reverse-SDE property fails; if it collapses to 0, Theorem 2 is false. This is a minimal parameter-free falsifier of the paper's central stochastic-generation claim.","verdict_should_be":"REJECT","load_bearing_attack":"The central stochastic-generation claim is not merely under-proved; it is wrong. Section 2 states that the stochastic generator (5), initialized at t=1 with ρ1 and using the exact velocity b* and score s*, has marginal ρ_t at every t, hence at t=0 outputs a draw from ρ0. When ρ0 is the empirical measure, that draw is a training sample. Theorem 2 instead claims N(X_i, σ² I_d). The proof in A.3.4 makes the specific error the Reader identifies: it defines a drift-only trajectory \\tilde Z but evaluates b along the noisy path Z, then writes Z0 = \\tilde Z0 + ∫_0^1 sqrt(2ζ(t)) dW_t. That stochastic integral is not an independent Gaussian added at the end; in a nonlinear SDE the terminal value is a functional of the whole Brownian path, and the score drift cancels the diffusion in the exact reverse-time dynamics. The failure is concrete. Take n=1, X1=0, α=1−t, β=t, γ=√(t(1−t)), ρ1=N(0,1), and any positive ζ for which the theorem's hypothesis is intended, e.g. ζ(t)=t(1−t). Then ρ_t=N(0,t), so b*(z,t)=z/(2t) and s*(z,t)=−z/t by Propositions 1–2. Solving the reverse SDE (5) explicitly gives marginal variance exactly t for every t; at t=0 the law is δ_0. The theorem's variance formula would instead give 2∫_0^1 ζ(t)dt = 1/3 for this ζ. Thus the headline result fails in the simplest possible oracle setting, independent of any regularity assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies stochastic interpolation models when the target distribution is the empirical measure of n training samples. It derives closed-form expressions for the optimal velocity and score (Propositions 1 and 2), claims that deterministic generation lands exactly on a training sample (Theorem 1), and claims that stochastic generation outputs a training sample plus isotropic Gaussian noise (Theorem 2). It then introduces an estimation-error model and proposes formal notions of underfitting and overfitting (Theorem 3 and Corollaries 1-3), with classification and contrastive-learning experiments intended to support the theory. The advertised message is that finite-sample stochastic interpolants memorize the training data, and that stochastic generation is effectively the training set plus Gaussian noise.","tokens_in":18825,"tokens_out":13400,"duration_ms":117138,"significance":"If the central results were correct, the paper would provide a sharp and useful characterization of finite-sample stochastic interpolants, with direct implications for data-copying, privacy, and the use of generated data in downstream tasks. The paper deserves credit for deriving the closed-form velocity and score fields, for connecting the deterministic analysis to an explicit softmax-weighted expression, and for attempting to translate the theory into empirical predictions. However, the stochastic-generation theorem is the headline claim, and it is internally inconsistent with the exact-marginal property already stated in Section 2; the proof in Appendix A.3.4 is invalid for a concrete reason. Because this claim drives the 'training samples plus Gaussian noise' narrative and the interpretation of the experiments, the paper's central contribution is not established. The deterministic memorization result and the estimation-error taxonomy are less affected and may be salvageable, but the manuscript in its current form is not acceptable.","major_comments":[{"comment":"Theorem 2 is false as stated, and the proof in A.3.4 is invalid. The proof defines a drift-only trajectory \\tilde Z but evaluates b along the noisy path Z, and then writes Z0 = \\tilde Z0 + \\int_0^1 \\sqrt{2\\zeta(t)} dW_t. This decoupling is not legitimate: the terminal value of a nonlinear SDE is a functional of the whole Brownian path, and the Itô integral is not an independent Gaussian added after the drift has been computed. The claimed conclusion also contradicts Section 2, where the paper states that stochastic generation with the exact b* and s* has marginal rho_t at every t; at t=0 this marginal is the empirical measure, not a Gaussian convolution. A concrete counterexample is n=1, X1=0, alpha(t)=1-t, beta(t)=t, gamma(t)=sqrt(t(1-t)), rho1=N(0,1). Then rho_t=N(0,t), b*(z,t)=z/(2t), and s*(z,t)=-z/t; the reverse SDE (5) has marginal N(0,t) at every t for any positive zeta (e.g., zeta(t)=t(1-t)), so at t=0 the law is delta_0. The theorem's variance formula 2 \\int_0^1 zeta(t)dt would instead give 1/3 for this zeta. Thus the central stochastic-generation claim fails already in the simplest oracle setting.","section":"Section 3.1, Theorem 2; Appendix A.3.4"},{"comment":"The proof of Corollary 2 assigns the asymptotic regimes in the opposite order to the statement. The theorem states that gamma(t) ≳ beta(t) gives convergence to the training set, gamma(t) ≍ beta(t) gives samples in the vicinity, and gamma(t) ≲ beta(t) gives divergence. After computing C3(t)/(C1(t)gamma(t)), the proof concludes: 'if gamma(t) ≲ beta(t), the limit is 0; if gamma(t) ≍ beta(t), constant; if gamma(t) ≳ beta(t), infinity', which reverses the statement. For beta(t)=t and gamma(t)=t^2, one has C3/(C1 gamma) ~ 1/t → infinity, so divergence corresponds to gamma ≲ beta as the statement says. The final paragraph of A.3.7 is therefore wrong, and the proof as written does not prove Corollary 2.","section":"Corollary 2; Appendix A.3.7"},{"comment":"The proof of the overfitting part of Corollary 3 is not valid as written. It defines C_T := exp^{(T)}(0) and then requires c3 lambda1 M < ln^{(T)}(C_T), but ln^{(T)}(exp^{(T)}(0)) = 0, so the inequality is impossible for positive c3, lambda1, and M. Even setting aside this notational error, the iterated-exponential bound is never rigorously derived from the one-step estimate; the conclusion that arbitrarily small lambda forces Z0 within tau of some Xi does not follow from the displayed inequalities.","section":"Appendix A.3.8, proof of Corollary 3, part (2)"},{"comment":"The error model used for Corollary 3 is not the model stated just before Eq. (10). The text says epsilon(z,t) is proportional to 1/rho_t(z), and the displayed expression contains a factor sqrt(C3(t))/beta(t) (or a similar polynomial factor) in addition to the inverse sum of exponentials, but Eq. (10) drops that factor without a precise statement of the limit in which the omission is valid. Since Corollary 3 concerns asymptotic behavior as t -> 0 and lambda -> 0 or infinity, the neglected polynomial factor can affect the thresholds and the proof. As written, Corollary 3 is a statement about a modified, more convenient error model, not about the model introduced in the text.","section":"Section 3.2, Eq. (10) and Corollary 3"}],"minor_comments":[{"comment":"The domain in the uniform bound is written as R × [0,1]; it should be R^d × [0,1] throughout.","section":"Corollary 1"},{"comment":"The variance formula 'sigma^2 = 2 \\int_0^t zeta(t)dt' uses t both as the upper limit and as the integration variable; it should be 2 \\int_0^1 zeta(u)du.","section":"Theorem 2 statement"},{"comment":"The tables appear misaligned: for example, the Full Samples entry at sample size 100 is 77.57, and the same number appears as the Half Samples entry at sample size 200. The rows should be regenerated from the raw results.","section":"Tables 1 and 2"},{"comment":"The proof contains a sign inconsistency: an intermediate line has the coefficient of z as gamma(t)gamma'(t) - beta'(t)beta(t) over C3(t), while the final definition of C1(t) uses a plus sign. The plus sign is the one used in the rest of the paper, but the displayed intermediate expression should be corrected.","section":"Appendix A.3.1, proof of Proposition 1"},{"comment":"There are numerous typographical errors, including 'empricial', 'Minikowski', 'determinstic', 'closded', and 'Minkowski' spelled inconsistently. A careful proofreading pass is needed.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my quick take. The closed-form finite-sample velocity and score (Propositions 1–2) and the deterministic memorization theorem (Theorem 1) are genuinely useful. The softmax concentration argument is standard and the derivation is clean, apart from a minor sign typo in the proof of Proposition 1. The estimation-error framework and the under/overfitting definitions give people a way to talk about data copying in diffusion/flow models, and the classification experiments point in the expected direction, though the experimental section is under-specified.\n\nThe problem is the paper's main advertised claim, Theorem 2. It is not just under-proved; it conflicts with Section 2. Section 2 says the stochastic generator (5), initialized at ρ1, has marginal ρ_t at every t, hence at t=0 it draws from the empirical ρ0. Theorem 2 says the output is N(X_i, σ²I). Those cannot both be true. The proof in A.3.4 makes the error concrete: it defines a drift-only trajectory, then writes the noisy solution as that trajectory plus an independent Gaussian integral. For a nonlinear SDE, the terminal value is a functional of the whole Brownian path; the Itô integral is not an independent add-on. In the n=1 example with α=1−t, β=t, γ=√(t(1−t)), ρ0=δ_0, ρ1=N(0,1), the exact b* is z/(2t), the exact score is −z/t, and the SDE (5) with ζ=t(1−t) has variance going to 0 at t=0, not to 2∫ζ dt = 1/3. So the theorem is false as stated, not just missing an assumption. Corollary 2's proof in A.3.7 also reverses the inequality ordering relative to its own statement.\n\nWho gets value from this paper: anyone working on finite-sample flow/diffusion theory will want the deterministic memorization result and the closed-form b* and s*. The stochastic section should not be trusted in its current form. I would send it to a serious referee, but with the expectation of major revision: either repair the stochastic analysis or honestly reframe the paper around the deterministic memorization and the estimation-error results. If the stochastic claim is retracted, the rest is publishable.","headline":"The finite-sample deterministic memorization result is solid, but the paper's headline stochastic-generation theorem is false as stated and contradicts its own setup.","tokens_in":19321,"tokens_out":11942,"would_cite":true,"duration_ms":98741,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that stochastic interpolation models trained on a finite set are memorization machines: with exact optimal velocity and score, deterministic generation lands exactly on a training sample, and stochastic generation lands…","keywords":["stochastic interpolation","finite-sample generative models","memorization","score-based generative models","flow matching","Gaussian noise","overfitting","underfitting"],"falsifier":"Run deterministic generation with the exact optimal velocity field from Proposition 1 on a known finite set; if the endpoint at $t=0$ is not one of the training samples, Theorem 1 fails. For the stochastic claim, generate many samples with known $\\zeta(t)$; if the empirical mean of $Z_0$ does not coincide with some training sample or the empirical covariance is not close to $2\\int_0^1 \\zeta(t)\\,dt$ times the identity matrix, Theorem 2's additive-Gaussian picture fails.","tokens_in":18169,"feed_emoji":"🎯","tokens_out":6441,"duration_ms":53936,"temperature":0.7,"pith_summary":"This paper argues that, once the training set is finite, stochastic interpolation models are memorization machines. With the optimal velocity field and score function, deterministic generation returns one of the training samples exactly, and stochastic generation returns a training sample plus Gaussian noise whose variance is set by the noise schedule. The paper also analyzes what happens when the velocity is estimated imperfectly: uniformly bounded errors still keep outputs inside the training set, while errors that grow near the endpoints either leave the output in a small neighborhood of a training sample or drive it to diverge, depending on the relative decay of the interpolation coefficients. This matters because it gives a formal explanation of data-copying in diffusion-style generative models and predicts that generated data used in downstream tasks is effectively noise-perturbed training data.","feed_headline":"Stochastic interpolants with finite data replay the training set","feed_subtitle":"Closed-form analysis shows deterministic outputs land on training points; stochastic outputs add Gaussian noise.","key_machinery":"The load-bearing object is the closed-form optimal velocity field $b^*(z,t)=\\sum_{i=1}^n \\frac{1}{C_3(t)}[C_1(t)z-C_2(t)X_i]\\,\\omega_i(z,t)$, where $\\omega$ is the softmax over squared distances $\\|z-\\alpha(t)X_i\\|^2/(2C_3(t))$. That softmax is the contraction engine: as $t\\to 0$ the weights concentrate on one training sample, and the ODE's integrating factor converts $b^*$ into exact selection of that sample. The score function enters through the Tweedie relation $s^*(z,t)=\\frac{\\alpha(t)}{B(t)}b^*(z,t)-\\frac{\\alpha'(t)}{B(t)}z$, which turns the stochastic generator into the same contraction with an extra Brownian integral.","core_discovery":"On the paper's own terms: when $\\rho_0$ is the empirical distribution $\\frac{1}{n}\\sum_i \\delta(X_i)$ and $\\rho_1$ is Gaussian, the optimal velocity field $b^*(z,t)$ is a softmax-weighted sum of vectors $z-X_i$ (Proposition 1), and the optimal score is a linear combination of $b^*$ and $z$ (Proposition 2). Theorem 1 states that the deterministic ODE driven by $b^*$ ends at one of the training samples. Theorem 2 states that, under the condition $\\zeta(t) \\lesssim B(t)$, the stochastic generator ends at that same training sample plus an independent Gaussian perturbation with covariance $2\\int_0^1 \\zeta(t)\\,dt$ times the identity. Theorem 3 extends the mechanism to estimated fields: with error $\\epsilon$, the endpoint is $X_i$ minus a limiting $\\epsilon$-term, so bounded errors vanish (Corollary 1), coefficients-controlled errors either converge to, stay near, or move away from the training set depending on the scaling of the noise coefficient against the interpolation coefficient (Corollary 2), and density-proportional errors produce underfitting for large error magnitude and overfitting for small error magnitude (Corollary 3).","pith_inferences":["The theory suggests that for small training sets, a diffusion-like model's apparent novelty is bounded by its noise schedule; without many training samples, diversity cannot come from smooth generalization beyond the empirical points.","A testable criterion follows: comparing the empirical covariance of generated samples to $2\\int_0^1 \\zeta(t)\\,dt$ times the identity could detect when a model behaves exactly in this memorization-plus-noise regime.","The same softmax-collapse argument should apply per class in conditional generation, so class-conditional training would memorize within each class; the paper leaves that case open.","The closed-form velocity is a kernel smoother with bandwidth $\\sqrt{C_3(t)}$, so the memorization result is conceptually a nearest-neighbor-type effect: the generator's effective bias is the training set itself."],"forward_implications":["Deterministic interpolation trained on $n$ points and using the oracle velocity is a lossless copy operator: every trajectory terminates at one of the $n$ training samples.","Stochastic oracle generation has the same memorization core and is equivalent to adding Gaussian noise of variance $2\\int_0^1 \\zeta(t)\\,dt$ to a training sample.","If the velocity estimation error is uniformly bounded, deterministic outputs stay in the training set despite the error, regardless of the bound's size.","The relative decay of the noise coefficient $\\gamma(t)$ and the interpolation coefficient $\\beta(t)$ decides whether finite-sample generation is faithful, concentrated near the training set, or divergent.","With density-inversely-proportional estimation error, large error gives divergent underfitting and small error gives collapse onto training samples, matching observed small-sample behavior."],"supporting_citations":[{"why":"Introduces the stochastic interpolant construction that the paper's generative processes are built on.","marker":"Albergo & Vanden-Eijnden, 2022"},{"why":"Supplies the unifying stochastic-interpolation framework for flows and diffusions used throughout the paper.","marker":"Albergo et al., 2023"},{"why":"Provides the score-velocity relation used in Proposition 2 and Theorem 2.","marker":"Huang et al., 2023"},{"why":"Shows flow matching is a special case, letting the memorization conclusions extend to flow models.","marker":"Lipman et al., 2022"},{"why":"Shows score-based diffusion models are a special case, giving the result broad reach across diffusion generators.","marker":"Song et al., 2020"},{"why":"Earlier finite-sample analysis of flow-matching memorization and generalization that motivates the finite-population question.","marker":"Gao & Li, 2024"},{"why":"Provides a closed-form flow-matching analysis on target stochasticity, a direct predecessor of the closed-form velocity here.","marker":"Bertrand et al., 2025"},{"why":"Supplies the memorization criterion used in the ImageNet experiments to quantify copied generated samples.","marker":"Yoon et al., 2023"}],"fun_headline_variants":["Finite training sets make stochastic interpolants memorize data","Stochastic generation: training points plus Gaussian noise","Optimal velocity field replays training samples exactly","With finite data, generative models must overfit or underfit"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof of Theorem 2 replaces the actual noisy trajectory by the noiseless drift trajectory and then adds the Brownian integral at the end; if the Brownian motion cannot be decoupled from the drift and treated as an independent additive Gaussian term, the conclusion that stochastic generation is just a training sample plus Gaussian noise does not follow.","fun_headline_variants_meta":{"raw":{"variants":["Finite training sets make stochastic interpolants memorize data","Stochastic generation: training points plus Gaussian noise","Optimal velocity field replays training samples exactly","With finite data, generative models must overfit or underfit"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000665,"raw_usage":{"total_tokens":3024,"prompt_tokens":920,"completion_tokens":2104,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":2041}},"tokens_in":536,"tokens_out":2104,"duration_ms":13042,"temperature":1.0,"reasoning_tokens":2041,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:47:21.218606+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run deterministic generation with the exact optimal velocity field from Proposition 1 on a known finite set; if the endpoint at $t=0$ is not one of the training samples, Theorem 1 fails. For the stochastic claim, generate many samples with known $\\zeta(t)$; if the empirical mean of $Z_0$ does not coincide with some training sample or the empirical covariance is not close to $2\\int_0^1 \\zeta(t)\\,dt$ times the identity matrix, Theorem 2's additive-Gaussian picture fails.","supporting_citations":[{"cited_title":"Diffusion probabilistic models generalize when they fail to memorize","cited_arxiv_id":null,"evidence_quote":"Supplies the memorization criterion used in the ImageNet experiments to quantify copied generated samples."}],"review_version":2}