{"id":"696dc6f4-d101-4ba3-bac1-27d0877d5fcc","arxiv_id":"2501.11318","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"NATS, a nested annealed training scheme for GANs, is claimed to improve FID/IS on CIFAR10, LSUN, CelebA, and ImageNet64, but its key theoretical justification is not provided in the preprint.","lead":"This paper proposes adding an 'annealed' weight to GAN training, so the discriminator's gradient is scaled down as training progresses, plus a nested loop that trains the generator several times per discriminator update. The authors report better FID and Inception scores on several image datasets, but the central proof is deferred to a missing appendix and the method depends on hand-tuned weights.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 4 is unsupported and appears false: differentiating Algorithm 3's log(1 - w[j]·D(G)) loss gives a different generator direction than Algorithm 2's w_m(x)∇_xD_m(x), and the cited proof is in a missing appendix.","rationale":"The strongest claim has two halves: a theoretical equivalence and a claimed empirical gain. The theoretical half is the load-bearing joint, and the reader's weakest-assumption analysis identifies it correctly. Algorithm 3's generator loss log(1 - w[j]D(G)) is differentiated with respect to generator parameters; Algorithm 2's annealed CFG generator direction is a scaled input gradient w_m(x)∇_xD_m(x). These are different objects even before considering the absent appendix: one carries a denominator (1 - wD), the other does not; one is a parameter-space chain-rule product involving G_θ, the other is an input-space functional direction accumulated over M steps. No formula in the paper connects them. The empirical half cannot rescue this because the reported tables lack code, seeds, error bars, and use per-dataset tuned schedules (Nd and w), so the quantitative claim is independently unverifiable. There is no machine-checked proof or released code that would supply independent support. Since the central theoretical mechanism rests on the unsupported Proposition 4, and direct computation indicates the printed loss does not produce the claimed field, the rejection stands. My read does not change the reader's verdict; it reinforces it.","tokens_in":21608,"tokens_out":4882,"duration_ms":57175,"concrete_test":"Fix the simplest nontrivial setting: x ∈ R, G(z) = a z, D(x) = x², and w[j] = c. Annealed CFG gives direction c·∇_xD = 2cx = 2caz. NATS gives d/da log(1 - cD(G(z))) = -2caz²/(1 - ca²z²). For any c≠0 and z≠0 these are not proportional: the ratio depends on a and z, so the gradient-vector fields cannot coincide. Running this one-dimensional check analytically, or asking the authors to supply the missing Appendix B derivation that recovers equality under explicit assumptions, settles whether Proposition 4 holds. If the derivation cannot be produced, the equivalence claimed in Proposition 4 should be withdrawn.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is Proposition 4: the NATS generator has the same gradient-vector field as the annealed CFG method. This equivalence is the only theoretical bridge that lets the authors transfer the annealed-CFG weight to arbitrary GAN losses. The bridge is not built in the text. Annealed CFG (Algorithm 2) advances the generator in data space by g_m(x) = w_m(x)∇_xD_m(x), as stated in Eq. (8). NATS (Algorithm 3, line 10) instead updates generator parameters by ascending the stochastic gradient of (1/m)Σ log(1 - w[j]·D(G(z))). Direct differentiation gives ∇_θ log(1 - w[j]D(G(z))) = - w[j] D'(G(z)) G_θ(z) / (1 - w[j]D(G(z))). This contains a denominator (1 - w[j]D) that has no counterpart in Algorithm 2, the coefficient w[j] multiplies the discriminator output inside a nonlinear log rather than scaling the input gradient, and the directional structure is D'(G)G_θ, not an accumulated sum of input-space gradients along an M-step composition. For the two vector fields to be equal, one would need a nontrivial identity relating the denominator, the Jacobian, and the nested accumulation; no such identity is derived. The proof pointer in Section III-A ('Appendix Section B') does not resolve this: Appendix B is not present in the submission. Thus the load-bearing theoretical claim is not merely unproven; as written, the printed algorithms describe different gradient objects.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims to establish a theoretical connection between the composite-functional-gradient GAN (CFG) and score-based models, to derive an annealed weighting of the CFG discriminator gradient, and to introduce a nested annealed training scheme (NATS) that transfers this annealing to arbitrary GAN losses. The central theoretical claim is Proposition 4, which states that the generator update in NATS has the same gradient-vector field as the annealed CFG method. The paper reports FID and IS improvements on CIFAR-10, LSUN, CelebA, and ImageNet64 across several GAN families, including BigGAN and DDGAN.","tokens_in":21965,"tokens_out":5273,"duration_ms":53078,"significance":"If Proposition 4 were true, the paper would offer a principled way to import annealed weighting from score-based models into any GAN, with broad practical value. The paper should be credited for Proposition 1, which is a correct and clean restatement of the optimal CFG discriminator gradient as a difference of score functions, and for a broad experimental comparison across original GAN, LSGAN, WGAN, HingeGAN, BigGAN, and DDGAN, including an ablation over N_d. However, the contribution rests on Proposition 4, and that proposition is not established: its proof is deferred to a missing appendix, and the algorithm-level gradient objects in Algorithms 2 and 3 are different. The empirical gains, while suggestive, are conditional on dataset-specific hyperparameter choices and are reported without statistical uncertainty.","major_comments":[{"comment":"The load-bearing claim is not supported and, as stated, is inconsistent with the printed algorithms. In annealed CFG, the generator is updated in data space by G_{m+1}(z) = G_m(z) + η_m w_m(x) ∇_x D_m(x), whereas Algorithm 3 updates generator parameters θ_g by ascending ∇_θ (1/m) Σ_i log(1 - w[j] D(G(z_i))). Differentiating the latter yields terms of the form - w[j] D'(G(z)) G_θ(z) / (1 - w[j] D(G(z))), which contain a denominator and a Jacobian structure that have no counterpart in Algorithm 2's scaled input-gradient accumulation. The text refers to Appendix Section B for the proof of Proposition 4, but no appendix is present in the preprint. The claimed equality of gradient-vector fields therefore requires either a corrected algorithm or a nontrivial identity that the manuscript does not provide.","section":"III-D, Proposition 4; Algorithm 3 line 10 vs Algorithm 2 lines 5-6"},{"comment":"These propositions are stated without proof, and Proposition 3's notation is not well-defined: Eq. (8) uses δ^(i)_m, r^(i)_m, and D^(i)_m without specifying what i indexes or how N relates to the M accumulation steps of Eq. (1), and the claim that w_m(x) = δ_m(x) r_m(x) is asserted rather than derived. Since Eq. (8) is the basis for the annealed weight and for Proposition 4, this gap is important.","section":"III-C, Propositions 2 and 3"},{"comment":"The annealed weight schedule is not derived from first principles: Technique 1 sets w_1 from the median pairwise distance D in Table I, and the remaining r_m are chosen as a geometric progression with a ratio set empirically; Table V then fixes different w(x) vectors for low- and high-resolution data. The reported improvements are therefore conditional on hyperparameters that are fitted to the evaluation datasets. The paper should report the selection procedure and its sensitivity more completely, for example through cross-validation or a fixed rule, before claiming that NATS significantly improves SOTA GANs.","section":"III-C, Initial Annealed Weight; Tables I and V"},{"comment":"The empirical claims are reported without standard deviations, number of seeds, or statistical tests, despite the use of the word 'significant'. Table X is also confusing: the CFG row lists Reverse NATS = 200.01, NATS = 16.34, NTS = 19.41, and CTS = 19.41, but NATS, NTS, and CTS are defined for GAN training schemes, and it is unclear how they apply to CFG. This weakens the support for the headline empirical claim.","section":"IV-B, Tables IX-XII"}],"minor_comments":[{"comment":"The affiliation line contains a typo: 'Minglu Li, Fellow, IEEE,,' has a double comma.","section":"Author affiliation line"},{"comment":"The text repeatedly refers to Appendix Sections B, C, D, and E, but none of these appendices appears in the preprint, making it impossible to verify the cited proofs and additional results.","section":"Section III-A and general references"},{"comment":"The update writes η_m g(G_m(z)) where the subscript on g is missing; it should be g_m to match Eq. (1).","section":"Algorithm 2, line 6"},{"comment":"The notation 'z ∼ N(x | 0, I)' appears to mean z ∼ N(0, I), and the double summation over i and m is notationally unclear.","section":"Proposition 2"},{"comment":"The text uses w(x), w_m(x), w[m], and w[j] for closely related quantities without a consistent convention; this makes Algorithm 3, line 10, ambiguous because w[j] is a scalar while w(x) is described as a vector.","section":"Notation for w"},{"comment":"The metric is labeled 'Fr´echet Distance'; it should be 'Fr´echet Inception Distance' to match the text and the cited implementation.","section":"Tables VII and VIII"}],"recommendation":"reject","confidential_remarks":"The stress-test concern is well founded. Proposition 4 is the hinge of the paper, and the discrepancy between Algorithms 2 and 3 is not a presentational issue: standard differentiation of the Algorithm 3 objective produces a different gradient object from the annealed CFG update. The repeated references to missing appendices further prevent verification of the theoretical claims. I do not see a route to publication in the current scope; a major revision would need to replace Proposition 4 with a correct statement and a real proof, or re-scope the paper as an empirical study with proper controls and statistical reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what is actually new: the NATS scheme—a nested training loop where the generator gets j updates per discriminator update, with a scalar w[j] multiplying the discriminator output inside the log-loss—is not in the cited prior work, and the reported FID/IS numbers across vanilla GAN, LSGAN, WGAN, HingeGAN, BigGAN, and DDGAN are concrete. The reverse-annealing ablation is a nice check that the direction of the weights matters. Proposition 1 is correctly derived from Eq. (7), though it is the standard density-ratio identity in f-divergence GAN theory; the score-function reading is fair.\n\nThe soft spots are not minor. Proposition 4 is the load-bearing claim that NATS's generator gradient-vector field equals annealed CFG's. The proof is deferred to an Appendix B that is not present. Direct differentiation of Algorithm 3's generator loss gives ∇θ log(1 - w[j]·D(G(z))) = -w[j] D'(G) Gθ / (1 - w[j]D(G)). That is not the same object as Algorithm 2's w_m(x) ∇x D_m(x) accumulated along the M-step composition: the denominator, the Jacobian structure, and the accumulation pattern have no counterpart. So as written, the paper is comparing two different gradient fields. This is not a missing-detail issue; it is an apparent contradiction between the printed algorithm and the proposition.\n\nBeyond that, Propositions 2 and 3 are stated without proof and have notation problems. The annealed weight schedule is fitted to the data: w1 is tied to median pairwise distances from Table I, r_m is a chosen geometric ratio, and Nd is tuned per model (Table VI). So the gains are not parameter-free predictions. The experiments report no code, seeds, or error bars; with a handful of numbers, the improvements are hard to verify.\n\nWhat is worth keeping: the nested-loop idea with an annealed scalar weight is a plausible practical trick, and if the empirical gains survive reproducibility, they would be a useful empirical note. But the theory as presented does not support itself. A rewrite that drops or fixes Proposition 4, labels the heuristic status honestly, and releases code with seeds could change my read. As submitted, I would reject. If I were an editor, I would still send it to a referee because the empirical recipe deserves scrutiny, but it would need heavy revision.","headline":"A concrete GAN training recipe with promising-looking FID numbers, but the central theory (Prop 4) is unsupported and as written appears to conflate two different gradient objects.","tokens_in":22502,"tokens_out":5512,"would_cite":false,"duration_ms":52807,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A nested annealed training scheme gives any GAN the generator dynamics of annealed CFG and reports lower FID and higher Inception Score across model families.","keywords":["generative adversarial networks","score-based model","annealed weight","composite functional gradient","nested training scheme","image generation","Fréchet Inception Distance","Inception Score"],"falsifier":"Compute the generator gradient of the NATS objective and compare it numerically to the annealed CFG gradient during an actual training run: the NATS gradient is $-w[j] D'(G(z))/(1-w[j]D(G(z))) \\cdot \\partial G/\\partial\\theta_g$, whereas the annealed CFG gradient is $w_m(x)\\nabla_x D_m(x)\\cdot\\partial G/\\partial\\theta_g$. If the two vector fields diverge in direction or magnitude, Proposition 4 fails; if they match, the claim is confirmed.","tokens_in":21384,"feed_emoji":"🖼️","tokens_out":8637,"duration_ms":77110,"temperature":0.7,"pith_summary":"This paper tries to give GAN training a score-function foundation. It argues that in the composite-functional-gradient GAN (CFG), the optimal discriminator's gradient equals the difference between the score functions of real and synthesized samples, and that the generator is best seen as minimizing that score difference. From score-based model annealing, it derives a decreasing weight on the discriminator gradient, calling the result annealed CFG, and then packages that weight into a nested training loop (NATS) that can be inserted into existing GANs without changing their loss or architecture. If the paper's central equivalence holds, NATS would let any GAN inherit annealed CFG's generator dynamics and improve sample quality and diversity. The experiments report lower FID and higher Inception Score from the original GAN through BigGAN and DDGAN.","feed_headline":"Nested annealing lowers GAN FID scores across model families","feed_subtitle":"The same trick improves classic GANs through BigGAN and DDGAN, with gains on CIFAR-10 and ImageNet-64.","key_machinery":"The load-bearing object is the annealed weight vector $w$, a decreasing sequence of scalars that multiplies discriminator gradients in annealed CFG and sits inside the generator loss in NATS. The nested training loop is the second piece: by training the discriminator once and the generator $j$ times at step $j$, it creates a sub-loop in which accumulated discriminator gradients can guide the generator, which the common one-step alternating scheme lacks. Proposition 4, claiming equality of the NATS and annealed CFG gradient-vector fields, is what lets the annealed weight transfer across architectures, losses, and regularizations.","core_discovery":"The central claim is that the CFG discriminator's gradient $\\nabla_x D(x)$ is $\\nabla_x \\log p^*(x) - \\nabla_x \\log p_g(x)$, the difference of the score functions of real and synthesized data, and that the CFG generator's objective is minimized when this difference vanishes. On this basis the paper introduces an annealed weight $w_m(x)$ in front of each discriminator gradient, inspired by annealed Langevin dynamics, and names the result annealed CFG. To make the idea usable with modern GANs, it proposes NATS: a nested scheme where the discriminator is updated once and the generator several times, with the generator objective $\\log(1 - w[j]D(G(z)))$ carrying the weight. Proposition 4 states that the NATS generator has the same gradient-vector field as annealed CFG, so the annealing benefit transfers to any GAN that adopts the scheme.","pith_inferences":["Editorial inference: if the score-difference identity is the mechanism behind the gains, then NATS training should visibly shrink the distance between the generator's score and the data score; measuring both scores during training would test that mechanism directly.","Editorial inference: the nested loop is itself a training-schedule change, so the paper's NTS-versus-NATS comparison can be read as separating the benefit of annealing from the benefit of extra generator steps per discriminator update.","Editorial inference: putting $w[j]$ inside the logarithm makes the weight interact with the discriminator output before the gradient is taken, while annealed CFG scales the input-gradient directly; a direct comparison of these two vector fields would show whether the equality claimed in Proposition 4 is exact or only approximate."],"forward_implications":["If Proposition 4 is correct, any GAN trained with NATS reproduces annealed CFG's generator dynamics without redesigning the model, so the annealed-training benefit becomes architecture-agnostic.","The scheme should keep working when the loss changes, since the modification is confined to the generator update and the annealed weight.","On the paper's empirical results, NATS lowers CIFAR-10 FID for all tested GAN families and improves the state-of-the-art baselines: BigGAN from 8.52 to 6.36 and DDGAN from 3.75 to 2.56.","The reverse-annealing experiment in the paper supports the direction of the weight schedule: reversing $w$ worsens FID, consistent with the annealing rationale.","The score-function identity gives GAN training a diagnostic handle: the discriminator gradient is interpretable as the mismatch between the real-data score and the generator's score."],"supporting_citations":[{"why":"It defines the CFG framework whose discriminator-gradient formula and generator update the paper builds on.","marker":"[1]"},{"why":"It provides the extended statement of the CFG framework used as the empirical baseline for annealed CFG.","marker":"[13]"},{"why":"It supplies the annealed Langevin dynamics and geometric noise-scale schedule that motivate the annealed weight $w_m(x)$.","marker":"[14]"},{"why":"It introduces the original GAN training loop that NATS extends, and provides one of the baseline models.","marker":"[9]"},{"why":"It is the state-of-the-art GAN used to test NATS, and its scores are the standard the paper claims to beat.","marker":"[24]"},{"why":"It is the denoising diffusion GAN used as the second state-of-the-art testbed for NATS.","marker":"[25]"},{"why":"It defines the Inception Score used to measure sample quality.","marker":"[26]"},{"why":"It defines the Fréchet Inception Distance used to measure quality and diversity.","marker":"[27]"}],"fun_headline_variants":["Nested annealing reduces GAN FID across model families","Annealed CFG and NATS: improved GAN sample quality","Nested annealed training: gains on CIFAR-10 and ImageNet-64","Universal GAN trick: nested annealing lowers FID","Nested annealing: better GANs on multiple benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the NATS generator update, which places the annealed weight $w[j]$ inside $\\log(1 - w[j]D(G(z)))$, yields exactly the same gradient-vector field as annealed CFG, which scales the input gradient $\\nabla_x D_m(x)$ by $w_m(x)$; the proof of this equality is deferred to an appendix that is absent from the preprint.","fun_headline_variants_meta":{"raw":{"variants":["Nested annealing reduces GAN FID across model families","Annealed CFG and NATS: improved GAN sample quality","Nested annealed training: gains on CIFAR-10 and ImageNet-64","Universal GAN trick: nested annealing lowers FID","Nested annealing: better GANs on multiple benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000874,"raw_usage":{"total_tokens":3825,"prompt_tokens":1030,"completion_tokens":2795,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":2706}},"tokens_in":646,"tokens_out":2795,"duration_ms":22656,"temperature":1.0,"reasoning_tokens":2706,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:24:38.532162+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the generator gradient of the NATS objective and compare it numerically to the annealed CFG gradient during an actual training run: the NATS gradient is $-w[j] D'(G(z))/(1-w[j]D(G(z))) \\cdot \\partial G/\\partial\\theta_g$, whereas the annealed CFG gradient is $w_m(x)\\nabla_x D_m(x)\\cdot\\partial G/\\partial\\theta_g$. If the two vector fields diverge in direction or magnitude, Proposition 4 fails; if they match, the claim is confirmed.","supporting_citations":[{"cited_title":"Composite functional gradient learning of generative adversarial models,","cited_arxiv_id":null,"evidence_quote":"It defines the CFG framework whose discriminator-gradient formula and generator update the paper builds on."},{"cited_title":"A framework of composite functional gradient methods for generative adversarial models,","cited_arxiv_id":null,"evidence_quote":"It provides the extended statement of the CFG framework used as the empirical baseline for annealed CFG."},{"cited_title":"Generative modeling by estimating gradients of the data distribution,","cited_arxiv_id":null,"evidence_quote":"It supplies the annealed Langevin dynamics and geometric noise-scale schedule that motivate the annealed weight $w_m(x)$."},{"cited_title":"Generative adversarial nets,","cited_arxiv_id":null,"evidence_quote":"It introduces the original GAN training loop that NATS extends, and provides one of the baseline models."},{"cited_title":"Improved techniques for training gans,","cited_arxiv_id":null,"evidence_quote":"It defines the Inception Score used to measure sample quality."},{"cited_title":"Gans trained by a two time-scale update rule converge to a local nash equilibrium,","cited_arxiv_id":null,"evidence_quote":"It defines the Fréchet Inception Distance used to measure quality and diversity."}],"review_version":1}