{"id":"65f0a74c-463d-47c5-a32e-2f27c62a6403","arxiv_id":"2506.22637","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"CaO2 selects confident diffusion-generated samples and optimizes their latents against the denoising objective, achieving state-of-the-art distilled-dataset accuracy on ImageNet subsets.","lead":"This paper proposes a two-stage recipe for dataset distillation: generate extra images with a diffusion model, keep the ones a small classifier is confident about, then tweak the kept images so the diffusion model likes them more. It reports top accuracy on ImageNet and its subsets, making synthetic mini-datasets a more practical substitute for real ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stage 2's mechanism is unverified: minimizing denoising loss w.r.t. the latent need not improve downstream classification, and Definition 2's 'Condition Inconsistency' is trivially true for any continuous density.","rationale":"The reader's conditional verdict is appropriate. The empirical evidence—large gains across ImageNet subsets, cross-architecture consistency, and ablations of each component—is real support for the headline accuracy claim, and I see no internal contradiction in the implementation. The weakest link is the paper's own characterization of why Stage 2 helps. The connection from denoising-loss minimization to downstream classification accuracy is asserted rather than derived; the formal definition of Condition Inconsistency is vacuous for continuous densities; and the ablations do not rule out a much simpler explanation (small-norm perturbation acting as regularization). A random-perturbation control would settle this. I also note the evaluation-protocol concern raised by the reader (best-of-two label paradigms, validation-based condition choice in Eq. (5)): it is a real selection-bias risk, but it concerns the magnitude of the reported 2.3% improvement rather than the existence of the effect, so it supports the conditional verdict without changing it. I therefore recommend the reader's verdict be kept, with the additional requested evidence: code release and the matched perturbation control.","tokens_in":17058,"tokens_out":3961,"duration_ms":46367,"concrete_test":"Use the released code to run a matched-control ablation on ImageNette/ImageWoof IPC=10, ResNet-18: replace Stage 2's Adam update in Eq. (4) with a single random perturbation x ← x + u, where u is drawn uniformly from the same L∞ ball of radius η used by the method, and keep Stage 1 selection, pool size, training schedule, and label paradigm identical. If the random-perturbation control matches the reported Stage-2 gains (within ~0.5%), the mechanism claim fails. As a secondary check, compute the diffusion classifier's softmax confidence pθ(c|x) before and after Stage 2; if confidence does not increase while accuracy does, Eq. (4) is not acting through conditional likelihood.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central conceptual claim is that CaO2 'rectifies' two inconsistencies, and the main empirical contribution is Stage 2 (Sec. 3.3). The asserted mechanism is that minimizing the denoising loss in Eq. (4) w.r.t. x increases the conditional likelihood pθ(x0|c), and that this makes the image a better training example for a downstream discriminative classifier. Neither leg of this inference is established. First, Definition 2 defines Condition Inconsistency as ∀i, ∃j≠i such that pθ(x_i0|c_j)>0; for a continuous diffusion density this holds for essentially every sample, including an ideal one, so it does not characterize the 'mismatch' the method claims to fix. Second, Eq. (4) is an ELBO-style objective, but maximizing an ELBO under a fixed approximate posterior does not guarantee an increase in the exact conditional likelihood or in pθ(c|x0) via Bayes' rule, and it certainly does not bound the risk of a discriminative classifier trained on the optimized samples. The ablations (Tabs. 3-5, Fig. 3d) show accuracy gains, but they do not include a perturbation-matched control: a random latent perturbation within the same L∞ ball could produce equal gains through regularization or data augmentation, which would invalidate the likelihood-maximization story. The paper's own visualization (Fig. 4) notes changes are 'imperceptible,' consistent with a small regularizing effect rather than a principled likelihood correction. Because the SOTA accuracy claim is empirical and the gains are consistent, this is a concern about attribution and mechanism, not a demonstrated failure; but the conceptual contribution of the paper depends on it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CaO2, a two-stage diffusion-based dataset distillation method. Stage 1 generates a pool of class-conditioned images and selects samples using a lightweight classifier's confidence, with the selection criterion (easiest vs hardest correct samples) chosen according to task difficulty. Stage 2 optimizes selected latents to minimize a denoising loss under a chosen condition, with an L-infinity regularizer, intended to increase the conditional likelihood of the sample. The authors frame the work around two 'inconsistencies' in diffusion-based distillation (Objective Inconsistency and Condition Inconsistency) and report state-of-the-art accuracy on ImageNet and its subsets, with an average improvement of 2.3% over baselines. The method is also integrated with a masked autoregressive backbone (MAR) and with Minimax Diffusion.","tokens_in":17406,"tokens_out":8507,"duration_ms":96991,"significance":"Strengths: the paper delivers extensive experiments across multiple datasets, architectures, and IPC settings; the gains over baselines are consistent though often modest; code is released; the method is simple, efficient, and works as a plug-and-play module on different backbones. The ablations cover pool size, selection strategy, condition choice, regularization form, stage ordering, and noise level. These are real empirical contributions. However, the conceptual claims about 'rectifying inconsistencies' are not rigorously established. Definition 2 of Condition Inconsistency is vacuous for continuous densities, and the Stage 2 likelihood-maximization mechanism is asserted rather than verified. The evaluation protocol that reports the higher of hard-label and soft-label accuracy for every method is non-standard and weakens the headline SOTA claim. The empirical method is likely useful, but the paper needs substantial revision to support its conceptual framing and to make the comparisons interpretable.","major_comments":[{"comment":"The evaluation protocol 'report the best result for each method' between hard-label and soft-label paradigms is not a fixed benchmark. Because each reported cell is a maximum over two evaluation protocols, this can systematically inflate accuracy, and the magnitude of the inflation may differ across methods. The average +2.3% claim therefore is not interpretable without showing both hard-label and soft-label numbers separately for every method and cell. Please report both paradigms separately (or pre-register one protocol), and analyze how the ranking changes under each.","section":"Sec. 4.1 and Tabs. 1-2"},{"comment":"Definition 2 states that Condition Inconsistency holds when for every i there exists j != i with p_theta(x_i0 | c_j) > 0. For any continuous density with overlapping class-conditional supports, this condition is true even for a perfectly trained diffusion model, since the likelihood of a sample under a wrong class label is generically nonzero. The definition therefore does not characterize a deficiency of empirical diffusion models or a 'mismatch' between condition and image. A quantitative definition (e.g., log-likelihood ratio p_theta(x|c_i)/p_theta(x|c_j), or the Bayes posterior p_theta(c_i|x)) is needed, or the paper should reframe this as 'suboptimal conditional likelihood' rather than an inconsistency.","section":"Sec. 3.3, Definition 2"},{"comment":"The mechanism claim underlying Stage 2 is not established. Minimizing the denoising loss with respect to x increases the ELBO lower bound in Eq. (2) for the current model, but it does not guarantee an increase in p_theta(x0|c) and, more importantly, no argument links the ELBO to the risk of a downstream discriminative classifier. The ablations in Tabs. 3-5 show accuracy gains, but they do not include a perturbation-matched control where x is randomly perturbed within the same norm budget. Such a control is needed to rule out the alternative explanation that the gains come from a small regularizing or data-augmentation effect rather than from likelihood alignment. Please add this control and, if possible, report a likelihood-based evaluation of the optimized samples.","section":"Sec. 3.3, Eq. (4)"},{"comment":"The text introduces a constraint ||x - x||_inf <= eta, but Eq. (4) uses an L_infinity norm of the denoising error as a penalty term; no projection or clipping step is described. As written, the optimization does not enforce the stated perturbation bound. This is a mismatch between the stated design and the implemented objective, and it affects reproducibility and the interpretation of the 'imperceptible changes' in Fig. 4. Please either enforce the hard constraint explicitly (with projection) or clearly state that the L_infinity error penalty is a proxy and remove the unused eta notation.","section":"Sec. 3.3, text vs Eq. (4)"},{"comment":"The choice of condition c_hat (true vs null) and the selection criterion (most vs least confident) are made per dataset based on validation accuracy, and the paper's ablations show this choice matters. This is effectively a task-level tuning knob. The paper should state explicitly how this choice is made in a new task without peeking at test accuracy, for example by a validation-set rule or a fixed default, and should report sensitivity of the main results to this choice.","section":"Sec. 3.4 and Sec. 4.3"}],"minor_comments":[{"comment":"'we use the size of the distilled dataset as an evidence' should read 'as evidence'.","section":"Sec. 3.4"},{"comment":"'We induce that' should be 'We infer that'.","section":"Appendix Sec. 9"},{"comment":"Ours(w/o SS) is below DiT-IGD on ImageWoof IPC=10 (65.0 vs 67.7); the sentence claiming 'standalone effectiveness' of the single-stage CaO2 should be qualified to reflect this exception.","section":"Appendix Tab. 7"},{"comment":"'Update x using Eq. (4), Eq. (5)' is ambiguous; Eq. (5) defines the condition choice rather than an update rule. Please rephrase.","section":"Algorithm 1, line 10"},{"comment":"References [20] and [21] appear to be the same paper ('Prioritize Alignment in Dataset Distillation') and should be merged.","section":"References"},{"comment":"The statement that latent optimization introduces 'imperceptible changes' is qualitative; please include a quantitative distance metric (e.g., mean absolute difference or PSNR) to support this claim.","section":"Sec. 4.5, Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The empirical core of the paper is plausible and the code release is a plus, but the conceptual framing (especially Definition 2) and the 'best of two protocols' evaluation need substantive work before the SOTA claim can be taken at face value. The lack of a perturbation-matched control for Stage 2 is a specific, testable gap. I would not reject: the method's gains appear consistent, and the requested controls are feasible within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this paper delivers a simple, model-agnostic two-stage pipeline that improves diffusion-based dataset distillation by a consistent few points across ImageNet subsets, and the experiments are extensive enough that the empirical effect is probably real. The conceptual story about 'rectifying inconsistencies' is weaker than the method, and the exact headline number should be treated as an upper estimate.\n\nWhat is new: combining classifier-confidence filtering of a diffusion-generated image pool with latent optimization against the denoising objective is not in the cited baselines. The MAR extension and the plug-and-play integration with Minimax Diffusion are useful. The ablations cover pool size, selection criterion, condition choice, regularization norm, noise level, stage order, and cross-architecture behavior. That is honest work, and the gains are consistent across most cells.\n\nWhere it gets soft. First, the evaluation protocol takes the better of hard-label and soft-label accuracy for every method, and hyperparameters are chosen after looking at validation accuracy. That is not fatal, but it inflates the average advantage and makes the 2.3% claim less clean than it looks. Second, the Stage 2 mechanism is under-supported. Minimizing an ELBO-style denoising loss with respect to the latent does not guarantee higher conditional likelihood, and even if it did, no argument connects that to downstream discriminative accuracy. The ablations show accuracy gains, but there is no perturbation-matched control: random latent perturbations in the same L∞ ball could produce similar gains via regularization. The paper's own visualization shows imperceptible changes, which is consistent with a small regularizing effect. Third, Definition 2 of Condition Inconsistency is essentially trivial for a continuous diffusion density: for any generated sample, the conditional likelihood under other class conditions will be positive. As stated, it does not characterize the mismatch the method addresses. These issues are about attribution and framing, not about whether the method works. I believe the empirical core holds.\n\nWho this is for: anyone working on diffusion-based dataset distillation or data condensation with generative priors would get value from the pipeline and the ablations. It deserves a serious referee, not a desk reject. In review, the main asks should be fixed evaluation protocol, release of code, and a control that isolates latent optimization from generic perturbation.\n\nRecommendation: send to peer review. Conditional accept after revision is the right shape.","headline":"CaO2 is a genuinely useful two-stage heuristic for diffusion-based dataset distillation, with solid empirical gains, but the conceptual framing overreaches and the reported margins should be read with a cautious eye.","tokens_in":17956,"tokens_out":1677,"would_cite":true,"duration_ms":18195,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CaO2 is a two-stage diffusion distillation method that claims a 2.3% average accuracy gain on ImageNet-family datasets by removing Objective and Condition Inconsistencies.","keywords":["dataset distillation","diffusion models","conditional likelihood","sample selection","latent optimization","image classification","ImageNet","classifier-free guidance"],"falsifier":"Two observations would settle the claim: measuring each selected latent's conditional likelihood or diffusion-classifier accuracy before and after Stage 2 — if it does not rise while downstream classifier accuracy rises, the stated mechanism is not driving the gain; or replacing Stage 2 with equal-magnitude random latent perturbations and checking whether the accuracy gain disappears.","tokens_in":16852,"feed_emoji":"🎯","tokens_out":5577,"duration_ms":57318,"temperature":0.7,"pith_summary":"CaO2 is a two-stage, training-free refinement method for diffusion-based dataset distillation. It claims that current diffusion-based distillation methods fail at evaluation time for two identifiable reasons — Objective Inconsistency and Condition Inconsistency — and that fixing both improves downstream classification on compact synthetic datasets. Concretely, the paper reports an average 2.3% accuracy gain over the best baselines on ImageNet and its subsets, with gains on ImageNette reaching 4.3% on average. A reader should care because dataset distillation is how large image collections are compressed for efficient training; making the distilled images carry their labels more faithfully raises the ceiling of every downstream model trained on them.","feed_headline":"CaO2 lifts diffusion-distilled datasets by 2.3% on average","feed_subtitle":"Sample selection plus latent refinement aligns generated images with labels, improving accuracy on ImageNet and subsets","key_machinery":"The carrying mechanism is the two-stage refinement loop. Stage 1 generates an image pool of size $m \\times \\text{IPC}$ per class from a frozen conditional diffusion model and keeps only samples a lightweight pre-trained classifier assigns to the conditioned class, choosing the most confident correct samples at low IPC and the least confident correct samples at high IPC. Stage 2 fixes the diffusion model and optimizes each selected latent $x$ against the objective $\\min_x \\mathbb{E}_{t,\\varepsilon}[\\|\\epsilon_\\theta(x_t, \\hat{c}, t) - \\varepsilon\\|_2^2 + \\lambda\\|\\epsilon_\\theta(x_t, \\hat{c}, t) - \\varepsilon\\|_\\infty]$, where $x_t = \\sqrt{\\alpha_t} x + \\sqrt{1-\\alpha_t}\\varepsilon$ is a mildly perturbed latent and the condition $\\hat{c}$ is chosen as the true class for easy tasks or the unconditional guidance label for hard tasks. This design avoids training or finetuning the generative backbone, which is why it can be transplanted onto DiT, Minimax Diffusion, and masked autoregressive backbones.","core_discovery":"The paper's central claim is that current diffusion-based dataset distillation overlooks the evaluation objective, creating two mismatches: Objective Inconsistency, where generated images can be classified into a different class than their conditioning label, and Condition Inconsistency, where an empirical diffusion model assigns nonzero likelihood to a sample under conditions other than its own. The proposed remedy is a two-stage pipeline: first, generate an image pool and select, per class, the samples a lightweight pre-trained classifier confidently assigns to the conditioned class; second, keep the diffusion model frozen and optimize each selected image latent to minimize the denoising loss under a chosen condition, thereby improving the sample's conditional likelihood. The paper establishes this largely through experiments on ImageNet, ImageNet-100, ImageWoof, and ImageNette across ResNet-18/50/101, EfficientNet-B0, and MobileNet-V2, where CaO2 outperforms the best baselines by an average of 2.3% accuracy.","pith_inferences":["Editorial inference: The latent-refinement step behaves like a lightweight label-alignment pass and should in principle transfer to any distilled dataset whose images were produced by a denoiser, not just to the two generative backbones tested.","Editorial inference: The task-dependent condition switch in Eq. (5) suggests a finer-grained recipe — choosing conditions or noise budgets per class or per difficulty cluster — that the paper does not test.","Editorial inference: Because the reported gains are often largest where samples are scarce, the 2.3% headline may understate the method's value at extreme compression ratios such as IPC=1."],"forward_implications":["Distilled datasets produced by CaO2 train classifiers to higher accuracy than datasets produced by Minimax Diffusion, D4M, RDED, and SRe2L across ImageNet and its subsets, at several images-per-class budgets.","The method improves both hard-label and soft-label evaluation settings, reporting the better of the two per configuration rather than relying on a single evaluation paradigm.","Because the generative backbone is frozen, CaO2 acts as a plug-and-play post-processing step: applying its two stages to a Minimax Diffusion or MAR generation pipeline yields consistent accuracy gains.","The easy-versus-hard condition choice suggests that task difficulty, not just class identity, should determine how synthetic samples are selected and refined during distillation.","The method is computationally light enough to run on a single GPU, making the reported gains available without large-scale optimization or additional model training."],"supporting_citations":[{"why":"DiT is the frozen conditional diffusion backbone used to generate the image pool and as the base model for the pipeline.","marker":"[28]"},{"why":"Minimax Diffusion is the main diffusion-based distillation baseline and one of the backbones CaO2 is plugged into.","marker":"[9]"},{"why":"Shows a diffusion model can act as a zero-shot classifier, providing the likelihood/classification link behind Objective and Condition Inconsistency.","marker":"[18]"},{"why":"Curriculum learning motivates choosing easy confident samples at low IPC and harder samples at high IPC.","marker":"[1]"},{"why":"Single-diffusion robust classification underpins the claim that latent optimization sharpens semantic features and resists adversarial pixels.","marker":"[4]"},{"why":"MAR supplies the masked autoregressive backbone that CaO2 extends to, with masking in place of noise perturbation.","marker":"[19]"},{"why":"RDED is the soft-label evaluation baseline and paradigm that CaO2 is compared against across datasets.","marker":"[34]"},{"why":"D4M is the prototype-clustering diffusion baseline used in ImageNet-1K comparisons.","marker":"[33]"}],"fun_headline_variants":["CaO2 fixes two inconsistencies in diffusion dataset distillation","CaO2 aligns distillation with evaluation to gain 2.3% accuracy","Objective and condition mismatches solved in diffusion distillation by CaO2","CaO2 improves distilled datasets via sample selection and latent refinement","Rectifying inconsistencies: CaO2 boosts diffusion-distilled accuracy by 2.3%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that minimizing the denoising loss of a latent under a fixed condition increases that sample's conditional likelihood, and that this makes the sample a better training example for a downstream classifier; if that link fails, Stage 2 adds nothing beyond Stage 1.","fun_headline_variants_meta":{"raw":{"variants":["CaO2 fixes two inconsistencies in diffusion dataset distillation","CaO2 aligns distillation with evaluation to gain 2.3% accuracy","Objective and condition mismatches solved in diffusion distillation by CaO2","CaO2 improves distilled datasets via sample selection and latent refinement","Rectifying inconsistencies: CaO2 boosts diffusion-distilled accuracy by 2.3%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000864,"raw_usage":{"total_tokens":3733,"prompt_tokens":919,"completion_tokens":2814,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":2720}},"tokens_in":535,"tokens_out":2814,"duration_ms":20150,"temperature":1.0,"reasoning_tokens":2720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:01:24.548098+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Two observations would settle the claim: measuring each selected latent's conditional likelihood or diffusion-classifier accuracy before and after Stage 2 — if it does not rise while downstream classifier accuracy rises, the stated mechanism is not driving the gain; or replacing Stage 2 with equal-magnitude random latent perturbations and checking whether the accuracy gain disappears.","supporting_citations":[{"cited_title":"Efficient dataset distillation via minimax diffusion","cited_arxiv_id":null,"evidence_quote":"Minimax Diffusion is the main diffusion-based distillation baseline and one of the backbones CaO2 is plugged into."},{"cited_title":"Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak","cited_arxiv_id":null,"evidence_quote":"Shows a diffusion model can act as a zero-shot classifier, providing the likelihood/classification link behind Objective and Condition Inconsistency."},{"cited_title":"Curriculum learning","cited_arxiv_id":null,"evidence_quote":"Curriculum learning motivates choosing easy confident samples at low IPC and harder samples at high IPC."},{"cited_title":"On the diversity and realism of distilled dataset: An efficient dataset distilla- tion paradigm","cited_arxiv_id":null,"evidence_quote":"RDED is the soft-label evaluation baseline and paradigm that CaO2 is compared against across datasets."},{"cited_title":"Dˆ4m: Dataset distillation via disentangled diffu- sion model","cited_arxiv_id":null,"evidence_quote":"D4M is the prototype-clustering diffusion baseline used in ImageNet-1K comparisons."}],"review_version":1}