{"id":"3e5a4eae-e414-4135-82a0-c7188c5aaca0","arxiv_id":"2505.06890","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Representation-conditioned diffusion pre-training improves hematoma classification accuracy by +6.15% and F1 by +13.60% over DINOv2 on a 179-image brain CT test set.","lead":"This paper proposes a way to pre-train an image classifier using a denoising model guided by learned image summaries, without needing labeled data. On brain CT scans, it reports higher accuracy and F1 for hematoma detection than a strong contrastive baseline, though the experiment has notable caveats.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported classification gain rests on an unspecified timestep t and a single unseeded evaluation, so the +6.15%/+13.60% margin is not yet established.","rationale":"The reader's verdict is CONDITIONAL with a high correctness risk, and my stress-test agrees that the paper should not be accepted as-is. My most load-bearing objection is slightly different from the reader's stated weakest assumption: rather than the pre-training-loss-to-features transfer assumption, I focus on the unreported and potentially test-selected timestep t in Diffusion Classifier Zero, together with the absence of error bars and repeated seeds. This is a direct threat to the empirical claim itself: if t was chosen favorably on the test set, the headline numbers are not evidence for the method. The reader's rationale does mention the unreported t and missing error bars, but their headline weakness is the transfer assumption; I see the empirical reproducibility issue as the more decisive one. No internal inconsistency or fraud is suggested; the concern is purely about the strength of evidence. The proposed check—pre-registered t selection on validation, multiple seeds, and epsilon averaging—would settle whether the reported margin is real. Until then, the paper remains a conditional accept requiring this evidence.","tokens_in":11507,"tokens_out":4633,"duration_ms":54200,"concrete_test":"Rerun the downstream classification in §5.2 with a pre-registered protocol: pre-train three independent seeds for each model, choose the single timestep t on the 178-image validation split (e.g., grid t = 100, 200, ..., 900), average Eq. (4) over 10 epsilon samples per image, and report test Accuracy/F1 as mean ± std over the three seeds. If the representation-conditioned model does not beat DINOv2 on the test split at the validation-selected t, the claimed +6.15%/+13.60% margin is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the accuracy/F1 improvement over DINOv2 in Table 4. The decisive weakness is that Diffusion Classifier Zero's output depends directly on the hyperparameter t in Eq. (4); the paper itself states in §3.2 that \"performance of image classification results is greatly affected, especially by t, and the selection of t takes time.\" Figure 9 then shows qualitatively different predicted latent images at different t values. Table 4 reports a single accuracy and F1 for each model with no indication of which t was used, how many timesteps were averaged, how many epsilon samples were drawn, or how many independent pre-training/tuning seeds were run. If t was chosen by looking at test-set outcomes, or if a single favorable t/epsilon was used, the reported margin could be a selection artifact rather than a property of the learned representations. The absence of confidence intervals or repeated runs makes it impossible to assess whether a 170/179 versus 159/179 test accuracy difference is stable. This is more directly load-bearing than the paper's mechanistic explanation of why representation-conditioned pre-training helps: even a wrong explanation would not invalidate a reproducible empirical gain, but an unreported test-selected hyperparameter would.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a self-supervised pre-training framework, the Representation-Conditioned Latent Diffusion Transformer, in which a latent diffusion transformer is conditioned on representations obtained from a ViT encoder that is trained jointly with the denoising network. The model is pre-trained on unlabeled brain CT data, then adapted to a binary hematoma classification task by removing the representation conditioning, adding class-label conditioning, and applying a zero-shot classification rule (Diffusion Classifier Zero) that compares the original latent image with predicted latent images per class. The central claim is that this method outperforms DINOv2 by +6.15% accuracy and +13.60% F1-score on a 179-image test set. The paper also reports generation metrics (FID, sFID, IS, Precision/Recall) and argues that the conditioning mechanism enables lower pre-training loss and higher-quality reconstructions.","tokens_in":11737,"tokens_out":2505,"duration_ms":24943,"significance":"If the claimed classification improvement is robust, the paper would offer a meaningful alternative to contrastive pre-training for small medical datasets, with the added benefit of having a generative model in the same framework. The architecture is clearly described, the use of an unlabeled pre-training corpus is a credible direction, and the paper ships a concrete experimental setting. However, the strength of the evidence is currently insufficient to support the central claim because the classification evaluation omits a key sensitive hyperparameter and lacks any uncertainty quantification, while the generation evaluation is partly circular due to conditioning on the target image's own representation.","major_comments":[{"comment":"The classification result in Table 4 is not reproducible as reported because the timestep t used in Eq. (4) is not specified, and the paper itself states in §3.2 that classification performance is greatly affected by t and that selecting t takes time. The number of timesteps averaged, the number of noise samples epsilon drawn, and the number of independent pre-training/tuning seeds are also not reported. Without this information, and without confidence intervals or repeated runs, the reported +6.15% accuracy and +13.60% F1 margin over DINOv2 could be a selection artifact rather than a property of the learned representations.","section":"§3.2 and Table 4"},{"comment":"The generation evaluation in Table 3 is confounded by the conditioning mechanism: the representation r is computed as r = f_phi(z0) from the target image's own latent representation (Eq. 1), so the representation-conditioned model is effectively tasked with reconstructing the input rather than generating new samples from the data distribution. The low FID of 10.00 is therefore expected and does not provide independent evidence that the model has learned a meaningful generative model of brain CTs; the comparison against unconditional and class-conditioned models is not a fair head-to-head generation test.","section":"§5.1 and Eq. (1)"},{"comment":"The comparison of pre-training losses (3.13×10^-2 vs 3.39×10^-4, a difference of about 10^2) is largely by construction: conditioning on the target image's own representation necessarily makes the denoising task easier, since the conditioning channel leaks information about the clean image. The claim that this lower loss indicates 'deeper training' or that it explains the downstream classification gain is not supported by any analysis linking pre-training loss to classifier performance, especially because the representation r is discarded during tuning and testing. The loss difference alone is not evidence that the learned representations are better.","section":"§5.2 and Figure 8"}],"minor_comments":[{"comment":"There are several typos in this section: 'Represention-Conditioned' should be 'Representation-Conditioned', 'it it combined' should be 'it is combined', 'optimizeing' should be 'optimizing', and 'an arbitary timestep' should be 'an arbitrary timestep'.","section":"§3.1"},{"comment":"The text contains typos such as 'Transfromer' and 'obtaines' in the description of Diffusion Classifier Zero; these should be corrected.","section":"§3.2"},{"comment":"The DINOv2 baseline is underdocumented: the paper does not specify the DINOv2 checkpoint size (ViT-S/B/L), the pre-training setup (number of epochs, optimizer, data augmentations) on the brain CT dataset, or the linear probe / fine-tuning procedure used for the downstream classification. This makes the comparison in Table 4 difficult to interpret.","section":"§4.3"},{"comment":"In Figure 7 the paper generates samples by starting from an intermediate timestep t=100 rather than from complete noise; while this is a reasonable visual comparison, the caption and text should state clearly that these are conditional reconstructions from partially noised latents, not unconditional samples from the learned distribution.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is interesting and the reported classification improvement is worth investigating, but the current experimental support is not strong enough for acceptance. The most pressing issue is the unreported timestep t and the absence of variance estimates; if the authors can supply this information and show the result is stable across seeds and t values, the paper could become publishable. The generation results should also be reframed as reconstruction, not generation, to avoid the appearance of circularity."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe genuinely new piece here is training the ViT encoder that produces the conditioning representation jointly with the diffusion denoising network. Prior work (Rombach et al., Bordes et al.) uses a frozen pretrained encoder for conditioning, so this is a real difference, and the paper says so clearly. Diffusion Classifier Zero, scoring by predicted latent image rather than predicted noise, is a small but legitimate variant of Li et al. The paper also picks a meaningful testbed: brain CT hematoma detection where labeled data is scarce, and DINOv2 is a strong baseline.\n\nThe problem is the experimental support for the headline claim. The paper's own Section 3.2 says classification performance is 'greatly affected' by timestep t and that choosing t takes time. Figure 9 shows qualitatively different predictions at different t. Table 4 reports a single accuracy and F1 with no t, no number of epsilon samples, no seeds, no error bars. On a 179-image test set, the difference between 170/179 and 159/179 could easily move with one or two images. If t was selected by peeking at test outcomes, the reported margin is an artifact. This is more load-bearing than the mechanistic explanation: even if the 'deeper training' story is wrong, a reproducible gain would stand, but an unreported test-selected hyperparameter would not.\n\nThe generation results are also partly circular: conditioning on r = f_phi(z0) makes the FID of 10.00 essentially a reconstruction score, and the 10^2 lower pre-training loss is by construction. The paper presents these as evidence for representation quality, which they are not. The loss comparison in Figure 8 cannot be used to argue that the conditioning mechanism learns better features.\n\nThe classification setup is also not 'zero-shot' in the usual sense: the denoising network is tuned on the downstream labeled training set before the Diffusion Classifier Zero rule is applied. The comparison to DINOv2 is fair in that both are tuned, but the label 'zero-shot' will mislead.\n\nNone of this kills the idea. Jointly trained conditioning is worth exploring, and the authors are honest about the t sensitivity. But the central empirical claim needs a proper evaluation: report t, average over several epsilon draws, run multiple seeds, give confidence intervals, and document the DINOv2 setup. The paper deserves peer review, not desk rejection, but the reviewers should press hard on these points.\n\nI'd bring it to our reading group mainly as a case study in evaluation pitfalls, and I wouldn't cite it until the timestep and error bars are resolved.","headline":"Genuine twist on conditioning, but the headline classification gain rests on an unreported timestep and a single unseeded run.","tokens_in":12274,"tokens_out":2379,"would_cite":false,"duration_ms":21594,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a self-supervised diffusion pre-training objective, conditioned on a jointly trained Vision Transformer representation of the input's own latent image, outperforms the contrastive DINOv2 baseline by +6.15% accuracy…","keywords":["self-supervised learning","diffusion models","latent diffusion transformer","representation conditioning","zero-shot classification","brain hematoma detection","medical image classification","contrastive learning"],"falsifier":"One experiment would settle it: pre-train two representation-conditioned models identically, except that in the control the conditioning encoder receives the latent image of a fixed random image rather than the input's own $z_0$. If the control matches the reported accuracy and F1, the classification gains are not caused by the representation learning the paper claims; if it does not, the conditioning mechanism is doing the work. A second check is to evaluate the reported tuned checkpoints on CT data from a facility excluded from both pre-training and downstream training: if the gains vanish under scanner shift, the acquired features are memorization-like rather than general.","tokens_in":2180,"feed_emoji":"🧠","tokens_out":2194,"duration_ms":96991,"temperature":0.7,"pith_summary":"This paper sets out to show that a diffusion model can act as a self-supervised pre-training backbone for image classification, competitive with or better than contrastive learning when labeled data are scarce. It proposes the Representation-Conditioned Latent Diffusion Transformer, where a Vision Transformer encoder maps the clean latent image $z_0$ to a representation $r=f_\\phi(z_0)$ that conditions the denoising network at every block, and the encoder is trained jointly with the denoiser by minimizing $\\|\\epsilon-g_\\theta(z_t,r,t)\\|^2$. Pre-trained on 10,313 unlabeled brain CT slices and adapted with 1,424 labeled slices, the method classifies hematoma presence through Diffusion Classifier Zero, selecting the class label whose predicted latent image is closest to the input in $\\mathbb{E}_{t,\\epsilon}\\|z_0-z'_0\\|^2$. The paper reports accuracy $0.9497$ and F1 $0.8235$, beating DINOv2 by $+6.15\\%$ accuracy and $+13.60\\%$ F1. If these results hold, diffusion pre-training offers a low-annotation route to high-performance medical image classification.","feed_headline":"Diffusion pre-training beats DINOv2 by 6.15% on brain CT","feed_subtitle":"Trained on unlabeled CT, it also lifts F1 by 13.60% over the contrastive baseline.","key_machinery":"The load-bearing object is the Representation-Conditioned Latent Diffusion Transformer, a DiT-style denoiser in the latent space of a fixed VAE, conditioned at two points—after the final decoding layer and at the start of each transformer block—on a representation $r=f_\\phi(z_0)$ produced by a Vision Transformer encoder from the noiseless latent image. The representation encoder is trained jointly with the denoiser through the loss $L=\\|\\epsilon-g_\\theta(z_t,r,t)\\|^2$, which distinguishes this method from prior work that conditions on features from a frozen pre-trained encoder. Classification is carried out by Diffusion Classifier Zero: for each candidate class label $c$, the tuned denoiser predicts the latent image $z'_0$ from the noised input, and the label minimizing $\\mathbb{E}_{t,\\epsilon}\\|z_0-z'_0(c)\\|^2$ is selected. The machinery therefore turns reconstruction fidelity at a controlled timestep into a class decision.","core_discovery":"The central discovery claimed is that conditioning a latent diffusion transformer on the input's own learned representation makes the denoiser both reconstruct better and transfer better to classification. With representation conditioning, pre-training loss falls to $3.39\\times10^{-4}$ compared with $3.13\\times10^{-2}$ for the unconditional model, and predicted latent images become close to the original at much earlier timesteps (around $t=800$, versus $t=400$ unconditionally). The paper interprets these as signs that the conditioning signal forces 'deeper training' of the denoising network, producing representations that survive the removal of the conditioning branch and remain useful when the model is re-tuned as a class-conditioned classifier. In the hematoma classification benchmark, that produces accuracy $0.9497$ and F1 $0.8235$, which is $+6.15\\%$ accuracy and $+13.60\\%$ F1 over DINOv2, and $+6.15\\%$ accuracy and $+11.77\\%$ F1 over the same architecture without representation conditioning.","pith_inferences":["Implicit but not established: the classification gain is attributed to 'deeper training' from easier conditioning; a direct test would be to freeze the ViT encoder during pre-training and check whether the downstream gain disappears, which would prove the joint optimization matters.","Implicit but not established: the representation $r$ is discarded after pre-training, so an alternative explanation is that the denoiser simply overfits to reconstruction of the pre-training distribution; a domain-shift test with a new scanner or facility would show whether the features generalize.","Testable extension: apply the same recipe to other small, imbalanced medical sets such as fundus images or skin lesions, where the paper argues annotation costs are high; the reported $+6.15\\%$ accuracy claim is currently a single-dataset result.","Testable extension: use $r=f_\\phi(z_0)$ directly as a frozen feature extractor for a linear probe; the paper's conditioning premise predicts that this representation should be more useful than the denoising network's hidden states, but this is not reported."],"forward_implications":["If correct, self-supervised diffusion pre-training on unlabeled medical scans can outperform a strong contrastive baseline (DINOv2) on a small downstream labeled set, reducing annotation burden.","Representation conditioning improves generation quality as well, with FID dropping to 10.00 from 24.42 (unconditional) and 25.69 (class-conditioned), so one pre-training recipe serves both generation and classification.","The large pre-training loss gap ($3.39\\times10^{-4}$ vs $3.13\\times10^{-2}$) suggests that representation conditioning stabilizes and deepens optimization, which the paper connects to finer and earlier prediction of the latent image.","Diffusion Classifier Zero, which classifies by comparing predicted latent images rather than predicted noise, extends zero-shot classification to models pre-trained without labels, avoiding the large labeled dataset required by prior diffusion classifiers.","The classification gains are concentrated in F1 and precision ($0.8235$ and $0.9130$, versus $0.6875$ and $0.6111$ for DINOv2), meaning the method reduces false positives in hematoma screening."],"supporting_citations":[{"why":"Supplies the DiT backbone, model sizes, and the class-conditioned latent diffusion paradigm that is generalized here.","marker":"(Peebles and Xie 2022)"},{"why":"Provides the latent diffusion framework and the fixed VAE encoder that maps images to $z_0$.","marker":"(Rombach et al. 2022)"},{"why":"Introduces the diffusion-as-zero-shot-classifier idea that Diffusion Classifier Zero adapts from noise comparison to latent-image comparison.","marker":"(Li et al. 2023)"},{"why":"Supplies the prior analysis of diffusion models as self-supervised encoders that this method builds on.","marker":"(Chen et al. 2024)"},{"why":"Defines the Vision Transformer used as the representation encoder $f_\\phi$.","marker":"(Dosovitskiy 2020)"},{"why":"Is the DINOv2 contrastive baseline whose accuracy and F1 the proposed method claims to beat.","marker":"(Oquab et al. 2023)"},{"why":"Provides the brain CT preprocessing and hematoma-labeling protocol that defines the benchmark.","marker":"(Hirano and Okita 2021)"}],"fun_headline_variants":["Diffusion pre-training: +6.15% accuracy, +13.6% F1 over DINOv2","Representation-conditioned diffusion ups brain CT accuracy by 6.15%","Self-supervised diffusion pre-training lifts F1 by 13.6% on hematoma","Diffusion pre-training with ViT conditioning beats DINOv2 on CT","Pre-training diffusion on unlabeled CT outperforms contrastive baseline"],"cache_read_input_tokens":14464,"weakest_assumption_plain":"The load-bearing premise is that a very low denoising loss during pre-training, achieved by giving the denoiser the image's own representation as a condition, is evidence that the network learned broadly useful features that transfer to classification even though that same representation is discarded during tuning and testing.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion pre-training: +6.15% accuracy, +13.6% F1 over DINOv2","Representation-conditioned diffusion ups brain CT accuracy by 6.15%","Self-supervised diffusion pre-training lifts F1 by 13.6% on hematoma","Diffusion pre-training with ViT conditioning beats DINOv2 on CT","Pre-training diffusion on unlabeled CT outperforms contrastive baseline"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001002,"raw_usage":{"total_tokens":4205,"prompt_tokens":879,"completion_tokens":3326,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":3216}},"tokens_in":495,"tokens_out":3326,"duration_ms":22042,"temperature":1.0,"reasoning_tokens":3216,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:30:07.696801+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One experiment would settle it: pre-train two representation-conditioned models identically, except that in the control the conditioning encoder receives the latent image of a fixed random image rather than the input's own $z_0$. If the control matches the reported accuracy and F1, the classification gains are not caused by the representation learning the paper claims; if it does not, the conditioning mechanism is doing the work. A second check is to evaluate the reported tuned checkpoints on CT data from a facility excluded from both pre-training and downstream training: if the gains vanish under scanner shift, the acquired features are memorization-like rather than general.","supporting_citations":[{"cited_title":"Classification of Hematoma: Joint Learning of Semantic Segmentation and Classification","cited_arxiv_id":"2103.17172","evidence_quote":"Provides the brain CT preprocessing and hematoma-labeling protocol that defines the benchmark."}],"review_version":1}