{"id":"6c16bd09-4cb4-433f-b5dc-3ae619a09ee3","arxiv_id":"2508.01719","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A diffusion model trained without labels provides features that let a small classifier recognize 24 modulation types at 88.27% accuracy with only 10 labeled signals per type.","lead":"This paper introduces ModFus-DM, a two-stage system that first trains a diffusion model on unlabeled radio signals and then uses its internal features to classify modulation types with very few labeled examples. It reports large gains over supervised and self-supervised baselines on four standard AMC datasets, including under distribution shifts and variable signal lengths.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II's T=100 mean cannot be reproduced from its own row (reported 89.29 vs 92.66 from the five listed per-SNR accuracies), and that T=100 is the setting behind every headline result, so the empirical core is currently unverified without code or raw logs.","rationale":"The reader identified temporal flexibility as the weakest assumption and noted the T=100 mean inconsistency as one of several weaknesses. I agree the crop-based variable-length protocol is unprotected, but the more load-bearing issue is upstream: the T=100 configuration is the one used for every headline experiment, and its mean in Table II is arithmetically inconsistent with its own row. The headline 88.27% claim lives in Table III, which inherits this configuration. Because the manuscript provides no code, no raw logs, and no error bars, the reader cannot determine whether the published numbers are accurate or contain transcription/computation errors. This concern is not about theoretical soundness: Equations (3)-(17) describe a coherent diffusion feature-extraction pipeline, and the ablation logic is reasonable. The problem is that the empirical basis for the central claim is not currently verifiable. The appropriate verdict is therefore UNVERDICTED rather than a stronger rejection: the method may work, but the evidence as printed, with an impossible summary statistic in a key table and no reproducibility artifacts, does not yet support acceptance.","tokens_in":20736,"tokens_out":9376,"duration_ms":110768,"concrete_test":"Recompute the T=100 row of Table II using the same mean definition applied to every other row (arithmetic mean of the five shown per-SNR columns). If the mean is 92.66, the published 89.29 is an error; if it is not, the paper must state which additional SNR values are included and provide the full per-SNR table. Independently recompute the RML2018.01A 12 dB, N=10 accuracy of 88.27% from the ten Monte Carlo runs and release the code, checkpoints, and raw logs for the MSDGM pretraining and DAFFus training so that the central claim can be rerun end to end. If the recomputed numbers match the published ones, the concern is resolved; if not, the empirical core is not reliable.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim of ModFus-DM is an empirical performance claim: a frozen diffusion U-Net plus DAFFus yields 88.27% on RML2018.01A with 10 labels per type and broadly outperforms existing methods. The most load-bearing support for that claim is the accuracy tables, and Section V.B.2, Table II contains an internally inconsistent summary statistic. For T=100, the listed per-SNR accuracies are 90.65, 92.93, 93.26, 93.96, and 92.48; their mean is 92.66, not the reported 89.29. For every other row in the same table, the reported mean is exactly the arithmetic mean of the five listed columns, so the intended definition is clear. The T=100 entry is below every individual value in its row. T=100 is then selected and used for all downstream experiments, including the headline 88.27% entry in Table III at RML2018.01A, 12 dB, N=10. If a key ablation table cannot be reproduced arithmetically, confidence in the remaining reported numbers is substantially weakened, and the paper provides no code, raw per-SNR logs, or error bars for the ten Monte Carlo runs to resolve the discrepancy. This is a verification risk in the empirical core of the paper, not a disagreement with the broader diffusion-representation idea.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ModFus-DM, a two-stage framework for automatic modulation classification. In the first stage, an unsupervised modulated-signal diffusion model (MSDGM) is trained on raw IQ signals to learn representations without labels or proxy tasks. In the second stage, the diffusion U-Net is frozen and a diffusion-aware feature fusion module (DAFFus) adaptively aggregates features from several U-Net blocks, followed by a linear classifier trained with a limited number of labeled signals per modulation type. The authors report extensive experiments on RML2016.10A, RML2016.10B, RML2018.01A, and RML2022, claiming consistent improvements over supervised, self-supervised, and semi-supervised baselines under limited-label settings, distribution shift, variable-length inputs, channel fading, and colored noise. The headline claim is over 88.27% accuracy on RML2018.01A with only 10 labeled signals per type.","tokens_in":21057,"tokens_out":2530,"duration_ms":30014,"significance":"If the empirical claims hold, the paper would make a useful contribution by showing that frozen diffusion-model features, trained without labels, provide a reuseable and label-efficient representation for radio modulation signals. The paper has notable strengths: it evaluates on four benchmarks, includes several challenging settings (limited labels, cross-dataset transfer, variable lengths, fading channels, colored noise), and compares against a broad set of baselines. The idea of using the diffusion process as an unsupervised representation learner for IQ signals is timely and plausible. However, the significance is currently gated by verification concerns in the central experimental tables, the lack of uncertainty quantification, and the use of test-set-based selection for key hyperparameters.","major_comments":[{"comment":"The T=100 row is internally inconsistent. The five per-SNR accuracies listed are 90.65, 92.93, 93.26, 93.96, and 92.48; their arithmetic mean is 92.66, not the reported 89.29. Every other row in the same table has a Mean exactly equal to the arithmetic mean of its five entries, so the intended definition is unambiguous. This matters because T=100 is the configuration used in all subsequent experiments, including the headline 88.27% result in Table III. As the paper stands, a load-bearing summary statistic cannot be reproduced from the table itself. The authors should provide corrected numbers, raw per-SNR logs, or code so that this discrepancy can be resolved.","section":"Section V.B.2, Table II"},{"comment":"The paper states that ten Monte Carlo experiments are run and the average accuracy is reported, but no error bars, standard deviations, or confidence intervals appear anywhere in the paper. Many reported advantages are large (e.g., Fig. 6, where ModFus-DM surpasses the second-best method by roughly 40–60 percentage points at N=2), but without variance information the reader cannot judge whether smaller differences, such as the 1.12% gap between DAFFus and Fusion-all in Fig. 4, are meaningful. I request that the authors report the spread of the ten runs, at least for the main tables and for the key limited-label comparisons.","section":"Section V.A and all experimental figures"},{"comment":"The diffusion step t=1 and the DAFFus fusion set (blocks b5–b8) are selected based on accuracy on the same benchmark test sets that are later used for the headline claims. The text says 'we fix t = 1 for all subsequent experiments' after observing that small t gives good results on RML2016.10A, RML2016.10B, RML2018, and RML2022. This is selection on test data. Since the central claim is an empirical performance claim, the reported numbers are optimistically biased unless the selection is accounted for, for example by reporting a validation split or by demonstrating that the chosen configuration is best across datasets in a pre-registered way. Please clarify the selection protocol or re-evaluate with a proper validation split.","section":"Section V.B.1, Fig. 4, and downstream experiments"},{"comment":"The temporal-flexibility claims are based on signals of lengths 64, 128, 256, 512, and 768 that are randomly cropped from full-length 1024 samples, as stated in the Table IV note. This is a proxy for natural variable-length signals, and the paper does not validate it against naturally captured variable-length data or compare baseline methods under the same crop protocol. Moreover, the U-Net is trained on length-1024 inputs and then applied to other lengths without any discussion of architectural changes or positional-encoding adaptation. If the crop proxy or the length-flexibility assumption fails, the temporal-flexibility conclusion in Section V.F would collapse. The authors should either add a baseline comparison under the same crops or temper the claim to what the experiment actually supports.","section":"Section V.F, Table IV, and Fig. 9"}],"minor_comments":[{"comment":"The abstract says 'over 88.27% accuracy in 24-type recognition tasks at SNR ≥ 12dB', but 88.27% appears to be the single 12dB N=10 entry in Table III. Please clarify whether the claim refers to a single SNR point or to an average over SNR ≥ 12 dB.","section":"Abstract and Table III"},{"comment":"The caption refers to 'MSGDM' while the text uses 'MSDGM'. Please correct the acronym for consistency.","section":"Fig. 2 caption"},{"comment":"The text contains the typo 'Rayleighigh' and later 'Rayleighigh fading'. Please fix these typos.","section":"Section V.G"},{"comment":"The definition of AP as 'pooling the final dimension to 1' is ambiguous. Please specify whether the pooling is over the time dimension, the channel dimension, or both, and how the pooled features are shaped.","section":"Eq. (13)"},{"comment":"The paper repeatedly describes ModFus-DM as a 'foundational model' and the 'first' such model for AMC. Given that the method is a frozen diffusion feature extractor plus a learned fusion head, the 'foundational model' claim is stronger than the evidence presented. Please soften or justify this terminology.","section":"General presentation"}],"recommendation":"major_revision","confidential_remarks":"The paper does not include a code availability statement. Given the arithmetic inconsistency in Table II, I would ask the editor to require the authors to provide code or raw result logs during revision. The central idea is plausible and the experimental scope is broad, but the empirical core is currently unverifiable without this material. The manuscript is within the scope of the journal, so the main question is whether the authors can close the verification gaps."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: ModFus-DM is a two-stage pipeline that takes the diffusion-as-representation idea from DDAE and Diffusion Model as Representation Learner, applies it to automatic modulation classification, and adds a DAFFus module that adaptively fuses U-Net features. The paper is coherent, the citations to the representation-learning lineage are in place, and the experimental scope is unusually broad: four datasets, limited labels, cross-dataset transfer, variable lengths, and fading/colored-noise scenarios. That breadth is the paper's real strength, and the DAFFus ablation showing its advantage over single-block or all-block fusion is convincing in structure.\n\nThe problem is in the empirical core. Table II, the ablation over total diffusion steps T, reports the T=100 row mean as 89.29, but the five per-SNR accuracies in that row (90.65, 92.93, 93.26, 93.96, 92.48) average to 92.66. Every other row in that table has a mean that is exactly the arithmetic mean of its columns, so this is not a rounding artifact. And T=100 is the setting used in every downstream experiment, including the headline 88.27% limited-label result on RML2018.01A. This is the load-bearing number of the paper, and it is currently internally inconsistent. The paper also reports Monte Carlo averaging over ten runs but gives no error bars, and no code or raw logs are provided to resolve the discrepancy. That is a verification risk, not a matter of taste.\n\nOther soft spots are smaller but still real. The choice of diffusion step t=1 and the DAFFus fusion set is made on test-set accuracy on the same benchmark datasets; the variable-length claim in Section V.F rests on random crops of length-1024 signals, which the paper does state, but no baseline methods are compared under that protocol; and calling ModFus-DM \"the first foundational model for AMC\" is a stretch given that it is an application of existing representation-learning recipes with a new fusion module. None of these individually sinks the approach, but they compound the core verification issue.\n\nWho should read it: anyone working on self-supervised or generative methods for RF signal classification. The direction is sensible and the experimental coverage is impressive. But before citing the numbers or building on them, we need code and raw per-SNR logs. If I were the editor, I would send this to review, with the request for code and corrected/verified tables as a mandatory condition. The idea deserves referee time; the current numbers do not deserve to be taken at face value.","headline":"A coherent diffusion-representation approach to AMC with a serious verification problem in its key ablation table, so the headline accuracy claims are not yet trustworthy.","tokens_in":21571,"tokens_out":1818,"would_cite":false,"duration_ms":22605,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion models trained to denoise unlabeled IQ signals produce modulation representations that rival fully supervised classifiers using only a handful of labeled examples per modulation type.","keywords":["automatic modulation classification","diffusion models","self-supervised representation learning","limited-label learning","U-Net features","modulation recognition","variable-length signals"],"falsifier":"Collect naturally recorded (uncropped) signals at lengths 64, 128, 256, 512, and 768 with known modulation labels, and evaluate the frozen MSDGM plus DAFFus trained on length-1024 data; if accuracy on the naturally short signals drops far below the random-crop results, or if the model fails on a length not seen in training, the temporal-flexibility claim would be refuted.","tokens_in":20544,"feed_emoji":"📡","tokens_out":8208,"duration_ms":80313,"temperature":0.7,"pith_summary":"This paper argues that a diffusion model trained purely on unlabeled radio signals can learn a representation of modulation type that is reusable and highly label-efficient. The proposed framework, ModFus-DM, first trains a noisy-signal reconstruction model (MSDGM) on raw IQ data, then freezes it and trains only a lightweight fusion module and classifier on a handful of labeled signals per class. The paper reports that with 10 labeled signals per modulation type it reaches over 88% accuracy on a 24-class recognition task at high SNR, and that the same frozen features generalize across datasets, signal lengths, and fading channels. If true, this would make generative self-supervision a practical source of modulation representations where annotation is scarce.","feed_headline":"Diffusion-trained radio features hit 88% with 10 labels per type","feed_subtitle":"Self-supervised diffusion features also survive distribution shifts and variable signal lengths.","key_machinery":"The modulated signal diffusion generation model (MSDGM): a diffusion model over IQ signals in which noise is added over T steps and a U-Net predicts the noise, trained with the standard denoising loss. The diffusion-aware feature fusion module (DAFFus) then takes the signals st at a small diffusion step t, extracts features from the U-Net's upsampling blocks b5-b8, concatenates them along the channel dimension, and projects them through a linear layer into a compact feature vector used for classification. The key operation is the adaptive fusion of these multi-scale features, which the paper shows outperforms any single block's features and both all-block and downsampling-only fusion.","core_discovery":"The paper's central claim is that intermediate features of a U-Net trained to denoise modulated signals carry discriminative modulation information, and that adaptively fusing the upsampling-block features yields a representation that separates modulation types better than any single layer or than all-block or downsampling-only fusion. The generative model, called MSDGM, is trained on raw unlabeled IQ signals with a standard noise-prediction loss; afterwards it is frozen and only the DAFFus fusion module and classifier are trained on as few as two to twenty labeled signals per class. Experiments on RML2016.10A, RML2016.10B, RML2018.01A, and RML2022 are reported as showing consistent gains over supervised and self-supervised baselines in limited-label settings, cross-dataset transfer, variable-length inputs, and fading channels.","pith_inferences":["The frozen representation should be label-set agnostic, so one could test whether DAFFus trained on one set of modulation types transfers to newly added types without retraining the diffusion model.","Since DAFFus uses features at a fixed denoising step t=1, a natural extension is to fuse features across several diffusion steps, which may further improve robustness to noise.","The superiority of upsampling-block features is demonstrated only for this U-Net and IQ-signal domain; comparing DAFFus on a masked-autoencoder backbone would separate the effect of diffusion from the effect of generative self-supervision in general.","If the representation is truly length-invariant, it should support zero-shot classification of modulation types that were absent from the labeled subset, testable with a nearest-prototype classifier over the frozen features."],"forward_implications":["Radio receivers could build recognition systems from unlabeled over-the-air captures, reducing annotation cost by orders of magnitude in spectrum monitoring and cognitive radio.","A single frozen diffusion feature extractor could be reused across datasets, SNRs, and channel conditions, so new deployments would only need to train a small fusion head.","Robustness to fading channels and colored noise suggests the learned representation captures signal structure that survives realistic propagation effects.","Length-flexible inference without architectural changes would allow models to process naturally variable-length transmissions instead of fixed-size windows."],"supporting_citations":[{"why":"Establishes the premise that U-Net encoder and decoder features from a denoising diffusion model carry discriminative semantic information usable for downstream tasks.","marker":"[19]"},{"why":"Shows diffusion models can act as representation learners, motivating the use of intermediate diffusion features for modulation recognition.","marker":"[20]"},{"why":"Supplies the denoising diffusion formulation and U-Net-based noise prediction architecture that MSDGM builds upon.","marker":"[18]"},{"why":"Provides the RML2016.10A dataset used for training and evaluating the method in limited-label settings.","marker":"[33]"},{"why":"Provides the RML2016.10B dataset used as a second benchmark and for cross-dataset generalization tests.","marker":"[34]"},{"why":"Defines the RML2018.01A dataset with 24 modulation types, the setting for the headline 88.27% accuracy claim.","marker":"[23]"},{"why":"Provides the RML2022 dataset used to test the method under more realistic channel conditions.","marker":"[35]"},{"why":"Represents recent work applying diffusion models to AMC as data augmenters, which this paper contrasts by using diffusion features for representation learning.","marker":"[17]"}],"fun_headline_variants":["Diffusion features classify radio signals with 10 labels per type","Unsupervised diffusion model hits 88% accuracy on 24 signal types","Few-shot radio classification via diffusion feature fusion","Self-supervised diffusion boosts AMC with scarce labels","Diffusion-trained features tackle variable-length radio signals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that a single trained model recognizes signals of any length assumes that randomly cropping long signals into short ones produces the same conditions as naturally captured short signals, and that a U-Net trained on one length can process any length without architectural changes.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion features classify radio signals with 10 labels per type","Unsupervised diffusion model hits 88% accuracy on 24 signal types","Few-shot radio classification via diffusion feature fusion","Self-supervised diffusion boosts AMC with scarce labels","Diffusion-trained features tackle variable-length radio signals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000187,"raw_usage":{"total_tokens":1328,"prompt_tokens":944,"completion_tokens":384,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":304}},"tokens_in":560,"tokens_out":384,"duration_ms":4204,"temperature":1.0,"reasoning_tokens":304,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:26:05.700538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect naturally recorded (uncropped) signals at lengths 64, 128, 256, 512, and 768 with known modulation labels, and evaluate the frozen MSDGM plus DAFFus trained on length-1024 data; if accuracy on the naturally short signals drops far below the random-crop results, or if the model fails on a length not seen in training, the temporal-flexibility claim would be refuted.","supporting_citations":[{"cited_title":"Denoising diffusion autoencoders are unified self-supervised learners,","cited_arxiv_id":null,"evidence_quote":"Establishes the premise that U-Net encoder and decoder features from a denoising diffusion model carry discriminative semantic information usable for downstream tasks."},{"cited_title":"Diffusion model as representation learner,","cited_arxiv_id":null,"evidence_quote":"Shows diffusion models can act as representation learners, motivating the use of intermediate diffusion features for modulation recognition."},{"cited_title":"Radio machine learning dataset generation with gnu radio,","cited_arxiv_id":null,"evidence_quote":"Provides the RML2016.10A dataset used for training and evaluating the method in limited-label settings."},{"cited_title":"Convolutional radio mod- ulation recognition networks,","cited_arxiv_id":null,"evidence_quote":"Provides the RML2016.10B dataset used as a second benchmark and for cross-dataset generalization tests."},{"cited_title":"Over-the-air deep learnig based radio signal classification,","cited_arxiv_id":null,"evidence_quote":"Defines the RML2018.01A dataset with 24 modulation types, the setting for the headline 88.27% accuracy claim."},{"cited_title":"RML22: Realistic dataset generation for wireless modulation classification,","cited_arxiv_id":null,"evidence_quote":"Provides the RML2022 dataset used to test the method under more realistic channel conditions."},{"cited_title":"Diffusion model empowered data augmentation for automatic modulation recognition,","cited_arxiv_id":null,"evidence_quote":"Represents recent work applying diffusion models to AMC as data augmenters, which this paper contrasts by using diffusion features for representation learning."}],"review_version":1}