{"id":"597cf588-2c63-4e19-9164-73290e7b8e49","arxiv_id":"2412.16207","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A comparison of three generative models for synthetic PCG signals reports low distribution-distance scores, but the evaluation lacks held-out splits, baselines, and error bars.","lead":"The paper applies three established generative models, WaveNet, DoppelGANger, and DiffWave, to synthesize phonocardiogram (PCG) heart sound signals, then compares the generated signals to real recordings using several similarity metrics. The work targets a practical need for synthetic medical time series, but its evaluation lacks held-out splits, baselines, and error bars, so the quality claim is not yet supported.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No held-out split for DGAN/DiffWave means low MMD/JSD and chance-level discriminator are consistent with memorizing training segments, so the central fidelity claim is unsupported.","rationale":"The reader identified the missing train/test split for the generative models as the weakest assumption, and this is indeed the load-bearing point. My independent reading of Sections IV-VI confirms that no split is described for DGAN or DiffWave, so the reported MMD, JSD, t-SNE, and discriminative scores cannot rule out memorization. The WaveNet experiment has a natural split because it forecasts the last 20% of each time series, but that evaluates prediction, not generation. The proposed concrete test would directly adjudicate whether memorization is occurring by comparing metrics on training versus held-out real segments. Because the concern is correct and the reader's CONDITIONAL verdict appropriately reflects that the flaws are fixable with additional experimental controls, no verdict change is needed.","tokens_in":8585,"tokens_out":3877,"duration_ms":36039,"concrete_test":"Train DGAN and DiffWave on 80% of the PCG segments, hold out 20%. Generate synthetic segments and compute MMD, JSD, and the 20-unit RNN discriminative score against (a) the training segments, (b) the held-out segments, and (c) a real-vs-real split of the held-out data. If metrics against held-out data are substantially worse than against training data (e.g., MMD increases 10x or discriminative accuracy exceeds 60%), memorization is the likely explanation and the paper's conclusion must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that generated PCG data closely resembles real PCG data is supported almost entirely by distributional metrics computed between generated samples and the real 110-sample PCG segments used to train the models. The paper never describes a train/test split for DGAN or DiffWave: preprocessing (Section IV) yields segments, and Sections VI-B and VI-C report MMD=0.0001, JSD around 0.014-0.015, t-SNE overlap, and 51-52% discriminative accuracy on a validation set. If the models memorized the training segments, they could emit near-copies, and all these metrics would look equally good even though the models have not learned the distribution of unseen PCG signals. The WaveNet evaluation (Section VI-A) does use a forecasting split, but prediction accuracy on the last 20% of each series is not evidence of distributional fidelity for synthesis. The weak RNN discriminator (20 hidden units, 50-100 epochs) is a secondary issue: it may underfit, but it cannot by itself distinguish memorization from generalization. The absence of a held-out real set is the primary load-bearing gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares three generative modeling approaches—WaveNet, DoppelGANger, and DiffWave—for synthesizing normal phonocardiogram (PCG) segments from the normal-subject subset of the PhysioNet Challenge 2022 dataset. The authors preprocess the recordings into 110-unit segments, train each model, and evaluate quality with forecasting metrics (MAE, MSE, SMAPE, ACD) for WaveNet and distributional metrics (t-SNE, discriminative score, MMD, JSD) for DGAN and DiffWave. The central claim is that all three models produce synthetic PCG data closely resembling the original data, supported by low MMD/JSD values and near-chance discriminative accuracy.","tokens_in":8769,"tokens_out":4816,"duration_ms":43412,"significance":"If the central claim were established, the paper would provide a useful comparative benchmark for raw-waveform PCG synthesis and a step toward data augmentation for murmur detection. The work uses a relevant public dataset, applies standard time-series generation metrics, and documents the preprocessing pipeline. However, because the evaluation does not separate training and test data for the generative models, and because WaveNet is evaluated only as a forecaster rather than as a full-sequence generator, the current evidence does not support the claimed fidelity. The paper would be strengthened by a held-out evaluation, a stronger discriminative classifier, and uncertainty quantification; these are fixable within the manuscript's scope.","major_comments":[{"comment":"No held-out split is described for DGAN or DiffWave. The MMD=0.0001 and JSD=0.014/0.015 values in Tables II and III are computed between generated samples and the real segments that were used to train the models. If the models memorized training segments, these metrics and the t-SNE overlap in Figures 2 and 3 would appear equally good even though the models have not learned the distribution of unseen PCG signals. The authors should split the real segments into training and held-out sets, train only on the former, and report MMD, JSD, and discriminative scores against the held-out real segments, ideally alongside a memorization baseline such as resampled training segments. Until this is done, the central claim that generated data closely resembles the original dataset is unsupported.","section":"VI-B, VI-C, V-B"},{"comment":"WaveNet is evaluated as a forecaster rather than as a whole-signal generator: it predicts the last 20% of each sequence, and Table I reports MAE=0.02, MSE=0.002, SMAPE=3.51, and ACD=0.006. Forecasting accuracy on tail portions does not measure whether WaveNet can synthesize a complete PCG segment from scratch, so the abstract's claim that WaveNet generates PCG data is not supported by these numbers. The authors should either generate full sequences by ancestral sampling from WaveNet and evaluate them with the same generative metrics used for the other two models, or explicitly reframe the WaveNet part as a forecasting baseline and remove it from the generative comparison.","section":"III-A, VI-A"},{"comment":"The discriminative score uses a single-layer RNN with 20 hidden units, trained for 50 epochs in Section V but for 100 epochs in Section VI, and the reported near-chance accuracies (52% and 51%) are point estimates without error bars. A weak classifier can fail to separate real and synthetic data even when the distributions are clearly different, so chance-level accuracy is not evidence of high fidelity. The authors should use a higher-capacity discriminator, report train and validation accuracy, include standard deviations over several random seeds, and resolve the inconsistent epoch count between Sections V and VI.","section":"V-B2, VI-B2, VI-C2"},{"comment":"All reported metrics are single point estimates with no confidence intervals or standard deviations across runs or bootstrap resamples. With only about 110 real segments, MMD values of 0.0001 and JSD values around 0.014 need uncertainty quantification before the claim of \"closely resembles\" can be assessed; the discriminative accuracy of 51-52% is also within random fluctuation for a small test set. Please report multiple-seed or bootstrap intervals for every metric.","section":"Tables I-III"}],"minor_comments":[{"comment":"The ACD metric is described only verbally as a coverage difference, and no formula is given in Section V-A4, while Table I reports ACD=0.006. Provide the exact computation or remove the metric.","section":"V-A4"},{"comment":"The Introduction states that no existing studies focus on generating high-quality PCG signals using generative models, but Section II cites Takezaki et al. [6], which applies a GAN to heart sound spectrogram data. Qualify the novelty claim to raw waveform synthesis or to the specific comparison performed here.","section":"I, II"},{"comment":"The thresholds RMSSD<0.1, zero-crossing ratio<0.3, and noise-free window ratio≥50% are introduced as \"my experimental analysis\" without a derivation or citation; describe how they were established and whether they were tuned on the same data.","section":"IV-B"},{"comment":"The metric name \"SAMPE\" appears in Table I; this should be \"SMAPE\" as defined in Section V-A3.","section":"III-A, Table I"},{"comment":"The affiliation for Alexander Gelbukh lists \"New Mexico, USA\" for Instituto Politécnico Nacional, which is located in Mexico; correct this error.","section":"Authors' affiliations"}],"recommendation":"major_revision","confidential_remarks":"The primary gap is the missing train/test split for the generative models, which undermines the central fidelity claim but is fixable with a re-evaluation. The paper's scope and dataset are appropriate for an applied ML/health-informatics venue. I would not reject on novelty grounds, but the authors should temper the claim that no prior work addresses PCG generation with generative models, given reference [6]."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the paper applies three off-the-shelf generative models to PCG segments and reports very low distributional distances, but the evaluation doesn't support the claim of realistic generation because the metrics are computed on the training data. The models could memorize the segments and produce exactly the results shown.\n\nWhat's actually new: applying WaveNet, DoppelGANger, and DiffWave to PCG signals from PhysioNet 2022 is a new application, and the preprocessing pipeline—filtering, quality thresholds, S1/S2 peak localization, fixed-length segmentation—is described carefully. The idea of comparing an autoregressive model, a GAN, and a diffusion model on a medical time series is sensible.\n\nWhat the paper does reasonably well: the metric suite (MAE, MSE, SMAPE, ACD for forecasting; t-SNE, discriminative score, MMD, JSD for generation) is standard, and the descriptions of the models are accurate. The paper is clearly written.\n\nThe soft spots are significant. For DGAN and DiffWave, no train/test split is described for the generative models. The real segments used to compute MMD, JSD, t-SNE, and the discriminative score appear to be the same segments used for training. With only 110 time steps per segment and no held-out real data, low MMD/JSD and chance-level discriminator accuracy are exactly what you'd expect from a model that memorized the training set. The discriminator is also too weak—20 hidden units, 50–100 epochs—so it may underfit regardless of fidelity. WaveNet is evaluated only as a forecaster on the last 20% of each series, which is not evidence of whole-signal generation.\n\nThere are no baselines (e.g., real-vs-real comparisons, a Gaussian model, or a simpler GAN), and no error bars or multiple runs, so the reported numbers are just point estimates. The claim that no prior study generated PCG data is contradicted by their own citation [6], which does GAN-based augmentation for heart sound classification. That's an overstatement.\n\nNone of these flaws are fatal to the underlying idea—this is a straightforward application that could be made solid with a proper held-out split, stronger discriminator, baselines, and multi-run statistics. But as it stands, the paper's central claim is unsupported. I wouldn't cite it, and I wouldn't send it to a serious venue in this state. It might be fine for a workshop after revision.","headline":"Routine application of three generative models to PCG signals, but the fidelity claim is unsupported because the metrics are computed on training data.","tokens_in":9321,"tokens_out":2643,"would_cite":false,"duration_ms":23971,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that WaveNet, DoppelGANger, and DiffWave can generate synthetic phonocardiogram signals that are statistically similar to real normal heart sounds, with low distribution-distance scores and a classifier that cannot…","keywords":["synthetic time series generation","phonocardiogram","PCG","generative models","WaveNet","DoppelGANger","DiffWave","maximum mean discrepancy"],"falsifier":"Retrain the three models on one group of patients and evaluate on a separate held-out group of patients; if the distance scores rise substantially or the classifier clearly separates real from synthetic, the claim that the generated PCG data closely resembles the original data would be falsified.","tokens_in":8350,"feed_emoji":"🫀","tokens_out":6270,"duration_ms":54529,"temperature":0.7,"pith_summary":"This paper tries to establish that three generative models from different families — an autoregressive network (WaveNet), a GAN (DoppelGANger), and a diffusion model (DiffWave) — can synthesize realistic phonocardiogram (PCG) recordings of normal heart sounds. Using heart-sound data from a public pediatric challenge dataset, the authors filter and segment the signals, train the models, and compare real versus synthetic segments with distribution-distance metrics and a trained discriminator. They report low maximum mean discrepancy and Jensen-Shannon divergence values and near-chance discriminative accuracy, which they take as evidence that the synthetic data closely resemble the original recordings. If true, this would make synthetic PCG data a practical resource for cardiac pre-screening and for augmenting scarce abnormal-heart-sound datasets.","feed_headline":"Three generative models make realistic synthetic heart sounds","feed_subtitle":"WaveNet, DoppelGANger, and DiffWave pass distribution tests against real PCG recordings.","key_machinery":"The argument is carried by three generative architectures paired with a fidelity-evaluation protocol. WaveNet uses stacked dilated causal convolutions with gated activations to predict future waveform samples; DoppelGANger uses two generators — one for static features and one for time series — plus a discriminator that sees both; DiffWave learns to reverse a forward process that adds Gaussian noise, reconstructing audio from noise. The evaluation machinery compares real and synthetic segments through maximum mean discrepancy in a reproducing kernel Hilbert space, Jensen-Shannon divergence between the two distributions, t-SNE overlap, and the accuracy of a small recurrent classifier trained to tell the sources apart. Low distances and near-chance classifier accuracy are the evidence that the synthetic segments sit close to the real data.","core_discovery":"The central claim is that generative models can produce PCG segments that are statistically indistinguishable from real normal heart sounds under the tested measures. For WaveNet, the paper reports forecasting errors of MAE 0.02, MSE 0.002, SMAPE 3.51, and ACD 0.006 on the last 20% of the time series. For DoppelGANger and DiffWave, it reports JSD values of 0.014 and 0.015 and MMD values of 0.0001 for both, with a 20-unit RNN discriminator reaching only 52% and 51% accuracy when trying to label real versus synthetic segments. The authors interpret these numbers as evidence that the generated data closely mirrors the distribution of the original PCG signals and that the models are effective at producing realistic synthetic heart-sound data.","pith_inferences":["The near-chance discriminative score depends entirely on the capacity of the 20-unit classifier; a stronger discriminator might separate the distributions, so the fidelity claim should be tested with a larger or convolutional classifier before relying on it.","If the fidelity holds on properly held-out patients, the same approach could be extended to conditional generation of murmur and abnormal classes, though those classes are rarer and more variable and will likely require conditioning or class-balanced training.","A stricter test would compare raw waveform statistics, such as power spectra or S1/S2 timing, rather than only the distance scores, which may not capture clinically meaningful features."],"forward_implications":["If the fidelity result holds, synthetic normal PCG segments can be produced in arbitrary volume without new patient recordings.","The same evaluation metrics give future work a ready benchmark for comparing new generative models on heart-sound data.","Synthetic normal segments could serve as the base class in a data-augmentation pipeline, which is the paper's stated next step toward detecting abnormal heart sounds and murmurs.","Sharing synthetic recordings instead of raw patient data becomes more plausible if the synthetic distribution is close enough for downstream diagnostic use."],"supporting_citations":[{"why":"Supplies the WaveNet architecture used for autoregressive synthesis.","marker":"[7]"},{"why":"Supplies the DoppelGANger dual-generator architecture used for GAN-based generation.","marker":"[8]"},{"why":"Supplies the DiffWave diffusion model used for denoising-based generation.","marker":"[9]"},{"why":"Source of the recurrent GAN framework and the MMD evaluation metric adopted here.","marker":"[1]"},{"why":"Proposes the discriminative score that trains a classifier to tell real from generated time series.","marker":"[17]"},{"why":"Public dataset from which the normal-subject PCG recordings are drawn.","marker":"[10]"},{"why":"Defines the clinical outcome labels used to select normal versus abnormal recordings.","marker":"[12]"},{"why":"Provides the quality-assessment protocol used to filter noisy PCG signals before training.","marker":"[13]"}],"fun_headline_variants":["Three AI models generate lifelike heart sounds","WaveNet, DoppelGANger, DiffWave craft realistic PCG data","Generative models produce realistic synthetic heart sounds","Synthetic heart sounds from three generative models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results assume that the models were tested on recordings they had not seen during training, and that the simple 20-unit classifier would have been able to detect any real differences between real and synthetic data if they existed.","fun_headline_variants_meta":{"raw":{"variants":["Three AI models generate lifelike heart sounds","WaveNet, DoppelGANger, DiffWave craft realistic PCG data","Generative models produce realistic synthetic heart sounds","Synthetic heart sounds from three generative models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000628,"raw_usage":{"total_tokens":2896,"prompt_tokens":933,"completion_tokens":1963,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":1901}},"tokens_in":549,"tokens_out":1963,"duration_ms":12398,"temperature":1.0,"reasoning_tokens":1901,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:22:35.081074+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the three models on one group of patients and evaluate on a separate held-out group of patients; if the distance scores rise substantially or the classifier clearly separates real from synthetic, the claim that the generated PCG data closely resembles the original data would be falsified.","supporting_citations":[{"cited_title":"Time-series generative ad- versarial networks,","cited_arxiv_id":null,"evidence_quote":"Proposes the discriminative score that trains a classifier to tell real from generated time series."},{"cited_title":"Heart murmur detection from phonocardiogram recordings: The george b. moody physionet challenge 2022,","cited_arxiv_id":null,"evidence_quote":"Defines the clinical outcome labels used to select normal versus abnormal recordings."},{"cited_title":"Analysis of pcg signals using quality assessment and homomorphic filters for localization and classification of heart sounds,","cited_arxiv_id":null,"evidence_quote":"Provides the quality-assessment protocol used to filter noisy PCG signals before training."}],"review_version":1}