{"id":"46ac5d11-ad46-48b6-ba4e-f535d72caca0","arxiv_id":"2607.19455","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Per-sample counterfactual optimization steers a fixed ensemble classifier's fault probability to arbitrary targets on CWRU/Paderborn vibration windows with ~0.005 error, outperforming a probability-regularized WGAN.","lead":"This paper compares two ways to edit real bearing-vibration signals so a fixed fault detector returns a chosen probability, such as 0.5, instead of a confident yes or no. A direct signal-optimization method hits the target almost exactly, while a trained generator is less precise but produces samples faster after training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CF's probability-steering numbers are likely correct, but the paper never validates that p*=0.5 samples are physically plausible rather than adversarial; its own caveat in §4.5.3 concedes the realism question is open.","rationale":"The reader's CONDITIONAL verdict is appropriate. I focus on the realism proxy rather than calibration because the technical claim of steering predicted probabilities does not require calibration, while the practical value claimed in the abstract and introduction depends on the generated signals being plausible borderline conditions. The paper's own §4.5.3 admits the p*=0.5 samples may be adversarial perturbations, which would defeat the stated purpose of populating decision-boundary samples for maintenance. The proposed discriminator test would settle whether this concern actually lands. I do not see an internal inconsistency in the probability-steering numbers; CF's high success rate is partly built into its stopping rule, but the paper acknowledges this confound (Section 4.5.1) and the MAE values still characterize its behavior. The calibration issue is a limitation but not a threat to the literal claim about predicted probabilities. Since the reader already captured the core concern and the verdict remains CONDITIONAL, no change is needed.","tokens_in":17102,"tokens_out":6047,"duration_ms":57216,"concrete_test":"Train a 1D CNN discriminator to distinguish real vibration windows from CF-generated p*=0.5 windows on the same dataset, using held-out real windows and the generated set, and report the held-out AUC. If AUC is near chance (≤0.6), the generated samples are on-manifold and this concern does not land; if AUC >0.8, they are identifiable as non-real, confirming the adversarial-perturbation worry and undermining the maintenance motivation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that CF reliably steers predicted fault probability to user targets while keeping signals close to real sources. The probability-steering part is well-supported: CF directly minimizes BCE and stops at tolerance ε=0.05, so SR_τ=1.000 and MAE 0.005–0.008 are unsurprising (Alg. 2, §3.8). The load-bearing weakness is the realism assumption. Section 3.9 states that ΔL1, ΔTV, and dPSD 'measure how aggressively a method edits the input rather than the realism of the result in any absolute sense.' Yet the abstract and introduction motivate the work by 'borderline samples' that are 'practically important' for maintenance decisions. Section 4.5.3 explicitly cautions that p*=0.5 samples 'may lie closer to adversarial perturbations of the source than to physically meaningful borderline conditions,' because the optimizer drives the input into regions the classifier rarely sees. If that is true, the central motivation fails even though the MAE numbers are correct. The paper leaves a direct realism evaluation to future work, so the practical value of the method is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript addresses the scarcity of samples with intermediate predicted fault probabilities in bearing vibration classifiers. It proposes two generation methods: PR-GAN, a WGAN-GP extension with a residual generator and a BCE probability-alignment term against a frozen heterogeneous ensemble CNN, and CF, a Wachter-style per-sample counterfactual that optimizes a BCE plus L2 proximity objective with early stopping at tolerance epsilon=0.05. Experiments on CWRU and Paderborn report that CF achieves MAE_p 0.005--0.008 and SR_tau=1.000 on retained samples, whereas PR-GAN achieves MAE_p 0.046--0.059 and SR_tau 0.501--0.680 on retained samples; CF also has lower Delta-L1 and dPSD in most settings, while PR-GAN has lower reported runtime in most settings. The authors position both methods as instantiations of existing templates applied to vibration windows, and they include a GitHub repository for reproducibility.","tokens_in":17502,"tokens_out":7152,"duration_ms":71966,"significance":"If the results are taken at face value, the paper provides a reproducible comparison of training-based and training-free probability-targeted generation on two standard bearing datasets. The manuscript is honest about several confounders and limitations, and it ships code. However, the headline CF success rate is largely a consequence of the method's own stopping rule, and the realism evaluation is only a source-proximity proxy, with the paper itself conceding that p*=0.5 samples may be adversarial rather than physically meaningful. The central practical motivation---generating plausible borderline samples for maintenance decisions---therefore remains unsupported. These issues are addressable with additional reporting and experiments, so the paper has a useful core if revised substantially.","major_comments":[{"comment":"CF's reported SR_tau=1.000 is a tautology. Algorithm 2 breaks when |p_f - p*| <= epsilon, with epsilon=0.05, and Eq. (18) defines success as |p_f - p*| <= tau with tau=0.05. Since N_failed=0, every returned sample satisfies the success criterion by construction; the 1.000 'success rate' is not an independent empirical outcome. The MAE_p values are not directly forced by the bound, but the comparison is still not like-for-like because PR-GAN has no early-exit oracle. Please report CF's error distribution under a fixed step budget (or at least how often the early-exit condition binds) and use the same evaluation procedure for both methods.","section":"§3.8, Algorithm 2; §3.9, Eq. (18)"},{"comment":"All probability metrics are computed only on samples retained after the 0.25 filtering threshold. For PR-GAN this excludes 77, 74, and 3 samples (CWRU) and 20, 15, and 0 (Paderborn) out of 500. Thus SR_tau is a conditional proportion among retained samples, not the success rate over all attempted generations. For example, CWRU p*=0.25 PR-GAN's full-attempt success rate is roughly (500-77)/500 * 0.501 = 0.424, not 0.501. The paper does disclose the conditioning, but the abstract's presentation of 'success rates between 0.501 and 0.680' is misleading without full-attempt rates. Please report full-attempt MAE/SR or a combined metric such as retention rate times SR_tau.","section":"§3.9, Eqs. (17)--(18); Tables 4--5"},{"comment":"The central motivation is to generate practically useful borderline samples, but the realism metrics Delta-L1, Delta-TV, and dPSD measure proximity to the source, not physical plausibility. Section 4.5.3 explicitly concedes that p*=0.5 samples 'may lie closer to adversarial perturbations of the source than to physically meaningful borderline conditions.' Because no direct realism check (envelope-spectrum analysis, a realism classifier, or a downstream task) is provided, the practical-value claim is unsupported. This is a load-bearing limitation; either add at least one direct realism evaluation or substantially narrow the claims to 'edits that steer a fixed classifier's probability' rather than 'realistic bearing vibration signals.'","section":"§3.9, §4.5.3, §5"}],"minor_comments":[{"comment":"The sentence 'The results are reported in Table 4.' appears twice consecutively. Please remove the duplicate.","section":"§4.3"},{"comment":"The 'Time (minute)' column mixes PR-GAN training time with CF generation time. The text explains this, but the table should annotate the distinction more clearly (e.g., separate rows for training and per-sample generation) to avoid an apples-to-oranges runtime comparison.","section":"Tables 4--5 and §4.5.1"},{"comment":"The values of alpha_ce and beta_l2 in the CF objective are not reported. Since they control the trade-off between probability alignment and source proximity, please include the values used in the experiments.","section":"§3.8, Algorithm 2"},{"comment":"The abstract says CF 'requires smaller average L1 changes,' but CF's optimization objective uses an L2 proximity term; Delta-L1 is only an evaluation metric. Clarify this distinction to avoid confusion.","section":"Abstract and §3.9"},{"comment":"The paper states that classifier calibration is not measured. This is a legitimate scoping choice, but it should be tied more prominently to the interpretation of p* in the abstract and conclusion, since 'fault probability' could be read as a physical probability rather than a model output.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest and well structured, but the two central issues are real: CF's SR_tau=1.000 is an artifact of its early-exit criterion, and the realism motivation is explicitly left unverified. Both are fixable within the manuscript's scope by additional reporting and at least one direct realism or downstream-task evaluation. I do not see a need for rejection; the comparison framework and reproducible code could be a useful contribution after these revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is exactly what it says it is: an application-level comparison of a Wachter-style counterfactual and a WGAN-GP variant with a probability-alignment loss, for steering a bearing-fault classifier's output to 0.25/0.5/0.75. Nothing here is methodologically new, and the authors say so. What is useful is that they run it on two real bearing datasets, ship code, and are unusually candid about the limits: the p=0.5 samples may be adversarial rather than physically meaningful, the metrics only measure edit distance, and calibration of the oracle is never checked.\n\nThe probability-steering result for CF is believable. It minimizes BCE against the target and stops at |p_fault - p*| <= 0.05, so MAE ~0.005 and SR=1.000 are essentially the stopping rule. That's not a flaw in the method, but it does mean the headline comparison to PR-GAN isn't a fair contest. PR-GAN's metrics are computed only on samples that survived a 0.25 retention filter, so its 0.046-0.059 MAE is conditional too. The paper acknowledges the filtering and the confounded comparison, but the abstract still advertises the contrast without the caveats.\n\nThe larger soft spot is what the numbers don't answer. The realism metrics (L1, TV, PSD) are edit-distance proxies, not plausibility checks. Their own Section 4.5.3 concedes that p*=0.5 outputs may simply be adversarial perturbations. With no calibration of the ensemble, a generated 0.5 doesn't mean '50% fault likelihood'. So the practical motivation—populating the decision boundary for maintenance decisions—remains unvalidated. The paper identifies this as future work, which is honest, but it means the abstract's promise is broader than the evidence.\n\nI'd send this to review. It's a competent empirical study, the limitations are stated clearly, and a referee could push the authors to report unfiltered results, measure calibration, and add one direct realism check (e.g., envelope spectrum at fault frequencies). I wouldn't cite it as a solution, but it's a useful cautionary data point for anyone working on classifier-guided signal generation.\n\nRecommended: accept a revision after those conditions.","headline":"A clean, honest application of known templates; the CF results are real but partly by design, and the realism question is left open.","tokens_in":17943,"tokens_out":2358,"would_cite":false,"duration_ms":22793,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Direct optimization of each signal against a frozen ensemble classifier can produce bearing vibration windows at user-chosen fault probabilities far more reliably than a trained probability-regularized GAN.","keywords":["bearing fault diagnosis","counterfactual generation","probability-targeted synthesis","vibration signal generation","generative adversarial networks","predictive maintenance","classifier calibration","gray-zone samples"],"falsifier":"Compute expected calibration error and reliability diagrams for the guidance ensemble, then run envelope-spectrum analysis at the characteristic bearing-fault frequencies on the CF-generated p* = 0.5 samples (or pass them through a second, independently trained classifier). If the samples are detected as out-of-distribution or the 0.5 outputs do not coincide with genuinely ambiguous vibration signatures, the central practical claim that these are useful borderline samples collapses.","tokens_in":17026,"feed_emoji":"🎯","tokens_out":6612,"duration_ms":61528,"temperature":0.7,"pith_summary":"This paper tries to solve a practical scarcity: in bearing vibration datasets, almost every window is confidently scored as healthy or faulty, so intermediate \"gray-zone\" fault probabilities are rare. It proposes two ways to synthesize windows at a user-chosen fault probability (0.25, 0.50, or 0.75) using a fixed, gradient-accessible ensemble classifier as a probability oracle: a trained residual GAN (PR-GAN) and a training-free per-sample counterfactual optimizer (CF). The paper's central empirical claim is that CF reaches the target with mean absolute probability error of 0.005–0.008 and a within-tolerance success rate of 1.000 on retained samples, whereas PR-GAN's error is 0.046–0.059 with success between 0.501 and 0.680. A sympathetic reader cares because controlled boundary-region samples could support maintenance-decision studies—but the authors explicitly caution that the uncalibrated oracle and source-closeness realism proxies leave the physical meaningfulness of the generated \"borderline\" samples open.","feed_headline":"Per-sample counterfactuals hit target fault probability 100% of the time","feed_subtitle":"Directly steering each vibration window against a frozen classifier beats a trained GAN at producing user-specified gray-zone signals.","key_machinery":"The load-bearing object is the \"probability oracle\": a fixed, gradient-accessible heterogeneous ensemble classifier whose averaged output is treated as a continuous function of the input. Both generation methods steer this output to a target probability: PR-GAN trains a residual generator x̂ = x + Δ(x, p*) with a Wasserstein-GAN adversarial loss, a binary-cross-entropy alignment term between the classifier output and p*, and an L1 residual penalty; CF runs per-sample gradient-based optimization on BCE(p*, f(x_cf)) plus an L2 proximity term to the source, with amplitude clipping and an early-stop tolerance of 0.05. The paper also gives an intermediate-value existence argument (continuity, pat","core_discovery":"On its own terms, the paper establishes that user-specified fault probabilities can be imposed on bearing vibration windows by directly optimizing each window against a frozen ensemble classifier: a per-sample counterfactual procedure using binary cross-entropy to a soft target plus an L2 proximity penalty achieved mean absolute error 0.005–0.008 and 100% within-tolerance success across all retained samples and both benchmark datasets, with smaller average L1 edits than the residual GAN. The trained PR-GAN, which augments a Wasserstein GAN with gradient penalty by adding a probability-alignment term and a residual edit, was systematically less reliable (MAE 0.046–0.059, success 0.501–0.680),","pith_inferences":["A strong reading of the CF result is that it is a targeted adversarial perturbation under an L2 budget; whether that is useful for maintenance hinges on external validation (e.g., a second held-out classifier or envelope-spectrum features agreeing with the oracle), which the paper leaves as future work.","The 100% success rate may partly reflect the generous 300-step optimization budget and the oracle's smoothness; a fairer apples-to-apples comparison would give PR-GAN per-sample verification or a comparable per-sample budget, which the paper does not do.","If the guidance ensemble were replaced by a calibrated one, the per-sample targeting mechanism would likely still work—binary-cross-entropy targeting depends on relative ordering—but the physical meaning of the resulting probability would become stronger; this is a testable extension.","A direct downstream test: use the generated gray-zone samples as training augmentation and measure whether boundary robustness or uncertainty estimates improve; the paper lists this as future work but does not run it."],"forward_implications":["For a fixed classifier with accessible gradients, per-sample counterfactual optimization can hit a user-chosen fault probability almost exactly (mean absolute error ≤ 0.008) on every retained window, while keeping time-domain and spectral edits smaller than the trained GAN's.","The trained residual GAN is not a reliable way to control probability: on roughly a third to half of its retained samples it misses the ±0.05 tolerance, so amortized inference speed does not compensate for probability-steering failure on individual outputs.","Generated p* = 0.5 samples should be read as \"the classifier is locally uncertain on this perturbation,\" not as calibrated 50% fault likelihood; the paper states this explicitly as a limitation.","The CF-versus-PR-GAN comparison is a system-level comparison, not a controlled ablation, so the reliability gap should not be attributed to any single design axis.","Probability-targeted samples provide a practical way to populate the boundary region of a fixed classifier with controlled probabilities that are otherwise scarce in standard bearing datasets, subject to the realism caveat."],"fun_headline_variants":["Counterfactuals beat GAN for targeted bearing fault signals","Per-sample edits hit fault probability 100% of time","Direct steering outperforms GAN for gray-zone vibration data","Counterfactual method nails target fault probabilities in bearings","Per-sample counterfactuals more reliable than PR-GAN for fault signals"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The approach assumes the ensemble classifier's predicted probability is a trustworthy guide to real fault likelihood; if that output is uncalibrated—or if a p* = 0.5 edit is just an adversarial perturbation of the source rather than a plausible vibration condition—the generated \"borderline\" samples do not deliver the maintenance value the paper motivates.","fun_headline_variants_meta":{"raw":{"variants":["Counterfactuals beat GAN for targeted bearing fault signals","Per-sample edits hit fault probability 100% of time","Direct steering outperforms GAN for gray-zone vibration data","Counterfactual method nails target fault probabilities in bearings","Per-sample counterfactuals more reliable than PR-GAN for fault signals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000656,"raw_usage":{"total_tokens":2917,"prompt_tokens":896,"completion_tokens":2021,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":640,"completion_tokens_details":{"reasoning_tokens":1937}},"tokens_in":640,"tokens_out":2021,"duration_ms":13661,"temperature":1.0,"reasoning_tokens":1937,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T13:17:18.728541+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute expected calibration error and reliability diagrams for the guidance ensemble, then run envelope-spectrum analysis at the characteristic bearing-fault frequencies on the CF-generated p* = 0.5 samples (or pass them through a second, independently trained classifier). If the samples are detected as out-of-distribution or the 0.5 outputs do not coincide with genuinely ambiguous vibration signatures, the central practical claim that these are useful borderline samples collapses.","supporting_citations":[],"review_version":1}