{"id":"1f007e60-9f85-4d74-8ab2-417a21228647","arxiv_id":"2505.20452","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new algorithm combines deep Gaussian process predictions, sliding-window spectral analysis, and an acquisition function that balances spectral change and uncertainty to detect change points while actively choosing where to sample.","lead":"The paper proposes an active learning method that combines deep Gaussian processes, spectral analysis, and a custom acquisition function to detect multiple change points in costly-to-sample time series. It reports improved accuracy over a derivative-based baseline and competitive results on real-world benchmarks with a limited number of data points.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The F1 comparison in Table 3 likely uses a different evaluation margin than the baselines from [29], so the real-world outperformance claim may be an artifact of an inflated tolerance.","rationale":"The reader's weakest assumption identifies exactly the issue I find most load-bearing: the F1 evaluation protocol is not held constant between the proposed method and the baselines from [29]. The paper sets M equal to the suppression interval δ (Section 4.3), which is 10 or 20 for the real datasets (Section 5.2.2), while the standard benchmark margin is 5. This can inflate the proposed method's F1 scores and invalidate the direct comparison in Table 3, which is the primary evidence for the abstract's claim of outperforming existing techniques on real-world data. The concern is concrete, testable, and does not require rejecting the algorithmic contributions. Other issues—such as the absence of a fixed rule for selecting β, window size A, and threshold b, and the lack of an explicit sampling-efficiency curve—are secondary but reinforce the need for a conditional verdict. I agree with the reader's assessment and would keep the verdict at CONDITIONAL: the paper is promising but the empirical claims require re-evaluation under a consistent protocol before acceptance.","tokens_in":14286,"tokens_out":6233,"duration_ms":63767,"concrete_test":"Recompute the F1 scores for the proposed method's change-point estimates on the five real-world datasets using the standard margin M=5 from the TCPD benchmark [29], instead of M=δ=10 or 20, and compare against the baseline F1 values in Table 3. If any proposed F1 value drops below the corresponding BOCPD or ECP score, the claim of outperforming existing methods on real data is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the proposed method 'outperforms existing techniques in terms of detection accuracy' on real data rests on Table 3, which compares F1 scores against baselines 'reported in [29]'. However, Section 4.3 states that 'the margin of error M is set to be equal to the suppression interval δ', and Section 5.2.2 sets δ=10 for Occupancy, Run log, and Well log, and δ=20 for Apple and Bee dance. The standard margin in the TCPD benchmark [29] is 5 samples (a fixed, dataset-independent value). If [29] used M=5, then the proposed method counts a detection as correct when it is up to 10 or 20 points away, while baselines require within 5 points. This systematically inflates the proposed method's F1 scores and makes the comparison in Table 3 invalid. Because the paper explicitly relies on these F1 scores to claim superiority over ACPD, BOCPD, ECP, KCPA, and RBOCPDMS, the accuracy claim for real-world data is not supported unless the margin is aligned. This is an evaluation-protocol mismatch, not an internal inconsistency, and it is directly testable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an offline multiple change point detection method, DGP-Spectral-AL, that combines deep Gaussian processes (DGPs), short-time Fourier transform spectral analysis, and active learning. The method first fits a DGP to a small initial set of points, converts the predicted mean and Monte Carlo sample paths into spectral representations, defines a Spectral Change Detection Metric (SCDM, Eq. 15) that combines spectral mean change and a weighted spectral gradient, and then selects new sampling locations by maximizing an acquisition function (Eq. 18) that balances SCDM with spectral uncertainty. Change points are finally extracted by thresholding SCDM with a suppression interval (Algorithm 1). Experiments compare the method against the authors' earlier DACD baseline on synthetic patterns and against several published change point detection methods on five real-world datasets.","tokens_in":14562,"tokens_out":5706,"duration_ms":55662,"significance":"If the claims were fully supported, the paper would offer a plausible and useful extension of active learning to change point detection under expensive data acquisition, with a spectral representation that is not tied to a single change pattern. The paper includes a sensitivity analysis and uses public benchmark datasets, which are strengths. However, the current evaluation does not substantiate the central claims: the real-world F1 comparison against baselines from [29] is likely invalid because the evaluation margin is inflated, the simulation claim of consistent superiority over DACD is contradicted by one of the reported pattern rows, and no uncertainty quantification is provided for any of the headline numbers. The algorithmic framework is interesting, but the evidence as presented is not yet convincing.","major_comments":[{"comment":"The F1 comparison against baselines from [29] is not valid as reported. Section 4.3 states that 'the margin of error M is set to be equal to the suppression interval δ', and Section 5.2.2 sets δ=10 for Occupancy, Run log, and Well log and δ=20 for Apple and Bee dance. The TCPD benchmark protocol in [29] uses a fixed margin (5 samples in the standard evaluation). Since the baseline F1 scores in Table 3 are taken from [29] rather than recomputed with the same margin, the proposed method is allowed a tolerance of 10 or 20 samples while the baselines are held to 5 samples. This systematically inflates the proposed method's F1 scores. The claim in the abstract that the method 'outperforms existing techniques in terms of detection accuracy' on real-world data is therefore not supported unless all baselines are recomputed under a common margin.","section":"Section 4.3, Section 5.2.2, Table 3"},{"comment":"The statement that 'our method consistently outperforms DACD in RMSE across all patterns' is contradicted by the Trend (TP) row of Table 2. DACD achieves RMSE 0.5, while the proposed method's best configuration in that row is 0.4 (Matérn, A=15, β=0.5) and most configurations are substantially worse (e.g., 2.1, 3.2, 5.1, 7.1). The claim needs at least to define what 'consistently' means across the 15 settings of β, kernel, and window size, and the current table does not support a blanket statement.","section":"Section 5.1.2, Table 2"},{"comment":"No standard deviations, confidence intervals, or significance tests are reported anywhere in the experimental sections. Section 5.1.2 says each simulation was repeated 10 times, but Table 2 reports only the average RMSE, and Table 3 reports single F1 values. Given the large variation across β, kernel, and window size (e.g., Well log F1 ranges from 0.1667 to 0.8571 across configurations), the reported differences—such as the 0.4 vs 0.5 RMSE for the Trend pattern—cannot be judged as meaningful. The experimental section needs at least error bars or a statistical comparison to support the claims of superiority.","section":"Section 5.1.2, Table 2 and Table 3"},{"comment":"The paper includes random and uniform sampling ablations in Table 3, but the text does not analyze them, and in several cases the ablations are competitive with or better than the proposed acquisition function. For example, for Well log with the RBF kernel, uniform sampling achieves F1 0.6400 while the best proposed RBF configuration achieves 0.6250; for Occupancy with the RBF kernel, uniform sampling (0.5882) beats several AF settings. Since one of the central contributions is the active learning acquisition function, the paper should explicitly compare the AF against these ablations and discuss where and why active selection helps or fails. Without this, the 'sampling efficiency' claim is not established.","section":"Table 3, Section 5.2.2"}],"minor_comments":[{"comment":"The definition of recall in Eq. (22) is unnecessarily convoluted: writing R as an average over individual ground-truth points T_k with |T_k|=1 is equivalent to |TP(T,C)|/|T| but obscures the standard definition. Please simplify.","section":"Section 4.3, Eq. (22)"},{"comment":"The sentence 'For both datasets, window sizes and estimated distances were chosen based on domain knowledge' is inaccurate because five datasets are used, not two. It should say 'For each dataset' or 'For all datasets'.","section":"Section 5.2.2"},{"comment":"The detection threshold b in Algorithm 1 is never specified for any experiment. Section 5.2.3 suggests a percentile-based threshold (e.g., top 5%) as a possibility, but the text does not state which threshold was actually used to produce Tables 2 and 3. This should be reported for reproducibility.","section":"Section 4.3 and Section 5.2.3"},{"comment":"There are several typos and grammatical errors, including 'it beneficial' in the Introduction, 'intruduce' in Section 3.2, 'Appedix A' in Section 5.1.2, and 'PryzlewiczV' in reference [15]. These should be corrected.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The core evaluation problems are fixable: the margins for the F1 comparison need to be aligned with the benchmark protocol, error bars or significance tests should be added, and the claims about consistency with respect to DACD need to be qualified. The proposed framework is plausible, but as submitted the abstract's claims are not supported by the experiments as reported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the good news: the algorithmic core is genuinely new and clearly described. Combining a deep GP with a spectral change metric and an active-learning acquisition function that balances SCDM with spectral uncertainty is a sensible integration, and the paper is transparent about borrowing DISS from remote sensing. The simulation study covers 15 patterns, which is more than most CPD papers attempt, and the method shows flexibility across diverse change types. That part holds up.\n\nThe soft spot is the real-world evaluation. Table 3 compares F1 scores against baselines from TCPD benchmark [29], which uses a fixed margin of M=5. This paper sets M equal to the suppression interval delta, giving 10 or 20. That systematically inflates the method's F1 relative to the baselines. The paper explicitly states M=delta, so it is not hidden, but it never acknowledges the divergence from the benchmark protocol, and the abstract's claim of outperforming existing techniques on real data is not supported by Table 3 as reported. Even with the inflated margin, BOCPD beats the method on Bee dance. This is a load-bearing issue, but it is directly fixable by re-running with M=5 or re-evaluating the baselines with the same margin.\n\nSecondary issues: no error bars on simulation RMSE, hyperparameters (beta, kernel, window size) are tuned per dataset without a fixed selection rule, and the only simulation baseline is the authors' own DACD. These are addressable and not fatal.\n\nI would send this to review, but only with a referee who will insist on fixing the evaluation protocol. The algorithm itself deserves a look. I would not cite it in this form, but a corrected version would be citable. For a reading group, it is a useful case study in how evaluation margins can flip a comparison.","headline":"The core algorithm is sensible and new, but the real-world F1 comparison is likely unfair to baselines because the evaluation margin is inflated, undermining the headline accuracy claim.","tokens_in":737,"tokens_out":1571,"would_cite":false,"duration_ms":36566,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that multiple change points in non-stationary time series can be detected accurately and cheaply by actively sampling where a deep Gaussian process's local spectrum changes, and it backs this with a new acquisition…","keywords":["change point detection","deep Gaussian process","active learning","spectral analysis","non-stationary time series","acquisition function","uncertainty quantification"],"falsifier":"Recompute the F1 scores of the baseline methods with the same tolerance window $M=\\delta$ used by the proposed method; if the baseline scores from the benchmark report change materially, the claim that the method beats existing techniques on real-world data is unsupported.","tokens_in":14093,"feed_emoji":"📈","tokens_out":4902,"duration_ms":46730,"temperature":0.7,"pith_summary":"Change point detection usually assumes a specific kind of change, but real non-stationary time series can shift in mean, variance, trend, periodicity, or a mix of these. This paper tries to establish that one algorithm can handle all these change types by looking for changes in the local frequency content of a deep Gaussian process fit to the data. The algorithm, called DGP-Spectral-AL, uses an acquisition function that balances a spectral-change score with spectral uncertainty, so it spends its limited measurement budget where changes are likely and where the model is unsure. The authors claim this beats a derivative-aware active-learning baseline on all fifteen simulated change patterns and matches or exceeds full-data baselines on five real-world datasets while sampling only a fraction of the points. If true, the payoff is cheaper, more flexible change point detection for expensive measurement settings such as drilling logs, pollution monitoring, and materials experiments.","feed_headline":"Change-point detection with fewer samples via spectral active learning","feed_subtitle":"Frequency-domain acquisition on deep GP predictions finds change points with fewer samples than baselines.","key_machinery":"The load-bearing object is the Spectral Change Detection Metric (SCDM), defined as $SCDM = SMC + SGD\\cdot(1-SCM)/2$, which combines spectral mean change, spectral correlation mapper, and spectral gradient distance to catch both abrupt and gradual spectral changes while staying insensitive to uniform amplitude scaling. This metric is computed on the sliding-window Fourier transform of the DGP's predicted mean, and it is paired with a spectral uncertainty term $SU$ obtained by Monte Carlo sampling of the DGP posterior and averaging the variance of the resulting spectra. The acquisition function $a(x)=\\beta\\cdot SCDM(x)+(1-\\beta)\\cdot SU(x)$ selects the next sampling locations, with $\\beta$ trading exploitation of detected spectral changes against exploration of high-uncertainty regions. The machinery works because the DGP's hierarchical warping smooths noise and models non-stationarity, so the spectral representation of its predictive mean carries the change signature that the thresholding step in Algorithm 1 converts into a set of change point estimates.","core_discovery":"On the paper's own terms, the central claim is that change points in non-stationary time series can be localized by treating the mean prediction of a Deep Gaussian Process as a signal, transforming it with a sliding-window Fourier transform, and actively querying new points where the local spectrum is changing and the spectral estimate is uncertain. The change evidence is summarized by a Spectral Change Detection Metric, $SCDM = SMC + SGD\\cdot(1-SCM)/2$, where SMC measures shifts in average spectral power, SCM measures correlation of spectral shapes, and SGD measures spectral gradient distance; this is combined with Monte-Carlo spectral uncertainty into an acquisition function $a(x)=\\beta\\cdot SCDM(x)+(1-\\beta)\\cdot SU(x)$. New samples are chosen at the maximizer, the DGP is refit, and after the active-learning budget is spent, change points are extracted by thresholding SCDM and suppressing a window around each detected peak. The paper reports that this method beats the DACD baseline in RMSE on every simulated pattern and reaches F1 scores up to 0.89 on real-world data while training on 30 initial points plus 10 batches, whereas the published baselines used the full datasets.","pith_inferences":["The SCDM formulation is not tied to short-time Fourier transforms; applying the same acquisition idea to wavelet or evolutionary spectral representations could extend the method to signals where a fixed window size is a poor match for the local dynamics.","The paper fixes $\\beta$ for the whole run; an adaptive schedule that starts with exploration and shifts toward exploitation as uncertainty shrinks could outperform the best fixed value without needing a grid search.","Because the uncertainty term is computed from DGP posterior samples, the method inherits any miscalibration in the DGP; a diagnostic that checks whether spectral variances match empirical error could make the acquisition function more reliable.","The F1 comparison would be airtight if the evaluation margin were explicitly matched to the margin used by the baseline benchmark; the paper sets the margin equal to its suppression interval, which is a sensible choice but leaves this matching as an assumption."],"forward_implications":["For simulated patterns including stratified, cycle, trend, systematic, shift, and their pairwise combinations, the method yields lower RMSE than DACD across all fifteen patterns.","On real-world datasets (Occupancy, Apple, Run log, Bee dance, Well log), the best $\\beta$ setting reaches F1 up to 0.8889 on Run log and 0.8571 on Well log, trained on far fewer points than the full-data baselines.","The method works with both Matérn 5/2 and RBF kernels, and the $\\beta$ parameter provides a meaningful exploration-exploitation dial: simpler shifts favor exploitation (high $\\beta$), while periodic and high-variability patterns favor exploration (low $\\beta$).","Because the selected samples concentrate around true change points as active learning proceeds, the method reduces measurement cost without sacrificing detection accuracy."],"supporting_citations":[{"why":"Introduces deep Gaussian processes, the modeling backbone that gives the method flexible non-stationary predictive means.","marker":"[7]"},{"why":"Supplies the doubly stochastic variational inference scheme used to train the DGP and to sample from its posterior.","marker":"[20]"},{"why":"Defines the DISS spectral-shape change metric that the paper adapts into its SCDM by adding spectral mean change.","marker":"[30]"},{"why":"Provides the DACD baseline, the derivative-aware active-learning method that the simulation experiments are compared against.","marker":"[35]"},{"why":"Supplies the five real-world benchmark datasets and the published F1 scores of the full-data baseline methods.","marker":"[29]"},{"why":"Establishes active learning for deep Gaussian process surrogates, the general sampling principle the acquisition function builds on.","marker":"[22]"}],"fun_headline_variants":["Active spectral learning finds change points with fewer samples","Deep GP spectral acquisition pinpoints change points with fewer queries","Change points from fewer samples: deep GP plus spectral active learning","Spectral change metric guides active deep GP sampling for change points","Active learning on deep GP spectral cues detects change points efficiently"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The real-world comparison assumes that the tolerance window used to count a detection as correct (the paper sets it equal to its suppression interval) is the same tolerance the published baselines used; if those tolerances differ, the F1 numbers are not directly comparable.","fun_headline_variants_meta":{"raw":{"variants":["Active spectral learning finds change points with fewer samples","Deep GP spectral acquisition pinpoints change points with fewer queries","Change points from fewer samples: deep GP plus spectral active learning","Spectral change metric guides active deep GP sampling for change points","Active learning on deep GP spectral cues detects change points efficiently"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000967,"raw_usage":{"total_tokens":4094,"prompt_tokens":905,"completion_tokens":3189,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":3108}},"tokens_in":521,"tokens_out":3189,"duration_ms":22330,"temperature":1.0,"reasoning_tokens":3108,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:54:17.395883+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the F1 scores of the baseline methods with the same tolerance window $M=\\delta$ used by the proposed method; if the baseline scores from the benchmark report change materially, the claim that the method beats existing techniques on real-world data is unsupported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces deep Gaussian processes, the modeling backbone that gives the method flexible non-stationary predictive means."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the doubly stochastic variational inference scheme used to train the DGP and to sample from its posterior."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the DISS spectral-shape change metric that the paper adapts into its SCDM by adding spectral mean change."},{"cited_title":"Active Learning for Abrupt Shifts Change-point Detection via Derivative-Aware Gaussian Processes","cited_arxiv_id":"2312.03176","evidence_quote":"Provides the DACD baseline, the derivative-aware active-learning method that the simulation experiments are compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes active learning for deep Gaussian process surrogates, the general sampling principle the acquisition function builds on."}],"review_version":1}