{"id":"2260a230-3fa7-4e5b-91fd-f961b45bef56","arxiv_id":"2608.00513","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"BayesSeg uses Bayesian optimization and a composite event-F1/NMI score to tune a threshold-based state segmentation method, reaching grid-search quality with 100 evaluations on a single-appliance dataset.","lead":"This paper presents BayesSeg, a pipeline that automatically tunes two thresholds used to split electricity consumption time series into steady and transition states, using Bayesian optimization and a new evaluation score. It reports that about 100 optimization rounds match a million-point grid search in quality while cutting search time from about 90 minutes to under a second, on one appliance dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Event_F1 matching rule in §2.2 is inverted: it rewards predicting a steady segment inside a true transition, so the reported event_F1 and composite scores do not measure event-detection quality.","rationale":"The paper's central architecture is segmentation–evaluation–optimization: the composite score (Eq. 9) is the objective that both grid search and Bayesian optimization maximize, and the abstract's headline numbers (composite 0.7149, event_F1 0.9340) are outputs of that objective. If the event_F1 matching rule is semantically inverted, then the evaluation layer does not measure event-detection quality, and the optimized parameters are those that place steady segments inside genuine transition intervals. This is not a minor calibration issue: it inverts the definition of a true positive. The correct semantics should match predicted transition segments to true transition events; the paper instead matches predicted steady segments to true transition events. Under the paper's rule, any algorithm that over-segments transitions into steady labels will be rewarded, while an accurate transition detector is penalized. The reported high event_F1 values are therefore not evidence of event-detection capability, and the composite score's 'broad plateau' is an artifact of a permissive, inverted match criterion. The BO efficiency claim—that 100 evaluations approximate the grid-search optimum of the paper's own objective—may still hold, but it becomes a statement about optimizing a flawed metric and cannot support the paper's claimed contributions. Other weaknesses (single appliance, in-sample optimization, unspecified τ in the abstract) are real but secondary; fixing them would not rescue the inverted metric. Therefore the appropriate verdict is REJECT, unless the authors re-run all experiments with a corrected event_F1 definition and the conclusions remain. The concrete test above would settle whether the inversion changes the results as expected.","tokens_in":18348,"tokens_out":5638,"duration_ms":65948,"concrete_test":"Reimplement event_F1 with the intended matching: for each true event interval g_i, count a match if a predicted transition segment t_j overlaps g_i within tolerance (e.g., max(start(g_i), start(t_j)) ≤ min(end(g_i), end(t_j)) + τ, or a boundary-distance variant). Recompute event_F1 and composite for the reported GS optimum (ε=4.20, Δ=2.07) and BO optimum (ε=3.97, Δ=3.80) at τ=2 on the same SustDataED2 Philips TV data. If event_F1 falls materially below 0.9340, or the composite-maximizing parameters move outside the reported plateau, the inverted matching rule is confirmed and the headline claims fail. A minimal diagnostic is to check whether a dummy predictor that labels all points 'steady' achieves near-perfect event_F1 under the paper's rule; under the corrected rule it should score near zero.","verdict_should_be":"REJECT","load_bearing_attack":"The load-bearing assumption is the event_F1 matching rule in Section 2.2. The paper defines real events as switching events (transitions) and then declares a successful match when a predicted steady-state segment s_j lies inside the true event interval: start(g_i)+τ ≤ start(s_j) and end(s_j) ≤ end(g_i)-τ. This is the inverse of event-detection semantics: a detector that labels the entire transition interval as 'steady' is rewarded, while a detector that correctly identifies the transition is not. As a result, event_F1 and the composite objective (Eq. 9) are not measures of switching-event quality; they measure the tendency to place steady segments inside transitions. This inflates the reported event_F1=0.9340 (Table 2, τ=2) and distorts parameter selection: both GS and BO select coarse-segmentation parameters (ε≈4, Δ≈2–4) where few transition segments are produced and many steady segments exist. Since the composite score is the objective being optimized, the BO-vs-gridsearch 0.35% comparison is internally consistent but optimizes a metric that contradicts the paper's stated contribution. The entire evaluation layer and all headline numbers depend on this definition.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BayesSeg, a pipeline for segmenting electricity consumption time series into steady and transition states using two thresholds (ε, Δ), evaluating with a composite score of event-level F1 and NMI, and tuning the thresholds via TPE-based Bayesian optimization. On a single Philips TV series from SustDataED2, it reports that BO with 100 evaluations reaches a composite score of 0.7149 and event_F1 of 0.9340, within 0.35% of an exhaustive 1,002,001-point grid search, at a claimed speedup over 5700×. It also compares BO with PSO, GA, and random search under paired seeds.","tokens_in":18693,"tokens_out":6716,"duration_ms":76898,"significance":"The BO-versus-grid-search comparison is a meaningful engineering result if the objective being optimized is legitimate, and the paper usefully documents the plateau structure of its composite score and performs a paired multi-optimizer study. The central evaluation layer, however, appears to define event matching in a way that rewards the inverse of event detection, so the reported event_F1 and composite scores do not currently measure the paper's stated contribution. The single-appliance, in-sample nature of the experiments further limits the generality claims. The significance is therefore conditional on correcting the matching definition and re-running the evaluation and optimization experiments.","major_comments":[{"comment":"The event_F1 matching rule is semantically inverted. The paper defines real events as switching events (transition intervals), but declares a successful match when a predicted steady-state segment s_j lies inside the true event interval: start(g_i)+τ ≤ start(s_j) and end(s_j) ≤ end(g_i)−τ. A correct event detector should output a transition segment overlapping the true event; a predicted steady segment inside the event is a false negative. Implemented literally, this rule rewards exactly the failure mode the abstract says the method avoids. Because the composite objective uses event_F1, the parameter selection and the headline numbers (event_F1=0.9340, composite=0.7149) measure the tendency to place steady segments inside transitions, not switching-event detection quality. The matching should be redefined in terms of predicted transition segments (or the complement of steady segments) an","section":"§2.2, Eq. (9), Table 2"},{"comment":"All reported optimal parameters are in-sample maxima computed on the same Philips TV series used to select them. There is no train/test split, cross-validation, or evaluation on other appliances/households. The conclusion acknowledges the single-appliance limitation, but this is not a minor caveat: the claimed 'automated configuration' and domain-agnostic generalization (Contribution 4) require evidence that optimized parameters transfer. As it stands, the 0.35% deviation claim is about matching the grid-search optimum of the same training objective, not about predictive performance on unseen data.","section":"§4.1–4.2, Table 2"},{"comment":"The convergence reporting is inconsistent. Table 4 states that for τ=2–6 the 99%-of-optimum threshold requires 28 iterations, while Table 5 reports for τ=4 a mean first_hit of 16.6±7.6 (50/50 success) for the same threshold. Table 4's first_hit values also appear to come from a single run rather than a distribution over seeds. Since convergence speed is part of the speedup claim, please clarify whether these are single-run or averaged values and reconcile the discrepancy.","section":"§4.3–4.4, Tables 4 and 5"},{"comment":"The statistical comparison shows that PSO is not significantly worse than BO for τ≥2 after Bonferroni correction (p>0.0167; Cohen's d≈0.27–0.30), and PSO is faster in wall-clock time (≈300 ms vs ≈530 ms). The text nevertheless concludes that BO 'leads in both solution quality and convergence speed.' The data support only a non-significant quality advantage and a runtime disadvantage. Please temper this claim or provide a stronger argument—beyond statistical significance—for preferring TPE in this application.","section":"§4.4, Table 5"}],"minor_comments":[{"comment":"Eq. (1) is typeset in a corrupted/malformed way; the steady-state condition should be written explicitly so that it matches the JudgeState pseudocode, which uses T=|s−es[|es|]| and M=|s−mean(es)|.","section":"Definition 1 / Eq. (1) and Algorithm 1"},{"comment":"Please define precisely whether each real event g_i is a point index or an interval, and how start(g_i) and end(g_i) are derived from the ground-truth transition labels. This ambiguity is critical given the tolerance-based matching.","section":"§2.2"},{"comment":"Report the random seed(s) and number of repetitions used for Tables 2–4. Table 5 uses seeds 42–91 and reports means and standard deviations, but Tables 2–4 appear to be single-run results, which makes the headline 0.35% deviation fragile.","section":"Tables 2–4"},{"comment":"The abstract reports event_F1=0.9340, but Table 1 shows that the maximum event_F1 at τ=2 is 0.978723 for a different parameter set. Please clarify that 0.9340 is the event_F1 at the BO-selected parameters, not the global maximum.","section":"Abstract and Table 2"},{"comment":"Several references are dated 2026 (Refs. 3, 19, 24). If these are in-press or preprints, please add appropriate arXiv/DOI identifiers so readers can access them.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The inverted event_F1 matching rule is the key issue; it affects the central evaluation metric and therefore all headline results. I do not read this as intentional gaming—the surrounding text suggests a genuine oversight in defining which predicted segment type matches a real event—but it requires re-running the experiments. The paper's scope is also narrower than claimed: one household, one appliance, and in-sample optimization. I would require the corrected metric and a clearer statement of generalization before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competent pipeline paper whose evaluation layer has a definitional bug that undercuts the headline numbers. The segmentation and optimization parts are standard but competently assembled, and the experiments are more thorough than most papers of this type.\n\nWhat's genuinely there: BayesSeg combines a dual-threshold steady/transition segmenter, a binary state mapping, NMI plus an event-level F1, and TPE Bayesian optimization. The integration is new as a configuration, and the experimental work is substantial: a full 1,002,001-point grid, paired 50-seed comparisons against PSO, GA, and random search with Bonferroni-corrected Wilcoxon tests, and clear tables. That is reproducible, honest legwork. The authors also state plainly in the conclusion that only one appliance at 0.5 Hz was tested and that alpha was fixed.\n\nThe soft spot is load-bearing. In Section 2.2, real events are switching transitions, but a match is declared when a predicted steady-state segment s_j satisfies start(g_i)+tau <= start(s_j) and end(s_j) <= end(g_i)-tau. That means a predicted steady segment contained inside the true transition interval counts as a successful event detection. That is the inverse of event detection: a segmenter that labels the transition as 'steady' is rewarded, and a segmenter that correctly marks the transition is not. The event_F1 values in Table 1 (0.9677-0.9840), the composite values, and the BO-vs-GS 0.35% comparison all inherit this inversion. The BO/GS comparison itself is internally consistent because both methods optimize the same objective, but the objective does not measure what the abstract claims.\n\nTwo smaller issues. First, the reported numbers are in-sample optima: the same labeled sequence is used to select parameters and report performance, with no train/test split or cross-validation. Second, the 5700x speedup is real but against a grid with step 0.01 in two dimensions; that's a strawman. Table 5's random-search baseline is more informative, and there BO's edge over PSO is small (and not always significant). The paper would be more honest if the abstract led with the continuous-search/runtime advantage rather than the million-point comparison.\n\nWho should read it: practitioners who want a working threshold-segmentation tuner for a known appliance might get a useful template. But no one should cite the event_F1 numbers until the matching rule is corrected to compare transition segments against transition intervals and the evaluation moves out of sample.\n\nMy recommendation: send it to peer review, because the framework is coherent and the flaw is corrigible, but flag the Section 2.2 definition as a must-fix before acceptance. As is, the headline claims don't survive.","headline":"The BO-vs-gridsearch story is internally consistent, but the event_F1 rule in §2.2 rewards placing a steady segment inside a transition, so the headline event and composite scores do not measure switching-event quality.","tokens_in":19133,"tokens_out":4285,"would_cite":false,"duration_ms":53360,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Bayesian optimization finds near-optimal segmentation parameters in roughly 100 evaluations, matching a million-point grid search within 0.35% on real household electricity data.","keywords":["non-intrusive load monitoring","time-series segmentation","steady-state detection","event-level F1","normalized mutual information","Bayesian optimization","TPE surrogate model","electricity consumption state time series"],"falsifier":"On the SustDataED2 Philips TV labels, implement the Section 2.2 matching condition exactly and inspect which predicted segments are counted as hits. If predicted steady segments lying inside ground-truth transition intervals are matched as events, the event_F1 objective is inverted and the composite scores in Tables 1–2 must be recomputed under the intended transition-overlap rule; if no such inversion appears, the reported event_F1 stands. A second, independent check: run BO with τ=2 and 100 evaluations over 50 seeds and verify the best composite stays within 0.35% of 0.717455.","tokens_in":18293,"feed_emoji":"⚡","tokens_out":9641,"duration_ms":111566,"temperature":0.7,"pith_summary":"BayesSeg claims that electricity-load segmentation can be made fully automatic: instead of hand-tuning thresholds, a Bayesian optimizer chooses them. The paper's pipeline labels each point as steady or transition using a local rule based on the tail and mean of the running steady segment, maps the result to a binary sequence, scores it with an equal-weighted combination of event-level F1 and normalized mutual information, then searches the two thresholds with a tree-structured Parzen estimator. On the SustDataED2 Philips TV record, roughly 100 evaluations find parameters whose composite score is within 0.35% of the best of 1,002,001 grid-search combinations, while cutting search time from about 5,300 seconds to under one second. The paper argues this removes the manual calibration bottleneck in non-intrusive load monitoring and transfers to any time series with stable and transient regimes.","feed_headline":"100 evaluations match a million-point search","feed_subtitle":"BayesSeg tunes steady/transition segmentation automatically, hitting grid-search quality at 5,700x speedup.","key_machinery":"The load-bearing object is the pair (ε, Δ) in the dual steady-state criterion: a new point s joins the current steady segment if |s − tail(es)| < ε, or if that fails but |s − mean(es)| < Δ. This local rule, applied sequentially and then complemented, fixes which segments exist. The evaluation objective is y = 0.5·event_F1 + 0.5·NMI, where event_F1 uses tolerance τ to match predicted steady segments to ground-truth switching events, and NMI compares the binary state label sequences. The optimizer is the tree-structured Parzen estimator: it splits past trials into good and bad sets by y, estimates densities l(x) and g(x), and proposes the next point maximizing l(x)/g(x). The argument depends o","core_discovery":"The paper's central claim is that a three-layer segmentation–evaluation–optimization loop can replace manual parameter tuning for electricity consumption state series. The segmentation layer defines steady-state segments by a dual local-statistics criterion—a point is absorbed into a steady candidate if it is close to the candidate's last value, or if that fails but is close to the candidate's mean—and extracts transition segments as the complement between consecutive steady segments. The evaluation layer projects both prediction and ground truth to binary steady/transient sequences, computes event_F1 by tolerance matching with slack parameter τ and NMI for global structure, and combines the","pith_inferences":["The published event-matching condition counts a steady segment inside a ground-truth transition interval as a hit for that transition; if implemented literally, reported event_F1 values would need recomputation under the inverse transition-overlap semantics before interpreting them as event-detection quality.","A natural stress test is to shrink the evaluation budget to 20–30 iterations and measure the spread of composite scores across many seeds; the broad plateau predicts little degradation, while a spike in variance would suggest the claim depends on the 100-iteration budget.","Since α=0.5 is fixed, the reported optimal parameters are only optimal for that weighting; treating event_F1 and NMI as separate objectives and tracing the Pareto frontier would show how the optimal (ε, Δ) region changes when detection rate matters more than structural consistency.","The single-appliance, single-household validation leaves the domain-agnostic claim open; the immediate next test is multi-appliance overlapping operation and coarser sampling rates, where the plateau structure may not survive."],"forward_implications":["Segmentation thresholds can be re-fit automatically for each new household, meter, or appliance in under a second, removing the manual calibration step.","Because the plateau of near-optimal composite scores is broad, precise threshold values matter less than entering the right region; a cheap optimizer suffices.","Combining event-level F1 with NMI gives a metric that can distinguish parameter quality where pointwise F1 saturates or collapses.","The framework's independence from domain features means the same segmentation–evaluation–optimization loop can be dropped into other steady/transient time-series tasks, such as industrial process or biomedical monitoring.","In deployment, the budget can be cut from 100 to roughly 30 evaluations with only about 1% loss, making the optimizer suitable for online adaptation."],"supporting_citations":[{"why":"Supplies the SustDataED2 residential dataset and ground-truth appliance state labels that every reported score is measured against.","marker":"[15]"},{"why":"Supplies the tolerance-based precision/recall matching idea that defines the event_F1 component of the composite objective.","marker":"[11]"},{"why":"Supplies Normalized Mutual Information for comparing predicted and ground-truth state sequences, the structural component of the composite score.","marker":"[12]"},{"why":"Prior application of Bayesian optimization to non-intrusive load disaggregation that motivates the optimization layer.","marker":"[26]"}],"fun_headline_variants":["BayesSeg: 100 evaluations, 0.35% off optimal","5700x faster segmentation tuning with BayesSeg","Automated segmentation: near-optimal in 100 evaluations","BayesSeg: from 5300s to under 1s for segmentation","Segmentation tuned in 100 tries, matches grid search"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The framework's headline numbers rest on treating a steady segment that lies inside a true transition interval as a successful detection of that transition; read literally, that rule rewards the opposite of what event detection should reward, so the high scores do not necessarily measure switching-event quality.","fun_headline_variants_meta":{"raw":{"variants":["BayesSeg: 100 evaluations, 0.35% off optimal","5700x faster segmentation tuning with BayesSeg","Automated segmentation: near-optimal in 100 evaluations","BayesSeg: from 5300s to under 1s for segmentation","Segmentation tuned in 100 tries, matches grid search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000732,"raw_usage":{"total_tokens":3146,"prompt_tokens":809,"completion_tokens":2337,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":2249}},"tokens_in":553,"tokens_out":2337,"duration_ms":19109,"temperature":1.0,"reasoning_tokens":2249,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T00:47:11.472612+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the SustDataED2 Philips TV labels, implement the Section 2.2 matching condition exactly and inspect which predicted segments are counted as hits. If predicted steady segments lying inside ground-truth transition intervals are matched as events, the event_F1 objective is inverted and the composite scores in Tables 1–2 must be recomputed under the intended transition-overlap rule; if no such inversion appears, the reported event_F1 stands. A second, independent check: run BO with τ=2 and 100 evaluations over 50 seeds and verify the best composite stays within 0.35% of 0.717455.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SustDataED2 residential dataset and ground-truth appliance state labels that every reported score is measured against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the tolerance-based precision/recall matching idea that defines the event_F1 component of the composite objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Normalized Mutual Information for comparing predicted and ground-truth state sequences, the structural component of the composite score."},{"cited_title":"A., ABID M","cited_arxiv_id":null,"evidence_quote":"Prior application of Bayesian optimization to non-intrusive load disaggregation that motivates the optimization layer."}],"review_version":1}