{"id":"969f5d35-4e3c-4c63-b5c6-14fff6384437","arxiv_id":"2505.07299","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Counterfactual fingerprints computed from sensor residuals can distinguish leakages from sensor faults in water distribution networks and provide operators with a contrastive explanation.","lead":"The authors propose counterfactual event fingerprints, a way to show water network operators which sensor readings would have to be different for an automated leak or sensor fault alarm to disappear. The method is tested on the Hanoi and L-Town benchmarks and performs slightly better than a non-interpretable baseline in distinguishing leaks from sensor faults.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The CDF optimization in Eq. (11) cannot produce the reported sensor-fault fingerprints, so the central isolation results are not reproducible from the stated algorithm.","rationale":"The reader's weakest assumption concerns the missing user-centric evaluation of interpretability. That is a real gap, but it is acknowledged in Section 11 and affects only the human-facing value proposition. The concern I identify is more load-bearing: the formal construction of the counterfactual event detection fingerprint is internally inconsistent with the event detection model and with the illustrative sensor-fault result. Because f_i ignores sensor i, the stated optimization in Eq. (11) cannot directly cancel the residual of a faulty sensor by changing that sensor's own previous value; doing so requires changing other sensors, while the remaining constraints force those other sensors to stay normal, making the feasible set empty for faults outside the normal prediction range. The paper gives no infeasibility handling or fallback, and Fig. 4(a) shows a pattern that the stated equations cannot produce. If this is a typo and the implementation actually uses a different formulation, the manuscript must state it; if not, the sensor-fault classification results are unexplained. I give credit for the L-Town transfer experiment, which is a genuine attempt at out-of-distribution evaluation, and for the generally clear writing. However, the central algorithm as written is under-specified on a point that determines the main quantitative claims. The appropriate verdict remains CONDITIONAL, but the required revision should focus on correcting or concretely specifying the CDF optimization and providing code or a clear feasibility analysis, rather than only adding a user study. This is why I keep the reader's verdict unchanged while disagreeing with the specific weakest assumption.","tokens_in":18338,"tokens_out":14431,"duration_ms":147960,"concrete_test":"Re-implement Eq. (11) for a Hanoi power-failure at sensor 13 using the paper's linear-regression event detection, the normal training set D, and the stated thresholds. Record (a) whether the feasible set is non-empty, and (b) which sensor has the largest |δ*|. If the feasible set is empty or the largest change is not on the faulty sensor, the sensor-fault fingerprint in Fig. 4(a) and the Table 2 isolation results cannot be obtained from the described algorithm. A useful second run is the same optimization with y(k) substituted for y(k−1), to test whether the reported figures actually correspond to a 'closest normal measurement' formulation rather than Eq. (11).","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 6 defines the closest CDF as a perturbation of the previous measurement vector y(k−1): choose x_cf ∈ D minimizing ||x_cf − y(k−1)||_1 subject to |f_i(x_cf) − y(k)_i| ≤ θ_i for every i (Eq. 11). In the event detection model, each f_i is a linear regression that deliberately ignores sensor i: 'The entry i of vector w_i is set to 0' (Section 5, Eq. 7). Hence the value of x_cf at sensor i does not enter f_i and cannot reduce the residual |f_i(x_cf) − y(k)_i|. For a sensor fault at sensor i, the alarm is raised because this residual is large; to make it small, x_cf values at sensors j≠i must be moved so that f_i predicts the faulty y(k)_i. But the constraints for j≠i require f_j(x_cf) to stay close to the normal observed y(k)_j, forcing those same x_cf_j to remain near normal. With x_cf restricted to the normal training set D, the feasible set is empty whenever the faulty value lies outside the normal prediction range (e.g., a power failure with y(k)_i=0 against normal pressures of tens of meters). The manuscript does not state how infeasible instances are handled. The illustrative sensor-fault CDF in Fig. 4(a) shows the largest change on the faulty sensor, which is exactly the component that Eq. (11) cannot use to change f_i. Therefore, either the implemented CDF differs from Definition 3 / Eq. (11), or the reported sensor-fault fingerprints and the F1 ≈ 0.94/0.97 isolation results are not produced by the stated algorithm. This is a correctness gap in the central construction, not merely a missing user study.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a framework for interpretable event diagnosis in water distribution networks. The authors introduce counterfactual event detection fingerprints (CDFs) and counterfactual event isolation fingerprints (CIFs), which are computed by finding the closest alternative sensor measurement pattern that would change an alarm or a classification decision. The CDFs are then used as features to train an interpretable classifier (decision tree or k-NN) to distinguish leakages from sensor faults. The methodology is evaluated on the Hanoi benchmark and on a modified L-Town network instance, reporting F1 scores around 0.94-0.97 for the proposed method, outperforming a residual-based baseline for small leakages. The paper also discusses limitations and future work, including the need for a user-centric evaluation.","tokens_in":18707,"tokens_out":6844,"duration_ms":58335,"significance":"If the results are reproducible and the definitions are tightened, the paper would make a valuable contribution to XAI for critical infrastructure: it is, to the authors' knowledge, the first systematic proposal of counterfactual fingerprints for WDN event diagnosis, and the experimental design (training on a simulated model and testing on a different network instance) is a genuine generalization test. The proposed representation is simple, visual, and potentially actionable for operators. However, the current manuscript contains a load-bearing gap in the definition of the CDF optimization (Eq. 11), which appears infeasible for the sensor-fault cases that are central to the evaluation, and the detection metrics in Tables 1 and 4 are internally inconsistent. The interpretability claim is also not yet supported by user evidence, though the authors explicitly acknowledge this.","major_comments":[{"comment":"The closest-CDF optimization as stated is infeasible for typical sensor faults. Since f_i is a linear regression that ignores sensor i (Section 5, Eq. (7)), the constraint |f_i(xcf) - y(k)_i| <= theta_i cannot be influenced by xcf_i. When y(k)_i is a power failure value (0 m) or a large offset while all other sensors are normal, any xcf in the normal training set D yields f_i(xcf) near the normal pressure range, so the constraint cannot be satisfied. The paper does not specify how infeasible cases are handled, and Fig. 4(a) shows the largest fingerprint change on the faulty sensor, which is exactly the component that Eq. (11) cannot use to satisfy the constraint for that sensor. Therefore, either the implemented CDF differs from Definition 3/Eq. (11), or the reported sensor-fault fingerprints and the corresponding classification results (Tables 2 and 5) are not produced by the stated algorithm. The authors should clarify the algorithm actually used and provide a feasibility analysis.","section":"Section 6, Eq. (11)"},{"comment":"The formal definition of a CDF as a perturbation of the observed time window is inconsistent with the implementation, which solves for a new previous measurement vector xcf = y(k-1) + delta* while holding y(k) fixed as the target. Consequently, the CDF plotted in Fig. 4 represents changes to the previous time step, not to the measurements at the moment of detection as the text and caption suggest. This should be stated explicitly, and Definition 2 and Eq. (9) should be rewritten to match the actual computational problem, including the T=1 specialization.","section":"Section 6, Definition 2 vs. Eqs. (10)-(12)"},{"comment":"The reported detection rates violate basic identities: TP+FN must equal 1 and FP+TN must equal 1 for rates computed on the same event/non-event bases. In Table 1, Leakage shows 0.98+0.08=1.06 and Small shows 0.96+0.22=1.18; similar inconsistencies appear in Table 4. The authors should report the exact definitions of TP, FP, FN, and TN (e.g., per time step vs. per event) and ensure the numbers are consistent.","section":"Tables 1 and 4"},{"comment":"The interpretability claim is not empirically validated. The paper acknowledges in Section 11 that 'A user-centric evaluation will be instrumental in demonstrating the methodology's real-world applicability,' but the abstract and title present interpretability as an achieved property. At minimum, the claims should be scoped to 'representation interpretability,' or a study with operators should be added. Without this, the central value proposition relative to non-interpretable baselines is not demonstrated.","section":"Sections 1, 7, and 11"}],"minor_comments":[{"comment":"The summation index j is not defined consistently with the argument y(k-i); there is a typo that should be corrected.","section":"Eq. (7)"},{"comment":"Eq. (8) uses a single threshold theta, while Eq. (11) uses per-sensor thresholds theta_i; the relationship between these thresholds should be clarified.","section":"Eq. (8) vs. Eq. (11)"},{"comment":"The baseline is called 'Non-interpretable baseline,' but a decision tree is itself interpretable; the distinction is in the input representation (residuals vs. CDFs), not the model class, so the label is misleading.","section":"Section 7"},{"comment":"The color scales differ between the subfigures within each figure; a shared scale would make the comparison more direct.","section":"Figures 4 and 6"},{"comment":"The number of test leakage scenarios (35) is much smaller than the training set; the authors should report confidence intervals or acknowledge this limitation explicitly.","section":"Section 10.1"},{"comment":"The text says 'mean an variance' but reports values with +/- that appear to be standard deviations; the notation should be made precise.","section":"Tables 1 and 4"}],"recommendation":"major_revision","confidential_remarks":"The main technical risk is the mismatch between Eq. (11) and the reported sensor-fault fingerprints; I recommend asking the authors for the actual optimization code or a feasibility argument. The k-NN results in Appendices A and B (F1=0.99/1.00) are suspiciously high and may indicate memorization or leakage from the training distribution; the authors should scrutinize these results. The paper's topic and scope are well suited to the journal, and the proposed framework is original if the correctness issues are resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: the paper brings counterfactual explanations to water-network fault diagnosis, and the two-stage trick of using counterfactual fingerprints as features for a classifier is a legitimate idea. The L-Town transfer experiment—training on one network instance, testing on another—is a real attempt at showing generalization. But the paper's central formal object, the CDF, is not what the figures and results say it is. The authors define the CDF as a perturbation of the past measurement window so that all ensemble residuals are within threshold (Eq. 11). Since each virtual sensor f_i deliberately ignores its own sensor, adjusting the faulty sensor's past value does nothing to the residual for that sensor. For a power failure (y_i=0) the feasible set is empty. The paper never says how infeasible cases are handled, yet reports F1≈0.95 for power failures and a fingerprint plot whose largest change sits on the faulty sensor—exactly the component Eq. (11) cannot change. Either the implementation differs from the described algorithm, or those results are not produced as claimed. That is a load-bearing reproducibility gap.\n\nThere are other issues. The detection tables violate rate identities: in Table 1, leakage has TP=0.98 and FN=0.08, which sums to 1.06; Table 4 has similar problems. These metrics need to be recomputed or reported correctly. The Hanoi evaluation appears to train and test on the same simulated scenarios, so the F1 values risk in-sample optimism; L-Town is cleaner. And the interpretability claim is untested—the authors admit no user study was done, which is honest but leaves the main motivation unmeasured. The CIF part is defined but not evaluated, so it's currently an appendix-level concept.\n\nIf the CDF computation is fixed to match the formalism (or the formalism revised to match the implementation) and the metrics corrected, this could be a solid contribution for the water-engineering XAI niche. As written, it needs major revision. I'd send it to review—the issues are catchable and fixable—but I would not cite it in its current form.","headline":"The paper's core idea is fine, but the CDF formalism doesn't match the reported fingerprints—following Eq. (11) can't produce the sensor-fault results, so the empirical claims aren't reproducible as written.","tokens_in":19294,"tokens_out":12589,"would_cite":false,"duration_ms":112403,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Counterfactual fingerprints can separate leaks from sensor faults in water distribution networks.","keywords":["water distribution networks","event diagnosis","counterfactual explanations","interpretable machine learning","leakage detection","sensor fault isolation","fault detection","counterfactual fingerprints"],"falsifier":"Show experienced water-network operators alarm logs from real events both with and without counterfactual fingerprint plots, and compare their event-type judgments: if the plots do not measurably improve accuracy or confidence, the claimed interpretability benefit is unsupported. For the classification claim itself, applying the same CDF decision-tree pipeline to a third, independent water network would reveal whether the leak-versus-sensor-fault separation is a general property or an artifact of the two benchmark networks.","tokens_in":18118,"feed_emoji":"💧","tokens_out":6524,"duration_ms":59246,"temperature":0.7,"pith_summary":"This paper tries to establish that a 'counterfactual event detection fingerprint'—the smallest set of changes to current sensor readings that would make a detected alarm disappear—is a useful and interpretable representation for diagnosing events in water distribution networks. The authors claim that leaks and sensor faults leave different fingerprint patterns: a sensor fault lights up mostly the faulty sensor, while a leak spreads changes across several sensors. On that basis, a decision tree trained on fingerprints distinguishes leakages from sensor faults with F1 scores of 0.94 for both classes on the Hanoi network and 0.97 for both on the L-Town network, outperforming a residual-based baseline especially for small leaks. The motivation is operator trust: instead of a black-box alarm, the operator sees what would have to change for the alarm to go away, which can be compared with their own experience. If correct, the framework turns event diagnosis into an inspectable, feedback-friendly loop rather than an opaque classification output.","feed_headline":"Fingerprint plots separate water-network leaks from sensor faults","feed_subtitle":"A minimal-change explanation reaches F1 0.94–0.97 and beats residual baselines on small leaks.","key_machinery":"The central object is the counterfactual event detection fingerprint (CDF): a vector $\\delta_{\\mathrm{CDF}}$ that, added to the observed sensor window, would flip the event-detection function from 'alarm' to 'no alarm', chosen to be closest to the observed data in the $\\ell^1$ norm and constrained to lie among recorded normal-operation measurements. It is computed as an ensemble-consistent explanation, meaning it must silence all virtual sensors in the detection ensemble at once. The companion counterfactual event isolation fingerprint (CIF) is the analogous minimal change that would flip the event-type classifier's output. The fingerprint carries the argument because its pattern of nonzero entries—spread out for leaks, concentrated on one sensor for faults—is what separates the two event classes and gives the operator a visual handle on the algorithm's decision.","core_discovery":"The central claim is that leakages and sensor faults produce robustly different counterfactual event detection fingerprints, and that these fingerprints are sufficient input for an interpretable classifier to identify the event type. Formally, given a residual-based detector that raises an alarm when a virtual-sensor forecast differs from observations, the closest CDF is the minimal change to the observed sensor window, restricted to plausible normal-operation measurements, that would turn the alarm off. The paper observes that leak CDFs tend to involve coordinated changes at many sensors, whereas sensor-fault CDFs concentrate on the faulty sensor; this contrast is what makes the downstream classification work. In the two case studies, the CDF-based decision tree reaches F1 0.94 for leaks and 0.94 for sensor faults in Hanoi and F1 0.97 for both classes in L-Town, while the direct-residual decision-tree baseline scores 0.70 on Hanoi leaks and 0.29 on Hanoi sensor faults. The authors also define counterfactual event isolation fingerprints (CIFs), which show what would have to change for the classifier to switch its diagnosis, extending the same explanation idea to the isolation stage.","pith_inferences":["The paper's motivating claim about operator trust is untested: no experiment measures whether operators actually understand fingerprint plots, trust them, or make better decisions with them; that would require a user study.","The discriminative power of the fingerprint likely depends on the residual detector's structure and on sensor placement; a network with very few or poorly placed sensors might not show the same leak-versus-fault contrast.","The same sparsity pattern that separates classes could be pushed further, for example to localize the leaking node or the faulty sensor, since a leak's fingerprint appears to emphasize sensors near the event.","Restricting fingerprints to recorded normal-operation data makes the explanation depend on the training period's demand patterns; unusual but legitimate hydraulic states could degrade both the explanation's plausibility and the downstream classification."],"forward_implications":["If the fingerprints really separate leaks from sensor faults, the same CDF computation can be reused to explain detection for any residual-based event detector, not only the linear-regression virtual sensors tested here.","Small leaks, which are the hardest cases for residual baselines, become classifiable from fingerprints: the L-Town small-leak F1 rises from 0.48 for the decision-tree baseline to 0.93 for the CDF-based decision tree.","Because the isolation classifier is a decision tree on fingerprints, the full pipeline remains inspectable: an operator can see both why an alarm fired and why a particular class label was assigned.","CIFs provide a concrete way to ask 'what would change this diagnosis?', which can support operator review, hypothetical reasoning, and feedback labeling in the monitoring loop.","The authors' stated boundary is that the method only handles event types present in training data; unknown event types are left for future work."],"supporting_citations":[{"why":"Supplies the definition of closest counterfactual explanations that the CDF adapts to the event-detection setting.","marker":"[29]"},{"why":"Provides the four sensor-fault types (noise, power failure, offset, drift) used to build fault scenarios.","marker":"[4]"},{"why":"Provides the Hanoi benchmark network used in the first case study.","marker":"[6]"},{"why":"Provides the realistic L-Town benchmark used in the second case study.","marker":"[45]"},{"why":"Defines ensemble-consistent explanations, the principle used to require a CDF to silence all virtual sensors.","marker":"[47]"},{"why":"Supplies the optimal pressure-sensor placement procedure used to choose the Hanoi sensor set.","marker":"[48]"},{"why":"Provides the hydraulic model equations used to simulate leakages and sensor faults on the network.","marker":"[42]"}],"fun_headline_variants":["Counterfactual fingerprints reveal leak vs sensor fault causes","Minimal-change explanations diagnose water-network events","Water leaks and sensor faults told apart by counterfactuals","Interpretable diagnosis: fingerprinting the closest alternative","Leak vs sensor fault? Counterfactual fingerprints explain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that operators will actually find counterfactual fingerprint plots intuitive and actionable; the paper does not test whether human understanding, trust, or decision quality improves when these explanations are shown.","fun_headline_variants_meta":{"raw":{"variants":["Counterfactual fingerprints reveal leak vs sensor fault causes","Minimal-change explanations diagnose water-network events","Water leaks and sensor faults told apart by counterfactuals","Interpretable diagnosis: fingerprinting the closest alternative","Leak vs sensor fault? Counterfactual fingerprints explain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000941,"raw_usage":{"total_tokens":4036,"prompt_tokens":977,"completion_tokens":3059,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":2984}},"tokens_in":593,"tokens_out":3059,"duration_ms":22282,"temperature":1.0,"reasoning_tokens":2984,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:20:16.498620+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Show experienced water-network operators alarm logs from real events both with and without counterfactual fingerprint plots, and compare their event-type judgments: if the plots do not measurably improve accuracy or confidence, the claimed interpretability benefit is unsupported. For the classification claim itself, applying the same CDF decision-tree pipeline to a third, independent water network would reveal whether the leak-versus-sensor-fault separation is a general property or an artifact of the two benchmark networks.","supporting_citations":[{"cited_title":"Reppa, M","cited_arxiv_id":null,"evidence_quote":"Provides the four sensor-fault types (noise, power failure, offset, drift) used to build fault scenarios."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the realistic L-Town benchmark used in the second case study."},{"cited_title":"One Explanation to Rule them All -- Ensemble Consistent Explanations","cited_arxiv_id":"2205.08974","evidence_quote":"Defines ensemble-consistent explanations, the principle used to require a CDF to silence all virtual sensors."},{"cited_title":"Casillas, V","cited_arxiv_id":null,"evidence_quote":"Supplies the optimal pressure-sensor placement procedure used to choose the Hanoi sensor set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the hydraulic model equations used to simulate leakages and sensor faults on the network."}],"review_version":1}