{"id":"e2930d01-9bd6-42fa-98b8-ea86b7be51e6","arxiv_id":"2505.00225","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A transformer that consumes the sequence of outage-update records predicts restoration time better on a customer-satisfaction-weighted metric than static tabular baselines on three utilities' storm data.","lead":"Power companies often estimate outage restoration times from a single snapshot or manual judgment. This paper tests a transformer that reads the full sequence of outage updates, reporting better customer-satisfaction-weighted accuracy on data from three utilities.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported CSI gains may stem from a data advantage: LTT sees the full update sequence (and likely trains on every prefix) while baselines get one static snapshot per event, with no specification of which snapshot.","rationale":"The reader's weakest assumption correctly targets whether baselines received a fair, information-matched input. My stress-test refines this into a concrete confound: LTT's longitudinal formulation naturally produces many more training examples (one per update prefix) than a static baseline (one per event), and the paper never states which snapshot is used for baselines. This is load-bearing because the 19.08% CSI improvement is only meaningful if the comparison controls for both the feature information and the training data volume. The reader noted the absence of error bars and malformed p-value, which are real but secondary; even perfect statistics would not rescue an unfair comparison. I recommend no change to the CONDITIONAL verdict because the issue is addressable by clarifying the protocol and, ideally, running the matched-information ablation. The concrete test I propose directly settles whether the gain is architectural or a data/information artifact.","tokens_in":11416,"tokens_out":6770,"duration_ms":69638,"concrete_test":"Obtain the exact preprocessing code and reproduce Table 1 under a matched-information control: give each baseline the feature vector of the latest available update (the same timestamp used as LTT's final input) plus simple aggregates such as update count and time since first update, and train all models on the same number of examples (e.g., baseline on the full set of update-level rows, or LTT only on final-update examples). If the CSI improvement over the best baseline falls below ~5% or loses significance, the headline claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that LTT improves CSI by 19.08% on average. This claim rests on the premise that the comparison isolates the longitudinal architecture. The paper never specifies what input the baselines receive. In 'Baseline Comparisons' it says baselines 'process only static, one-dimensional data,' and in 'Results and Discussion' it repeats that they are 'treated as a conventional tabular regression problem.' It does not state whether the static snapshot is the first update, the last update, or a fixed horizon, nor whether the baselines were trained on one row per event while LTT was trained on all update prefixes (200k updates vs 34k events). If baselines lack the latest crew-dispatch and restoration-progress features, or if LTT trains on ~6x more examples, the reported CSI gain is confounded by information and data volume, not attributable to the transformer. The p-value in the abstract is also malformed ('p > 0.001' indicates non-significance), but that is secondary.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Longitudinal Tabular Transformer (LTT) for predicting estimated times of restoration (ETR) from sequences of outage-event updates, using data from three utilities covering about 34,000 storm-related events and 200,000 updates. To reflect customer satisfaction, the authors introduce an asymmetric loss and three associated metrics (UPR, OPR-8, WAE, CSI), and report that LTT improves the CSI metric by an average of 19.08% over eight baselines. They also provide qualitative attention-map and SHAP analyses to interpret temporal dependencies and feature contributions.","tokens_in":11668,"tokens_out":3668,"duration_ms":38272,"significance":"If the reported results can be established under a fair comparison, the work addresses a practically important operational problem and offers a sensible longitudinal treatment of outage updates. The integration of customer-satisfaction considerations into the loss and metrics is useful and could influence industrial practice. However, the current evaluation does not yet support the central claim: the baselines receive an unspecified static input, the reported metrics partly coincide with the training objective, and no statistical significance or variability is reported. These issues must be resolved before the claimed 19.08% CSI improvement can be considered credible.","major_comments":[{"comment":"The manuscript does not state what static input the baseline models receive. The text in 'Baseline Comparisons' says baselines 'process only static, one-dimensional data,' and in 'Results and Discussion' says they are 'treated as a conventional tabular regression problem,' but it never specifies whether the snapshot is the first update, the most recent update, or a fixed time horizon, nor whether baselines are trained on one row per event while LTT is trained on every update prefix. Given that the data include about 200,000 updates for 34,000 events, LTT may be seeing all updates and many more training examples than the baselines. The reported CSI gains could therefore stem from an information and data-volume advantage rather than from the longitudinal architecture. This is the load-bearing comparison for the abstract's headline claim and must be clarified and, if necessary, re-run with information-matched baselines.","section":"Baseline Comparisons / Results and Discussion"},{"comment":"The training loss (Eq. 3), the WAE metric (Eq. 7), and the CSI metric (Eq. 8) share the same piecewise alpha/beta structure. Since LTT is explicitly trained to minimize this exact loss, reporting that LTT has lower WAE and higher CSI than baselines is partly a tautology; the baselines, unless trained with the same loss, are being compared on a metric that the proposed model directly optimizes. The paper should demonstrate improvements on independent metrics that are not part of the training objective, and should report standard deviations or confidence intervals for the five-trial averages in Table 1, along with a valid significance test. The abstract's notation '(p > 0.001)' is not a meaningful significance statement; if a p-value was computed, report the actual value and the test used.","section":"Evaluation Metrics and Asymmetric Loss (Eqs. 3, 7, 8)"},{"comment":"The data split is described as stratified sampling based on storm magnitude, but the paper does not state whether the split is performed at the event level or the update level. If updates from the same outage event appear in both training and test sets, the longitudinal model could memorize event-specific restoration outcomes, inflating its apparent performance. Please clarify that all updates of a given event are confined to a single split, and describe how the temporal ordering of events is handled to prevent leakage.","section":"Experimental Setup / Data"}],"minor_comments":[{"comment":"The caption contains a typo: 'Storm-Realted' should be 'Storm-Related'.","section":"Figure 1 caption"},{"comment":"In the sentence 'OPCO-3's LLT model predictions,' 'LLT' should be 'LTT' to match the model name used elsewhere.","section":"SHAP Analysis section"},{"comment":"The phrase '(p>0.001)' is self-contradictory as written; p>0.001 indicates non-significance, whereas the surrounding context implies a significant difference. If a test was performed, report the exact p-value or a confidence interval.","section":"Abstract"},{"comment":"The time delta is computed relative to the first update, so the first update always has delta 0. Please clarify whether this is intentional and how it interacts with the positional encoding for the first update.","section":"Eq. (10)"},{"comment":"The attention visualizations and SHAP feature-importance discussion are qualitative. Consider providing quantitative summaries, such as attention concentration metrics or rank correlations across revisions, to support the claimed temporal patterns.","section":"Attention and SHAP interpretation"}],"recommendation":"major_revision","confidential_remarks":"The core architectural idea is reasonable, but the evaluation as written cannot support the headline 19.08% CSI improvement because of the undefined baseline input and the overlap between the training loss and the reported metrics. The authors should be asked to provide an information-matched baseline protocol, full event-level splits, standard deviations, and valid significance tests. The paper also lacks pointers to code or data, which is unusual for a reproducibility-oriented ML submission; this may be a fit issue for the venue. I recommend major revision rather than rejection because the issues appear fixable within the scope of the manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the paper's core claim—that the Longitudinal Tabular Transformer beats eight baselines on customer-weighted metrics—is plausible but not yet demonstrated. The biggest problem is that the baselines are not information-matched. The authors describe them as 'conventional tabular regression problems' with 'static, one-dimensional data' but never say which update snapshot they get. If the baselines see only the first update (or any single snapshot) while LTT sees the full sequence, the reported 19.08% CSI improvement could reflect data advantage, not architecture. The paper also trains LTT on all update prefixes (200k examples) while baselines likely train on one row per event (34k examples). That's a six-fold data-volume difference. The authors need to specify the baseline input, ideally feeding them the same cumulative features at the same update times, and report results with error bars.\n\nSecond, the evaluation metric is the training objective. The asymmetric loss (Eq 3) is identical to WAE (Eq 7), and CSI (Eq 8) uses the same alpha and beta. So LTT is trained to minimize exactly the metric it's judged on, while the baselines presumably train on MSE. That's a mismatch. The paper should either train baselines with the same asymmetric loss or evaluate with a metric not used during training. The RMSE results do show LTT ahead, but that's still confounded by the information issue.\n\nThe abstract's 'p > 0.001' is not a valid significance statement; p > 0.001 means the result is not significant at that level. There are no standard deviations or confidence intervals despite 'averaged over five trials.' These are easy fixes.\n\nNow the good part. The longitudinal framing is a natural fit for ETR prediction, and the customer-satisfaction-weighted metrics are genuinely useful, grounded in survey data. The paper is clearly written and uses real data from three utilities. The attention and SHAP analyses are standard but provide evidence that the model uses temporal information in a sensible way.\n\nThis paper deserves peer review. The flaws are addressable, and the problem is important. A referee should ask for (1) a precise description of baseline inputs, ideally a fair comparison where baselines get the same cumulative features, (2) uncertainty estimates, and (3) a proper significance test. If those are addressed, the contribution could hold up. As written, the headline number isn't credible, but the underlying idea and the new metrics are worth taking seriously.","headline":"Sensible longitudinal framing and useful customer-centered metrics, but the headline CSI gain is not credible yet because the baselines get less information and the evaluation metric equals the training loss.","tokens_in":12138,"tokens_out":3529,"would_cite":false,"duration_ms":35172,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A model that reads the entire sequence of outage updates predicts restoration times and improves a customer-satisfaction score by an average of 19.08%.","keywords":["estimated time of restoration","longitudinal tabular transformer","power outage prediction","asymmetric loss function","customer satisfaction impact","tabular transformer","self-attention","feature attribution"],"falsifier":"Retrain every baseline on the same data the LTT sees—for instance, by flattening all updates into one feature set or feeding the update sequence to a standard sequence model—and compare the Customer Satisfaction Impact. If the 19.08% average improvement shrinks to a few points or disappears, the paper's claim that the longitudinal architecture drives the gain would be refuted, because the gain would then be an artifact of information disparity.","tokens_in":11255,"feed_emoji":"⚡","tokens_out":7938,"duration_ms":74786,"temperature":0.7,"pith_summary":"The paper sets out to show that Estimated Times of Restoration for storm-related power outages should be predicted from the full sequence of event updates, not from a single static snapshot. It introduces a Longitudinal Tabular Transformer that encodes each update's categorical and continuous features, adds time-delta positional information, and uses self-attention across updates to produce a prediction from the latest update representation. Trained with an asymmetric loss that penalizes under-prediction more than over-prediction, and over-prediction beyond eight hours again, the model is claimed to improve the Customer Satisfaction Impact score by an average of 19.08% over eight baselines across 34,000 outage events at three utilities. If correct, this gives utilities a way to refresh restoration estimates automatically as crews file updates, with errors weighted toward what customers actually experience.","feed_headline":"Outage ETR model cuts customer-impact error by 19%","feed_subtitle":"A longitudinal transformer beats eight static baselines on 34,000 storm outages across three utilities.","key_machinery":"The load-bearing object is the Longitudinal Tabular Transformer: each outage event is a variable-length sequence of updates; categorical features are embedded, concatenated with Z-normalized continuous features, linearly projected to $d_{\\text{model}}=128$, and added to a positional encoding of the time delta $\\Delta t_{i,j}$ since the first update. Six transformer encoder layers with 16 heads, masked for variable length, attend over the update sequence, and the representation of the last update is passed through a fully connected head to predict the ETR. The second piece of machinery is the asymmetric scoring scheme: a piecewise loss $\\ell(\\epsilon_i)$ with penalties $\\alpha=5$ for under-prediction, $1$ for over-prediction within eight hours, and $\\beta=2$ for over-prediction beyond eight hours, together with the derived metrics UPR, OPR-8, WAE, and CSI.","core_discovery":"The central claim is that ETR prediction is a longitudinal regression problem, and that modeling the update sequence directly with a transformer is what improves performance. Concretely, the paper reports that the Longitudinal Tabular Transformer achieves the best Weighted Asymmetric Error and Customer Satisfaction Impact among all tested methods on every storm-size category for all three operating companies, with an average CSI improvement of 19.08% over the baselines. It also introduces the CSI metric itself, defined as $1 - (\\alpha \\cdot \\mathrm{UPR} + \\beta \\cdot \\mathrm{OPR\\text{-}8})/(\\alpha+\\beta)$, with $\\alpha=5$ and $\\beta=2$ set from customer-survey research, and proposes an asymmetric loss whose piecewise form matches the measured satisfaction curve: under-predictions are punished hardest, small over-predictions up to eight hours are punished least, and large over-predictions are punished again.","pith_inferences":["Editorial inference: the same longitudinal framing is a natural fit for water, gas, or telecom outage restoration, where similar update streams exist; the paper does not test those domains.","Editorial inference: the size of the claimed advantage may depend on the specific tolerance curve—the eight-hour threshold and the $\\alpha:\\beta$ ratio—so utilities with different customer-survey curves would need to recalibrate before seeing similar gains.","Editorial inference: a decisive test of the architectural claim would be to give a sequence-capable baseline, such as an LSTM, the same full update history; the paper's comparison treats baselines as static models, so part of the gain could come from access to later updates rather than from self-attention per se."],"forward_implications":["If the reported CSI gain holds, utilities can replace manual or static ETR estimates with predictions that refresh whenever an outage update arrives, reducing the rate at which customers are told restoration will happen sooner than it actually does.","The asymmetric loss and the eight-hour tolerance give an explicit, tunable link between a regression objective and customer satisfaction; the same piecewise loss could be used for any service-recovery prediction with asymmetric tolerance.","The feature-attribution and attention results imply that early updates—customer count and priority, then crew-dispatch status and recent restoration rolling averages—carry most of the signal, giving operations teams a concrete list of which field data to keep accurate.","The architecture's modularity over static inputs, dynamic variables, and variable-length sequences suggests it can be adapted to other longitudinal tabular regression problems without changing the core attention mechanism."],"supporting_citations":[{"why":"Supplies the industrial customer-satisfaction ratios that set $\\alpha=5$ and $\\beta=2$ in the loss, WAE, and CSI.","marker":"(Geno 2023)"},{"why":"XGBoost baseline, one of the eight models the LTT must beat in the comparison.","marker":"(Chen and Guestrin 2016)"},{"why":"LightGBM baseline, the strongest tree-based competitor in several OPCO and storm-size cells.","marker":"(Ke et al. 2017)"},{"why":"CatBoost baseline, providing a third categorical-capable boosting comparison.","marker":"(Prokhorenkova et al. 2018)"},{"why":"TabTransformer baseline, the closest transformer-for-tabular competitor, needed to isolate the longitudinal contribution.","marker":"(Huang et al. 2020)"},{"why":"FT-Transformer baseline, a strong deep tabular model that anchors the deep-learning comparison.","marker":"(Gorishniy et al. 2021)"},{"why":"ResNet baseline; its poor performance in some cells frames the gap the LTT must explain.","marker":"(He et al. 2016)"},{"why":"Supplies the feature-attribution values used for the feature-importance and temporal-significance analysis.","marker":"(Lundberg and Lee 2017)"}],"fun_headline_variants":["Transformer predicts outage ETR, cuts customer-impact error 19%","Longitudinal transformer improves outage ETR by 19% on 34k events","Modeling outage update sequences boosts ETR accuracy 19%","Outage ETR transformer: 19% better on three utility datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results assume the baseline models were given a fair, information-matched input; the paper says baselines were treated as conventional static tabular regression problems while the LTT saw the full update sequence, and it never clarifies whether baselines saw only the first update or a less informative snapshot.","fun_headline_variants_meta":{"raw":{"variants":["Transformer predicts outage ETR, cuts customer-impact error 19%","Longitudinal transformer improves outage ETR by 19% on 34k events","Modeling outage update sequences boosts ETR accuracy 19%","Outage ETR transformer: 19% better on three utility datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000514,"raw_usage":{"total_tokens":2518,"prompt_tokens":988,"completion_tokens":1530,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":604,"completion_tokens_details":{"reasoning_tokens":1452}},"tokens_in":604,"tokens_out":1530,"duration_ms":10945,"temperature":1.0,"reasoning_tokens":1452,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:47:22.821263+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain every baseline on the same data the LTT sees—for instance, by flattening all updates into one feature set or feeding the update sequence to a standard sequence model—and compare the Customer Satisfaction Impact. If the 19.08% average improvement shrinks to a few points or disappears, the paper's claim that the longitudinal architecture drives the gain would be refuted, because the gain would then be an artifact of information disparity.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the industrial customer-satisfaction ratios that set $\\alpha=5$ and $\\beta=2$ in the loss, WAE, and CSI."}],"review_version":1}