{"id":"514ca354-bca5-429b-8091-a543e51c6345","arxiv_id":"2501.15085","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A T-symmetry regularized graph-neural-network offline RL policy, deployed in a production data center, reduced air-side cooling energy by 14-21% versus PID control over 2000 hours without observed safety violations.","lead":"Researchers built an offline reinforcement learning system that controls air-cooling units in a working data center, reporting 14-21% less cooling energy than the existing PID controllers. The result matters because data centers spend a large share of electricity on cooling, and this is one of the first real deployments of offline RL in a safety-critical industrial loop.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Field savings claim rests on unadjusted before-after comparisons; weather and water-side confounders are not controlled, and the 4-ACU comparison uses only two PID days.","rationale":"The reader's weakest assumption correctly identifies the risk of confounding in the field comparison, but the specific claim that all RL days are in September-November is inaccurate: Table 1 clearly places three RL days (May 7-9) immediately after the two PID baseline days (May 5-6) in each room. Those adjacent-day comparisons substantially reduce the seasonal confound, though they still lack weather data and replicate only the PID side twice. The more serious confound applies to the 6-ACU and all-ACU comparisons in September and November, which are months apart and show higher server loads, making the ACLF reductions partly a denominator artifact. The 14-day experiment in Figure 4 also compares two different two-week windows without weather covariates. No error bars or replications are provided for any field or testbed result, so the effect size is not statistically anchored. This does not undermine the safety or feasibility claims, but it does mean the headline energy-savings number is not robustly attributable to the RL policy. The CONDITIONAL verdict is therefore appropriate: the paper should be accepted only if the authors supply covariate-adjusted or randomized evidence. My read does not change that verdict.","tokens_in":18753,"tokens_out":11090,"duration_ms":95681,"concrete_test":"Obtain hourly outdoor temperature, relative humidity, chilled water supply temperature, and server power time series for all Table 1 and Figure 4 periods; fit a linear mixed model of hourly ACLF (or ACU power) with a controller indicator, server power, and these weather/water-side covariates. Report the controller coefficient and 95% CI. If the coefficient is not significantly negative or the implied saving drops below about 10%, the 14-21% claim is confounded. As a secondary check, report per-ACU energy for controlled vs uncontrolled ACUs in the 4-ACU experiments to verify that the total-ACU saving is not an artifact of load shifting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim - 14-21% cooling energy savings - is established entirely by observational comparisons of PID and RL periods separated in time (Section 4.1, Table 1, Figures 3-4). The reader's seasonality objection is partially misplaced: the 'Ours (4 ACUs)' columns in Table 1 are May 7-9, adjacent to the May 5-6 PID baseline, so those three comparisons are not seasonally confounded. But the 6-ACU (Sep 23-29) and all-ACU (Nov 11-12) comparisons are months later, and no outdoor temperature, humidity, or water-side data are reported for any period. Server loads also differ (e.g., Room A: PID 552-555 kW vs Sep/Nov 572-578 kW), so the ACLF drops are partly denominator effects. The 14-day experiments (Fig. 4) compare RL (Jun 17-Jul 1) with PID (Jul 2-16); July is typically hotter, which would bias against RL, but no weather data are shown. Across all field comparisons there is no replication: PID has two single-day baselines per room, and each RL condition has one multi-day run. Without covariate adjustment or error bars, the effect size could be inflated by day-to-day and seasonal variation. The testbed experiments (Fig. 6) are controlled but run each method once per load condition, so they provide no uncertainty estimate. This does not invalidate the deployment, but it means the headline savings figure is not yet causally established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a physics-informed offline reinforcement learning (RL) framework for optimizing air-side cooling in data centers. The method trains a T-symmetry-enforced graph neural network dynamics model (TTDM) to learn latent state representations, then performs offline Q-learning in the latent space with a TD3+BC-style policy regularizer. The framework is deployed in a commercial data center for closed-loop control of air-cooling units, with over 2000 hours of experiments, and the authors report 14–21% reductions in air-side cooling load factor (ACLF) relative to PID control, with no safety violations. The paper also includes testbed experiments comparing the method with PID, MPC, CCA, IQL, CQL, and FISOR, as well as ablations on the GNN and T-symmetry components.","tokens_in":19085,"tokens_out":8369,"duration_ms":69329,"significance":"If the reported 14–21% energy savings are real, this paper would be a significant demonstration of offline RL in a safety-critical industrial deployment, showing that a policy learned from limited historical data can replace conventional PID control. The strengths include a real production deployment with no safety violations, a physically motivated dynamics model combining GNN and T-symmetry regularization, and a testbed evaluation against multiple baselines. However, the evidence as presented does not yet rigorously establish the causal effect: the field comparisons are observational before-after studies without weather or water-side controls, and the testbed comparisons lack replication. The paper's contribution is therefore best evaluated as a promising case study rather than a definitive proof of the method's superiority.","major_comments":[{"comment":"The headline 14–21% savings claim is based on non-randomized before-after comparisons. The PID baseline consists of two single days (May 5–6, 2024) per room, while the RL periods are May 7–9 (4 ACUs), September 23–29 (6 ACUs), and November 11–12 (all ACUs). No outdoor temperature, humidity, or water-side data are reported for any of these periods, and the server loads differ (e.g., Room A: PID 552.17–555.31 kW vs. 572.77 kW in the September period). Because ACLF is defined as ACU energy divided by server energy, a higher server load can mechanically lower the ACLF even with the same ACU power. The text says the periods were selected for 'similar server load patterns,' but no matching procedure or balance table is provided, and the durations are not comparable: the September period lasts 6 days while the PID baseline is 5.5–7.5 hours. Without covariate adjustment, confidence intervals, or a formal causal design, the 14–21% figure cannot be causally attributed to the RL policy based on the evidence presented.","section":"Section 4.1, Table 1"},{"comment":"The experimental comparisons are single-run with no replication. In the production DC, each RL condition is one multi-day run and the PID baseline is only two days per room; in the testbed, each method is run once per load condition. The paper therefore provides no estimate of day-to-day variability, sensor noise, or controller performance variance. The claim that the method outperforms all baselines under all load conditions in the testbed is based on single measurements, and the ACLF values in Figure 6 show no error bars. To support the reported effect sizes, the authors should provide multiple runs (or a resampling-based uncertainty estimate) for at least the testbed experiments.","section":"Section 4.2, Figure 6; Section 4.1, Figures 3–4"},{"comment":"The policy regularization coefficient α was tuned in the range [2.5, 10] and the best-performing model was selected for deployment, and the reward weights β1–β4 and r0 are determined from the dataset statistics. This selection procedure means the reported results may partly reflect tuning on the deployment target rather than the inherent sample efficiency of the method. The paper should report the sensitivity of the deployed policy to α and to the reward weight scales, and ideally evaluate the offline training procedure across multiple seeds or via a validation protocol that avoids selecting the model on the same data used for the final comparison.","section":"Section 3.3, Eq. (9), and Appendix D.1"},{"comment":"The analysis intended to show that the air-side optimization does not affect the water-side system is insufficient. It compares CWP frequency and EWT before and after the RL policy takes over in only two time windows (November 10–11 for Room A, October 29–30 for Room B), with no quantitative change measures, significance tests, or simultaneous outdoor/weather conditions. This does not rule out water-side or ambient confounders in the main field comparisons of Table 1, which span different seasons.","section":"Appendix C.3, Figure 12"}],"minor_comments":[{"comment":"The phrase '14∼21% energy savings' should be qualified as a reduction in air-side cooling load factor (ACLF) relative to the PID baseline, not total data-center energy, to avoid overstatement.","section":"Abstract"},{"comment":"The sentence 'we select several time periods (lengths from 5.5 to 7.5 hours)' is inconsistent with Table 1, which includes a 6-day period (September 23–29) and a 24-hour period (November 11–12). Please correct or clarify the matching criteria.","section":"Section 4.1, paragraph after Table 1"},{"comment":"The notation \\dot{s} is used but never defined; define \\dot{s} = (s' - s)/Δt to make the chain-rule derivation self-contained.","section":"Equations (3) and (4)"},{"comment":"In the TTDM learning loop, the mini-batch is written as {(s,a,s',a')} but a' is never used; the reverse dynamics in Eq. (4) uses the same action a. Change the notation to {(s,a,s')}.","section":"Algorithm 1"},{"comment":"The listing of which ACUs were controlled (ACU 1-6, 1-5, 2-5, 2-4 in Room A; ACU 1-1, 1-2, 2-1, 2-2 in Room B) is hard to parse without the layout; consider adding a table or annotated figure.","section":"Appendix B.1"}],"recommendation":"major_revision","confidential_remarks":"This paper is an interesting application of offline RL with a real deployment, which is rare. However, the central claim in the abstract overstates the evidence: the field comparison is an uncontrolled before-after study. The authors should be encouraged to either add a stronger causal analysis (e.g., matching on outdoor temperature and server load, reporting confidence intervals) or soften the claim to 'observed ACLF reduction under specified conditions.' The testbed experiments also need error bars or multiple runs. The methodology section is generally sound, and the deployment itself seems credible, so I believe the paper can be revised to an acceptable level."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this is a serious systems paper with a real deployment, 2000 hours of closed-loop operation, and a concrete efficiency gain. The 14-21% ACLF reduction is the hook, but it is not proven to the standard the abstract suggests. The field evidence is observational, some comparisons span seasons, and there are no error bars anywhere.\n\nWhat is genuinely new: the authors build a T-symmetry regularized latent-space offline RL method on top of their own TSRL framework, add a GNN-based thermal dynamics model (TTDM), and take the whole thing into a production data center. That is not a simulation exercise. They controlled 4, 6, and eventually all ACUs in two server rooms, ran for 2000 hours, report no safety violations, and back the method with controlled testbed comparisons against PID, MPC, CQL, IQL, and FISOR. The ablations show both the GNN structure and T-symmetry enforcement reduce prediction error and improve policy performance. That is real engineering and real evidence of a working system.\n\nThe soft spot is the central savings claim. The 4-ACU comparisons in Table 1 are actually temporally adjacent (May 5-6 PID vs May 7-9 RL), so the seasonality objection is less damaging there. But the 6-ACU (September) and all-ACU (November) runs are months later, with no outdoor temperature, humidity, or water-side data reported. Server loads differ across periods, so the ACLF denominator is not stable. The 14-day experiment compares RL (Jun 17-Jul 1) with PID (Jul 2-16), and July being hotter would bias against RL, but no weather data are shown to establish that. On the testbed, each method runs once per load condition, so there is no uncertainty estimate. The paper's own Appendix F admits the single-facility scope and reward-penalty approach to safety. These are specific, addressable gaps, not fatal flaws — but they mean the impressive-sounding 14-21% is not yet a measured causal effect, it is an observed difference under uncontrolled conditions.\n\nWho is this for? Researchers in offline RL who care about real-world deployment, and anyone working on data center cooling or industrial control. It deserves a serious referee, not a desk reject. The recommendation: send it to peer review, and insist the authors add covariate-adjusted or randomized field evidence, report weather and water-side conditions, provide replication runs or error bars on the testbed, and soften the abstract to match the evidence. The deployment is valuable even if the exact savings number needs another round of experiments.","headline":"A credible real-world deployment of offline RL for data center cooling, but the headline 14-21% savings claim is not yet causally established due to observational field comparisons lacking weather and replication controls.","tokens_in":19632,"tokens_out":1855,"would_cite":true,"duration_ms":18253,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An offline reinforcement learning controller replaced PID control in a production data center's cooling system and achieved 14–21% energy savings over 2,000 hours of operation without any safety constraint violations.","keywords":["offline reinforcement learning","data center cooling","time-reversal symmetry","graph neural network","latent dynamics model","energy efficiency","real-world deployment","thermal safety"],"falsifier":"Run both controllers in the same server room on alternating days within the same week, or hold outdoor temperature, humidity, and entering water temperature fixed (or regress them out), and check whether the 14–21% ACLF drop survives; if the gap shrinks to near zero, the claimed energy savings are an artifact of period differences.","tokens_in":18597,"feed_emoji":"❄️","tokens_out":7428,"duration_ms":61646,"temperature":0.7,"pith_summary":"This paper claims that a data center's air-side cooling can be safely optimized purely from historical operational data, without a simulator, using an offline reinforcement learning approach. The proposed framework builds a graph-neural-network dynamics model that respects time-reversal symmetry, learns compact latent representations of the cooling system, and trains the control policy in that latent space. Deployed in a large commercial data center, the policy controlled up to all of the air-cooling units and reduced cooling energy by 14–21% compared with the default PID controllers, with no safety or operational constraint violations over 2,000 hours of experiments. If true, the result would mean that data-limited, safety-critical industrial control problems need not rely on hand-tuned PID loops or expensive simulation environments.","feed_headline":"Offline RL cuts data-center cooling energy by 14–21%","feed_subtitle":"A physics-informed offline RL policy ran 2,000 hours in a live data center, beating PID with zero thermal violations.","key_machinery":"The key machinery is the T-symmetry enforced Thermal Dynamics Model (TTDM): a graph neural network encoder that maps states and actions to latent codes, plus forward and reverse latent ODE dynamics constrained to be negatives of each other, i.e. $f(z_s,z_a) = -g(z_{s'},z_a)$ where $z_{s'} = z_s + f(z_s,z_a)$. Time-reversal symmetry means the underlying transition law is unchanged when time runs backwards; enforcing it acts as a physics-based regularizer that keeps policy-induced samples physically plausible even outside the training distribution. The Q-function and policy are then optimized in this latent space, using a TD3+BC-style objective with an added T-symmetry consistency loss, which is what enables sample-efficient, safe policy learning from limited historical data.","core_discovery":"The central discovery is that enforcing time-reversal symmetry in a learned latent dynamics model makes offline RL practical for real-world data center cooling. The paper's T-symmetry enforced Thermal Dynamics Model (TTDM) couples a pair of latent ODE dynamics, forward $f(z_s,z_a)$ and reverse $g(z_{s'},z_a)$, with the constraint $f(z_s,z_a)=-g(z_{s'},z_a)$, and a GNN encoder that captures spatial and control dependencies among sensor and ACU nodes. This yields well-behaved latent states in which the Q-function and policy can be learned from roughly 140,000 to 180,000 historical samples, far fewer than typical offline RL benchmarks. In field tests in two production server rooms, the learned policy achieved 14–21% lower air-side cooling load factor than PID controllers, and ablations on a 22-server testbed show that both the GNN structure and the T-symmetry term improve prediction error and policy performance.","pith_inferences":["Since the reported savings come from comparing May PID days with September through November RL days, a cleaner causal test would be a randomized crossover or a matched-period comparison that also logs outdoor temperature, humidity, and chilled-water conditions; until then, some of the 14–21% could reflect seasonal differences.","The same latent T-symmetry recipe could be tried on other thermal-regulation tasks, such as HVAC in buildings or liquid cooling loops, where dynamics are approximately reversible and data are scarce.","The paper's claim that controlling more ACUs increases savings suggests an extrapolation to full-facility control across multiple rooms, but that scale-up remains unverified beyond the two rooms tested."],"forward_implications":["Replacing local PID control with the offline RL policy lowers the air-side cooling load factor by 14–21% in the tested rooms, with larger savings when more ACUs are under policy control.","The policy keeps cold-aisle temperatures below the operational threshold and yields a more concentrated hot-aisle temperature distribution, indicating safe and stable long-term operation.","The ablations show that removing the GNN structure or the T-symmetry enforcement degrades multi-step prediction and policy energy efficiency, identifying these two design choices as the source of the gains.","Because training uses only historical operational data and no simulator, the same framework should transfer to other data-limited industrial control settings with similar constraints."],"supporting_citations":[{"why":"Supplies the T-symmetry regularization for offline RL that the framework adapts to enforce physical consistency of policy-induced samples.","marker":"(Cheng et al., 2023)"},{"why":"Provides the graph convolutional blocks used in TTDM to encode spatial and control dependencies among sensors and ACUs.","marker":"(Kipf & Welling, 2017)"},{"why":"Provides the TD3+BC-style policy objective that the paper modifies by evaluating the Q-function in latent space.","marker":"(Fujimoto & Gu, 2021)"},{"why":"Supplies the latent ODE formulation used for the forward and reverse dynamics, encouraging parsimonious models.","marker":"(Champion et al., 2019)"},{"why":"Is the data-driven MPC baseline for DC cooling control compared on the testbed.","marker":"(Lazic et al., 2018)"},{"why":"Provides CCA, the off-policy RL baseline that competes in the testbed evaluation.","marker":"(Li et al., 2019)"},{"why":"Provides IQL, a mainstream offline RL baseline used for comparison.","marker":"(Kostrikov et al., 2022)"},{"why":"Provides CQL, a conservative offline RL baseline used for comparison.","marker":"(Kumar et al., 2020)"},{"why":"Provides FISOR, the safe offline RL baseline used for comparison.","marker":"(Zheng et al., 2024)"},{"why":"Documents the conventional PID approach that serves as the main field-test baseline.","marker":"(Durand-Estebe et al., 2013)"}],"fun_headline_variants":["Physics-informed offline RL saves 14–21% cooling energy in live DC","Time-reversal symmetry enables offline RL for data center cooling","Offline RL with physics priors cuts data center cooling energy 14–21%","AI cooling policy saves 14–21% energy in production data centers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The crux is that the days when the PID controller ran and the days when the new policy ran are assumed to be otherwise comparable, even though they fall in different months with possibly different weather and water-side conditions.","fun_headline_variants_meta":{"raw":{"variants":["Physics-informed offline RL saves 14–21% cooling energy in live DC","Time-reversal symmetry enables offline RL for data center cooling","Offline RL with physics priors cuts data center cooling energy 14–21%","AI cooling policy saves 14–21% energy in production data centers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000541,"raw_usage":{"total_tokens":2636,"prompt_tokens":1033,"completion_tokens":1603,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":1523}},"tokens_in":649,"tokens_out":1603,"duration_ms":10601,"temperature":1.0,"reasoning_tokens":1523,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:37:59.412854+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run both controllers in the same server room on alternating days within the same week, or hold outdoor temperature, humidity, and entering water temperature fixed (or regress them out), and check whether the 14–21% ACLF drop survives; if the gap shrinks to near zero, the claimed energy savings are an artifact of period differences.","supporting_citations":[{"cited_title":"We provide detailed descriptions of these baseline methods as follows","cited_arxiv_id":null,"evidence_quote":"Provides CQL, a conservative offline RL baseline used for comparison."}],"review_version":1}