{"id":"7a646e61-77e0-4770-8dc2-0ba52befea88","arxiv_id":"2412.08293","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Sinergym is an open-source, Gymnasium-compatible virtual testbed that standardizes building energy optimization experiments with 87 preconfigured environments, weather variability, and experiment tracking.","lead":"This paper introduces Sinergym, an open-source software testbed that links building energy simulators with reinforcement learning agents. It lets researchers train and compare AI controllers for building heating, cooling, and ventilation in simulated environments, supporting the move toward intelligent building operations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Weather-variability equation in §4.2 is not an Ornstein-Uhlenbeck process, so a headline reproducibility feature is documented incorrectly; conditional acceptance should require the code/formula fix.","rationale":"This is a software-infrastructure contribution, and the strongest evidence is the public repository, documentation, 87 predefined environments, and apparent community uptake; I credit that independent support. The reader's weakest_assumption about sim-to-real fidelity is a real limitation, but it is not the most load-bearing concern for this paper: the examples are framed as demonstrations, and no benchmark-paper standard requires validation against physical buildings for the central claim of providing a shared testbed. The load-bearing condition is that the environments behave as documented, because the claimed contribution is reproducibility and fair comparison. Eq. (1) fails that condition for the WeatherVar feature, a headline differentiator. The demonstration experiments' single-seed results are secondary because §5 says the goal is to show possibilities rather than to establish a best controller. The reader's verdict is already conditional, and my concern keeps it conditional: the concrete check will determine whether the issue is a code bug or a documentation typo, and the paper should not be used as a benchmark reference until the equation and implementation are reconciled.","tokens_in":18138,"tokens_out":6264,"duration_ms":67126,"concrete_test":"Set σ=0, μ=0, τ=1 and start from T0=20. Eq. (1) yields T1=19, T2=18, ... (a constant downward drift), whereas an OU process with zero mean and time constant 1 should return toward 0. Run this case in Sinergym's weather-variability code and compare the generated sequence to both Eq. (1) and the standard Euler OU update T_{t+1} = T_t + (μ - T_t) Δt/τ. Also inspect the source to see which formula is implemented. If the code matches Eq. (1), WeatherVar is broken as documented; if it matches the OU update, the paper's Eq. (1) must be corrected before the paper can serve as a reproducibility reference.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Sinergym removes reproducibility and comparability barriers for RL-based BEO. That claim loads on the correctness of the environment mechanics, not just on the availability of code. Section 4.2 presents Eq. (1) as the Ornstein-Uhlenbeck (OU) weather-noise model used by the WeatherVar feature, which Table 2 identifies as a differentiator relative to other frameworks. The printed equation, T_{t+1} = (1 - μ)T_t - τ + σ(W_{t+1} - W_t), is not an OU discretization. A standard Euler or exact OU update pulls T_t toward the mean μ; the printed deterministic part instead has fixed point -τ/μ and, for typical parameter settings, drives the temperature away from the recorded TMY3 value rather than fluctuating around it. Because only Sinergym offers WeatherVar among the compared frameworks, this is a load-bearing reproducibility defect: users who follow the paper to implement or audit stochastic-weather experiments will not be running the described process. The demonstration experiments in §5.3 explicitly rely on mixed weather with noise-added outdoor temperature, so the error affects the reported examples as well. The single-seed DRL runs are secondary because §5 frames them as illustrative; the equation error is primary because it misdescribes the environment itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Sinergym, an open-source Python testbed for RL-based building energy optimization (BEO) built on EnergyPlus and the Gymnasium API. It compares Sinergym with existing frameworks (RL Testbed, Energym, BOPTEST-Gym, CityLearn), describes its architecture and core features (87 predefined environments, weather variability, custom rewards, wrappers, controllers, experiment tracking), and illustrates usage with default, rule-based, and deep RL controllers. The central claim is that Sinergym addresses the setup, scale, reproducibility, and comparison barriers that hinder RL-based BEO, providing a shared benchmarking infrastructure.","tokens_in":18376,"tokens_out":7869,"duration_ms":81813,"significance":"If the implementation matches the corrected description, Sinergym would be a valuable community resource: it is open-source, uses a standard API, ships PyPI/Docker distributions, provides documentation, and its central functionality is independently checkable from the public repository. The paper also offers explicit comparison tables against other frameworks and supports experiment tracking through Weights & Biases and CSV loggers. However, the manuscript currently contains two formula-level inconsistencies, Eq. (1) in §4.2 and Eq. (2) in §4.4, that affect the documentation of the WeatherVar feature and the reward function, respectively; these need correction before the paper can serve as a reliable reference for the tool. The experimental demonstrations are intentionally illustrative and the authors say so, which mitigates concerns about the absence of multi-seed comparisons.","major_comments":[{"comment":"Equation (1) is not an Ornstein-Uhlenbeck (OU) process as stated. The deterministic part (1 - μ)T_t - τ has fixed point T = -τ/μ rather than fluctuating around the recorded TMY3 value, and the parameter τ is described as a time constant but appears in the equation as a constant offset; a standard Euler-Maruyama discretization of an OU process would read T_{t+1} = T_t + θ(μ - T_t)Δt + σ√(Δt) ε_t, or the corresponding exact form. Because WeatherVar is listed as a differentiator in Table 2 and is used in the §5 experiments (e.g., Eplus-datacenter-mixed-continuous-stochastic-v1), the printed formula must be corrected to match the implementation, and the parameter meanings and discretization scheme must be specified.","section":"§4.2, Eq. (1)"},{"comment":"The reward function in Eq. (2) contradicts the accompanying statement that the penalty is 0 inside the comfort range. For T_t ∈ [T_low, T_up], the expression (|T_t - T_up| + |T_t - T_low|) equals T_up - T_low, so the penalty term is (1 - ω)λ_T(T_up - T_low) > 0. The formula should use a distance to the interval, e.g., max(0, T_t - T_up) + max(0, T_low - T_t), or an equivalent hinge loss. Since all reported rewards and the reward-customization functionality depend on this definition, the equation and the implementation must be reconciled.","section":"§4.4, Eq. (2)"}],"minor_comments":[{"comment":"The caption contains a typo: 'uring' should be 'during'.","section":"§5.3, Figure 6 caption"},{"comment":"The text says 'di cussed' where 'discussed' is intended.","section":"§5.1"},{"comment":"The sentence referring to 'Figure 9a show' should be 'Figure 9a shows'.","section":"§5.5"},{"comment":"Section 5.1 reports the comfort range as [16, 27] ℃, while Appendix A, Table A.9 specifies the reward comfort range as (18.0, 27.0) ℃; these values should be aligned for reproducibility of the reported examples.","section":"§5.1 vs. Appendix A"},{"comment":"The paper states that policies can transition from virtual environments to real-world building operations, but no validation against measured building data is presented; please soften this claim or explicitly mark it as future work.","section":"§1 and §3.1"},{"comment":"The comparative statements about stability and relative performance of SAC, TD3, and PPO are based on a single training run per algorithm; please add multiple seeds with variance information or explicitly label the comparison as illustrative only.","section":"§5.3 and §5.5"},{"comment":"The description of 'lambda temperature' says it calibrates the energy reward component, but it should refer to the temperature/comfort component.","section":"Appendix A, Table A.9"}],"recommendation":"major_revision","confidential_remarks":"This is a software/tool contribution to a domain journal. The two formula errors are genuine and affect the reproducibility of the WeatherVar and reward-function features, but both are straightforward to fix, since the public code presumably contains the correct implementations. The experimental demonstration is thin (single seed, no hyperparameter sensitivity for the main comparisons), but the authors already frame it as illustrative; I would not require a full benchmark study for this paper. I recommend major revision and a check of the code against the corrected formulas before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a software-tool paper, and the software is real. Sinergym gives the building-RL community a solid Gymnasium-compatible wrapper around EnergyPlus with 87 predefined environments, dynamic action/observation spaces, custom rewards, wrappers, and logging. The comparison with RL Testbed, BOPTEST-Gym, Energym, and CityLearn is clear and honest. The repo is public, and the examples run. That is real value, and the paper deserves serious referee time.\n\nThe smartest thing in the paper is the weather-variability feature: adding noise to TMY3 temperature across episodes is a simple way to make trained policies less brittle. But the paper makes a load-bearing documentation error. Equation (1) in §4.2 is presented as the Ornstein-Uhlenbeck update, and it is not one. As written, T_{t+1} = (1-μ)T_t - τ + σ(W_{t+1}-W_t) has a deterministic fixed point at -τ/μ, and for the parameter values they show it drives temperature away from the recorded value rather than reverting to a mean. The surrounding text says the parameters are σ, μ, τ, but the equation does not match the text or any standard OU discretization. Since Table 2 lists WeatherVar as a differentiator, and Section 5.3 uses mixed weather with noise-added outdoor temperature, this is a primary reproducibility defect: a reader cannot implement or audit the feature from the paper. The authors presumably have correct code in the repo, but the paper as written is wrong.\n\nThe other soft spots are minor. The DRL experiments are single runs per algorithm, no seeds, no intervals, so the 15-18% savings are illustrative, which the paper mostly says. The sim-to-real claim in Section 1 is a goal, not a validated result; that is a limitation, not a flaw, for a testbed paper. The two self-citations as evidence of adoption are fine because the artifact is independently checkable.\n\nBottom line: the central claim — a functional, feature-rich testbed — holds up because the code is public. The equation needs to be corrected, and the experiments would benefit from uncertainty reporting. I'd accept it for peer review, and I'd cite it if I worked in building RL. Bring it to the reading group if you want to discuss what a benchmark infrastructure paper owes its readers.","headline":"Sinergym is a genuinely useful open-source RL testbed for building control, but the weather-noise equation in §4.2 is not an Ornstein-Uhlenbeck process and must be fixed before the paper is citable as a benchmark reference.","tokens_in":18966,"tokens_out":2631,"would_cite":true,"duration_ms":25701,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper presents Sinergym, an open-source EnergyPlus-based virtual testbed that gives reinforcement learning agents a standardized, reproducible way to train, evaluate, and compare HVAC control strategies across 87 building scenarios.","keywords":["Building energy optimization","Reinforcement learning","HVAC control","EnergyPlus","Simulation testbed","Reproducible benchmarking","Weather variability","Digital twin"],"falsifier":"Take a policy trained in Sinergym's data-center environment and apply it to an actual building, or to a measured high-fidelity replay of one, over a comparable season; then compare energy use and comfort violations against the same building under the default or rule-based controller. If the simulated 15-18% savings vanish or comfort is violated, the sim-to-real pathway that motivates the testbed is not established.","tokens_in":17919,"feed_emoji":"⚡","tokens_out":7407,"duration_ms":79738,"temperature":0.7,"pith_summary":"Sinergym is an open-source Python virtual testbed that lets reinforcement learning agents control simulated buildings through the same standard step/reset interface used across the RL field. The paper argues that this one tool addresses four problems that have blocked RL-based building energy optimization: the effort of wiring simulators into a control loop, the need to run and monitor thousands of training episodes, the difficulty of reproducing experiments, and the absence of a shared benchmark for comparing controllers. It bundles 87 predefined environments built from reference building models and weather files, with configurable actions, observations, rewards, and wrappers, plus logging for experiment tracking. If the claims hold, researchers gain a common infrastructure for training and fairly comparing data-driven HVAC controllers, a step toward using simulation as a digital twin for real building operations.","feed_headline":"RL agents get a shared testbed of 87 simulated buildings","feed_subtitle":"Open-source Sinergym wraps EnergyPlus in a standard RL loop, with weather noise and logging for fair HVAC comparisons.","key_machinery":"The load-bearing mechanism is the Environment class, a step/reset wrapper that takes control of the EnergyPlus engine through its Python API. It overrides the building's default schedulers during each control step, interrupts the simulation, writes the control signal, and resumes, turning a batch simulator into an interactive RL environment. Around that core sit the configuration machinery of building definition files, weather files, custom reward functions, nested wrappers, and stochastic weather-noise generation, which together make the 87 scenarios reproducible and modifiable.","core_discovery":"The paper's central claim is that Sinergym solves the setup, scale, reproducibility, and comparison problems in RL-based building energy optimization. It does so by wrapping the EnergyPlus simulation engine behind a standard reinforcement-learning interaction interface, so any controller written for that interface can drive a building without touching simulator internals. Underneath, the environment reads an EnergyPlus building definition file and a weather file, applies optional stochastic temperature noise for weather variability, and exposes configurable action spaces, observation spaces, reward functions, and wrappers. The included 87 environments combine four reference buildings with multiple climate regions, and the logging and benchmarking tooling records hyperparameters, metrics, and outputs. Demonstrations with default, rule-based, and deep-RL controllers on a data-center environment show learned policies saving 15-18% energy relative to reactive baselines, with further gains after hyperparameter tuning.","pith_inferences":["A shared 87-environment benchmark could let the building-control community publish standard leaderboards, in the same way established game and locomotion benchmarks did for reinforcement learning; the paper lists this as a medium-term goal rather than a demonstrated result.","Because every scenario is a combination of building file, weather file, action space, observation space, and reward, the testbed could be repurposed for adjacent problems such as demand response, fault detection, or multi-building coordination without new simulator plumbing.","The weather-noise mechanism suggests a testable robustness protocol: train under different noise levels and measure how policy performance degrades on held-out real weather years; the paper does not report such a transfer evaluation.","The clearest next experiment is sim-to-real: deploy a policy trained in the testbed on a real HVAC system and compare actual energy use and comfort against the simulated numbers, which would test the digital-twin premise the paper uses to motivate the tool."],"forward_implications":["Researchers can compare deep-RL HVAC controllers on identical predefined environments with logging and reproducible seeds, making reported results directly comparable.","New building models can be added by supplying an EnergyPlus building definition file and letting the tool detect sensors and actuators automatically, so the testbed can grow beyond the four base buildings.","Controllers trained with the included reward functions and wrappers can be evaluated against default and rule-based baselines out of the box, giving every new algorithm a fair reference point.","Weather variability through stochastic noise lets policies be tested under environmental perturbations, supporting claims about robustness.","Because the environments expose a standard RL interaction interface, existing RL training libraries and algorithms can be benchmarked without custom integration code."],"supporting_citations":[{"why":"Defines the standard step/reset interaction contract that Sinergym implements.","marker":"[32]"},{"why":"An earlier EnergyPlus RL environment whose limitations motivate the new design.","marker":"[38]"},{"why":"A Modelica-based control benchmarking framework that provides the comparison baseline for benchmarking features.","marker":"[39]"},{"why":"A building-model library that serves as the comparison baseline for weather and reward customization.","marker":"[40]"},{"why":"Introduces the testing-framework concept for simulation-based benchmarking of building control.","marker":"[7]"},{"why":"Supplies one of the reference building models included in the testbed.","marker":"[49]"},{"why":"Supplies the data-center building model used in the demonstration experiments.","marker":"[48]"},{"why":"Defines the stochastic process used for weather variability.","marker":"[52]"},{"why":"Provides the experiment-tracking platform used for logging and comparing runs.","marker":"[53]"}],"fun_headline_variants":["Open-source Sinergym puts 87 buildings on the RL grid","RL building control gets a shared 87-building playground","Sinergym: one RL loop for 87 simulated buildings","Open testbed wraps EnergyPlus in RL for 87 buildings","A shared RL gym for 87 building simulations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim depends on reference building models and typical-year weather files standing in for real buildings accurately enough that a control policy trained in the virtual testbed can transfer to actual operations; the paper does not validate this against measured building data.","fun_headline_variants_meta":{"raw":{"variants":["Open-source Sinergym puts 87 buildings on the RL grid","RL building control gets a shared 87-building playground","Sinergym: one RL loop for 87 simulated buildings","Open testbed wraps EnergyPlus in RL for 87 buildings","A shared RL gym for 87 building simulations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001285,"raw_usage":{"total_tokens":5244,"prompt_tokens":933,"completion_tokens":4311,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":4229}},"tokens_in":549,"tokens_out":4311,"duration_ms":30569,"temperature":1.0,"reasoning_tokens":4229,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:58:55.511575+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a policy trained in Sinergym's data-center environment and apply it to an actual building, or to a measured high-fidelity replay of one, over a comparable season; then compare energy use and comfort violations against the same building under the default or rule-based controller. If the simulated 15-18% savings vanish or comfort is violated, the sim-to-real pathway that motivates the testbed is not established.","supporting_citations":[{"cited_title":"Moriyama, G","cited_arxiv_id":null,"evidence_quote":"An earlier EnergyPlus RL environment whose limitations motivate the new design."},{"cited_title":"Arroyo, C","cited_arxiv_id":null,"evidence_quote":"A Modelica-based control benchmarking framework that provides the comparison baseline for benchmarking features."},{"cited_title":"Scharnhorst, B","cited_arxiv_id":null,"evidence_quote":"A building-model library that serves as the comparison baseline for weather and reward customization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies one of the reference building models included in the testbed."},{"cited_title":"Pinto, D","cited_arxiv_id":null,"evidence_quote":"Supplies the data-center building model used in the demonstration experiments."},{"cited_title":"Z´ arate-Mi˜ nano, M","cited_arxiv_id":null,"evidence_quote":"Defines the stochastic process used for weather variability."},{"cited_title":"Biewald, Experiment tracking with weights and biases, software available from wandb.com (2020)","cited_arxiv_id":null,"evidence_quote":"Provides the experiment-tracking platform used for logging and comparing runs."}],"review_version":1}