Pith. sign in

REVIEW 2 major objections 7 minor 56 references

SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning

T0 review · 2 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper presents Sinergym, an open-source EnergyPlus-based virtual testbed that gives reinforcement learning agents a standardized, reproducible way to train, evaluate, and compare HVAC control strategies across 87 building scenarios.

desk verdict Sinergym is a genuinely useful open-source RL testbed for building control, but the weather-noise equation in §4.2 is not an Ornstein-Uhlenbeck process and must be fixed before the paper is citable as a benchmark reference. read the letter →

arxiv 2412.08293 v1 pith:J6PBESEY submitted 2024-12-11 cs.LG cs.SYeess.SY

Alejandro Campoy-Nieves , Antonio Manjavacas , Javier Jiménez-Raboso , Miguel Molina-Solana , Juan Gómez-Romero This is my paper · ORCID
classification cs.LGcs.SYeess.SY
keywords BuildingenergyoptimizationReinforcementlearningHVACcontrolPlusSimulationtestbedReproduciblebenchmarkingWeathervariabilityDigitaltwin
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sinergym is an open-source Python virtual testbed that lets reinforcement learning agents control simulated buildings through the same standard step/reset interface used across the RL field. The paper argues that this one tool addresses four problems that have blocked RL-based building energy optimization: the effort of wiring simulators into a control loop, the need to run and monitor thousands of training episodes, the difficulty of reproducing experiments, and the absence of a shared benchmark for comparing controllers. It bundles 87 predefined environments built from reference building models and weather files, with configurable actions, observations, rewards, and wrappers, plus logging for experiment tracking. If the claims hold, researchers gain a common infrastructure for training and fairly comparing data-driven HVAC controllers, a step toward using simulation as a digital twin for real building operations.

What carries the argument

The load-bearing mechanism is the Environment class, a step/reset wrapper that takes control of the EnergyPlus engine through its Python API. It overrides the building's default schedulers during each control step, interrupts the simulation, writes the control signal, and resumes, turning a batch simulator into an interactive RL environment. Around that core sit the configuration machinery of building definition files, weather files, custom reward functions, nested wrappers, and stochastic weather-noise generation, which together make the 87 scenarios reproducible and modifiable.

What would settle it

Take a policy trained in Sinergym's data-center environment and apply it to an actual building, or to a measured high-fidelity replay of one, over a comparable season; then compare energy use and comfort violations against the same building under the default or rule-based controller. If the simulated 15-18% savings vanish or comfort is violated, the sim-to-real pathway that motivates the testbed is not established.

Watch

Extended reading notes

Core claim

The paper's central claim is that Sinergym solves the setup, scale, reproducibility, and comparison problems in RL-based building energy optimization. It does so by wrapping the EnergyPlus simulation engine behind a standard reinforcement-learning interaction interface, so any controller written for that interface can drive a building without touching simulator internals. Underneath, the environment reads an EnergyPlus building definition file and a weather file, applies optional stochastic temperature noise for weather variability, and exposes configurable action spaces, observation spaces, reward functions, and wrappers. The included 87 environments combine four reference buildings with multiple climate regions, and the logging and benchmarking tooling records hyperparameters, metrics, and outputs. Demonstrations with default, rule-based, and deep-RL controllers on a data-center environment show learned policies saving 15-18% energy relative to reactive baselines, with further gains after hyperparameter tuning.

Load-bearing premise

The claim depends on reference building models and typical-year weather files standing in for real buildings accurately enough that a control policy trained in the virtual testbed can transfer to actual operations; the paper does not validate this against measured building data.

Editorial extensions

If this is right

  • Researchers can compare deep-RL HVAC controllers on identical predefined environments with logging and reproducible seeds, making reported results directly comparable.
  • New building models can be added by supplying an EnergyPlus building definition file and letting the tool detect sensors and actuators automatically, so the testbed can grow beyond the four base buildings.
  • Controllers trained with the included reward functions and wrappers can be evaluated against default and rule-based baselines out of the box, giving every new algorithm a fair reference point.
  • Weather variability through stochastic noise lets policies be tested under environmental perturbations, supporting claims about robustness.
  • Because the environments expose a standard RL interaction interface, existing RL training libraries and algorithms can be benchmarked without custom integration code.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A shared 87-environment benchmark could let the building-control community publish standard leaderboards, in the same way established game and locomotion benchmarks did for reinforcement learning; the paper lists this as a medium-term goal rather than a demonstrated result.
  • Because every scenario is a combination of building file, weather file, action space, observation space, and reward, the testbed could be repurposed for adjacent problems such as demand response, fault detection, or multi-building coordination without new simulator plumbing.
  • The weather-noise mechanism suggests a testable robustness protocol: train under different noise levels and measure how policy performance degrades on held-out real weather years; the paper does not report such a transfer evaluation.
  • The clearest next experiment is sim-to-real: deploy a policy trained in the testbed on a real HVAC system and compare actual energy use and comfort against the simulated numbers, which would test the digital-twin premise the paper uses to motivate the tool.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper presents Sinergym, an open-source Python testbed for RL-based building energy optimization (BEO) built on EnergyPlus and the Gymnasium API. It compares Sinergym with existing frameworks (RL Testbed, Energym, BOPTEST-Gym, CityLearn), describes its architecture and core features (87 predefined environments, weather variability, custom rewards, wrappers, controllers, experiment tracking), and illustrates usage with default, rule-based, and deep RL controllers. The central claim is that Sinergym addresses the setup, scale, reproducibility, and comparison barriers that hinder RL-based BEO, providing a shared benchmarking infrastructure.

Significance. If the implementation matches the corrected description, Sinergym would be a valuable community resource: it is open-source, uses a standard API, ships PyPI/Docker distributions, provides documentation, and its central functionality is independently checkable from the public repository. The paper also offers explicit comparison tables against other frameworks and supports experiment tracking through Weights & Biases and CSV loggers. However, the manuscript currently contains two formula-level inconsistencies, Eq. (1) in §4.2 and Eq. (2) in §4.4, that affect the documentation of the WeatherVar feature and the reward function, respectively; these need correction before the paper can serve as a reliable reference for the tool. The experimental demonstrations are intentionally illustrative and the authors say so, which mitigates concerns about the absence of multi-seed comparisons.

major comments (2)
  1. [§4.2, Eq. (1)] Equation (1) is not an Ornstein-Uhlenbeck (OU) process as stated. The deterministic part (1 - μ)T_t - τ has fixed point T = -τ/μ rather than fluctuating around the recorded TMY3 value, and the parameter τ is described as a time constant but appears in the equation as a constant offset; a standard Euler-Maruyama discretization of an OU process would read T_{t+1} = T_t + θ(μ - T_t)Δt + σ√(Δt) ε_t, or the corresponding exact form. Because WeatherVar is listed as a differentiator in Table 2 and is used in the §5 experiments (e.g., Eplus-datacenter-mixed-continuous-stochastic-v1), the printed formula must be corrected to match the implementation, and the parameter meanings and discretization scheme must be specified.
  2. [§4.4, Eq. (2)] The reward function in Eq. (2) contradicts the accompanying statement that the penalty is 0 inside the comfort range. For T_t ∈ [T_low, T_up], the expression (|T_t - T_up| + |T_t - T_low|) equals T_up - T_low, so the penalty term is (1 - ω)λ_T(T_up - T_low) > 0. The formula should use a distance to the interval, e.g., max(0, T_t - T_up) + max(0, T_low - T_t), or an equivalent hinge loss. Since all reported rewards and the reward-customization functionality depend on this definition, the equation and the implementation must be reconciled.
minor comments (7)
  1. [§5.3, Figure 6 caption] The caption contains a typo: 'uring' should be 'during'.
  2. [§5.1] The text says 'di cussed' where 'discussed' is intended.
  3. [§5.5] The sentence referring to 'Figure 9a show' should be 'Figure 9a shows'.
  4. [§5.1 vs. Appendix A] Section 5.1 reports the comfort range as [16, 27] ℃, while Appendix A, Table A.9 specifies the reward comfort range as (18.0, 27.0) ℃; these values should be aligned for reproducibility of the reported examples.
  5. [§1 and §3.1] The paper states that policies can transition from virtual environments to real-world building operations, but no validation against measured building data is presented; please soften this claim or explicitly mark it as future work.
  6. [§5.3 and §5.5] The comparative statements about stability and relative performance of SAC, TD3, and PPO are based on a single training run per algorithm; please add multiple seeds with variance information or explicitly label the comparison as illustrative only.
  7. [Appendix A, Table A.9] The description of 'lambda temperature' says it calibrates the energy reward component, but it should refer to the temperature/comfort component.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Sinergym's central claims are software-functionality claims independently verifiable from the public repository; the demonstrative RL experiments are illustrations, not fitted predictions.

full rationale

The paper's central claims are that Sinergym provides an open-source Gymnasium-compatible testbed with 87 predefined environments, weather variability, custom rewards, wrappers, and experiment tracking. These are implementation claims whose truth is checkable against the public GitHub repository and the installed package, not derived from the paper's own prior results. The citations to [22], [23], [28], and [29] in Section 1 are adoption/examples-of-use citations; they do not serve as premises for any conclusion about Sinergym's functionality, and the fact that some are self-citations is not load-bearing because the feature set is documented and testable independently. Section 5's RL experiments are explicitly illustrative ('the main objective of these examples is not to find the best controller... but to show the possibilities of the tool'), so the reported energy savings are not presented as an independent prediction forced by a fitted parameter. Equation (1) in Section 4.2 describes the WeatherVar noise mechanism; while it may be a correctness/documentation defect (the printed recurrence is not a standard Ornstein-Uhlenbeck discretization), a mislabeled formula is not a circular derivation: no claim is reduced to its own input. The reward function (Eq. 2) and environment configurations are explicitly user-defined inputs, and the paper never 'predicts' a quantity from data fitted to that same quantity. Hence no circular step can be exhibited under the standards of this review.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

All entries listed affect the demonstration experiments; only the last two axioms touch the core value of the testbed. The reward parameters are user configs, not fitted, so they do not make the demo results circular.

free parameters (6)
  • energy weight ω = 0.5
    Hand-chosen weight in the default linear reward (Eq. 2) balancing power cost against comfort penalty; used in all demo environments, directly shapes the learned policies and the reported savings.
  • energy scaling λ_P = 0.00005
    Hand-chosen scaling constant for the power term in the reward, set to bring energy values into a comparable range; affects the magnitude of the reward and thus the policy trade-off.
  • temperature scaling λ_T = 1.0
    Hand-chosen scaling for the comfort violation term; together with λ_P sets the relative importance of comfort in the reward.
  • comfort range (T_low, T_up) = (18.0, 27.0) °C
    ASHRAE-recommended data center comfort range used in the reward; defines the zero-penalty zone and therefore influences what the controllers optimize.
  • OU weather noise parameters (σ, μ, τ) = not specified in paper text
    Parameters of the Ornstein-Uhlenbeck process for weather variability (§4.2); values are chosen in environment configuration but not given in the paper, which limits reproducibility of the stochastic weather feature.
  • observation normalization range = -5e6 to 5e6
    Hard-coded min and max used by the normalizing wrapper in the demo (Table A.7); arbitrary and not physically meaningful, but they affect the scaled observations seen by the RL agents.
assumptions (6)
  • domain assumption EnergyPlus is a valid and accurate building simulation engine for evaluating HVAC control strategies.
    The entire testbed is built on EnergyPlus (§3.2); if EnergyPlus misrepresents building thermodynamics, any policy trained in Sinergym may not reflect real building behavior.
  • domain assumption The ASHRAE and DOE reference building models and TMY3 weather files are representative of real building stock and climate.
    The 87 environments are combinations of these models and weather files (§4.1, §4.2); the usefulness of the benchmark depends on this representativeness.
  • domain assumption The Gymnasium interface is an appropriate abstraction for continuous building control.
    Sinergym's design assumes the step and reset observation-action loop of Gymnasium maps naturally onto building control (§3.2); this is a community convention rather than an established physical truth.
  • domain assumption Weather variability simulated with Ornstein-Uhlenbeck noise improves robustness of controllers.
    Section 4.2 introduces weather noise to make training robust across conditions; the paper does not validate that this noise model reproduces real weather variability.
  • domain assumption Reinforcement learning agents trained in simulation can be transferred to real buildings.
    The paper's motivating goal is 'transition from virtual environments to real-world building operations' (Section 1, Section 6), but no sim-to-real validation is provided.
  • domain assumption The reward function formulation (linear combination of energy and comfort penalties) adequately captures the control objective.
    Section 4.4 defines the default reward; the entire evaluation and controller comparison rests on this scalarization being a valid objective for building energy optimization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning." pith.science (2026). https://pith.science/paper/J6PBESEY

@misc{pith2026241208293,
  author       = {Pith},
  title        = {Pith review of: SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J6PBESEY}},
  note         = {Machine review of arXiv:2412.08293}
}
read the original abstract

Simulation has become a crucial tool for Building Energy Optimization (BEO) as it enables the evaluation of different design and control strategies at a low cost. Machine Learning (ML) algorithms can leverage large-scale simulations to learn optimal control from vast amounts of data without supervision, particularly under the Reinforcement Learning (RL) paradigm. Unfortunately, the lack of open and standardized tools has hindered the widespread application of ML and RL to BEO. To address this issue, this paper presents Sinergym, an open-source Python-based virtual testbed for large-scale building simulation, data collection, continuous control, and experiment monitoring. Sinergym provides a consistent interface for training and running controllers, predefined benchmarks, experiment visualization and replication support, and comprehensive documentation in a ready-to-use software library. This paper 1) highlights the main features of Sinergym in comparison to other existing frameworks, 2) describes its basic usage, and 3) demonstrates its applicability for RL-based BEO through several representative examples. By integrating simulation, data, and control, Sinergym supports the development of intelligent, data-driven applications for more efficient and responsive building operations, aligning with the objectives of digital twin technology.

Figures

Figures reproduced from arXiv: 2412.08293 by the authors.

Figure 1
Figure 1. Sinergym’s general architecture, depicting its three main layers: communication, middleware and simulator simulator is launched through the EnergyPlus Python API. This step also involves the creation of handlers for the simulator, which are used to manage and communicate with the underlying engine. Now, the simulator is ready to be launched and the first episode can start after calling reset. Before starting each ep… view at source ↗
Figure 2
Figure 2. Overview of the Sinergym workflow the model components, which are programmatically defined when the environment is created and internally materialized. Furthermore, Sinergym offers additional functionalities to facilitate the integration of new building models, such as the automatic detection of sensors and actuators in epJSON files, and the subsequent preparation of these files to be accessed through the Gymnasium … view at source ↗
Figure 3
Figure 3. Ornstein-Uhlenbeck noise applied to a mixed weather dataset (New York) with different values for [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Example of intermediate values computed by the reward function [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Indoor air temperatures achieved by each controller for a 1-year evaluation period in [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Mean reward of DRL agents uring training [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Mean reward obtained by Default, RBC, SAC, TD3, PPO and optimized PPO controllers during evaluation (10 [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Monthly average power demand for each controller [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Energy savings achieved by each DRL controller during evaluation. Darker bars represent DRL versus the default [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 30 canonical work pages

  1. [1]

    P´ erez-Lombard, J

    L. P´ erez-Lombard, J. Ortiz, C. Pout, A review on buildings energy consumption information, Energy and Buildings 40 (3) (2008) 394–398. doi:10.1016/j.enbuild.2007.03.007

    work page doi:10.1016/j.enbuild.2007.03.007 2008

  2. [2]

    C. Wang, K. Pattawi, H. Lee, Energy saving impact of occupancy-driven thermostat for residential buildings, Energy and Buildings 211 (2020) 109791. doi:10.1016/j.enbuild.2020.109791

    arXiv 2020

  3. [3]

    V. J. Mawson, B. R. Hughes, Optimisation of HV AC control and manufacturing schedules for the reduction of peak energy demand in the manufacturing sector, Energy 227 (2021) 120436. doi:10.1016/j.energy.2021.120436

    work page arXiv 2021

  4. [4]

    Gholamzadehmir, C

    M. Gholamzadehmir, C. Del Pero, S. Buffa, R. Fedrizzi, N. Aste, Adaptive-predictive control strategy for HV AC systems in smart buildings – A review, Sustainable Cities and Society 63 (2020) 102480. doi:10.1016/j.scs.2020.102480

    arXiv 2020

  5. [5]

    Rolnick, P

    D. Rolnick, P. L. Donti, L. H. Kaack, K. Kochanski, A. Lacoste, K. Sankaran, A. S. Ross, N. Milojevic-Dupont, N. Jaques, A. Waldman-Brown, A. S. Luccioni, T. Maharaj, E. D. Sherwin, S. K. Mukkavilli, K. P. Kording, C. P. Gomes, A. Y. Ng, D. Hassabis, J. C. Platt, F. Creutzig, J. Chayes, Y. Bengio, Tackling climate change with machine learning, ACM Computi...

    doi:10.1145/3485128 2022

  6. [6]

    Findeis, F

    A. Findeis, F. Kazhamiaka, S. Jeen, S. Keshav, Beobench: A toolkit for unified access to building simulations for reinforce- ment learning, in: Procs. Thirteenth ACM International Conference on Future Energy Systems, e-Energy ’22, Association for Computing Machinery, New York, NY, USA, 2022, pp. 374—-382. doi:10.1145/3538637.3538866

    arXiv 2022

  7. [7]

    D. Blum, J. Arroyo, S. Huang, J. Drgoˇ na, F. Jorissen, H. T. Walnum, Y. Chen, K. Benne, D. Vrabie, M. Wetter, L. Helsen, Building optimization testing framework (boptest) for simulation-based benchmarking of control strategies in buildings, Journal of Building Performance Simulation 14 (5) (2021) 586–610. doi:10.1080/19401493.2021.1986574

    arXiv 2021

  8. [8]

    Sutton, A

    R. Sutton, A. Barto, Reinforcement Learning, second edition: An Introduction, Adaptive Computation and Machine Learning series, MIT Press, 2018

    work page 2018

Show all 56 references
  1. [9]

    H. Dong, Z. Ding, S. Zhang, H. Yuan, H. Zhang, J. Zhang, Y. Huang, T. Yu, H. Zhang, R. Huang, Deep Reinforcement Learning: Fundamentals, Research, and Applications, Springer Nature, 2020. doi:10.1007/978-981-15-4095-0 . URL http://www.deepreinforcementlearningbook.org

    2020 doi

  2. [10]

    T. Wei, Y. Wang, Q. Zhu, Deep Reinforcement Learning for Building HV AC Control, in: Proceedings of the 54th Annual Design Automation Conference 2017, DAC ’17, Association for Computing Machinery, New York, NY, USA, 2017, pp. 1–6. doi:10.1145/3061639.3062224. URL https://githu...

    2017

  3. [11]

    Mason, S

    K. Mason, S. Grijalva, A review of reinforcement learning for autonomous building energy management, Computers & Electrical Engineering 78 (2019) 300–312. doi:10.1016/j.compeleceng.2019.07.019

    2019 doi

  4. [12]

    Zhang, X

    D. Zhang, X. Han, C. Deng, Review on the research and practice of deep learning and reinforcement learning in smart grids, CSEE Journal of Power and Energy Systems 4 (3) (2018) 362–370. doi:10.17775/CSEEJPES.2018.00520

    2018

  5. [13]

    J. R. V´ azquez-Canteli, Z. Nagy, Reinforcement learning for demand response: A review of algorithms and modeling techniques, Applied Energy 235 (2019) 1072–1089. doi:10.1016/j.apenergy.2018.11.002

    2019 doi

  6. [14]

    Brandi, M

    S. Brandi, M. S. Piscitelli, M. Martellacci, A. Capozzoli, Deep reinforcement learning to optimise indoor temperature control and heating energy consumption in buildings, Energy and Buildings 224 (2020) 110225. doi:10.1016/j.enbuild. 2020.110225

    2020

  7. [15]

    Azuatalam, W.-L

    D. Azuatalam, W.-L. Lee, F. de Nijs, A. Liebman, Reinforcement learning for whole-building HV AC control and demand response, Energy and AI 2 (2020) 100020. doi:10.1016/j.egyai.2020.100020

    2020

  8. [16]

    L. Yu, S. Qin, M. Zhang, C. Shen, T. Jiang, X. Guan, A Review of Deep Reinforcement Learning for Smart Building Energy Management, IEEE Internet of Things Journal 8 (15) (2021) 12046–12063. doi:10.1109/JIOT.2021.3078462

    2021

  9. [17]

    A. T. D. Perera, P. Kamalaruban, Applications of reinforcement learning in energy systems, Renewable and Sustainable Energy Reviews 137 (2021) 110618. doi:10.1016/j.rser.2020.110618

    2021

  10. [18]

    Q. Fu, Z. Han, J. Chen, Y. Lu, H. Wu, Y. Wang, Applications of reinforcement learning for building energy efficiency control: A review, Journal of Building Engineering 50 (2022) 104165. doi:10.1016/j.jobe.2022.104165

    2022

  11. [19]

    X. Deng, Y. Zhang, Y. Zhang, H. Qi, Towards optimal HV AC control in non-stationary building environments combining active change detection and deep reinforcement learning, Building and Environment 211 (2022) 108680. doi:10.1016/j. buildenv.2021.108680

    2022

  12. [20]

    M. H. B. Mahbod, C. B. Chng, P. S. Lee, C. K. Chui, Energy saving evaluation of an energy efficient data center using a model-free reinforcement learning approach, Applied Energy 322 (2022) 119392. doi:10.1016/j.apenergy.2022.119392

    2022

  13. [21]

    Z. Nagy, G. Henze, S. Dey, J. Arroyo, L. Helsen, X. Zhang, B. Chen, K. Amasyali, K. Kurte, A. Zamzam, H. Zandi, J. Drgoˇ na, M. Quintana, S. McCullogh, J. Y. Park, H. Li, T. Hong, S. Brandi, G. Pinto, A. Capozzoli, D. Vrabie, M. Berg´ es, K. Nweye, T. Marzullo, A. Bernstein, T...

    2023

  14. [22]

    M. Wang, J. Willes, T. Jiralerspong, M. Moezzi, A comparison of classical and deep reinforcement learning methods for hvac control (2023). doi:10.48550/arXiv.2308.05711

    2023 doi arXiv

  15. [23]

    Manjavacas, A

    A. Manjavacas, A. Campoy Nieves, J. Jim´ enez-Raboso, M. Molina-Solana, J. G´ omez-Romero, An experimental evaluation of deep reinforcement learning algorithms for hvac control, Artificial Intelligence Review 57 (06 2024). doi:10.1007/ s10462-024-10819-x

    2024

  16. [24]

    Dmitrewski, M

    A. Dmitrewski, M. Molina-Solana, R. Arcucci, Cntrlda: A building energy management control system with real-time adjustments. application to indoor temperature, Building and Environment 215 (2022) 108938. doi:10.1016/j.buildenv. 2022.108938

    2022

  17. [26]

    H.-Y. Liu, X. Fu, B. Balaji, R. Gupta, D. Hong, B2RL: An Open-Source Dataset for Building Batch Reinforcement Learning, in: Proceedings of the 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, BuildSys ’22, Association for ...

    2022

  18. [27]

    A. Naug, A. Guillen, R. Luna Guti´ errez, V. Gundecha, S. Ghorbanpour, L. Dheeraj Kashyap, D. Markovikj, L. Krause, S. Mousavi, A. R. Babu, S. Sarkar, PyDCM: Custom Data Center Models with Reinforcement Learning for Sustainabil- ity, in: Proceedings of the 10th ACM Internation...

    2023

  19. [28]

    Jim´ enez-Raboso, A

    J. Jim´ enez-Raboso, A. Manjavacas, A. Campoy-Nieves, M. Molina-Solana, J. G´ omez-Romero, Explaining deep reinforce- ment learning-based methods for control of building hvac systems, in: L. Longo (Ed.), Explainable Artificial Intelligence, Springer Nature Switzerland, Cham, 2...

    2023

  20. [29]

    URL http://arxiv.org/abs/2405.00389

    Employing federated learning for training autonomous HV AC systems. URL http://arxiv.org/abs/2405.00389

    arXiv

  21. [30]

    W¨ olfle, S

    D. W¨ olfle, S. R. Vossberg, H. Schmeck, TBMEnv - An environment to assess the accuracy of thermal building models under realistic conditions, in: Proceedings of the 10th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, BuildS...

    2023

  22. [31]

    Jang, Active reinforcement learning for robust building control, Master’s thesis, EECS Department, University of California, Berkeley (May 2023)

    A. Jang, Active reinforcement learning for robust building control, Master’s thesis, EECS Department, University of California, Berkeley (May 2023). URL http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-101.html

    2023

  23. [32]

    Kwiatkowski, M

    A. Kwiatkowski, M. Towers, J. Terry, J. U. Balis, G. D. Cola, T. Deleu, M. Goul˜ ao, A. Kallinteris, M. Krimmel, A. KG, R. Perez-Vicente, A. Pierr´ e, S. Schulhoff, J. J. Tai, H. Tan, O. G. Younis, Gymnasium: A standard interface for reinforce- ment learning environments (2024...

    2024 doi arXiv

  24. [33]

    Brockman, V

    G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, W. Zaremba, OpenAI Gym, arXiv:1606.01540 [cs] (Jun. 2016). doi:10.48550/arXiv.1606.01540

    2016 doi arXiv

  25. [34]

    Raffin, A

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, N. Dormann, Stable-baselines3: Reliable reinforcement learning implementations, Journal of Machine Learning Research 22 (268) (2021) 1–8. URL https://stable-baselines3.readthedocs.io/en/master/

    2021

  26. [35]

    Liang, R

    E. Liang, R. Liaw, R. Nishihara, P. Moritz, R. Fox, K. Goldberg, J. Gonzalez, M. Jordan, I. Stoica, RLlib: Abstractions for Distributed Reinforcement Learning, in: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Vol. 80 of Proceedings of Machin...

    2018

  27. [36]

    Z. Wang, T. Hong, Reinforcement learning for building controls: The opportunities and challenges, Applied Energy 269 (2020) 115036. doi:10.1016/j.apenergy.2020.115036

    2020

  28. [37]

    Biemann, F

    M. Biemann, F. Scheller, X. Liu, L. Huang, Experimental evaluation of model-free reinforcement learning algorithms for continuous HV AC control, Applied Energy 298 (2021) 117164. doi:10.1016/j.apenergy.2021.117164

    2021

  29. [38]

    Moriyama, G

    T. Moriyama, G. De Magistris, M. Tatsubori, T.-H. Pham, A. Munawar, R. Tachibana, Reinforcement Learning Testbed for Power-Consumption Optimization, in: L. Li, K. Hasegawa, S. Tanaka (Eds.), Methods and Applications for Modeling and Simulation of Complex Systems, Communication...

    2018 doi

  30. [39]

    Arroyo, C

    J. Arroyo, C. Manna, F. Spiessens, L. Helsen, An OpenAI-Gym Environment for the Building Optimization Testing (BOPTEST) Framework, in: Proceedings of the 17th IBPSA Conference, 2021, p. 9

    2021

  31. [40]

    Scharnhorst, B

    P. Scharnhorst, B. Schubnel, C. Fern´ andez-Bandera, J. Salom, P. Taddeo, M. Boegli, T. Gorecki, Y. Stauffer, A. Peppas, C. Politi, Energym: A Building Model Library for Controller Benchmarking, Applied Sciences 11 (8) (2021) 3518. doi: 10.3390/app11083518. URL https://github....

    2021 doi

  32. [41]

    Lukianykhin, T

    O. Lukianykhin, T. Bogodorova, ModelicaGym: applying reinforcement learning to Modelica models, in: Proceedings of the 9th International Workshop on Equation-based Object-oriented Modeling Languages and Tools, EOOLT’19, Associa- tion for Computing Machinery, New York, NY, USA,...

    2020

  33. [42]

    W¨ olfle, A

    D. W¨ olfle, A. Vishwanath, H. Schmeck, A Guide for the Design of Benchmark Environments for Building Energy Optimization, in: Procs. 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, BuildSys ’20, Association for Computing...

    2020

  34. [43]

    Zhang, O

    T. Zhang, O. Ardakanian, Cobs: Comprehensive building simulator, in: Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, BuildSys ’20, Association for Computing Machinery, New York, NY, USA, 2020, pp. 314––...

    2020

  35. [44]

    Nweye, K

    K. Nweye, K. Kaspar, G. Buscemi, T. Fonseca, G. Pinto, D. Ghose, S. Duddukuru, P. Pratapa, H. Li, J. Mohammadi, L. L. Ferreira, T. Hong, M. Ouf, A. Capozzoli, Z. Nagy, CityLearn v2: energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive c...

    2024

  36. [45]

    Pigott, C

    A. Pigott, C. Crozier, K. Baker, Z. Nagy, GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management, arXiv:2110.06396 [cs] (Oct. 2021). doi:10.48550/arXiv.2110.06396. URL https://github.com/apigott/GridLearn

    2021 doi arXiv

  37. [46]

    Marot, B

    A. Marot, B. Donnot, G. Dulac-Arnold, A. Kelly, A. O’Sullivan, J. Viebahn, M. Awad, I. Guyon, P. Panciatici, C. Romero, Learning to run a Power Network Challenge: a Retrospective Analysis, in: Proceedings of the NeurIPS 2020 Competition and Demonstration Track, PMLR, 2021, pp....

    2020

  38. [47]

    Zhang, O

    C. Zhang, O. Vinyals, R. Munos, S. Bengio, A study on overfitting in deep reinforcement learning (2018). doi:10.48550/ ARXIV.1804.06893

    2018 doi arXiv

  39. [48]

    Pinto, D

    G. Pinto, D. Deltetto, A. Capozzoli, Data-driven district energy management with surrogate models and deep reinforcement learning, Applied Energy 304 (2021) 117642. doi:10.1016/j.apenergy.2021.117642

    2021

  40. [49]

    X. Ding, W. Du, A. E. Cerpa, MB2C: Model-Based Deep Reinforcement Learning for Multi-zone Building Control, in: Pro- ceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, BuildSys ’20, Association for Computing ...

    2020

  41. [50]

    Y. Li, Y. Wen, D. Tao, K. Guan, Transforming cooling optimization for green data center via deep reinforcement learning, IEEE Transactions on Cybernetics 50 (5) (2020) 2002–2013. doi:10.1109/TCYB.2019.2927410

    2020

  42. [51]

    H. Cho, B. Liu, K. Gowri, Energy saving impact of ASHRAE 90.1 Vestibule requirements: Modeling of air infiltration through door openings, Tech. Rep. PNNL-20026, Pacific Northwest National Lab, Richland, W A (United States) (2010)

    2010

  43. [52]

    Z´ arate-Mi˜ nano, M

    R. Z´ arate-Mi˜ nano, M. Anghel, F. Milano, Continuous wind speed models based on stochastic differential equations, Applied Energy 104 (2013) 42–49. doi:10.1016/j.apenergy.2012.10.064

    2013 doi

  44. [53]

    Biewald, Experiment tracking with weights and biases, software available from wandb.com (2020)

    L. Biewald, Experiment tracking with weights and biases, software available from wandb.com (2020). URL https://www.wandb.com/

    2020

  45. [54]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, in: J. G. Dy, A. Krause (Eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Vol. 80 of Proceeding...

    2018

  46. [55]

    Fujimoto, H

    S. Fujimoto, H. van Hoof, D. Meger, Addressing function approximation error in actor-critic methods, in: J. G. Dy, A. Krause (Eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Vol. 80 of Proceed- ings of Machine Learning Research, PMLR, St...

    2018

  47. [56]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal Policy Optimization Algorithms, arXiv:1707.06347 [cs] (Aug. 2017). doi:10.48550/arXiv.1707.06347

    2017 doi arXiv

  48. [57]

    ASHRAE, ASHRAE TC9.9: Data Center Power Equipment Thermal Guidelines and Best Practices systems, ASHRAE (2016). 26

    2016

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.