Pith. sign in

REVIEW 5 major objections 5 minor 28 references

BuildEvo: Designing Building Energy Consumption Forecasting Heuristics via LLM-driven Evolution

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BuildEvo claims that an LLM-driven evolutionary loop can write transparent, physically grounded building-energy forecasting heuristics that beat neural baselines on the Building Data Genome Project 2 benchmark.

desk verdict A plausible PIFL extension of ReEvo to building energy forecasting, but the SOTA claim is unsupported by a single no-error-bar table and an unpolished manuscript. read the letter →

arxiv 2507.12207 v1 pith:N47UG7PF submitted 2025-07-16 cs.AI cs.NE

Subin Lin , Chuanbo Hua This is my paper
classification cs.AIcs.NE
keywords buildingenergyforecastingLLM-drivenevolutionevolutionaryalgorithmheuristicsearchinterpretablephysicalinsightsDataGenomeProject2time-series
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BuildEvo is an attempt to close the gap between handcrafted forecasting rules, which are transparent but imprecise, and neural networks, which are accurate but opaque. The paper claims that a large language model placed inside an evolutionary loop can automatically discover code-based forecasting heuristics that are both physically grounded and competitive with deep learning. Guided by building metadata, weather, and operational data from the Building Data Genome Project 2, the LLM generates, recombines, and mutates candidate heuristics, with a feedback loop that grades which internal rules actually help. Evaluations across several building types report lower error than linear, kernel, random-forest, LSTM, and Informer baselines. If the claim holds, accurate and auditable forecasting models could be obtained without training a black box.

What carries the argument

The carrying mechanism is an evolutionary loop with an LLM as the genetic operator, adapted from a reflective-evolution scheme: the LLM generates an initial population of Python heuristics, short-term reflections compare parents to guide crossover, long-term reflections accumulate textual 'verbal gradients' about effective design patterns, and elite heuristics are sampled across generations from a softmax-weighted archive (Cross-Generation Elite Sampling) so the search can escape local optima. The distinctive component is the Physical Insights Feedback Loop, which decomposes each heuristic into logical segments (base-load rules, weather adjustments, occupancy schedules) and returns statistical feedback on each segment's empirical utility to the mutation and reflection prompts. That loop converts the global error signal $J$ (such as RMSE or MAPE) into component-level credit assignment for physically grounded rules.

What would settle it

Run the identical BuildEvo pipeline on BDG2 buildings with little or no missing data using only observed records, and compare MAPE and RMSE against the Informer baseline on the same buildings; if the gap shrinks or reverses on unimputed data, the reported state-of-the-art claim is an artifact of imputation rather than of the evolutionary search.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that physical priors do not have to be imposed on a model from outside; they can be fed into the evolutionary search itself. The generator LLM receives building characteristics such as square footage, primary space usage, weather, and calendar features, and the search's mutation and crossover operations are steered by reflective analyses that identify which physically motivated components — base-load estimates from occupancy classes, weather-normalized adjustments, time-of-day rules — actually reduce forecasting error. With the Physical Insights Feedback Loop that grades individual internal segments of a heuristic, the evolved programs reach a mean absolute percentage error below that of the deep-learning baselines on the tested buildings, while remaining readable Python code with no trained parameters. The paper therefore argues that interpretability and accuracy are not in tension when the search itself is the learning device.

Load-bearing premise

The load-bearing premise is that the imputed BDG2 data used for evolution and evaluation preserves real consumption behavior: if the rule-, model-, and donor-based imputation step injects artificial patterns, the evolved heuristics could score well by fitting those patterns rather than by capturing true building dynamics.

Editorial extensions

If this is right

  • On the tested BDG2 buildings, BuildEvo's heuristics report lower MAPE, RMSE, and MAE than LR, SVR, RF, LSTM, Informer, and PSO-Informer, with no learnable parameters.
  • The evolved heuristics remain readable Python code, so their forecasting logic can be inspected, debugged, and audited in building energy management systems.
  • Ablation shows that removing the Physical Insights Feedback Loop degrades performance but still stays competitive with neural baselines, attributing the gain to component-level physical feedback.
  • Per-building results across classrooms, dormitories, offices, and public-service buildings are consistently better than the baselines, which the paper reads as improved generalization across building types.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same component-level feedback loop should transfer to other forecasting targets such as peak loads, cooling demand, or anomaly detection, and even to control policies, since it only needs a scalar error signal and a way to grade internal segments; the paper does not test these settings.
  • Because the evolved artifact is executable Python, it could be transpiled into embedded or edge controllers where a trained neural network would not fit; this is a consequence of the design, not a result the paper reports.
  • The paper's own preprocessing makes every building's test set partly synthetic, so an independent check — evolving on buildings with near-complete records and testing on untouched ones — would clarify whether the gains come from the search or from imputation patterns.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes BuildEvo, an LLM-driven evolutionary framework that automatically generates interpretable Python-code heuristics for building energy consumption forecasting. The method adapts the ReEvo hyper-heuristic approach and Cross-Generation Elite Sampling, and adds a Physical Insights Feedback Loop (PIFL) that provides statistical feedback on the utility of individual heuristic components. Experiments on five buildings from the Building Data Genome Project 2 (BDG2) compare BuildEvo against LR, SVR, RF, LSTM, Informer, and PSO-Informer, reporting MAPE/RMSE/MAE values in Table 1. The abstract claims state-of-the-art performance and improved generalization, while Section 4.2 more cautiously claims competitive performance. The paper also argues that the generated heuristics are transparent and have no learnable parameters, in contrast to neural baselines.

Significance. If the performance claim were substantiated, BuildEvo would be a valuable step toward interpretable and accurate forecasting heuristics, showing that LLM-driven program search combined with physical knowledge can match or beat deep learning on building energy benchmarks. The problem is well motivated, the use of a public dataset is appropriate, and the PIFL mechanism is an interesting addition to existing LLM+EA frameworks. However, the current evidence is insufficient: the central result rests on a single table with no error bars, no significance tests, and no reproducibility artifacts, and the evaluation protocol is under-specified. The contribution is potentially significant for the building energy community, but the paper as submitted does not yet establish its main claim.

major comments (5)
  1. [Table 1, Section 4.2] The central claim of state-of-the-art performance is not supported by the reported evidence. Each cell in Table 1 contains a single value with no error bars, no number of evolutionary runs, no significance tests, and no description of the train/test split, forecasting horizon, building selection criteria, or baseline hyperparameters. The numerical margins (e.g., MAPE 7.10 vs 8.01 on Classroom) could easily be within run-to-run noise, especially given the stochasticity of LLM-driven evolution. Moreover, Section 4.2 only claims 'competitive performance', which is weaker than the abstract's 'state-of-the-art'; these two statements need to be reconciled and the stronger claim requires statistical support.
  2. [Section 3.2, Appendix A] The evaluation uses a heavily imputed version of BDG2, with rule-based, model-based, donor-based, and interpolation methods filling missing values. No validation of the imputation against held-out ground truth is provided, and the fraction of imputed values per building is not reported. Because all models are compared on the same imputed data, it is possible that the imputation injects artificial patterns (e.g., zero-filling HVAC schedules) that favor heuristics exploiting those patterns, making the reported improvements an artifact of preprocessing. The paper should report metrics on raw data where feasible, or at least analyze the sensitivity of results to the imputation strategy.
  3. [Section 4.2] The PIFL ablation is not statistically grounded. The text states that performance 'significantly improved' with PIFL, but Table 1 reports single values without variance and no significance test is performed. The '- PIFL' row is also defined only as 'Without the PIFL' without specifying which components of the loop are removed, making the ablation difficult to interpret or reproduce.
  4. [Sections 2, 3.1, Abstract] The manuscript contains internal inconsistencies that undermine its clarity: Section 2 ends with 'motivating our EEMD-PSO-Informer approach', which is unrelated to the proposed method; Section 3.1 defines the task as 'using deep learning models' although the method generates heuristics; and the abstract contains a placeholder '[evolutionary/iterative-refinement]'. These artifacts make it difficult to determine exactly what was evaluated and suggest that the manuscript was assembled from unrelated text. They must be corrected before the paper can be considered.
  5. [Section 4.1] No code, data, evolved heuristics, prompts, or random seeds are provided. For an LLM-driven method, the prompt template, generation temperature, and evolutionary budget materially affect results; without these details, the single-run results cannot be reproduced or compared against future work. The paper should release all artifacts or, at minimum, provide a detailed protocol and multiple seeds.
minor comments (5)
  1. [Section 3.3] The population size N is introduced but its value is not specified; the paper should state the population size, number of generations, and total compute budget used in the experiments.
  2. [Table 1] The column headers such as 'To College Classroom' and 'To Public Services-B' appear truncated or misformatted; the building identifiers should be clarified or mapped to the BDG2 building IDs.
  3. [Section 4.2] The statement that BuildEvo 'doesn't have learnable parameters' is misleading, since the evolved heuristics contain constants and thresholds that are selected by the evolutionary search; the paper should clarify that these are not trained by gradient descent but are still tuned parameters.
  4. [References] The Google Gemini reference has an incomplete access date ('YYYY-MM-DD') and a placeholder URL; these should be completed.
  5. [Abstract, Section 3.2] The dataset name is inconsistently written as 'BDGP2' and 'BDG2'; please standardize to a single form.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the reported accuracy is an external benchmark comparison, and the cited ReEvo machinery is a prior method rather than a self-referential input.

full rationale

I walked the paper's claimed derivation chain: an LLM generator produces code heuristics from a task specification and building metadata; evolution, reflection, CGES, and PIFL guide refinement; the final heuristics are scored in Table 1 against LR, SVR, RF, LSTM, Informer, and PSO-Informer on BDG2. The only potentially circular-looking elements do not reduce to the paper's inputs. The fitness objective J (RMSE) is computed on the same imputed BDG2 set used for final evaluation, but this is standard model selection: the baselines are evaluated on the same data, and no equation identifies the evolved heuristic with the imputation model or with J itself. The hierarchical imputation (rule-, model-, and donor-based, Appendix A) is applied to all methods, so any bias it introduces is a data-quality threat, not a constructional identity. Section 3.3 states 'Our evolutionary framework adapts the Reflective Evolution approach (Ye et al., 2024)', and Ye et al. includes a coauthor of this paper; this is a provenance self-citation for the search operators, not a load-bearing uniqueness claim, and the numerical claims rest on Table 1 against external baselines rather than on that citation. The PIFL ablation ('- PIFL' vs 'BUILD EVO' in Table 1) is an internal comparison whose improvement is empirical, not guaranteed by definition. I also flag, per the completeness rule, the manuscript's internal-consistency problems: Section 2 ends with 'motivating our EEMD-PSO-Informer approach', Section 3.1 defines the task 'using deep learning models', and the abstract contains an unresolved '[evolutionary/iterative-refinement]' placeholder; these are residual-revision artifacts that undermine confidence in careful evaluation, but they are not evidence that a result equals its input by construction. The abstract's 'state-of-the-art' wording exceeds Section 4.2's 'competitive performance', but an overclaim is a correctness/reproducibility concern, not circularity. I found no equation, fit, or self-citation chain that forces the reported predictions from the inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three unverified supports: (1) ReEvo's reflective evolution works as a substrate, taken from the authors' own prior work; (2) the imputed BDG2 dataset preserves real consumption dynamics; (3) LLM-generated Python code executes faithfully enough that fitness values guide search. No independent evidence for these is provided beyond internal consistency.

free parameters (3)
  • evolution population size N = not reported
    Initial population size N is a core EA hyperparameter; without reporting N, the search budget is unspecified, and the final heuristic's performance is conditional on it.
  • number of generations / evolution budget = not reported
    No generation count or compute budget reported; the SOTA claim is tied to an unspecified search effort.
  • heuristic constants (base-load multipliers, weather thresholds) = not reported
    Evolved heuristics contain numeric constants selected by the evolutionary search; these are effectively fitted to the training data, but the paper claims 'no learnable parameters'.
assumptions (3)
  • domain assumption ReEvo reflective evolution is a sound basis for heuristic generation (Ye et al., 2024)
    The framework is adapted from ReEvo, co-authored by the same group; the paper does not re-validate the base mechanism.
  • domain assumption Imputed BDG2 data preserves true consumption dynamics
    All evaluations run on the preprocessed, imputed dataset; if imputation injects artificial patterns, the ranking of methods could be distorted.
  • domain assumption LLM-generated Python heuristics execute faithfully and reflect intended logic
    The fitness J is computed by running generated code; silent bugs or LLM hallucination in code would corrupt the evolutionary signal.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BuildEvo: Designing Building Energy Consumption Forecasting Heuristics via LLM-driven Evolution." pith.science (2026). https://pith.science/paper/N47UG7PF

@misc{pith2026250712207,
  author       = {Pith},
  title        = {Pith review of: BuildEvo: Designing Building Energy Consumption Forecasting Heuristics via LLM-driven Evolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N47UG7PF}},
  note         = {Machine review of arXiv:2507.12207}
}
read the original abstract

Accurate building energy forecasting is essential, yet traditional heuristics often lack precision, while advanced models can be opaque and struggle with generalization by neglecting physical principles. This paper introduces BuildEvo, a novel framework that uses Large Language Models (LLMs) to automatically design effective and interpretable energy prediction heuristics. Within an evolutionary process, BuildEvo guides LLMs to construct and enhance heuristics by systematically incorporating physical insights from building characteristics and operational data (e.g., from the Building Data Genome Project 2). Evaluations show BuildEvo achieves state-of-the-art performance on benchmarks, offering improved generalization and transparent prediction logic. This work advances the automated design of robust, physically grounded heuristics, promoting trustworthy models for complex energy systems.

Figures

Figures reproduced from arXiv: 2507.12207 by the authors.

Figure 1
Figure 1. Comparision between human expert heuristic design, hu￾man expert hyper heuristic design, and LLM hyper heuristic de￾sign. Learning-based Methods for Building Energy Consump￾tion Forecasting Machine learning and deep learning have notably advanced building energy consumption fore￾casting, yet challenges in handling complex time series data persist. Initial statistical models like Linear Regres￾sion (LR) and machine l… view at source ↗
Figure 2
Figure 2. Visual overview of the built-environment-informed imputation process, demonstrating its effectiveness in reducing missing data while preserving key data characteristics across multiple dimensions. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 21 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    M., Hassan, M

    Ahmed, D. M., Hassan, M. M., and Mstafa, R. J. A review on deep sequential models for forecasting time series data. Applied Computational Intelligence and Soft Computing, 2022 0 (1): 0 6596397, 2022

    work page 2022

  3. [3]

    S., Koh, S

    Bayasgalan, A., Park, Y. S., Koh, S. B., and Son, S.-Y. Comprehensive review of building energy management models: Grid-interactive efficient building perspective. Energies, 17 0 (19): 0 4794, 2024

    work page 2024

  4. [4]

    Bishop, C. M. and Nasrabadi, N. M. Pattern recognition and machine learning, volume 4. Springer, 2006

    2006

  5. [5]

    K., Liu, J., and Liu, X

    Chen, G., Lu, S., Zhou, S., Tian, Z., Kim, M. K., Liu, J., and Liu, X. A systematic review of building energy consumption prediction: From perspectives of load classification, data-driven frameworks, and future directions. Applied Sciences, 15 0 (6): 0 3086, 2025

    work page 2025

  6. [6]

    Gemini 2.0 flash

    Google . Gemini 2.0 flash. [Large language model], 2025. URL https://deepmind.google/technologies/gemini/. Accessed: YYYY-MM-DD. Available from: [URL for Gemini 2.5 Flash or general Gemini access point]

    work page 2025

  7. [7]

    Connecting large language models with evolutionary algorithms yields powerful prompt optimizers

    Guo, Q., Wang, R., Guo, J., Li, B., Song, K., Tan, X., Liu, G., Bian, J., and Yang, Y. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=ZG3RaNIsO8

    work page 2024

  8. [8]

    and Schmidhuber, J

    Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural computation, 9 0 (8): 0 1735--1780, 1997

    1997

Show all 28 references
  1. [9]

    Energy demand forecasting using a novel remnant gm (1, 1) model

    Hu, Y.-C. Energy demand forecasting using a novel remnant gm (1, 1) model. Soft Computing, 24 0 (18): 0 13903--13912, 2020

    2020

  2. [10]

    and Dong, B

    Jiang, Z. and Dong, B. Modularized neural network incorporating physical priors for future building energy modeling. Patterns, 5 0 (8), 2024

    2024

  3. [11]

    and Eberhart, R

    Kennedy, J. and Eberhart, R. Particle swarm optimization. In Proceedings of ICNN'95-international conference on neural networks, volume 4, pp.\ 1942--1948. ieee, 1995

    1942

  4. [12]

    S., Pourmirza, Z., Pazhoohesh, M., and Walker, S

    Khalil, M., McGough, A. S., Pourmirza, Z., Pazhoohesh, M., and Walker, S. Machine learning, deep learning and statistical analysis for forecasting building energy consumption—a systematic review. Engineering Applications of Artificial Intelligence, 115: 0 105287, 2022

    2022

  5. [13]

    and Wen, J

    Li, X. and Wen, J. Review of building energy modeling for control and operation. Renewable and Sustainable Energy Reviews, 37: 0 517--537, 2014

    2014

  6. [14]

    Grey-box modeling and application for building energy simulations-a critical review

    Li, Y., O'Neill, Z., Zhang, L., Chen, J., Im, P., and DeGraw, J. Grey-box modeling and application for building energy simulations-a critical review. Renewable and Sustainable Energy Reviews, 146: 0 111174, 2021

    2021

  7. [15]

    and Zhai, Z

    Lim, H. and Zhai, Z. J. Review on stochastic modeling methods for building stock energy prediction. In Building Simulation, volume 10, pp.\ 607--624. Springer, 2017

    2017

  8. [16]

    S., Duque-Perez, O., and Zorita-Lamadrid, A

    Mariano-Hern \'a ndez, D., Hern \'a ndez-Callejo, L., Garc \' a, F. S., Duque-Perez, O., and Zorita-Lamadrid, A. L. A review of energy consumption forecasting in smart buildings: Methods, input variables, forecasting horizon and metrics. Applied Sciences, 10 0 (23): 0 8323, 2020

    2020

  9. [17]

    More buildings make more generalizable models—benchmarking prediction methods on open electrical meter data

    Miller, C. More buildings make more generalizable models—benchmarking prediction methods on open electrical meter data. Machine Learning and Knowledge Extraction, 1 0 (3): 0 974--993, 2019

    2019

  10. [18]

    Y., Nagy, Z., Raftery, P., Hobson, B

    Miller, C., Kathirgamanathan, A., Picchetti, B., Arjunan, P., Park, J. Y., Nagy, Z., Raftery, P., Hobson, B. W., Shi, Z., and Meggers, F. The building data genome project 2, energy meter data from the ASHRAE great energy predictor III competition. Scientific Data, 7: 0 368, 2020

    2020

  11. [19]

    Osuna, E. C. and Sudholt, D. Runtime analysis of probabilistic crowding and restricted tournament selection for bimodal optimisation. In Proceedings of the Genetic and Evolutionary Computation Conference, pp.\ 929--936, 2018

    2018

  12. [20]

    I., Monedero, I., and Le \'o n, C

    Pe \ n a, M., Biscarri, F., Guerrero, J. I., Monedero, I., and Le \'o n, C. Rule-based system to detect energy efficiency anomalies in smart buildings, a data mining approach. Expert Systems with Applications, 56: 0 242--255, 2016

    2016

  13. [21]

    gradient descent

    Pryzant, R., Iter, D., Li, J., Lee, Y. T., Zhu, C., and Zeng, M. Automatic prompt optimization with" gradient descent" and beam search. arXiv preprint arXiv:2305.03495, 2023

    2023 arXiv

  14. [22]

    P., Dupont, E., Ruiz, F

    Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., et al. Mathematical discoveries from program search with large language models. Nature, 625 0 (7995): 0 468--475, 2024

    2024

  15. [23]

    and Zmeureanu, R

    Runge, J. and Zmeureanu, R. A review of deep learning techniques for forecasting energy use in buildings. Energies, 14 0 (3): 0 608, 2021

    2021

  16. [24]

    Enhancing grid stability and efficiency in buildings through forecasting and intelligent energy management of distributed energy resources

    Senthil Kumar, S., Srinivasan, C., and Sridhar, P. Enhancing grid stability and efficiency in buildings through forecasting and intelligent energy management of distributed energy resources. Electrical Engineering, pp.\ 1--18, 2024

    2024

  17. [25]

    and Kotsakis, E

    Thomas, D. and Kotsakis, E. Energy management and optimal power scheduling in a smart building under uncertainty. In Smart Cities-Their Framework and Applications. IntechOpen, 2020

    2020

  18. [26]

    Combining physics-based and data-driven modeling for building energy systems

    Von Krannichfeldt, L., Orehounig, K., and Fink, O. Combining physics-based and data-driven modeling for building energy systems. arXiv preprint arXiv:2411.01055, 2024

    2024 arXiv

  19. [27]

    Reevo: Large language models as hyper-heuristics with reflective evolution

    Ye, H., Wang, J., Cao, Z., Berto, F., Hua, C., Kim, H., Park, J., and Song, G. Reevo: Large language models as hyper-heuristics with reflective evolution. arXiv preprint arXiv:2402.01145, 2024

    2024 arXiv

  20. [28]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp.\ 11106--11115, 2021

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.