Pith. sign in

REVIEW 4 major objections 6 minor 58 references

LLM-based Question-Answer Framework for Sensor-driven HVAC System Interaction

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read JARVIS claims that a two-stage design—a fine-tuned planning LLM plus a deterministic query/processing agent—answers vague questions about live HVAC sensor data more accurately than a generic text-to-SQL+RAG baseline.

desk verdict A genuinely new system architecture with a coherent evaluation that is too narrow to support the headline claims; deserves review, not desk rejection. read the letter →

arxiv 2507.04748 v1 pith:Z35T552S submitted 2025-07-07 cs.AI

Sungmin Lee , Minju Kang , Joonhee Lee , Seungyong Lee , Dongju Kim , Jingi Hong , Jun Shin , Pei Zhang
show 1 more author
JeongGil Ko
This is my paper
classification cs.AI
keywords LLM-basedquestionansweringHVACsystemssensordatatext-to-SQLretrieval-augmentedgenerationchain-of-thoughtreasoningLLM-as-a-judgesmartbuildings
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a two-stage architecture—a fine-tuned 'expert' language model that plans, plus a deterministic agent that retrieves and processes data—can give non-expert users reliable question answering over live HVAC sensor data. The proposed system, JARVIS, is claimed to outperform both a general-purpose text-to-SQL-with-retrieval baseline and every ablation that removes one of its design components, on expert-curated questions from a real commercial building. The paper's evidence pairs automated scoring by three LLM judges with a 22-participant user study, and it reports that the full system scores above 4.0 out of 5 on cohesiveness and truthfulness while also being among the fastest configurations. If the claim holds, building occupants and managers could ask colloquial questions about temperature, energy, and comfort and receive grounded, interpretable answers without writing SQL or interpreting raw sensor streams themselves.

What carries the argument

The load-bearing mechanism is the master–worker split between a fine-tuned Expert-LLM and a deterministic Agent. The Expert-LLM, fine-tuned on HVAC-common knowledge and prompted with deployment-specific metadata (a mapping from user language to database names plus current time), first generates a chain-of-thought 'thinking' trace, then an 'expectation' that fixes the final answer format, and only then emits JSON execution instructions; this bottom-up planning ensures the querying and processing steps supply exactly what the response needs. The Agent's parameterized SQL builder, rather than letting the model write SQL, takes high-level query intentions and expands them into complete statements with boilerplate, joins, and null filtering, while a Python data-processing library computes statistics and formats results for a general-purpose response-generation LLM. The same data is thus never fed raw to the final LLM; it is summarized and shaped by deterministic code.

What would settle it

Collect a second test set written by actual residents and building managers in a different building and language, run JARVIS and the text-to-SQL baseline on it, and have humans rate responses; if the LLM-judge ranking no longer matches human rankings, or if JARVIS's advantage shrinks on questions with new vocabulary and layouts, the paper's generalization claim is contradicted. A narrower check: if a change of metadata alone, without retraining, causes the Expert-LLM to fail on questions of the same type, the adaptive-context claim fails.

Watch

Extended reading notes

Core claim

JARVIS's central claim is that decomposing the QA task into a planning stage and an execution stage, with domain knowledge injected at the right place, solves the three failure modes that plague generic text-to-SQL for HVAC data: user-native ambiguity, fragile monolithic SQL, and missing statistical operations. The Expert-LLM produces structured JSON instructions containing a thinking trace, an expected-answer template, and modular query/processing steps; the Agent's parameterized SQL builder converts those high-level intentions into executable statements with null filtering and joins handled deterministically, and a Python-based processor computes operations such as argmax that SQL cannot express. The paper reports that JARVIS outperforms the text-to-SQL-with-retrieval baseline across cohesiveness, helpfulness, and truthfulness, that ablation variants all score lower, and that the query-execution and processing modules interact super-additively: accurate retrieval alone is not enough, because response quality drops sharply when either module is removed. A user study on ten questions confirms the same ranking of configurations that the LLM judges produce.

Load-bearing premise

The load-bearing premise is that the 80 expert-curated QA pairs, especially the 27 held-out questions from one commercial building in one language, represent the breadth of real non-expert HVAC questions, and that the LLM judges' scores track human preferences closely enough to rank systems correctly.

Editorial extensions

If this is right

  • Non-expert users can ask vague, user-native questions such as 'how hot is our room?' and receive answers grounded in live sensor data, because metadata injection resolves the user-to-database vocabulary gap.
  • Data retrieval becomes more reliable because the model never writes raw SQL; modular parameterized queries avoid the syntax errors and null-handling failures that dominate generic text-to-SQL errors.
  • Queries that require operations SQL cannot express, such as finding the time of maximum temperature, become answerable through Python-based post-processing in the Agent.
  • Each design component earns its place: removing metadata plus thinking, expectation, query execution, or processing lowers response quality, and removing both backend modules shows that accurate retrieval alone is insufficient.
  • The system can run on-site with small language models while remaining among the fastest configurations tested, suggesting practical deployment without cloud dependency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit: the same context taxonomy—common knowledge fine-tuned, deployment metadata prompted, sensor data processed before prompting—could transfer to other sensor-rich domains such as water, energy, or industrial monitoring, since none of the categories are HVAC-specific.
  • The judge alignment result is based on ten questions from one building and one language; a stronger test would replicate the LLM-judge and user-study comparison on a multilingual or multi-building dataset, where judge bias is more likely to surface.
  • The residual failure mode the paper observes, malformed JSON in 1 of 27 test cases, suggests that constrained or programmatic decoding could eliminate most remaining execution errors and push accuracy higher; the paper names this as future work but does not test it.
  • Because the Expert-LLM centralizes semantic interpretation, the architecture could be reused as a general 'semantic context manager' for other agent-based systems, with the planning LLM swapped for a different domain expert.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes JARVIS, a two-stage LLM-based question-answering framework for sensor-driven HVAC systems. An Expert-LLM translates user queries into structured execution instructions, and a deterministic Agent performs parameterized SQL retrieval, Python-based data processing, and final response generation. The design incorporates adaptive context injection, a thinking component, expectation-based bottom-up planning, and modular query/processing modules. The system is evaluated on 27 held-out expert-curated test queries from one commercial building against the TAG baseline and several ablations, using LLM-as-a-Judge scores and a 22-participant user study. The central claim is that JARVIS consistently outperforms the baseline and all ablations on response quality and query accuracy.

Significance. If the empirical claims hold, JARVIS is a useful practical contribution: the architecture is coherent, the parameterized SQL builder addresses a known text-to-SQL brittleness problem, and the component-wise ablation analysis is informative. The paper is also candid about limitations such as malformed JSON outputs and the tension between template-guided and natural responses. However, the significance is currently bounded by the narrowness of the evaluation: 27 test queries from one building and one language, with the headline comparison resting on averaged LLM-as-a-Judge scores that are not validated against human ratings for the baseline. The design ideas merit publication after the evaluation is strengthened or the claims are appropriately scoped.

major comments (4)
  1. [Section 5.3, Figure 5] The central claim that JARVIS outperforms the TAG baseline on all three response quality metrics rests entirely on LLM-as-a-Judge scores averaged over three models and two runs, with no confidence intervals, no per-judge breakdown, and no inter-judge agreement analysis. The user study in Section 5.5, which could have validated this comparison, only rates 10 query-response pairs drawn from JARVIS configurations and does not include TAG. As it stands, Figure 5 cannot support the strong comparative claim. Please add agreement metrics (e.g., Krippendorff's alpha or pairwise Cohen's kappa), report variance, and obtain human ratings for TAG responses on the same test set, or explicitly restrict the claim to the automated evaluation.
  2. [Section 5.1.2, Section 5.4] The test set consists of 27 queries curated by HVAC experts from a single commercial building and in a single language, and the same experts authored the 53 fine-tuning samples. The disjoint persona split is a good design choice, but it does not address overlap in building, language, or curators. The abstract's claim that JARVIS 'consistently outperforms' baselines generalizes well beyond what this evaluation can support. Please either temper the generalization claims throughout the paper or add evidence from additional deployments, languages, or independently authored queries.
  3. [Section 5.2, Section 5.5] The LLM-as-a-Judge scores are reported as a single average of six scores per response, with no measure of dispersion or agreement among GPT-4o, GPT-3.5-turbo, and Gemini 2.5. Figure 8 claims that the automated scores 'closely track' human evaluations, but no correlation coefficient or agreement statistic is reported, and the human comparison covers only 10 JARVIS ablation pairs rather than the full set of configurations and the TAG baseline. Please report per-model scores, agreement coefficients, and a human-judge correlation computed on a set that includes all variants and the baseline.
  4. [Section 5.6, Figure 10] The query execution accuracy, precision, and recall values (e.g., 0.92 execution accuracy for full JARVIS) are calculated on only 27 test queries and are reported as averages without confidence intervals or a per-query distribution. Given the small sample size, exact binomial confidence intervals and per-query counts are needed. In addition, Section 5.3 reports that TAG failed SQL generation in 4 of 27 cases and exceeded context limits in 3 instances; please clarify how these execution failures were scored in the response-quality comparison, since a failed query cannot receive a meaningful truthfulness rating.
minor comments (6)
  1. [Figure 5] The y-axis label 'Coheseiveness' is misspelled; it should be 'Cohesiveness'.
  2. [Figure 11] The fitted equation 'y = 0.03e−2x -1.31' appears to have a malformed exponent or missing superscript; please correct the formatting.
  3. [Section 5.3] The phrase 'This performance gap is primarily stems from' contains a grammatical error; it should be 'primarily stems from' or 'primarily results from'.
  4. [Section 5.7, Table 2] The table would be more self-contained if the column headers and the 'Proc. O'/'Proc. X' notation were defined in the caption or table notes.
  5. [Section 6] The regex-based post-processing patch for malformed JSON is mentioned in the discussion but is not described in the implementation section; a brief description of its behavior would help reproducibility.
  6. [Section 3.2] The terms 'semantic context' and 'sensor context' are introduced somewhat abruptly; consider defining them immediately after Table 1 rather than at the end of the subsection.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central claims rest on disjoint-persona held-out evaluation against an external baseline, not on a derivation that reduces to its inputs.

full rationale

This is an empirical systems paper, not a derivation: JARVIS's reported superiority is a measured outcome. The Expert-LLM is fine-tuned on 53 expert-authored QA pairs, but the headline evaluation uses 27 held-out test pairs from two personas whose metadata was disjoint from the fine-tuning personas (Section 5.1.2), so the test scores are not re-statements of training data. The JARVIS-vs-TAG comparison (Figure 5) is against an independent external baseline [5], and the response-quality scores come from three external LLM judges (GPT-4o, GPT-3.5-turbo, Gemini 2.5) plus a 22-participant user study; query accuracy is checked against ground-truth cells using execution accuracy, precision, and recall (Section 5.2). The user study covers only 10 response pairs and does not include the TAG baseline, and the single-building, single-language, 27-question test set limits external validity; likewise, using LLM judges that are given the dataset's expectations could introduce stylistic bias. These are legitimate external-validity and statistical-support concerns, not circularity: no claim reduces by construction to a fitted parameter, and no load-bearing step depends on a self-citation. The paper also self-reports limitations (Section 5.3 rigid 'larger by 0' responses; Section 6 malformed JSON in 1/27 cases and the expectation-template richness trade-off), which further shows the evaluation is not engineered to be unfalsifiable. I therefore find no circular step requiring a score above zero.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a fitted Expert-LLM and on assumptions about data representativeness, judge validity, and metadata availability. No new physical entities or theoretical constructs are introduced.

free parameters (1)
  • Expert-LLM LoRA weights = not released (fine-tuned on 53 expert QA samples)
    The planner's decomposition of queries into thinking, expectation, and execution instructions is produced by this fitted model. The paper reports LLaMA3.1-8B-Instruct with LoRA but provides neither the weights nor detailed training hyperparameters (Section 4).
assumptions (4)
  • domain assumption The expert-curated 80 QA pairs capture realistic user queries and correct answers for the target deployment.
    These pairs serve as ground truth for fine-tuning (53 samples) and evaluation (27 samples); Section 5.1.2.
  • domain assumption LLM-as-a-Judge scores from GPT-4o, GPT-3.5-turbo, and Gemini 2.5 correlate with human quality judgments.
    Section 5.2 relies on the six-score average for all non-user-study metrics; Figure 8 checks against 22 participants on only 10 questions.
  • ad hoc to paper Deployment-specific metadata (user-native to database-native taxonomy mapping, current timestamp) is available and accurate for each new deployment.
    The Expert-LLM depends on these metadata dictionaries for ambiguity resolution, as shown in Sections 3.3.1 and 5.5 (Question 1); the paper does not provide a method to derive the mapping automatically.
  • domain assumption Sensor-data nulls are missing at random and can be safely excluded by a fixed filtering rule.
    The parameterized SQL builder automatically appends null-exclusion clauses based on the preliminary failure analysis in Section 2.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-based Question-Answer Framework for Sensor-driven HVAC System Interaction." pith.science (2026). https://pith.science/paper/Z35T552S

@misc{pith2026250704748,
  author       = {Pith},
  title        = {Pith review of: LLM-based Question-Answer Framework for Sensor-driven HVAC System Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z35T552S}},
  note         = {Machine review of arXiv:2507.04748}
}
read the original abstract

Question-answering (QA) interfaces powered by large language models (LLMs) present a promising direction for improving interactivity with HVAC system insights, particularly for non-expert users. However, enabling accurate, real-time, and context-aware interactions with HVAC systems introduces unique challenges, including the integration of frequently updated sensor data, domain-specific knowledge grounding, and coherent multi-stage reasoning. In this paper, we present JARVIS, a two-stage LLM-based QA framework tailored for sensor data-driven HVAC system interaction. JARVIS employs an Expert-LLM to translate high-level user queries into structured execution instructions, and an Agent that performs SQL-based data retrieval, statistical processing, and final response generation. To address HVAC-specific challenges, JARVIS integrates (1) an adaptive context injection strategy for efficient HVAC and deployment-specific information integration, (2) a parameterized SQL builder and executor to improve data access reliability, and (3) a bottom-up planning scheme to ensure consistency across multi-stage response generation. We evaluate JARVIS using real-world data collected from a commercial HVAC system and a ground truth QA dataset curated by HVAC experts to demonstrate its effectiveness in delivering accurate and interpretable responses across diverse queries. Results show that JARVIS consistently outperforms baseline and ablation variants in both automated and user-centered assessments, achieving high response quality and accuracy.

Figures

Figures reproduced from arXiv: 2507.04748 by the authors.

Figure 1
Figure 1. Failure analysis of generic text-to-sql operations in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Operational flow of JARVIS. Semantic context Sensor context HVAC-common knowledge Deployment and user specific context Small-scale sensor data Large-scale sensor data Occurrence time Before deployment Before request In-situ (real-time) In-situ (real-time) Context length Long Short Short Long Pretraining X X X X Finetuning O X X X Direct Prompting - O O X Processed Prompting - - O O Examples Presuppositions Per-user … view at source ↗
Figure 3
Figure 3. Sample operation of Expert-LLM generating execution instructions based on user input. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Query type distribution for QA dataset. 5 Evaluation We evaluate JARVIS using data from a real-world, commercial-scale HVAC deployment at a large building site. Based on this data, we constructed a QA dataset curated by HVAC experts, which serves as the foundation for …
Figure 5
Figure 5. Figure 5: Response quality comparison vs Baseline TAG [ [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Avg. response quality vs. Query execution accuracy. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Response quality measure comparison between the LLM-as-a-Judge approach and user study. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: User study results for Questions 1, 5 and 9. Error bars indicate 25th–75th percentile range. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Query execution accuracy, recall and precision for [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Expert-LLM latency for varying token lengths. [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 29 canonical work pages

  1. [1]

    Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. 2024. Make your llm fully utilize the context. Advances in Neural Information Processing Systems 37 (2024), 62160–62188

    work page 2024

  2. [2]

    Akari Asai, Sewon Min, Zexuan Zhong, and Danqi Chen. 2023. Retrieval-based Language Models and Applications. arXiv preprint arXiv: (2023)

    work page 2023

  3. [3]

    ASHRAE. 2024. Certified HVAC Designer (CHD). ASHRAE Official Web- site. Available from: https://www.ashrae.org/professional-development/ashrae- certification/hvac-designer-certification

    work page 2024

  4. [4]

    Anirban Basu, Yujia Wang, Yifan Li, et al . 2024. Seven Failure Points When Engineering a Retrieval Augmented Generation System. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice. ACM, 1–10

    work page 2024

  5. [5]

    Asim Biswal, Liana Patel, Siddarth Jha, Amog Kamsetty, Shu Liu, Joseph E Gon- zalez, Carlos Guestrin, and Matei Zaharia. 2024. Text2sql is not enough: Unifying ai and databases with tag. arXiv preprint arXiv:2408.14717 (2024)

    arXiv 2024

  6. [6]

    Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, and Shengxin Zhu. 2023. Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review. arXiv preprint arXiv:2310.14735 (2023)

    arXiv 2023

  7. [7]

    Xinghao Chen, Anhao Zhao, Heming Xia, Xuan Lu, Hanlin Wang, Yanjun Chen, Wei Zhang, Jian Wang, Wenjie Li, and Xiaoyu Shen. 2025. From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge. arXiv preprint arXiv:2411.16594 (2025)

    arXiv 2025

  8. [8]

    Yuxuan Cheng, Yuvraj Agarwal, and Rajesh Gupta. 2019. HVACMeter: Ap- portionment of HVAC Power to Thermal Zones and Air Handler Units. In Proceedings of the 6th ACM International Conference on Systems for Energy- Efficient Buildings, Cities, and Transportation (BuildSys ’19) . ACM, 61–70. https: //doi.org/10.1145/3360322.3360840

    work page arXiv 2019

Show all 58 references
  1. [9]

    Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu, Bing Qin, and Ting Liu. 2023. Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Fron- tiers and Future. arXiv preprint arXiv:2309.15402 (2023)

    2023 arXiv

  2. [10]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. arXiv preprint arXiv:2405.06211 (2024)

    2024 arXiv

  3. [11]

    Yiming Fan, Xinyu Zhang, Yifan Wang, Yuxiang Li, Bill Yuchen Lin, and Xiang Ren. 2024. EplusGPT: A Large Language Model for Building Energy Simulation and Control. Energy and Buildings 288 (2024), 113278

    2024

  4. [12]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2023. Retrieval-Augmented Gen- eration for Large Language Models: A Survey. arXiv preprint arXiv:2312.10997 (2023)

    2023 arXiv

  5. [13]

    Aaron Grattafiori et al . 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783

    2024 arXiv

  6. [14]

    Saurabh Gupta et al. 2024. Students Level of Trust in the Use of AI Chatbots in Higher Education: A Quantitative Analysis Adopting Technology Acceptance Model. IEEE Access (2024). User study with 98 respondents, measuring perceived usefulness, ease of use, and trust (truthfulness)

    2024

  7. [15]

    Shailja Gupta, Rajesh Ranjan, and Surya Narayan Singh. 2024. A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions. arXiv preprint arXiv:2410.12837 (2024)

    2024 arXiv

  8. [16]

    Xiaoxin He, Zhiwei Liu, Ziqi Wang, Yuxuan Wang, Yiqun Liu, and Qingyao Ai

  9. [17]

    Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, and Xiao Huang. 2024. Next-generation database interfaces: A survey of llm-based text-to-sql. arXiv preprint arXiv:2406.08426 (2024)

    2024

  10. [18]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685 [cs.CL] https://arxiv.org/abs/2106.09685

    2021 arXiv

  11. [19]

    Gang Jiang, Zhihao Ma, Liang Zhang, and Jianli Chen. 2025. Prompt engineering to inform large language model in automated building energy modeling. Energy 316 (2025), 134548

    2025

  12. [20]

    Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024. Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models. arXiv preprint arXiv:2402.14848 (2024). https://arxiv.org/abs/2402.14848 Accepted to ACL 2024

    2024 arXiv

  13. [21]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing s...

    2020

  14. [22]

    Dan Li, Yuxun Zhou, Guoqiang Hu, and Costas J. Spanos. 2020. Handling Incom- plete Sensor Measurements in Fault Detection and Diagnosis for Building HVAC Systems. IEEE Transactions on Automation Science and Engineering 17, 2 (2020), 833–846. https://doi.org/10.1109/TASE.2019.2948101

    2020

  15. [23]

    Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024. LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods. arXiv preprint arXiv:2412.05579 (2024)

    2024 arXiv

  16. [24]

    Siran Li, Linus Stenzel, Carsten Eickhoff, and Seyed Ali Bahrainian. 2025. Enhanc- ing Retrieval-Augmented Generation: A Study of Best Practices. In Proceedings of the 31st International Conference on Computational Linguistics (COLING) . 6705– 6717

    2025

  17. [25]

    Yifan Li, Yichao Liu, Banghao Chen, and Shengxin Zhu. 2024. A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. arXiv preprint arXiv:2402.07927 (2024)

    2024 arXiv

  18. [26]

    Zhiwei Li, Zhihao Wang, Banghao Chen, and Shengxin Zhu. 2023. On the Use of Virtual Knowledge Graphs to Improve Environmental Sensor Data Accessibility. IEEE Transactions on Industrial Informatics 20, 2 (2023), 1234–1245

    2023

  19. [27]

    Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang, and Zhenyu Yan. 2025. TaskSense: A Translation-like Approach for Tasking Heterogeneous Sensor Systems with LLMs. Association for Computing Ma- chinery, New York, NY, USA, 213–225. ...

    2025

  20. [28]

    Yichao Liu, Zhiwei Wang, Banghao Chen, and Shengxin Zhu. 2024. Be- Honest: Benchmarking Honesty in Large Language Models. arXiv preprint arXiv:2406.13261 (2024)

    2024 arXiv

  21. [29]

    Yifan Liu, Zhihao Wang, Zhaofeng Zhang, Yichao Liu, Zhiwei Li, Banghao Chen, and Shengxin Zhu. 2025. A Survey of Automatic Prompt Engineering: An Opti- mization Perspective. arXiv preprint arXiv:2502.08930 (2025)

    2025

  22. [30]

    Yang Liu, Yujia Zhang, Wenxuan Zhang, Xinyu Wang, Zhen Li, Yujie Xu, Yichao Liu, Zhiwei Li, Banghao Chen, and Shengxin Zhu. 2023. A Survey on Large Language Model based Autonomous Agents. Frontiers of Computer Science 17, 6 (2023), 176301

    2023

  23. [31]

    Jie Lu, Xiangning Tian, Chaobo Zhang, Yang Zhao, Jian Zhang, Wenkai Zhang, Chenxin Feng, Jianing He, Jiaxi Wang, and Fengtai He. 2024. Evaluation of large language models (LLMs) on the mastery of knowledge and skills in the heating, ventilation and air conditioning (HVAC) indu...

    2024

  24. [32]

    Wes McKinney. 2010. Data Structures for Statistical Computing in Python. In Proceedings of the 9th Python in Science Conference . 51–56

    2010

  25. [33]

    Zhang, Kashif Rasul, Anderson Schneider, Lintao Ma, Yuriy Nevmyvaka, and Dongjin Song

    Kanghui Ning, Zijie Pan, Yu Liu, Yushan Jiang, James Y. Zhang, Kashif Rasul, Anderson Schneider, Lintao Ma, Yuriy Nevmyvaka, and Dongjin Song. 2025. TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecaster. arXiv preprint arX...

    2025

  26. [34]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, a...

    2022 arXiv

  27. [35]

    Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction. arXiv preprint arXiv:2304.11015 (2023)

    2023 arXiv

  28. [36]

    Muhammad Usman Rasheed et al . 2023. Leveraging SQL for Effective Data Acquisition in Wireless Sensor Networks: An Empirical Approach. IEEE Sensors Journal 23, 21 (2023), 23456–23467. 13

    2023

  29. [37]

    Jeff Reback, Wes McKinney, jbrockmendel, Joris Van den Bossche, Tom Augspurger, Simon Cloud, Kyle Hawkins, Gfyoung, Sinhrks, Michael Roeschke, Stephan Klein, Terji Petersen, Marc Tratner, Chang She, Mortada Mehyar Ayd, William Hoefler, Eyal Naveh, Samuel Garcia, Jeremy Schende...

    2020 doi

  30. [38]

    LG AI Research, Soyoung An, Kyunghoon Bae, Eunbi Choi, Kibong Choi, Stan- ley Jungkyu Choi, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Gerrard Jeong- won Jo, Hyunjik Jo, Jiyeon Jung, Yountae Jung, Hyosang Kim, Joonkee Kim, Seonghwan Kim, Soyeon Kim, Sunkyoung Kim, Yireun Kim, Yo...

    2024

  31. [39]

    Ellen Riloff and Michael Thelen. 2000. A Rule-based Question Answering System for Reading Comprehension Tests. In ANLP-NAACL 2000 Workshop: Reading Comprehension Tests as Evaluation for Computer-based Language Understanding Systems. Association for Computational Linguistics, S...

    2000

  32. [40]

    Prasanta Kumar Sahoo et al. 2025. A Systematic Review on Prompt Engineering for Large Language Models. IEEE Transactions on Artificial Intelligence (2025)

    2025

  33. [41]

    Satwinder Singh and Himanshu Beniwal. 2022. A survey on near-human conver- sational agents. Expert Systems with Applications 202 (2022), 117116

    2022

  34. [42]

    Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. 2024. DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models. arXiv preprint arXiv:2403.10081 (2024)

    2024 arXiv

  35. [43]

    The Chill Brothers. 2025. Smart Home and HV AC: How Energy Management Is Shaping Future Spending. https://thechillbrothers.com/smart-home-and-hvac- how-energy-management-is-shaping-future-spending/ By 2029, smart AC and heating systems are expected to generate $8.31 billion

    2025

  36. [44]

    Timescale

    Inc. Timescale. 2017. TimescaleDB: An Open-Source Time-Series SQL Database. https://www.timescale.com/

    2017

  37. [45]

    Ziyang Wang, Yuxiang Yang, Yuxuan Wang, Yuxiang Li, Bill Yuchen Lin, and Xiang Ren. 2023. The Rise and Potential of Large Language Model Based Agents: A Survey. arXiv preprint arXiv:2309.07864 (2023)

    2023 arXiv

  38. [46]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems. https://arxiv.org/abs/2201.11903

    2022 arXiv

  39. [47]

    Tong Xiao and Peng Xu. 2024. Exploring automated energy optimization with unstructured building data: A multi-agent based framework leveraging large language models. Energy and Buildings 322 (2024), 114691

    2024

  40. [48]

    Shichao Xu, Yangyang Fu, Yixuan Wang, Zhuoran Yang, Chao Huang, Zheng O’Neill, Zhaoran Wang, and Qi Zhu. 2025. Efficient and assured reinforcement learning-based building HVAC control with heterogeneous expert-guided train- ing. Scientific Reports 15, 1 (2025), 7677

    2025

  41. [49]

    Juan Yang, Jing Yu, and Shijing Wang. 2024. Heating ventilation air-conditioner system for multi-regional commercial buildings based on deep reinforcement learning. Advanced Control for Applications: Engineering and Industrial Systems 6, 2 (2024), e190

    2024

  42. [50]

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887 (2018)

    2018 arXiv

  43. [51]

    Beichen Zhang, Kun Zhou, Zheng Gong, Wayne Xin Zhao, and Ji-Rong Wen

  44. [52]

    Jian Zhang, Chaobo Zhang, Jie Lu, and Yang Zhao. 2025. Domain-specific large language models for fault diagnosis of heating, ventilation, and air condition- ing systems by labeled-data-supervised fine-tuning. Applied Energy 377 (2025), 124378

    2025

  45. [53]

    Xiaoyu Zhang, Yifan Wang, Yuxiang Li, Bill Yuchen Lin, and Xiang Ren. 2024. SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL)

    2024

  46. [54]

    Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. 2024. Retrieval-Augmented Generation for AI-Generated Content: A Survey. CoRR (2024). arXiv:2401.00001

    2024 arXiv

  47. [55]

    Sisi Zheng, Yuxuan Wang, Yuchen Zhang, Bill Yuchen Lin, and Xiang Ren. 2023. Judging LLM-as-a-Judge with LLM-as-a-Judge. arXiv preprint arXiv:2306.05685 (2023)

    2023 arXiv

  48. [56]

    Xiaohu Zhu, Qian Li, Lizhen Cui, and Yongkang Liu. 2024. Large Language Model Enhanced Text-to-SQL Generation: A Survey. arXiv preprint arXiv:2410.06011 (2024). 14

    2024 arXiv

  49. [2021]

    In Proceedings of the 29th ACM International Conference on Multimedia

    ViCA: Combining visual, social, and task-oriented conversational AI in a healthcare setting. In Proceedings of the 29th ACM International Conference on Multimedia. 5174–5182. User study with 21 participants, evaluation of helpfulness and entertainment

  50. [2024]

    arXiv preprint arXiv:2402.07630 (2024)

    G-Retriever: Retrieval-Augmented Generation for Textual Graph Under- standing and Question Answering. arXiv preprint arXiv:2402.07630 (2024)

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.