Pith. sign in

REVIEW 4 major objections 4 minor 30 references

Explingo: Explaining AI Predictions using Large Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that one large language model can turn SHAP explanations into fluent, accurate narratives, and a second LLM can grade those narratives on accuracy, completeness, fluency, and conciseness.

desk verdict A useful open-source pipeline for narrative SHAP explanations, but the headline claim leans on an unvalidated LLM grader and needs human evaluation on the actual outputs. read the letter →

arxiv 2412.05145 v1 pith:SGH66KHX submitted 2024-12-06 cs.CL cs.AIcs.LG

Alexandra Zytek , Sara Pido , Sarah Alnegheimish , Laure Berti-Equille , Kalyan Veeramachaneni This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords explainableAISHAPlargelanguagemodelsnarrativeexplanationsLLMevaluationfew-shotpromptingautomatedgradingnatural
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Explingo claims that a large language model can turn SHAP feature-contribution tables into natural-language narratives that preserve the numbers and direction of the original explanation, and that a second LLM can grade those narratives on accuracy, completeness, fluency, and conciseness. The paper's evidence comes from experiments across nine exemplar datasets built from four public prediction tasks (house prices, student pass/fail, mushroom toxicity, and PDF malware), in which the Narrator's best configurations reached average total grades near 15 out of 16 according to the Grader. If true, this would let non-experts read model explanations without wading through SHAP tables: a user would write only a few example narratives in the desired style, and the same prompt would produce and quality-check readable explanations for future predictions. The paper also identifies where the pipeline currently fails, most notably when narratives use comparative language and when the Grader must score complex domains.

What carries the argument

The load-bearing machinery is a pair of LLM prompts. The Narrator receives the SHAP explanation parsed into triples of feature name, feature value, and SHAP contribution, along with a format descriptor, a one-line context such as “the model predicts house prices,” and a handful of exemplar narratives; it outputs a single narrative. The Grader receives an explanation, a narrative, and a rubric for one metric, and returns a numeric grade: accuracy and completeness use explicit chain-of-thought instructions to list features one by one, fluency compares the narrative against five exemplars, and conciseness is computed deterministically from word count. The two are tied together by a weighted total grade $G = \alpha_a A + \alpha_f F + \alpha_c C + \alpha_s S$, and the Narrator is optimized by bootstrapping new exemplars that receive perfect Grader scores. That feedback loop—Grader selects exemplars, exemplars shape Narrator output, Narrator output is graded again—is what carries the argument from “LLMs can write prose” to “LLMs can write and check application-specific explanations.”

What would settle it

Collect a fresh set of 100 generated narratives from the same datasets and domains, have several human annotators grade them under the paper's rubrics, and compare their average ratings with the Grader's scores on narratives not used in prompt tuning. If agreement is no better than chance, especially on accuracy, the evidence for high-quality narratives collapses. A faster check is to reproduce the reported Mush 2 failure, where a narrative that faithfully restated the SHAP direction and values received an accuracy score of 0.

Watch

Extended reading notes

Core claim

The central discovery, on the paper's own terms, is that generation is not the bottleneck: evaluation is. With no exemplars, the Narrator already produces perfectly accurate and complete narratives, scoring 4.0 out of 4.0 on both metrics, but fluency and conciseness are low. Adding one hand-written exemplar and three bootstrapped exemplars raises the total grade from about 11.3 to 15.0 out of 16, at only a small cost to accuracy and completeness. The Grader, after prompt tuning with chain-of-thought instructions and explicit rubrics, agrees with human labels on 96% of the 50-example accuracy and completeness validation sets, and can reliably distinguish narrative styles once five exemplars are supplied. The paper's main claim is therefore that LLMs can reliably transform SHAP explanations into high-quality narratives, particularly when guided by a small number of human-labeled and bootstrapped examples.

Load-bearing premise

The main claim rests on trusting the Grader's scores as a valid measure of narrative quality, and the Grader was verified against human labels only on the small sets used to tune its prompts, with no held-out evaluation for the main results.

Editorial extensions

If this is right

  • A user needs to write only around four exemplar narratives (one hand-written plus three bootstrapped) to reach the highest measured quality; adding more exemplars improves style but measurably lowers accuracy and completeness.
  • The Grader can double as a deployment guardrail: narratives below a quality threshold can be rejected automatically and replaced with standard graph-based explanations.
  • The pipeline works across at least four domains with no per-domain changes other than the exemplars and the context line, suggesting the approach is application-agnostic.
  • Bootstrapped exemplars are safer than hand-written ones for correctness because the Grader filters them for perfect accuracy and completeness before they enter the prompt.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the explanation format is passed separately from the narrative style, the same Narrator–Grader loop should transfer to other explanation types such as counterfactuals or LIME with only rubric changes; the paper lists this as future work rather than a demonstrated result.
  • The paper's reported difficulty with comparative terms like “larger” suggests a concrete extension: pass feature distributions or reference values into both subsystems so that relative language can be scored fairly; this is my extrapolation from the paper's failure analysis, not a tested result.
  • If the Grader is trusted as a guardrail, its calibration becomes a safety property, which is why the decisive next experiment is a held-out human study: none of the main results are evaluated against human labels on narratives that were not used to tune the prompts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces Explingo, a two-part LLM-based system for transforming SHAP feature-contribution explanations into natural-language narratives. The Narrator (GPT-4o) converts explanations into narratives, optionally guided by hand-written and bootstrapped few-shot exemplars; the Grader (also GPT-4o) scores narratives on accuracy, completeness, fluency, and conciseness. Experiments on nine exemplar datasets from four public ML datasets compare three base prompts and ten few-shot settings, reporting that a small number of hand-written and bootstrapped exemplars yields the highest Grader-assessed quality (e.g., H=1, B=3 gives the best total grade in Table IV). The system is integrated into the open-source Pyreal library, and the exemplar datasets are released. The paper's central claim is that LLMs can reliably transform SHAP explanations into high-quality narratives when guided by a small number of examples.

Significance. If the claim is established, this is a useful contribution to explainable AI: it offers a practical pipeline for turning quantitative SHAP explanations into readable narratives, with an automated grader that could serve as a deployment guardrail. The paper is systematic in comparing prompting and few-shot strategies, and it is transparent about known Grader failures on specific datasets. The release of the exemplar datasets and open-source implementation is a concrete asset for follow-up work. However, the main evidence for narrative quality consists entirely of LLM-Grader scores, and the Grader's validity for the final generated narratives is not established by the current evaluation. The paper itself documents several Grader failures, which makes the central claim currently unsupported by the evidence presented.

major comments (4)
  1. [Sections V-A, V-B, and VII] The Grader's accuracy and completeness prompts were tuned on the same 50-example human-labeled sets used to report 96% agreement (Section V-A, Table III), with no held-out set. This means the 96% figure reflects fit to the tuning set, not predictive validity for new narratives. Since all headline results in Tables IV and V are Grader scores on the 2,197 generated narratives, the central claim that LLMs produce high-quality narratives is not yet supported. A human evaluation on a sample of the final generated narratives, or at least a held-out Grader validation set, is required.
  2. [Section VII, Table V] The paper documents Grader failures: a Mush 2 narrative that the authors describe as accurate receives an accuracy grade of 0, and PDF 2 receives accuracy 0.000 in both best-performing settings. The authors attribute these low scores to the Grader rather than the Narrator, and this may be correct, but the data cannot distinguish between 'the Narrator produces low-quality narratives in these domains' and 'the Grader mis-scores high-quality narratives in these domains.' Because Table V is presented as evidence of narrative quality, this ambiguity directly affects the paper's load-bearing claim.
  3. [Section VI] Bootstrapped few-shot exemplars are selected by requiring perfect Grader scores on accuracy, completeness, and fluency, and at least 3.5 on conciseness. If the Grader is biased, as Table V suggests, that bias is propagated into the exemplars used to prompt the Narrator, and subsequently into the Grader-based evaluation of those Narrator outputs. This creates a self-referential optimization loop that makes the reported improvements from bootstrapping in Table IV difficult to interpret as genuine gains in narrative quality. An independent human evaluation is necessary to break this loop.
  4. [Section V-C] The fluency metric is validated only by checking that Grader scores differentiate same-dataset from other-dataset narratives (Fig. 5). This establishes discriminative ability but does not establish agreement with human judgments of style similarity. Since fluency is one of the four reported metrics and contributes to the total grade, the claim that the Narrator 'matches the style' of exemplar narratives is not fully supported. A human study on style similarity, or at least a human agreement measurement for fluency, would strengthen the evaluation.
minor comments (4)
  1. [Section VII, Experimental Setup] The text states '3 base prompts and 11 few-shot (exemplar) settings,' but Table IV lists only 10 few-shot rows (plus 3 base prompts), totaling 13 techniques. The count '11' should be corrected to '10'.
  2. [Section V-D, Eq. (2)] The description of Lmax is unclear: the text says 'we set the maximum input length to 90% of the longest feature description from the exemplar narratives,' while the equation defines Lmax as the ideal number of words per feature. Please clarify how the 90% value maps to Lmax and whether the same Lmax is used across all datasets.
  3. [Section I] The phrase 'for sure as a guardrail in deployment' appears to be a typo; it should likely read 'as a guardrail in deployment.'
  4. [Section V-A] The sentence beginning 'We then started with a basic prompt...' is awkwardly phrased; consider rephrasing for clarity, as the current wording makes the iterative tuning process harder to follow.

Circularity Check

1 steps flagged · score 4.0 of 10

Bootstrapped few-shot exemplars are selected by the same GPT-4o Grader that reports the paper's headline quality scores, creating a partial self-referential loop.

  1. fitted input called prediction [Section VI 'Optimizing the Narrator', bootstrapped few-shot paragraph; outcome reported in Section VII, Table IV]
    "we used DSPy's BootstrapFewShot to create more exemplar narratives that score highly according to our GRADER. To do this we give the bootstrapper: ... a requirement that exemplars it produces achieve a perfect score on accuracy, completeness, and fluency and at least 3.5 on conciseness as measured by our GRADER. ... TABLE IV: Narrative quality scores for narratives generated by our NARRATOR, as evaluated by our GRADER."

    The Grader's score is used twice: once as the selection criterion for bootstrapped few-shot exemplars that steer the Narrator's prompts, and again as the reported measure of narrative quality in Table IV. Any systematic bias of the Grader is therefore baked into the demonstrations and then read back as high-quality output, so the headline 'high scores across all metrics' is partially forced by the optimization target rather than independently verified. The paper itself shows the Grader is not a neutral instrument, giving an accurate Mush 2 narrative a 0 on accuracy and PDF 2 accuracy 0.000 in Table V, while no human evaluation was run on the 2,197 generated narratives.

full rationale

Most of Explingo's pipeline is self-contained: the Narrator transforms externally generated SHAP values, the four metrics are defined in Section V from prior literature, and the accuracy and completeness Grader prompts were checked against 50 human-labeled pairs with 96% agreement. The fluency metric was validated by cross-dataset style differentiation, not by circular reference to the generated narratives. However, the central claim that LLMs generate high-quality narratives is weakened by the bootstrapping loop: Section VI selects few-shot exemplars using the Grader's own scores as a hard requirement, and Section VII reports the same Grader's scores as evidence of quality. The paper's own Table V and discussion acknowledge Grader errors on Mush 2 and PDF 2, which shows the Grader is not a transparent measurement device and that the reported scores are not an independent ground truth. Because no human evaluation was performed on the final generated narratives, the evidence cannot fully separate Narrator quality from Grader bias. This is best characterized as moderate partial circularity in the evaluation loop, while the system contribution, open-source integration, and curated exemplar datasets remain independently valuable and are not themselves circular.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim depends on these chosen weights, thresholds, and assumptions about LLM grader validity. The free parameters are mostly evaluation-setup choices, while the domain assumptions about SHAP, metrics, and grader reliability are the real load-bearing premises.

free parameters (4)
  • Alpha weights in total grade = alpha_a=4, alpha_c=2, alpha_f=1, alpha_s=1
    Set by authors to equalize metric scales (Section V, Eq. 1). Central claim comparisons depend on this weighting.
  • Lmax, max ideal words per feature for conciseness = 90% of the longest feature description from exemplar narratives
    Hyperparameter in Eq. 2; authors choose this value in Section VII.
  • Exemplar counts H and B = H=1, B=3 best; H=1, B=1 close second
    Chosen based on Grader-based evaluation results; the optimal count is an empirical finding, not a principled parameter.
  • Bootstrapping thresholds = Perfect accuracy, completeness, fluency; conciseness >= 3.5
    Thresholds used by DSPy BootstrapFewShot to select exemplars (Section VI).
assumptions (5)
  • domain assumption SHAP values are an appropriate explanation format for ML predictions
    The system is built around SHAP contributions; the paper cites [1] but does not justify this choice.
  • domain assumption The four metrics (accuracy, completeness, fluency, conciseness) capture narrative quality
    Adapted from [28]; alternative metrics exist, and the paper acknowledges this is subjective.
  • domain assumption GPT-4o's grading correlates with human judgments
    Relies on prior LLM-as-judge results [21], [22], [23] and its own 96% agreement on small validation sets.
  • domain assumption Hand-written exemplar narratives are high-quality and representative of user preferences
    The Narrator imitates these exemplars, so their quality bounds the output; the paper notes some exemplars may be inaccurate.
  • domain assumption DSPy's BootstrapFewShot produces valid training exemplars from Grader feedback
    Assumes the DSPy optimization loop and the Grader scores are reliable guides for exemplar selection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Explingo: Explaining AI Predictions using Large Language Models." pith.science (2026). https://pith.science/paper/SGH66KHX

@misc{pith2026241205145,
  author       = {Pith},
  title        = {Pith review of: Explingo: Explaining AI Predictions using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SGH66KHX}},
  note         = {Machine review of arXiv:2412.05145}
}
read the original abstract

Explanations of machine learning (ML) model predictions generated by Explainable AI (XAI) techniques such as SHAP are essential for people using ML outputs for decision-making. We explore the potential of Large Language Models (LLMs) to transform these explanations into human-readable, narrative formats that align with natural communication. We address two key research questions: (1) Can LLMs reliably transform traditional explanations into high-quality narratives? and (2) How can we effectively evaluate the quality of narrative explanations? To answer these questions, we introduce Explingo, which consists of two LLM-based subsystems, a Narrator and Grader. The Narrator takes in ML explanations and transforms them into natural-language descriptions. The Grader scores these narratives on a set of metrics including accuracy, completeness, fluency, and conciseness. Our experiments demonstrate that LLMs can generate high-quality narratives that achieve high scores across all metrics, particularly when guided by a small number of human-labeled and bootstrapped examples. We also identified areas that remain challenging, in particular for effectively scoring narratives in complex domains. The findings from this work have been integrated into an open-source tool that makes narrative explanations available for further applications.

Figures

Figures reproduced from arXiv: 2412.05145 by the authors.

Figure 1
Figure 1. Sample NARRATOR inputs and outputs. The items in blue make up the prompt passed to the NARRATOR to transform ML explanations into narratives for a house-pricing example. The item in green is a narrative the NARRATOR LLM may generate based on this prompt. Some components, like the output instructions, are provided by DSPy [6]. shot strategies on narrative generation, and evaluate po￾tential sources of reduced narrati… view at source ↗
Figure 2
Figure 2. Prompt passed to the GRADER to compute the accuracy metric. Experiments suggested that asking for a grade from a 2-point rubric resulted in better performance than a simple yes/no question. We saw that the GRADER incorrectly considered narratives that were accurate but did not include all feature values from the input explanation as inaccurate, and regularly did not notice when contribution directions were wrong if … view at source ↗
Figure 3
Figure 3. Prompt passed to the GRADER to compute the completeness metric. We found that while the original version of the prompt allowed the LLM to correctly identify missing feature values, it was regularly not identifying when features were missing altogether or did not have their feature directions listed. Through experimentation, we determined that a chain-of￾thought [30] approach that explicitly asked the GRADER to list … view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Prompt passed to the GRADER to compute the fluency metric. We found that the GRADER was over-emphasizing narrative content, as shown by a significant difference in score between narratives from different datasets with original prompt (How well does the narrative match …
Figure 5
Figure 5. Figure 5: Difference between fluency scores on narratives com [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Correlation between mean metric grades (color) over [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Open-source library implementation of narrative gener [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 17 canonical work pages

  1. [1]

    A unified approach to interpreting model predictions,

    S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” Advances in neural information processing systems, vol. 30, 2017

    2017

  2. [2]

    Rethinking Explainability as a Dialogue: A Practitioner’s Perspective,

    H. Lakkaraju, D. Slack, Y . Chen, C. Tan, and S. Singh, “Rethinking Explainability as a Dialogue: A Practitioner’s Perspective,” Feb. 2022, arXiv:2202.01875 [cs]. [Online]. Available: http://arxiv.org/abs/2202. 01875

    arXiv 2022

  3. [3]

    LLMs for XAI: Future Directions for Explaining Explanations,

    A. Zytek, S. Pid `o, and K. Veeramachaneni, “LLMs for XAI: Future Directions for Explaining Explanations,” May 2024, arXiv:2405.06064 [cs]. [Online]. Available: http://arxiv.org/abs/2405.06064

    arXiv 2024

  4. [4]

    Towards LLM-guided Causal Explainability for Black-box Text Classifiers,

    A. Bhattacharjee, R. Moraffah, J. Garland, and H. Liu, “Towards LLM-guided Causal Explainability for Black-box Text Classifiers,” in AAAI ReLM 2024 . AAAI, 2024. [Online]. Available: http: //arxiv.org/abs/2309.13340

    arXiv 2024

  5. [5]

    Are Large Language Models Post Hoc Explainers?

    N. Kroeger, D. Ley, S. Krishna, C. Agarwal, and H. Lakkaraju, “Are Large Language Models Post Hoc Explainers?” in NeurIPS XAIA 2023. arXiv, 2023. [Online]. Available: http://arxiv.org/abs/2310.05797

    arXiv 2023

  6. [6]

    Dspy: Compiling declarative language model calls into self-improving pipelines,

    O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts, “Dspy: Compiling declarative language model calls into self-improving pipelines,” 2023. [Online]. Available: https://arxiv.org/abs/2310.03714

    arXiv 2023

  7. [7]

    Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making,

    A. Zytek, D. Liu, R. Vaithianathan, and K. Veeramachaneni, “Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making,” IEEE Transactions on Visualization and Computer Graphics, pp. 1–1, 2021, conference Name: IEEE Transactions on Visualization and Computer Graphics

    work page 2021

  8. [8]

    More Questions than Answers? Lessons from Integrating Explainable AI into a Cyber-AI Tool

    A. Suh, H. Li, C. Kenney, K. Alperin, and S. R. Gomez, “More Questions than Answers? Lessons from Integrating Explainable AI into a Cyber-AI Tool,” in CHI HCXAI . ACM, 2023. [Online]. Available: http://arxiv.org/abs/2408.04746

    work page Pith review arXiv 2023

Show all 30 references
  1. [9]

    Explainable machine learning in deployment,

    U. Bhatt, A. Xiang, S. Sharma, A. Weller, A. Taly, Y . Jia, J. Ghosh, R. Puri, J. M. F. Moura, and P. Eckersley, “Explainable machine learning in deployment,” in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , ser. FAT* ’20. New York, NY , US...

    2020

  2. [10]

    On Two XAI Cultures: A Case Study of Non- technical Explanations in Deployed AI System,

    H. Jiang and E. Senge, “On Two XAI Cultures: A Case Study of Non- technical Explanations in Deployed AI System,” in Human Centered AI (HCAI) workshop at NeurIPS 2021 , Dec. 2021, arXiv:2112.01016 [cs]. [Online]. Available: http://arxiv.org/abs/2112.01016

    2021 arXiv

  3. [11]

    Survey on explainable ai: From approaches, lim- itations and applications aspects,

    W. Yang, Y . Wei, H. Wei, Y . Chen, G. Huang, X. Li, R. Li, N. Yao, X. Wang, X. Gu et al., “Survey on explainable ai: From approaches, lim- itations and applications aspects,” Human-Centric Intelligent Systems , vol. 3, no. 3, pp. 161–188, 2023

    2023

  4. [12]

    From black boxes to conversations: Incorporating xai in a conversational agent,

    V . B. Nguyen, J. Schl ¨otterer, and C. Seifert, “From black boxes to conversations: Incorporating xai in a conversational agent,” in World Conference on Explainable Artificial Intelligence . Springer, 2023, pp. 71–96

    2023

  5. [13]

    ConvXAI : Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing,

    H. Shen, C.-Y . Huang, T. Wu, and T.-H. K. Huang, “ConvXAI : Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing,” Computer Supported Cooperative Work and Social Computing , pp. 384–387, Oct. 2023

    2023

  6. [14]

    Explaining machine learning models with interactive natural language conversations using talktomodel,

    D. Slack, S. Krishna, H. Lakkaraju, and S. Singh, “Explaining machine learning models with interactive natural language conversations using talktomodel,” Nature Machine Intelligence , vol. 5, no. 8, pp. 873–883, 2023

    2023

  7. [15]

    Natural language expla- nations for machine learning classification decisions,

    J. Burton, N. Al Moubayed, and A. Enshaei, “Natural language expla- nations for machine learning classification decisions,” in 2023 Interna- tional Joint Conference on Neural Networks (IJCNN) . IEEE, 2023, pp. 1–9

    2023

  8. [16]

    Adapting Prompt for Few-shot Table-to-Text Generation,

    Z. Guo, M. Yan, J. Qi, J. Zhou, Z. He, Z. Lin, G. Zheng, and X. Wang, “Adapting Prompt for Few-shot Table-to-Text Generation,” Aug. 2023

    2023

  9. [17]

    Evaluating Large Language Models: A Comprehensive Survey,

    Z. Guo, R. Jin, C. Liu, Y . Huang, D. Shi, Supryadi, L. Yu, Y . Liu, J. Li, B. Xiong, and D. Xiong, “Evaluating Large Language Models: A Comprehensive Survey,” Nov. 2023, arXiv:2310.19736 [cs]. [Online]. Available: http://arxiv.org/abs/2310.19736

    2023 arXiv

  10. [18]

    Benchmarking Foundation Models with Language-Model-as-an-Examiner,

    Y . Bai, J. Ying, Y . Cao, X. Lv, Y . He, X. Wang, J. Yu, K. Zeng, Y . Xiao, H. Lyu, J. Zhang, J. Li, and L. Hou, “Benchmarking Foundation Models with Language-Model-as-an-Examiner,” in NeurIPS 2023 Datasets and Benchmarks. arXiv, Nov. 2023

    2023

  11. [19]

    Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study,

    Y . Chen, R. Wang, H. Jiang, S. Shi, and R. Xu, “Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study,” Sep. 2023, arXiv:2304.00723 [cs]. [Online]. Available: http://arxiv.org/abs/2304.00723

    2023 arXiv

  12. [20]

    Exploring ChatGPT’s Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences,

    Y . Ji, Y . Gong, Y . Peng, C. Ni, P. Sun, D. Pan, B. Ma, and X. Li, “Exploring ChatGPT’s Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences,” Mar. 2023, arXiv:2303.07610 [cs]. [Online]. Available: http://arxiv.org/abs/2303.07610

    2023 arXiv

  13. [21]

    GPTScore: Evaluate as You Desire,

    J. Fu, S.-K. Ng, Z. Jiang, and P. Liu, “GPTScore: Evaluate as You Desire,” Feb. 2023, arXiv:2302.04166 [cs]. [Online]. Available: http://arxiv.org/abs/2302.04166

    2023 arXiv

  14. [22]

    Is ChatGPT a Good NLG Evaluator? A Preliminary Study,

    J. Wang, Y . Liang, F. Meng, Z. Sun, H. Shi, Z. Li, J. Xu, J. Qu, and J. Zhou, “Is ChatGPT a Good NLG Evaluator? A Preliminary Study,” in NewSumm@EMNLP. arXiv, Oct. 2023

    2023

  15. [23]

    G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment,

    Y . Liu, D. Iter, Y . Xu, S. Wang, R. Xu, and C. Zhu, “G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment,” May 2023, arXiv:2303.16634 [cs]. [Online]. Available: http://arxiv.org/abs/2303. 16634

    2023 arXiv

  16. [24]

    Ames, Iowa: Alternative to the Boston Housing Data as an End of Semester Regression Project,

    D. De Cock, “Ames, Iowa: Alternative to the Boston Housing Data as an End of Semester Regression Project,” Journal of Statistics Education , vol. 19, no. 3, Nov. 2011. [Online]. Available: https://www.tandfonline.com/doi/full/10.1080/10691898.2011.11889627

    2011

  17. [25]

    Using data mining to predict secondary school student performance,

    P. Cortez and A. Silva, “Using data mining to predict secondary school student performance,” EUROSIS, Jan. 2008

    2008

  18. [26]

    Mushroom,

    “Mushroom,” UCI Machine Learning Repository, 1987, DOI: https://doi.org/10.24432/C5959T

    1987 doi

  19. [27]

    PDF Malware Detection based on Stacking Learning:,

    M. Issakhani, P. Victor, A. Tekeoglu, and A. Lashkari, “PDF Malware Detection based on Stacking Learning:,” in Proceedings of the 8th International Conference on Information Systems Security and Privacy . SCITEPRESS - Science and Technology Publications, 2022, pp. 562– 570

    2022

  20. [28]

    Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics,

    J. Zhou, A. H. Gandomi, F. Chen, and A. Holzinger, “Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics,” Electronics, vol. 10, no. 5, p. 593, Jan. 2021

    2021

  21. [29]

    METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,” in Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , J. Goldstein, A. Lavie, C....

    2005

  22. [30]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” Jan. 2023, arXiv:2201.11903 [cs]. [Online]. Available: http://arxiv.org/abs/2201.11903

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.