{"id":"4c93b742-851a-4d56-8d15-37899363fca6","arxiv_id":"2412.05145","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Explingo uses GPT-4o to convert SHAP explanations into natural-language narratives and to grade them on four quality metrics, with a few exemplars improving style but nudging correctness down.","lead":"This paper introduces Explingo, a two-part LLM system that turns SHAP explanations of ML predictions into readable narratives and automatically grades those narratives on accuracy, completeness, fluency, and conciseness. It reports that a few hand-written and bootstrapped examples guide GPT-4o to produce highly scored narratives, while flagging areas where automated grading still fails.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on Grader scores as evidence of narrative quality, but Grader validity is untested on the actual generated narratives, and the paper's own Table V shows Grader failures on two datasets.","rationale":"The reader's weakest assumption is exactly the load-bearing point: the Grader's LLM-based grades are treated as valid measures of narrative quality without a held-out human validation on the actual Narrator outputs. My review agrees with that identification. The paper is transparent about its limitations, including the admission that the Grader mis-scored accurate Mush 2 narratives and struggled with comparative terms in PDF 2, and the open-source artifacts and reproducible pipeline are real contributions. However, those admissions undercut the strength of the central claim: the reported 'high scores' are Grader scores, and for two datasets even the Grader's own scores are far below 'high'. A human evaluation of the final generated narratives is the single experiment that would settle whether the central claim holds. I do not see a more fundamental logical flaw in the system design; the missing experiment is the issue. Since the reader already assigned CONDITIONAL with high confidence and identified this concern, no verdict change is needed, but the revision should either add the human evaluation or weaken the abstract's claim about high-quality narratives across metrics.","tokens_in":16348,"tokens_out":3716,"duration_ms":41735,"concrete_test":"Run a held-out human evaluation on the generated narratives, oversampling the datasets where Grader scores are suspicious. Have at least two independent human annotators score a stratified sample (e.g., 60 narratives from the H=1, B=3 condition, including all Mush 2 and PDF 2 narratives) using the same accuracy, completeness, and fluency rubrics as the Grader. Compute human-human agreement and Grader-human agreement (e.g., Cohen's kappa and exact-match accuracy). If Grader-human agreement on these final generated narratives is close to the 96% reported on the tuning set and human ratings reproduce the Table IV/V pattern, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim that LLMs can generate high-quality narratives is supported almost entirely by Grader scores in Tables IV and V. That support is load-bearing on the Grader being a valid measure of narrative quality for the final generated narratives, and this is the least secure step in the argument. The Grader was tuned and validated on the same 50 curated explanation-narrative pairs used to iteratively adjust the prompts, with no held-out set; human agreement of 96% is reported on that same tuning set, not on any of the 2,197 Narrator-generated narratives. Fluency validation used style differentiation between datasets, not human ratings. Bootstrapped exemplars are admitted only if the Grader gives perfect accuracy/completeness/fluency scores (Section VI), so any Grader bias is propagated into the very examples used to steer the Narrator. The paper itself documents Grader failures: in Section VII and Table V, an accurate Mush 2 narrative receives accuracy 0, and PDF 2 receives accuracy 0.000 in both best-performing settings, which the authors attribute to the Grader rather than the Narrator. Because no human evaluation was run on the final generated narratives, the data cannot distinguish between 'the Narrator produces low-quality narratives for Mush 2 and PDF 2' and 'the Grader mis-scores high-quality narratives in these domains'. The central claim is therefore not currently established by the evidence presented. This is an internal evidence gap, not a disagreement with consensus: it can be closed by a human evaluation of the actual outputs.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Explingo, a two-part LLM-based system for transforming SHAP feature-contribution explanations into natural-language narratives. The Narrator (GPT-4o) converts explanations into narratives, optionally guided by hand-written and bootstrapped few-shot exemplars; the Grader (also GPT-4o) scores narratives on accuracy, completeness, fluency, and conciseness. Experiments on nine exemplar datasets from four public ML datasets compare three base prompts and ten few-shot settings, reporting that a small number of hand-written and bootstrapped exemplars yields the highest Grader-assessed quality (e.g., H=1, B=3 gives the best total grade in Table IV). The system is integrated into the open-source Pyreal library, and the exemplar datasets are released. The paper's central claim is that LLMs can reliably transform SHAP explanations into high-quality narratives when guided by a small number of examples.","tokens_in":16752,"tokens_out":4386,"duration_ms":40483,"significance":"If the claim is established, this is a useful contribution to explainable AI: it offers a practical pipeline for turning quantitative SHAP explanations into readable narratives, with an automated grader that could serve as a deployment guardrail. The paper is systematic in comparing prompting and few-shot strategies, and it is transparent about known Grader failures on specific datasets. The release of the exemplar datasets and open-source implementation is a concrete asset for follow-up work. However, the main evidence for narrative quality consists entirely of LLM-Grader scores, and the Grader's validity for the final generated narratives is not established by the current evaluation. The paper itself documents several Grader failures, which makes the central claim currently unsupported by the evidence presented.","major_comments":[{"comment":"The Grader's accuracy and completeness prompts were tuned on the same 50-example human-labeled sets used to report 96% agreement (Section V-A, Table III), with no held-out set. This means the 96% figure reflects fit to the tuning set, not predictive validity for new narratives. Since all headline results in Tables IV and V are Grader scores on the 2,197 generated narratives, the central claim that LLMs produce high-quality narratives is not yet supported. A human evaluation on a sample of the final generated narratives, or at least a held-out Grader validation set, is required.","section":"Sections V-A, V-B, and VII"},{"comment":"The paper documents Grader failures: a Mush 2 narrative that the authors describe as accurate receives an accuracy grade of 0, and PDF 2 receives accuracy 0.000 in both best-performing settings. The authors attribute these low scores to the Grader rather than the Narrator, and this may be correct, but the data cannot distinguish between 'the Narrator produces low-quality narratives in these domains' and 'the Grader mis-scores high-quality narratives in these domains.' Because Table V is presented as evidence of narrative quality, this ambiguity directly affects the paper's load-bearing claim.","section":"Section VII, Table V"},{"comment":"Bootstrapped few-shot exemplars are selected by requiring perfect Grader scores on accuracy, completeness, and fluency, and at least 3.5 on conciseness. If the Grader is biased, as Table V suggests, that bias is propagated into the exemplars used to prompt the Narrator, and subsequently into the Grader-based evaluation of those Narrator outputs. This creates a self-referential optimization loop that makes the reported improvements from bootstrapping in Table IV difficult to interpret as genuine gains in narrative quality. An independent human evaluation is necessary to break this loop.","section":"Section VI"},{"comment":"The fluency metric is validated only by checking that Grader scores differentiate same-dataset from other-dataset narratives (Fig. 5). This establishes discriminative ability but does not establish agreement with human judgments of style similarity. Since fluency is one of the four reported metrics and contributes to the total grade, the claim that the Narrator 'matches the style' of exemplar narratives is not fully supported. A human study on style similarity, or at least a human agreement measurement for fluency, would strengthen the evaluation.","section":"Section V-C"}],"minor_comments":[{"comment":"The text states '3 base prompts and 11 few-shot (exemplar) settings,' but Table IV lists only 10 few-shot rows (plus 3 base prompts), totaling 13 techniques. The count '11' should be corrected to '10'.","section":"Section VII, Experimental Setup"},{"comment":"The description of Lmax is unclear: the text says 'we set the maximum input length to 90% of the longest feature description from the exemplar narratives,' while the equation defines Lmax as the ideal number of words per feature. Please clarify how the 90% value maps to Lmax and whether the same Lmax is used across all datasets.","section":"Section V-D, Eq. (2)"},{"comment":"The phrase 'for sure as a guardrail in deployment' appears to be a typo; it should likely read 'as a guardrail in deployment.'","section":"Section I"},{"comment":"The sentence beginning 'We then started with a basic prompt...' is awkwardly phrased; consider rephrasing for clarity, as the current wording makes the iterative tuning process harder to follow.","section":"Section V-A"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and useful problem, and the open-source release is valuable. The main concern is the validity of the LLM-based Grader as the sole evidence for the central claim. I would encourage the editor to request that the authors either run a human evaluation on a sample of the generated narratives (even a few hundred) or provide a held-out validation of the Grader on narratives outside the tuning set. Without that, the headline claim rests on a self-referential evaluation loop. The paper is otherwise well structured and the per-dataset reporting in Table V is commendable for its transparency."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Sarah, quick take on Explingo (2412.05145). The real contribution is a concrete, open-source pipeline for turning SHAP values into readable narratives, with a multi-metric LLM grader plus bootstrapped few-shot selection. The authors also report a useful empirical trade-off: more exemplars buy fluency and conciseness but cost accuracy and completeness. That is worth knowing.\n\nWhat I like: the system is reproducible — code, exemplar datasets, and cost estimates are all public. The paper is also unusually honest: it documents Grader failures on Mush 2 and PDF 2, and explicitly says the low scores appear to be Grader errors, not Narrator errors. That kind of transparency is rare.\n\nThe soft spot is the evaluation loop. The Grader is a GPT-4o LLM that was tuned and validated on the same 50-example sets used to report 96% agreement; there is no held-out test set for the Grader. The final 2,197 narratives were never shown to humans. The bootstrapped exemplars are selected by the same Grader, so any bias in the Grader propagates into the examples that steer the Narrator. The paper's own Table V shows Grader failures on two datasets. So the headline claim — 'LLMs can generate high-quality narratives that achieve high scores across all metrics' — is not actually established for the final outputs. It is established that the narratives score well according to an LLM Grader whose reliability on those very outputs was never checked.\n\nThat said, this is an internal evidence gap, not a fatal flaw. The authors acknowledge the limitation and call for further investigation. For a systems paper, the artifacts are solid and the approach is sensible. The missing piece is a modest human evaluation of, say, 100 sampled narratives across the four metrics; that would close the loop.\n\nWho should read this: anyone working on LLM-based explanation or using LLMs as evaluators. It is a good cautionary case study on grader validation. I would send it to a serious referee if I were the editor, with a request to push for human evaluation and to tone down the abstract. My own verdict: conditional accept, with the evaluation gap clearly flagged.","headline":"A useful open-source pipeline for narrative SHAP explanations, but the headline claim leans on an unvalidated LLM grader and needs human evaluation on the actual outputs.","tokens_in":17226,"tokens_out":2488,"would_cite":true,"duration_ms":23611,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that one large language model can turn SHAP explanations into fluent, accurate narratives, and a second LLM can grade those narratives on accuracy, completeness, fluency, and conciseness.","keywords":["explainable AI","SHAP","large language models","narrative explanations","LLM evaluation","few-shot prompting","automated grading","natural language explanations"],"falsifier":"Collect a fresh set of 100 generated narratives from the same datasets and domains, have several human annotators grade them under the paper's rubrics, and compare their average ratings with the Grader's scores on narratives not used in prompt tuning. If agreement is no better than chance, especially on accuracy, the evidence for high-quality narratives collapses. A faster check is to reproduce the reported Mush 2 failure, where a narrative that faithfully restated the SHAP direction and values received an accuracy score of 0.","tokens_in":16123,"feed_emoji":"📝","tokens_out":6993,"duration_ms":70919,"temperature":0.7,"pith_summary":"Explingo claims that a large language model can turn SHAP feature-contribution tables into natural-language narratives that preserve the numbers and direction of the original explanation, and that a second LLM can grade those narratives on accuracy, completeness, fluency, and conciseness. The paper's evidence comes from experiments across nine exemplar datasets built from four public prediction tasks (house prices, student pass/fail, mushroom toxicity, and PDF malware), in which the Narrator's best configurations reached average total grades near 15 out of 16 according to the Grader. If true, this would let non-experts read model explanations without wading through SHAP tables: a user would write only a few example narratives in the desired style, and the same prompt would produce and quality-check readable explanations for future predictions. The paper also identifies where the pipeline currently fails, most notably when narratives use comparative language and when the Grader must score complex domains.","feed_headline":"LLM narrators turn SHAP tables into plain-language stories","feed_subtitle":"A small set of style examples yields fluent, accurate narratives; a second LLM checks their quality.","key_machinery":"The load-bearing machinery is a pair of LLM prompts. The Narrator receives the SHAP explanation parsed into triples of feature name, feature value, and SHAP contribution, along with a format descriptor, a one-line context such as “the model predicts house prices,” and a handful of exemplar narratives; it outputs a single narrative. The Grader receives an explanation, a narrative, and a rubric for one metric, and returns a numeric grade: accuracy and completeness use explicit chain-of-thought instructions to list features one by one, fluency compares the narrative against five exemplars, and conciseness is computed deterministically from word count. The two are tied together by a weighted total grade $G = \\alpha_a A + \\alpha_f F + \\alpha_c C + \\alpha_s S$, and the Narrator is optimized by bootstrapping new exemplars that receive perfect Grader scores. That feedback loop—Grader selects exemplars, exemplars shape Narrator output, Narrator output is graded again—is what carries the argument from “LLMs can write prose” to “LLMs can write and check application-specific explanations.”","core_discovery":"The central discovery, on the paper's own terms, is that generation is not the bottleneck: evaluation is. With no exemplars, the Narrator already produces perfectly accurate and complete narratives, scoring 4.0 out of 4.0 on both metrics, but fluency and conciseness are low. Adding one hand-written exemplar and three bootstrapped exemplars raises the total grade from about 11.3 to 15.0 out of 16, at only a small cost to accuracy and completeness. The Grader, after prompt tuning with chain-of-thought instructions and explicit rubrics, agrees with human labels on 96% of the 50-example accuracy and completeness validation sets, and can reliably distinguish narrative styles once five exemplars are supplied. The paper's main claim is therefore that LLMs can reliably transform SHAP explanations into high-quality narratives, particularly when guided by a small number of human-labeled and bootstrapped examples.","pith_inferences":["Because the explanation format is passed separately from the narrative style, the same Narrator–Grader loop should transfer to other explanation types such as counterfactuals or LIME with only rubric changes; the paper lists this as future work rather than a demonstrated result.","The paper's reported difficulty with comparative terms like “larger” suggests a concrete extension: pass feature distributions or reference values into both subsystems so that relative language can be scored fairly; this is my extrapolation from the paper's failure analysis, not a tested result.","If the Grader is trusted as a guardrail, its calibration becomes a safety property, which is why the decisive next experiment is a held-out human study: none of the main results are evaluated against human labels on narratives that were not used to tune the prompts."],"forward_implications":["A user needs to write only around four exemplar narratives (one hand-written plus three bootstrapped) to reach the highest measured quality; adding more exemplars improves style but measurably lowers accuracy and completeness.","The Grader can double as a deployment guardrail: narratives below a quality threshold can be rejected automatically and replaced with standard graph-based explanations.","The pipeline works across at least four domains with no per-domain changes other than the exemplars and the context line, suggesting the approach is application-agnostic.","Bootstrapped exemplars are safer than hand-written ones for correctness because the Grader filters them for perfect accuracy and completeness before they enter the prompt."],"supporting_citations":[{"why":"Supplies the SHAP feature-contribution explanations that the Narrator rewrites into prose.","marker":"[1]"},{"why":"Provides the four metric definitions (accuracy, completeness, fluency, conciseness) that the Grader implements.","marker":"[28]"},{"why":"Chain-of-thought prompting is used in the final Grader prompts to fix missing-feature and contribution-direction errors.","marker":"[30]"},{"why":"Prior evidence that LLM-based graders can align with human judgments motivates the automated Grader design.","marker":"[23]"}],"fun_headline_variants":["Explaining AI with LLM narrators: evaluation is the hard part","From SHAP to stories: LLMs narrate, but grading lags","Bootstrapped LLM narrators ace fluency with just 4 examples","LLMs write AI explanations, but the real test is judging them"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The main claim rests on trusting the Grader's scores as a valid measure of narrative quality, and the Grader was verified against human labels only on the small sets used to tune its prompts, with no held-out evaluation for the main results.","fun_headline_variants_meta":{"raw":{"variants":["Explaining AI with LLM narrators: evaluation is the hard part","From SHAP to stories: LLMs narrate, but grading lags","Bootstrapped LLM narrators ace fluency with just 4 examples","LLMs write AI explanations, but the real test is judging them"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000127,"raw_usage":{"total_tokens":1115,"prompt_tokens":943,"completion_tokens":172,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":93}},"tokens_in":559,"tokens_out":172,"duration_ms":2369,"temperature":1.0,"reasoning_tokens":93,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:48:54.854828+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a fresh set of 100 generated narratives from the same datasets and domains, have several human annotators grade them under the paper's rubrics, and compare their average ratings with the Grader's scores on narratives not used in prompt tuning. If agreement is no better than chance, especially on accuracy, the evidence for high-quality narratives collapses. A faster check is to reproduce the reported Mush 2 failure, where a narrative that faithfully restated the SHAP direction and values received an accuracy score of 0.","supporting_citations":[{"cited_title":"Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics,","cited_arxiv_id":null,"evidence_quote":"Provides the four metric definitions (accuracy, completeness, fluency, conciseness) that the Grader implements."}],"review_version":1}