{"id":"85c09b0b-b140-4d07-9890-1082b4eaf01b","arxiv_id":"2608.08126","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A constrained-prompt LLM credit-risk explainer inverted the sign of three of four supplied drivers in an audited case, while the stacking ensemble's AUC gain over random forest was real but operationally small.","lead":"Using a public credit dataset, this paper builds a stacking ensemble and an LLM-based explanation system, then shows the model's generated rationales can contradict the attributions they were given, including reversing risk direction. The result matters because regulated credit decisions increasingly rely on LLM explanations, and this is concrete evidence that input-side prompt controls do not guarantee faithful narratives.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The audited sign-inversion count is inflated: per Appendix A, only top-4 drivers carry signed impacts, so person_age/person_income were not 'supplied' with the signs Table V audits.","rationale":"In good faith, the paper is a careful empirical study and its general conclusion—that input-side constraints such as constrained prompting and greedy decoding do not guarantee faithful narratives—is plausible and well motivated. The reader's weakest assumption points to the validity of the fused SHAP+LIME attribution as ground truth, and that is a real issue. My stress-test identifies a more direct and more easily settled problem: the audit's own table treats attributions as 'supplied in the prompt' when the paper's prompt specification says they were not. Only the top four drivers receive signed impacts; person_income and person_age, whose supposed sign inversions dominate the headline, fall outside that set. This makes the central existence proof weaker than stated, though not empty. The clean evidence remains that loan_intent_VENTURE was supplied with a negative sign and the narrative called it risk-increasing, and the top driver loan_amnt was omitted. The membership criterion does catch age and income as features named outside the driver set, but the paper's language conflates membership violations with sign inversions of supplied evidence. A revision should separate these categories and count them accurately. Because the underlying practical recommendation—verify narratives after generation—does not depend on the inflated count, the reader's conditional verdict remains appropriate; no change of verdict is needed.","tokens_in":18208,"tokens_out":7321,"duration_ms":75260,"concrete_test":"Regenerate the exact prompt for the Table V worked case from the released code and Appendix A, and record which features actually carry signed impacts in the prompt's driver block. Then recompute the audit's direction and membership counts using only those signed entries. If person_income and person_age appear without signs, revise the paper's claim to one signed inversion (loan_intent_VENTURE) plus the coverage failure, and remove 'feature never given to it' for person_age. Optionally, also recompute the audit using raw SHAP and raw LIME signs separately to check whether the remaining sign-inversion claim is robust to the fusion reference.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central exhibit (Section X.B, Table V) claims that the generated narrative inverts the sign of supplied evidence for age, income, and venture intent. But the prompt template in Appendix A and the description in Section V.B say that signed impacts are provided only for the top four local feature drivers. In Table V, the top-four set is loan_amnt (+0.3297), person_home_ownership_RENT (+0.1179), person_home_ownership_OWN (+0.0815), and loan_intent_VENTURE (−0.0720). person_income (−0.0569) and person_age (−0.0092) are therefore fifth and sixth; they are not among the signed entries handed to the language model. They may appear in the raw profile, global context, or sensitive-features lists, but those lists carry no signs. Consequently, the narrative's statements about age and income are membership violations under the paper's own audit definition, not sign inversions of supplied attributions. The cleanly supported sign inversion is loan_intent_VENTURE alone. The claim that the model 'introduced a feature never given to it' is also inaccurate because person_age appears in the raw applicant profile in the prompt. The broad conclusion that constrained prompting is insufficient still survives on the venture-intent inversion and the omission of loan_amnt, but the headline 'three of four checkable propositions invert the sign of the evidence' is materially overstated. This concern is internal to the paper's own prompt specification and does not depend on whether the fused SHAP+LIME sign is the correct ground truth.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper builds an end-to-end credit-risk system consisting of a multi-scale stacking ensemble (four gradient-boosting learners plus a residual MLP fused by a neural meta-learner trained on out-of-fold predictions) and an LLM-based explanation layer that feeds the top-four fused SHAP/LIME drivers to Phi-2 under a constrained prompt. The predictive claim is that the ensemble reaches test ROC-AUC 0.9539 and PR-AUC 0.9137, with a statistically significant but operationally small improvement over a tuned random forest. The explanatory claim is that, despite constrained prompting, the LLM-generated narrative in an audited case inverted the sign of three of four supplied drivers, omitted the dominant driver, and introduced a feature never given to it. The paper also reports trustworthiness diagnostics showing calibration and perturbation stability below the authors' own thresholds and concludes that constrained prompting is necessary but not sufficient, so grounding must be verified after generation.","tokens_in":18546,"tokens_out":6378,"duration_ms":58812,"significance":"If the narrative-failure finding is correct, it is an important negative result for a widely proposed architecture: input-side constraints on LLM-generated credit explanations do not guarantee grounded output, and cheap post-generation audits are necessary. The paper's strengths include leakage-controlled preprocessing, out-of-fold meta-features, conservative AUC significance testing operating under an independence assumption that disadvantages the authors' own claim, an operating-point cost analysis that qualifies the AUC gain, released reproduction scripts, and honest reporting of failures (miscalibration, perturbation instability, sign instability). The audit methodology (membership, direction, coverage) is simple and transferable. However, the strength of the central exhibit is reduced by an internal inconsistency between the prompt specification and the audit's sign-inversion count, as detailed below.","major_comments":[{"comment":"The sign-inversion count is inflated because the prompt supplies signed impacts only for the top four drivers. In Table V the top-four set is {loan_amnt, person_home_ownership_RENT, person_home_ownership_OWN, loan_intent_VENTURE}; person_income (−0.0569) and person_age (−0.0092) are fifth and sixth and therefore were not among the signed entries in the prompt. Consequently, the narrative's statements about age and income are unsupported directional assertions, not inversions of supplied evidence. The only cleanly supported sign inversion is loan_intent_VENTURE (−0.0720 supplied as decreasing risk, narrative asserts increasing). The coverage failure (loan_amnt omitted) is also supported. The headline 'three of four checkable propositions invert the sign of the evidence' (Abstract, Section X.B, Section XIV) must be corrected, because it is material to the paper's central claim.","section":"Section X.B / Table V / Appendix A"},{"comment":"The claim that the model 'introduced a feature never given to it' is contradicted by the prompt template in Appendix A, which includes the raw applicant profile with person_age. The model was given person_age; it was not given person_age in the signed driver list. Section X.B item 3 states this correctly ('not among the four drivers supplied'), but the abstract and conclusion overstate it. Please revise to avoid the inaccurate 'never given' phrasing.","section":"Abstract and Section XIV"},{"comment":"The audit uses the fused SHAP+LIME sign as the ground truth that the narrative must match, but the paper's own measurements show this reference is fragile: SHAP and LIME agree on set membership (overlap@10 = 0.80) but not on order (tau = 0.43, p = 0.18), and the modal-sign share for person_income is 0.53 across instances. For loan_intent_VENTURE, the sign is relatively stable (79% modal-sign share in Fig. 8a), so the single clean inversion is robust. For person_income, a 'sign inversion' claim is of questionable meaning because the fusion reference itself is near-random in direction. The audit definition should explicitly state that the reference is the fused attribution, not a verified ground-truth direction, and the paper should temper the language accordingly.","section":"Sections V.A, IX.C, X.A"}],"minor_comments":[{"comment":"References [10]–[13] are listed without author names; please complete them.","section":"References"},{"comment":"The denominator of the Hanley–McNeil variance term appears as '1/n+n−'; this should be 1/(n_+ n_−) with n_+ and n_− the numbers of positive and negative cases.","section":"Section VI.B, Eq. (7)"},{"comment":"Table V would be clearer if the 'Impact' column explicitly stated that these are fused min–max-normalized SHAP/LIME scores, not raw attribution values.","section":"Table V"},{"comment":"Section IX.C reports Kendall tau and Spearman rho with p-values for n = 8; please state in the text that n is the number of features in the common top-10 set.","section":"Section IX.C"},{"comment":"SHAP attributions are computed for 100 test instances and LIME for 50; please justify the different sample sizes.","section":"Section V.A"},{"comment":"The x-axis label 'mean |Δp| under ± perturbation' should specify the perturbation strength (0.01) and units.","section":"Figure 8(b)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's self-correction regarding the earlier 0.96 AUC figure is commendable. The reference list contains several anonymous preprints ([10]–[13]) that appear to be placeholders; these will need to be properly attributed before publication. The 'three of four' phrasing in the abstract is likely to attract scrutiny and should be corrected in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is an honest, well-executed empirical study of a stacked credit-risk model plus an LLM explanation layer. The predictive half is not the point: the ensemble beats a tuned random forest by 0.014 AUC but saves only six defaults at the chosen threshold, and they report that plainly. The point is the audit: cheap membership/direction/coverage checks for LLM-generated rationales, applied to a concrete case where the narrative inverts a supplied sign, omits the dominant driver, and asserts a feature outside the driver set. That is a real existence proof that input-side prompt control is insufficient.\n\nSecond, the headline failure count is overstated. The prompt supplies signed impacts only for the top four drivers. In the audited case those are loan_amnt, RENT, OWN, and VENTURE. person_income and person_age are fifth and sixth, so they were not supplied with signs; the narrative's claims about them are membership violations, not sign inversions of supplied evidence. And person_age appears in the raw borrower profile, so the abstract's 'introduced a feature never given to it' is inaccurate. The cleanly supported inversion is loan_intent_VENTURE alone, plus the omission of the top driver loan_amnt. That is enough to sustain the main conclusion—constrained prompting is necessary but not sufficient—but the 'three of four checkable propositions invert the sign' line is materially too strong.\n\nThere is a second, deeper soft spot: the audit uses a fused SHAP+LIME sign as ground truth, and the paper itself shows the two methods disagree on order (tau=0.43, p=0.18) and that person_income's sign is near a coin flip (0.53). The venture-intent inversion is solid regardless of ground truth, because the prompt gave a signed value and the narrative contradicted it. But for other features the reference itself is shaky.\n\nWhat the paper does well: leakage-controlled preprocessing, out-of-fold meta-features, conservative AUC significance testing, honest calibration and perturbation diagnostics, and a clear statement of what the result is not. The limitation section is unusually candid. Some related-work citations are unverifiable preprints, but the core finding does not depend on them.\n\nVerdict: this deserves a serious referee. It is a reproducible, specific negative result for a widely recommended architecture, and the audit definition is a useful contribution. It needs revision to fix the audited-case accounting and to address the ground-truth fusion issue. I would send it to review, and as a referee I would ask for those two fixes rather than reject. For a reading group, it is a useful example of a paper that reports its own failure honestly.","headline":"A genuinely honest negative result about LLM-generated credit explanations, but the headline sign-inversion count is inflated and the audit's ground truth is shaky.","tokens_in":19067,"tokens_out":1880,"would_cite":true,"duration_ms":18107,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A constrained LLM credit explanation inverted the risk directions it was handed, and the paper shows why prompt-side controls alone fail.","keywords":["credit risk","stacking ensemble","LLM-generated explanations","explanation fidelity","SHAP","LIME","attribution instability","model calibration"],"falsifier":"Re-score the audited case with SHAP only and with LIME only as the reference, regenerate the narrative under the same prompt, and check whether the narrative matches either reference; if it matches one, the reported sign inversion is an artifact of the fusion step rather than a failure of the language model to follow supplied signs. Alternatively, run the paper's three audit checks on a stratified sample of several hundred test instances; violation rates near zero across that sample would undercut the claim that this failure is a systematic property of the architecture.","tokens_in":1778,"feed_emoji":"⚠️","tokens_out":2787,"duration_ms":80452,"temperature":0.7,"pith_summary":"This paper builds a credit-risk explanation pipeline of the kind now common in the literature—a high-accuracy stacking ensemble, SHAP and LIME attributions, and a constrained language model that writes the applicant-facing rationale—and then asks whether the generated narrative matches the attributions it was given. It finds that the predictive gains are real but small at the deployed threshold, while the narrative layer fails: in a documented case the model said age, income, and venture intent increase risk when the supplied attributions scored all three as risk-reducing, omitted the dominant driver, and introduced a feature never provided. The paper's central conclusion is that constrained prompting is necessary but not sufficient, and that grounding must be verified after generation with cheap membership, direction, and coverage checks. A sympathetic reader should take this as an existence proof that the standard input-side control architecture can produce fluent, sign-inverted rationales.","feed_headline":"LLM credit explanation inverted three risk directions it was given","feed_subtitle":"A fidelity audit shows the rationale contradicts the attributions it was handed; verify after generation, not just at the prompt.","key_machinery":"The load-bearing object is the dual-layer explanation pipeline and the audit built to inspect it. Layer one fuses min-max-normalized SHAP and LIME scores into a single score per feature, truncates to the top four drivers, and hands them with signed impacts to a 2.7-billion-parameter decoder-only language model under a deliberately constrained prompt with greedy decoding. Layer two is the fidelity audit introduced by the paper: membership (every feature named in the narrative must be in the supplied driver set), direction (the narrative's asserted risk direction must match the sign of the fused attribution), and coverage (the top driver must be named). The audit is what carries the argument, because it turns \"the narrative reads well\" into checkable propositions that can contradict the evidence.","core_discovery":"On the paper's own terms, the central discovery is asymmetric. The multi-scale stacking ensemble reaches test ROC-AUC 0.9539 and PR-AUC 0.9137, beating the best single model by $\\Delta\\text{AUC} = 0.0143$ that survives a conservative significance test, yet at the F1-optimal threshold this prevents only six additional defaults out of 1,422 and cuts cost-weighted loss by under two percent. The explanatory half fails where the literature usually does not look: in the audited worked case, the generated narrative asserted that person_age ($-0.0092$), person_income ($-0.0569$), and loan_intent_VENTURE ($-0.0720$) raise risk even though the fused attributions supplied in the prompt recorded all three as lowering risk; it never mentioned loan_amnt ($+0.3297$), the strongest driver; and it introduced person_age, which was not in the supplied driver set. The paper ties this to measured properties of the attribution layer rather than to a one-off glitch: SHAP and LIME agree on which features matter (overlap@10 $= 0.80$) but not on their order ($\\tau = 0.43$, $p = 0.18$), and the sign of the attribution for the ensemble's most sensitive input, person_income, flips near a coin flip across applicants (modal-sign share $0.53$). From this it concludes that constrained prompting is necessary but not sufficient, and that the three audit checks would have blocked the failure.","pith_inferences":["Editorial inference: If the fused SHAP-LIME sign is itself an artifact of averaging two methods that disagree on order, then the documented sign inversion may understate or misstate the model's true risk direction; a SHAP-only or LIME-only reference would disentangle the language model's defiance from the fusion step's ambiguity.","Editorial inference: The one documented case cannot support a failure rate, but the paper's own audit definition makes a rate study cheap: scoring several hundred narratives against the three checks would turn the existence proof into a systematic measurement and allow comparison across model scales and prompt designs.","Editorial inference: The same verify-after-generation pattern likely transfers to other regulated domains with generated explanations, such as medical or hiring decision letters, where a fluent but sign-inverted rationale would have similar consequences."],"forward_implications":["If the central claim is right, any deployment that treats prompt design as the safety mechanism is relying on a control with no enforcement, so generation-side verification becomes a mandatory component rather than an optional refinement.","The same audit checks are cheap enough to run on every decision: they require no model calls, no labels, and no human review, and they would have blocked the documented failure.","Explanation quality inherits attribution quality, so sign stability and inter-method rank agreement should be reported alongside importance rankings; without them, directional statements about an applicant are not supportable.","Because the ensemble's 0.0143 AUC gain translates into under two percent cost reduction at the deployed threshold, threshold selection against an explicit cost ratio matters more than further architecture search.","A narrative layer in a regulated setting should fall back to a deterministic template whenever the audit checks fail."],"supporting_citations":[{"why":"Supplies the Shapley-value attribution method that forms half of the fused quantitative explanation layer.","marker":"[4]"},{"why":"Supplies the local surrogate attribution method that forms the other half of the fused explanation layer.","marker":"[5]"},{"why":"Documents that language models generate fluent text whether or not it is grounded, the hazard the audit is designed to catch.","marker":"[14]"},{"why":"Provides the faithfulness definition the paper's membership, direction, and coverage checks operationalize.","marker":"[15]"},{"why":"Proposes the two-stage generate-then-verify architecture the paper's failure motivates as necessary.","marker":"[12]"},{"why":"Establishes that interpretability methods such as LIME are unstable across runs, supporting the measured attribution instability.","marker":"[17]"}],"fun_headline_variants":["LLM credit rationale contradicts supplied attributions, audit finds","Credit-risk ensemble: big AUC, tiny gains, but narrative fails hard","Fidelity audit: LLM flips three risk signs, omits dominant driver","Why constrained prompting alone won't fix LLM credit explanations","SHAP and LIME agree on features but not on sign or order"],"cache_read_input_tokens":21120,"weakest_assumption_plain":"The audit treats the sign of the fused SHAP-plus-LIME attribution as the ground truth the narrative must match; if that fused sign is itself an artifact of averaging two methods that disagree on ordering, the documented inversion shows the model disobeyed the prompt but not that it misrepresented the model's true risk direction.","fun_headline_variants_meta":{"raw":{"variants":["LLM credit rationale contradicts supplied attributions, audit finds","Credit-risk ensemble: big AUC, tiny gains, but narrative fails hard","Fidelity audit: LLM flips three risk signs, omits dominant driver","Why constrained prompting alone won't fix LLM credit explanations","SHAP and LIME agree on features but not on sign or order"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000279,"raw_usage":{"total_tokens":1827,"prompt_tokens":1283,"completion_tokens":544,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":899,"completion_tokens_details":{"reasoning_tokens":452}},"tokens_in":899,"tokens_out":544,"duration_ms":6556,"temperature":1.0,"reasoning_tokens":452,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:22:41.220017+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-score the audited case with SHAP only and with LIME only as the reference, regenerate the narrative under the same prompt, and check whether the narrative matches either reference; if it matches one, the reported sign inversion is an artifact of the fusion step rather than a failure of the language model to follow supplied signs. Alternatively, run the paper's three audit checks on a stratified sample of several hundred test instances; violation rates near zero across that sample would undercut the claim that this failure is a systematic property of the architecture.","supporting_citations":[{"cited_title":"‘Why should I trust you?’ Ex- plaining the predictions of any classifier,","cited_arxiv_id":null,"evidence_quote":"Supplies the local surrogate attribution method that forms the other half of the fused explanation layer."},{"cited_title":"Survey of hallucination in natural language generation,","cited_arxiv_id":null,"evidence_quote":"Documents that language models generate fluent text whether or not it is grounded, the hazard the audit is designed to catch."},{"cited_title":"Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?,","cited_arxiv_id":null,"evidence_quote":"Provides the faithfulness definition the paper's membership, direction, and coverage checks operationalize."},{"cited_title":"A Two-Stage LLM Framework for Accessible and Verified XAI Explanations","cited_arxiv_id":"2604.12543","evidence_quote":"Proposes the two-stage generate-then-verify architecture the paper's failure motivates as necessary."}],"review_version":1}