{"id":"57b8ccb0-e1da-4a4f-b89c-b87a6bca5f1d","arxiv_id":"2608.02879","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"The authors build a sentence-level interpreter by first training an energy-based surrogate of a black-box LLM, then training a lightweight network that scores prompt sentences without further API queries.","lead":"This paper trains an energy-based model to imitate how a black-box language model connects prompts to responses, then uses that imitation to train a second network that scores which prompt sentences most influenced a chosen reply. Once trained, the scoring network needs no further calls to the closed API, which could make auditing proprietary models cheaper and more practical.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2's faithfulness gap may merely reflect topical selection: no baseline controls for target-similar sentence choice, and §6 concedes only behavioral simulation, so the causal-dependency claim lacks decisive support.","rationale":"The paper is honest and technically competent: the EBM architecture is standard, the interpreter is end-to-end differentiable, and the faithfulness experiment is a genuine interventional check against the actual LLM, which is more than many attribution papers do. The ablation study (Table 1) and plausibility matrices (Fig. 3) are informative. However, the single load-bearing premise is that the learned energy landscape captures causal concept dependencies rather than surface or statistical regularities. The reader's weakest_assumption points to the same region; my stress-test sharpens it: the faithfulness metric itself cannot distinguish causation from topical association, and the paper provides no control that rules out a trivial semantic-similarity selector. This is not an accusation of dishonesty; the Limitations section is explicit about the gap between behavioral simulation and mechanistic alignment. The concrete test I propose is cheap and decisive: add an SBERT-similarity baseline to Table 2, with matched selection sparsity. If that baseline matches ESCI, the central claim is unsupported and the verdict should remain conditional pending that comparison; if it does not, the concern is falsified and the paper's claim stands. I recommend UNCHANGED because the reader's CONDITIONAL verdict already captures the need for additional evidence; my proposed test is a specific condition that should be satisfied before ACCEPT.","tokens_in":23536,"tokens_out":9385,"duration_ms":84792,"concrete_test":"Add a no-training baseline to the Table 2 protocol: rank prompt sentences by Sentence-BERT cosine similarity to the target sentence, select the top-K with K matched to ESCI's average selection size, and run the same Sufficiency/Comprehensiveness evaluation. If this baseline achieves a Faithfulness Gap statistically indistinguishable from ESCI's 0.195 (with 95% CI over the 2,000 targets), then the EBM training and Eq. (3) are not the source of the reported causal faithfulness, and the central claim lacks support. If the baseline is clearly worse, the concern is resolved in the paper's favor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the EBM-trained interpreter identifies the prompt sentences causally responsible for a target output. The only direct evidence is Table 2: Generative Sufficiency/Comprehensiveness computed as cosine similarity between the embedded target sentence and the LLM's responses to the selected/complement prompts. This metric rewards topical/semantic overlap, not mechanism. A sentence that is topically related to the target will, when kept, raise similarity; when removed, lower it. The interpreter's training objective (Eq. 3) maximizes the EBM energy gap between complement and selected subsets, but the EBM is trained contrastively (InfoNCE) to separate authentic pairs from negatives such as human responses, GPT-2 outputs, and masked variants. Such losses can be satisfied by surface style differences, and Appendix E.1's diagnostic evaluates the EBM by its own energy drops, not by LLM interventions. The paper itself concedes in §6 that results 'confirm behavioral simulation rather than mechanistic alignment.' Therefore, the positive gap in Table 2 could be produced by any sentence selector that picks the most target-similar sentences (e.g., an SBERT nearest-neighbor selector), with no causal modeling. If that is the case, 'accurately simulates' and 'causal faithfulness' overstate what the experiments establish.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a post-hoc, model-agnostic sentence-level attribution method for black-box LLMs. It first trains a transformer-based energy-based model (EBM) on 20,000 prompt-response pairs with a hybrid InfoNCE objective, then trains a lightweight interpreter network whose selection mask maximizes the EBM energy gap between complement and selected prompt subsets (Eq. 3). The trained interpreter is claimed to identify the prompt sentences causally responsible for a user-specified target output, with zero additional LLM API calls at inference. Experiments include an ablation of the EBM objective, alignment of interpreter scores with five LLM 'oracles', and a causal faithfulness study in which the target LLM is queried with selected and removed sentence subsets, measured by embedding cosine similarity.","tokens_in":23778,"tokens_out":4226,"duration_ms":37066,"significance":"If the faithfulness claim were established, the framework would offer a practical zero-query-at-inference sentence-level attribution tool for closed APIs. The paper does several things right: the faithfulness experiment in Section 4.3 is a genuine interventional test against the target LLM, not only against the surrogate; the limitations section explicitly concedes that the results support behavioral simulation rather than mechanistic alignment; and the zero-API-at-inference property is clearly stated and useful. However, the central causal claim is not yet supported by the evidence as presented, because the faithfulness metric is embedding similarity and no control rules out a purely topical selector.","major_comments":[{"comment":"The central faithfulness claim rests on Generative Sufficiency/Comprehensiveness measured as cosine similarity between the embedded target sentence and the LLM's counterfactual response (Eqs. G4-G5). This metric rewards topical or stylistic overlap, not causal dependence. A selector that simply picks the prompt sentences most embedding-similar to the target sentence would likely produce a positive gap, so the reported Gap of 0.195 for ESCI versus 0.245 for LIME does not by itself demonstrate causal attribution. Please add a control baseline that selects sentences by maximum SBERT cosine similarity to the target sentence (or an equivalent topical selector), and report its Gap in Table 2. Also provide bootstrap confidence intervals or significance tests over the 200 prompts, since the ESCI/LIME difference may be within sampling noise.","section":"Section 4.3, Table 2"},{"comment":"The interpreter is optimized against the EBM energy gap of Eq. (3), so its behavior is only as valid as the EBM's approximation of the target LLM. The EBM is trained contrastively on 20,000 observational pairs, and the paper's own Section 6 states that 'our current results confirm behavioral simulation rather than mechanistic alignment.' Appendix E.1's diagnostic evaluates the EBM using its own energy drops, not against target-LLM interventions. The abstract's claim that 'our EBM accurately simulates the target LLM' is therefore stronger than the evidence. Please either soften the abstract and introduction accordingly, or add a direct validation in which EBM energy changes on ablated prompts are compared with changes in the target LLM's actual generations (e.g., semantic similarity or likelihood under the LLM).","section":"Section 3.3, Eq. (3); Section 6"},{"comment":"The plausibility evaluation compares against five LLM 'oracles' rather than human judgments, and the authors acknowledge this is not ground truth. The qualitative case studies are said to reflect manual analysis by two evaluators of all 2,000 combinations, but no annotation protocol, inter-annotator agreement, or random sample details are given. Please report at least Cohen's kappa on a held-out subset, or restrict the claims to 'alignment with LLM preferences' rather than 'plausibility.'","section":"Section 4.2"}],"minor_comments":[{"comment":"The acronym ESCI is used without definition before Section 4.2; define it at first introduction of the interpreter (Section 3.3).","section":"Section 3.3 and 4.2"},{"comment":"The sentence indices shown are [0], [1], [3]; [2] is missing, and the score columns are not aligned with the number of prompt sentences. Please fix the snippet or annotation.","section":"Figure 4, sample ID 33"},{"comment":"Table 4 omits the value of K (number of Gumbel-Softmax draws) used by the interpreter; Appendix C states K independent samples are drawn, but no value is reported in Table 4.","section":"Table 4"},{"comment":"The kernel width sigma=0.25*sqrt(n) is stated, but the regularization strength for the Ridge regression in LIME is not reported; this is needed to reproduce the LIME baseline.","section":"Appendix G, Eq. (G2)"},{"comment":"Several metrics (e.g., Art. Delta E) are reported without variance or significance information; adding confidence intervals would make the ablation validation more robust.","section":"Section 4.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper you asked about proposes ESCI: train an EBM surrogate on 20k prompt-response pairs from a target LLM, then train a lightweight interpreter to select prompt sentences that maximize the EBM's energy gap between removed and selected subsets. Once trained, the interpreter needs no further API calls. The combination—two-stage EBM-to-interpreter transfer with alternating fine-tuning, at sentence granularity—is genuinely new relative to SOInter and TextGenSHAP, and the writing is refreshingly honest about what is and isn't established.\n\nWhat's well done: the limitations section explicitly concedes \"behavioral simulation rather than mechanistic alignment.\" The faithfulness experiment is a real interventional test against the target LLM with selected and removed sentences, not just an internal diagnostic. The ablations of the EBM objective are thoughtful, and the trivial-target analysis in Appendix G is the kind of transparency you rarely see. The appendices give enough detail to reproduce the setup.\n\nThe soft spots, in rough proportion: the headline causal evidence is Table 2, which measures cosine similarity between the embedded target sentence and the LLM's response to kept/removed prompts. That metric rewards topical overlap. A selector that simply picks the most target-similar sentences would produce a positive gap without modeling any causal dependency. The paper does not include such a control, so the \"causal faithfulness\" claim is overwrought. The authors know this—§6 says behavioral simulation only—but the abstract's \"accurately simulates\" and §4.3's \"causal faithfulness\" push further than the data support.\n\nSecond, the evaluation is thin: 200 prompts, 2000 targets, no error bars or significance tests. The faithfulness metric was switched from NLI to cosine after observing NLI's floor effect; that's plausible but post-hoc. The target filtering (removing trivial fillers) is defensible but should have been pre-registered or at least accompanied by sensitivity analysis.\n\nThird, the plausibility comparison uses LLM oracles as ground truth, which is reasonable for a black-box setting but not a substitute for human or white-box validation.\n\nI don't think the central idea collapses. The energy gap objective and the interventional evaluation are coherent. But \"approaches LIME with zero API queries\" is only interesting if the gap isn't an artifact of topical selection, and the paper doesn't yet establish that. The fix is straightforward: add a target-similarity baseline (e.g., SBERT nearest-neighbor selection), report variance across bootstrap resamples or multiple seed runs, and pre-specify the faithfulness metric. This is a paper for interpretability researchers working on black-box audit tools; it deserves a serious referee, but it needs a strong revision.\n\nRecommendation: send to peer review, with the expectation of heavy revision and a request for the control baseline.","headline":"A coherent, honestly written two-stage EBM-to-interpreter attribution method for black-box LLMs; the causal faithfulness claim rests on a cosine-similarity metric that likely rewards topical overlap, so it needs a strong revision.","tokens_in":24378,"tokens_out":3528,"would_cite":false,"duration_ms":29942,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Contrastive energy modeling of prompt-response pairs lets a lightweight interpreter attribute a black-box LLM's outputs to the prompt sentences that drove them, with no further API calls after training.","keywords":["energy-based models","black-box LLM interpretability","sentence-level attribution","post-hoc explanation","contrastive learning","Gumbel-Softmax selection","causal faithfulness","surrogate model"],"falsifier":"Train the same two-stage interpreter on an open-weights LLM for which internal gradients or activations are available, then compare ESCI's selected prompt sentences with gradient- or activation-based attributions on 500 counterfactual triplets; if the agreement is near chance, the energy landscape would be shown to encode something other than the target's causal computation.","tokens_in":23275,"feed_emoji":"🔍","tokens_out":12314,"duration_ms":97362,"temperature":0.7,"pith_summary":"The paper claims that a black-box LLM can be attributed at the sentence level by first training an energy-based surrogate over prompt-response pairs and then training a lightweight interpreter to select the prompt sentences whose removal changes the surrogate's energy most. The interpreter is meant to be standalone: after a one-time training phase on 20,000 pairs from the target LLM, it can attribute a new prompt without making any further API calls. The authors report that this attribution approaches the causal faithfulness of a sentence-level LIME baseline (Gap 0.195 versus 0.245) while running in $O(1)$ inference time, and that it breaks even with LIME on API cost after roughly 400 interpreted prompts. The pay-off, if the claims hold, is a model-agnostic post-hoc explanation tool for proprietary models that are otherwise only reachable through closed APIs.","feed_headline":"Energy landscape links LLM outputs to their causal prompt sentences","feed_subtitle":"Trained on 20,000 prompt-response pairs, the interpreter then needs no API queries and rivals much costlier baselines.","key_machinery":"The load-bearing object is the contrastively trained energy function $E_{\\mathrm{LM}}(x,y;\\theta)$, a transformer-based EBM that maps sentence-embedded prompts and responses to a scalar compatibility score. It serves as a differentiable surrogate for the black-box LLM, turning the discrete search over prompt-sentence subsets into a smooth optimization: the interpreter learns to pick the mask that maximizes the energy gap between the removed and retained sentences. The training combines a fidelity InfoNCE term with a local-dependency term built from two partial-sequence samplers, and the interpreter's discrete selection is made differentiable with a Gumbel-Softmax relaxation. This machinery is what lets an energy landscape act as the supervision signal for attribution.","core_discovery":"The central claim, on the paper's own terms, is that the generation behavior of an opaque LLM can be compressed into a scalar energy landscape over sentences, and that an interpreter trained to maximize the energy gap between selected and unselected prompt sentences inherits the causal structure of that landscape. The method defines each sentence as a concept, embeds prompt and response sentences with a frozen Sentence-BERT model, and passes them through transformer blocks that project into a task-adapted concept space. A contrastive dual objective—global fidelity against human or other-LM responses, plus local dependency against mismatched partial prompts and responses—shapes the energy so that low energy marks authentic target-LLM pairs. Given a target output sentence, the interpreter produces a sparse binary mask over prompt sentences by optimizing the difference between the energy of the complement and the energy of the selected subset. The experiments show that these selections align with five LLM oracles on nDCG and soft top-1 metrics and, on generative sufficiency and comprehensiveness, come close to a much more expensive LIME baseline.","pith_inferences":["Editorial inference: the same energy landscape is a natural substrate for auditing chain-of-thought: if an intermediate reasoning step is unfaithful, its energy relative to the prompt and final answer should be anomalously high; the paper lists this as future work but does not test it.","Editorial inference: the reported break-even at 400 prompts means the method is a bet on deployment volume; for one-off explanations, paying LIME's per-query cost is likely the rational choice, and the paper's own numbers do not resolve this trade-off.","Editorial inference: the plausibility comparison uses other LLMs as oracles, so shared stylistic biases among LLMs could inflate the alignment scores; a check against white-box explanations on an open-weights model would arbitrate this, as the paper itself notes in its limitations."],"forward_implications":["Attribution for closed-API models becomes a one-time training cost rather than a per-query cost, so high-volume monitoring of proprietary LLM outputs becomes feasible.","Sentence-level attributions align with rationale-evaluation standards, giving auditors units they can read and verify rather than token-level salience maps.","If the energy gap truly tracks causal influence, the same interpreter can be used to prune irrelevant prompt sentences before generation, reducing token and API costs without degrading the answer.","The faithfulness results imply that a roughly 71-million-parameter trainable interpreter can rival instruction-tuned LLM judges and a perturbation-based baseline on standard Q&A-style tasks.","Because the interpreter is trained across a distribution of prompts rather than a single neighborhood, attributions should generalize more reliably across inputs of the same task type."],"supporting_citations":[{"why":"Supplies the LIME local-interpretation baseline that the paper adapts to sentence level and compares against for faithfulness.","marker":"Ribeiro et al., 2016"},{"why":"Provides the energy-based modeling foundation that justifies using an unnormalized compatibility score as the surrogate.","marker":"LeCun et al., 2006"},{"why":"The transformer-based EBM discriminator architecture whose ablations the paper builds on for its energy network.","marker":"Bakhtin et al. (2019)"},{"why":"Sentence-BERT supplies the frozen sentence embeddings that define the paper's concept-level representation.","marker":"Reimers and Gurevych, 2019"},{"why":"Gumbel-Softmax makes the interpreter's binary sentence selection differentiable during training.","marker":"Jang et al., 2017"},{"why":"Defines the sufficiency and comprehensiveness protocols that ground the faithfulness evaluation.","marker":"DeYoung et al., 2020"},{"why":"The HC3 corpus provides the prompt-response data and human-written negative samples used to train the EBM surrogate.","marker":"Guo et al., 2023"},{"why":"Establishes that attention weights are unreliable explanations, motivating the need for a separate attribution mechanism.","marker":"Jain and Wallace, 2019"}],"fun_headline_variants":["Energy landscape reveals which prompt sentences drive LLM outputs","No-API interpreter reads LLM minds via sentence energy","Sentence-level energy maps black-box LLM reasoning","Train once, interpret any LLM: sentence energy attributor","Energy-based interpreter exposes causal prompt sentences without extra queries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method depends on the trained energy scores matching the target LLM's real cause-and-effect structure over prompt and response sentences, not just its surface style; the paper says its results confirm behavioral simulation, not mechanistic alignment.","fun_headline_variants_meta":{"raw":{"variants":["Energy landscape reveals which prompt sentences drive LLM outputs","No-API interpreter reads LLM minds via sentence energy","Sentence-level energy maps black-box LLM reasoning","Train once, interpret any LLM: sentence energy attributor","Energy-based interpreter exposes causal prompt sentences without extra queries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000539,"raw_usage":{"total_tokens":2575,"prompt_tokens":922,"completion_tokens":1653,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":1575}},"tokens_in":538,"tokens_out":1653,"duration_ms":9108,"temperature":1.0,"reasoning_tokens":1575,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:56:53.134997+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same two-stage interpreter on an open-weights LLM for which internal gradients or activations are available, then compare ESCI's selected prompt sentences with gradient- or activation-based attributions on 500 counterfactual triplets; if the agreement is near chance, the energy landscape would be shown to encode something other than the target's causal computation.","supporting_citations":[],"review_version":1}