Enhancing the Comprehensibility of Text Explanations via Unsupervised Concept Discovery

Danding Wang; Jintao Li; Juan Cao; Qiang Sheng; Yifan Sun

REVIEW 3 major objections 5 minor 22 references

Enhancing the Comprehensibility of Text Explanations via Unsupervised Concept Discovery

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that using LLM feedback as a training loss makes automatically discovered text concepts more comprehensible to humans while keeping accuracy on par with black-box models.

desk verdict A genuinely novel LLM-in-the-loop concept discovery method with credible human studies, but the causal role of the LLM feedback is not fully isolated and statistical reporting is thin. read the letter →

arxiv 2505.20293 v1 pith:PRTCY5LH submitted 2025-05-26 cs.CL

Yifan Sun , Danding Wang , Qiang Sheng , Juan Cao , Jintao Li This is my paper · ORCID

classification cs.CL

keywords concept-basedexplanationsunsupervisedconceptdiscoveryslotattentionLLMashumanproxytextclassificationinterpretabilitycomprehensibilitylossexplainableAI

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ECO-Concept aims to build text classifiers that explain their own predictions through concepts discovered from data, with no human-written concept labels. The paper claims that concept comprehensibility can be made an explicit training objective: a large language model summarizes each discovered concept and highlights which tokens belong to it, and the model is fine-tuned to minimize the gap between those highlights and its own slot attention. On seven classification datasets, the resulting concepts are meant to be both task-useful and human-understandable, matching or exceeding black-box accuracy while beating existing unsupervised and post-hoc concept methods in human evaluations. The value of the claim is that interpretability no longer has to be justified after the fact or paid for with performance.

What carries the argument

The central object is the ECO-Concept feedback loop joining a slot-attention concept extractor with an LLM concept evaluator. Slot attention makes up to $M$ trainable prototypes compete to explain encoded tokens, producing a concept attention matrix $A$ and concept features $U$. A linear classifier predicts from total concept activations, creating a concept bottleneck. The evaluator selects highly activated exemplars, prompts an LLM to summarize each concept and highlight related tokens (matrix $S$), and defines the comprehensibility loss $\mathcal{L}_{\mathrm{com}} = \frac{1}{M} \sum_m \beta_m \, \mathrm{MSE}(A_{m,:}, S_{m,:})$, with $\beta_m$ encoding concept importance. This loss, together with consistency and distinctiveness regularizers, carries the argument: it converts "human-understandable" into a differentiable objective.

What would settle it

An experiment that would settle it: recruit humans to rate or simulate two models trained identically except that one uses $\mathcal{L}_{\mathrm{com}}$ and the other does not (or uses random highlights). If the $\mathcal{L}_{\mathrm{com}}$ model's concepts are not rated as more comprehensible or do not improve simulation accuracy, the central claim fails. Also, if replacing the GPT-4o highlights with a deliberately poor highlighter still improves human ratings, then the improvement is not caused by LLM alignment.

Watch

Extended reading notes

Core claim

The central claim is that comprehensibility is not a byproduct of concept learning but a trainable signal. ECO-Concept extracts concepts via slot attention over text tokens, turns the summed attention into a concept bottleneck for classification, and then uses LLM-generated concept summaries and token highlightings as a pseudo-ground truth. The comprehensibility loss $\mathcal{L}_{\mathrm{com}}$ compares the model's concept attention matrix to the LLM highlight matrix, weighted by each concept's importance (average activation times classifier weight magnitude), and this loss is added to cross-entropy, consistency, and distinctiveness losses. The paper reports that across CEBaB, Beer, Hotel, IMDB, AGnews, Twitter, and SciCite, this procedure yields classification accuracy at or above black-box and supervised concept models and concept evaluation scores higher than Cockatiel, Concept-Shap, and ProtoTEx, and that human subjects detect intruders, rate quality, and simulate model outputs better with ECO-Concept explanations.

Load-bearing premise

The framework assumes that what the LLM highlights as concept-related tokens is a trustworthy target for what humans find comprehensible, and that pushing slot attention toward those highlights makes explanations better rather than merely different.

Editorial extensions

If this is right

Any text classification dataset without concept annotations gets built-in, global concept explanations; the four annotation-free datasets in the paper are direct evidence.
Because the LLM is consulted only during training, deployed inference runs entirely on the local RoBERTa-based model, incurring no extra API cost at prediction time.
The framework's concept bottleneck keeps accuracy competitive with black-box classifiers, so the interpretability gain does not require a large accuracy trade-off.
The comprehensibility enhancement stage improves concept semantics, distinctiveness, and consistency over the base slot-attention model, and it can be applied after a first phase of regular concept training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test the paper does not run: compare human-rated comprehensibility of the same model with $\mathcal{L}_{\mathrm{com}}$ on versus off in a randomized A/B study. If the loss only reshapes concepts without raising human agreement or simulation accuracy, the mechanism would not be responsible for the reported gains.
The LLM-as-proxy idea could generalize beyond ECO-Concept: any concept-based explainer that produces token-level or segment-level activations could be fine-tuned with the same summarization-highlighting loop, including prototype networks or vision concept models.
The paper's fixed concept count and API-cost constraint suggest an obvious extension: adaptive concept discovery (add or merge concepts during training) combined with cheaper open LLM evaluators would determine whether the comprehensibility gains persist when the evaluator is weaker.
The importance weighting $\beta_m$ means the framework trades off comprehensibility of low-importance concepts; an implicit consequence is that the model may be allowed to keep obscure low-importance concepts if they help accuracy.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, this paper is worth a serious look. The central idea—use LLM comprehensibility judgments as a training signal to reshape slot-attention concepts in text—is new to me and it works about as well as claimed. The authors put in real effort on evaluation: seven tasks, multiple baselines, and three kinds of human studies including forward simulatability. The forward simulatability gains on Beer and AGnews are meaningful despite the small sample. The ablation w/o L_com is the right control: it shows that the improvement isn't just from extra fine-tuning. I believe the core claim that LLM-guided enhancement improves human comprehensibility is plausible and largely supported.

The soft spots are real but not fatal. First, Tables 1–3 are point estimates with no error bars or confidence intervals, and the pairwise t-tests named in §4.2 are never shown. That's fixable but should be asked for. No code is released either. Second, the reviewer's circularity concern has teeth: the same kind of LLM serves as teacher for the highlight target S in Eq. (9) and as the scorer for Semantics/Distinctiveness in Table 2. The human intruder detection and ratings break the loop, but the automatic metrics could still be inflated. Third, the stress-test note about S=0 for 'meaningless' concepts is on target—I haven't seen a control that separates 'making concepts clearer' from 'deleting hard concepts.' A random-highlight target or a human-rationale-based target would settle this.

One thing I'd push back on: the claim that LLM highlights are validated as a human proxy via Appendix B is indeed too weak, but the human studies later in the paper do show that the final concepts are more comprehensible. So I wouldn't call the central argument circular; I'd call the mechanism-understanding incomplete.

Who is this for? Anyone working on concept-based interpretability or LLM-guided representation learning. It deserves peer review—send it to a careful referee with a request for code, error bars, and the proposed controls. My verdict: conditional accept at best, but definitely not a desk reject.

Referee Report

3 major / 5 minor

Summary. The paper proposes ECO-Concept, an intrinsically interpretable framework for text classification that discovers concepts without any concept annotations. It uses a slot-attention concept extractor to obtain M concept prototypes, regularized by consistency and distinctiveness losses. A second stage evaluates concept comprehensibility with an LLM that summarizes the top-activated exemplars per concept and highlights concept-related segments; the MSE between the model's slot attention and the LLM highlight matrix is used as a comprehensibility loss to fine-tune the model. Experiments on seven text datasets compare ECO-Concept with black-box, supervised, and unsupervised baselines, and report classification accuracy/F1, LLM-based concept quality metrics (Semantics, Distinctiveness, Consistency), and human studies (intruder detection, subjective ratings, forward simulatability).

Significance. If the central claim holds, the paper makes a valuable contribution by incorporating human-comprehensibility feedback into the training loop of an unsupervised concept discovery method, without needing concept annotations. The design is coherent, and the human evaluations (intruder detection, subjective ratings, forward simulatability) are a genuine attempt to measure what the title promises. The paper also ships a project page with code, includes parameter sensitivity and robustness analyses, and attempts to control for additional fine-tuning in the ablation of §4.4. However, the LLM-in-the-loop design creates a circularity concern that must be addressed before the claim that LLM feedback improves human comprehensibility can be accepted.

major comments (3)

[§3.4, Eq. (9), §4.3, Appendix B] The central claim that the LLM-based comprehensibility loss makes concepts more human-understandable is partially circular. The loss trains the model to match LLM-generated highlights S, and then the Semantics and Distinctiveness metrics in Table 2 are obtained by summarizing and evaluating concepts with the same (or closely related) LLM. Appendix B validates the LLM as a human proxy by asking humans to rate agreement with LLM summaries and highlights on the model's own final concepts, which have themselves been trained to match those LLM outputs; high agreement can therefore reflect self-consistency in the training loop rather than the validity of S as a target for human comprehensibility. I ask for a control that breaks this loop, for example training with human-annotated rationales or with random highlights of the same density, and reporting whether the human-rated comprehensibility gains persist. Without such a control, the paper has not isolated that LLM feedback—rather than the suppression mechanism in Eq. (9) or merely longer fine-tuning—causes the gains.
[§4.2, Tables 1 and 2] The paper states in §4.2 that ECO-Concept shows "significant improvements" over unsupervised baselines with pairwise t-tests at a 95% confidence level, but it does not report the t-statistics, p-values, degrees of freedom, or any multiple-comparison correction. Tables 1 and 2 report only point estimates without standard deviations or confidence intervals, which is insufficient to support the significance claim for a method whose metrics (especially Semantics and Distinctiveness) are produced by a stochastic LLM. Since the human studies in Tables 3-4 also report only point accuracies, the statistical evidence for the method's superiority is incomplete.
[§4.4, Fig. 4, Eq. (9)] The ablation does not isolate the causal role of the LLM feedback. In Eq. (9), concepts judged semantically meaningless by the LLM are assigned S=0, which drives their activations toward zero; suppressing ambiguous concepts could by itself raise the Semantics and Distinctiveness scores by removing hard cases. The current comparison with the w/o L_com variant controls for additional fine-tuning, but it does not control for the suppression mechanism or for the information added by the positive highlights (S=1 tokens). A control that fine-tunes with only the suppression term (or with random highlight targets of the same density) is needed to attribute the human-rated comprehensibility gains to the LLM-generated highlighting specifically.

minor comments (5)

[Abstract] The word "adaption" should be "adoption" or "adaptation".
[§3.2, Eqs. (3)-(4)] The notation k(k-1) in the denominators of the consistency and distinctiveness losses is unclear because k is only defined in the surrounding text; the summation ranges over pairs (u_m, u'_m) and pairs of average features, which the equations do not make explicit.
[§3.4, Eq. (9)] When |B_m ∩ D_high| is zero for some concept m, the denominator in Eq. (9) would be zero; the paper should state how such concepts are handled.
[§3.4, Training Strategies] The criterion for deciding that a concept's meaning has "remained unchanged" during the iterative re-summarization is not specified; this could matter for reproducibility.
[§4.3, Tables 3-4, Fig. 3] The human evaluation results are reported as point estimates without confidence intervals or significance tests across participants; given the small sample sizes (42 and 24 participants), the spread should be reported, especially for the forward simulatability claim in Table 4.

Circularity Check

2 steps flagged · score 5.0 of 10

LLM is both the teacher that generates the Eq. (9) highlighting target S and the judge that scores Semantics/Distinctiveness in Table 2, and the Appendix B proxy validation is run on concepts already trained to imitate that same LLM; the automated comprehensibility gains therefore partially reduce to the training signal, though the independent human studies partially break the loop.

self definitional [Section 3.4, Eq. (9); Section 4.3, Quantitative Metrics and Table 2]
"The comprehensibility score of each concept is obtained by averaging the MSE loss between the slot attention matrix A and the highlighted matrix S for the highlighted samples within a mini-batch ℬ. ... Semantics ... proportion of concepts with identifiable semantics to the total number of extracted concepts. Distinctiveness ... proportion of unique concepts identified by an LLM in all extracted ones."

Eq. (9) trains the slot attention A to match the LLM-generated highlight matrix S, and for concepts the LLM deems semantically meaningless, S is set to zero so the model suppresses those activations. When the same LLM (GPT-4o/GPT-4o-mini) is then used in Table 2 to judge whether concepts have clear semantics and whether they are distinct, the reported improvement in comprehensibility is measured with the very model that supplied the training target. The automated metric is therefore not independent of the training objective: the model is being rewarded for imitating the LLM and then judged by that same LLM, so high Semantics/Distinctiveness scores can reflect self-consistency in the training loop rather than an external standard of human comprehensibility.
fitted input called prediction [Appendix B, Human Evaluation of using LLMs as Human Proxies]
"For each task, we presented the concepts extracted using our method to human evaluators and asked them to rate their level of agreement with the conceptual summaries and segment highlightings generated by LLMs. ... Our human study, conducted with diverse participants, confirms that LLMs closely align with human judgment in concept evaluation without introducing much bias. This validates our method of employing LLMs as human proxies for concept evaluation and leveraging their feedback to refine the model."

The validation of the LLM as a human proxy is performed on the final concepts of ECO-Concept, which have been fine-tuned in Section 3.4 to match exactly those LLM summaries and highlightings via Eq. (9). High human agreement with the LLM outputs on those concepts is therefore consistent with the model having learned to imitate the teacher; it does not independently establish that the LLM highlight targets S are a valid training signal for human comprehensibility. The validation is circular with the training loop because the concepts used in the test were constructed to agree with the very LLM being validated.

full rationale

The paper's main contribution is a feedback loop in which LLM evaluations of concept comprehensibility are used as a loss to refine the concept extractor. The central circularity concern is that the same LLM serves as both the teacher generating the highlighting target S in Eq. (9) and the judge computing the Semantics and Distinctiveness metrics in Table 2. Consequently, the automated evidence for the enhancement stage (Fig. 4 and Table 2) partially reduces to the training signal: the model is trained to match LLM highlights and then evaluated by LLM judgments of whether the resulting concepts are meaningful. Appendix B attempts to validate the LLM as a human proxy, but it does so on the model's own final concepts, which have already been optimized to agree with the LLM, so the validation is confounded by the training loop. These are genuine circularity issues for the specific claim that LLM feedback improves comprehensibility. However, the paper also provides independent human evaluations: intruder detection (Table 3), subjective ratings (Fig. 3), and forward simulatability (Table 4). These human studies show that ECO-Concept's final concepts are more comprehensible than the baselines, and they are not generated by the LLM. The weakness is that they do not isolate the causal effect of the comprehensibility loss: no human experiment compares the Base model to ECO-Concept to show that the LLM-guided fine-tuning, rather than the suppression of 'meaningless' concepts or additional training, is what drives the human-rated gains. Overall, the automated evaluation loop is partially circular, but the existence of independent human benchmarks prevents the entire derivation from reducing to the LLM's own judgments. Score 5 reflects this partial circularity: the central enhancement claim is not fully externally validated, yet the method has independent human support for its final output.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central claim rests on the adequacy of learned concept prototypes, the validity of LLM judgments as human proxies, and a set of hand-chosen hyperparameters. The paper validates the LLM proxy only by agreement ratings on its own extracted concepts, leaving the optimization target partially unvalidated.

free parameters (6)

M, number of concepts = 20
Set by hand across all tasks; Appendix E.1 reports sensitivity analysis showing 20 is near-optimal for balancing performance and interpretability.
lambda_con, consistency loss weight = 0.1
Selected by grid search over {0.01, 0.05, 0.1, 0.3, 0.5} to optimize validation accuracy (Appendix E.2).
lambda_dist, distinctiveness loss weight = -0.01
Selected by grid search over {-0.001, -0.005, -0.01, -0.05, -0.1} to optimize validation accuracy (Appendix E.2).
lambda_com, comprehensibility loss weight = 1.0
Selected by grid search over {0.1, 0.5, 1.0, 1.5, 2.0} to optimize validation accuracy (Appendix E.2).
k, number of top-activated samples per concept in consistency and distinctiveness losses = Not reported
Equations (3) and (4) depend on k, but the paper never states its value, which is a reproducibility gap.
Number of exemplars for LLM summarization and highlighting = 10 summarization, 100 highlighting
Implementation choice in Section 4.1; directly affects which samples define the comprehensibility loss and the measured concept quality.

assumptions (5)

domain assumption Slot attention over text token embeddings yields distinct, human-meaningful concepts.
The method borrows object-centric slot attention from vision (Section 3.2, Equations 1-2); no proof is given that sparse competition among slots groups tokens into concepts aligned with human semantics.
domain assumption LLM-generated highlights S are a valid training target for human comprehensibility.
The comprehensibility loss in Eq. (9) directly minimizes the difference between slot attention and LLM highlights; Appendix B validates agreement on the model's own concepts, but not the suitability of highlights as an optimization target.
ad hoc to paper MSE between slot attention and LLM highlight matrices measures comprehensibility.
This is a modeling choice introduced for this paper (Eq. 9) with no independent evidence that MSE is the best or even a valid measure of human comprehensibility.
domain assumption Pretrained RoBERTa/BERT representations provide a suitable concept space.
The concept extractor operates directly on encoder outputs X (Section 3.1); if these embeddings do not preserve human-meaningful structure, the discovered concepts will not align with human understanding.
ad hoc to paper A fixed number of concepts, M=20, is adequate across all seven tasks.
The paper uses 20 concepts everywhere; sensitivity analysis in Appendix E.1 shows robustness, but there is no principled way to determine the correct concept count per task.

invented entities (2)

Trainable concept prototype vectors C (M by D)
purpose: Act as a concept bottleneck: token-level slot attention is aggregated into concept activations t, the only input to the classifier.
Internal latent variables introduced for this model; they have no falsifiable external handle.
LLM-based comprehensibility score
purpose: Serves as a training signal to fine-tune the concept extractor toward human-understandable concepts.
A constructed metric defined via MSE between slot attention and LLM highlights; not independently measured outside the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing the Comprehensibility of Text Explanations via Unsupervised Concept Discovery." pith.science (2026). https://pith.science/paper/PRTCY5LH

@misc{pith2026250520293,
  author       = {Pith},
  title        = {Pith review of: Enhancing the Comprehensibility of Text Explanations via Unsupervised Concept Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PRTCY5LH}},
  note         = {Machine review of arXiv:2505.20293}
}

read the original abstract

Concept-based explainable approaches have emerged as a promising method in explainable AI because they can interpret models in a way that aligns with human reasoning. However, their adaption in the text domain remains limited. Most existing methods rely on predefined concept annotations and cannot discover unseen concepts, while other methods that extract concepts without supervision often produce explanations that are not intuitively comprehensible to humans, potentially diminishing user trust. These methods fall short of discovering comprehensible concepts automatically. To address this issue, we propose \textbf{ECO-Concept}, an intrinsically interpretable framework to discover comprehensible concepts with no concept annotations. ECO-Concept first utilizes an object-centric architecture to extract semantic concepts automatically. Then the comprehensibility of the extracted concepts is evaluated by large language models. Finally, the evaluation result guides the subsequent model fine-tuning to obtain more understandable explanations. Experiments show that our method achieves superior performance across diverse tasks. Further concept evaluations validate that the concepts learned by ECO-Concept surpassed current counterparts in comprehensibility.

Figures

Figures reproduced from arXiv: 2505.20293 by the authors.

**Figure 1.** Comparison of explanations between our proposed ➂ ECO-Concept and existing typical ➀ supervised and ➁ unsupervised concept-based methods. Supervised methods explain based on predefined concepts, while ECOConcept and unsupervised methods explain via concept-related highlighted text. ECO-Concept eliminates the need for concept annotations and can discover unseen concepts with improved comprehensibility. comprehensibi… view at source ↗

**Figure 2.** (a) Illustration of the proposed framework ECO-Concept. ECO-Concept consists of a concept extractor, a classifier, and a concept evaluator. (b) The concept extractor takes the encoded text X as input and interacts with the concept prototypes C to obtain a slot attention matrix A and concept features U. The concept prototypes are optimized using consistency and distinctiveness loss. (c) The concept evaluator utilizes… view at source ↗

**Figure 3.** Human subjective ratings on concept quality infer the model’s outputs based on the explanations and rated their confidence. For comparison, we also measured the accuracy of human judgments without explanations (NE). We conducted forward simulatability experiments with explanations on the two most challenging tasks for human judgment (Beer and AGnews), as determined by the lowest human accuracy rates in no-explanatio… view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Classification performance and concept metrics comparison of the model before concept enhancement (the Base model), ECO-Concept, and ECO-Concept (w/o ℒcom). 5. Conclusion To automatically extract human-understandable concept explanations with no predefined concept anno…

**Figure 5.** Figure 5: Human ratings of provided explanations (Und. indicating understandability, Plaus. indicating plausibility, and Help. indicating helpfulness) D. Further Analysis on Ablation Study D.1. Further Analysis on Impact of Concept Comprehensibility Enhancement As shown in [PIT…

**Figure 6.** Figure 6: Comparison of concept explanations before and after concept enhancement in a beer review case 20 [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]

**Figure 7.** Figure 7: Classification performance and concept metrics of our method with different concept numbers. E.2. Grid search Details The trade-off parameters λcon, λdist, and λcom were selected through grid search, aiming to optimize the model’s classification performance on the vali…

**Figure 8.** Figure 8: Comparison of concept explanations provided by ECO-Concept and other concept-based methods 24 [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]

**Figure 9.** Figure 9: Screenshots of the survey interface 25 [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 17 canonical work pages

[1]

Model Robustness Evaluation To test whether our method can yield reliable explanations for noisy samples, we performed experiments using seven datasets

677 .793 .882 .882 .977 .977 .934 .934 .927 .939 .824 .801 .856 .879 (0.1, -0.01, 0.5) .683 .799 .883 .883 .979 .979 .931 .931 .936 .951 .826 .804 .856 .879 (0.05, -0.01, 0.5) .686 .799 .883 .883 .977 .977 .929 .929 .929 .948 .823 .799 .858 .880 F. Model Robustness Evaluation To test whether our method can yield reliable explanations for noisy samples, we...

work page 1995
[7]

Linearly-Interpretable Concept Embedding Models for Text Analysis

Self-supervised Interpretable Concept-based Models for Text Classification.Preprint, arXiv:2406.14335. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

work page Pith review arXiv
[8]

InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 4171–4186

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 4171–4186. Association for Computational Linguistics. Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Ca...

work page 2019
[9]

Antonio Gulli

Explaining classifiers with causal concept effect (cace).Preprint, arXiv:1907.07165. Antonio Gulli

arXiv 1907
[10]

In61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), pages 5120–5136

COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP tasks. In61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), pages 5120–5136. Maurice G Kendall

work page 2023
[11]

Preprint, arXiv:1907.11692

RoBERTa: A Robustly Optimized BERT Pretraining Approach. Preprint, arXiv:1907.11692. Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf

arXiv 1907
[12]

Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts

Interpretable-by-Design Text Classification with Iteratively Generated Concept Bottleneck.Preprint, arXiv:2310.19660. Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts

arXiv
[13]

Dheeraj Rajagopal, Vidhisha Balachandran, Eduard H Hovy, and Yulia Tsvetkov

Concept-based Explainable Artificial Intelligence: A Survey.Preprint, arXiv:2312.12936. Dheeraj Rajagopal, Vidhisha Balachandran, Eduard H Hovy, and Yulia Tsvetkov

arXiv

Show all 22 references

[14]

InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 836–850

SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 836–850. Michael Röder, Andreas Both, and Alexander Hinneburg

2021
[15]

InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics, pages 3082–3101

Analyzing Encoded Concepts in Transformer Language Models. InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics, pages 3082–3101. Qiang Sheng, Xueyao Zhang, Juan Cao, and Lei Zhong

2022
[16]

https://transformer-circuits.pub/2024/ scaling-monosemanticity/index.html

Scaling Monosemanticity: Extract- ing Interpretable Features from Claude 3 Sonnet. https://transformer-circuits.pub/2024/ scaling-monosemanticity/index.html. Bowen Wang, Liangzhi Li, Yuta Nakashima, and Hajime Nagahara

2024
[17]

InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 3964–3979

Proto-lm: A Prototypical Network-Based Framework for Built-in Interpretability in Large Language Models. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 3964–3979. Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, a...

2023
[20]

Each citation is categorized into one of three classes: method, background, or result

is a dataset designed for classifying citation intents in academic papers. Each citation is categorized into one of three classes: method, background, or result. A.2. Baselines The details of the baseline methods are shown as follows. BERT-/RoBERTa-based classifier(Devlin et a...

2019
[21]

It provides both global and local concept explanations for each sample while performing the classification tasks

is an unsupervised self-explaining model. It provides both global and local concept explanations for each sample while performing the classification tasks. PROTOTEX(Dasetal.,2022)isanunsupervisedself-explainingclassificationarchitecturebasedonprototype networks. It explains mo...

2022
[2010]

Since the original labels are on a scale of 0 to 5, we utilize the binarized version proposed by Bao et al

is a multi-aspect hotel reviews dataset, comprising review texts annotated with seven concept labels: value, rooms, location, cleanliness, check-in/front desk, service, and business service. Since the original labels are on a scale of 0 to 5, we utilize the binarized version p...

2018
[2012]

Each review includes sentiment ratings across five aspects: appearance, aroma, palate, taste, and overall impression

is a multi-aspect beer reviews dataset. Each review includes sentiment ratings across five aspects: appearance, aroma, palate, taste, and overall impression. Following Jourdan et al. (2023), we train the model to predict whether the overall score exceeds 3, indicating a positi...

2023
[2018]

InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1903–1913

Deriving Machine Attention from Human Rationales. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1903–1913. Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and W...

2018
[2019]

InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 3586–3596

Structural Scaffolds for Citation Intent Classification in Scientific Publications. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 3586–3596. Jonathan Crabbé and Mihaela van der Schaar

2019
[2021]

InFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 761–775

Rationalization through Concepts. InFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 761–775. Yujia Bao, Shiyu Chang, Mo Yu, and Regina Barzilay

2021
[2022]

13 Enhancing the Comprehensibility of Text Explanations via Unsupervised Concept Discovery Anubrata Das, Chitrank Gupta, Venelin Kovatchev, Matthew Lease, and Junyi Jessy Li

Discovering latent concepts learned in BERT.Preprint, arXiv:2205.07237. 13 Enhancing the Comprehensibility of Text Explanations via Unsupervised Concept Discovery Anubrata Das, Chitrank Gupta, Venelin Kovatchev, Matthew Lease, and Junyi Jessy Li

arXiv
[2023]

Fateme Hashemi Chaleshtori, Atreya Ghosal, Alexander Gill, Purbid Bambroo, and Ana Marasović

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning.https://transformer-circuits.pub/ 2023/monosemantic-features/index.html. Fateme Hashemi Chaleshtori, Atreya Ghosal, Alexander Gill, Purbid Bambroo, and Ana Marasović

2023
[2024]

Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady

On Evaluating Explanation Utility for Human-AI Decision Making in NLP.Preprint, arXiv:2407.03545. Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Model Robustness Evaluation To test whether our method can yield reliable explanations for noisy samples, we performed experiments using seven datasets

677 .793 .882 .882 .977 .977 .934 .934 .927 .939 .824 .801 .856 .879 (0.1, -0.01, 0.5) .683 .799 .883 .883 .979 .979 .931 .931 .936 .951 .826 .804 .856 .879 (0.05, -0.01, 0.5) .686 .799 .883 .883 .977 .977 .929 .929 .929 .948 .823 .799 .858 .880 F. Model Robustness Evaluation To test whether our method can yield reliable explanations for noisy samples, we...

work page 1995

[2] [7]

Linearly-Interpretable Concept Embedding Models for Text Analysis

Self-supervised Interpretable Concept-based Models for Text Classification.Preprint, arXiv:2406.14335. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

work page Pith review arXiv

[3] [8]

InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 4171–4186

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 4171–4186. Association for Computational Linguistics. Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Ca...

work page 2019

[4] [9]

Antonio Gulli

Explaining classifiers with causal concept effect (cace).Preprint, arXiv:1907.07165. Antonio Gulli

arXiv 1907

[5] [10]

In61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), pages 5120–5136

COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP tasks. In61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), pages 5120–5136. Maurice G Kendall

work page 2023

[6] [11]

Preprint, arXiv:1907.11692

RoBERTa: A Robustly Optimized BERT Pretraining Approach. Preprint, arXiv:1907.11692. Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf

arXiv 1907

[7] [12]

Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts

Interpretable-by-Design Text Classification with Iteratively Generated Concept Bottleneck.Preprint, arXiv:2310.19660. Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts

arXiv

[8] [13]

Dheeraj Rajagopal, Vidhisha Balachandran, Eduard H Hovy, and Yulia Tsvetkov

Concept-based Explainable Artificial Intelligence: A Survey.Preprint, arXiv:2312.12936. Dheeraj Rajagopal, Vidhisha Balachandran, Eduard H Hovy, and Yulia Tsvetkov

arXiv