{"id":"6b821b17-4ade-491a-92ab-f4e8af381c95","arxiv_id":"2506.02092","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"LCBM, a variational concept-based model with per-concept embeddings, outperforms unsupervised concept baselines and nearly matches black-box classifiers on seven image datasets.","lead":"This paper introduces LCBM, an image classifier that learns human-readable concepts from data without labels, representing each concept as both a binary score and a dense embedding vector. In tests across seven image datasets, LCBM beat other unsupervised concept-based models and came within 1-2% of uninterpretable black-box accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Table 4 contradicts the 'always less than 1-2%' gap claim: on CUB-200 with ResNet-18, LCBM trails E2E by 7.44 points, and on MNIST Even/Odd LCBM trails two baselines, so the unqualified 'surpasses' and 'nearly matches' statements need scoping.","rationale":"The strongest claim has two parts: (i) LCBM surpasses all unsupervised concept-based models and (ii) it nearly matches black-box models, with a generalization loss always below 1-2%. Both parts are more categorical than the reported numbers justify. The CUB-200 gap of 7.44 points is not a small fluctuation: the standard deviations are about 0.34 and 1.01, so the gap is decisive and appears in the exact appendix the authors cite for detailed results. On MNIST Even/Odd, LCBM is 0.76 points below BotCL(contr) and 0.43 below LF-CBM, so 'surpasses' is only true on most datasets or on average, not universally. These contradictions are directly checkable from the paper's own evidence, which makes them more load-bearing than the capacity confound identified by the reader: even if one fully accepts that concept embeddings cause the improvement, the magnitude and universality of the headline claim are overstated. The capacity concern is also valid and deserves a capacity-matched baseline, but it is secondary here. The methodology remains promising, a code repository is provided, and the user study and information-plane results give additional support, so a rejection is not warranted. A revision that scopes the claims to the actual reported numbers, and ideally adds a matched-capacity baseline, would make the paper internally consistent and better aligned with its evidence.","tokens_in":17654,"tokens_out":5016,"duration_ms":50528,"concrete_test":"Recompute the E2E-minus-LCBM accuracy gap from Table 4 for every dataset using the reported means. If the CUB-200 gap is 67.96 - 60.52 = 7.44 points, the 'always less than 1-2%' sentence in Section 4.2 is false and must be revised or scoped. Separately, verify the MNIST Even/Odd column to confirm that LCBM (97.25) is below BotCL(contr) (98.01) and LF-CBM (97.68); if both rows match, the unqualified 'surpasses existing unsupervised concept-based models' should be qualified. This check uses the paper's own tables and requires no new computation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim in the abstract and Section 4.2 is that LCBM 'surpasses existing unsupervised concept-based models in generalization capability' and 'almost matches' black-box models, with the generalization gap 'always less than 1-2%.' The paper's own detailed results, Table 4 in Appendix F, do not support that universal statement. On CUB-200 with ResNet-18, E2E accuracy is 67.96±0.34 while LCBM is 60.52±1.01, a gap of 7.44 percentage points, over three times the stated bound of 1-2%. On MNIST Even/Odd, LCBM at 97.25±0.97 trails BotCL(contr) at 98.01±0.29 and LF-CBM at 97.68±0.32, so even the categorical 'surpasses' claim fails on at least one dataset outside the acknowledged 'few decimals' caveat. Because the headline contribution is precisely that unsupervised concept-based models can be nearly as accurate as black boxes, an unqualified exception on a fine-grained dataset materially weakens the claim as written. The reader's capacity-confound complaint is real and worth testing separately, but this numerical inconsistency is more directly load-bearing: it can be checked against the paper's own reported numbers without any new experiments. The appropriate fix is to scope the claim to 'on most benchmarks' or report the CUB gap explicitly; the method still appears promising and the code is released, so a conditional acceptance with mandatory rewording is the right outcome.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces the Learnable Concept-Based Model (LCBM), an unsupervised concept-based image classifier whose latent concepts are Bernoulli random variables augmented with per-concept continuous embeddings in R^d. The model is trained with an ELBO objective combining a reconstruction term, a classification term, and a KL term with a batch-regularized Bernoulli prior; predictions are made as a local linear combination of concept scores, with sample-dependent weights predicted from the embeddings. The authors evaluate LCBM on seven image datasets against SENN, BotCL (two variants), ProtoPNet, LF-CBM, and an end-to-end black-box baseline, using task accuracy, information-plane curves, reconstruction MSE, concept-F1/CAS alignment metrics, a 72-participant user study, and negative intervention experiments. The central claims are that LCBM surpasses existing unsupervised concept-based models in generalization, nearly matches black-box accuracy with a gap \"always less than 1-2%,\" and yields more interpretable concepts.","tokens_in":18015,"tokens_out":3124,"duration_ms":30133,"significance":"If the claims hold, the paper makes a useful contribution to interpretable machine learning: it proposes a concrete mechanism—concept embeddings combined with Bernoulli scores—for improving the accuracy of unsupervised concept bottlenecks while retaining linear, local interpretability. The release of code, the use of multiple datasets and baselines, the standard variational derivation, and the user study with 72 participants are strengths that make the empirical claims partly reproducible. However, the headline generalization claims are overstated in their current form, and the attribution of the accuracy gain to the embedding mechanism is not yet substantiated because no capacity-matched baseline is presented. The contribution is potentially significant, but the present version requires scoping and additional experiments before the central claims can be accepted as stated.","major_comments":[{"comment":"The claim in Section 4.2 that \"the generalization loss is always less than 1-2%\" relative to the E2E black-box model is contradicted by the paper's own Table 4. On CUB-200 with ResNet-18, LCBM achieves 60.52±1.01% accuracy while E2E achieves 67.96±0.34%, a gap of 7.44 percentage points—more than three times the stated bound. In addition, the claim that LCBM \"surpasses existing unsupervised concept-based models\" fails on MNIST Even/Odd, where LCBM at 97.25±0.97% trails BotCL (contr) at 98.01±0.29% and LF-CBM at 97.68±0.32%. The qualitative statement in the text that \"only ... a few methods perform better, by a few decimals\" understates these gaps. The authors should scope the claim (e.g., to \"most benchmarks\") or explicitly report and discuss the CUB-200 gap; as written, the unqualified statements in the abstract and Section 4.2 are not supported by the reported numbers.","section":"Section 4.2, Fig. 4, Table 4 (Appendix F)"},{"comment":"The paper attributes LCBM's accuracy improvement to \"unsupervised concept embeddings,\" but the comparison baselines (SENN, BotCL, ProtoPNet, LF-CBM) use a single scalar value per concept, whereas LCBM uses a 128-dimensional vector per concept. This is a substantial difference in bottleneck capacity and parameter count, yet no capacity-matched baseline is provided: Appendix C only varies the embedding dimension within LCBM itself (Fig. 10), and does not compare against baselines given an equivalent per-concept parameter budget. Without such a comparison, the claim that the gain comes from the embedding mechanism rather than from the larger representation capacity is not established. A concrete test would be to train the baselines with an increased per-concept capacity (or to ablate LCBM to a single-scalar concept representation while keeping the same training procedure) and report whether the accuracy advantage persists.","section":"Section 3.2, Appendix C"},{"comment":"The information-plane analysis in Fig. 5 is central to the claim that LCBM \"retains more information regarding both the input and the output,\" but the mutual information estimator is not specified. For continuous concept embeddings and discrete/continuous inputs, I(X,C) and I(C,Y) are generally intractable, and the reported values depend heavily on the estimator, the discretization, and the number of samples. Without specifying the estimator (e.g., k-NN, histogram, variational bound) and its hyperparameters, these curves cannot be reproduced or compared across models with different representation types. The authors should state the estimator and settings, or otherwise present an alternative quantitative justification for the information-retention claim.","section":"Section 4.3, Fig. 5"}],"minor_comments":[{"comment":"The label \"Aignement\" in Eq. (13) is a typo for \"Alignment.\"","section":"Appendix A, Eq. (13)"},{"comment":"The figure captions contain typos: \"extracted form\" should be \"extracted from,\" and \"MNIST-Evan/Odd\" should be \"MNIST Even/Odd.\"","section":"Figures 17, 18, 24"},{"comment":"The description of the information-plane metric refers to \"the evolution of the mutual information ... as the training epoch increases,\" but Fig. 5 appears to show only the final information-plane position per model. If the figure shows trajectories, the axis and legend should be clarified; if it shows only endpoint values, the text should be updated accordingly.","section":"Section 4.1"},{"comment":"The ViT results are presented in an appendix without a corresponding discussion of whether the observed patterns (e.g., LCBM trailing BotCL on MNIST Even/Odd by 0.3%) affect the generalizability of the main claims. A sentence relating these results to the abstract claims would help.","section":"Table 3 (Appendix B)"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be an accepted ECML-PKDD 2025 paper (per the footnote), but this review was conducted on the arXiv version. The main technical concerns—overbroad accuracy claims and the lack of a capacity-matched baseline—are addressable in a revision, and I do not see an internal inconsistency in the core methodology. I would encourage the editor to request the scoping and additional experiments described in the major comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nI've read the LCBM paper. The core idea is a genuinely new combination: unsupervised concept-based model with Bernoulli-distributed concepts, per-concept embeddings, and a local linear classifier. The experimental sweep is thorough—seven datasets, two backbones, code released—and they include a user study plus interpretability metrics beyond raw accuracy. That earns a serious read.\n\nBut the headline claim is overstated. The abstract and Section 4.2 say the generalization gap to black-box models is \"always less than 1-2%\" and that LCBM \"surpasses\" existing unsupervised concept models. Their own Table 4 shows a 7.44-point gap on CUB-200 with ResNet-18, and on MNIST Even/Odd LCBM trails BotCL (contr) and LF-CBM. That's not a minor caveat; it changes how strong the contribution is. The fix is easy: scope the claim to \"most benchmarks\" and report the exceptions.\n\nThe other real soft spot is attribution of the accuracy gain. LCBM uses 128-dimensional concept embeddings; baselines use single scalar scores. The embedding-size ablation in Appendix C does not match baseline capacity, so we can't tell whether the gain comes from richer representations or just a larger bottleneck. A capacity-matched baseline would settle this, and it's worth asking for.\n\nMinor issues: the information-plane plots don't specify the mutual information estimator, which hampers reproduction; the user study population is narrow (72 mostly CS professionals, age 20-29). Neither is load-bearing, but both should be tightened.\n\nThe math is standard variational inference; the ELBO derivation is clean. I don't see circular reasoning or data leakage. The code release is a plus.\n\nWho's this for? Researchers working on interpretable ML, especially unsupervised concept-based models. It's a useful subfield contribution, not a field-changer.\n\nI'd send it to peer review with a request to fix the overclaim, add or discuss a capacity-matched baseline, and document the MI estimator. With those changes, it's acceptable.\n\nRecommendation: send to review, conditional on revision.","headline":"LCBM is a genuinely new unsupervised concept-based model with strong empirical work, but the paper's headline accuracy claims are overstated and need scoping.","tokens_in":18528,"tokens_out":3206,"would_cite":true,"duration_ms":28693,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Unsupervised concept-based image classifiers can stay interpretable and still nearly match black-box accuracy.","keywords":["concept-based models","unsupervised learning","interpretability","XAI","variational inference","concept embeddings","image classification","Bernoulli latent space"],"falsifier":"Construct a capacity-matched baseline by giving SENN or BotCL the same 128-dimensional embedding per concept (and a comparable parameter count) while keeping their original losses and interpretable linear head; if its task accuracy reaches LCBM's 1-2% gap on CIFAR-100 and Tiny ImageNet, the embedding-size effect rather than the proposed mechanism explains the result. A second check would verify whether the mutual information curves $I(X,C)$ and $I(C,Y)$ in Fig. 5 remain monotonic for that baseline; if they do, the information-retention claim is not specific to LCBM.","tokens_in":17494,"feed_emoji":"🧠","tokens_out":5723,"duration_ms":56030,"temperature":0.7,"pith_summary":"The paper introduces LCBM, an unsupervised concept-based model for image classification that learns concepts as Bernoulli random variables in a latent space, each paired with a learned vector embedding. Its central claim is that this embedding-based concept representation closes most of the accuracy gap that has historically separated unsupervised interpretable models from black-box classifiers: across seven image datasets LCBM's task accuracy is within 1-2% of an end-to-end black-box baseline and often higher than prior unsupervised concept models. The paper also claims the learned concepts retain more information about both input and label, align better with human-annotated concepts, and are judged more plausible and understandable by users. If these claims hold, an interpretable model no longer has to sacrifice accuracy to explain itself, and concept annotations from humans or large language models become unnecessary.","feed_headline":"Unsupervised concept models nearly match black-box accuracy","feed_subtitle":"LCBM's concept embeddings close the gap to within 1–2% and keep predictions interpretable.","key_machinery":"The load-bearing mechanism is the concept embedding pair $(c_j, t_j)$: for each concept $j$, a per-concept MLP $h_j$ maps a frozen backbone's features into a $d=128$ vector $c_j$, and a learned prototype $t_j$ converts alignment $c_j \\cdot t_j$ through a sigmoid into a Bernoulli activation probability, sampled via the reparameterization trick to give the concept score. The same embedding feeds both the decoder for reconstruction and a class-specific network that predicts local linear weights, so each prediction stays a linear combination of concept scores while the representations carry more than one bit per concept. The ELBO objective with batch-level KL regularization ties representativity, completeness, and alignment together.","core_discovery":"On the paper's own terms, the discovery is that modelling each unsupervised concept as a Bernoulli random variable with an associated 128-dimensional embedding, rather than a single scalar activation, lets a concept bottleneck carry enough information to nearly match black-box accuracy without losing local linear interpretability. The model is trained with a variational ELBO whose three terms correspond to representativity (input reconstruction), completeness (task classification), and alignment (KL to a prior), and the final class prediction is a linear combination of concept scores with weights produced from the embeddings. The reported evidence includes accuracy gaps below 1-2% on CIFAR-10, CIFAR-100, Tiny ImageNet, Skin Lesions, and CUB-200, higher mutual information with both input and label during training, lower reconstruction error, higher concept-F1 and concept-alignment scores against human annotations, and a user study where participants more often identify intruder and completing images for LCBM concepts than for the BotCL baseline.","pith_inferences":["The reported gains may be partly a capacity effect rather than a property of the embedding mechanism: the baselines pass one scalar per concept, while LCBM passes 128 dimensions, and the Appendix C ablation varies embedding size but does not give a capacity-matched baseline.","Because the classifier weights are sample-dependent, the 'linear combination' is local, not global; a user cannot read a single fixed rule from the model, only per-image explanations, and whether that satisfies regulated interpretability requirements is an open question the paper does not settle.","The authors themselves note the CNN decoder limits reconstruction quality and that manual inspection of concepts is still required, so the representativity and interpretability gains are bounded by that component.","If concept embeddings are this informative, the same representation could be reused for auxiliary tasks such as concept-based retrieval, dataset audit, or generating natural-language concept descriptions, but those applications are not tested here."],"forward_implications":["Unsupervised concept models become usable in domains where concept labels and reliable language-model priors are unavailable, since LCBM derives concepts from data alone while staying within roughly 1-2% of black-box accuracy.","With a small fixed number of concepts (10-30), LCBM scales to 100-200 class datasets, suggesting concept bottlenecks do not need class-scaled concept sets.","Prediction remains locally interpretable as a weighted sum of concept activations, and negative concept interventions change predictions, so users can probe and counterfactually edit decisions.","The higher concept-F1 and CAS scores imply the discovered concepts can double as a dataset annotation source, reducing manual labelling effort."],"supporting_citations":[{"why":"Supplies the main unsupervised concept-basis baseline (SENN) and the dictionary-based interpretability evaluation that LCBM is compared against.","marker":"[3]"},{"why":"Defines the supervised concept bottleneck formulation and the linear interpretable classifier that LCBM extends to the unsupervised setting.","marker":"[16]"},{"why":"Provides BotCL, the state-of-the-art unsupervised baseline used for accuracy comparisons and the user study.","marker":"[40]"},{"why":"Provides ProtoPNet, the prototype-based baseline that constrains representation capacity and is compared across datasets.","marker":"[6]"},{"why":"Provides LF-CBM, the label-free baseline using vision-language priors that LCBM is claimed to beat despite not using external concepts.","marker":"[26]"},{"why":"Supplies the concept-embedding idea and the Concept Alignment Score (CAS) metric used to measure human alignment.","marker":"[8]"},{"why":"Supplies the probabilistic graphical model factorization $p(x|c)p(y|c)p(c)$ that underlies the ELBO objective.","marker":"[25]"},{"why":"Provides the concrete distribution / reparameterization trick that makes Bernoulli concept sampling differentiable during training.","marker":"[22]"}],"fun_headline_variants":["Unsupervised concept model with Bernoulli latents nearly matches black-box accuracy","Learnable Concept-Based Model: Unsupervised concepts, near black-box accuracy","Unsupervised concept embeddings bridge accuracy gap to black-box models","Unsupervised concept model: high accuracy, human-intuitive concepts","Unsupervised concept model almost matches black-box accuracy with better interpretability"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that LCBM's architecture, rather than its larger concept representation, drives the accuracy gain is the load-bearing assumption: baselines use a single scalar per concept while LCBM uses a 128-dimensional embedding, and the Appendix C ablation only changes embedding size without ever matching baselines in capacity.","fun_headline_variants_meta":{"raw":{"variants":["Unsupervised concept model with Bernoulli latents nearly matches black-box accuracy","Learnable Concept-Based Model: Unsupervised concepts, near black-box accuracy","Unsupervised concept embeddings bridge accuracy gap to black-box models","Unsupervised concept model: high accuracy, human-intuitive concepts","Unsupervised concept model almost matches black-box accuracy with better interpretability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000916,"raw_usage":{"total_tokens":3896,"prompt_tokens":873,"completion_tokens":3023,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":2929}},"tokens_in":489,"tokens_out":3023,"duration_ms":22482,"temperature":1.0,"reasoning_tokens":2929,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:32:51.110803+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a capacity-matched baseline by giving SENN or BotCL the same 128-dimensional embedding per concept (and a comparable parameter count) while keeping their original losses and interpretable linear head; if its task accuracy reaches LCBM's 1-2% gap on CIFAR-100 and Tiny ImageNet, the embedding-size effect rather than the proposed mechanism explains the result. A second check would verify whether the mutual information curves $I(X,C)$ and $I(C,Y)$ in Fig. 5 remain monotonic for that baseline; if they do, the information-retention claim is not specific to LCBM.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Provides BotCL, the state-of-the-art unsupervised baseline used for accuracy comparisons and the user study."},{"cited_title":"In: The Eleventh International Conference on Learning Representations (2023),https://openreview.net/forum?id=FlCg47MNvBA 16 F","cited_arxiv_id":null,"evidence_quote":"Provides LF-CBM, the label-free baseline using vision-language priors that LCBM is claimed to beat despite not using external concepts."},{"cited_title":"In: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A","cited_arxiv_id":null,"evidence_quote":"Supplies the concept-embedding idea and the Concept Alignment Score (CAS) metric used to measure human alignment."},{"cited_title":"Advances in Neural Information Processing Sys- tems35, 4667–4679 (2022)","cited_arxiv_id":null,"evidence_quote":"Supplies the probabilistic graphical model factorization $p(x|c)p(y|c)p(c)$ that underlies the ELBO objective."},{"cited_title":"In: International Conference on Learning Representations (2022)","cited_arxiv_id":null,"evidence_quote":"Provides the concrete distribution / reparameterization trick that makes Bernoulli concept sampling differentiable during training."}],"review_version":1}