{"id":"dbed53b3-445d-4fd7-b5e9-e1b11d945274","arxiv_id":"2607.07264","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"By fixing CLIP-derived language similarity maps as the coefficient matrix in non-negative matrix factorization, this method produces named, faithful concept explanations for frozen image classifiers.","lead":"This paper introduces a method to explain what a trained image classifier is looking at by decomposing its internal activations using language-anchored concepts. It matters because it produces human-readable, named explanations without retraining the model, which is useful for auditing AI in high-stakes domains like medicine.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Faithfulness metrics (C-Ins, C-Del) test decision-relevance but not naming accuracy; the 'named' pillar of the central claim rests on qualitative evidence and an ablation that doesn't isolate language content from structural constraint.","rationale":"The reader identifies CLIP map quality as the weakest assumption, which is adjacent to my concern about naming accuracy. But the reader frames it as an input-quality issue (noisy CLIP maps), while the more precise concern is an evaluation gap: the paper's faithfulness metrics don't test whether names match content, and the ablation doesn't isolate language content from structural constraint. Despite this, the verdict should remain ACCEPT. The paper provides reasonable qualitative evidence for naming accuracy (Figures 3–5 show clean localization), the cross-image consistency result (Figure 4) is compelling for the examples shown, and the ablation does demonstrate that fixing S improves faithfulness even if it doesn't isolate the language component. The concern is shared across the entire concept-discovery literature — no existing method quantitatively tests naming accuracy. The paper is honest about its limitations and doesn't overclaim. The central claim (named + faithful + no modification) holds at the level of evidence provided, though a quantitative naming-accuracy test and a random-fixed-S control would substantially strengthen the 'functional anchor' sub-claim. Confidence should be slightly reduced from HIGH to MEDIUM-HIGH to reflect the untested naming property.","tokens_in":18297,"tokens_out":8565,"duration_ms":563360,"concrete_test":"Two checks: (1) On a subset of ImageNet classes with ground-truth part/attribute annotations (e.g., PartImageNet), compute IoU between LAD's concept heatmaps (from Ŝ at inference) and the actual annotated part locations for matching concept names. If mean IoU is below ~0.3, the names are not reliably localized and the 'named' property is weak. (2) Replace S with a random non-negative matrix matched in sparsity and scale to the CLIP-derived S, relearn W, and recompute C-Del on ImageNet. If random-fixed-S achieves C-Del within 0.05 of LAD's 0.902, the language content is not specifically functional — the improvement comes from fixing S, not from CLIP semantics.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that LAD produces concepts that are 'simultaneously named, faithful, and obtained without modifying the model.' The 'faithful' property is tested quantitatively via C-Ins and C-Del (Table 1). The 'no modification' property is structural. But the 'named' property — that concept names actually describe the visual content of the localized regions — is supported only qualitatively (Figures 3–5) and by one cross-image consistency example (Figure 4, 'maple neck').\n\nThe gap: C-Ins and C-Del measure whether the decomposition captures decision-relevant directions, not whether the names attached to those directions are correct. A concept could achieve high C-Del (the basis vector captures decision-relevant features) while having a misleading name (the CLIP-derived coefficient matrix S localized the concept to regions that don't actually contain the named attribute). The basis W is learned to minimize ||Ā − SW^T||_F; if CLIP mislocalizes a concept, W[:,k] will be learned to reconstruct features in the wrong region, producing a faithful but misnamed direction.\n\nThe ablation in Table 3 is the paper's main evidence that the language anchor is 'functional, not cosmetic.' It compares LAD (fixed CLIP-derived S) against unsupervised NMF (learned S). But this conflates two changes: (1) fixing S vs. learning S, and (2) using CLIP-derived semantic values vs. unconstrained values. A random fixed S (matched in sparsity and scale) might also improve C-Del over learned S, because any structured coefficient matrix constrains the optimization. Without this control, the ablation shows that fixing S helps, not that the language content specifically helps.\n\nThe paper is honest about limitations (ViT failure in Sec. A7, dermoscopy boundary bias in Sec. A5), and these don't invalidate the central claim. But the conjunction 'named AND faithful' requires that the names be accurate, and no quantitative test verifies this.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"The paper proposes Language-Anchored Decomposition (LAD), a post-hoc concept-discovery method that inverts standard NMF by fixing CLIP-derived language-grounded similarity maps as the coefficient matrix S and learning only the concept basis W against frozen encoder activations. The central claim is that LAD produces concepts that are simultaneously named, faithful, and obtained without modifying the model. The method is evaluated on ImageNet (ResNet34, 500 classes), Places365 (ResNet50, 364 classes), and two clinical domains (retinal fundus on ODIR-5K with DenseNet-121, dermoscopy on HAM10000 with ConvNeXt-Small). The paper provides an ablation (Table 3) showing that removing the language anchor preserves accuracy but collapses deletion faithfulness (C-Del drops from 0.902 to 0.507 on ImageNet). Code is publicly available.","tokens_in":19183,"tokens_out":1516,"duration_ms":308303,"significance":"The paper addresses a genuine gap in the interpretability literature: post-hoc concept methods (ACE, ICE, CRAFT, FACE) recover faithful but unnamed factors, while by-design methods (Concept Bottleneck Models, LaViSE) attach names only by retraining or altering the classifier. LAD's approach of inverting NMF to fix language-grounded coefficients and learn only the basis is a clean and novel formulation. The convergence analysis (Sec. A3) is correct: with S fixed, the objective is a convex quadratic in W, and projected gradient descent converges to a global minimizer. The computational efficiency of fixing S (0.03s/class vs. 0.5s for CRAFT and 2.6s for FACE) is a practical advantage. The ViT case study (Sec. A7) is a commendable negative result that honestly documents where the method does not transfer. The publicly available code and the breadth of evaluation (natural images, scenes, two medical domains, five CNN backbones, one ViT) strengthen reproducibility.","major_comments":[{"comment":"The ablation comparing LAD (fixed CLIP-derived S) against unsupervised NMF (learned S) is the paper's primary evidence that the language anchor is 'functional, not cosmetic.' However, this comparison conflates two changes: (1) fixing S vs. learning S, and (2) using CLIP-derived semantic values vs. unconstrained values. A random or structured-random fixed S (matched in sparsity and scale to the CLIP maps) might also improve C-Del over fully learned S, because fixing S regularizes the decomposition regardless of semantic content. To isolate whether the language content of S matters — as opposed to the structural constraint of fixing S — the paper should include a control where S is fixed but semantically arbitrary (e.g., random non-negative matrices or shuffled CLIP maps). If such a control also achieves high C-Del, the claim that 'language supervision determines which concept directions' ","section":"Sec. 5.5, Table 3"},{"comment":"The central claim includes three pillars: named, faithful, and no modification. The 'faithful' pillar is tested quantitatively via C-Ins and C-Del (Table 1). The 'no modification' pillar is structural. But the 'named' pillar — that concept names accurately describe the visual content of the localized regions — is supported only qualitatively (Figures 3-5) and by one cross-image consistency example (Figure 4, 'maple neck'). C-Ins and C-Del measure whether the decomposition captures decision-relevant directions, not whether the names attached to those directions are correct. A concept could achieve high C-Del while having a misleading name if CLIP mislocalizes the concept. The paper should acknowledge this gap explicitly and, ideally, provide a quantitative or semi-quantitative evaluation of naming accuracy (e.g., human ratings on a sample of concept-region pairs, or automated verification","section":"Central claim (Abstract, Sec. 1, Sec. 6)"}],"minor_comments":[{"comment":"LAD's C-Del on ImageNet (0.902) is lower than FACE (0.939). The paper frames this as an intended trade-off (Sec. 5.2: 'Trading a portion of deletion completeness for named, verifiable concepts'). This framing is reasonable but could be stated more prominently, as readers may initially read the table as LAD underperforming.","section":"Table 1"},{"comment":"The red-circle visual prompting technique for constructing CLIP similarity maps is a key component, but the choice of circle radius r and grid resolution (7x7 or 14x14) is not thoroughly justified or analyzed. Table A6 shows grid sensitivity for cost but not for faithfulness metrics. A brief analysis of how grid resolution affects C-Ins/C-Del would strengthen the method.","section":"Sec. 3.3"},{"comment":"The dermoscopy results show substantially lower C-Del (0.418) compared to retinal fundus (0.954). The paper attributes this to boundary bias in concept localization (Fig. A1). This is an honest and important finding, but it raises a question about the generality of the method in domains where discriminative features are interior rather than boundary-localized. A brief discussion of when LAD is expected to succeed or struggle would help practitioners.","section":"Sec. A5, Table A2"},{"comment":"The ViT case study reports C-Del = 0.02, which the paper attributes to the preserved [CLS] pathway and diffuse patch-token localization. This is a valuable negative result. The paper could note more explicitly that the near-zero C-Del is partly an artifact of the evaluation protocol (the [CLS] token carries original information), making the metric not directly comparable to the CNN setting.","section":"Sec. A7"},{"comment":"The diagram is informative but dense. The relationship between the CLIP-derived S matrix and the learned W basis could be made clearer with a small worked example showing dimensions for a single image.","section":"Fig. 2"},{"comment":"The claim that 'naming becomes a structural property of the factorization rather than a post-hoc step' is a strong statement. It would be more precise to say naming is enforced by the structure of S, since the names are still ultimately derived from CLIP similarity and could be incorrect if CLIP's localization is poor.","section":"Sec. 3.2"}],"recommendation":"minor_revision","confidential_remarks":"The skeptic's concern about the Table 3 ablation conflating structural constraint with semantic content is the most substantive issue. On reading the paper, I find this concern partially valid: the ablation does not isolate language content from the act of fixing S. However, the paper's qualitative evidence (Figures 3-5) and the cross-image consistency example (Figure 4) provide some support for the naming claim, and the method's formulation is sound. The fix is a control experiment, which is feasible within the manuscript's scope. I do not see this as a load-bearing error that invalidates the central claim, but it does need to be addressed. The dermoscopy boundary bias and ViT negative result are honestly reported limitations, not flaws."},"author_rebuttal":null,"desk_editor":{"model":"glm-5.2","letter":"The main thing to know: this paper inverts NMF in a way I haven't seen before—fixing CLIP-derived language similarity maps as the coefficient matrix S and learning only the basis W against frozen encoder activations. That gives you named concepts without retraining the model, which is a real gap in the interpretability literature that nobody had cleanly filled. The per-image named-concept rollout (Figure 4) is a genuine new capability; baselines can only emit “factor 1, factor 2.” Code is public, which matters here because the method is simple enough to reproduce and check yourself. The clinical results on retinal fundus are strong—LAD leads C-Del by a wide margin (0.954 vs. 0.863 for FACE)—and the computational cost is an order of magnitude below the baselines. The paper is also unusually honest about where the method fails, which I respect. The ViT case study (Sec. A7) is a clean negative result: reconstruction transfers but localization collapses, and they explain why. The dermoscopy boundary bias (Sec. A5) is reported as a limitation, not hidden. Now the soft spots. The stress-test note lands on a real gap: the “named” pillar of the central claim rests on qualitative evidence (Figures 3–5) and one cross-image consistency example. C-Ins and C-Del test decision-relevance, not whether the name attached to a concept actually describes what's in the highlighted region. No human study, no automated naming-accuracy metric. That's a hole. The ablation in Table 3 is the other weak point. It compares fixed CLIP-derived S against learned S (standard NMF), but this conflates two changes: fixing S vs. learning S, and using CLIP-derived semantic content vs. unconstrained values. A random fixed S—matched in sparsity and scale—might also improve C-Del over learned S, because any structured coefficient matrix constrains the optimization. Without that control, the ablation shows that fixing S helps, not that the language content specifically helps. The paper claims the anchor is “functional, not cosmetic,” but the evidence supports “fixing S helps,” which is weaker. These are evaluation gaps, not method flaws. The core formulation is sound, the convergence analysis is clean, and the honest reporting of failures builds trust. The paper deserves a serious referee who can push for a random-anchor control and some quantitative naming validation. I'd recommend accepting for peer review.","headline":"Writing letter on LAD paper","tokens_in":19178,"tokens_out":2083,"would_cite":false,"duration_ms":152229,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Named concepts, faithful explanations, no retraining","keywords":["concept-based explanation","non-negative matrix factorization","CLIP","vision-language models","post-hoc interpretability","faithfulness","medical imaging"],"falsifier":"If replacing the fixed language-grounded coefficient matrix with learned coefficients (standard unsupervised NMF) produced equivalent or better deletion faithfulness (C-Del), the central claim that the language anchor is functional rather than cosmetic would be falsified. The paper reports the opposite: C-Del drops from 0.902 to 0.507 when the anchor is removed.","tokens_in":18631,"feed_emoji":"🔍","tokens_out":1147,"duration_ms":372421,"temperature":0.7,"pith_summary":"The paper introduces Language-Anchored Decomposition (LAD), a method for explaining what a trained image classifier relies on when making a decision. Existing explanation methods face a trade-off: those that recover factors faithful to the model's internal behavior leave those factors unnamed, while those that attach human-readable concept names require retraining or altering the classifier. LAD breaks this trade-off by inverting a standard matrix factorization technique. Instead of learning both the concept coefficients and the concept basis from the data, LAD fixes the coefficients to spatial similarity maps derived from a vision-language model (CLIP), where each map corresponds to a named concept proposed by a large language model. It then learns only the concept basis that best reconstructs the frozen classifier's own activations. This makes naming a structural constraint rather than a post-hoc label, while the model's own feature geometry determines which named concepts survive. The paper shows that this language anchor is functional: removing it preserves classification accuracy but collapses the faithfulness of attributions, demonstrating that language supervision determines which concept directions are discovered. Across natural images, scenes, and medical imaging, LAD produces spatially precise, named concepts that are decision-relevant under insertion and deletion tests.","feed_headline":"Named concepts, faithful explanations, no retraining","feed_subtitle":"By inverting matrix factorization to fix language-grounded concept maps, LAD explains what a classifier sees without altering the model.","key_machinery":"The key machinery is the inversion of non-negative matrix factorization: fixing a language-grounded coefficient matrix S (built from CLIP spatial similarity maps for LLM-proposed named concepts) and learning only a non-negative concept basis W that reconstructs the frozen encoder's spatial activations via projected gradient descent. At inference, non-negative least squares estimates per-image concept coefficients against the fixed basis.","core_discovery":"The central mechanism is the inversion of non-negative matrix factorization (NMF). Standard NMF decomposes a model's activations into a coefficient matrix and a basis matrix, both learned from data, producing unnamed factors. LAD fixes the coefficient matrix S to CLIP-derived spatial similarity maps for named concepts and learns only the basis W against the frozen encoder's activations. This means each basis vector is structurally tied to a named concept from the outset, and the reconstruction objective reshapes initial CLIP similarities so that concepts inconsistent with the encoder's evidence are suppressed. The paper's key empirical finding is that this language anchor changes which basis","pith_inferences":["The dependence on CLIP-derived similarity maps as the fixed coefficient matrix means LAD's concept quality is bounded by CLIP's spatial grounding fidelity; if CLIP's red-circle prompting mislocalizes a concept, the learned basis will be anchored to the wrong region, a risk the paper acknowledges but does not systematically quantify across concept types.","The observation that the language anchor changes which basis directions are discovered suggests a broader principle: any externally fixed coefficient structure, not necessarily language-derived, could steer factorization toward specific semantic regimes, opening the question of whether non-linguistic priors (e.g., anatomical atlases, spectral signatures) could serve as alternative anchors for doma","The trade-off between naming completeness and deletion faithfulness (LAD leads insertion but sometimes trails on deletion) implies that semantically constrained explanations may systematically under-cover decision-relevant but non-interpretable evidence, raising the question of whether the unnamed residual factors represent genuinely non-conceptual computation or simply concepts outside the LLM's "],"forward_implications":["If the language anchor genuinely steers decomposition toward classifier-meaningful structure, then concept-level auditing of deployed models becomes possible without retraining, enabling domain experts to inspect not just where a model looks but what named evidence it attributes to those regions.","The finding that removing the language anchor preserves accuracy but collapses deletion faithfulness suggests that accuracy alone is an insufficient proxy for explanation quality; faithfulness metrics may require semantic grounding to be meaningful.","The clinical domain results, where named concepts anchor to focal disease evidence and lead deletion faithfulness by a wide margin, suggest the method may be most valuable in domains where discriminative evidence is spatially localized and clinically named.","The negative result on Vision Transformers, where reconstruction transfers but spatial localization fails, indicates that the approach's success depends on the encoder's spatial geometry and that transformer-native formulations are needed for patch-token architectures.","The boundary bias observed in dermoscopy, where concepts localize to lesion borders rather than diagnostic interiors, reveals that named concept faithfulness is domain-dependent and that the method may surface where the model's evidence sits rather than asserting clinically complete reasoning."],"fun_headline_variants":["Inverted NMF ties classifier concepts to CLIP-grounded names","LAD names what classifiers see without retraining or model modification","Fixing language maps as NMF coefficients yields faithful named explanations","Removing the language anchor preserves accuracy but breaks attribution","Post-hoc concept naming via inverted NMF on a frozen encoder"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method assumes that CLIP-based localized similarity maps, constructed via red-circle visual prompting on a grid, provide a semantically meaningful and spatially accurate coefficient matrix that can anchor the decomposition. If these CLIP maps are noisy or do not align with the encoder's actual feature geometry, the learned basis may not correspond to faithful concepts.","fun_headline_variants_meta":{"raw":{"variants":["Inverted NMF ties classifier concepts to CLIP-grounded names","LAD names what classifiers see without retraining or model modification","Fixing language maps as NMF coefficients yields faithful named explanations","Removing the language anchor preserves accuracy but breaks attribution","Post-hoc concept naming via inverted NMF on a frozen encoder"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":1090,"prompt_tokens":1022,"completion_tokens":68,"prompt_tokens_details":null},"tokens_in":1022,"tokens_out":68,"duration_ms":25803,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T15:36:45.256214+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If replacing the fixed language-grounded coefficient matrix with learned coefficients (standard unsupervised NMF) produced equivalent or better deletion faithfulness (C-Del), the central claim that the language anchor is functional rather than cosmetic would be falsified. The paper reports the opposite: C-Del drops from 0.902 to 0.507 when the anchor is removed.","supporting_citations":[],"review_version":1}