Pith. sign in

REVIEW 4 major objections 5 minor 30 references

Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that meaningful concepts in CLIP embeddings are directional structures that break rotational invariance, and that this property can be turned into a statistically grounded post-hoc decomposition, with removing flagged…

desk verdict The rotation test is broken as written, but the decomposition and identification theory are worth a second look. read the letter →

arxiv 2506.13831 v1 pith:GDJG23MN submitted 2025-06-16 cs.LG cs.AI

Jitian Zhao , Chenghui Li , Frederic Sala , Karl Rohe This is my paper
classification cs.LGcs.AI MSC 62H2562F0368T07
keywords CLIPembeddingsconceptdecompositionVarimaxrotationinvariancehypothesistestingspuriouscorrelationreconstructionfidelityinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that meaningful concepts inside CLIP embeddings are directional structures that break rotational invariance, and that this property can be turned into a hypothesis test. It argues that random noise is statistically unchanged by rotation, whereas true semantic concepts prefer certain directions, so a test that rejects rotation invariance signals real structure. On top of the test, the paper develops a post-hoc decomposition that applies Varimax rotation to the singular vectors of the embedding matrix, yielding sparse, interpretable concept loadings while keeping near-SVD reconstruction fidelity. If the central claim is right, concept decompositions can be validated statistically and corrected without retraining, and the paper reports a 22.6% improvement in worst-group accuracy on Waterbirds after removing the concepts it flags as spurious.

What carries the argument

The load-bearing object is the left singular vector matrix $U$ of the normalized embedding matrix, together with the Varimax objective $v(U,R)$, which measures the maximum sparsity achievable by rotating the columns. The test generates a null distribution by independently rotating each row of $U$ with a random orthogonal matrix (Algorithm 4), then compares TS1 and TS2 against that null to produce p-values. The decomposition uses the Varimax-rotated SVD factors: image loadings are $\hat{Z} = UDR$ and the concept dictionary is $\hat{Y} = VR$, so concepts are orthogonal directions that are sparse over data points and interpretable through top-loading images and text descriptions.

What would settle it

Generate an $n \times d$ matrix with i.i.d. standard Gaussian entries, which the paper itself treats as its null model, compute the truncated left singular vectors $U$, and run the paper's test procedure (Algorithm 4 resampling plus TS1/TS2). If the row-wise rotation is a valid null, p-values across many independent datasets should be approximately uniform over $[0,1]$; if the null is invalid, small p-values will appear far more often than 5% of the time.

Watch

Extended reading notes

Core claim

The central claim is that rotation-sensitivity testing plus Varimax rotation yields concepts that represent robust, reproducible patterns rather than method-specific artifacts, while preserving SVD-level reconstruction fidelity. The paper formalizes "no concept structure" as a null model in which singular vectors are rotationally invariant, and proposes two test statistics: average absolute kurtosis across singular-vector columns (TS1) and the maximum Varimax sparsity objective over rotations (TS2). It then decomposes the embedding matrix as sparse loadings times an orthogonal concept dictionary obtained by rotating the SVD factors, and proves that under stated assumptions the Varimax objective recovers the true concept rotation up to permutation and sign. It also proves a lower bound showing that fixed, pre-defined concept vocabularies suffer unavoidable reconstruction error when misspecified, which motivates learning concepts from data. Empirically, the paper reports that removing the spurious concepts identified by the method improves Waterbirds worst-group accuracy by 22.6% and triples iWildCam prediction accuracy.

Load-bearing premise

The test's validity rests on one premise: that independently spinning each row of the singular-vector matrix around a random rotation mimics what the matrix would look like if the data had no preferred directions, and if that simulated "noise" is not the right comparison, every p-value in the paper is on shaky ground.

Editorial extensions

If this is right

  • Concepts extracted by the method come with a p-value for rotation-sensitive structure, so a practitioner can tell whether a reported concept is distinguishable from noise.
  • Because reconstruction fidelity stays near the SVD baseline at moderate concept counts, downstream zero-shot classification can be run in the concept space with little information loss.
  • Deleting the loadings of flagged spurious concepts is a retraining-free intervention that the paper shows lifts Waterbirds worst-group accuracy by 22.6% and triples iWildCam accuracy.
  • The reconstruction lower bound for fixed concept vocabularies implies that word-based or otherwise pre-specified concept dictionaries will leave an irreducible error whenever the true concepts are not a projection of that dictionary.
  • The identification theorem implies that when loadings are independent and super-Gaussian, the Varimax rotation finds the true concept axes up to permutation and sign, which is what makes the decomposition reproducible across runs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next check, not reported in the paper, is whether the row-wise rotation null yields calibrated p-values when $U$ is drawn from the paper's own Gaussian null; under that null $U$ is Haar-distributed on the Stiefel manifold, so the independent row rotations may not match the null distribution.
  • If calibrated, the same test could be applied to other foundation models' embeddings to audit which interpretable directions are statistically real rather than artifacts of a particular SVD run.
  • The demonstrated concept arithmetic (group-of-dogs minus single-dog plus single-bird retrieving groups of birds) suggests concepts behave like word embeddings do for analogies, which could be tested systematically across concept pairs.
  • Removing spurious concepts improved worst-group accuracy on three benchmarks, which points toward a general bias-mitigation recipe, but the paper does not examine whether removal degrades other axes of behavior, so that trade-off is an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a statistical framework for interpreting CLIP embeddings: a hypothesis test for rotation-sensitive structure in the left singular vectors of an embedding matrix, followed by a Varimax-based post-hoc concept decomposition with automatic text labeling. It claims that the test validates that discovered concepts are robust and reproducible rather than method-specific artifacts, that the decomposition achieves better reconstruction fidelity than SpLiCE, and that removing the identified spurious concepts improves Waterbirds worst-group accuracy by 22.6%. The paper also contains an identification theorem for Varimax rotations, a reconstruction-error lower bound for fixed-concept methods, and supporting empirical studies on ImageNet, Waterbirds, iWildCam, and CelebA.

Significance. If the statistical guarantees were valid, the framework would be a useful contribution: it offers a way to validate concept decomposition without retraining, a principled link between rotation invariance and the absence of concept structure, and an external comparison against SpLiCE. The empirical spurious-correlation removal results are interesting and the reconstruction-fidelity comparison is clearly presented. The paper also extends a published Varimax identification result and provides a clean lower bound for fixed-concept methods. However, the central statistical claim is not supported as written because the resampling procedure does not generate draws from the null model it defines, and the proof of the main distributional theorem does not cover the test statistic actually used.

major comments (4)
  1. [Section 2.2, Algorithm 4, Proposition 1, Example 2] Algorithm 4 does not sample from the null distribution defined by the paper. Under Example 2, which the paper proves in Appendix G.2, the left singular-vector matrix U of Gaussian noise is Haar-distributed on the Stiefel manifold: its rows are dependent and its columns are almost surely orthonormal. Algorithm 4 instead independently rotates each row of U by a uniform rotation in SO_k, producing a matrix whose columns are almost surely not orthonormal; such matrices have measure zero under the null. Consequently, the Monte-Carlo p-values in Algorithm 1 are not calibrated, because TS1 and TS2 are evaluated on matrices that are not draws from the null model. The white-noise control experiments cannot validate the procedure because the same misspecified resampler generates both the null samples and the p-values. A valid resampling for the paper's null would right-multiply U by a single Haar-distributed rotation in SO_k, which preserves the Stiefel constraint.
  2. [Theorem 1, Eq. (6), Appendix G.6] The theorem claims a standard normal limit for an equivalent rescaled version of TS1, but the proof does not establish this. Equation (6) defines TS3 using |kurtosis|, but the proof immediately replaces it with n * sum_j U_ji^4 - 3n/(n+2), dropping the absolute value. The proof then states that the relevant variables are i.i.d. and invokes the central limit theorem. However, the columns of a Haar-distributed U are dependent (they are exchangeable, not independent), and the absolute value is a nonlinear transformation that is not accounted for in the rescaling. Thus the proof does not cover the absolute-kurtosis statistic TS1 defined in Section 2.3.
  3. [Assumption 1, Theorem 2, Appendix G.7] The equality characterization in the proof of Theorem 2 is not valid under the stated assumptions. The proof's leading term has coefficient proportional to eta_j - 3*sigma_j^4, and the equality claim requires this coefficient to be strictly positive. Assumption 1 allows eta_j = 3*sigma_j^4 (with equal variances this makes the objective independent of the rotation), so the claimed uniqueness up to permutation can fail. In addition, Assumption 1 states kurtosis kappa >= 3 using the excess-kurtosis convention of Section 2.3, while the proof uses eta_j >= 3*sigma_j^4, which corresponds to raw kurtosis at least 3; these conditions are inconsistent. The theorem needs a strict super-Gaussianity condition stated with the same kurtosis convention used in the paper.
  4. [Example 4, Appendix G.4] The proof that Gaussian-mixture singular vectors are rotation-sensitive contains a numerical error. For the mixture (1/2)N(1,1) + (1/2)N(-1,1), each entry has variance 2, not 1, so Var((Av)_i) = Var((Av')_i) = 2 for the two directions considered in the proof. The claim that the variance of (Av')_i is 'apparently smaller than 1' is therefore false, and the proof does not establish the stated rotation sensitivity as written.
minor comments (5)
  1. [Section 5.3 / Table 2] The claimed 22.6% improvement on Waterbirds is the difference between the full zero-shot worst-group accuracy (38.1) and the spurious-removed accuracy (60.7); the baseline used for the comparison should be stated explicitly in the text, since Table 2 also reports an SVD-reconstructed baseline of 39.0.
  2. [Section 3 / Algorithm 2] Algorithm 2 performs Varimax on U D, whereas the hypothesis test in Section 2 operates on U. The relationship between the two objects should be clarified, especially because the theoretical identification result in Theorem 2 is stated for Z, not for U D.
  3. [References [23] and [24]] References [23] and [24] appear to cite the same work in two versions; they should be consolidated to avoid duplicate entries.
  4. [Section 5.2] The paper says its method 'outperforms other techniques in terms of reconstruction error' but only compares against SpLiCE. The claim should be restricted to the methods actually evaluated, or additional baselines should be included.
  5. [Appendix G.6] The notation in the proof of Theorem 1 switches between kurtosis of columns and sums of fourth powers without defining the centered or scaled quantities consistently; this makes it difficult to verify the moment calculations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the central identification and recovery results are proved in the paper, and the empirical claims are interventions rather than fitted predictions.

full rationale

The paper's derivation chain does not exhibit a circular reduction. The main identification result (Theorem 2) is proved in Appendix G.7 rather than imported from the co-authored Rohe-Zeng work; that prior work is published and is used mainly as motivation, and the proof in the paper states and uses Assumption 1 directly. The reconstruction lower bound (Theorem 3) is proved from Lemma 1, whose proof is also included in Appendix E. The hypothesis test is a standard Monte Carlo comparison: Algorithm 4 generates null replicas and Algorithm 1 compares observed statistics to their empirical null distribution; this is not a case where the target quantity is fitted and then relabeled a prediction. The most serious technical concern is the validity of the null resampling: Proposition 1 assumes i.i.d. rows, whereas under the paper's own Gaussian null (Example 2, Appendix G.2) the left singular-vector matrix U is Haar-distributed on the Stiefel manifold, so row-wise independent rotations do not preserve column orthogonality. That is a statistical mis-specification and a correctness risk, but it is not an identity between input and output, and no equation in the paper reduces the p-value to the observed statistic by construction. Similarly, the spurious-concept removal uses dataset-specific selection and is evaluated on the same benchmark, which weakens the external validity of the reported 22.6% improvement, but that figure is the outcome of an explicit intervention rather than a fitted value being passed off as a prediction. Self-citations occur (notably Rohe and Zeng, and Li et al.), but the load-bearing statements are accompanied by in-paper proofs, so the derivation does not reduce to those citations. Overall, no significant circularity is present.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The paper introduces no new physical entities, but it does introduce a modeling ontology: concepts as orthogonal linear directions, and rotation-sensitive structure as the property that makes those directions discoverable. The strongest assumptions are unverified for CLIP: independent entries and kurtosis of latent loadings, exact linear factorization, and the validity of row-wise rotation of singular vectors as a null sampler. The last assumption is the most fragile because it is used to produce all reported p-values.

free parameters (2)
  • Number of concepts k = 50 for spurious removal; varied in Figure 4
    User-chosen; all downstream numbers depend on it and no principled selection rule is given.
  • Number of spurious concepts removed per dataset = Not reported
    The spurious-removal experiment zeroes an unreported number of concept coefficients per dataset; without this count and a validation split the 22.6% gain is not reproducible.
assumptions (4)
  • ad hoc to paper Rows of U are i.i.d. from a rotationally invariant distribution, so independent row rotations reproduce the null distribution of the singular vectors.
    Proposition 1 requires i.i.d. rows. Under the Gaussian null (Example 2), U is Haar-distributed on the Stiefel manifold and its rows are not independent; Algorithm 4 therefore samples from a different distribution.
  • domain assumption Assumption 1: concept loadings Z have i.i.d. rows, independent entries, and kurtosis greater than or equal to 3.
    Needed for Theorem 2 identification; not checked on CLIP embeddings.
  • domain assumption Embeddings admit an exact linear factorization A = Z* C*^T with an orthogonal concept dictionary.
    Used in Theorem 3 and Algorithm 2; the paper states linearity as a limitation in the conclusion.
  • domain assumption The curated text descriptions from Gandelsman et al. faithfully label the semantics of learned concepts.
    Concept interpretation in Algorithm 3 and the spurious-removal experiments depend on the quality and coverage of these descriptions, as the paper itself acknowledges.
invented entities (2)
  • Orthogonal concept dictionary Y
    purpose: Represents each concept as an orthogonal direction in embedding space so loadings are sparse and independent
    Introduced as part of Algorithm 2; identifiability relies on Assumption 1, which is not verified, and no external benchmark confirms the axes are the true semantic concepts.
  • Rotation-sensitive concept structure
    purpose: The latent property that the hypothesis test detects and that justifies applying Varimax
    Postulated by Definition 1 and Examples 2 and 3; the proposed null test for it is invalid as written.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretation." pith.science (2026). https://pith.science/paper/GDJG23MN

@misc{pith2026250613831,
  author       = {Pith},
  title        = {Pith review of: Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GDJG23MN}},
  note         = {Machine review of arXiv:2506.13831}
}
read the original abstract

Concept-based approaches, which aim to identify human-understandable concepts within a model's internal representations, are a promising method for interpreting embeddings from deep neural network models, such as CLIP. While these approaches help explain model behavior, current methods lack statistical rigor, making it challenging to validate identified concepts and compare different techniques. To address this challenge, we introduce a hypothesis testing framework that quantifies rotation-sensitive structures within the CLIP embedding space. Once such structures are identified, we propose a post-hoc concept decomposition method. Unlike existing approaches, it offers theoretical guarantees that discovered concepts represent robust, reproducible patterns (rather than method-specific artifacts) and outperforms other techniques in terms of reconstruction error. Empirically, we demonstrate that our concept-based decomposition algorithm effectively balances reconstruction accuracy with concept interpretability and helps mitigate spurious cues in data. Applied to a popular spurious correlation dataset, our method yields a 22.6% increase in worst-group accuracy after removing spurious background concepts.

Figures

Figures reproduced from arXiv: 2506.13831 by the authors.

Figure 1
Figure 1. Visualization of singular vector loadings from CLIP embeddings from the ViT-B/32 backbone model, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Pipeline overview. CLIP embeddings from images and texts are processed to extract interpretable concepts. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Comparison of concept clusters obtained by our Varimax-rotated decomposition (left column) and raw [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Reconstruction quality versus number of concepts. Higher cosine similarity indicates better preservation [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Illustration of how p-values change with rank [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Comparison of bootstrap distributions and observed test statistics. The blue histograms show the distribu [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Top-24 concepts using our method with leading images and corresponding text descriptions. We observe [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Top-6 Waterbrids concepts with text descriptions. We noticed there are bird-focused concepts (e.g., first [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Demonstration of analogical reasoning with concepts. The equation [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 22 canonical work pages

  1. [1]

    Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP

    Sriram Balasubramanian, Samyadeep Basu, and Soheil Feizi. Decomposing and interpreting image represen- tations via text in vits beyond clip. arXiv preprint arXiv:2406.01583, 2024

    work page Pith review arXiv 2024

  2. [2]

    The iWildCam 2020 Competition Dataset

    Sara Beery, Elijah Cole, and Arvi Gjoka. The iwildcam 2020 competition dataset. arXiv preprint arXiv:2004.10340, 2020

    work page Pith review arXiv 2020

  3. [3]

    Interpreting clip with sparse linear concept embeddings (splice)

    Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P Calmon, and Himabindu Lakkaraju. Interpreting clip with sparse linear concept embeddings (splice). arXiv preprint arXiv:2402.10376, 2024

    arXiv 2024

  4. [4]

    Tests for high-dimensional covariance matrices

    Song Xi Chen, Li-Xin Zhang, and Ping-Shou Zhong. Tests for high-dimensional covariance matrices. Journal of the American Statistical Association, 105(490):810–819, 2010

    work page 2010

  5. [5]

    A holistic approach to unifying automatic concept extraction and concept importance estimation

    Thomas Fel, Victor Boutin, Louis Béthune, Rémi Cadène, Mazda Moayeri, Léo Andéol, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems, 36, 2024

    work page 2024

  6. [6]

    Interpreting clip’s image representation via text-based decomposition

    Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip’s image representation via text-based decomposition. In The Twelfth International Conference on Learning Representations, 2024

    work page 2024

  7. [7]

    Interpreting the second-order effects of neurons in clip

    Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting the second-order effects of neurons in clip. arXiv preprint arXiv:2406.04341, 2024

    arXiv 2024

  8. [8]

    Concept discovery and dataset exploration with singular value decomposition

    Mara Graziani, An-phi Nguyen, Laura O’Mahony, Henning Müller, and Vincent Andrearczyk. Concept discovery and dataset exploration with singular value decomposition. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy ML, 2023

    work page 2023

Show all 30 references
  1. [9]

    Nonlinear independent component analysis for principled disentanglement in unsupervised deep learning

    Aapo Hyvärinen, Ilyes Khemakhem, and Hiroshi Morioka. Nonlinear independent component analysis for principled disentanglement in unsupervised deep learning. Patterns, 4(10), 2023

    2023

  2. [10]

    Some optimal multivariate tests

    S John. Some optimal multivariate tests. Biometrika, 58(1):123–127, 1971

    1971

  3. [11]

    The varimax criterion for analytic rotation in factor analysis

    Henry F Kaiser. The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23(3):187–200, 1958. 10

    1958

  4. [12]

    Earnshaw, Imran S

    Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton A. Earnshaw, Imran S. Haque, Sara Beery, Jure Leskovec, A...

    2021

  5. [13]

    Concentration and regularization of random graphs

    Can M Le, Elizaveta Levina, and Roman Vershynin. Concentration and regularization of random graphs. Random Structures & Algorithms, 51(3):538–561, 2017

    2017

  6. [14]

    Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size

    Olivier Ledoit and Michael Wolf. Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size. The Annals of Statistics, 30(4):1081–1102, 2002

    2002

  7. [15]

    Spectral neural networks: Approximation theory and optimization landscape

    Chenghui Li, Rishi Sonthalia, and Nicolas Garcia Trillos. Spectral neural networks: Approximation theory and optimization landscape. arXiv preprint arXiv:2310.00729, 2023

    2023 arXiv

  8. [16]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023

    2023

  9. [17]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015

    2015

  10. [18]

    Significance test for sphericity of a normal n-variate distribution

    John W Mauchly. Significance test for sphericity of a normal n-variate distribution. The Annals of Mathemati- cal Statistics, 11(2):204–209, 1940

    1940

  11. [19]

    Linguistic regularities in continuous space word rep- resentations

    Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word rep- resentations. In Lucy Vanderwende, Hal Daumé III, and Katrin Kirchhoff, editors, Proceedings of the 2013 Conference of the North American Chapter of the Association for Comput...

    2013

  12. [20]

    Rotated word vector representations and their interpretability

    Sungjoon Park, JinYeong Bak, and Alice Oh. Rotated word vector representations and their interpretability. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 401–411, 2017

    2017

  13. [21]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  14. [22]

    On linear identifiability of learned representations

    Geoffrey Roeder, Luke Metz, and Durk Kingma. On linear identifiability of learned representations. In International Conference on Machine Learning, pages 9030–9039. PMLR, 2021

    2021

  15. [23]

    Vintage factor analysis with varimax performs statistical inference

    Karl Rohe and Muzhe Zeng. Vintage factor analysis with varimax performs statistical inference. Journal of the Royal Statistical Society Series B: Statistical Methodology, 2020

    2020

  16. [24]

    Vintage factor analysis with varimax performs statistical inference

    Karl Rohe and Muzhe Zeng. Vintage factor analysis with varimax performs statistical inference. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(4):1037–1060, 07 2023. ISSN 1369-7412. doi: 10.1093/jrsssb/qkad029. URL https://doi.org/10.1093/jrsssb/qkad029

    2023 doi

  17. [25]

    Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019

    1911 arXiv

  18. [26]

    Ehinger, and Benjamin I

    Ruihan Zhang, Prashan Madumal, Tim Miller, Krista A. Ehinger, and Benjamin I. P. Rubinstein. Invertible concept-based explanations for cnn models with non-negative concept activation vectors. Proceedings of the AAAI Conference on Artificial Intelligence, 35(13):11682–11690, Ma...

    2021 doi

  19. [27]

    no preferred direction

    Yilin Zhang and Karl Rohe. Understanding regularized spectral clustering via graph conductance. Advances in Neural Information Processing Systems, 31, 2018. 11 Supplementary materials contain additional experiment details, results, and proofs. We provide glossary table in Sect...

    2018

  20. [28]

    This step introduces randomness into the data transformation process

    Row-wise Random Rotations: To generate conditionally rotation-invariant data, we applied random rotations to each row of ˜U. This step introduces randomness into the data transformation process

  21. [29]

    Generation of Synthetic Data: The white-noise image and pure white-noise embeddings were generated using random sampling from standard Gaussian distributions. 13 Dataset Task Spurious Attribute #Images #Classes Waterbirds Bird Species Background 4,795 2 iWildCam Animal Species...

  22. [30]

    A {bird_type} with a {background_type} background

    We observed that for white noise image embeddings, p-values increase as k increases, which aligns with our expectation. For white noise embedding, we observed p-values oscillate around 0.5 and show no clear pattern as k changes. This aligns with our theoretical results from Ex...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.