Pith. sign in

REVIEW 4 major objections 8 minor 1 cited by

Image Reconstruction as a Tool for Feature Analysis

T0 review · 4 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The central claim is that reconstruction fidelity measures how much visual information an encoder's features preserve: multitask-trained SigLIP2 beats contrastive-only SigLIP at every resolution, and orthogonal feature-space rotations…

desk verdict A useful reconstruction tool for probing vision encoders, but the central SigLIP-vs-SigLIP2 attribution is undercut by an internal contradiction about training data. read the letter →

arxiv 2506.07803 v1 pith:NCLZVOGG submitted 2025-06-09 cs.CV

Eduard Allakhverdov , Dmitrii Tarasov , Elizaveta Goncharova , Andrey Kuznetsov This is my paper
classification cs.CV
keywords imagereconstructionvisionencoderinterpretabilityfeatureinversioncontrastivepretrainingmultitasklatentspacemanipulationorthogonaltransformationscolorediting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that how faithfully an image can be rebuilt from a vision encoder's internal feature tensor is a direct measure of how much visual information those features preserve. Applying this reconstruction test to SigLIP and SigLIP2, two encoder families that share architecture and parameter count but differ mainly in training objective, the paper finds that the multitask-trained SigLIP2 produces significantly higher-fidelity reconstructions than the contrastive-only SigLIP at every tested resolution. The same pipeline ranks a wider set of ViT encoders by feature informativeness and shows that higher input resolutions yield more invertible features. The paper further shows that learned linear operators on feature tensors behave like their pixel-space counterparts: an orthogonal self-inverse rotation swaps red and blue channels, a linear operator suppresses the blue channel like a projection, and a linear map performs credible colorization. If correct, the method turns any frozen vision encoder plus a small trained decoder into a general probe of what the encoder keeps.

What carries the argument

The load-bearing object is the trained reconstructor $R_\theta$: a small transformer composed of four blocks with multi-head self-attention and feed-forward sublayers, followed by upsampling layers interleaved with residual blocks, mapping a feature tensor $f = E(i)$ back to the original image $i$ under an $\ell^2$ reconstruction loss $\mathcal{L}_{\mathrm{rec}} = \mathbb{E}\|i - R_\theta(E(i))\|_2^2$. Each spatial feature vector is channel-wise normalized before decoding, which suppresses norm outliers common in CLIP-style features. For the manipulation experiments, the central identity is the orthogonal Procrustes solution $Q = \arg\min_{Q} \sum_j \|f_j^* - Q f_j\|^2$ subject to $Q^\top Q = I_c$ and $QQ = I_c$, learned from paired features $(E(i), E(i^*))$ of images and their pixel-space transforms; applying $Q$ in latent space is then shown to reproduce the pixel-space transform after decoding, and eigenvalue analysis of learned operators links them to the algebra of their image-space counterparts.

What would settle it

Train a contrastive-only SigLIP model on the same multilingual WebLI data that SigLIP2 used: if its reconstruction fidelity matches SigLIP2's rather than SigLIP's, the paper's attribution of the gap to the training objective is falsified.

Watch

Extended reading notes

Core claim

The central discovery, stated on the paper's own terms, is that the training objective determines how much of the original image survives in a vision transformer's feature tensor. Because SigLIP and SigLIP2 share architecture, parameter count, and the same WebLI data source, the reconstruction gap between them is attributed to pretraining: the multitask objective (sigmoid contrastive plus image captioning, self-distillation, and masked prediction) preserves structural detail that pure contrastive learning discards. Fidelity is measured by training a decoder on COCO image-feature pairs with a frozen encoder and scoring reconstructions on the COCO validation split with CLIP and SigLIP2 cosine similarity, and SigLIP2 wins at 224, 256, 384, and 512 pixels with statistically significant margins under both Wilcoxon signed-rank and bootstrap tests. The same pipeline yields two further claims: higher-resolution feature tensors retain more visual information, and pixel-space color operations correspond to simple linear or orthogonal operators in feature space, whose eigenvalue behavior mirrors the algebra of the corresponding image-space operation (self-inverse rotations for channel swaps, projection-like spectra for blue-channel suppression).

Load-bearing premise

The load-bearing premise is that the reconstruction gap between SigLIP and SigLIP2 comes from the training objective alone, even though the two models were trained on different data subsets (English-only versus multilingual WebLI) and every reconstruction is judged through a decoder trained on COCO.

Editorial extensions

If this is right

  • Reconstruction fidelity becomes a model-agnostic proxy for how much visual information an encoder keeps, usable on any ViT-based encoder without training probes or linear readouts.
  • For applications that need dense, invertible image features, multitask pretraining such as SigLIP2's is preferable to contrastive-only pretraining, since it preserves more structural detail at equal architecture and parameter count.
  • Input resolution should be part of encoder selection alongside architecture and objective, because higher-resolution feature tensors consistently prove more informative.
  • Pixel-space color edits can be driven by simple learned operators in feature space, opening a route to predictable, semantically grounded image editing through latent-space manipulation.
  • The encoder rankings from the appendix suggest image-based or multitask pretraining and higher resolution, more than raw parameter count, drive reconstruction fidelity across model families.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cleanest follow-up control would be a contrastive-only SigLIP retrained on SigLIP2's exact multilingual data subset, since that comparison separates the training objective from the training data as causes of the reconstruction gap.
  • If the linear colorization result generalizes, the same paired-feature linear-mapping recipe could probe whether other semantic attributes such as texture, lighting, or object identity are likewise encoded linearly in the same feature space.
  • Because all reconstructions are scored through decoders and similarity measures trained on natural-image statistics, the encoder ranking may partly reflect distribution overlap with photographic images; stability of the ranking under a different reconstruction dataset would test that.
  • The paper's own limitation note names convolutional encoders as the next target, so under the paper's logic a ConvNeXt-style encoder trained with an image-reconstruction objective should reconstruct more faithfully than a contrastive-only ViT of comparable size.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes a reconstruction-based interpretability method for vision encoders: a learned decoder maps frozen encoder features back to pixels, and reconstruction fidelity is used as a proxy for how much visual information the encoder preserves. The authors compare SigLIP and SigLIP2 across four resolutions, report that SigLIP2 reconstructions are statistically better under CLIP and SigLIP2 similarity scores, and extend the method to a range of ViT-based encoders. They also study feature-space manipulations: a learned orthogonal/self-conjugate matrix that swaps red and blue channels, a linear operator for blue-channel suppression, and a linear operator for colorization. The central claims are that image-based pretraining objectives retain more visual information than contrastive-only objectives, that higher resolution increases reconstruction fidelity, and that orthogonal feature rotations correspond to interpretable color edits.

Significance. If the central claims were established, the reconstruction pipeline would be a useful and inexpensive tool for probing vision-encoder representations, and the SigLIP/SigLIP2 comparison would provide a clean causal statement about training objectives. The paper ships code and weights, and the statistical testing on the SigLIP/SigLIP2 difference is thorough (permutation/wilcoxon with very small p-values). The feature-manipulation experiments, especially the color-swap operator, are creative and the visual results are striking. However, the main causal attribution is currently confounded, and several supporting analyses are either internally inconsistent or less decisive than the text suggests. The method itself is promising and the empirical material is substantial, but the paper needs additional control experiments and more careful framing before the headline conclusions can be accepted.

major comments (4)
  1. [Section 5.1 / Section 7] The central claim that SigLIP and SigLIP2 'differ only in their training objective' is contradicted within the manuscript. Section 5.1 states that SigLIP uses the English-only WebLI subset while SigLIP2 uses a multilingual corpus, and explicitly concedes that 'these subsets are not identical.' Section 7, however, asserts that these models 'share identical architectures, parameter counts, and datasets but differ only in their optimization objectives.' This is an internal inconsistency. Because training-data distribution and scale are known to affect representation quality, the statistically significant reconstruction gap in Table 1 cannot be attributed to the objective alone. A control that holds the pretraining data fixed (e.g., training a SigLIP-style contrastive-only model on the same multilingual corpus, or at minimum explicitly re-framing the conclusion as comparing a combination of objective and data) is required before the headline causal statement is supported.
  2. [Section 4.2.1 / Section 5.2] The claim that 'orthogonal rotations — rather than spatial transformations — control color encoding' (Abstract) is not established by the experiments. The hypothesis in Section 4.2.1 posits that the feature-space operator Af is orthogonal and self-inverse; the estimation then solves an orthogonal Procrustes problem, thereby imposing orthogonality rather than discovering it. The ablation in Figure 6 compares orthogonal self-conjugated, orthogonal, and linear solutions, but all three yield visually similar reconstructions (Figure 12, columns 4-7), and the linear operator, after normalization, also reproduces the swap. This does not demonstrate that orthogonality is a uniquely correct characterization. Furthermore, the text never tests spatial transformations or other non-orthogonal alternatives, so the comparative claim 'rather than spatial transformations' is unsupported. The authors should either soften the claim to 'orthogonal operators provide a good fit' or add experiments that distinguish among transformation families (e.g., by measuring reconstruction error quantitatively against a spatial-shift baseline).
  3. [Section 4.2.2 / Section 5.3] The eigenvalue analysis for blue-channel suppression is largely tautological and does not provide evidence for the proposed feature-space hypothesis. The argument that repeated application of Af converges to a projection and hence its eigenvalues are either 1 or have magnitude less than 1 holds for any linear contraction; it is not a distinctive prediction of the claim that feature-space operations mirror image-space operations. The empirical validation in Figure 7 is qualitative (visual inspection of eigenvalue scatter and reconstructions), with no quantitative metric or comparison to a null distribution. To make this experiment informative, the authors should compare the learned Af against a baseline (e.g., a random operator with the same spectral radius) and report a quantitative measure of how well the repeated operator matches the pixel-space projection.
  4. [Section 3 / Section 5.1 / Appendix A] The use of reconstruction fidelity as an absolute measure of 'how much visual information an encoder preserves' is not validated for decoder dependence. All experiments use a single reconstructor architecture (four transformer blocks plus upsampling) trained on a fixed 115k-image COCO subset with an ℓ2 loss. Because the decoder and training distribution are fixed, the ranking of encoders in Figure 9 and Table 1 could reflect the decoder's inductive bias or COCO-specific coverage rather than intrinsic properties of the encoders. The paper's Limitations section acknowledges this, but no control experiment is provided (e.g., a different decoder architecture, a different training set, or a perceptual loss). Additionally, the evaluations in Appendix A use only CLIP and SigLIP2 encoders as scoring functions, which may favor encoders with similar pretraining; no reconstruction-based metrics (PSNR, LPIPS) are reported. Without these controls, the strong information-ranking claims are not fully supported.
minor comments (8)
  1. [Section 5.1] The sentence 'SigLIP models are variant of CLIP' contains a grammar error ('are variant' should be 'are variants').
  2. [Table 2] The column header 'Sequence dength' is misspelled ('dength' should be 'length').
  3. [Figure 13 caption] The caption refers to 'fisheye space' in column 4; this appears to be a typo for 'feature space'.
  4. [References] The reference to 'Caron et al. (2024)' for WebLI in Section 5.1 is misplaced: WebLI is a web image-text dataset introduced in the context of SigLIP/PaLI, not in the cited 'Web-scale visual entity recognition' paper. Please cite the correct source for WebLI.
  5. [References] Radford et al. (2021a) and Radford et al. (2021b) are the same paper and should be merged into a single reference.
  6. [Section 6.1] The notation 'Rθ ⊥ ⊥(Ai, Af) | E' for conditional independence is nonstandard and is not defined; a brief explanation would help readers.
  7. [Section 5.4] The colorization experiment does not compare against a trivial baseline (e.g., predicting the mean color of the training set), so it is unclear how much of the success is due to the linear feature-space mapping versus simple global color priors.
  8. [Table 1] The caption states 'n = 1000samples' but the space is missing; also the relationship between 'n=1000' and the bootstrap 'B=100,000' is not explained in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reconstruction comparison and color-swap experiment are empirical, with held-out evaluation and an unconstrained ablation; the data-subset difference in Section 5.1 is a validity confound, not a definitional circularity.

full rationale

The paper's derivation chain is not circular. Reconstruction quality is defined operationally as the ability of an ℓ2-trained decoder to invert a frozen encoder, measured on COCO-val (Sections 3 and 5.1). The SigLIP/SigLIP2 ranking is obtained from measured reconstructions and statistical tests, not derived from the conclusion that image-based objectives retain more information. The paper does admit in Section 5.1 that SigLIP uses the English-only WebLI subset while SigLIP2 uses a multilingual corpus, and the Conclusion's 'identical datasets' statement is inconsistent with that admission; however, this is an attribution/confound problem, not a circular reduction, because the reconstruction scores are independent measurements of held-out images. The color-swap operator is fitted via orthogonal Procrustes with a self-inverse projection, so the orthogonality of Q is partly imposed; but the experiment's substantive content is the held-out generalization of Q and the reconstructor to unseen COCO-val images, and the ablation in Figure 6 includes an unconstrained linear operator whose eigenvalues also cluster near ±1, providing independent evidence beyond the imposed constraint. No load-bearing self-citation or imported uniqueness theorem appears. Thus the central claims rest on external measurements rather than on equations that reproduce their inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the fitted operators and reconstructor are models, not independently evidenced entities. The central claim rests on several domain assumptions about the validity of reconstruction-based metrics and on fitted linear operators whose properties are partly imposed by the method.

free parameters (4)
  • color-swap operator Q = Orthogonal matrix in R^{768x768}, fit via Procrustes on 1024 COCO-val pairs
    The claim that color swaps are orthogonal rotations in feature space is tested using this fitted matrix; the orthogonality and self-inverse constraints are imposed before fitting.
  • blue-suppression linear operator Af = Linear matrix fit to paired features with alpha=0.9
    Eigenvalue analysis in Section 5.3 is applied to this fitted operator; the property that eigenvalues have magnitude <1 is reported as confirmation of the Section 4.2.2 hypothesis.
  • colorization linear operator Af = Linear matrix fit to grayscale/color feature pairs
    Used in Section 5.4 to claim that colorization can be approximated linearly in feature space.
  • reconstructor hyperparameters = lr=3e-4, batch size 10, 40 epochs, Adam
    Chosen by hand in Table 3; reconstruction quality, and therefore the encoder comparison, depends on these choices.
assumptions (4)
  • domain assumption Encoder features are approximately invertible by a learned decoder.
    Section 3.1 defines the ideal R_theta(f)=i and trains R_theta with L2 loss; the validity of reconstruction quality as an information measure depends on this invertibility being achievable.
  • domain assumption L2 reconstruction loss and cosine similarity in CLIP/SigLIP2 space are adequate fidelity measures.
    Section 5.1 uses CLIP and SigLIP2 cosine similarity as the evaluation metric; no perceptual loss or human evaluation is used, and scoring SigLIP2 reconstructions with SigLIP2 partly measures the model against itself.
  • domain assumption SigLIP and SigLIP2 differ only by training objective.
    Section 5.1 describes SigLIP as English-only WebLI and SigLIP2 as multilingual WebLI, so the datasets are not identical; the paper's central comparison assumes the difference is attributable to the objective.
  • ad hoc to paper Pixel-space transformations correspond to linear or orthogonal feature-space operators.
    Section 4.1 hypothesizes the correspondence; Section 4.2.1 imposes orthogonality and self-inverse structure rather than deriving it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Image Reconstruction as a Tool for Feature Analysis." pith.science (2026). https://pith.science/paper/NCLZVOGG

@misc{pith2026250607803,
  author       = {Pith},
  title        = {Pith review of: Image Reconstruction as a Tool for Feature Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NCLZVOGG}},
  note         = {Machine review of arXiv:2506.07803}
}
read the original abstract

Vision encoders are increasingly used in modern applications, from vision-only models to multimodal systems such as vision-language models. Despite their remarkable success, it remains unclear how these architectures represent features internally. Here, we propose a novel approach for interpreting vision features via image reconstruction. We compare two related model families, SigLIP and SigLIP2, which differ only in their training objective, and show that encoders pre-trained on image-based tasks retain significantly more image information than those trained on non-image tasks such as contrastive learning. We further apply our method to a range of vision encoders, ranking them by the informativeness of their feature representations. Finally, we demonstrate that manipulating the feature space yields predictable changes in reconstructed images, revealing that orthogonal rotations (rather than spatial transformations) control color encoding. Our approach can be applied to any vision encoder, shedding light on the inner structure of its feature space. The code and model weights to reproduce the experiments are available in GitHub.

Figures

Figures reproduced from arXiv: 2506.07803 by the authors.

Figure 1
Figure 1. A frozen vision model generates image embeddings, which are then processed by a reconstructor model that learns to approximate image reconstruction. Let i ∈ R H×W×3 denote an input image, where H, W are the width and height of the input image, and 3 is the number of input channels. A feature extractor E : R H×W×C −→ R h×w×c maps i to a latent feature tensor f = E(i) (h and w are patch size, and c is a hidden dimensi… view at source ↗
Figure 2
Figure 2. Original image (left) and spatial activation maps for four selected channels of its encoded feature tensor f, illustrating how individual feature channels capture coherent image structures. (a) Q Matrix Calculation (b) Q Matrix Application [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Scheme for computing and subsequently applying the matrix Q in feature space. channels of i, the reconstructed channels still mirror the modified image. Hence, the corresponding feature-space operator Af must preserve this per-channel replication property. A natural choice is an orthogonal transformation applied independently to each spatial token. In addition, just as swapping the red and blue channels twice leaves… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Comparison of SigLIP and SigLIP2 reconstruction performance and visual results. The motivation for choosing this series of models lies in the similarity of the training and architectural details, whereas the only difference is in the training objectives. Both models ar…
Figure 5
Figure 5. Figure 5: Color-swap via orthogonal rotations in SigLIP2 feature space. Each row presents: (1) the original image, (2) its reconstruction from encoder features, (3) the image after swapping red and blue channels in pixel space, (4) the reconstruction of the pixel-swapped image, …
Figure 6
Figure 6. Figure 6: Color swap operator properties ablation. Rows: (1) sample for transformed image reconstruction, (2) Af operator eigenvalues visualization, (3) values of real part of Af eigenvalues. Columns: (1) self-conjugated and orthogonal operator constraints, (2) orthogonal only c…
Figure 7
Figure 7. Figure 7: Ablation of operator properties for blue-channel suppression with coefficient α = 0.9. Rows: (1) Sample for transformed image reconstruction, (2) Af operator eigenvalues visualization, (3) Values of real part of Af eigenvalues. Columns: (1) Single application of the op…
Figure 8
Figure 8. Figure 8: Examples of solving the colorization problem by applying a linear transformation in the feature space. remains constant. Higher input resolution makes E more nearly invertible, but does not guarantee exact recovery. 2. Finite training coverage. We train Rθ on a limited…
Figure 9
Figure 9. Figure 9: Encoder performance comparison on the COCO val set, showing average CLIP similarity and SigLIP2 similarity between original images and their reconstructions for each vision encoder. Higher bars indicate better alignment of reconstructed images with the originals under …
Figure 10
Figure 10. Figure 10: Qualitative comparison of original images and their reconstructions obtained from different vision encoders. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Qualitative analysis of the SigLIP and SigLIP2 reconstruction samples. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Color-swap via simple transformations in SigLIP2 feature space. Each row presents: (1) the original image, (2) its reconstruction from encoder features, (3) the image after swapping red and blue channels in pixel space, (4) the reconstruction of the pixel-swapped imag…
Figure 13
Figure 13. Figure 13: B-channel suppression via linear transformations in SigLIP2 feature space. Each row presents: (1) the original image, (2) its reconstruction from encoder features (3) reconstruction obtained by quadrupling the corresponding linear blue channel suppression operator dir…
Figure 14
Figure 14. Figure 14: Examples of solving the colorization problem by applying a linear transformation in the feature space. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reconstruction Alignment Improves Unified Multimodal Models

    cs.CV 2025-09 conditional novelty 6.0 of 10

    RECA, a self-supervised post-training objective that conditions unified multimodal models on their own visual understanding embeddings to reconstruct input images, improves text-to-image and editing benchmarks across ...

Reference graph

Works this paper leans on

29 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    Caron, M., Fathi, A., Schmid, C., and Iscen, A. (2024). Web-scale visual entity recognition: An llm-driven data approach

    work page 2024

  2. [2]

    Caron, M., Touvron, H., Misra, I., J \'e gou, H., Mairal, J., Bojanowski, P., and Joulin, A. (2021a). Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision , pages 9650--9660

    work page 2021

  3. [3]

    Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., and Joulin, A. (2021b). Emerging properties in self-supervised vision transformers. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 9630--9640

    work page 2021

  4. [4]

    Chen, Z., Wang, W., Cao, Y., Liu, Y., Gao, Z., Cui, E., Zhu, J., Ye, S., Tian, H., Liu, Z., Gu, L., Wang, X., Li, Q., Ren, Y., Chen, Z., Luo, J., Wang, J., Jiang, T., Wang, B., He, C., Shi, B., Zhang, X., Lv, H., Wang, Y., Shao, W., Chu, P., Tu, Z., He, T., Wu, Z., Deng, H., Ge, J., Chen, K., Zhang, K., Wang, L., Dou, M., Lu, L., Zhu, X., Lu, T., Lin, D.,...

    work page 2025

  5. [5]

    Darcet, T., Oquab, M., Mairal, J., and Bojanowski, P. (2024). Vision transformers need registers. In The Twelfth International Conference on Learning Representations

    work page 2024

  6. [6]

    K., and Wickstrøm, K

    Dorszewski, T., Tětková, L., Jenssen, R., Hansen, L. K., and Wickstrøm, K. K. (2025). From colors to classes: Emergence of concepts in vision transformers

    work page 2025

  7. [7]

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations

    2021

  8. [8]

    and Brox, T

    Dosovitskiy, A. and Brox, T. (2016). Inverting visual representations with convolutional networks. pages 4829--4837

    work page 2016

Show all 29 references
  1. [9]

    M., Jain, A., Schmidt, L., Toshev, A., and Shankar, V

    Fang, A., Jose, A. M., Jain, A., Schmidt, L., Toshev, A., and Shankar, V. (2023). Data filtering networks. arXiv preprint arXiv:2309.17425

    2023 arXiv

  2. [10]

    Fang, Y., Sun, Q., Wang, X., Huang, T., Wang, X., and Cao, Y. (2024). Eva-02: A visual representation for neon genesis. Image and Vision Computing , 149:105171

    2024

  3. [11]

    Gao, Z., Chen, Z., Cui, E., Ren, Y., Wang, W., Zhu, J., Tian, H., Ye, S., He, J., Zhu, X., Lu, L., Lu, T., Qiao, Y., Dai, J., and Wang, W. (2024). Mini-internvl: A flexible-transfer pocket multimodal model with 5

    2024

  4. [12]

    L., and Choi, Y

    Hessel, J., Holtzman, A., Forbes, M., Bras, R. L., and Choi, Y. (2021). Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718

    2021 arXiv

  5. [13]

    Kazemi, H., Chegini, A., Geiping, J., Feizi, S., and Goldstein, T. (2024). What do we learn from inverting clip models?

    2024

  6. [14]

    Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  7. [15]

    C., Lo, W.-Y., Doll \'a r, P., and Girshick, R

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Doll \'a r, P., and Girshick, R. (2023). Segment anything. arXiv:2304.02643

    2023 arXiv

  8. [16]

    Kornblith, S., Norouzi, M., Lee, H., and Hinton, G. (2019). Similarity of neural network representations revisited. In Proceedings of the 36th International Conference on Machine Learning , volume 97, pages 3519--3529. PMLR

    2019

  9. [17]

    T., Isola, P., Globerson, A., Irani, M., and Mosseri, I

    Lang, O., Gandelsman, Y., Yarom, M., Wald, Y., Elidan, G., Hassidim, A., Freeman, W. T., Isola, P., Globerson, A., Irani, M., and Mosseri, I. (2021). Explaining in style: Training a gan to explain a classifier in stylespace. arXiv preprint arXiv:2104.13369

    2021 arXiv

  10. [18]

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In Computer vision--ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v...

    2014

  11. [19]

    Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022). A convnet for the 2020s. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 11966--11976

    2022

  12. [20]

    and Vedaldi, A

    Mahendran, A. and Vedaldi, A. (2016). Visualizing deep convolutional neural networks using natural pre-images. 120(3):233–255

    2016

  13. [21]

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H. V., Szafraniec, M., Khalidov, V., Fernandez, P., HAZIZA, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.-Y., Li, S.-W., Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jegou, H., Maira...

    2024

  14. [22]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021a). Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748--8763. PmLR

    2021

  15. [23]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021b). Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748--8763. PmLR

    2021

  16. [24]

    Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., and Dosovitskiy, A. (2021). Do vision transformers see like convolutional neural networks? In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W., editors, Advances in Neural Information Processing Syst...

    2021

  17. [25]

    F., Alabdulmohsin, I., Parthasarathy, N., Evans, T., Beyer, L., Xia, Y., Mustafa, B., et al

    Tschannen, M., Gritsenko, A., Wang, X., Naeem, M. F., Alabdulmohsin, I., Parthasarathy, N., Evans, T., Beyer, L., Xia, Y., Mustafa, B., et al. (2025). Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv ...

    2025 arXiv

  18. [26]

    Wu, Z., Lischinski, D., and Shechtman, E. (2021). Stylespace analysis: Disentangled controls for stylegan image generation. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 12858--12867

    2021

  19. [27]

    E., Huang, P.-Y., Howes, R., Sharma, V., Li, S.-W., Ghosh, G., Zettlemoyer, L., and Feichtenhofer, C

    Xu, H., Xie, S., Tan, X. E., Huang, P.-Y., Howes, R., Sharma, V., Li, S.-W., Ghosh, G., Zettlemoyer, L., and Feichtenhofer, C. (2024). Demystifying clip data

    2024

  20. [28]

    Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. (2023). Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision , pages 11975--11986

    2023

  21. [29]

    Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A., and Kong, T. (2022). Image BERT pre-training with online tokenizer. In International Conference on Learning Representations

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.