Pith. sign in

REVIEW 4 major objections 6 minor 49 references

Multimodal Structure Learning: Disentangling Shared and Specific Topology via Cross-Modal Graphical Lasso

T0 review · 4 major / 6 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Download PDF Read on arXiv ↗

Pith's one-line read Cross-Modal Graphical Lasso recovers shared and class-specific conditional-dependency graphs from aligned vision-language features, improving generative classification and dense segmentation.

desk verdict Solid multimodal assembly of joint GLasso/CSSL with attention-footprint priors; SOTA claims are plausible but rest on an unvalidated prior-to-partial-correlation link and decoupled training. read the letter →

arxiv 2604.03953 v1 submitted 2026-04-05 cs.CV cs.LG

Fei Wang , Yutong Zhang , Xiong Wang This is my paper · ORCID
classification cs.CVcs.LG
keywords graphicallassomultimodallearningprecisionmatrixcommon-specificstructurecross-attentionvision-languagesemanticsegmentationADMM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Interpretable multimodal learning depends on sparse graphs of conditional feature dependencies, yet standard Graphical Lasso breaks on vision-language data because of high-dimensional noise, misaligned modalities, and the mixing of shared versus category-specific structure. CM-GLasso renders text as images so a single vision-language encoder places both modalities in one embedding space, then distills patches into a small set of semantic nodes whose attention footprints supply a spatial prior for the graph. It unifies tailored Graphical Lasso and common-specific structure learning in one ADMM objective that estimates an invariant precision matrix and class-specific residual matrices together, using the prior to soften penalties on likely edges. The resulting graphs drive a parameter-free generative classifier and a topology-aware segmentation decoder. Across eight natural and medical benchmarks the method reports new state-of-the-art accuracy and mIoU, arguing that explicit shared-plus-specific topology is a practical inductive bias for both tasks.

What carries the argument

Cross-Modal Graphical Lasso (CM-GLasso): the joint objective that unifies tailored Graphical Lasso with common-specific structure learning, solved by ADMM, where cosine similarity of cross-attention footprints from an auxiliary modality reweights the ℓ1 penalties on the class-specific precision residuals.

What would settle it

On synthetic data with known ground-truth precision supports embedded into a vision-language space, check whether CM-GLasso recovers the true edges more accurately than prior-free joint glasso or two-stage baselines; if the cross-modal attention prior does not improve edge recovery, the central mechanism fails.

Watch

Extended reading notes

Core claim

CM-GLasso establishes that jointly estimating a shared precision matrix and class-specific residual matrices under cross-modal attention priors, inside a single ADMM-optimized objective, disentangles invariant and category-specific topologies without the error accumulation of two-stage pipelines, and that those graphs improve generative classification and dense semantic segmentation on natural and medical benchmarks.

Load-bearing premise

The method treats how much two semantic nodes co-attend to the same patches in one modality as a reliable signal of whether those nodes are truly conditionally dependent in the other modality.

Editorial extensions

If this is right

  • Generative classification can score classes by Gaussian log-likelihood under the learned precision matrices with no extra trainable parameters.
  • Segmentation can route messages separately along synergistic (negative) and competitive (positive) edges according to the sign of precision entries.
  • Rendering text as images lets language attention footprints serve as structural priors for vision graphs, and vice versa, inside one encoder.
  • When eBIC selects zero prior strength the method reduces to ordinary joint glasso, so uninformative priors do not force structure.
  • The same shared-plus-specific topology backbone feeds both the classification and segmentation heads.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same attention-footprint prior could transfer structure from any dense modality once a shared encoder exists.
  • Offline ADMM after proxy supervision offers a practical pattern for injecting exact convex graphical models into deep nets without unrolling.
  • Scaling to thousands of classes will require low-rank or hierarchical substitutes for per-class eigendecompositions, as the cubic cost already flags.
  • If attention co-occurrence approximates partial correlation, similar priors could regularize other conditional-independence models beyond Gaussian graphical models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CM-GLasso, a framework that estimates sparse multimodal precision matrices by (i) rendering text as images and encoding both modalities with a shared SigLIP 2 ViT, (ii) distilling patches into p semantic nodes via learnable cross-attention prototypes whose attention footprints yield a cross-modal prior, and (iii) jointly optimizing a common precision matrix Θ_com and class-specific matrices S^(c) under a tailored-GLasso + CSSL objective solved by ADMM. Nonparanormal transforms enforce approximate Gaussianity; decoupled proxy CE training freezes features before offline ADMM. Learned topologies drive a parameter-free generative classification head and a sign-aware graph message-passing segmentation head. Across eight natural and medical benchmarks the method reports SOTA generative classification and competitive dense segmentation, with ablations supporting text rendering, cross-attention nodes, nonparanormalization, joint vs two-stage optimization, and combined Θ_com+S^(c).

Significance. If the central claims hold, the work is a meaningful bridge between classical sparse GGMs and modern vision–language encoders: it gives a concrete recipe for transferring spatial attention structure as a prior into joint graphical lasso, and shows that the resulting shared/specific topologies can serve as inductive bias for both generative classification and topology-aware segmentation. Strengths include a clean ADMM derivation with explicit positive-definiteness handling via Z^(c), systematic ablations (Tables 3–10), an eBIC-controlled prior strength with a k*=0 fallback, and evaluation spanning natural and medical domains. The contribution is primarily methodological and empirical rather than a new statistical theory result; its lasting value depends on whether the attention-footprint prior is shown to recover genuine conditional dependence rather than merely acting as a useful regularizer.

major comments (4)
  1. §3.2.4 Eqs. (4)–(6) and Eq. (11): The load-bearing novelty is that cosine similarity of auxiliary-modality attention footprints is a reliable prior on target-modality conditional dependence under HDLSS, so that the sigmoid-reweighted ℓ1 on S^(c) preserves the right edges. The paper never measures agreement between high-W_ij edges and nonzeros of an oracle or high-sample precision matrix, nor recovery of known partial correlations. Table 8 only shows that eBIC rejects uninformative self-priors (k*≈0 in >84% of cases); it does not validate that accepted cross-modal priors coincide with true dependence. Without such a check (synthetic GGMs with known Θ, or high-n reference estimates), the “tailored cross-modal” advantage and the attribution of SOTA gains to structural disentanglement remain under-supported relative to “joint GLasso + a helpful regularizer from encoder attention.”
  2. Abstract / §3.4 / Tables 1–2: The claim of a “new state-of-the-art in generative classification” is not cleanly separated from strong pretrained features and selective baselines. CIFAR-10 ACC 94.71% and several Table 1 comparators are well below standard modern discriminative numbers; if the intended scope is strictly GGM-style generative heads, that framing should be explicit and baselines should include other generative or precision-matrix classifiers under the same SigLIP 2 features. Otherwise the SOTA attribution overstates what the joint topology contributes versus the encoder and node distillation alone (cf. Table 4).
  3. §3.6 (Decoupled proxy supervision): Downstream losses never flow through ADMM; topologies are estimated offline from frozen proxy-trained features. The manuscript acknowledges suboptimality, but the Abstract and contribution bullets still speak of a “unified” joint objective that “guarantees” simultaneous disentanglement without multi-step error accumulation. That language should be tempered: joint ADMM avoids two-stage GLasso→CSSL error, yet the overall pipeline remains multi-stage (proxy CE → offline ADMM → frozen heads). A controlled experiment quantifying the gap versus (approximate) unrolled or alternating task-aware updates would strengthen the central claim.
  4. §4.1 / free parameters: The method depends on ρ, γ_s, k* (via eBIC), p, μ, and γ, with p constrained by p < n_c. Table 10 shows local stability of ρ,γ_s, but there is no systematic report of how p is chosen per dataset, how sensitive Θ_com vs S^(c) sparsity is to p, or whether the shared/specific split remains stable under reasonable p ranges. Because the interpretability and HDLSS story rest on the semantic-node graph, this choice is load-bearing and should be specified and ablated.
minor comments (6)
  1. Abstract opening sentence is ungrammatical: “However, sparse graph estimation techniques, such as Graphical Lasso (GLasso), to visual-linguistic domains is severely bottlenecked…” — missing verb (e.g., “applying … is severely bottlenecked”).
  2. §3.4 Eq. (10) and §4.1: γ appears in the implementation details but is not clearly identified with a term in Eq. (10) (which uses ρ and γ_s). Align notation.
  3. Algorithm for ADMM is deferred to “supplementary materials” / “Algorithm??” (§3.4.1, §3.6) and is not in the main text; for reproducibility the full pseudocode, stopping criteria, and initialization should be in the main paper or a clearly linked supplement.
  4. §3.1: Sign interpretation of θ_ij (θ_ij<0 synergistic, θ_ij>0 exclusive) is used heavily in H_S (Eqs. 19–20). A short sanity check or citation that this sign convention is preserved under nonparanormal transforms would help readers.
  5. Figures 2–3 captions are informative, but the main text does not quantify how often long-range edges in Θ^(c) align with the qualitative “sky–water” examples; a small quantitative edge-type breakdown would help.
  6. Related work on joint graphical lasso and tailored GLasso is appropriate; a brief contrast with other multimodal graph / structure-learning VLM methods (beyond PaliGemma/Ja) would situate novelty more clearly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; priors, eBIC selection, joint ADMM, and task heads are independent of the claimed precision-matrix outputs by construction.

full rationale

The derivation chain is a standard prior-guided sparse estimation pipeline, not a closed loop. Cross-modal priors W^(c,m' o m) are constructed solely from frozen SigLIP-2 attention footprints (Eqs. 4–6) before any precision-matrix estimation; they are never defined in terms of Θ_com or S^(c). The joint objective (Eq. 10) and ADMM updates (Eqs. 13–16) take empirical covariances Σ̂^(c) (after nonparanormal transform) plus those external priors as inputs and produce the precision matrices as outputs; nothing forces the recovered edges to equal the input attention co-occurrences. eBIC merely selects the scalar sharpness k* (with explicit k*=0 fallback that recovers ordinary joint GLasso), a conventional model-selection step that does not redefine the target quantity. Proxy CE training of prototypes is deliberately decoupled and frozen before offline ADMM, so the subsequent generative likelihoods (Eq. 18) and message-passing (Eq. 19) are not tautological re-statements of the CE loss. All load-bearing citations (GLasso, joint GLasso, nonparanormal, ADMM, eBIC, SigLIP) are external; no uniqueness theorem or ansatz is imported from the present authors. Empirical SOTA claims rest on held-out benchmark metrics, not on quantities that equal fitted inputs by construction. Hence the paper is self-contained against the circularity criteria.

Assumptions & free parameters 6 free parameters · 6 assumptions · 3 invented entities

The central claim rests on standard GGM/ADMM mathematics plus several domain and paper-specific modeling choices: Gaussianity after nonparanormal transform, reliability of attention co-occurrence as a partial-correlation prior, sufficiency of decoupled proxy training, and the text-as-image alignment trick. Free parameters (penalties, k*, node count) are grid- or eBIC-selected on the evaluation regime. Invented entities are methodological constructs (CM-GLasso, attention-footprint priors, cross-attention distillation nodes) validated only inside the paper’s benchmarks, without external independent measurements.

free parameters (6)
  • ρ (common ℓ1 penalty) = grid in {0.01,0.05,0.1,0.2}; best often ~0.05
    Grid-searched over {0.01,0.05,0.1,0.2}; controls sparsity of Θ_com and directly affects reported ACC/mIoU (Table 10).
  • γ_s (class-specific weighted ℓ1 penalty) = grid in {0.01,0.05,0.1,0.2}
    Grid-searched jointly with ρ; scales prior-weighted soft-thresholding on S^(c).
  • k* (sigmoid sharpness for prior weights) = candidate set {0..50}; mean ~15.6 for text→image
    Selected by eBIC per setting; decides how strongly cross-modal priors reweight edges (Eq. 11).
  • p (number of semantic graph nodes / prototypes) = dataset-dependent; example complexity uses p=50
    Chosen under hard constraint p < n_c to avoid HDLSS collapse; defines precision matrix size and capacity.
  • μ (ADMM augmented-Lagrangian penalty) = 1.0
    Fixed to 1.0 in experiments; affects Z/Θ/S update balance and convergence path.
  • γ (additional joint-objective coefficient, §4.1) = 0.5
    Set to 0.5 in implementation details; influences relative regularization strength.
assumptions (6)
  • domain assumption After nonparanormal rank transform, class-conditional features are adequately multivariate Gaussian for GLasso/GGM likelihoods.
    Invoked in §3.3–3.5; Shapiro–Wilk pass rate only rises to ~88%, not certainty.
  • standard math ADMM on the joint objective converges so that Z^(c) may be used as the positive-definite precision estimate for inference.
    Cited via Boyd et al.; used in Remark after Eq. 16–17.
  • ad hoc to paper Cosine similarity of auxiliary-modality attention footprints approximates useful priors on target-modality conditional dependence.
    Core modeling claim of §3.2.4 Eqs. 4–6; not derived from GGM theory.
  • ad hoc to paper Rendering text as a 224×224 image and encoding with the same SigLIP 2 ViT yields attention structures comparable to natural images.
    §3.2.1–3.2.2; efficacy depends on encoder OCR/layout capacity.
  • ad hoc to paper Decoupled proxy CE training of prototypes yields features whose offline covariances still support the claimed topologies.
    §3.6 explicitly trades global end-to-end optimality for stability.
  • domain assumption Sign of precision entries cleanly separates synergistic (θ<0) vs competitive (θ>0) pathways for message passing.
    §3.1 and Eq. 19–20; standard partial-correlation sign interpretation applied operationally.
invented entities (3)
  • CM-GLasso joint objective (tailored GLasso + CSSL with cross-modal prior weights)
    purpose: Simultaneously estimate Θ_com and {S^(c)} under attention-derived adaptive penalties without two-stage error accumulation.
    Defined in Eq. 10–11; performance evidence is only the paper’s own benchmarks.
  • Cross-attention distillation semantic prototypes Q_proto producing graph nodes and A footprints
    purpose: Reduce N_p patches to p nodes and extract spatial co-occurrence priors plus back-projection for segmentation.
    §3.2.3–3.2.4; architectural construct validated only internally.
  • Attention-footprint cross-modal prior matrix W_attn / W^(c,m'→m)
    purpose: Replace noisy empirical correlations with encoder attention co-occurrence as GLasso prior.
    Eqs. 4–6; no external measurement that these equal true partial correlations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Structure Learning: Disentangling Shared and Specific Topology via Cross-Modal Graphical Lasso." pith.science (2026). https://pith.science/paper/2604.03953

@misc{pith2026260403953,
  author       = {Pith},
  title        = {Pith review of: Multimodal Structure Learning: Disentangling Shared and Specific Topology via Cross-Modal Graphical Lasso},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2604.03953}},
  note         = {Machine review of arXiv:2604.03953}
}
read the original abstract

Learning interpretable multimodal representations inherently relies on uncovering the conditional dependencies between heterogeneous features. However, sparse graph estimation techniques, such as Graphical Lasso (GLasso), to visual-linguistic domains is severely bottlenecked by high-dimensional noise, modality misalignment, and the confounding of shared versus category-specific topologies. In this paper, we propose Cross-Modal Graphical Lasso (CM-GLasso) that overcomes these fundamental limitations. By coupling a novel text-visualization strategy with a unified vision-language encoder, we strictly align multimodal features into a shared latent space. We introduce a cross-attention distillation mechanism that condenses high-dimensional patches into explicit semantic nodes, naturally extracting spatial-aware cross-modal priors. Furthermore, we unify tailored GLasso estimation and Common-Specific Structure Learning (CSSL) into a joint objective optimized via the Alternating Direction Method of Multiplier (ADMM). This formulation guarantees the simultaneous disentanglement of invariant and class-specific precision matrices without multi-step error accumulation. Extensive experiments across eight benchmarks covering both natural and medical domains demonstrate that CM-GLasso establishes a new state-of-the-art in generative classification and dense semantic segmentation tasks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 4 linked inside Pith

  1. [1]

    Al Haris, M

    M. Al Haris, M. Dzeaulfath, and R. Wasono. Principal component analysis on convolutional neural network using transfer learning method for image classification of CIFAR-10 dataset. Register: Jurnal Ilmiah Teknologi Sistem Informasi, 10(2):141–150, 2024

    2024

  2. [2]

    Inferring sparse gaussian graph- ical models with latent structure.Electronic Journal of Statistics, 3:205–238, 2009

    Christophe Ambroise, Julien Chiquet, and Catherine Matias. Inferring sparse gaussian graph- ical models with latent structure.Electronic Journal of Statistics, 3:205–238, 2009

    2009

  3. [3]

    Distributed optimization and statistical learning via the alternating direction method of multipliers.Foun- dations and Trends in Machine Learning, 3(1):1–122, 2011

    Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers.Foun- dations and Trends in Machine Learning, 3(1):1–122, 2011

    2011

  4. [4]

    R. Dagli. Astroformer: More data might not be all you need for classification.arXiv preprint arXiv:2304.05350, 2023

    arXiv 2023

  5. [5]

    Patrick Danaher, Pei Wang, and Daniela M. Witten. The joint graphical lasso for inverse covariance estimation across multiple classes.Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(2):373–397, 2014

    2014

  6. [6]

    B. Dong, W. Wang, D.-P. Fan, J. Li, H. Fu, and L. Shao. Polyp-PVT: Polyp segmentation with pyramid vision transformers.CAAI Artificial Intelligence Research, 2:9150015, 2023

    2023

  7. [7]

    S. Duan, X. Yang, and N. Wang. Multi-label prototype visual spatial search for weakly supervised semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  8. [8]

    Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes challenge: A retrospective.International Journal of Computer Vision (IJCV), 111(1):98–136, 2015

    2015

Show all 49 references
  1. [9]

    G. J. Ferdous et al. SPT-Swin: A shifted patch tokenization swin transformer for image classification.IEEE Access, 2024

    2024

  2. [10]

    Extended bayesian information criteria for gaussian graphical models

    Rina Foygel and Mathias Drton. Extended bayesian information criteria for gaussian graphical models. InAdvances in Neural Information Processing Systems 23 (NeurIPS), 2010

    2010

  3. [11]

    Sparse inverse covariance estimation with the graphical lasso.Biostatistics, 9(3):432–441, 2008

    Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Sparse inverse covariance estimation with the graphical lasso.Biostatistics, 9(3):432–441, 2008. 17

    2008

  4. [12]

    Yuan Gao, Jiayi Ma, Mingbo Zhao, Wei Liu, and Alan L. Yuille. Nddr-CNN: Layerwise feature fusing in multi-task CNNs by neural discriminative dimensionality reduction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3205–3214, 2019

    2019

  5. [13]

    Caltech-256 object category dataset

    Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. Tech- nical Report 7694, California Institute of Technology, 2007

    2007

  6. [14]

    Learning a common substructure of multiple graphical gaussian models.Neural Networks, 38:23–38, 2013

    Satoshi Hara and Takashi Washio. Learning a common substructure of multiple graphical gaussian models.Neural Networks, 38:23–38, 2013

    2013

  7. [15]

    Hyun and H

    C. Hyun and H. Park. Multi-task learning with self-defined tasks for adversarial robustness of deep networks.IEEE Access, 12:83248–83259, 2024

    2024

  8. [16]

    Jahanaray, M

    S. Jahanaray, M. Ezoji, and Z. Imani. EEG-based feature space for supporting deep neural networks in image classification.International Journal of Engineering, 38(6):1264–1273, 2025

    2025

  9. [17]

    Oneformer: One transformer to rule universal image segmentation

    Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orber, and Humphrey Shi. Oneformer: One transformer to rule universal image segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  10. [18]

    Smedsrud, Michael A

    Debesh Jha, Pia H. Smedsrud, Michael A. Riegler, P˚ al Halvorsen, Thomas de Lange, Dag Johansen, and H˚ avard D. Johansen. Kvasir-SEG: A segmented polyp dataset. InProceedings of the International Conference on Multimedia Modeling (MMM), pages 451–462, 2020

    2020

  11. [19]

    Jia et al

    X. Jia et al. Polypmixnet: Enhancing semi-supervised polyp segmentation with polyp-aware augmentation.Computers in Biology and Medicine, 170:108006, 2024

    2024

  12. [20]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    2009

  13. [21]

    D. Lian, D. Zhou, J. Feng, and X. Wang. Scaling & shifting your features: A new baseline for efficient model tuning.Advances in Neural Information Processing Systems (NeurIPS), 35:109–123, 2022

    2022

  14. [22]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ ar, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In Proceedings of the European Conference on Computer Vision (ECCV), 2014

    2014

  15. [23]

    Y. Lin, M. Chen, W. Wang, et al. CLIP is also an efficient segmenter: A text-driven approach for weakly supervised semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15305–15314, 2023

    2023

  16. [24]

    Lingjærde, B

    Ole C. Lingjærde, B. Mangrud, M. Robrecht, G. F. Pettersen, and O. C. Lingjærde. Tailored graphical lasso for data integration in gene network reconstruction.BMC Bioinformatics, 22:498, 2021. 18

    2021

  17. [25]

    D. Liu, Y. Wang, K. Mase, et al. MT-ASM: A multi-task attention strengthening model for fine-grained object recognition.Multimedia Systems, 30:297, 2024

    2024

  18. [26]

    The nonparanormal: Semiparametric estima- tion of high dimensional undirected graphs.Journal of Machine Learning Research (JMLR), 10:2295–2328, 2009

    Han Liu, John Lafferty, and Larry Wasserman. The nonparanormal: Semiparametric estima- tion of high dimensional undirected graphs.Journal of Machine Learning Research (JMLR), 10:2295–2328, 2009

    2009

  19. [27]

    T. Y. Liu and S. Soatto. Tangent model composition for ensembling and continual fine-tuning. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  20. [28]

    Lu et al

    H. Lu et al. Learning with mixture of prototypes for out-of-distribution detection.arXiv preprint arXiv:2402.02653, 2024

    2024 arXiv

  21. [29]

    Shufflenet V2: Practical guide- lines for efficient CNN architecture design

    Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet V2: Practical guide- lines for efficient CNN architecture design. InProceedings of the European Conference on Computer Vision (ECCV), 2018

    2018

  22. [30]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InProceed- ings of...

    2021

  23. [31]

    Ren et al

    L. Ren et al. DA-VPT: Semantic-guided visual prompt tuning for vision transformers. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  24. [32]

    Rong et al

    S. Rong et al. Boundary-enhanced co-training for weakly supervised semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  25. [33]

    Shang et al

    C. Shang et al. PRO-VPT: Distribution-adaptive visual prompt tuning via prompt relocation. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025

    2025

  26. [34]

    M. M. H. Shawon.MedFoundX: A Foundation Model for Biomedical Image Classification and Segmentation. PhD thesis, BRAC University, 2025

    2025

  27. [35]

    Scene labeling using sparse precision matrix

    Nasim Souly and Mubarak Shah. Scene labeling using sparse precision matrix. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3650– 3658, 2016

    2016

  28. [36]

    Srivastava and G

    S. Srivastava and G. Sharma. Omnivec2: A novel transformer based network for large scale multimodal and multitask learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  29. [37]

    PaliGemma 2: A family of versatile VLMs for transfer.arXiv preprint arXiv:2412.03555, 2024

    Andreas Steiner et al. PaliGemma 2: A family of versatile VLMs for transfer.arXiv preprint arXiv:2412.03555, 2024. 19

    2024 arXiv

  30. [38]

    C. Tay, V. Subbaraju, and T. Kandappu. Privobfnet: A weakly supervised semantic segmen- tation model for data protection. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024

    2024

  31. [39]

    SigLIP 2: Multilin- gual vision-language encoders with improved semantic understanding, localization, and dense features.arXiv preprint arXiv:2502.14786, 2025

    Michael Tschannen, Alexey Gritsenko, Xiaohua Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nithya Parthasarathy, Thomas Evans, Lucas Beyer, Ying Xia, Basil Mustafa, Olivier H´ enaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. SigLIP 2: Multilin- gual vision-lan...

    2025 arXiv

  32. [40]

    The caltech-UCSD birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-UCSD birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Insti- tute of Technology, 2011

    2011

  33. [41]

    Internimage: Exploring large- scale vision foundation models with deformable convolutions

    Wenhai Wang, Jifeng Dai, Zhe Chen, Zhen Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, Xiaogang Wang, and Yu Qiao. Internimage: Exploring large- scale vision foundation models with deformable convolutions. InProceedings of the IEEE/CVF Conference on ...

    2023

  34. [42]

    Y. Wei, J. Ye, Z. Huang, et al. Online prototype learning for online continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 18764–18774, 2023

    2023

  35. [43]

    K. Wu, Y. Chen, H. Zhang, et al. Ja: Joint understanding and generation with decoupled visual encoding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  36. [44]

    Groupvit: Semantic segmentation emerges from text supervision

    Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiao- long Wang. Groupvit: Semantic segmentation emerges from text supervision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18134–18144, 2022

    2022

  37. [45]

    Xu et al

    L. Xu et al. Auxiliary tasks enhanced dual-affinity learning for weakly supervised semantic segmentation.IEEE Transactions on Neural Networks and Learning Systems, 36(3):5082–5096, 2024

    2024

  38. [46]

    Zeng et al

    R. Zeng et al. Visual fourier prompt tuning.Advances in Neural Information Processing Systems (NeurIPS), 37:5552–5585, 2024

    2024

  39. [47]

    Sigmoid loss for lan- guage image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for lan- guage image pre-training. InProceedings of the IEEE/CVF International Conference on Com- puter Vision (ICCV), 2023

    2023

  40. [48]

    C. Zhi, J. Zhuo, and S. Wang. Confusing pair correction based on category prototype for domain adaptation under noisy environments.AAAI Conference on Artificial Intelligence, 38(15), 2024. 20

    2024

  41. [49]

    Semantic understanding of scenes through the ADE20K dataset.International Journal of Computer Vision (IJCV), 127(3):302–321, 2019

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ADE20K dataset.International Journal of Computer Vision (IJCV), 127(3):302–321, 2019. 21

    2019

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.