Pith. sign in

REVIEW 4 major objections 5 minor 44 references

Evaluating Compositional Generalisation in VLMs and Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that vision-language models—including a diffusion-based classifier—bind attributes to objects well but fail at relational compositional generalisation.

desk verdict Useful benchmark extension, but the headline relational-GZSL result is undercut by a likely split confound and a table error; still worth refereeing. read the letter →

arxiv 2508.20783 v1 pith:XRQ3225L submitted 2025-08-28 cs.CV cs.AI

Beth Pearson , Bilal Boulbarss , Michael Wray , Martha Lewis This is my paper · ORCID
classification cs.CVcs.AI
keywords compositionalgeneralisationvision-languagemodelsdiffusionclassifierzero-shotlearninggeneralisedconceptbindingrelationalreasoningCLEVR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether generative models—specifically a Diffusion Classifier built on Stable Diffusion—compose familiar concepts into novel meanings better than discriminative vision-language models. To answer, it extends a synthetic concept-binding benchmark built from CLEVR-style images into zero-shot and generalised zero-shot versions for single objects, two objects, and spatial relations. The results show that fine-tuned ViLT and the Diffusion Classifier handle attribute–object binding well—ViLT exceeds 99% on two-object binding—but every model drops to near chance on the relational generalised zero-shot task, where distractors swap left and right. The paper reads this as evidence that current models rely on object recognition rather than structured relational composition, and it locates part of the cause in CLIP's text embeddings, where left/right concepts are not disentangled.

What carries the argument

The load-bearing object is an extension of the Concept Binding Benchmark, rendered with CLEVR's Blender script into three synthetic datasets—single-object, two-object, and relational—each split into training, in-distribution, and out-of-distribution sets with disjoint label spaces. The critical device is the hard-negative distractor: in the relational GZSL setting, every choice set contains the true label, a label with the relation swapped, and a label with object order swapped, so a model cannot succeed by recognising which two shapes are present. Three classifiers are compared: the generative Diffusion Classifier, which scores each candidate prompt by the noise-prediction error of a Stable

What would settle it

Run the relational GZSL protocol with an out-of-distribution test set of several thousand images balanced across all shape pairs and both relations, scoring accuracy only on trials whose hard negative is the relation swap (e.g., cube left sphere vs cube right sphere). If any model reaches well above 20% on that subset, the paper's conclusion that all tested VLMs fail relational composition would be refuted. Re-rendering the CLEVR scenes with randomised camera positions and lighting would also reveal whether the near-chance scores are caused by low-level image cues rather than relational semant

Watch

Extended reading notes

Core claim

The central claim is that the ability to bind attributes to objects does not transfer to relational composition. In the paper's extended benchmark, fine-tuned ViLT reaches over 99% accuracy on two-object zero-shot and generalised zero-shot binding, and the fine-tuned Diffusion Classifier reaches 99.5% on out-of-distribution single-object labels. But on the relational GZSL task—where the correct label must be chosen from options that include a hard negative with the same two objects and the opposite relation—all three models hover near the 20% chance level: fine-tuned ViLT scores 25–26% on out-of-distribution test labels, fine-tuned CLIP 34.75%, and fine-tuned Diffusion Classifier 38.25%. The

Load-bearing premise

The claim depends on the synthetic CLEVR images and prompt templates having no accidental cues that let models solve out-of-distribution labels without genuinely composing familiar colours, shapes, and relations; the paper asserts this design property but does not verify it empirically.

Editorial extensions

If this is right

  • Relational GZSL with swapped-order hard negatives is a sharper test of compositionality than attribute–object binding; high two-object accuracy does not imply any relational competence.
  • Because all three architectures fail, the generative-versus-discriminative choice is not what determines relational composition; the bottleneck lies in how relations are represented in text conditioning.
  • Positive-only fine-tuning, used to align with DreamBooth, is credited with CLIP's improved out-of-distribution generalisation relative to earlier benchmark results; negative examples may suppress labels that later appear as novel positives.
  • The failure pattern predicts that benchmarks without GZSL hard negatives overestimate relational understanding in vision-language models.
  • If relational reasoning is needed in deployment, pre-training objectives must change rather than expecting task-specific fine-tuning to teach it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test, not run in the paper, is to vary camera viewpoint, lighting, or object size in the CLEVR renders; if relational GZSL scores move with low-level image changes, part of the failure is perceptual rather than conceptual.
  • The text-embedding analysis suggests a concrete intervention: prompt templates that force left and right into distinct syntactic positions may shift embeddings enough to test whether the bottleneck is lexical overlap or relational semantics.
  • The same benchmark design could be applied to other relations, like above/below or front/behind, to see whether the near-chance relational GZSL result is specific to left/right or generalises to all spatial relations.
  • If the positive-only fine-tuning hypothesis is right, existing compositional fine-tuning methods that mine hard negatives may be hurting out-of-distribution generalisation, and a positive-only variant should be compared on the same splits.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper extends the Concept Binding Benchmark of Lewis et al. (2024) to zero-shot (ZSL) and generalized zero-shot (GZSL) settings, using synthetic CLEVR-style images with single-object, two-object, and relational (left/right) tasks. It evaluates frozen and fine-tuned versions of CLIP, ViLT, and the Diffusion Classifier (DC). The main claims are that ViLT and DC perform well on attribute-object binding, with ViLT best on two-object tasks and DC best on single-object OOD generalization, but that all models fail on relational GZSL, achieving near-chance accuracy on OOD splits. The paper also analyzes CLIP embeddings and Stable Diffusion generations to argue that relational concepts are not disentangled.

Significance. If the findings hold, they provide a useful diagnostic result: current VLMs and diffusion-based classifiers can bind attributes to objects but do not compositionally generalize spatial relations, particularly under GZSL where seen and unseen labels compete. The benchmark extension with explicitly defined train/ID/OOD splits and the public code/data are strengths. The paper also makes a concrete empirical contribution by comparing generative and discriminative classifiers under the same compositional tasks. However, the central claim about relational GZSL is weakened by a plausible shortcut based on shape-position correlations in the training split, by very small OOD validation sets with few prompts, and by the absence of statistical significance testing.

major comments (4)
  1. [§3, Relational dataset; Limitations] The relational GZSL conclusion ('all models struggle significantly') depends on the benchmark ruling out shortcuts. The only stated balancing condition is that 'the relational train split contains at least one of each shape.' If a shape appears predominantly on the left or right in training, a model can predict the relation from shape identity alone, and this prior would fail on OOD shape pairs, producing exactly the near-chance accuracies reported in Table 6. The Limitations assertion that synthetic images 'reduce the risk of spurious correlations' is not verified for this split. Please report per-shape left/right counts in each split, or rebalance the splits so shape identity and relation are independent, and rerun the experiments. This is load-bearing for the paper's central claim.
  2. [Table 6] The ViLT-FT row appears erroneous: its entries (16.55, 22.53, 26.50, 25.50) are the same as the Frozen ViLT row shifted one column right (Frozen ViLT: 13.94, 16.55, 22.53, 26.50). As ViLT-FT's near-chance relational GZSL performance is a key piece of evidence, this table must be corrected and checked against the raw results.
  3. [§4.2, §4.4, Table 6] Several load-bearing comparisons rest on very small OOD validation sets (250 relational images; 'only 4 very distinct prompts' for two-object OOD val) and on means with large standard deviations (e.g., CLIP-FT relational GZSL OOD Val 42.80±18.39 in Table 6). The paper reports no significance tests against chance (20% in GZSL) or between models. Please provide confidence intervals or significance tests, and/or increase the OOD validation size, so that claims such as 'ViLT-FT has the strongest performance' and 'all models are near chance' are supported beyond raw means.
  4. [§3, §4, Appendix A] The cross-model comparisons are confounded by different fine-tuning protocols: CLIP uses contrastive loss with 20-40 images/class, ViLT uses LoRA with different ranks, and DC uses DreamBooth with 30-40 images/class and large step counts. Differences attributed to model architecture (discriminative vs generative) could instead reflect the fine-tuning method, data quantity, or optimization. Please add a control (e.g., use the same LoRA or contrastive setup where possible) or explicitly frame the results as tied to the specific fine-tuning pipelines rather than to model class.
minor comments (5)
  1. [§4.5] Typo: 'which may be not have been required' should read 'which may not have been required'.
  2. [Tables 2–6] The tables print mean and standard deviation without a visible separator (e.g., '13.940.43'). Please use '±' or parentheses consistently to avoid misreading.
  3. [§5, Relational prompts] The prompt example 'a cylinder to the left of a cone' in §5 differs from the formal label notation (cylinder, left, cone). Please state the exact prompt template used for all relational splits, including how OOD prompts are constructed.
  4. [Figure 6] The t-SNE analysis is qualitative. Consider reporting a quantitative metric (e.g., nearest-neighbor accuracy or centroid distances) to support the claim that left/right text embeddings are overly similar.
  5. [§4.2] The statement that OOD validation has 'only 4 very distinct prompts' should be made specific: list the prompts and the number of images per prompt for each dataset.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reported accuracies are measurements, not derived from fitted parameters; the self-citation to Lewis et al. is not load-bearing.

full rationale

This paper is an empirical benchmark evaluation, not a derivation. The central claims (e.g., Section 4.5, Tables 2-6) are measured accuracies of frozen/fine-tuned CLIP, ViLT, and Diffusion Classifier on a rendered CLEVR-style dataset with explicitly defined train/ID/OOD splits (Section 3). No parameter is fit to the reported accuracies and then renamed a prediction; the OOD GZSL scores are out-of-sample measurements. The benchmark is an extension of Lewis et al. (2024), and that citation shares an author (Lewis), but the extension is defined in this paper and the original benchmark is publicly available; the self-citation is not load-bearing for the conclusion. The only passage that could resemble a circularity is the Section 2/Limitations assertion that 'we use simple, synthetic images to ensure no spurious correlations.' That is an unverified validity assumption, not a derivation; the skeptic's concern about per-shape left/right imbalance producing a nontransferable position prior is a potential confound or correctness risk, not a circularity. Accordingly, no circular step is identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The evaluation depends on benchmark-design assumptions rather than fitted parameters; no mathematical model is derived. The key assumptions concern the diagnostic validity of the synthetic scenes and the fairness of the comparison protocol.

assumptions (4)
  • domain assumption Synthetic CLEVR-style images with simple shapes and colors are a valid diagnostic for compositional generalization in VLMs.
    Stated in Section 3 and Limitations; the paper relies on this to generalize from the benchmark to broader VLM understanding.
  • domain assumption The OOD splits test novel combinations of seen components, so accuracy on them isolates compositional recombination.
    Split definitions in Section 3 guarantee no label overlap, but the assumption that this isolates composition from low-level cues is asserted, not verified.
  • domain assumption Positive-only fine-tuning of CLIP (to match DreamBooth) is a fair comparison across model types.
    Section 3 states this; differing fine-tuning protocols (contrastive, LoRA, DreamBooth) may confound relative performance.
  • domain assumption Relational hard negatives require relational reasoning and cannot be solved by bag-of-words shortcuts.
    Section 3.3 asserts this; the t-SNE analysis provides qualitative support, but no quantitative proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Compositional Generalisation in VLMs and Diffusion Models." pith.science (2026). https://pith.science/paper/XRQ3225L

@misc{pith2026250820783,
  author       = {Pith},
  title        = {Pith review of: Evaluating Compositional Generalisation in VLMs and Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XRQ3225L}},
  note         = {Machine review of arXiv:2508.20783}
}
read the original abstract

A fundamental aspect of the semantics of natural language is that novel meanings can be formed from the composition of previously known parts. Vision-language models (VLMs) have made significant progress in recent years, however, there is evidence that they are unable to perform this kind of composition. For example, given an image of a red cube and a blue cylinder, a VLM such as CLIP is likely to incorrectly label the image as a red cylinder or a blue cube, indicating it represents the image as a `bag-of-words' and fails to capture compositional semantics. Diffusion models have recently gained significant attention for their impressive generative abilities, and zero-shot classifiers based on diffusion models have been shown to perform competitively with CLIP in certain compositional tasks. In this work we explore whether the generative Diffusion Classifier has improved compositional generalisation abilities compared to discriminative models. We assess three models -- Diffusion Classifier, CLIP, and ViLT -- on their ability to bind objects with attributes and relations in both zero-shot learning (ZSL) and generalised zero-shot learning (GZSL) settings. Our results show that the Diffusion Classifier and ViLT perform well at concept binding tasks, but that all models struggle significantly with the relational GZSL task, underscoring the broader challenges VLMs face with relational reasoning. Analysis of CLIP embeddings suggests that the difficulty may stem from overly similar representations of relational concepts such as left and right. Code and dataset are available at: https://github.com/otmive/diffusion_classifier_clip

Figures

Figures reproduced from arXiv: 2508.20783 by the authors.

Figure 1
Figure 1. We evaluate the compositional generalisa [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Single and Two-Object dataset design. Class [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Relational dataset design. Class labels be [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Samples from our extended benchmark with two example classes displayed from each dataset—single, [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Images generated by Frozen and Fine-Tuned Diffusion-Classifier using prompts from the single, two [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: t-SNE visualisations of CLIP text and image embeddings for relational prompts after fine-tuning. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 31 canonical work pages

  1. [1]

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258

    arXiv 2021

  2. [2]

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. 2023. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18392--18402

    work page 2023

  3. [3]

    Paola Cascante-Bonilla, Khaled Shehada, James Seale Smith, Sivan Doveh, Donghyun Kim, Rameswar Panda, Gul Varol, Aude Oliva, Vicente Ordonez, Rogerio Feris, et al. 2023. Going beyond nouns with vision & language models using synthetic data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20155--20165

    2023

  4. [4]

    Yang, Chen-Dong Duan, Hang Su, and Jun Zhu

    Huanran Chen, Yinpeng Dong, Zhengyi Wang, X. Yang, Chen-Dong Duan, Hang Su, and Jun Zhu. 2023. https://api.semanticscholar.org/CorpusID:258865189 Robust classification via a single diffusion model . ArXiv, abs/2305.15241

    arXiv 2023

  5. [5]

    Noam Chomsky. 1957. Syntactic Structures. Mouton and Co., The Hague

    work page 1957

  6. [6]

    Kevin Clark and Priyank Jaini. 2023. Text-to-image diffusion models are zero shot classifiers. Advances in Neural Information Processing Systems, 36:58921--58937

    work page 2023

  7. [7]

    Blender Online Community. 2018. http://www.blender.org Blender - a 3D modelling and rendering package . Blender Foundation, Stichting Blender Foundation, Amsterdam

    work page 2018

  8. [8]

    Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780--8794

    2021

Show all 44 references
  1. [9]

    Anuj Diwan, Layne Berry, Eunsol Choi, David Harwath, and Kyle Mahowald. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.143 Why is winoground hard? investigating failures in visuolinguistic compositionality . In Proceedings of the 2022 Conference on Empirical Methods in Natu...

    2022 doi

  2. [10]

    Sivan Doveh, Assaf Arbelle, Sivan Harary, Eli Schwartz, Roei Herzig, Raja Giryes, Rogerio Feris, Rameswar Panda, Shimon Ullman, and Leonid Karlinsky. 2023. Teaching structured vision & language concepts to vision & language models. In Proceedings of the IEEE/CVF Conference on ...

    2023

  3. [11]

    Sri Harsha Dumpala, Aman Jaiswal, Chandramouli Sastry, Evangelos Milios, Sageev Oore, and Hassan Sajjad. 2025. Sugarcrepe++ dataset: vision-language model sensitivity to semantic and lexical alterations. In Proceedings of the 38th International Conference on Neural Information...

    2025

  4. [12]

    Xuehai He, Weixi Feng, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Pradyumna Narayana, S Basu, William Yang Wang, and Xin Eric Wang. 2024. https://openreview.net/forum?id=GtnipgAomT Discffusion: Discriminative diffusion models as few-shot vision and language learners . Trans...

    2024

  5. [13]

    Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna. 2024. Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality. Advances in Neural Information Processing Systems, 36

    2024

  6. [14]

    Hang Hua, Jing Shi, Kushal Kafle, Simon Jenni, Daoan Zhang, John Collomosse, Scott Cohen, and Jiebo Luo. 2024 a . Finematch: Aspect-based fine-grained image and text mismatch detection and correction. In European Conference on Computer Vision, pages 474--491. Springer

    2024

  7. [15]

    Hang Hua, Yunlong Tang, Ziyun Zeng, Liangliang Cao, Zhengyuan Yang, Hangfeng He, Chenliang Xu, and Jiebo Luo. 2024 b . Mmcomposition: Revisiting the compositionality of pre-trained vision-language models. arXiv preprint arXiv:2410.09733

    2024 arXiv

  8. [16]

    Irene Huang, Wei Lin, Muhammad Jehanzeb Mirza, Jacob Hansen, Sivan Doveh, Victor Butoi, Roei Herzig, Assaf Arbelle, Hilde Kuehne, Trevor Darrell, Chuang Gan, Aude Oliva, Rogerio Feris, and Leonid Karlinsky. 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/28aad...

    2024

  9. [17]

    Theo MV Janssen and Barbara H Partee. 1997. Compositionality. In Handbook of logic and language, pages 417--473. Elsevier

    1997

  10. [18]

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. 2017. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern reco...

    2017

  11. [19]

    Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021. Vilt: Vision-and-language transformer without convolution or region supervision. In International conference on machine learning, pages 5583--5594. PMLR

    2021

  12. [20]

    Benno Krojer, Elinor Poole-Dayan, Vikram Voleti, Christopher Pal, and Siva Reddy. 2023. Are diffusion models vision-and-language reasoners? In NeurIPS

    2023

  13. [21]

    Martha Lewis, Nihal Nayak, Peilin Yu, Jack Merullo, Qinan Yu, Stephen Bach, and Ellie Pavlick. 2024. https://aclanthology.org/2024.findings-eacl.101 Does CLIP bind concepts? probing compositionality in large image models . In Findings EACL 2024, pages 1487--1500, St. Julian ' ...

    2024

  14. [22]

    Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. 2023. Your diffusion model is secretly a zero-shot classifier. In ICCV

    2023

  15. [23]

    Zixian Ma, Jerry Hong, Mustafa Omer Gul, Mona Gandhi, Irena Gao, and Ranjay Krishna. 2023. Crepe: Can vision-language foundation models reason compositionally? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10910--10921

    2023

  16. [24]

    Rohan Pandey, Rulin Shao, Paul Pu Liang, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2023. https://doi.org/10.18653/v1/2023.acl-long.298 Cross-modal attention congruence regularization for vision-language relation alignment . In Proceedings of the 61st Annual Meeting of ...

    2023 doi

  17. [25]

    Farhad Pourpanah, Moloud Abdar, Yuxuan Luo, Xinlei Zhou, Ran Wang, Chee Peng Lim, Xi-Zhao Wang, and QM Jonathan Wu. 2022. A review of generalized zero-shot learning methods. IEEE transactions on pattern analysis and machine intelligence, 45(4):4051--4070

    2022

  18. [26]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  19. [27]

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3

    2022 arXiv

  20. [28]

    Arijit Ray, Filip Radenovic, Abhimanyu Dubey, Bryan Plummer, Ranjay Krishna, and Kate Saenko. 2024. Cola: A benchmark for compositional text-to-image retrieval. Advances in Neural Information Processing Systems, 36

    2024

  21. [29]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695

    2022

  22. [30]

    Ugur Sahin, Hang Li, Qadeer Khan, Daniel Cremers, and Volker Tresp. 2024. Enhancing multimodal compositional reasoning of visual language models with generative negative mining. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5563--5573

    2024

  23. [31]

    Ziyi Shou and Fangzhen Lin. 2024. Enhancing semantic understanding in vision language models using meaning representation negative generation. In Fourth Workshop on Knowledge-infused Learning

    2024

  24. [32]

    Harman Singh, Pengchuan Zhang, Qifan Wang, Mengjiao Wang, Wenhan Xiong, Jingfei Du, and Yu Chen. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.56 Coarse-to-fine contrastive learning in image-text-graph space for improved vision-language compositionality . In Proceedings of...

    2023 doi

  25. [33]

    Sania Sinha, Tanawan Premsri, and Parisa Kordjamshidi. 2024. https://openreview.net/forum?id=BXDxwItNqQ A survey on compositional learning of AI models: Theoretical and experimental practices . Transactions on Machine Learning Research. Survey Certification

    2024

  26. [34]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063

    2024

  27. [35]

    Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. 2022. Winoground: Probing vision and language models for visio-linguistic compositionality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2022

  28. [36]

    Ankit Vani, Bac Nguyen, Samuel Lavoie, Ranjay Krishna, and Aaron Courville. 2024. Sparo: Selective attention for robust and compositional transformer encodings for vision. In European Conference on Computer Vision, pages 233--251. Springer

    2024

  29. [37]

    I Can't Believe It's Not Better: Failure Modes in the Age of Foundation Models

    Chenwei Wu, Li Erran Li, Stefano Ermon, Patrick Haffner, Rong Ge, and Zaiwei Zhang. 2023. https://proceedings.mlr.press/v239/wu23a.html The role of linguistic priors in measuring compositional generalization of vision-language models . In Proceedings on "I Can't Believe It's N...

    2023

  30. [38]

    Yongqin Xian, Bernt Schiele, and Zeynep Akata. 2017. Zero-shot learning-the good, the bad and the ugly. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4582--4591

    2017

  31. [39]

    Nir Yellinek, Leonid Karlinsky, and Raja Giryes. 2025. https://doi.org/10.1109/TIP.2024.3523801 3vl: Using trees to improve vision-language models’ interpretability . Trans. Img. Proc., 34:495–509

    2025

  32. [40]

    Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. 2022. When and why vision-language models behave like bags-of-words, and what to do about it? In The Eleventh International Conference on Learning Representations

    2022

  33. [41]

    Le Zhang, Rabiul Awal, and Aishwarya Agrawal. 2024. Contrasting intra-modal and ranking cross-modal hard negatives to enhance visio-linguistic compositional understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13774--13784

    2024

  34. [42]

    Tiancheng Zhao, Tianqi Zhang, Mingwei Zhu, Haozhan Shen, Kyusong Lee, Xiaopeng Lu, and Jianwei Yin. 2022. An explainable toolbox for evaluating pre-trained vision-language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System...

    2022

  35. [43]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.