Contrastive vision-language learning with paraphrasing and negation

Artur d'Avila Garcez; Joe Townsend; Kwun Ho Ngan; Saman Sadeghi Afgeh

REVIEW 4 major objections 6 minor 14 references

A projection-based extension of CLIP jointly learns paraphrasing and negation, raising original-over-negation accuracy from 68.1% to 78.1% while preserving retrieval.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 21:05 UTC pith:R6A27DVJ

load-bearing objection SemCLIP is a genuine first attempt at joint paraphrase+negation training for CLIP with a solid held-out CC-Neg result, but the Sugarcrepe++ evidence is compromised by an unstated train/test split and the paraphrase side of the claim does not hold up. the 4 major comments →

arxiv 2511.16527 v2 pith:R6A27DVJ submitted 2025-11-20 cs.CV cs.LG

Contrastive vision-language learning with paraphrasing and negation

Kwun Ho Ngan , Saman Sadeghi Afgeh , Joe Townsend , Artur d'Avila Garcez This is my paper

classification cs.CV cs.LG MSC 68T0768T45

keywords CLIPcontrastive learningnegationparaphrasingembedding projectionvision-languagezero-shot classificationsemantic robustness

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to show that CLIP-style vision-language models can learn both paraphrasing and negation in a single training objective — something prior work treated separately. SemCLIP adds two projection-based losses: one pulls the embeddings of paraphrased captions in the same direction as the original caption, and the other pushes negated captions away. On CC-Neg, this raises the original-over-negation accuracy from 68.1% to 78.1% while holding retrieval accuracy on original captions at 33.1%. On five zero-shot classification datasets, the SemCLIP model finetuned on Sugarcrepe++ shows the largest accuracy gap between positive and negated prompts. The underlying claim is that semantic equivalence and contradiction reduce to angular constraints in a low-dimensional projection of the embedding space.

Core claim

On its own terms, the central discovery is that a small projection subspace — one or two random orthonormal directions — is enough to teach a CLIP text encoder the difference between paraphrase and negation. The paraphrasing loss minimizes 1 - cosine similarity between the projections of the original and the paraphrased caption; the negation loss maximizes the distance by penalizing positive cosine similarity between original and negated projections. Combined with CLIP's original contrastive loss in a weighted sum, these terms produce a model that matches CLIP's top-1 accuracy on original and paraphrased captions (33.1% and 21.0% on CC-Neg) while raising original-over-negation accuracy from

What carries the argument

The central mechanism is a low-dimensional projection subspace: a small number n (1 or 2) of random orthonormal vectors onto which all text embeddings are projected, with two new losses computed on those projections. The paraphrasing loss L_paraphrase = 1 - cos(p(t), p(t+)) pulls the paraphrase's projection toward the original's; the negation loss L_negation = max(0, cos(p(t), p(t-))) pushes the negation's projection to be orthogonal or opposite. These are combined with CLIP's standard contrastive loss in a weighted sum (Eq. 7), so the semantic relations of equivalence and contradiction become geometric directions in the subspace.

Load-bearing premise

The reported Sugarcrepe++ retrieval numbers assume the model is evaluated on caption pairs it did not see during finetuning, but the paper does not define a held-out split for those 4,757 pairs.

What would settle it

Run the SemCLIP finetuning on Sugarcrepe++ with an explicit train/eval split of the 4,757 pairs; if Table 1's top-1 accuracy on original and paraphrased captions drops toward the CLIP baseline, the gain comes from memorizing the training pairs rather than from the loss.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If the result holds, negation robustness can be added to existing CLIP models by finetuning only the text encoder with LLM-generated triplets.
Retrieval quality on normal captions is preserved, so the extra semantic constraints come at little cost to standard benchmarks.
Zero-shot classifiers built on SemCLIP will correctly reject negated prompts, which is useful for safe and truthful image search.
The same angular-loss recipe is a template for other semantic relations: any pair of text transformations could be pushed together or apart in projection space.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

One extension the paper leaves implicit: the same projection loss could define entailment — if A entails B, A's projection should sit in a monotone relation to B's — turning natural-language inference into a geometric constraint testable on standard inference datasets.
The reported Sugarcrepe++ retrieval numbers should be re-verified on a strictly held-out split before being taken as generalization evidence; the paper does not state that such a split exists.
Because the projection has as few as one dimension, the learned direction can be inspected directly: a testable prediction is that the projection coordinate correlates with human ratings of paraphrase-versus-negation strength.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

SemCLIP is a genuine first attempt at joint paraphrase+negation training for CLIP with a solid held-out CC-Neg result, but the Sugarcrepe++ evidence is compromised by an unstated train/test split and the paraphrase side of the claim does not hold up.

read the letter

Here's the short version: SemCLIP is the first CLIP variant I know of that trains with both paraphrase and negation triplets in one loss, and its held-out CC-Neg result is genuine. But the paraphrase half of the claim is weaker than the paper's abstract suggests, and the Sugarcrepe++ numbers in Table 1 need a train/test split before they can be read as generalization.

What's actually new: the projection-based paraphrase and negation losses (Eq. 5-6) layered onto CLIP's contrastive loss, with the geometric interpretation that paraphrases land near the original and negations are pushed away in a low-dimensional subspace. That's a modest but real extension of NegCLIP/CoN-CLIP and ParaCLIP. The held-out CC-Neg evaluation (22,825 examples) is the strongest evidence: original-over-negation accuracy jumps from 68.1% to 78.1%, and standard retrieval on originals stays flat at 33.1%. The downstream zero-shot results on five classification datasets are a nice plus, especially when finetuned on Sugarcrepe++.

Now the soft spots. The SCPP column is the big one. The paper never states a train/test split for the 4,757 SCPP pairs; Figure 2 labels the model as 'SCPP finetuning', and Table 1 reports retrieval on SCPP. If those pairs are the same ones used for finetuning, the SCPP numbers are in-sample and cannot support a generalization claim. The reader's concern here is right, and it matters because the SCPP results are already mixed: SemCLIP's paraphrased-caption accuracy is 53.1 vs 60.0 for CLIP and 59.1 for the paraphrase-only model, and original-caption accuracy drops from 64.2 to 57.3. So the 'preserving performance' claim only holds on CC-Neg, not SCPP. The stress-test note is accurate: the paraphrase robustness claim in the introduction is not supported by the data; the joint model sacrifices paraphrase performance to get negation robustness. The paper does say 'results are mixed' in the abstract, but the intro and conclusion go further than that.

Also minor: Table 1 reports single scalars from 25 runs with no error bars or selection rule. The ablation in Appendix C gives some ranges, but the main table hides the variance. And the comparison to CoN-CLIP's 99.7% on CC-Neg is striking; the paper credits CoN-CLIP but doesn't test it on SCPP. That's a missing baseline for the joint task, though understandable given CoN-CLIP's code/data may not be set up for it.

Overall, the central idea holds up better than the SCPP evidence. With a clean train/test split, error bars, and language that matches the standard-accuracy numbers, this would be a useful contribution. I'd send it to a serious referee rather than desk reject it; the issues are addressable and the CC-Neg result is worth taking seriously.

Referee Report

4 major / 6 minor

Summary. SemCLIP extends CLIP with two additional loss terms: a paraphrase loss that pulls the projections of original and paraphrased captions together, and a negation loss that pushes the projections of original and negated captions apart. Captions are generated by a two-stage LLM pipeline (Phi-4 for generation, Mistral-7B for validation). The method is evaluated on CC-Neg and Sugarcrepe++ (SCPP) for image-caption retrieval and on five downstream zero-shot classification datasets. The headline result is an improvement in original-over-negation accuracy on CC-Neg from 68.1% (CLIP baseline) to 78.1% (SemCLIP), with original-caption retrieval accuracy essentially unchanged (33.1% vs 33.1%). On SCPP, results are mixed, with lower retrieval accuracies than the CLIP baseline (original caption 57.3 vs 64.2; paraphrased 53.1 vs 60.0) but comparable original-over-negation accuracy (82.8 vs 82.7).

Significance. The joint treatment of paraphrasing and negation in a CLIP-style objective is novel, and the CC-Neg original-over-negation improvement is substantial and reported on a held-out test split. The method is simple, modular, and the paper includes a systematic ablation over the projection hyperparameters. If the results are robust, SemCLIP offers a practical way to improve negation robustness at modest engineering cost. The significance is tempered, however, by the absence of a stated train/test split for SCPP, the lack of error bars or a selection rule for the main table, and the apparent degradation in paraphrase retrieval accuracy, which undercuts the broad claim of joint robustness.

major comments (4)

[Section 4, Table 1] The SCPP evaluation split is not stated. The paper describes Sugarcrepe++ as a dataset with a total of 4,757 image-caption pairs and Figure 2 labels models as 'SCPP finetuning'. If the same 4,757 pairs are used for both finetuning and the Table 1 retrieval evaluation, those numbers reflect in-sample fit rather than generalization. Please specify the exact train/test split used for SCPP and report held-out evaluation results.
[Table 1 and Section 4] Table 1 reports single scalars for each model and metric, but Section 4 states the results are based on 25 individual training runs that systematically sweep hyperparameters. No selection rule (best, average, median) or variance is reported. Without confidence intervals or a description of how the reported values were selected, the SCPP comparisons are uninterpretable, especially where differences are small (e.g., original-over-negation 82.8 vs 82.7).
[Abstract, Section 1, Table 1] The claim of 'robustness to paraphrased queries' and 'preserving roughly the same performance' is not supported by the data. On SCPP, SemCLIP's paraphrased-caption accuracy is 53.1% versus 60.0% for CLIP and 59.1% for Paraphrase-only. On CC-Neg, paraphrased accuracy is 21.0% versus 21.9% for CLIP. The paraphrase loss appears to degrade paraphrase retrieval performance, particularly on the larger and more carefully curated SCPP benchmark. Please either temper the claim or provide evidence that the paraphrase loss alone or in combination preserves paraphrase invariance.
[Section 3.3, Eqs. (5)-(7)] The proposed losses operate on low-dimensional projections (n=1 or 2) of the text embeddings, while the evaluation metrics—Top-1 retrieval and original-over-negation—use full-embedding cosine similarity. The paper does not explain why optimizing the projection losses should change the full-embedding geometry sufficiently to produce the observed CC-Neg original-over-negation improvement. With fixed projection vectors, gradients are confined to a low-rank subspace; with learnable V, the objective can be minimized by changing V alone. Please provide a theoretical or empirical link between the projected losses and the full-embedding retrieval behavior, or report diagnostics such as full-embedding distance changes before/after training.

minor comments (6)

[Eq. (7)] The denominator 'α+β+γ' appears on the same line as the numerator with no parentheses; it should be written as (αL_contrastive + βL_paraphrase + γL_negation) / (α+β+γ).
[Appendix C] Appendix C reports a peak original-over-negation accuracy of 89.1%, while Table 1 reports 78.1% for SemCLIP on CC-Neg. Please clarify which hyperparameter configuration is used for Table 1 and reconcile the discrepancy.
[Figure 2 and text] The text refers to 'SPCC' in a few places (e.g., 'finetuned on SPCC'); this should be 'SCPP'.
[Section 4] The phrase 'All four models recover the correct image for approximately 33% of the 22,825 held-out examples' should clarify whether these are the CC-Neg test set examples and how they relate to the 228,246 total pairs.
[General] The capitalization of 'Sugarcrepe++' is inconsistent across the manuscript; the dataset's official capitalization should be used throughout.
[Appendix A] The prompt templates are clear, but no mention is made of releasing the generated paraphrase/negation data or the code. For reproducibility, consider making these available.

Circularity Check

0 steps flagged

No significant circularity: SemCLIP's losses are explicit training priors and all headline results are measured on external held-out benchmarks.

full rationale

SemCLIP adds two loss terms (Eqs. 5-6) to CLIP's contrastive loss (Eq. 7). These terms encode a chosen inductive bias: paraphrase projections should have high cosine similarity and negation projections should have non-positive cosine similarity. The evaluation metrics are distinct: Top-1 image retrieval on original/paraphrased captions and original-over-negation accuracy are computed in the full embedding space, not in the low-dimensional projection, so improving Eqs. 5-6 does not by construction force the reported test numbers. The CC-Neg table reports held-out examples (22,825 test examples), and the downstream zero-shot tasks use prompts like 'This is a photo of a <class>' that are independent of the LLM-generated training triplets. The paper's own admission of mixed Sugarcrepe++ paraphrased-caption results (60.0 to 53.1) is a performance inconsistency, not a circularity. The only self-citation (d'Avila Garcez & Lamb 2020) is motivational and carries no load-bearing claim; no uniqueness theorem or imported ansatz is invoked. The reader's SCPP train/test overlap concern is a leakage/generalization issue, not an instance of a result reducing to its inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new physical entities are introduced. The projection subspace is an architectural component, not an external entity. The main burden is carried by learned models and LLM-generated labels rather than first-principles derivation.

free parameters (4)

loss weights alpha, beta, gamma = alpha=beta=gamma=1
Chosen from {0,1} for simplicity (Section 3.3, Eq. 7); no tuning or principled selection is reported.
projection dimension n = 1 or 2
Chosen by hand; ablation shows 1D vs 2D changes results, so the choice affects the outcome.
projection normalization = True in reported experiments
Boolean hyperparameter swept in ablations; paper says normalization was used in all reported experiments.
learnable projection vectors V = False (learnable variant tested, no improvement)
When enabled, V would be fit to training data; final vectors are not released either way.

axioms (5)

domain assumption LLM-generated paraphrases and negations are correct semantic labels.
Section 3.1 and Appendix A rely on Phi-4 generation and Mistral-7B Yes/No validation as ground truth.
domain assumption Random orthonormal low-dimensional projections preserve enough semantic signal for the loss to transfer to full CLIP embeddings.
Section 3.2 assumes projection directions impose meaningful geometric constraints, with no theoretical guarantee.
domain assumption Cosine similarity in CLIP embedding space is a valid proxy for image-text matching.
Standard CLIP assumption used in L_contrastive and in the evaluation metrics.
domain assumption Frozen ViT-B/32 initialized from LAION-2B is an appropriate base model.
Section 4 uses OpenCLIP ViT-B-32 pretrained on LAION-2B with frozen vision encoder.
domain assumption CC-Neg and Sugarcrepe++ are valid benchmarks with correctly labeled negatives.
Section 4 adopts both datasets as proxies for semantic understanding without re-auditing their labels.

pith-pipeline@v1.3.0-alltime-deepseek · 13089 in / 11462 out tokens · 99549 ms · 2026-08-03T21:05:36.852508+00:00 · methodology

0 comments

read the original abstract

Contrastive vision-language models continue to be the dominant approach for image-text retrieval. Contrastive Language-Image Pre-training (CLIP) trains two neural networks to align their image and text embeddings in a shared latent space. As a challenging case-study for neurosymbolic AI, recent results evaluating CLIP on negated or paraphrased text have shown mixed performance as these are difficult to define formally for text data. Negation produces the opposite meaning using various possible but small lexical changes. Paraphrasing may use very different textual expressions to denote essentially the same thing. As a result, learning of paraphrasing and negation together poses a significant challenge because of the above mismatch between changes in syntax and intended meaning expected to be captured by distances in embedding space. This paper proposes a new CLIP contrastive loss function capable of balancing the requirements of having both paraphrasing and negation. It applies training triplets consisting of original, paraphrased and negated text generated by multiple large language models to the evaluation of CLIP models. The approach, called SemCLIP, aims to learn semantically-relevant and simple embeddings, placing paraphrased captions nearer to the original image embeddings while at the same time pushing negated captions farther away. Empirically, SemCLIP is shown to be capable of preserving roughly the same performance as CLIP augmented with either negation or paraphrasing. Although direct comparisons are difficult to make because the problem of learning with both negation and paraphrasing is different, an expected benefit of SemCLIP should be robustness when applied zero-shot to downstream image classification tasks. Our experiments confirm such robustness as measured by difference in accuracy (mean-accuracy delta) between original and negated captions on five downstream datasets.

Figures

Figures reproduced from arXiv: 2511.16527 by Artur d'Avila Garcez, Joe Townsend, Kwun Ho Ngan, Saman Sadeghi Afgeh.

**Figure 2.** Figure 2: Model robustness to negation is measured by the difference in accuracy (Mean Accuracy Delta) between [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Figure 3: Effect of setting the number of projection vectors on image matching accuracies using trained model (finetuned [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

**Figure 4.** Figure 4: Effect of setting the use of learnable projection vectors on image matching accuracies using trained model [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗

**Figure 5.** Figure 5: Effect of setting the projection vectors normalization on image matching accuracies using trained model [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗

**Figure 6.** Figure 6: Effect of setting the number of projection vectors on image matching accuracies using trained model (finetuned [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗

**Figure 7.** Figure 7: Effect of setting the use of learnable projection vectors on image matching accuracies using trained model [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗

**Figure 8.** Figure 8: Effect of setting the projection vectors normalization on image matching accuracies using trained model [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗

**Figure 9.** Figure 9: Mean accuracy on downstream classification tasks by models with different training loss terms (finetuned [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗

**Figure 10.** Figure 10: Mean accuracy on downstream classification tasks by models with different training loss terms (finetuned [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

14 extracted references · 10 linked inside Pith

[3]

URLhttp://arxiv.org/abs/2306.14610. Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July

Pith/arXiv arXiv
[7]

Steven J Leon, Åke Björck, and Walter Gander

URL http: //arxiv.org/abs/2405.00740. Steven J Leon, Åke Björck, and Walter Gander. Gram-schmidt orthogonalization: 100 years and more.Numerical Linear Algebra with Applications, 20(3):492–532,

Pith/arXiv arXiv
[9]

Aarohi Srivastava, Abhinav Rastogi, and Abhishek Rao et al

URLhttp://arxiv.org/abs/2403.20312. Aarohi Srivastava, Abhinav Rastogi, and Abhishek Rao et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,

Pith/arXiv arXiv
[10]

Norbert Vanek and Haoruo Zhang

URLhttps://arxiv.org/abs/2206.04615. Norbert Vanek and Haoruo Zhang. On truth and polarity in negation processing: language-specific effects in non- linguistic contexts.Frontiers in Psychology, V olume 14 - 2023,

Pith/arXiv arXiv 2023
[11]

doi: 10.3389/fpsyg.2023

ISSN 1664-1078. doi: 10.3389/fpsyg.2023. 1244249. URL https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023. 1244249. Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it?,

doi:10.3389/fpsyg.2023 2023
[12]

Kecheng Zheng, Yifei Zhang, Wei Wu, Fan Lu, Shuailei Ma, Xin Jin, Wei Chen, and Yujun Shen

URLhttp://arxiv.org/abs/2210.01936. Kecheng Zheng, Yifei Zhang, Wei Wu, Fan Lu, Shuailei Ma, Xin Jin, Wei Chen, and Yujun Shen. DreamLIP: Language-image pre-training with long captions,

Pith/arXiv arXiv
[13]

{original}

URLhttp://arxiv.org/abs/2403.17007. 11 A Prompt Templates for Caption Generation The following templates provide an example of paraphrased and negated caption candidate generation by either Phi-4 or Mistral-7B model as well as the subsequent quality validation of generation by Mistral-7B. Caption Generation Template 1Paraphrased caption candidate 2(Phi-4/...

Pith/arXiv arXiv 2024
[14]

(b) Effect of setting the number of projection vectors on Top-1 accuracy using paraphrased caption for image matching

Downstream Models Tasks SemCLIP SemCLIP CoN-CLIP (CC-Neg) (SCPP) CIFAR-10 90.691.690.45 CIFAR-100 62.566.562.31 Food-101 50.7 62.783.39 Flowers102 36.0 37.564.74 Oxford-IIIT Pet 53.5 69.581.66 14 (a) Effect of setting the number of projection vectors on Top-1 accuracy using original caption for image matching. (b) Effect of setting the number of projectio...

2024
[2009]

Technical Report, CIFAR-10 and CIFAR-100 datasets

URL https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf . Technical Report, CIFAR-10 and CIFAR-100 datasets. Samuel Lavoie, Polina Kirichenko, Mark Ibrahim, Mahmoud Assran, Andrew Gordon Wilson, Aaron Courville, and Nicolas Ballas. Modeling caption diversity in contrastive vision-language pretraining,

2009
[2018]

Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna

URLhttps://arxiv.org/abs/1812.02230. Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna. SugarCrepe: Fixing hackable benchmarks for vision-language compositionality,

Pith/arXiv arXiv
[2021]

URLhttps://doi.org/10.5281/zenodo.5143773. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El...

doi:10.5281/zenodo.5143773
[2022]

Fixing weight decay regularization in adam.CoRR, abs/1711.05101, 2017a

Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam.CoRR, abs/1711.05101, 2017a. URL http://arxiv.org/abs/1711.05101. Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts, 2017b. URL https: //arxiv.org/abs/1608.03983. Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over ...

Pith/arXiv arXiv
[2023]

Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner

URLhttp://arxiv.org/abs/2305.20088. Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. Towards a definition of disentangled representations,

Pith/arXiv arXiv
[2024]

Alex Krizhevsky

URLhttp://arxiv.org/abs/2402.15120. Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto,

Pith/arXiv arXiv

[1] [3]

URLhttp://arxiv.org/abs/2306.14610. Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July

Pith/arXiv arXiv

[2] [7]

Steven J Leon, Åke Björck, and Walter Gander

URL http: //arxiv.org/abs/2405.00740. Steven J Leon, Åke Björck, and Walter Gander. Gram-schmidt orthogonalization: 100 years and more.Numerical Linear Algebra with Applications, 20(3):492–532,

Pith/arXiv arXiv

[3] [9]

Aarohi Srivastava, Abhinav Rastogi, and Abhishek Rao et al

URLhttp://arxiv.org/abs/2403.20312. Aarohi Srivastava, Abhinav Rastogi, and Abhishek Rao et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,

Pith/arXiv arXiv

[4] [10]

Norbert Vanek and Haoruo Zhang

URLhttps://arxiv.org/abs/2206.04615. Norbert Vanek and Haoruo Zhang. On truth and polarity in negation processing: language-specific effects in non- linguistic contexts.Frontiers in Psychology, V olume 14 - 2023,

Pith/arXiv arXiv 2023

[5] [11]

doi: 10.3389/fpsyg.2023

ISSN 1664-1078. doi: 10.3389/fpsyg.2023. 1244249. URL https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023. 1244249. Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it?,

doi:10.3389/fpsyg.2023 2023

[6] [12]

Kecheng Zheng, Yifei Zhang, Wei Wu, Fan Lu, Shuailei Ma, Xin Jin, Wei Chen, and Yujun Shen

URLhttp://arxiv.org/abs/2210.01936. Kecheng Zheng, Yifei Zhang, Wei Wu, Fan Lu, Shuailei Ma, Xin Jin, Wei Chen, and Yujun Shen. DreamLIP: Language-image pre-training with long captions,

Pith/arXiv arXiv

[7] [13]

{original}

URLhttp://arxiv.org/abs/2403.17007. 11 A Prompt Templates for Caption Generation The following templates provide an example of paraphrased and negated caption candidate generation by either Phi-4 or Mistral-7B model as well as the subsequent quality validation of generation by Mistral-7B. Caption Generation Template 1Paraphrased caption candidate 2(Phi-4/...

Pith/arXiv arXiv 2024

[8] [14]

(b) Effect of setting the number of projection vectors on Top-1 accuracy using paraphrased caption for image matching

Downstream Models Tasks SemCLIP SemCLIP CoN-CLIP (CC-Neg) (SCPP) CIFAR-10 90.691.690.45 CIFAR-100 62.566.562.31 Food-101 50.7 62.783.39 Flowers102 36.0 37.564.74 Oxford-IIIT Pet 53.5 69.581.66 14 (a) Effect of setting the number of projection vectors on Top-1 accuracy using original caption for image matching. (b) Effect of setting the number of projectio...

2024

[9] [2009]

Technical Report, CIFAR-10 and CIFAR-100 datasets

URL https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf . Technical Report, CIFAR-10 and CIFAR-100 datasets. Samuel Lavoie, Polina Kirichenko, Mark Ibrahim, Mahmoud Assran, Andrew Gordon Wilson, Aaron Courville, and Nicolas Ballas. Modeling caption diversity in contrastive vision-language pretraining,

2009

[10] [2018]

Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna

URLhttps://arxiv.org/abs/1812.02230. Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna. SugarCrepe: Fixing hackable benchmarks for vision-language compositionality,

Pith/arXiv arXiv

[11] [2021]

URLhttps://doi.org/10.5281/zenodo.5143773. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El...

doi:10.5281/zenodo.5143773

[12] [2022]

Fixing weight decay regularization in adam.CoRR, abs/1711.05101, 2017a

Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam.CoRR, abs/1711.05101, 2017a. URL http://arxiv.org/abs/1711.05101. Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts, 2017b. URL https: //arxiv.org/abs/1608.03983. Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over ...

Pith/arXiv arXiv

[13] [2023]

Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner

URLhttp://arxiv.org/abs/2305.20088. Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. Towards a definition of disentangled representations,

Pith/arXiv arXiv

[14] [2024]

Alex Krizhevsky

URLhttp://arxiv.org/abs/2402.15120. Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto,

Pith/arXiv arXiv