Pith. sign in

REVIEW 4 cited by

Jina CLIP: Your CLIP Model Is Also Your Text Retriever

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.20204 v2 pith:VPRIFEC4 submitted 2024-05-30 cs.CL cs.AIcs.CVcs.IR

Andreas Koukounas , Georgios Mastrapas , Michael Günther , Bo Wang , Scott Martens , Isabelle Mohr , Saba Sturua , Mohammad Kalim Akram
show 6 more authors
Joan Fontanals Martínez Saahil Ognawala Susana Guzman Maximilian Werk Nan Wang Han Xiao
This is my paper · ORCID
classification cs.CLcs.AIcs.CVcs.IR
keywords modelscliptasksretrievalcontrastiveinformationmodelmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Contrastive Language-Image Pretraining (CLIP) is widely used to train models to align images and texts in a common embedding space by mapping them to fixed-sized vectors. These models are key to multimodal information retrieval and related tasks. However, CLIP models generally underperform in text-only tasks compared to specialized text models. This creates inefficiencies for information retrieval systems that keep separate embeddings and models for text-only and multimodal tasks. We propose a novel, multi-task contrastive training method to address this issue, which we use to train the jina-clip-v1 model to achieve the state-of-the-art performance on both text-image and text-text retrieval tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning

    cs.CV 2025-08 reject novelty 6.0 of 10

    A new audio-visual reasoning benchmark and a two-part metric (factual consistency, core inference) claim to expose a gap between answer accuracy and reasoning quality in AV-LLMs.

  2. MM-R5: MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrieval

    cs.AI 2025-06 conditional novelty 6.0 of 10

    MM-R5, a 7B multimodal re-ranker trained with SFT and GRPO, achieves state-of-the-art page-level recall on MMDocIR by generating per-page reasoning chains.

  3. Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning

    cs.CL 2025-06 conditional novelty 5.0 of 10

    State-of-the-art text embeddings lag far behind on tasks requiring pragmatic inference, stance detection, and social meaning, relative to their strong performance on surface semantic benchmarks.

  4. Distill CLIP (DCLIP): Enhancing Image-Text Retrieval via Cross-Modal Transformer Distillation

    cs.CV 2025-05 conditional novelty 5.0 of 10

    DCLIP fine-tunes a CLIP student's image encoder to match a YOLO-region, bidirectional cross-attention teacher, improving retrieval while retaining most zero-shot accuracy.

Pith tools