REVIEW 10 cited by
Jina CLIP: Your CLIP Model Is Also Your Text Retriever
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Contrastive Language-Image Pretraining (CLIP) is widely used to train models to align images and texts in a common embedding space by mapping them to fixed-sized vectors. These models are key to multimodal information retrieval and related tasks. However, CLIP models generally underperform in text-only tasks compared to specialized text models. This creates inefficiencies for information retrieval systems that keep separate embeddings and models for text-only and multimodal tasks. We propose a novel, multi-task contrastive training method to address this issue, which we use to train the jina-clip-v1 model to achieve the state-of-the-art performance on both text-image and text-text retrieval tasks.
Forward citations
Cited by 10 Pith papers
-
AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning
A new audio-visual reasoning benchmark and a two-part metric (factual consistency, core inference) claim to expose a gap between answer accuracy and reasoning quality in AV-LLMs.
-
CONCAP: Seeing Beyond English with Concepts Retrieval-Augmented Captioning
CONCAP combines retrieved captions with retrieved concepts to improve multilingual image captioning, reaching 34.2 average CIDEr on XM3600 against 31.8 for Pangea and 25.9 for mBLIP while training on 566K pairs.
-
jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval
jina-embeddings-v4 is a multimodal embedding model built on Qwen2.5-VL with LoRA adapters and dual single/multi-vector outputs, and it introduces the Jina-VDR benchmark for visually rich retrieval.
-
MM-R5: MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrieval
MM-R5, a 7B multimodal re-ranker trained with SFT and GRPO, achieves state-of-the-art page-level recall on MMDocIR by generating per-page reasoning chains.
-
Progressive Multimodal Reasoning via Active Retrieval
AR-MCTS retrieves per-step multimodal knowledge during tree search and trains a process reward model, improving MLLM reasoning accuracy on three benchmarks by a few points.
-
FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training
Frozen LLM text encoders, combined with multi-prompt hidden-state extraction and cached embeddings, make CLIP-style pre-training data-efficient, long-context aware, and multilingual.
-
Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning
State-of-the-art text embeddings lag far behind on tasks requiring pragmatic inference, stance detection, and social meaning, relative to their strong performance on surface semantic benchmarks.
-
Distill CLIP (DCLIP): Enhancing Image-Text Retrieval via Cross-Modal Transformer Distillation
DCLIP fine-tunes a CLIP student's image encoder to match a YOLO-region, bidirectional cross-attention teacher, improving retrieval while retaining most zero-shot accuracy.
-
jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images
jina-clip-v2, an 865M-parameter multilingual dual-encoder, outperforms prior CLIP-style models on text-only and crossmodal retrieval, and on visually rich document retrieval, while supporting flexible embedding dimensions.
-
Human Action CLIPs: Detecting AI-generated Human Motion
CLIP-based semantic embeddings, with a fine-tuned variant, detect AI-generated human-motion video with high accuracy (up to 99.2% video-level) and generalize to unseen generators.
Discussion (0). Continue with ORCID to comment.