REVIEW 14 cited by
Large Dual Encoders Are Generalizable Retrievers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
It has been shown that dual encoders trained on one domain often fail to generalize to other domains for retrieval tasks. One widespread belief is that the bottleneck layer of a dual encoder, where the final score is simply a dot-product between a query vector and a passage vector, is too limited to make dual encoders an effective retrieval model for out-of-domain generalization. In this paper, we challenge this belief by scaling up the size of the dual encoder model {\em while keeping the bottleneck embedding size fixed.} With multi-stage training, surprisingly, scaling up the model size brings significant improvement on a variety of retrieval tasks, especially for out-of-domain generalization. Experimental results show that our dual encoders, \textbf{G}eneralizable \textbf{T}5-based dense \textbf{R}etrievers (GTR), outperform %ColBERT~\cite{khattab2020colbert} and existing sparse and dense retrievers on the BEIR dataset~\cite{thakur2021beir} significantly. Most surprisingly, our ablation study finds that GTR is very data efficient, as it only needs 10\% of MS Marco supervised data to achieve the best out-of-domain performance. All the GTR models are released at https://tfhub.dev/google/collections/gtr/1.
Forward citations
Cited by 14 Pith papers
-
Certified Domain Consistency for Multi-Domain Retrieval: Label-Free Per-Domain Contamination Control with Conformal Risk Guarantees
C3R certifies per-domain retrieval contamination budgets using a two-split conformal scheme, without query-time domain labels.
-
KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking
KaLM-Reranker-V1 uses encoder–decoder FBNL with Matryoshka pooling to match Qwen3-class reranking quality at substantially lower online cost.
-
GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models
A LoRA-tuned LLM with trie-constrained decoding improves grocery category recommendation and yields a 7.5% cart-add lift in production.
-
Depth Gives a False Sense of Privacy: LLM Internal States Inversion
LLM internal states at intermediate layers contain enough information to recover long, sensitive user prompts with high accuracy.
-
When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR
GradNormIR uses gradient norms from a retriever's own contrastive loss to detect out-of-distribution documents and schedule retriever updates before queries arrive.
-
Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings
A new benchmark (ConTEB) and training method (InSeNT) show that context-aware chunk embeddings greatly improve retrieval on context-dependent queries, with minimal computational overhead.
-
LogiCoL: Logically-Informed Contrastive Learning for Set-based Dense Retrieval
A dense retriever trained with subset and exclusion constraints on logically related query pairs improves recall on queries with AND, OR, and NOT connectives.
-
IDEAlign: Comparing Large Language Models to Human Experts in Open-ended Interpretive Annotations
IDEAlign uses a pick-the-odd-one-out triplet task to measure idea-level similarity between LLMs and expert human annotations, and shows LLM judges using this protocol outperform lexical and vector-based baselines.
-
From Attack Descriptions to Vulnerabilities: A Sentence Transformer-Based Approach
A sentence-transformer benchmark finds MMPNet best links ATT&CK attack technique descriptions to CVE vulnerability records (F1=89) and proposes 275 manually vetted new links.
-
Optimizing RAG Pipelines for Arabic: A Systematic Analysis of Core Components
For Arabic retrieval-augmented generation, sentence-aware chunking, BGE-M3 and Multilingual-E5-large embeddings, bge-reranker-v2-m3, and Aya-8B yield the highest RAGAS scores across six Arabic datasets.
-
Efficient Data Selection at Scale via Influence Distillation
Influence Distillation selects LLM fine-tuning data by approximating each sample's gradient influence on a target task via landmarks and JVP embeddings, matching or beating RDS+ accuracy at roughly one third the selec...
-
QZhou-Embedding Technical Report
QZhou-Embedding reports state-of-the-art average scores on MTEB and CMTEB as of August 27, 2025, using a two-stage multi-task pipeline with LLM-based data synthesis.
-
A Multi-Task Evaluation of LLMs' Processing of Academic Text Input
The abstract reports Gemini underperforms on four academic text tasks, but the attached full text is an unrelated biomedical retrieval paper, leaving the claims unverifiable.
-
Choosing a Text Embedding Model: A Practical Benchmarking and Decision Framework
On four BEIR subsets T3EM leads nDCG@10 (0.638) but mE5-L is the recommended open default; training objective and chunk size dominate size, and no model wins every MTEB task.
Discussion (0). Sign in to comment.