Pith. sign in

REVIEW 4 cited by

In Defense of Cross-Encoders for Zero-Shot Retrieval

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2212.06121 v1 pith:O6ZEAPRT submitted 2022-12-12 cs.IR cs.CL

Guilherme Rosa , Luiz Bonifacio , Vitor Jeronymo , Hugo Abonizio , Marzieh Fadaee , Roberto Lotufo , Rodrigo Nogueira This is my paper · ORCID
classification cs.IRcs.CL
keywords cross-encodersbi-encodersgainsretrievalabilitygeneralizationin-domainout-of-domain
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Bi-encoders and cross-encoders are widely used in many state-of-the-art retrieval pipelines. In this work we study the generalization ability of these two types of architectures on a wide range of parameter count on both in-domain and out-of-domain scenarios. We find that the number of parameters and early query-document interactions of cross-encoders play a significant role in the generalization ability of retrieval models. Our experiments show that increasing model size results in marginal gains on in-domain test sets, but much larger gains in new domains never seen during fine-tuning. Furthermore, we show that cross-encoders largely outperform bi-encoders of similar size in several tasks. In the BEIR benchmark, our largest cross-encoder surpasses a state-of-the-art bi-encoder by more than 4 average points. Finally, we show that using bi-encoders as first-stage retrievers provides no gains in comparison to a simpler retriever such as BM25 on out-of-domain tasks. The code is available at https://github.com/guilhermemr04/scaling-zero-shot-retrieval.git

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MICE: Minimal Interaction Cross-Encoders for efficient Re-ranking

    cs.IR 2026-02 conditional novelty 6.0 of 10

    MICE is a cross-encoder-derived late-interaction ranker that retains most in-domain effectiveness and beats same-size ColBERT by 5-8 nDCG@10 points while cutting latency up to 4x with precomputed document vectors.

  2. Automating AI Failure Tracking: Semantic Association of Reports in AI Incident Database

    cs.CY 2025-07 conditional novelty 4.0 of 10

    Sentence-embedding retrieval ranks the correct AI Incident in the top three for about 98% of test reports when titles and descriptions are combined, but possible train/test leakage likely inflates that number.

  3. Optimizing Legal Document Retrieval in Vietnamese with Semi-Hard Negative Mining

    cs.IR 2025-07 conditional novelty 4.0 of 10

    A lightweight Bi-Encoder plus Cross-Encoder pipeline with random top-candidate negative sampling achieves 79.1% MRR@10 on Vietnamese legal retrieval.

  4. Beyond Retrieval: Ensembling Cross-Encoders and GPT Rerankers with LLMs for Biomedical QA

    cs.IR 2025-07 conditional novelty 4.0 of 10

    An ensemble of finetuned cross-encoders and GPT-4o rerankers over dense-vector retrieval achieved rank 10 in BioASQ 2025 Phase A retrieval and rank 1 on factoid answers.

Pith tools