Pith. sign in

REVIEW 5 cited by

Fine-Tuning LLaMA for Multi-Stage Text Retrieval

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.08319 v1 pith:KJ4KQ32N submitted 2023-10-12 cs.IR

Xueguang Ma , Liang Wang , Nan Yang , Furu Wei , Jimmy Lin This is my paper · ORCID
classification cs.IR
keywords modelsretrievaleffectivenesslanguagellmsdemonstratefine-tuninglarge
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The effectiveness of multi-stage text retrieval has been solidly demonstrated since before the era of pre-trained language models. However, most existing studies utilize models that predate recent advances in large language models (LLMs). This study seeks to explore potential improvements that state-of-the-art LLMs can bring. We conduct a comprehensive study, fine-tuning the latest LLaMA model both as a dense retriever (RepLLaMA) and as a pointwise reranker (RankLLaMA) for both passage retrieval and document retrieval using the MS MARCO datasets. Our findings demonstrate that the effectiveness of large language models indeed surpasses that of smaller models. Additionally, since LLMs can inherently handle longer contexts, they can represent entire documents holistically, obviating the need for traditional segmenting and pooling strategies. Furthermore, evaluations on BEIR demonstrate that our RepLLaMA-RankLLaMA pipeline exhibits strong zero-shot effectiveness. Model checkpoints from this study are available on HuggingFace.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG

    cs.CL 2025-09 conditional novelty 6.0 of 10

    In multilingual retrieval-augmented generation, models cite English evidence more accurately than translated evidence, and this language preference can outweigh document relevance.

  2. RaDeR: Reasoning-aware Dense Retrieval Models

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A math-trained dense retriever and reranker, built from MCTS reasoning trajectories and self-reflection, outperforms strong baselines on reasoning-intensive retrieval benchmarks and beats BM25 on chain-of-thought queries.

  3. Reranking with Compressed Document Representation

    cs.IR 2025-05 conditional novelty 6.0 of 10

    A reranker trained on 8-token PISCO document embeddings plus a short query achieves near-identical nDCG@10 to full-text rerankers on BeIR and TREC-DL while running up to 16x faster.

  4. Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval

    cs.CL 2025-08 conditional novelty 4.0 of 10

    Reasoning-infused text embedding, which prepends LLM-generated reasoning to queries before embedding, improves zero-shot dense retrieval on BRIGHT.

  5. LineRetriever: Planning-Aware Observation Reduction for Web Agents

    cs.CL 2025-06 conditional novelty 4.0 of 10

    LineRetriever uses a small LM to select relevant lines from web page observations, cutting context by up to 73% with only small success-rate drops on web agent benchmarks.

Pith tools