REVIEW 22 cited by
Overview of the TREC 2020 deep learning track
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime. We again have a document retrieval task and a passage retrieval task, each with hundreds of thousands of human-labeled training queries. We evaluate using single-shot TREC-style evaluation, to give us a picture of which ranking methods work best when large data is available, with much more comprehensive relevance labeling on the small number of test queries. This year we have further evidence that rankers with BERT-style pretraining outperform other rankers in the large data regime.
Forward citations
Cited by 22 Pith papers
-
Mutual Linearity in and out of Stationarity for Markov Jump Processes: A Trajectory-Based Approach
Trajectory-level linear response yields mutual linearity of observables under single-edge rate perturbation for Markov jump processes, including non-stationary state and counting observables.
-
The Matryoshka Hypencoder
A Matryoshka-style loss lets one Hypencoder generate Q-Nets of different widths, so a single model can be deployed at multiple efficiency-effectiveness points.
-
LLMs Encode Relevance as a Layer-Wise Cross-Lingual Signal
Large language models encode query-document relevance as a linearly decodable internal signal that strengthens in middle-to-late layers and, in several models, outperforms their own generated judgments.
-
Tool-Adaptive LLM Reranker
Pointwise LLM reranking cast as an agentic MDP with language-preserving warm-up and asymmetric cost-aware RL yields adaptive tool use, SOTA NDCG, and high throughput.
-
Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain
Misleading health documents in RAG context sharply lower LLM accuracy, and heavily helpful-biased retrieval pools restore it.
-
Upcycling Candidate Tokens of Large Language Models for Query Expansion
Using unselected top-k candidate tokens from a single LLM decoding pass as extra query terms improves retrieval over standard keyword expansion while using far fewer tokens than document-level methods.
-
ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking
ERank trains a pointwise reasoning-LLM reranker with fine-grained integer scoring and a listwise reinforcement-learning reward, reporting nDCG@10 of 40.2 on BRIGHT for its 32B variant with BM25 hybrid scoring.
-
Shifting from Ranking to Set Selection for Retrieval Augmented Generation
SETR identifies a query's information requirements with chain-of-thought reasoning and selects a compact passage set, improving multi-hop RAG accuracy over fixed-top-k reranking baselines.
-
Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information
A summary-based anchor document enables contrastive pointwise scoring that, when averaged with ordinary pointwise scores, improves zero-shot LLM reranking.
-
Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval
Dense retrieval embeddings can be decomposed into interpretable latent concepts that serve both as explanations and as efficient sparse indexing units for retrieval.
-
RaDeR: Reasoning-aware Dense Retrieval Models
A math-trained dense retriever and reranker, built from MCTS reasoning trajectories and self-reflection, outperforms strong baselines on reasoning-intensive retrieval benchmarks and beats BM25 on chain-of-thought queries.
-
Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?
Standard pointwise rerankers beat reasoning-based pointwise rerankers under identical training, and removing the reasoning process at inference partially recovers the gap.
-
Reranking with Compressed Document Representation
A reranker trained on 8-token PISCO document embeddings plus a short query achieves near-identical nDCG@10 to full-text rerankers on BeIR and TREC-DL while running up to 16x faster.
-
Rank-K: Test-Time Reasoning for Listwise Reranking
Rank-K, a reasoning-model-based listwise reranker distilled from DeepSeek R1 traces, beats RankZephyr on several benchmarks but only marginally on TREC DL 2019/2020.
-
Does UMBRELA Work on Other LLMs?
UMBRELA relevance judgments made with DeepSeek V3 are close to GPT-4o, and even small models preserve leaderboard rankings although per-document agreement with humans drops.
-
Are Optimal Algorithms Still Optimal? Rethinking Sorting in LLM-Based Pairwise Ranking with Batching and Caching
Under an LLM-inference cost model, Quicksort with batching uses roughly 44% fewer inference calls than Heapsort for pairwise document ranking, reversing the classical comparison-count ordering.
-
RankLLM: A Python Package for Reranking with LLMs
RankLLM is an open-source Python package that modularly supports pointwise, pairwise, and listwise LLM rerankers, with integrated retrieval, evaluation, training, and response analysis, and reproduces results from Ran...
-
Measuring Hypothesis Testing Errors in the Evaluation of Retrieval Systems
The paper adds Type II error metrics to the evaluation of relevance judgment sets and shows that balanced accuracy and Matthews correlation can summarize qrels' discriminative power in one number.
-
The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers
The paper proposes a full-paper, evidence-aware retrieval agenda for scientific fact-checking and shows that combining verification feedback with semantic reranking improves evidence recall on SciFact-Open and Check-COVID.
-
Comparative Analysis of Lion and AdamW Optimizers for Cross-Encoder Reranking with MiniLM, GTE, and ModernBERT
Lion and AdamW are compared for reranker fine-tuning, but the reported Lion gains are confounded by a 10x learning-rate difference and an inverted GPU-utilization metric.
-
GOLFer: Smaller LM-Generated Documents Hallucination Filter & Combiner for Query Expansion in Information Retrieval
GOLFer filters hallucinated sentences from small-LM-generated hypothetical documents and reweights the rest into the query, improving retrieval at lower cost than large LLM expansion.
-
Exp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query Expansion
Exp4Fuse improves sparse retrieval by fusing the ranked lists from the original query and an LLM-expanded query using a modified reciprocal rank fusion.
Discussion (0). Continue with ORCID to comment.