Pith. sign in

REVIEW 3 major objections 7 minor 76 references

Relevance is better treated as a posterior refined by offline document priors plus online query likelihoods, not a single query-time score.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 18:58 UTC pith:GRKPXNQS

load-bearing objection Solid systems framing that reconnects neural IR to classical priors, with narrow but honest fusion experiments—not a new law of retrieval. the 3 major comments →

arxiv 2607.23561 v1 pith:GRKPXNQS submitted 2026-07-26 cs.IR

Towards a Relevance Posterior in Neural Information Access

Andrew Parry , Emmanouil Georgios Lionis , Debasis Ganguly , Sean MacAvaney This is my paper
classification cs.IR
keywords Neural RankingStatic RankingDocument priorsLikelihood-prior decompositionScore fusionMulti-stage retrievalRAGApproximate posterior inference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern search systems almost always score relevance only when a query arrives, packing document quality, authority, and match into one expensive online score. This paper argues that view is limiting. Relevance is more naturally a posterior belief refined in stages: stable, query-independent beliefs about documents can be learned once offline as priors, while query-conditioned models supply the likelihood that resolves intent. Making that split explicit lets systems cache document utility, fuse it with existing rankers at almost no query-time cost, and allocate modelling capacity between offline and online work. On standard TREC Deep Learning passage tests, a learned quality prior fused by simple addition lifts BM25 first-stage ranking and, especially, LLM listwise re-ranking, while the same fusion can hurt a strong learned sparse retriever—evidence that complementarity is real but not automatic. The broader payoffs the authors target are efficiency (less online work), effectiveness (new document-side signal), and control (priors as tunable policy knobs across ranking, RAG, and recommendation).

Core claim

The authors claim modern information access is approximate posterior inference over relevance, and that an explicit likelihood–prior decomposition—extending classical probabilistic retrieval to neural pipelines—lets a cached query-independent document utility complement deployed rankers through score fusion alone, improving first-stage and re-ranking effectiveness without extra document-side inference at query time.

What carries the argument

Likelihood–prior (relevance posterior) decomposition: treat a cached document utility u_ϕ(d) as a log-prior potential and fuse it additively with a query-conditioned score, S = s_θ(q,d) + γ u_ϕ(d), so offline document beliefs and online interaction remain separable, updatable, and composable under staged approximation (truncation, ANN, context budgets).

Load-bearing premise

The load-bearing premise is that a query-free quality model’s scores act as a complementary document prior under simple fixed-weight addition, rather than mostly repeating signals strong neural retrievers already carry.

What would settle it

On the same TREC DL setups, if uncalibrated additive fusion of the quality prior with BM25 and with RankZephyr never improves nDCG@10 (and never beats pruning), or if tuned fusion still cannot help without hurting strong sparse first-stages for reasons other than scale mismatch, the complementarity claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Stable document properties (quality, trust, usability) can be paid for once offline and reused across retrievers, re-rankers, and RAG context filters.
  • First-stage depth, re-ranking depth, and prior strength become joint design variables under a fixed latency or token budget.
  • Policy, safety, freshness, and personalization can be adjusted as prior terms without retraining the query-time model.
  • Ad-hoc search, static ranking, and recommendation sit on one posterior family differing mainly in which of document, query, and user are conditioned on.
  • LLM re-rankers and generators may benefit disproportionately when candidates are pre-ordered by document-side utility before listwise or context-budgeted interaction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The SPLADE first-stage drops already hint that “prior” training must be residual to the deployed likelihood, not a generic quality classifier, or fusion will double-count.
  • Directly training the offline utility against RAG answer quality or listwise LLM preference, still without the query in the prior’s input, is a natural next experiment the results invite.
  • If priors become the control surface, A/B tests and audits can move from full ranker swaps to prior-only updates—closer to how recommenders already ship item-side changes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper argues that modern neural IR pipelines are best understood as staged approximate posterior inference over relevance, with query-independent document beliefs (priors) computed offline and query-conditioned evidence (likelihoods) resolved online, mediated by approximation operators (truncation, ANN recall, context budgets). Sections 3–4 develop this vocabulary from classical probabilistic retrieval (Eq. 1–9) and a system taxonomy (Table 1). Section 5 gives preliminary empirical support: additive score fusion (Eq. 12, γ=1.0, uncalibrated) of a cached QualT5 quality log-odds prior with BM25 improves nDCG@10 significantly on TREC DL-2019/2020, degrades SPLADE first-stage (reported openly), and yields its largest gains under RankZephyr LLM re-ranking; a pruning control isolates fusion vs. gating effects. Section 6 develops implications (efficiency, effectiveness, control) and a recommender-systems connection (Eq. 14). The contribution is primarily a perspective with supporting experiments, positioned as preliminary.

Significance. If the results hold, the paper contributes a coherent, historically grounded vocabulary for allocating computation between offline priors and online scoring — timely given LLM re-ranking and RAG cost pressures — plus a practically cheap recipe: cached prior + score fusion at near-zero query-time cost, with +0.03–0.05 nDCG@10 on standard benchmarks. Credit is due for several good-practice elements: paired t-tests with Bonferroni correction, a pruning ablation separating prior-as-gate from prior-as-signal, honest reporting of the SPLADE first-stage degradations rather than hiding them, a qualitative per-intent analysis, and a public code release (Zenodo). The negative result (fusion hurts strong learned-sparse retrievers at first stage) is itself informative for the community.

major comments (3)
  1. [§5.1–5.2, Eq. (12), Table 2] Eq. (12) and Table 2: all headline gains rest on additive fusion with γ=1.0 and no calibration between BM25/SPLADE scores and QualT5 log-odds, which live on incomparable scales. The motivation ('isolate whether the signal is complementary in principle') is reasonable, but the abstract's quantitative claims could partly reflect a fortunate scale coincidence; the SPLADE degradation is itself consistent with scale mismatch. A small γ sweep or per-system score normalisation (e.g., z-score) on a dev split would show the effect's sign is robust rather than scale-dependent, and is cheap given the released code. As written, readers cannot tell how fragile the +0.046/+0.054 numbers are.
  2. [§5.2 'Inference Setting'; §5.1] The claim that independent θ and ϕ ensure 'improvements reflect complementary utility estimation' is stronger than the design supports. QualT5 is trained as MonoT5-minus-the-query on MSMARCO relevants vs BM25 hard negatives (Chang et al.) — the same relevance-label process that supervises the rankers. The prior is therefore query-free in input but not supervision-independent: it may encode 'relevance-without-query' on the same label distribution rather than a distinct utility factor, and the §5.1 'functionally orthogonal' argument covers only data access. Please soften the complementarity language or add a control prior trained without relevance labels (e.g., readability/spam) to separate the two mechanisms.
  3. [§5.2, §5.3; cf. §4, §6.1] Fusion is applied only to the retriever's top-1000, so the prior can reorder but never augment the candidate set; R@1000 preservation (§5.3) holds by construction, not as a finding. This sits uneasily with the framework's central motivation that priors shape posterior support (§4 'support restriction failures', §6.1 index-side filtering): the experiment tests prior-as-reweighting only. Either add a full-corpus fusion condition (e.g., CombSUM/RRF of BM25 with the prior ranking, which could surface documents BM25 missed) or scope the support/recall claims to reweighting within a fixed candidate pool.
minor comments (7)
  1. [§6.4, Eq. (14)] Eq. (14): p(d|q,u) ∝ p(q|d)p(d|u)p(d) double-counts the document prior. Under q⊥u|d, Bayes gives p(d|q,u) ∝ p(q|d)p(u|d)p(d) = p(q|d)p(d|u)p(u), i.e. the third factor should be the user marginal p(u) (a constant per request), not p(d). Please restate or derive.
  2. [Table 2, §5.2] Table 2: (i) the Bonferroni correction family is not defined — how many comparisons are corrected per cell block? (ii) 'Overall values are computed via geometric mean' over two collections is non-standard for nDCG@10; please justify or report macro-average. (iii) DL-2019/2020 have 43/45 queries; a note on test power for the small CE-reranking deltas would help interpret the unmarked cells.
  3. [Figures 1–2] Figure 2 caption ends mid-sentence ('with and without reranking.' — unclosed parenthesis) and only mentions BM25, while the panels show MAP/R@100 for DL-2020 only; please state collection and metric per panel and fix the caption. Figure 1 is dense and hard to parse at column width.
  4. [§5.2 'Quality Estimators'] QualT5-Tiny/Small/Base are referenced from Chang et al. but never specified here (parameter counts, training data size); since a capacity trend is claimed in §5.3, a one-line summary table would make the paper self-contained.
  5. [§5.1, §3] 'Zero marginal query-time cost' (§5.1) should note the lookup/addition cost and the one-time offline cost of scoring all 8.8M MSMARCO passages with a T5 encoder–decoder, plus index storage for the cached scores — small, but relevant to the amortised-inference argument of §3.
  6. [§2, §5.2, Figure 1] Typographical: 'prior likelihood–decomposition' in §2 (RAG paragraph) has reversed hyphenation ('prior–likelihood'); 'monoELECTRA'/'MonoELECTRA' inconsistent capitalisation; a stray 'Efficiency' token appears in the Figure 1 text block (layout artefact).
  7. [§5.3 'Qualitative Differences'] The qualitative finding that gains concentrate on navigational/definitional intents (§5.3) is interesting; reporting the per-intent query counts or a small breakdown table would let readers judge how much of the mean gain this subset explains.

Circularity Check

1 steps flagged

No load-bearing circular derivation; mild interpretive renaming of cached quality scores as log-priors, with empirical claims checked on held-out TREC judgments.

specific steps
  1. renaming known result [Section 5.1, Eqs. (10)–(12)]
    "We define a query-independent document utility u_φ(d)∈R, interpreted as a log prior potential, u_φ(d)≈log p̃_φ(d), ... We adopt additive fusion, S_add(q,d)=s_θ(q,d)+γ u_φ(d), γ≥0, which recovers the log-posterior of Eq. (1) when s_θ is calibrated as a log-likelihood and u_φ as a log-prior"

    Calling an offline quality classifier’s log-odds a ‘log prior’ and additive fusion a ‘log-posterior’ renames standard static-feature fusion in posterior language. The identity is interpretive (and the paper admits the uncalibrated γ=1 setting), not a derivation that forces the TREC gains; still a mild definitional gloss rather than a predictive collapse.

full rationale

The paper’s central move is a systems perspective (retrieval as staged approximate posterior refinement) plus an empirical score-fusion study. Classical Bayes ranking p(d|q)∝p(q|d)p(d) is used as motivation, not as a uniqueness theorem that forces the results. The only near-circular step is definitional: QualT5 log-odds are labeled a log-prior potential u_φ(d)≈log p̃_φ(d) and additively fused with fixed γ=1.0; that is an interpretive ansatz, not a fit that manufactures the reported nDCG deltas. Fusion weights are not learned on the test metrics, and effectiveness is measured on external TREC DL-2019/2020 judgments against BM25/SPLADE and re-rankers. Reuse of Chang et al. QualT5 checkpoints (overlapping coauthor MacAvaney) is tool reuse for a static signal, not a self-cited uniqueness result that the argument depends on. SPLADE first-stage degradations further show the outcome is not forced by construction. Overall circularity is negligible.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The load-bearing content is a modeling stance (retrieval ≈ staged posterior refinement) plus an empirical protocol that treats an existing quality classifier as log-prior potential and adds it to deployed scores. Little new mathematics; dependence is on PRP/query-likelihood folklore, amortised-inference rhetoric, MSMARCO-trained QualT5 labels, and hand-set fusion choices.

free parameters (5)
  • fusion weight γ = 1.0 (hand-set)
    Additive fusion S = s_θ(q,d) + γ u_ϕ(d); authors fix γ=1.0 with no learning or calibration to isolate complementarity.
  • fusion candidate depth = top-1000 fuse / top-100 re-rank
    Fusion applied on retriever top-1000; re-rank top-100—pipeline cutoffs that affect who can benefit from the prior.
  • pruning threshold = 20th percentile
    Baseline static pruning removes bottom 20th percentile by QualT5-Base quality.
  • BM25 hyperparameters = k1=1.2, b=0.75, k3=8
    Robust04-tuned defaults in framework (k1, b, k3) shape the lexical likelihood baseline being ‘corrected’.
  • RankZephyr window/stride = window 20, stride 10
    Listwise LLM re-ranking configuration taken from prior art; affects measured +LLM gains.
axioms (5)
  • domain assumption Probability Ranking Principle / query-likelihood decomposition p(d|q) ∝ p(q|d)p(d) justifies treating static utility as a prior term composable with a query score.
    Section 3 imports classical probabilistic IR as the formal backbone for neural pipelines.
  • ad hoc to paper Unnormalised additive fusion in score space is a valid stand-in for log-posterior combination when s_θ and u_ϕ are uncalibrated.
    Section 5.1 sets S_add = s_θ + γ u_ϕ and explicitly skips calibration; ranking claims rest on this operational choice.
  • domain assumption QualT5 query-free quality log-odds approximate a document utility prior log p̃(d) rather than only a pruning gate.
    Section 5.2 reuses Chang et al. models trained with BCE on MSMARCO relevants vs BM25 hard negatives without query text.
  • domain assumption Stable document-side factors can be amortised offline without harming intent resolution when composed later as priors.
    Core perspective in Sections 1 and 4; motivates shifting compute offline.
  • ad hoc to paper Independence of θ and ϕ (no joint training / no mutual awareness) isolates complementary utility rather than co-adapted scoring.
    Stated experimental design in Section 5.2; interprets gains as evidence for decomposition.
invented entities (2)
  • Learned neural document prior u_ϕ(d) as first-class persistent posterior factor independent evidence
    purpose: Name and operationalise query-independent neural utility as a reusable offline belief composed throughout retrieval/RAG, not only as index pruning.
    Mostly a reframing of existing quality estimators and classical priors; little new ontology beyond the systems role assigned in Sections 4–6.
  • Staged posterior refinement operators (truncation, ANN, context budget) as explicit approximation vocabulary no independent evidence
    purpose: Taxonomise lexical/dense/multi-stage/RAG pipelines uniformly as prior/likelihood/approximation triples (Table 1).
    Descriptive systems language rather than a new physical/statistical object; usefulness is organizational.

pith-pipeline@v1.2.0-grok45-kimik3 · 25621 in / 3725 out tokens · 90553 ms · 2026-07-30T18:58:34.240870+00:00 · methodology

0 comments
read the original abstract

Modern information retrieval systems typically operationalise relevance as a query-conditional score computed at inference time. This design choice has become dominant such that alternative decompositions of relevance are rarely discussed, despite the long history of document and query priors in probabilistic retrieval and large-scale search. As neural ranking models grow more computationally expensive and retrieval pipelines expand to include multi-stage ranking, recommendation, and retrieval-augmented generation, this monolithic view of query-time scoring becomes increasingly limiting. We argue that modern information access systems are more naturally understood as performing approximate posterior inference, in which relevance is refined through a staged combination of query-dependent likelihoods and query-independent priors. We extend classical probabilistic retrieval formalisms to contemporary learned systems and show how explicit likelihood-prior decomposition exposes new opportunities to shift computation offline while disentangling document-level and interaction-level beliefs. We present empirical evidence that incorporating query-independent document utility can complement existing rankers and improve effectiveness with minimal query-time computation (solely score fusion). Concretely, a learned prior improves first-stage retrieval through rank fusion (up to 0.046 nDCG@10 on TREC DL-2019 and 0.029 nDCG@10 on TREC DL-2020) and also improves downstream re-ranking, with the largest gains observed for the LLM re-ranker RankZephyr (up to 0.054 nDCG@10 on TREC DL-2020). Finally, we discuss how this decomposition connects to broader information access and outline research directions for designing retrieval systems that explicitly allocate modelling capacity between offline priors and online interaction.

Figures

Figures reproduced from arXiv: 2607.23561 by Andrew Parry, Debasis Ganguly, Emmanouil Georgios Lionis, Sean MacAvaney.

Figure 1
Figure 1. Figure 1: An illustration of prior-likelihood decomposition [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: MAP and R@100 on TREC DL-2020 for BM25 under quality priors (QualT5-Tiny/Small/Base fusion and pruning, with [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An illustration of how posterior inference of [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

76 extracted references · 1 canonical work pages

  1. [1]

    Leif Azzopardi and Vishwa Vinay. 2008. Retrievability: an evaluation measure for higher order information access tasks. InProceedings of the 17th ACM Conference on Information and Knowledge Management(Napa Valley, California, USA)(CIKM ’08). Association for Computing Machinery, New York, NY, USA, 561–570. doi:10. 1145/1458082.1458157

    arXiv 2008

  2. [2]

    Elias Bassani, Pranav Kasela, and Gabriella Pasi. 2024. Denoising Attention for Query-aware User Modeling. InNAACL-HLT. https://api.semanticscholar.org/ CorpusID:271520023

    2024

  3. [3]

    NJ BELKIN. 1980. Anomalous States of Knowledge as a Basis for Information Retrieval.The Canadian Journal of Information Science5 (1980), 133–143

    1980

  4. [4]

    Blei, Andrew Y

    David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation.J. Mach. Learn. Res.3, null (March 2003), 993–1022

    2003

  5. [5]

    Marco Braga, Sean MacAvaney, Craig Macdonald, and Gabriella Pasi. 2025. Re- vealing MonoT5’s Learning Mechanisms via Prompt-Token Adaptation. InECIR

    2025

  6. [6]

    Barla Cambazoglu and Ricardo Baeza-Yates

    B. Barla Cambazoglu and Ricardo Baeza-Yates. 2016. Scalability and Efficiency Challenges in Large-Scale Web Search Engines. InProceedings of the 39th In- ternational ACM SIGIR Conference on Research and Development in Information Retrieval(Pisa, Italy)(SIGIR ’16). Association for Computing Machinery, New York, NY, USA, 1223–1226. doi:10.1145/2911451.2914808

    arXiv 2016

  7. [7]

    Manish Chandra, Debasis Ganguly, and Iadh Ounis. 2026. LURE-RAG: Light- weight Utility-Driven Reranking for Efficient RAG. InECIR (1) (Lecture Notes in Computer Science). Springer, 565–581

    2026

  8. [8]

    Xuejun Chang, Debabrata Mishra, Craig Macdonald, and Sean MacAvaney. 2024. Neural Passage Quality Estimation for Static Pruning. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024. ACM. doi:10.1145/3626772.3657765

    arXiv 2024

  9. [9]

    Le, and Christopher D

    Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. InICLR. https://openreview.net/pdf?id=r1xMH1BtvB

    2020

  10. [10]

    Cormack, Charles L A Clarke, and Stefan Buettcher

    Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval(Boston, MA, USA)(SIGIR ’09). Association for Computing Machinery, New York, NY, USA, 758–759...

    doi:10.1145/1571941 2009

  11. [11]

    Cormack, José María Gómez Hidalgo, and Enrique Puertas Sánz

    Gordon V. Cormack, José María Gómez Hidalgo, and Enrique Puertas Sánz. 2007. Spam filtering for short messages. InProceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management(Lisbon, Portugal) (CIKM ’07). Association for Computing Machinery, New York, NY, USA, 313–320. doi:10.1145/1321440.1321486

    arXiv 2007

  12. [12]

    Cormack, Mark D

    Gordon V. Cormack, Mark D. Smucker, and Charles L. A. Clarke. 2010. Efficient and effective spam filtering and re-ranking for large web datasets.Information Retrieval14 (2010), 441–465. https://api.semanticscholar.org/CorpusID:11861526 Towards a Relevance Posterior in Neural Information Access SIGIR ’26, July 20–24, 2026, Melbourne, VIC, Australia

    2010

  13. [13]

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2020. Overview of the TREC 2020 Deep Learning Track. InProceedings of the 29th Text REtrieval Conference, TREC 2020, Virtual Event, Gaithersburg, MD, USA, November 16-20, 2020 (NIST Special Publication, Vol. 1266), Ellen M. Voorhees and Angela Ellis (Eds.). National Institute of Standards and ...

    2020

  14. [14]

    Voorhees

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2019. Overview of the TREC 2019 Deep Learning Track. In28th International Text Retrieval Conference, TREC 2019, Gaithersburg, Maryland, USA (NIST Special Publication), Ellen M. Voorhees and Angela Ellis (Eds.). National Institute of Standards and Technology (NIST)

    2019

  15. [15]

    Jeffrey Dean. 2009. Challenges in building large-scale information retrieval systems: invited talk. InProceedings of the Second ACM International Conference on Web Search and Data Mining(Barcelona, Spain)(WSDM ’09). Association for Computing Machinery, New York, NY, USA, 1. doi:10.1145/1498759.1498761

    arXiv 2009

  16. [16]

    Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant

  17. [17]

    Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. InProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval(Virtual Event, Canada)(SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 2288–2292. doi:1...

    doi:10.1145/3404835 2021

  18. [18]

    Fox and Joseph A

    Edward A. Fox and Joseph A. Shaw. 1993. Combination of Multiple Searches. In Text Retrieval Conference. https://api.semanticscholar.org/CorpusID:1309301

    1993

  19. [19]

    Debasis Ganguly, Johannes Leveling, and Gareth J. F. Jones. 2013. An LDA- smoothed relevance model for document expansion: a case study for spoken document retrieval. InSIGIR. ACM, 1057–1060

    2013

  20. [20]

    Debasis Ganguly, Dwaipayan Roy, Mandar Mitra, and Gareth J.F. Jones. 2015. Word Embedding based Generalized Language Model for Information Retrieval. InProceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval(Santiago, Chile)(SIGIR ’15). Association for Computing Machinery, New York, NY, USA, 795–798....

    doi:10.1145/2766462 2015

  21. [21]

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer Feed-Forward Layers Are Key-Value Memories. InProceedings of the 2021 Con- ference on Empirical Methods in Natural Language Processing, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, Online and Punta Cana...

    doi:10.18653/v1/2021.emnlp-main.446 2021

  22. [22]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang

  23. [23]

    Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. InProceedings of the 26th International Conference on World Wide Web(Perth, Australia)(WWW ’17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 173–182. doi:10.1145/3038912.3052569

    arXiv 2017

  24. [24]

    Djoerd Hiemstra. 2001. Using language models for information retrieval. (2001)

    2001

  25. [25]

    Thomas Hofmann. 1999. Probabilistic latent semantic analysis. InProceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence(Stockholm, Sweden)(UAI’99). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 289–296

    1999

  26. [26]

    Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation.CoRRabs/2010.02666 (2020). arXiv:2010.02666 https://arxiv.org/abs/2010.02666

    Pith/arXiv arXiv 2020

  27. [27]

    Ingwersen

    P. Ingwersen. 1996. Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory.J. Documentation52 (1996), 3–50. https: //api.semanticscholar.org/CorpusID:11877851

    1996

  28. [28]

    Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques.ACM Trans. Inf. Syst.20, 4 (2002), 422–446. doi:10.1145/582415. 582418

    doi:10.1145/582415 2002

  29. [29]

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516(2025)

    Pith/arXiv arXiv 2025

  30. [30]

    Jeff Johnson, Matthijs Douze, and Herve Jegou. 2021. Billion-Scale Similarity Search with GPUs .IEEE Transactions on Big Data7, 03 (July 2021), 535–547. doi:10.1109/TBDATA.2019.2921572

    arXiv 2021

  31. [31]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, Bonnie Webber, Trevor Cohn, Yulan He, a...

    doi:10.18653/v1/2020.emnlp- 2020

  32. [32]

    Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. InProceedings of the 43rd International ACM SIGIR conference on research and development in Information Re- trieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, Jimmy Huang, Yi Chang, Xueqi Cheng, Jaap Kamps, Vanessa...

    arXiv 2020

  33. [33]

    Kleinberg

    Jon M. Kleinberg. 1999. Authoritative sources in a hyperlinked environment.J. ACM46, 5 (Sept. 1999), 604–632. doi:10.1145/324133.324140

    arXiv 1999

  34. [34]

    Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix Factorization Tech- niques for Recommender Systems.Computer42, 8 (2009), 30–37. doi:10.1109/ MC.2009.263

    2009

  35. [35]

    Kschischang, B.J

    F.R. Kschischang, B.J. Frey, and H.-A. Loeliger. 2001. Factor graphs and the sum- product algorithm.IEEE Transactions on Information Theory47, 2 (2001), 498–519. doi:10.1109/18.910572

    doi:10.1109/18.910572 2001

  36. [36]

    John Lafferty and Chengxiang Zhai. 2001. Document language models, query models, and risk minimization for information retrieval. InProceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval(New Orleans, Louisiana, USA)(SIGIR ’01). Association for Computing Machinery, New York, NY, USA, 111–119. ...

    arXiv 2001

  37. [37]

    Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.ArXivabs/2005.11401 (2020). https://api.semanticscholar.org/CorpusID:218869575

    Pith/arXiv arXiv 2020

  38. [38]

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic Search-Enhanced Large Reasoning Models.CoRRabs/2501.05366 (2025). arXiv:2501.05366 doi:10.48550/ ARXIV.2501.05366

    Pith/arXiv arXiv doi:10.48550/arxiv.2501.05366 2025

  39. [39]

    Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2020. Distilling Dense Rep- resentations for Ranking using Tightly-Coupled Teachers.CoRRabs/2010.11386 (2020). arXiv:2010.11386 https://arxiv.org/abs/2010.11386

    Pith/arXiv arXiv 2020

  40. [40]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts.Transactions of the Association for Computational Linguistics 12 (2024), 157–173. doi:10.1162/tacl_a_00638

    doi:10.1162/tacl_a_00638 2024

  41. [41]

    Tie-Yan Liu. 2009. Learning to Rank for Information Retrieval.Found. Trends Inf. Retr.3, 3 (March 2009), 225–331. doi:10.1561/1500000016

    doi:10.1561/1500000016 2009

  42. [42]

    Sean MacAvaney, Andrew Yates, Kai Hui, and Ophir Frieder. 2019. Content- Based Weak Supervision for Ad-Hoc Re-Ranking. InProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, Benjamin Piwowarski, Max Chevalier, Éric Gaussier, Yoelle Maarek, Jian-Yun Nie...

    arXiv 2019

  43. [43]

    Craig Macdonald and Nicola Tonellotto. 2020. Declarative Experimentation inInformation Retrieval using PyTerrier. InProceedings of ICTIR 2020

    2020

  44. [44]

    Malkov and D

    Yu A. Malkov and D. A. Yashunin. 2020. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE Trans. Pattern Anal. Mach. Intell.42, 4 (April 2020), 824–836. doi:10.1109/ TPAMI.2018.2889473

    arXiv 2020

  45. [45]

    Stefano Mizzaro. 1997. Relevance: The whole history.Journal of the American Society for Information Science48, 9 (1997), 810–832. doi:10.1002/(SICI)1097- 4571(199709)48:9<810::AID-ASI6>3.0.CO;2-U

    doi:10.1002/(sici)1097- 1997

  46. [46]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Ma- jumder, and Li Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. InProceedings of the Workshop on Cognitive Computa- tion: Integrating neural and symbolic approaches 2016 co-located with the 30th An- nual Conference on Neural Information Processing S...

    2016

  47. [47]

    Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage Re-ranking with BERT. CoRRabs/1901.04085 (2019). arXiv:1901.04085 http://arxiv.org/abs/1901.04085

    Pith/arXiv arXiv 2019

  48. [48]

    Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. Docu- ment Ranking with a Pretrained Sequence-to-Sequence Model. InFindings of the Association for Computational Linguistics: EMNLP 2020, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 708–718. doi:10.18653/v1/2020.findings-emnlp.63

    doi:10.18653/v1/2020.findings-emnlp.63 2020

  49. [49]

    Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document Expansion by Query Prediction.CoRRabs/1904.08375 (2019). arXiv:1904.08375 http://arxiv.org/abs/1904.08375

    Pith/arXiv arXiv 2019

  50. [50]

    Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank Citation Ranking : Bringing Order to the Web. InThe Web Conference. https://api.semanticscholar.org/CorpusID:1508503 SIGIR ’26, July 20–24, 2026, Melbourne, VIC, Australia. Andrew Parry, Emmanouil Georgios Lionis, Debasis Ganguly, & Sean MacAvaney

    1999

  51. [51]

    Andrew Parry, Maik Fröbe, Harrisen Scells, Ferdinand Schlatt, Guglielmo Faggioli, Saber Zerhoudi, Sean MacAvaney, and Eugene Yang. 2025. Variations in Relevance Judgments and the Shelf Life of Test Collections. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval(Padua, Italy)(SIGIR ’25). Associ...

    arXiv 2025

  52. [52]

    Andrew Parry, Sean MacAvaney, and Debasis Ganguly. 2024. Top-Down Partitioning for Efficient List-Wise Ranking. arXiv:2405.14589 [cs.IR] https: //arxiv.org/abs/2405.14589

    Pith/arXiv arXiv 2024

  53. [53]

    Gabriella Pasi, Gareth J. F. Jones, Keith Curtis, Stefania Marrara, Camilla Sanvitto, Debasis Ganguly, and Procheta Sen. 2018. Evaluation of Personalised Information Retrieval at CLEF 2018 (PIR-CLEF). InExperimental IR Meets Multilinguality, Mul- timodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, A vignon, Franc...

    2018

  54. [54]

    Jie Peng, Craig Macdonald, and Iadh Ounis. 2008. Automatic document prior feature selection for web retrieval. InProceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’08). 761–762

    2008

  55. [55]

    Ponte and W

    Jay M. Ponte and W. Bruce Croft. 1998. A language modeling approach to information retrieval. InProceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval(Melbourne, Australia)(SIGIR ’98). Association for Computing Machinery, New York, NY, USA, 275–281. doi:10.1145/290941.291008

    arXiv 1998

  56. [56]

    Ronak Pradeep, Rodrigo Frassetto Nogueira, and Jimmy Lin. 2021. The Expando- Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to- Sequence Models.CoRRabs/2101.05667 (2021). arXiv:2101.05667 https://arxiv. org/abs/2101.05667

    Pith/arXiv arXiv 2021

  57. [57]

    Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023. RankZephyr: Ef- fective and Robust Zero-Shot Listwise Reranking is a Breeze!CoRRabs/2312.02724 (2023). arXiv:2312.02724 doi:10.48550/ARXIV.2312.02724

    Pith/arXiv arXiv doi:10.48550/arxiv.2312.02724 2023

  58. [58]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer.J. Mach. Learn. Res.21, 1, Article 140 (Jan. 2020), 67 pages

    2020

  59. [59]

    Stephen Robertson. 1977. The Probability Ranking Principle in IR.Journal of Documentation33 (12 1977), 294–304. doi:10.1108/eb026647

    doi:10.1108/eb026647 1977

  60. [60]

    Robertson and Karen Spärck Jones

    Stephen E. Robertson and Karen Spärck Jones. 1976. Relevance weighting of search terms.J. Am. Soc. Inf. Sci.27, 3 (1976), 129–146. doi:10.1002/ASI.4630270302

    work page doi:10.1002/asi.4630270302 1976

  61. [61]

    Robertson, Steve Walker, Micheline Hancock-Beaulieu, Mike Gatford, and A

    Stephen E. Robertson, Steve Walker, Micheline Hancock-Beaulieu, Mike Gatford, and A. Payne. 1995. Okapi at TREC-4. InProceedings of The Fourth Text REtrieval Conference, TREC 1995, Gaithersburg, Maryland, USA, November 1-3, 1995 (NIST Special Publication, Vol. 500-236), Donna K. Harman (Ed.). National Institute of Standards and Technology (NIST). http://t...

    1995

  62. [62]

    Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024. LaMP: When Large Language Models Meet Personalization. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thail...

    doi:10.18653/v1/ 2024

  63. [63]

    Tefko Saracevic. 2007. Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance.J. Assoc. Inf. Sci. Technol.58, 13 (2007), 1915–1933. doi:10.1002/ASI. 20682

    doi:10.1002/asi 2007

  64. [64]

    Ferdinand Schlatt, Maik Fröbe, Harrisen Scells, Shengyao Zhuang, Bevan Koop- man, Guido Zuccon, Benno Stein, Martin Potthast, and Matthias Hagen. 2024. Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders.arXiv preprint arXiv:2404.06912(2024)

    Pith/arXiv arXiv 2024

  65. [65]

    Ferdinand Schlatt, Maik Fröbe, Harrisen Scells, Shengyao Zhuang, Bevan Koop- man, Guido Zuccon, Benno Stein, Martin Potthast, and Matthias Hagen. 2024. A Systematic Investigation of Distilling Large Language Models into Cross- Encoders for Passage Re-ranking. arXiv:2405.07920 [cs.IR] https://arxiv.org/abs/ 2405.07920

    Pith/arXiv arXiv 2024

  66. [66]

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. InProceedings of the 40th International Conference on Machine Learning(Honolulu, Hawaii, USA)(ICML’23). JMLR.org, Article 1291, 18 pages

    2023

  67. [67]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent.CoRRabs/2304.09542 (2023). arXiv:2304.09542 doi:10.48550/ ARXIV.2304.09542

    Pith/arXiv arXiv doi:10.48550/arxiv.2304.09542 2023

  68. [68]

    Fangzheng Tian, Debasis Ganguly, and Craig Macdonald. 2025. Is Relevance Propagated from Retriever to Generator in RAG?. InECIR (1) (Lecture Notes in Computer Science). Springer, 32–48

    2025

  69. [69]

    Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Ra- sul, Younes Belkada, Shengyi Huang, Leandro Von Werra, Clémentine Fourrier, Nathan Habib, et al. 2023. Zephyr: Direct distillation of lm alignment.arXiv preprint arXiv:2310.16944(2023)

    Pith/arXiv arXiv 2023

  70. [70]

    Lidan Wang, Jimmy Lin, and Donald Metzler. 2011. A cascade ranking model for efficient ranked retrieval. InProceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval(Beijing, China) (SIGIR ’11). Association for Computing Machinery, New York, NY, USA, 105–114. doi:10.1145/2009916.2009934

    arXiv 2011

  71. [71]

    Bruce Croft

    Xing Wei and W. Bruce Croft. 2006. LDA-based document models for ad- hoc retrieval.Proceedings of the 29th annual international ACM SIGIR con- ference on Research and development in information retrieval(2006). https: //api.semanticscholar.org/CorpusID:3343003

    2006

  72. [72]

    Bruce Croft, Erik Learned-Miller, and Jaap Kamps

    Hamed Zamani, Mostafa Dehghani, W. Bruce Croft, Erik Learned-Miller, and Jaap Kamps. 2018. From Neural Re-Ranking to Neural Ranking: Learning a Sparse Representation for Inverted Indexing. InProceedings of the 27th ACM International Conference on Information and Knowledge Management(Torino, Italy)(CIKM ’18). Association for Computing Machinery, New York, ...

    arXiv 2018

  73. [73]

    Chengxiang Zhai and John Lafferty. 2004. A study of smoothing methods for lan- guage models applied to information retrieval.ACM Transactions on Information Systems (TOIS)22, 2 (2004), 179–214

    2004

  74. [2020]

    InProceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol

    Retrieval Augmented Language Model Pre-Training. InProceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119). PMLR, 3929–3938. http://proceedings.mlr.press/v119/guu20a.html

    2020

  75. [2022]

    InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval(Madrid, Spain) (SIGIR ’22)

    From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective. InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval(Madrid, Spain) (SIGIR ’22). Association for Computing Machinery, New York, NY, USA, 2353–

  76. [2359]

    doi:10.1145/3477495.3531857

    arXiv