Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information

Chen Xu; Jintao Tang; Kehan Long; Shasha Li; Ting Wang

REVIEW 4 major objections 5 minor 46 references

Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Zero-shot pointwise LLM ranking can be made substantially more accurate without losing efficiency by comparing every candidate document against a single automatically built anchor summary and linearly combining the contrastive scores with…

desk verdict A useful, mostly solid pointwise reranking paper whose main attribution claim is undercut by a missing closest baseline; worth refereeing carefully. read the letter →

arxiv 2506.10859 v1 pith:HGISXMPE submitted 2025-06-12 cs.IR cs.AI

Kehan Long , Shasha Li , Chen Xu , Jintao Tang , Ting Wang This is my paper

classification cs.IRcs.AI MSC 68P2068T50

keywords LLMsforZero-ShotRankingPointwiseGlobalContextRankAggregationAnchorDocumentSpectralMulti-DocumentSummarizationContrastiveRelevanceScoringReranking

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Zero-shot pointwise LLM rankers score each document in isolation, which makes their relevance judgments inconsistent and leaves comparative information on the table. This paper claims that a single anchor document—an unsupervised, query-focused summary of the top candidate set—can supply that missing global context without switching to pairwise or listwise prompting. The proposed GCCP strategy scores every candidate against the anchor and then linearly averages those contrastive scores with existing pointwise scores, a training-free post-aggregation the authors call PAGC. Across TREC DL and BEIR, adding this anchor consistently beats the underlying pointwise methods at nearly the same efficiency, and the best combinations stay competitive with far more expensive comparative approaches.

What carries the argument

The load-bearing object is the anchor document $d_a$: a short, query-focused, globally representative summary of the candidate set, built without training or generation. Construction proceeds from the top-$m$ pseudo-relevant documents, a sentence graph with affinity matrix $A$ of TF-IDF cosine similarities thresholded by $\theta$ (Eq. 5), the normalized Laplacian $L = I - D^{-1/2} A D^{-1/2}$ (Eq. 6), and the Fiedler vector $\mathbf{v}_2$, the eigenvector for the second-smallest eigenvalue, whose sign partitions the sentences into a core cluster and a periphery (Eqs. 7-8). The first $z$ sentences of the larger cluster, reordered by original position, become the anchor (Eq. 9). The anchor does the work of injecting global context: every candidate is compared to the same reference, and Eq. 11's linear average lets that signal be grafted onto any existing pointwise scorer at negligible extra cost.

What would settle it

Rebuild the anchor from the top-50 or top-100 BM25 documents instead of the top-10 used in the paper: if a large fraction of queries show substantially changed final rankings, the anchor is not capturing a stable global structure. Alternatively, replace the Fiedler-vector cluster with a random subset of sentences of the same size; if the reported gains largely vanish, the specific spectral construction is what carries the result rather than mere content coverage.

Watch

Extended reading notes

Core claim

The paper's central claim is that independent per-document scoring, not the LLM itself, is what caps pointwise ranker quality, and that this can be fixed while keeping pointwise cost. GCCP constructs an anchor document $d_a$ by spectral multi-document summarization: the top-$m$ BM25 documents are split into sentences, a graph is built from thresholded TF-IDF cosine similarities (Eq. 5), and the Fiedler vector of the normalized Laplacian separates the sentences into a central cluster and a peripheral one; the larger cluster's first $z$ sentences, in original order, form the anchor (Eq. 9). Each candidate $d_i$ is then scored against this anchor through a standard pairwise prompt, giving a contrastive score (Eq. 10). PAGC combines these contrastive scores with any existing pointwise scores by the linear average in Eq. 11, reusing the $O(N)$ parallel inference pattern of pointwise methods. The experiments with three open-source instruction-tuned LLM sizes report that every pointwise baseline improves when post-aggregated with GCCP and that the best PAGC variants are competitive with, and in several configurations superior to, pairwise and listwise baselines while using far fewer LLM calls.

Load-bearing premise

The method assumes that the automatically built summary used as the reference point is a fair, query-focused representative of the whole candidate set, so that comparing each document against that single reference produces consistent and meaningful scores; if the reference is biased or noisy, the bias propagates to every score.

Editorial extensions

If this is right

Any existing pointwise ranker (QG, RG-YN, RG-S) can be improved by averaging its scores with GCCP contrastive scores, with no retraining and no change to its inference pattern.
The method preserves $O(N)$ LLM calls and pointwise-level latency, with theoretical parallel time complexity $O(1)$, unlike comparative methods whose serial passes are $O(N)$ or $O(\log N)$.
On TREC DL and BEIR, PAGC configurations are competitive with pairwise and listwise baselines, and in several settings outperform them, while using roughly 200 to 300 LLM calls per query instead of up to 9,900.
Heterogeneous aggregation—mixing independent pointwise scores with contrastive anchor scores—consistently beats aggregating multiple instances of the same scoring type, indicating that the global-context signal is complementary rather than redundant.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper's setup, the same post-aggregation should transfer to supervised pointwise rankers, where an anchor-based contrastive score might correct calibration errors without retraining.
A natural diagnostic the authors leave implicit is per-query anchor quality: measuring how well the larger spectral cluster captures relevant content would show when the method can be trusted and when it should be skipped.
Because the final aggregation is a simple linear average, PAGC can be combined with other zero-shot ranking post-processing tricks, so the reported gains may stack with further refinements.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper is a legitimate within-subfield advance. The new configuration—building a query-focused anchor via unsupervised spectral MDS and using it for pairwise contrastive scores that are linearly aggregated with pointwise scores—is not something I have seen in the cited literature. The experiments are broad (TREC DL and eight BEIR sets, three Flan backbones), the gains over the included pointwise baselines are consistent, and the efficiency/cost analysis is honest and helpful. The code is linked. That is real evidence and should count in its favor.

The soft spots are real but mostly addressable. The biggest one is the missing closest baseline. The related work explicitly cites Yan et al. 2024 for using pairwise comparisons as a post-processing step on pointwise scores, and that method never appears in any table. Without it, the claimed gains over pointwise baselines could come from generic pairwise post-processing rather than from the global-context anchor. The anchor ablations in Table 5 do not settle it: Spectral vs Top differs by 0.001 on TREC DL for PAGC-QSG, and the BEIR differences, while larger, come with no significance tests. So the central attribution—that the spectral anchor is what matters—is not yet established. This is a load-bearing weakness, but it is fixable by adding the baseline.

Smaller issues: theta in Eq. 5 is never reported; m and z are fixed at 10 after tuning on these benchmarks; no variance or error bars anywhere, despite paired t-tests; no multiple-comparison correction; and the GCCP x2 and GCCP x3 homogeneous comparisons in Table 4 use different m values, which is a bit apples-to-oranges. None of these sink the method, but they are exactly the kind of thing a referee should ask to be cleaned up.

The theoretical efficiency analysis is fine, though the O(1) time complexity claim is just parallel pointwise inference, which is not new. The paper is honest about that.

Bottom line: read it if you work on LLM reranking; it is a solid practical contribution with a clear mechanism. The missing baseline needs to be added before the central claim is fully convincing, and the tuning/reporting issues should be tightened. Definitely deserves peer review rather than desk rejection.

Referee Report

4 major / 5 minor

Summary. The paper proposes a zero-shot LLM reranking strategy, GCCP, in which an anchor document constructed by unsupervised spectral multi-document summarization (Eqs. 5–9) is used as a single global reference for pointwise comparisons against each candidate (Eq. 10). These contrastive scores are then linearly aggregated with conventional pointwise scores such as QG, RG-YN, and RG-S(0,4) (Eq. 11), yielding the PAGC framework. The authors evaluate on TREC DL 2019/2020 and eight BEIR sets with Flan-t5-large, Flan-t5-xl, and Flan-ul2, reporting that PAGC outperforms previous pointwise methods at similar efficiency and remains competitive with pairwise/listwise methods while using fewer LLM calls.

Significance. The proposed method is simple, training-free, and plausibly practical: it adds a small number of LLM calls to a pointwise pipeline, and the code is released. The evaluation is broad and includes three backbone LLMs, two TREC DL test sets, eight BEIR datasets, a complexity comparison in Table 3, and an aggregation analysis in Table 4. If the central claim holds, PAGC offers a useful accuracy–efficiency middle ground for zero-shot reranking. However, the significance is conditional: the closest existing post-processing baseline is not compared, the advantage of the spectral anchor over simpler anchors is numerically small and not tested for significance, and the key hyperparameters are selected on the evaluation test sets without an independent validation procedure.

major comments (4)

[§2.2, §4.3.1, Tables 1–2] The related work (Section 2.2) explicitly identifies Yan et al. [39] as a method that uses pairwise comparisons to post-process pointwise relevance scores, which is the same mechanism as GCCP/PAGC. Yet [39] never appears in any comparison table, figure, or ablation. Without this baseline, the observed gains over QG, RG-YN, and RG-S(0,4) cannot be attributed to the global-context anchor rather than to generic pairwise post-processing; Table 5 does not resolve this because spectral and Top-anchor variants differ by as little as 0.001 (0.6703 vs. 0.6693 on TREC DL for PAGC-QSG). Including [39] under the same backbone and prompt settings is necessary to support the paper's central claim of a new pointwise state of the art.
[§4.2, §4.4.2, Eq. (5)] The hyperparameters m and z are fixed to 10 in Section 4.2, but Section 4.4.2 shows that m=10 is chosen because it is 'almost optimal' on the very TREC DL and BEIR test sets, and no z sensitivity study is reported. In addition, the threshold theta in Eq. (5) is never given a value or a sensitivity analysis. This test-set selection weakens the claim of significant improvement over prior pointwise methods. The authors should add a validation-based selection protocol, report theta, and provide error bars or confidence intervals for at least the main tables.
[§4.4.4, Table 5] The anchor-strategy comparison does not establish that the spectral MDS anchor is superior. On TREC DL, GCCP-Spectral (0.6076) is actually below GCCP-Top (0.6099), and PAGC-QSG Spectral (0.6703) exceeds Top (0.6693) by only 0.001; on BEIR the largest spectral-vs-random gap shown is 0.0124 (0.4740 vs. 0.4616), still without a significance test. The text claims 'consistently strong performance' and 'strategic anchor construction,' but the data support only a weak preference for Spectral at best. Significance tests (or per-query paired comparisons) and variance estimates are needed before the contribution can be attributed to the anchor.
[§3.2.2, Eqs. (5)–(9)] The construction of the anchor assumes that the larger cluster from the Fiedler-vector partition contains the key shared information and that truncation to z sentences preserves query focus and content coverage. No validation of these properties is provided: there are no qualitative examples of anchors, no measurement of anchor relevance to the query, and no discussion of the sensitivity of the partition to the threshold theta. Since every contrastive score in Eq. (10) inherits the anchor, this is a correctness-risk point. I suggest reporting the theta value, adding anchor-quality diagnostics, and testing whether a query-independent 'generic language' cluster could dominate the larger cluster on some datasets.

minor comments (5)

[§3.2.2, Eq. (5)] Equation (5) introduces n for the number of sentences, while N is used for documents in Table 3; please disambiguate the notation.
[§4.3.4, Figure 3] Figure 3 and the surrounding text compute cost and latency using GPT-4-o pricing, while all headline experiments use open-source Flan models; clarify whether these are simulated costs or measured values and how they transfer to the actual models.
[§4.4.1, Figure 4] The aggregation-method comparison reports nearly identical NDCG@10 values without error bars; a sentence explaining the variance or the per-query consistency would help the reader assess the 'robustness' claim.
[Table 2, §4.3.2] PAGC-QSG is described as aggregating QG, RG-S(0,4), and GCCP, but Table 1 does not show this three-way combination directly; adding an explicit row or equation would make the relationship between the tables easier to verify.
[§4.3.3, Table 3] The statement that GCCP and PAGC have O(1) time complexity refers to parallel time, whereas the same table lists O(N) and O(2N)/O(3N) LLM calls; spelling out this distinction would prevent a misleading reading of the complexity column.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the contrastive scores are independent LLM judgments against an unsupervised anchor, and the aggregation equation is a plain average, not a fit.

full rationale

The derivation chain is self-contained and does not reduce to its own inputs. The contrastive score f_c in Eq. 10 is an independent LLM judgment comparing each candidate with an anchor document; the anchor itself is constructed by an unsupervised spectral multi-document summarization procedure (Eqs. 5-9) over BM25 top-m documents, with no use of relevance labels or of the final ranking target. The post-aggregation in Eq. 11 is an unweighted linear average of the contrastive score with existing pointwise scores, so no fitted parameter is renamed as a prediction and no equation is definitionally equivalent to another. The anchor-strategy ablation in Table 5 compares Spectral, Top, Random, and Synthetic anchors; this is an empirical ablation, not a fitted input. The choice m=10 and z=10 is tuned on the evaluation benchmarks, which weakens the zero-shot framing but is not circularity under the stated definitions. The related-work passage citing Yan et al. [39] as using pairwise comparison to post-process pointwise scores identifies a potentially important omitted baseline, and its absence from the comparison tables is a correctness/comparison risk rather than a circularity: the paper does not claim to derive [39]'s results, and its central claim is not justified by a self-citation. No load-bearing self-citation, imported uniqueness theorem, or ansatz smuggled via citation appears in the paper. Hence the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central method rests on several heuristic choices: the BM25 pool, the spectral clustering anchor, the pairwise probability as a score, and the linear aggregation. The paper does not justify these from first principles and tunes some of them on the evaluation data, so they are recorded as axioms and free parameters rather than derived facts.

free parameters (3)

m (number of BM25 candidates in MDS input) = 10
Hand-set in Section 4.2; Section 4.4.2 shows m=10 is near-optimal on the test benchmarks, indicating it was selected using the evaluation data.
z (number of sentences in the anchor) = 10
Hand-set in Section 4.2; no sensitivity analysis is provided for z, yet it directly controls anchor length and content.
theta (cosine threshold in affinity matrix) = not reported
Threshold in Eq. 5 controls graph sparsity and clustering; the value is never specified, so the anchor construction is not fully reproducible as stated.

assumptions (4)

domain assumption BM25 top-100 pooling contains enough relevant documents for reranking
Used to generate candidate sets and to filter top-m documents for multi-document summarization in Section 4.2. If relevant documents are absent from the pool, no reranker can recover them.
domain assumption The Fiedler-vector partition separates common from peripheral content, so the larger cluster represents the global context
Invoked in Section 3.2.2 and Eqs. 7-8 for anchor generation. This is a heuristic from spectral clustering, not proven for TF-IDF sentence graphs.
domain assumption The probability that the LLM outputs the candidate rather than the anchor is a comparable, query-focused relevance score across candidates
Eq. 10 defines the contrastive score this way using a pairwise prompt; pairwise outputs are not guaranteed transitive or calibrated across candidates.
domain assumption TF-IDF embeddings of sentences capture semantic similarity well enough for the affinity matrix
Eq. 5 builds edges from TF-IDF cosine similarity. This is a traditional bag-of-words assumption and may miss paraphrastic or long-range semantic relations.

invented entities (1)

Anchor document (d_a)
purpose: Global reference point for contrastive scoring; every candidate is compared against it
Constructed by spectral multi-document summarization, not an externally observable entity. Its usefulness is only demonstrated by benchmark results, so it carries no falsifiable prediction outside the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information." pith.science (2026). https://pith.science/paper/HGISXMPE

@misc{pith2026250610859,
  author       = {Pith},
  title        = {Pith review of: Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HGISXMPE}},
  note         = {Machine review of arXiv:2506.10859}
}

read the original abstract

Recent advancements have successfully harnessed the power of Large Language Models (LLMs) for zero-shot document ranking, exploring a variety of prompting strategies. Comparative approaches like pairwise and listwise achieve high effectiveness but are computationally intensive and thus less practical for larger-scale applications. Scoring-based pointwise approaches exhibit superior efficiency by independently and simultaneously generating the relevance scores for each candidate document. However, this independence ignores critical comparative insights between documents, resulting in inconsistent scoring and suboptimal performance. In this paper, we aim to improve the effectiveness of pointwise methods while preserving their efficiency through two key innovations: (1) We propose a novel Global-Consistent Comparative Pointwise Ranking (GCCP) strategy that incorporates global reference comparisons between each candidate and an anchor document to generate contrastive relevance scores. We strategically design the anchor document as a query-focused summary of pseudo-relevant candidates, which serves as an effective reference point by capturing the global context for document comparison. (2) These contrastive relevance scores can be efficiently Post-Aggregated with existing pointwise methods, seamlessly integrating essential Global Context information in a training-free manner (PAGC). Extensive experiments on the TREC DL and BEIR benchmark demonstrate that our approach significantly outperforms previous pointwise methods while maintaining comparable efficiency. Our method also achieves competitive performance against comparative methods that require substantially more computational resources. More analyses further validate the efficacy of our anchor construction strategy.

Figures

Figures reproduced from arXiv: 2506.10859 by the authors.

**Figure 1.** Illustration of different prompting strategies. Comparative approaches include (a) Pairwise and (b) Listwise. Pointwise [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** The overall framework of our proposed pointwise ranking with Post-Aggregated Global Context information (PAGC), [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Relationship between Cost / Latency and NDCG@10 on the TREC DL benchmark. (a) shows the trade-off between [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Performance results of GCCP and PAGC with dif [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 22 canonical work pages

[39]

Le Yan, Zhen Qin, Honglei Zhuang, Rolf Jagerman, Xuanhui Wang, Michael Bendersky, and Harrie Oosterhuis. 2024. Consolidating Ranking and Relevance Predictions of Large Language Models through Post-Processing. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, Yaser Al...

work page 2024
[1]

Leonidas Akritidis, Athanasios Fevgas, Panayiotis Bozanis, and Yannis Manolopoulos. 2022. An unsupervised distance-based model for weighted rank aggregation with list pruning.Expert Syst. Appl.202 (2022), 117435. doi:10.1016/J.ESWA.2022.117435

work page arXiv 2022
[2]

Yiqun Chen, Qi Liu, Yi Zhang, Weiwei Sun, Daiting Shi, Jiaxin Mao, and Dawei Yin. 2024. TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy.CoRRabs/2406.11678 (2024). doi:10.48550/ ARXIV.2406.11678 arXiv:2406.11678

arXiv doi:10.48550/arxiv.2406.11678 2024
[3]

Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2021. Overview of the TREC 2020 deep learning track.CoRRabs/2102.07662 (2021). arXiv:2102.07662 https://arxiv.org/abs/2102.07662

arXiv 2021
[4]

Voorhees

Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track.CoRR abs/2003.07820 (2020). arXiv:2003.07820 https://arxiv.org/abs/2003.07820

arXiv 2020
[5]

Sunhao Dai, Yuqi Zhou, Liang Pang, Weihao Liu, Xiaolin Hu, Yong Liu, Xiao Zhang, Gang Wang, and Jun Xu. 2024. Neural Retrievers are Biased Towards LLM-Generated Content. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024, Ricardo Baeza-Yates and Francesco Bonchi (Eds.). AC...

arXiv 2024
[6]

Rob Deconde, Sarah Hawley, Seth Falcon, Nigel Clegg, Beatrice Knudsen, and Ruth Etzioni. 2006. Combining Results of Microarray Experiments: A Rank Aggregation Approach.Statistical Applications in Genetics and Molecular Biology 5 (2006), 1–25. https://api.semanticscholar.org/CorpusID:117085811

work page 2006
[7]

Guglielmo Faggioli, Laura Dietz, Charles L. A. Clarke, Gianluca Demartini, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, and Henning Wachsmuth. 2023. Perspectives on Large Language Models for Relevance Judgment. InProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval,...

arXiv 2023

Show all 46 references

[8]

Mohamed Farah and Daniel Vanderpooten. 2007. An outranking approach for rank aggregation in information retrieval. InSIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, ...

2007
[9]

Miroslav Fiedler. 1975. A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory.Czechoslovak Mathematical Journal 25 (1975), 619–633. https://api.semanticscholar.org/CorpusID:123006817

1975
[10]

Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. Precise Zero-Shot Dense Retrieval without Relevance Labels. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, ...

2023 doi
[11]

Fang Guo, Wenyu Li, Honglei Zhuang, Yun Luo, Yafu Li, Le Yan, and Yue Zhang
[12]

Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bo- janowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised Dense Infor- mation Retrieval with Contrastive Learning.Trans. Mach. Learn. Res.(2022)

2022
[13]

Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques.ACM Trans. Inf. Syst.20, 4 (2002), 422–446. doi:10.1145/582415. 582418

2002 doi
[15]

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large Language Models are Zero-Shot Reasoners. InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Informa- tion Processing Systems 2022, NeurIPS 2022, New ...

2022
[16]

Manning, Christopher Ré, Diana Acosta-Navas, Drew A

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michi- hiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Nav...

2023
[17]

Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Frassetto Nogueira. 2021. Pyserini: A Python Toolkit for Repro- ducible Information Retrieval Research with Sparse and Dense Representations. InSIGIR ’21: The 44th International ACM SIGIR Conf...

2021
[18]

Jian Luo, Xuanang Chen, Ben He, and Le Sun. 2024. PRP-Graph: Pairwise Ranking Prompting to LLMs with Graph Aggregation for Effective Text Re-ranking. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, B...

2024 doi
[19]

Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. 2023. Zero-Shot List- wise Document Reranking with a Large Language Model.CoRRabs/2305.02156 (2023). doi:10.48550/ARXIV.2305.02156 arXiv:2305.02156

2023 doi arXiv
[20]

Rodrigo Frassetto Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. Document Ranking with a Pretrained Sequence-to-Sequence Model. InFindings of the Association for Computational Linguistics: EMNLP. 708–718

2020
[21]

Andrew Parry, Sean MacAvaney, and Debasis Ganguly. 2024. Top-Down Partitioning for Efficient List-Wise Ranking.CoRRabs/2405.14589 (2024). doi:10.48550/ARXIV.2405.14589 arXiv:2405.14589

2024 doi arXiv
[22]

Ponte and W

Jay M. Ponte and W. Bruce Croft. 1998. A Language Modeling Approach to Information Retrieval. InSIGIR ’98: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 24-28 1998, Melbourne, Australia, W. Bruce ...

1998
[23]

Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023. RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models.ArXivabs/2309.15088 (2023)

2023 arXiv
[24]

Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Ben- dersky. 2024. Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting. InFindings of the Association...

2024
[25]

Arafat Sultan, Deevya Swain, Avirup Sil, and Heng Ji

Revanth Gangi Reddy, JaeHyeok Doo, Yifei Xu, Md. Arafat Sultan, Deevya Swain, Avirup Sil, and Heng Ji. 2024. FIRST: Faster Improved Listwise Reranking with Single Token Decoding. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 20...

2024
[26]

Elena Renda and Umberto Straccia

M. Elena Renda and Umberto Straccia. 2003. Web Metasearch: Rank vs. Score Based Rank Aggregation Methods. InProceedings of the 2003 ACM Symposium on Applied Computing (SAC), March 9-12, 2003, Melbourne, FL, USA, Gary B. Lamont, Hisham Haddad, George A. Papadopoulos, and Brajen...

2003
[27]

Saari and Vincent Merlin

Donald G. Saari and Vincent Merlin. 1996. The Copeland method.Economic Theory8 (1996), 51–76. https://api.semanticscholar.org/CorpusID:153454330

1996
[28]

Devendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, and Luke Zettlemoyer. 2022. Improving Passage Retrieval with Zero-Shot Question Generation. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, ...

2022 doi
[29]

Shuoqi Sun, Shengyao Zhuang, Shuai Wang, and Guido Zuccon. 2024. An Investigation of Prompt Variations for Zero-shot LLM-based Rankers.CoRR abs/2406.14117 (2024). doi:10.48550/ARXIV.2406.14117 arXiv:2406.14117

2024 doi arXiv
[30]

Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? Investi- gating Large Language Models as Re-Ranking Agents. InEMNLP. 14918–14937

2023
[31]

Raphael Tang, Xinyu Crystina Zhang, Xueguang Ma, Jimmy Lin, and Ferhan Ture. 2024. Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Co...

2024
[32]

Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby, and Donald Metzler

Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby, and Donald Metzler. 2023. UL2: Unifying Language Learning Paradigms. InThe Eleventh International Conferenc...

2023
[33]

Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

2021
[35]

Kexiang Wang, Baobao Chang, and Zhifang Sui. 2020. A Spectral Method for Unsu- pervised Multi-Document Summarization. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, Bonnie Webber, Trevor Cohn,...

2020 doi
[36]

Shuai Wang, Harrisen Scells, Bevan Koopman, and Guido Zuccon. 2023. Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, T...

2023
[37]

Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M

Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned Language Models are Zero-Shot Learners. InThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 2...

2022
[38]

Bennett, Junaid Ahmed, and Arnold Overwijk

Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. InICLR

2021
[40]

Xiaowei Yuan, Zhao Yang, Yequan Wang, Jun Zhao, and Kang Liu. 2024. Improving Zero-shot LLM Re-Ranker with Risk Minimization. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, Yaser Al-On...

2024
[41]

Jinming Zhao, Ming Liu, Longxiang Gao, Yuan Jin, Lan Du, He Zhao, He Zhang, and Gholamreza Haffari. 2020. SummPip: Unsupervised Multi-Document Summa- rization with Sentence Graph Compression. InProceedings of the 43rd International ACM SIGIR conference on research and developm...

2020
[42]

Hongli Zhou, Zhihao Jia, Haiyang Zhu, and Zhizheng Zhang. 2024. CLLP: Con- trastive Learning Framework Based on Latent Preferences for Next POI Recom- mendation. InProceedings of the 47th International ACM SIGIR Conference on Re- search and Development in Information Retrieval...

2024
[43]

Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chen- long Deng, Zhicheng Dou, and Ji-Rong Wen. 2023. Large Language Models for Information Retrieval: A Survey. (2023). arXiv:2308.07107

2023
[44]

Honglei Zhuang, Zhen Qin, Kai Hui, Junru Wu, Le Yan, Xuanhui Wang, and Michael Bendersky. 2024. Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels. InProceedings of the 2024 Conference of the North American Chapter of the Association f...

2024
[45]

Honglei Zhuang, Zhen Qin, Rolf Jagerman, Kai Hui, Ji Ma, Jing Lu, Jianmo Ni, Xuanhui Wang, and Michael Bendersky. 2023. RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Info...

2023
[46]

Shengyao Zhuang, Bing Liu, Bevan Koopman, and Guido Zuccon. 2023. Open- source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking. InFindings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, Houd...

2023
[47]

Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. 2024. A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models. InSIGIR

2024
[2024]

CoRRabs/2404.11960 (2024)

Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankers. CoRRabs/2404.11960 (2024). doi:10.48550/ARXIV.2404.11960 arXiv:2404.11960

2024 doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [39]

Le Yan, Zhen Qin, Honglei Zhuang, Rolf Jagerman, Xuanhui Wang, Michael Bendersky, and Harrie Oosterhuis. 2024. Consolidating Ranking and Relevance Predictions of Large Language Models through Post-Processing. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, Yaser Al...

work page 2024

[2] [1]

Leonidas Akritidis, Athanasios Fevgas, Panayiotis Bozanis, and Yannis Manolopoulos. 2022. An unsupervised distance-based model for weighted rank aggregation with list pruning.Expert Syst. Appl.202 (2022), 117435. doi:10.1016/J.ESWA.2022.117435

work page arXiv 2022

[3] [2]

Yiqun Chen, Qi Liu, Yi Zhang, Weiwei Sun, Daiting Shi, Jiaxin Mao, and Dawei Yin. 2024. TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy.CoRRabs/2406.11678 (2024). doi:10.48550/ ARXIV.2406.11678 arXiv:2406.11678

arXiv doi:10.48550/arxiv.2406.11678 2024

[4] [3]

Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2021. Overview of the TREC 2020 deep learning track.CoRRabs/2102.07662 (2021). arXiv:2102.07662 https://arxiv.org/abs/2102.07662

arXiv 2021

[5] [4]

Voorhees

Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track.CoRR abs/2003.07820 (2020). arXiv:2003.07820 https://arxiv.org/abs/2003.07820

arXiv 2020

[6] [5]

Sunhao Dai, Yuqi Zhou, Liang Pang, Weihao Liu, Xiaolin Hu, Yong Liu, Xiao Zhang, Gang Wang, and Jun Xu. 2024. Neural Retrievers are Biased Towards LLM-Generated Content. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024, Ricardo Baeza-Yates and Francesco Bonchi (Eds.). AC...

arXiv 2024

[7] [6]

Rob Deconde, Sarah Hawley, Seth Falcon, Nigel Clegg, Beatrice Knudsen, and Ruth Etzioni. 2006. Combining Results of Microarray Experiments: A Rank Aggregation Approach.Statistical Applications in Genetics and Molecular Biology 5 (2006), 1–25. https://api.semanticscholar.org/CorpusID:117085811

work page 2006

[8] [7]

Guglielmo Faggioli, Laura Dietz, Charles L. A. Clarke, Gianluca Demartini, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, and Henning Wachsmuth. 2023. Perspectives on Large Language Models for Relevance Judgment. InProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval,...

arXiv 2023