Pith. sign in

REVIEW 2 major objections 3 minor 86 references

Hypencoder: Hypernetworks for Information Retrieval

T0 review · 2 major / 3 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A retrieval model can score documents with a query-specific neural network instead of an inner product, and it outperforms strong dense retrievers on standard and hard benchmarks.

desk verdict Genuinely novel first-stage retrieval idea with solid empirical support; the efficiency and theoretical claims need scoping before publication. read the letter →

arxiv 2502.05364 v2 pith:NJK6GN7F submitted 2025-02-07 cs.IR cs.LG

Julian Killingback , Hansi Zeng , Hamed Zamani This is my paper · ORCID
classification cs.IRcs.LG
keywords hypernetworksdenseretrievallearnedrelevancefunctionsquery-specificscoringapproximatenearestneighborsearchtip-of-the-tongueinstructionfollowingTRECDeepLearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Hypencoder, a retrieval model that replaces the standard inner-product similarity function with a small neural network generated per query by a hypernetwork. The paper argues that inner-product scoring cannot express all relevance patterns, and proves that for any fixed linear similarity there are query-relevant document sets that cannot be perfectly ranked. In experiments on MS MARCO and TREC Deep Learning data, the approach outperforms strong dense retrievers and several rerankers and larger models, and the relative gains grow on harder tasks like tip-of-the-tongue and instruction-following retrieval. The paper also reports retrieving from an 8.8 million passage corpus in under 60 milliseconds using a graph-based approximate search.

What carries the argument

The load-bearing component is the hyperhead layer paired with the q-net. A hyperhead uses scaled-dot-product attention over the contextualized query embeddings to produce each weight matrix and bias vector of the q-net, adding a query-independent base weight $\theta^H_i$ so the model can learn universal patterns. The q-net is a small feed-forward network with residual connections and layer normalization that takes a single 768-dimensional document vector and outputs a scalar score. This arrangement makes the scoring function nonlinear and query-specific while keeping document representations precomputable, which is what allows Hypencoder to claim both expressive scoring and efficient approximate retrieval. The efficiency machinery is a small-world graph over document vectors built with $\ell^2$ distances, searched greedily by scoring only a limited candidate set with the q-net.

What would settle it

Run the paper's graph-search algorithm on a collection where Euclidean neighbors of documents are deliberately unrelated to q-net scores (for example, embeddings shuffled after graph construction) and compare recall@1000 against exhaustive q-net scoring; if the recall gap stays the same as on MS MARCO, the proposed efficiency explanation is wrong, whereas a large drop would confirm that the latency result depends on $\ell^2$ locality.

Watch

Extended reading notes

Core claim

The central claim is that the query encoder should not output a vector at all, but rather the weights of a query-dependent scoring network, the q-net, which maps a single document vector to a relevance score. The paper proves that any inner-product-based scoring function has a ceiling: by Radon's theorem, once a corpus has more than dimension-plus-one documents, some set of relevant documents cannot be linearly separated from the rest, so a linear similarity cannot produce a perfect ranking for every query. Hypencoder's q-net, being a multilayer neural network with query-generated weights, is not subject to that ceiling. In experiments, this design yields a new state-of-the-art on TREC DL '19 and '20 among BERT-sized single-vector dense encoders, beats BE-Base, TAS-B, CL-DRD, and reference models including RepLLaMA and both rerankers on key metrics, and produces larger relative gains on TOT, DL-HARD, and FollowIR. The paper positions this as a new category of retrieval model that combines bi-encoder efficiency with more expressive cross-encoder-like matching.

Load-bearing premise

The efficiency result rests on the assumption that a graph built from Euclidean distances between document vectors is a reliable map of where the nonlinear q-net will score well, an assumption the paper itself flags as unproven because small input changes can produce large score changes.

Editorial extensions

If this is right

  • If the central claim holds, dense retrieval is no longer limited to linear scoring, so query complexity can be added without giving up first-stage efficiency.
  • Hard retrieval tasks with verbose, multi-facet, or instruction-bearing queries should benefit more from Hypencoder than standard passage ranking, as the paper's TOT, DL-HARD, and FollowIR results indicate.
  • The approximate search results imply that nonlinear query-specific scoring can still be served from a precomputed document graph at practical latencies for collections of at least tens of millions of passages.
  • Because the framework is generic, its gains could compound with multi-vector document representations, harder-negative training, distillation, and other techniques developed for dense retrieval.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims, one could directly test the theoretical ceiling by constructing a query whose relevant set is a Radon partition and checking whether any trained linear retriever fails to rank it perfectly while a q-net with enough capacity succeeds.
  • The 60-millisecond latency claim likely depends on the document graph preserving q-net score locality, which the paper itself notes is not guaranteed; a testable extension is building the graph using q-net score gradients or learned neighbor selection instead of $\ell^2$ distance alone.
  • Beyond retrieval, the same query-conditioned weight generation could transfer to other matching problems such as recommendation, where learned similarity functions are already used and where per-user scoring networks could replace fixed inner products.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper introduces Hypencoder, a retrieval architecture in which a query encoder (a hypernetwork) produces the weights of a small query-specific scoring network (the q-net) that operates on a single document embedding. The authors argue that inner-product similarity fundamentally limits retrieval quality, supporting this with a Radon theorem argument, and they present a graph-based approximate search algorithm to make the method practical. Experiments on MSMARCO, TREC Deep Learning 2019/2020, BEIR-style out-of-domain tasks, TREC Tip-of-the-Tongue, and FollowIR show improved effectiveness over strong dense retrieval baselines and several larger reference models, with a reported average latency of 59.6 ms on an 8.8M-passage collection.

Significance. If the empirical results are credited, the work makes a useful contribution: it introduces a new modeling paradigm for first-stage retrieval with a more expressive query-conditioned scoring function, provides an open-sourced implementation, and demonstrates consistent gains over standard bi-encoders—especially on hard tasks such as tip-of-the-tongue and instruction-following retrieval. The controlled BE-Base comparison and the use of multiple standard benchmarks strengthen the empirical core. However, the theoretical motivation is not established as stated, and the approximate-search efficiency claim rests on a heuristic that is only evaluated on two TREC Deep Learning test sets. The significance is therefore conditional on a careful revision of these two claims.

major comments (2)
  1. [Section 3, 'On the Limitations of Linear Similarity Functions such as Inner Product'] The Radon theorem argument does not prove the claim in the abstract that 'there is always a set of relevant documents which cannot be perfectly retrieved regardless of the query vector and specific encoder model.' Radon's theorem guarantees that any h+2 points in h dimensions can be partitioned into two non-linearly-separable subsets, but it says nothing about whether such subsets coincide with the relevance sets of real queries. The sentence 'If any two of these subsets contain all the relevant documents for a query' is an unsubstantiated condition that is not derived from the argument. This is a load-bearing theoretical claim in the motivating section; please either provide a rigorous connection between Radon partitions and query relevance distributions or weaken the claim to a motivating observation.
  2. [Section 3.6, Algorithm 1, and Table 4] The efficiency claim that the model 'is able to retrieve from a corpus of 8.8M documents in under 60 milliseconds' depends on the assumption that a graph built from l2 distances between document embeddings is a good navigation structure for the non-linear q-net score. This assumption is not validated beyond TREC DL '19 and '20, and the early-stopping condition in Algorithm 1 (line 8) is not sound for arbitrary non-linear scorers: the score of an unvisited neighbor is not upper-bounded by the current kth-best score, so genuinely better documents can be missed even when the stopping condition fires. The paper itself notes in Section 3.6 that the non-linear nature of the Hypencoder scoring function could make small input differences produce large score differences. Please provide additional empirical evidence on other collections or query distributions, or explicitly scope the latency result as a proof-of-concept on the MSMARCO/TREC DL collections.
minor comments (3)
  1. [Section 4.4.3, Table 3] The text states that 'Hypencoder is the only model to achieve a positive p-MRR', but Table 3 shows a positive p-MRR only on FollowIR News '21 (2.0), while the Robust '04 and Core '17 subsets have negative p-MRR values (-3.5 and -11.8). Please clarify that the positive p-MRR is only on one subset.
  2. [Section 4.4.2, Table 2] On NFCorpus, Hypencoder's nDCG@10 (0.324) is slightly below BE-Base (0.327), yet the accompanying text says Hypencoder 'remains dominant' on out-of-domain tasks. Please qualify the claim to reflect this exception.
  3. [Figure 2] The caption contains a typographical error: 'n_{Candidates}' is spelled 'n_{Candidates}' in two places as 'n_{Candidates}' and 'n_{Candidates}'? In the text it appears as 'n_Candidates' and 'n_{Candidates}' with an extra 'd': 'n_{Candidates}' should be 'n_{Candidates}'. Please correct the spelling of 'n_{Candidates}' and 'n_{Candidates}' in the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: retrieval-quality results follow from external teacher distillation and external test labels; the efficiency claim is an empirical heuristic evaluation, not a derivation from the model's own fitted values.

full rationale

The paper's central retrieval-quality claim is not circular: Hypencoder is trained end-to-end with distillation labels from an external MiniLM cross-encoder (Section 4.1.1), and its in-domain, out-of-domain, and hard-task numbers are measured against externally defined test collections (TREC DL '19/'20, MSMARCO Dev, BEIR, TOT, FollowIR). The theoretical motivation (Radon's theorem) is an independent external mathematical result and is used only to motivate a more expressive scorer, not to predict the empirical numbers. The approximate-search latency claim is an empirical evaluation of Algorithm 1 on MSMARCO, and although the l2-graph navigation heuristic is not theoretically guaranteed for a non-linear q-net, that is a correctness/robustness concern, not circularity. The paper does cite its own prior work (e.g., SNRM [75], Prakash et al. [52], CL-DRD [78]) and uses an early Hypencoder iteration to mine training candidates, but none of these is a load-bearing step that reduces a prediction to its own input: labels come from an external teacher and test labels are external. No step was found where Eq. X is equivalent to Eq. Y by construction or where a fitted parameter is renamed as a prediction.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central empirical claim depends mainly on standard ML assumptions and a few domain assumptions. The most important unstated premise is that the Radon partition subsets correspond to actual relevance sets, which the paper asserts rather than proves. The efficient search claim additionally depends on a heuristic graph assumption. No free parameters are fitted to the evaluation labels in the sense of a scientific prediction; the listed parameters are tuned by validation or chosen for the latency demonstration.

free parameters (4)
  • q-net depth (number of linear layers) = 6 layers
    Chosen by validation experiments on TREC DL '19/'20 (Figure 3); performance plateaus at 6 and degrades at 8. The reported results all use this depth, so it is a hand-selected architectural parameter that the empirical claim depends on.
  • Efficient search configuration (initial candidates, neighbors per iteration, max iterations) = Efficient 1: 10000, 64, 16
    Tuned to report the 59.6 ms latency on MS MARCO; approximates exhaustive search with some nDCG loss. This configuration is part of the practicality claim.
  • Efficient search configuration (quality variant) = Efficient 2: 100000, 328, 20
    Second tuned configuration for the quality-speed trade-off reported in Table 4.
  • Document graph degree = 100 neighbors per document
    Chosen for the graph-based search; affects both search quality and index size, and is not justified beyond the experiments.
assumptions (6)
  • standard math Radon's theorem: any h+2 points in R^h can be partitioned into two subsets with intersecting convex hulls
    Used in Section 3 to argue that some subsets of document vectors are not linearly separable, motivating a non-linear scoring function.
  • standard math Universal approximation: multilayer feedforward networks can approximate a wide class of functions
    Invoked in Section 3 to claim the q-net can express more complex relevance functions than inner product.
  • ad hoc to paper Relevance sets for real queries can be arbitrary subsets of the corpus, so non-separable subsets correspond to genuine queries
    Needed to turn the Radon partition existence result into the abstract's claim that there is always a set of relevant documents that cannot be perfectly retrieved. The paper does not justify this step.
  • domain assumption The [CLS] document embedding produced by BERT preserves enough information for the non-linear scorer to improve over inner product
    The empirical method uses only the [CLS] vector as document representation (Section 3.2); if this representation discarded information needed for complex relevance, the q-net could not recover it.
  • domain assumption The MiniLM cross-encoder provides reliable relevance labels for query-passage pairs used in training
    Training data is generated by sampling passages from an early Hypencoder and labeling with MiniLM (Section 4.1.1). The quality of the teacher bounds the quality of the student.
  • domain assumption The l2-distance document graph is a good navigation structure for the non-linear q-net scores
    Algorithm 1 and the latency results assume that greedy best-first search over l2 neighbors finds high-scoring documents for the q-net; the paper provides no proof or analysis for this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hypencoder: Hypernetworks for Information Retrieval." pith.science (2026). https://pith.science/paper/NJK6GN7F

@misc{pith2026250205364,
  author       = {Pith},
  title        = {Pith review of: Hypencoder: Hypernetworks for Information Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NJK6GN7F}},
  note         = {Machine review of arXiv:2502.05364}
}
read the original abstract

Existing information retrieval systems are largely constrained by their reliance on vector inner products to assess query-document relevance, which naturally limits the expressiveness of the relevance score they can produce. We propose a new paradigm; instead of representing a query as a vector, we use a small neural network that acts as a learned query-specific relevance function. This small neural network takes a document representation as input (in this work we use a single vector) and produces a scalar relevance score. To produce the small neural network we use a hypernetwork, a network that produces the weights of other networks, as our query encoder. We name this category of encoder models Hypencoders. Experiments on in-domain search tasks show that Hypencoders significantly outperform strong dense retrieval models and even surpass reranking models and retrieval models with an order of magnitude more parameters. To assess the extent of Hypencoders' capabilities, we evaluate on a set of hard retrieval tasks including tip-of-the-tongue and instruction-following retrieval tasks. On harder tasks, we find that the performance gap widens substantially compared to standard retrieval tasks. Furthermore, to demonstrate the practicality of our method, we implement an approximate search algorithm and show that our model is able to retrieve from a corpus of 8.8M documents in under 60 milliseconds.

Figures

Figures reproduced from arXiv: 2502.05364 by the authors.

Figure 1
Figure 1. Overview comparing Hypencoder to existing retrieval and reranking paradigms. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Relationship between the three main parameters of our efficient search: the size of the initial set of candidates [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Average nDCG@10 on TREC DL ’19 and ’20 versus [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

86 extracted references · 28 canonical work pages

  1. [1]

    Abien Fred Agarap. 2018. Deep Learning using Rectified Linear Units (ReLU). CoRR abs/1803.08375 (2018). arXiv:1803.08375 http://arxiv.org/abs/1803.08375

    arXiv 2018

  2. [2]

    Harman, E

    James Allan, Donna K. Harman, E. Kanoulas, Dan Li, Christophe Van Gysel, and Ellen M. Voorhees. 2017. TREC 2017 Common Core Track Overview. In Text Retrieval Conference. https://api.semanticscholar.org/CorpusID:38019792

    2017

  3. [3]

    Jaime Arguello, Samarth Bhargav, Fernando Diaz, Evangelos Kanoulas, and Bhaskar Mitra. 2023. Overview of the TREC 2023 Tip-of-the-Tongue Track. In The Thirty-Second Text REtrieval Conference Proceedings (TREC 2023), Gaithers- burg, MD, USA, November 14-17, 2023 (NIST Special Publication, Vol. 500-xxx) , Ian Soboroff and Angela Ellis (Eds.). National Insti...

    2023

  4. [4]

    Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer Normaliza- tion. CoRR abs/1607.06450 (2016). arXiv:1607.06450 http://arxiv.org/abs/1607. 06450

    arXiv 2016

  5. [5]

    Samarth Bhargav, Georgios Sidiropoulos, and Evangelos Kanoulas. 2022. ’It’s on the tip of my tongue’: A new Dataset for Known-Item Retrieval. In WSDM ’22: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21 - 25, 2022 , K. Selcuk Candan, Huan Liu, Leman Akoglu, Xin Luna Dong, and Jiliang Ta...

    arXiv 2022

  6. [6]

    Alexander Bondarenko, Maik Fröbe, Meriem Beloucif, Lukas Gienapp, Yamen Ajjour, Alexander Panchenko, Chris Biemann, Benno Stein, Henning Wachsmuth, Martin Potthast, and Matthias Hagen. 2020. Overview of Touché 2020: Argument Retrieval. In Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum, Thessaloniki, Greece, September 22-25, 2020 ...

    2020

  7. [7]

    Vera Boteva, Demian Gholipour Ghalandari, Artem Sokolov, and Stefan Riezler

  8. [8]

    Christopher Burges, Robert Ragno, and Quoc Le. 2006. Learning to rank with nonsmooth cost functions. Advances in neural information processing systems 19 (2006)

    2006

Show all 86 references
  1. [9]

    Christopher J. C. Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Gregory N. Hullender. 2005. Learning to rank using gradient descent. In Machine Learning, Proceedings of the Twenty-Second International Conference (ICML 2005), Bonn, Germany, Augu...

    2005

  2. [10]

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2020. Overview of the TREC 2020 Deep Learning Track. In Proceedings of the Twenty-Ninth Text REtrieval Conference, TREC 2020, Virtual Event [Gaithersburg, Maryland, USA], November 16-20, 2020 (NIST Special Publicat...

    2020

  3. [11]

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Jimmy Lin

  4. [12]

    Voorhees, and Ian Soboroff

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, and Ian Soboroff. 2022. Overview of the TREC 2022 Deep Learning Track. In Proceedings of the Thirty-First Text REtrieval Conference, TREC 2022, online, November 15-19, 2022 (NIST Special P...

    2022

  5. [13]

    Voorhees

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. CoRR abs/2003.07820 (2020). arXiv:2003.07820 https://arxiv.org/abs/2003.07820

    2020 arXiv

  6. [15]

    Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hutter, and Joel Veness. 2024. Language Modeling Is Compression. In The Twelfth International C...

    2024

  7. [16]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Associa- tion for Computational Linguistics: Hum...

    2019

  8. [17]

    Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant

  9. [18]

    Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing...

    2021

  10. [19]

    Luyu Gao and Jamie Callan. 2022. Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022 , S...

    2022 doi

  11. [20]

    Luyu Gao, Zhuyun Dai, and Jamie Callan. 2021. COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List. In North American Chapter of the Association for Computational Linguistics . https: //api.semanticscholar.org/CorpusID:233241070

    2021

  12. [21]

    Bruce Croft, and Xueqi Cheng

    Jiafeng Guo, Yixing Fan, Liang Pang, Liu Yang, Qingyao Ai, Hamed Zamani, Chen Wu, W. Bruce Croft, and Xueqi Cheng. 2020. A Deep Look into neural ranking models for information retrieval. Information Processing & Management 57, 6 (2020), 102067. https://doi.org/10.1016/j.ipm.20...

    2020

  13. [22]

    Dai, and Quoc V

    David Ha, Andrew M. Dai, and Quoc V. Le. 2017. HyperNetworks. In 5th Interna- tional Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings . OpenReview.net. https://openreview. net/forum?id=rkpACe1lx

    2017

  14. [24]

    Xiangnan He and Tat-Seng Chua. 2017. Neural Factorization Machines for Sparse Predictive Analytics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, August 7-11, 2017 , Noriko Kando, Tet...

    2017

  15. [26]

    Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. CoRR abs/2010.02666 (2020). arXiv:2010.02666 https://arxiv.org/abs/2010.02666

    2020 arXiv

  16. [27]

    Khattab, Sophia Althammer, Mete Sertkan, and Allan Hanbury

    Sebastian Hofstätter, O. Khattab, Sophia Althammer, Mete Sertkan, and Allan Hanbury. 2022. colberter. Proceedings of the 31st ACM International Conference on Information & Knowledge Management (2022). https://api.semanticscholar. org/CorpusID:247628023

    2022

  17. [28]

    Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021. Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling. InSIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information...

    2021

  18. [29]

    Stinchcombe, and Halbert White

    Kurt Hornik, Maxwell B. Stinchcombe, and Halbert White. 1989. Multilayer feedforward networks are universal approximators. Neural Networks 2, 5 (1989), 359–366. https://doi.org/10.1016/0893-6080(89)90020-8

    1989 doi

  19. [30]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bo- janowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised Dense Infor- mation Retrieval with Contrastive Learning. Trans. Mach. Learn. Res. 2022 (2022). https://openreview.net/forum?id=jKN1pXi7b0

    2022

  20. [31]

    Reddi, Sadeep Jayasumana, Ankit Singh Rawat, Aditya Krishna Menon, Felix Yu, and Sanjiv Kumar

    Ziwei Ji, Himanshu Jain, Andreas Veit, Sashank J. Reddi, Sadeep Jayasumana, Ankit Singh Rawat, Aditya Krishna Menon, Felix Yu, and Sanjiv Kumar. 2024. Effi- cient Document Ranking with Learnable Late Interactions. CoRR abs/2406.17968 Hypencoder: Hypernetworks for Information R...

    2024 doi arXiv

  21. [32]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, ...

    2020

  22. [34]

    Lin, Yashar Mehdad, Wen tau Yih, and Xilun Chen

    Minghan Li, Sheng-Chieh Lin, Barlas Oğuz, Asish Ghoshal, Jimmy J. Lin, Yashar Mehdad, Wen tau Yih, and Xilun Chen. 2022. CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval. In Annual Meeting of the Association ...

    2022

  23. [35]

    Yury Lifshits and Shengyu Zhang. 2009. Combinatorial algorithms for nearest neighbors, near-duplicates and small-world design. InProceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2009, New York, NY, USA, January 4-6, 2009 , Claire Mathieu (Ed....

    2009

  24. [36]

    J. Lin, R. Nogueira, and A. Yates. 2022. Pretrained Transformers for Text Ranking: BERT and Beyond. Springer International Publishing

    2022

  25. [37]

    Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2020. Distilling Dense Rep- resentations for Ranking using Tightly-Coupled Teachers. CoRR abs/2010.11386 (2020). arXiv:2010.11386 https://arxiv.org/abs/2010.11386

    2020 arXiv

  26. [38]

    Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. 2023. How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval. In Findings of the Asso- ciation for Computational Linguistics: EMNLP 2023 ,...

    2023 doi

  27. [39]

    Zhenghao Lin, Yeyun Gong, Xiao Liu, Hang Zhang, Chen Lin, Anlei Dong, Jian Jiao, Jingwen Lu, Daxin Jiang, Rangan Majumder, and Nan Duan. 2023. PROD: Progressive Distillation for Dense Retrieval. In Proceedings of the ACM Web Conference 2023 (Austin, TX, USA) (WWW ’23) . Associ...

    2023 doi

  28. [40]

    Xinyu Ma, Jiafeng Guo, Ruqing Zhang, Yixing Fan, Yingyan Li, and Xueqi Cheng. 2021. B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval. Proceedings of the 44th International ACM SI- GIR Conference on Research and Development in Informat...

    2021

  29. [41]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine- Tuning LLaMA for Multi-Stage Text Retrieval. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18...

    2024

  30. [42]

    Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Nazli Goharian, and Ophir Frieder. 2020. Efficient Document Re-Ranking for Trans- formers by Precomputing Term Representations. In Proceedings of the 43rd In- ternational ACM SIGIR conference on research...

    2020

  31. [43]

    Iain Mackie, Jeffrey Dalton, and Andrew Yates. 2021. How Deep is your Learning: the DL-HARD Annotated Deep Learning Dataset. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

    2021

  32. [44]

    Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. 2018. WWW’18 Open Challenge: Fi- nancial Opinion Mining and Question Answering. In Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018,...

    2018

  33. [45]

    Malkov and Dmitry A

    Yury A. Malkov and Dmitry A. Yashunin. 2020. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs.IEEE Trans. Pattern Anal. Mach. Intell. 42, 4 (2020), 824–836. https://doi.org/10.1109/ TPAMI.2018.2889473

    2020

  34. [46]

    Bhaskar Mitra and Nick Craswell. 2018. An Introduction to Neural Information Retrieval. Found. Trends Inf. Retr. 13, 1 (Dec. 2018), 1–126. https://doi.org/10. 1561/1500000061

    2018

  35. [47]

    Bhaskar Mitra and Nick Craswell. 2019. An Updated Duet Model for Passage Re-ranking. CoRR abs/1903.07666 (2019). arXiv:1903.07666 http://arxiv.org/abs/ 1903.07666

    2019 arXiv

  36. [48]

    Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017. Learning to Match using Local and Distributed Representations of Text for Web Search. In Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, Rick Barrett, Rick C...

    2017

  37. [49]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In Proceedings of the Workshop on Cogni- tive Computation: Integrating neural and symbolic approaches 2016 ...

    2016

  38. [50]

    Rodrigo Frassetto Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-Stage Document Ranking with BERT. CoRR abs/1910.14424 (2019). arXiv:1910.14424 http://arxiv.org/abs/1910.14424

    2019 arXiv

  39. [51]

    Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Shengxian Wan, and Xueqi Cheng

  40. [52]

    Prafull Prakash, Julian Killingback, and Hamed Zamani. 2021. Learning Robust Dense Retrieval Models from Incomplete Relevance Labels. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-...

    2021

  41. [53]

    Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxi- ang Dong, Hua Wu, and Haifeng Wang. 2021. RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2021 Conference of the North Am...

    2021

  42. [54]

    Johann Radon. 1921. Mengen konvexer Körper, die einen gemeinsamen Punkt enthalten. Math. Ann. 83, 1 (1921). https://doi.org/10.1007/BF01464231

    1921 doi

  43. [55]

    In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA , Dale Schuurmans and Michael P

    Text Matching as Image Recognition. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA , Dale Schuurmans and Michael P. Wellman (Eds.). AAAI Press, 2793–2799. https: //doi.org/10.1609/AAAI.V30I1.10341

    2016 doi

  44. [56]

    Voorhees, Lucy Lu Wang, and William R

    Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen M. Voorhees, Lucy Lu Wang, and William R. Hersh. 2021. Searching for scientific evidence in a pandemic: An overview of TREC-COVID. J. Biomed. Informatics 121 (2021), 103865. https://d...

    2021

  45. [57]

    Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell

    Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. 2019. Meta-Learning with Latent Embedding Optimization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 ....

    2019

  46. [58]

    Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational ...

    2022

  47. [59]

    Marcin Sendera, Marcin Przewiezlikowski, Konrad Karanowski, Maciej Zieba, Jacek Tabor, and Przemyslaw Spurek. 2023. HyperShot: Few-Shot Learning by Kernel HyperNetworks. In IEEE/CVF Winter Conference on Applications of Computer Vision, W ACV 2023, Waikoloa, HI, USA, January 2-...

    2023

  48. [60]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Em- pirical Methods in Natural Language Processing . Association for Computational Linguistics. https://arxiv.org/abs/1908.10084

    2019 arXiv

  49. [61]

    Shulong Tan, Weijie Zhao, and Ping Li. 2021. Fast Neural Ranking on Bipartite Graph Indices. Proc. VLDB Endow. 15, 4 (2021), 794–803. https://doi.org/10.14778/ 3503585.3503589

    2021

  50. [62]

    Shulong Tan, Zhixin Zhou, Zhaozhuo Xu, and Ping Li. 2020. Fast Item Ranking under Neural Network based Measures. In WSDM ’20: The Thirteenth ACM Inter- national Conference on Web Search and Data Mining, Houston, TX, USA, February 3-7, 2020, James Caverlee, Xia (Ben) Hu, Mounia...

    2020

  51. [63]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evalua- tion of Information Retrieval Models. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks ...

    2021

  52. [64]

    V. N. Vapnik and A. Ya. Chervonenkis. 1971. On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities. Theory of Probability & Its Applications 16, 2 (1971), 264–280. https://doi.org/10.1137/1116025

    1971 doi

  53. [65]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: An- nual Conference on Neural Information Processing Systems ...

    2017

  54. [66]

    Johannes von Oswald, Christian Henning, João Sacramento, and Benjamin F. Grewe. 2020. Continual learning with hypernetworks. In 8th International Con- ference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30,

    2020

  55. [67]

    Ian Soboroff, Shudong Huang, and Donna Harman. 2020. TREC 2020 News Track Overview. In Proceedings of the Twenty-Ninth Text REtrieval Conference, TREC 2020, Virtual Event [Gaithersburg, Maryland, USA], November 16-20, 2020 (NIST Special Publication, Vol. 1266), Ellen M. Voorhe...

    2020

  56. [68]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2023. SimLM: Pre-training with Represen- tation Bottleneck for Dense Passage Retrieval. In Proceedings of the 61st An- nual Meeting of the Association for Computational...

    2023 doi

  57. [69]

    Lawrie, and Luca Soldaini

    Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn J. Lawrie, and Luca Soldaini. 2024. FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instruc- tions. CoRR abs/2403.15246 (2024). https://doi.org/10.48550/ARXIV....

    2024 doi arXiv

  58. [70]

    Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel

    Orion Weller, Benjamin Van Durme, Dawn J. Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel. 2024. Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models. CoRR abs/2409.11136 (2024). https: //doi.org/10.48550/ARXIV.2409.11136 arXiv:2409.11136

    2024 doi arXiv

  59. [71]

    Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. 2022. RetroMAE: Pre- Training Retrieval-oriented Language Models Via Masked Auto-Encoder. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Pro- cessing, EMNLP 2022, Abu Dhabi, United Arab Emirate...

    2022 doi

  60. [72]

    Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power

  61. [73]

    Bennett, Junaid Ahmed, and Arnold Overwijk

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. In 9th International Conference on Learning Representations, ICLR 2021, V...

    2021

  62. [74]

    Zhou Yang, Qingfeng Lan, Jiafeng Guo, Yixing Fan, Xiaofei Zhu, Yanyan Lan, Yue Wang, and Xueqi Cheng. 2018. A Deep Top-K Relevance Matching Model for Ad- hoc Retrieval. In Information Retrieval - 24th China Conference, CCIR 2018, Guilin, China, September 27-29, 2018, Proceedin...

    2018 doi

  63. [75]

    Voorhees

    Ellen M. Voorhees. 2004. Overview of the TREC 2004 Robust Track. In Proceed- ings of the Thirteenth Text REtrieval Conference, TREC 2004, Gaithersburg, Mary- land, USA, November 16-19, 2004 (NIST Special Publication, Vol. 500-261) , Ellen M. Voorhees and Lori P. Buckland (Eds....

    2004

  64. [76]

    Hansi Zeng, Surya Kallumadi, Zaid Alibadi, Rodrigo Nogueira, and Hamed Zamani. 2023. A Personalized Dense Retrieval Framework for Unified In- formation Access. In Proceedings of the 46th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval ...

    2023

  65. [77]

    Hansi Zeng, Julian Killingback, and Hamed Zamani. 2025. Scaling Sparse and Dense Retrieval in Decoder-Only LLMs. ArXiv abs/2502.15526 (2025). https: //api.semanticscholar.org/CorpusID:276558404

    2025 arXiv

  66. [78]

    Hansi Zeng, Hamed Zamani, and Vishwa Vinay. 2022. Curriculum Learning for Dense Retrieval Distillation. In SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022 , Enrique Amigó, Pablo Cast...

    2022

  67. [79]

    Zhang, and Shaoping Ma

    Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, M. Zhang, and Shaoping Ma. 2021. Optimizing Dense Retrieval Model Training with Hard Negatives. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021). https://api...

    2021

  68. [80]

    Chris Zhang, Mengye Ren, and Raquel Urtasun. 2019. Graph HyperNetworks for Neural Architecture Search. In 7th International Conference on Learning Rep- resentations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net. https://openreview.net/forum?id=rkgW0oA9FX

    2019

  69. [81]

    Weijie Zhao, Shulong Tan, and Ping Li. 2024. GUITAR: Gradient Pruning toward Fast Neural Ranking. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024 , Grace Hui Yan...

    2024

  70. [84]

    Bruce Croft, Erik G

    Hamed Zamani, Mostafa Dehghani, W. Bruce Croft, Erik G. Learned-Miller, and Jaap Kamps. 2018. From Neural Re-Ranking to Neural Ranking: Learning a Sparse Representation for Inverted Indexing. InProceedings of the 27th ACM International Conference on Information and Knowledge M...

    2018

  71. [1732]

    https://doi.org/10.1145/3404835.3463106

  72. [2016]

    In Advances in Information Retrieval - 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20-23, 2016

    A Full-Text Learning to Rank Dataset for Medical Information Retrieval. In Advances in Information Retrieval - 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20-23, 2016. Proceedings (Lecture Notes in Computer Science, Vol. 9626), Nicola Ferro, Fabio C...

    2016 doi

  73. [2017]

    End-to-End Neural Ad-hoc Ranking with Kernel Pooling. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, August 7-11, 2017 , Noriko Kando, Tetsuya Sakai, Hideo Joho, Hang Li, Arjen P. de V...

    2017

  74. [2020]

    https://openreview.net/forum?id=SJgwNerKvB

    OpenReview.net. https://openreview.net/forum?id=SJgwNerKvB

  75. [2021]

    In Proceedings of the Thirtieth Text REtrieval Conference, TREC 2021, online, November 15-19, 2021 (NIST Special Publication, Vol

    Overview of the TREC 2021 Deep Learning Track. In Proceedings of the Thirtieth Text REtrieval Conference, TREC 2021, online, November 15-19, 2021 (NIST Special Publication, Vol. 500-335), Ian Soboroff and Angela Ellis (Eds.). National Institute of Standards and Technology (NIS...

    2021

  76. [2022]

    From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective. In SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022, Enrique Amigó, Pablo Castells, Julio G...

    2022

  77. [2477]

    https://doi.org/10.1109/WACV56688.2023.00250

    2023

  78. [3734]

    https://doi.org/10.18653/v1/2022.naacl-main.272

    2022 doi

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.