Pith. sign in

REVIEW 3 major objections 5 minor 34 references

Counterfactual Query Rewriting to Use Historical Relevance Feedback

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Using historical relevance feedback to rewrite recurring queries lets a lightweight BM25 retriever beat transformer-based rankers.

desk verdict Useful, reproducible counterfactual-feedback study whose generalization claim outruns Table 3; solid paper but needs reframing. read the letter →

arxiv 2502.03891 v1 pith:4L6HRDQI submitted 2025-02-06 cs.IR

Jüri Keller , Maik Fröbe , Gijs Hendriksen , Daria Alexander , Martin Potthast , Matthias Hagen , Philipp Schaer This is my paper · ORCID
classification cs.IR
keywords QueryrewritingKeyqueriesLongitudinalevaluationRelevancefeedbackTemporalinformationretrievalWebsearchBM25LongEval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when a search engine sees a query it has seen before, the documents users previously found relevant can be rewritten into a better query even if those documents have since been edited or deleted. The authors call this a counterfactual use of historical relevance feedback: they use the version of the document from the time the relevance judgment was made, as though that version still exists. On the LongEval web-search collection, three BM25-based approaches built on this idea—boosting with old relevance labels, expanding the query with tf-idf terms from old relevant documents, and deriving minimal 'keyqueries' that place old relevant documents at the top of the current corpus—all improve nDCG@10 over the plain BM25 baseline, and the keyquery variant is the most effective. The claim matters because the rewrites can be computed offline, so a lightweight system could serve recurring queries better than expensive transformer-based rankers without running them at query time.

What carries the argument

The central mechanism is the keyquery, a reformulation that is built and then tested against the historical relevant documents before it is used. For a query $q$, the target set $D^+$ contains all documents with positive historical judgments; a keyquery is a minimal query such that every document in $D^+$ appears in the top-k results of the current corpus, the query has more than a threshold number of results, and no proper subquery satisfies both conditions. Candidate keyqueries are constructed from the top ten RM3 terms, and the candidate with the highest nDCG@10 on $D^+$ is selected. The ranking check is what separates this approach from ordinary relevance feedback: it ensures the rewritten query actually positions the previously relevant documents at the top, trading off specificity against the risk of overfitting. A simpler variant used in the paper expands the query with the top ten tf-idf terms from $D^+$ without this verification.

What would settle it

Run the same three rewrites on a longitudinal collection in which most previously relevant documents have been deleted and replaced with genuinely new information, and compare nDCG@10 against plain BM25; if the improvement disappears or turns negative while transformer baselines still help, the counterfactual availability of old documents is what drives the reported gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that historical relevance feedback, applied counterfactually to an evolved corpus, can be converted into query rewrites that outperform both ordinary BM25 and transformer-based ranking models. The keyquery is the load-bearing device: for a recurring query $q$ and the set $D^+$ of documents previously judged relevant, a keyquery $q_k$ is a minimal expanded query that retrieves every document in $D^+$ within the top-k results against the current corpus while still returning more than a threshold number of results. Candidate rewrites are generated from the top ten RM3 expansion terms and the best one is chosen by its nDCG@10 on $D^+$. In the LongEval evaluation, this keyquery approach reaches higher nDCG@10 than ColBERT, monoT5, and List-in-T5 on every tested timestamp, with simple tf-idf expansion also beating them on every timestamp.

Load-bearing premise

The load-bearing premise is the counterfactual assumption that a document judged relevant at an earlier time remains a usable signal even if it has been edited or deleted; if relevance drifts quickly, the rewrites would push rankings toward outdated content.

Editorial extensions

If this is right

  • Search engines that see the same query repeatedly can precompute rewritten queries from accumulated relevance feedback, so each repeat query costs one BM25 lookup while ranking better.
  • The keyquery variant gives a parameter-light way to keep improving recurring-query results as feedback accumulates, without retraining or GPU inference.
  • The ablation shows the rewrites can surface documents that were never seen before, where pure boosting of old documents contributes nothing.
  • In longitudinal setups like LongEval, historical-feedback lexical systems become a reference point that neural rankers without such feedback must be measured against.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit is feeding noisy click logs rather than graded judgments into the same rewrites; if a handful of imperfect clicks suffice, the approach becomes deployable with interaction data alone.
  • The paper's Matthews-effect caveat suggests a testable boundary: with a time-decay penalty on historical feedback, one could find how quickly the rewrite signal turns from helpful to harmful as document turnover accelerates.
  • Because the ablation removes previously seen documents and most of the gain disappears, the method may be best understood as a recall booster for old documents; a freshness threshold on the target set would be a natural safeguard before deployment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses the problem of using historical relevance feedback for recurring queries in evolving web corpora. It proposes three approaches built on BM25: boosting previously relevant documents by score adjustment (BM25Boost), expanding queries with top tf-idf terms from previously relevant documents (BM25RF), and constructing keyqueries that retrieve previously relevant documents in the top ranks (BM25keyquery). The approaches are evaluated on the LongEval collection at timestamps with overlapping queries, compared against BM25, BM25RM3, ColBERT, List-in-T5, and monoT5, using nDCG@10 and nDCG@10' (with unjudged documents removed). Table 2 reports large gains for the proposed methods. An ablation (Table 3) that removes all previously seen documents from runs and judgments is meant to test generalization to new documents. The paper concludes that rewriting queries with historical relevance feedback improves effectiveness and generalizes beyond known query-document pairs.

Significance. If the claims are valid, the contribution is practically relevant: a lightweight, precomputable query-rewriting strategy could exploit historical clicks or judgments to improve retrieval on recurring queries without retraining expensive neural models. The paper is clearly written, makes code available, and includes significance tests and an ablation that directly targets generalization. However, the current evidence does not support the full strength of the claims. The comparison with transformer baselines is asymmetric because those baselines do not receive the historical relevance signal, and the ablation shows that gains on never-before-seen documents are largely not significant. The realistic significance is therefore narrower than the paper argues: the methods appear to help mainly by retrieving previously relevant documents under the counterfactual assumption that those documents remain relevant, rather than by demonstrating robust generalization to genuinely new documents.

major comments (3)
  1. [Section 4.3, Table 3, Section 5] The generalization claim is not supported by the ablation. Table 3 shows that after removing all documents seen in previous timestamps from runs and relevance judgments, BM25RF never achieves a statistically significant improvement over BM25 at any of the five timestamps, and BM25keyquery is significant at only one timestamp (01/24, +0.032). Therefore the statements in Section 4.3 ('indicating that these approaches generalize to newly created or modified documents') and Section 5 ('The ablation study suggests that the advanced approaches generalize beyond known query-document pairs') overstate the evidence. The large margins in Table 2 may be largely attributable to retrieving the very documents whose historical relevance labels were used to construct the rewrites. The paper should either supply additional evidence of generalization (e.g., a more powerful analysis focused on new relevant documents, or per-query breakdowns) or explicitly restrict the contribution to improving retrieval of previously relevant documents.
  2. [Section 4.1, Table 2] The comparison with transformer-based baselines is asymmetric. BM25RF and BM25keyquery are constructed using historical relevance judgments (previous qrels), whereas ColBERT, List-in-T5, and monoT5 are run on the current query and corpus only, with no access to the same historical relevance signal. The abstract's claim that the approach 'even outperforms computationally expensive transformer-based approaches' therefore conflates the effect of the extra relevance signal with a difference in retrieval models. The authors should either give the transformer baselines access to historical feedback in a comparable way (e.g., by fine-tuning on previous qrels or using a feedback-based expansion) or reframe the comparison as 'with historical feedback versus without' and soften the claim of general superiority over transformers.
  3. [Section 4.3] The sentence 'Keyqueries are the most effective approach in all cases, outperforming the best transformer by a large margin' is not consistent with Table 2. For nDCG@10', BM25RF is higher than BM25keyquery at timestamps 01/23 (0.590 vs 0.574) and 08/23 (0.426 vs 0.422); for nDCG@10, BM25Boost is higher at 07/22, 01/23, and 06/23. Please correct the claim and, if keyqueries are to be recommended as the best variant, report significance tests for the pairwise differences among the proposed approaches.
minor comments (5)
  1. [Throughout] The manuscript contains several typos and small language errors, e.g., 'We proposed' in the Abstract, 'recommendet' and 'whreas' in Section 4.3, 'tocuments' in Section 4.1, 'beyound' in Section 4.3, and 'students t-test' in Table 3's caption. A careful proofreading pass is needed.
  2. [Figure 1 and Table 2] The timestamps in Figure 1 (06/23, 07/23, 09/23, 01/24, 06/24, 08/24) are inconsistent with Table 2 (07/22, 09/22, 01/23, 06/23, 08/23), and the text says the evaluation runs between June 2022 and August 2023. Please align the labels and clarify the actual evaluation timestamps.
  3. [Table 3] Table 3 reports only the improvement over BM25; including the BM25 baseline nDCG@10' values would help readers judge the practical magnitude of the deltas and the variability across timestamps.
  4. [Section 4.2 and Section 5] The paper notes the limitation that previously relevant documents may no longer be available or relevant, but it would be informative to connect this directly to the observed S3 distribution (about 50% of overlapping URLs have similarity below 0.8) and to the weak generalization results in Table 3; currently the connection is not drawn.
  5. [Section 3] In the formal definition of keyqueries, the parameter l is not defined; it only appears later as the 'more than 25 results' constraint in Section 4.1. Please define l in the definition.

Circularity Check

3 steps flagged · score 6.0 of 10

The keyquery variant is fit by construction to retrieve the historical feedback documents, and the Table 3 ablation fails to support the generalization half of the claim, so the main comparison to transformers is partially circular.

  1. self definitional [Section 3, 'Keyqueries for Previously Relevant Documents']
    "A query qk is a keyquery for the set of target documents D+ = {d|(q,d,t) ∈ H ∧ rel(q,d,t) > 0} previously known relevant for a query q against the corpus D+ ∪ D for the given retrieval model, iff (1) every d ∈ D+ is in the top-k results, (2) qk has more than l results, and (3) no subquery q′k ⊂ qk satisfies the above. If multiple candidates are keyqueries, we use the one with the highest nDCG@10 on D+."

    D+ is by definition the set of documents carrying the historical relevance labels. The keyquery is selected so that every d in D+ is in the top-k against D+∪D, and the best candidate is the one with the highest nDCG@10 on D+. When that keyquery is run against the current corpus D, which under the paper's counterfactual still contains those documents, the output is constructed to put the labeled relevant documents at the top. Evaluating with nDCG@10 and nDCG@10' on qrels for those same documents therefore rewards exactly the construction signal.

  2. fitted input called prediction [Section 3, 'Previously Relevant Documents as Relevance Feedback']
    "For a query q, the set D+ = {d|(q,d,t) ∈ H ∧ rel(q,d,t) > 0} specifies the previously positive documents on which we calculate the tf-idf scores. The top k terms with the highest tf-idf scores are obtained for query expansion and appended to the original query."

    BM25RF is constructed from the very documents whose relevance labels define the success metric; expanding with their tf-idf terms is a soft way of retrieving those same documents again. The paper's own Table 3 shows that once all previously seen documents are removed from runs and qrels, BM25RF never significantly beats BM25 at any timestamp. Thus the large nDCG@10' gains in Table 2 cannot be attributed to a learned generalization to new relevant documents; they come from re-finding the feedback documents whose labels were used to build the expansion.

1 more flagged steps
  1. other [Section 5, 'Conclusion and Future Work']
    "The ablation study suggests that the advanced approaches generalize beyond known query-document pairs, making them effective for new documents as well."

    This sentence is the load-bearing bridge for the central claim that query rewriting with historical relevance feedback is preferable to simple boosting and outperforms transformers. It is contradicted by the paper's own Table 3: BM25RF shows no significant improvement over BM25 on never-before-seen documents at any timestamp, and BM25keyquery is significant at only one of five timestamps. The only demonstrated source of the Table 2 gains is the construction-based retrieval of the feedback documents, so the asserted generalization beyond known query-document pairs is unsupported by the reported evidence.

full rationale

Both central rewriting approaches use the historical relevance labels as their input, and for keyqueries the method's definition explicitly requires the previously relevant documents D+ to be in the top-k while optimizing nDCG@10 on D+ itself. Submitting the resulting keyquery to the current corpus and evaluating on qrels for the same documents is therefore partly self-definitional: the system is rewarded for retrieving the exact documents used to construct the query. TF-IDF query expansion is a softer version of the same fitting, since the expansion terms come from the feedback documents whose labels are also the success criterion. The authors disclose the counterfactual assumption and include a Table 3 ablation that removes all previously seen documents, which is a genuine attempt to break the loop; however, the ablation results are weak (BM25RF: no significant improvement; BM25keyquery: significant only at 01/24), so the Section 5 conclusion that the approaches generalize is not backed by the data. Self-citations to prior keyquery work and boosting are not the core problem here; the circularity is in the construction itself, not in an unverified cited theorem. The paper is transparent and the ablation earns credit, but the central 'outperforms transformers' claim rests on an evaluation that largely rewards retrieving the same documents used to build the queries, giving a partial circularity score of 6.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new theoretical entities are introduced; the approach reuses existing retrieval components (BM25, RM3-style expansion, boosting, keyqueries) with hyperparameters taken from prior work. The load-bearing assumptions are the stability of historical relevance over time and the quality of the click-derived evaluation labels.

free parameters (4)
  • Boosting weight lambda = 0.7
    Weight applied in Eq. (1) to historical relevance judgments; set based on previous experiments by the same authors (Keller et al., CLEF 2024), not tuned on the LongEval data used here.
  • Boosting multiplier mu = 2
    Additional multiplier for highly relevant documents in Eq. (1); from the same prior work.
  • Feedback term count k = 10
    Number of tf-idf terms added to the query in BM25RF and number of RM3 candidates in keyquery generation; chosen to match the PyTerrier RM3 default.
  • Keyquery constraints (top-k, min result count) = top-10, >25 results
    Target documents must appear in the top 10 results and the query must return more than 25 results; set by the authors for the LongEval scenario (Section 4.1).
assumptions (4)
  • domain assumption Counterfactual stability of historical relevance: previously relevant documents remain indicative for retrieval even after deletion or content change.
    Stated in Section 3 and Section 5; without this, rewriting queries with the old documents could rank stale content and the measured gains would not transfer.
  • domain assumption Click-derived relevance labels in LongEval are valid enough for evaluation.
    The LongEval labels come from Qwant click logs (Section 4.1); the paper relies on these for computing nDCG@10 and nDCG@10'.
  • domain assumption The top-10 RM3 terms are an adequate vocabulary for keyquery enumeration.
    Keyquery candidates are generated only from the top-10 RM3 terms, following the re-implementation of Fröbe et al. [12]; a richer or different vocabulary could change which keyqueries exist.
  • domain assumption Removing unjudged documents yields a valid effectiveness estimate (nDCG@10').
    Adopted from Sakai [28] to handle sparse click-derived judgments; the removal changes the result list for all systems, so it is fair across systems but can inflate scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Counterfactual Query Rewriting to Use Historical Relevance Feedback." pith.science (2026). https://pith.science/paper/4L6HRDQI

@misc{pith2026250203891,
  author       = {Pith},
  title        = {Pith review of: Counterfactual Query Rewriting to Use Historical Relevance Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4L6HRDQI}},
  note         = {Machine review of arXiv:2502.03891}
}
read the original abstract

When a retrieval system receives a query it has encountered before, previous relevance feedback, such as clicks or explicit judgments can help to improve retrieval results. However, the content of a previously relevant document may have changed, or the document might not be available anymore. Despite this evolved corpus, we counterfactually use these previously relevant documents as relevance signals. In this paper we proposed approaches to rewrite user queries and compare them against a system that directly uses the previous qrels for the ranking. We expand queries with terms extracted from the previously relevant documents or derive so-called keyqueries that rank the previously relevant documents to the top of the current corpus. Our evaluation in the CLEF LongEval scenario shows that rewriting queries with historical relevance feedback improves the retrieval effectiveness and even outperforms computationally expensive transformer-based approaches.

Figures

Figures reproduced from arXiv: 2502.03891 by the authors.

Figure 2
Figure 2. S3 Similarities of documents with overlapping URLs as eCDF plot. 4 Evaluation We evaluate our counterfactual query rewriting in the LongEval scenario that comes with overlapping queries across six points in time between June 2022 and August 2023 [2,4,16]. We modify the LongEval datasets to focus on queries that re-occur across multiple timestamps to study the effects of evolving documents. We evaluate the retrieval … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 29 canonical work pages

  1. [1]

    In: WSDM

    Adar, E., Teevan, J., Dumais, S.T., Elsas, J.L.: The web changes everything: understanding the dynamics of web content. In: WSDM. pp. 282–291. ACM (2009)

    work page 2009

  2. [2]

    In: CLEF (Working Notes)

    Alkhalifa, R., Bilal, I.M., Borkakoty, H., Camacho-Collados, J., Deveaud, R., El-Ebshihy, A., Anke, L.E., Sáez, G.N.G., Galuscáková, P., Goeuriot, L., Kochkina, E., Liakata, M., Loureiro, D., Mulhem, P., Piroi, F., Popel, M., Servan, C., Madabushi, H.T., Zubiaga, A.: Extended overview of the CLEF-2023 longeval lab on longitudinal evaluation of model perfo...

    work page 2023

  3. [3]

    In: CLEF (Working Notes)

    Alkhalifa, R., Borkakoty, H., Deveaud, R., El-Ebshihy, A., Anke, L.E., Fink, T., Galuscáková, P., Sáez, G.G., Goeuriot, L., Iommi, D., Liakata, M., Madabushi, H.T., Medina-Alias, P., Mulhem, P., Piroi, F., Popel, M., Zubiaga, A.: Extended overview of the CLEF 2024 longeval lab on longitudinal evaluation of model performance. In: CLEF (Working Notes). CEUR...

    work page 2024

  4. [4]

    In: Faggioli, G., Ferro, N., Galuscáková, P., de Herrera, A.G.S

    Alkhalifa, R., Borkakoty, H., Deveaud, R., El-Ebshihy, A., Anke, L.E., Fink, T., Galuscáková, P., Sáez, G.G., Goeuriot, L., Iommi, D., Liakata, M., Madabushi, H.T., Medina-Alias, P., Mulhem, P., Piroi, F., Popel, M., Zubiaga, A.: Extended overview of the CLEF 2024 longeval lab on longitudinal evaluation of model performance. In: Faggioli, G., Ferro, N., G...

    work page 2024

  5. [5]

    In: Herzog, O., Schek, H., Fuhr, N., Chowdhury, A., Teiken, W

    Bernstein, Y., Zobel, J.: Redundant documents and search effectiveness. In: Herzog, O., Schek, H., Fuhr, N., Chowdhury, A., Teiken, W. (eds.) Proceedings of the 2005 ACM CIKM International Conference on Information and Knowledge Management, Bremen, Germany, October 31 - November 5, 2005. pp. 736–743. ACM (2005)

    work page 2005

  6. [6]

    ACM Comput

    Campos, R., Dias, G., Jorge, A.M., Jatowt, A.: Survey of temporal information retrieval and related applications. ACM Comput. Surv.47(2), 15:1–15:41 (2014)

    work page 2014

  7. [7]

    Synthesis Lectures on Information Concepts, Retrieval, and Services, Morgan & Claypool Publishers (2015)

    Chuklin, A., Markov, I., de Rijke, M.: Click Models for Web Search. Synthesis Lectures on Information Concepts, Retrieval, and Services, Morgan & Claypool Publishers (2015). https://doi.org/10.2200/S00654ED1V01Y201507ICR043

    work page doi:10.2200/s00654ed1v01y201507icr043 2015

  8. [8]

    In: SIGIR

    Dumais, S.T.: Putting searchers into search. In: SIGIR. pp. 1–2. ACM (2014)

    work page 2014

Show all 34 references
  1. [9]

    In: WSDM

    Elsas, J.L., Dumais, S.T.: Leveraging temporal dynamics of document content in relevance ranking. In: WSDM. pp. 1–10. ACM (2010)

    2010

  2. [10]

    In: Barrón-Cedeño, A., Da San Martino, G., Esposti, M.D., Sebastiani, F., Macdonald, C., Pasi, G., Hanbury, A., Potthast, M., Faggioli, G., Ferro, N

    Fröbe, M., Akiki, C., Potthast, M., Hagen, M.: Noise-reduction for automatically transferred relevance judgments. In: Barrón-Cedeño, A., Da San Martino, G., Esposti, M.D., Sebastiani, F., Macdonald, C., Pasi, G., Hanbury, A., Potthast, M., Faggioli, G., Ferro, N. (eds.) Experi...

    2022

  3. [11]

    In: Diaz, F., Shah, C., Suel, T., Castells, P., Jones, R., Sakai, T

    Fröbe, M., Bevendorff, J., Gienapp, L., Völske, M., Stein, B., Potthast, M., Hagen, M.: CopyCat: Near-Duplicates within and between the ClueWeb and the Common Crawl. In: Diaz, F., Shah, C., Suel, T., Castells, P., Jones, R., Sakai, T. (eds.) 44th International ACM Conference o...

    2021

  4. [12]

    In: 2nd Workshop on Reducing Online Misinformation through Credible Information Retrieval (ROMCIR 2022)

    Fröbe, M., Günther, S., Bondarenko, A., Huck, J., Hagen, M.: Using Keyqueries to Reduce Misinformation in Health-Related Search Results. In: 2nd Workshop on Reducing Online Misinformation through Credible Information Retrieval (ROMCIR 2022). CEUR Workshop Proceedings, CEUR-WS....

    2022

  5. [13]

    In: Chen, H.H., Duh, W., Huang, H.H., Kato, M., Mothe, J., Poblete, B

    Fröbe, M., Reimer, J.H., MacAvaney, S., Deckers, N., Reich, S., Bevendorff, J., Stein, B., Hagen, M., Potthast, M.: The Information Retrieval Experiment Platform. In: Chen, H.H., Duh, W., Huang, H.H., Kato, M., Mothe, J., Poblete, B. (eds.) 46th International ACM SIGIR Confere...

    2023

  6. [14]

    In: IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology

    Fröbe, M., Schmidt, E.O., Hagen, M.: Efficient query obfuscation with keyqueries. In: IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology. pp. 154–161 (2021)

    2021

  7. [15]

    In: Kamps, J., Goeuriot, L., Crestani, F., Maistro, M., Joho, H., Davis, B., Gurrin, C., Kruschwitz, U., Caputo, A

    Fröbe, M., Wiegmann, M., Kolyada, N., Grahm, B., Elstner, T., Loebe, F., Hagen, M., Stein, B., Potthast, M.: Continuous Integration for Reproducible Shared Tasks with TIRA.io. In: Kamps, J., Goeuriot, L., Crestani, F., Maistro, M., Joho, H., Davis, B., Gurrin, C., Kruschwitz, ...

    2023

  8. [16]

    In: Chen, H., Duh, W.E., Huang, H., Kato, M.P., Mothe, J., Poblete, B

    Galuscáková, P., Deveaud, R., Sáez, G.G., Mulhem, P., Goeuriot, L., Piroi, F., Popel, M.: Longeval-retrieval: French-english dynamic test collection for continuous web search evaluation. In: Chen, H., Duh, W.E., Huang, H., Kato, M.P., Mothe, J., Poblete, B. (eds.) Proceedings ...

    2023

  9. [17]

    In: Gurrin, C., Jones, G., Kelly, D., Kruschwitz, U., de Rijke, M., Sakai, T., Sheridan, P

    Gollub, T., Hagen, M., Michel, M., Stein, B.: From Keywords to Keyqueries: Content Descriptors for the Web. In: Gurrin, C., Jones, G., Kelly, D., Kruschwitz, U., de Rijke, M., Sakai, T., Sheridan, P. (eds.) 36th International ACM Conference on Research and Development in Infor...

    2013

  10. [18]

    In: Ferro, N., Crestani, F., Moens, M.F., Mothe, J., Silvestri, F., Di Nunzio, G., Hauff, C., Silvello, G

    Hagen, M., Beyer, A., Gollub, T., Komlossy, K., Stein, B.: Supporting Scholarly Search with Keyqueries. In: Ferro, N., Crestani, F., Moens, M.F., Mothe, J., Silvestri, F., Di Nunzio, G., Hauff, C., Silvello, G. (eds.) Advances in Information Retrieval. 38th European Conference...

    2016 doi

  11. [19]

    Kanhabua, N., Blanco, R., Nørvåg, K.: Temporal information retrieval. Found. Trends Inf. Retr.9(2), 91–208 (2015).https://doi.org/10.1561/1500000043

    2015 doi

  12. [20]

    In: Oosterhuis, H., Bast, H., Xiong, C

    Keller, J., Breuer, T., Schaer, P.: Evaluation of temporal change in IR test collections. In: Oosterhuis, H., Bast, H., Xiong, C. (eds.) Proceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR 2024, Washington, DC, USA, 13 July 2024...

    2024

  13. [21]

    In: Faggioli, G., Ferro, N., Galuscáková, P., de Herrera, A.G.S

    Keller, J., Breuer, T., Schaer, P.: Leveraging prior relevance signals in web search. In: Faggioli, G., Ferro, N., Galuscáková, P., de Herrera, A.G.S. (eds.) Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 10 Keller et al. 2024), Grenoble, France, 9-12 S...

    2024

  14. [22]

    In: Huang, J.X., Chang, Y., Cheng, X., Kamps, J., Murdock, V., Wen, J., Liu, Y

    Khattab, O., Zaharia, M.: Colbert: Efficient and effective passage search via contextualized late interaction over BERT. In: Huang, J.X., Chang, Y., Cheng, X., Kamps, J., Murdock, V., Wen, J., Liu, Y. (eds.) Proceedings of the 43rd International ACM SIGIR conference on researc...

    2020

  15. [23]

    Synthesis Lectures on Human Language Technologies, Morgan & Claypool Publishers (2021)

    Lin, J., Nogueira, R.F., Yates, A.: Pretrained Transformers for Text Ranking: BERT and Beyond. Synthesis Lectures on Human Language Technologies, Morgan & Claypool Publishers (2021). https://doi.org/10.2200/S01123ED1V01Y202108HLT053

    2021 doi

  16. [24]

    Springer (2011)

    Liu, T.: Learning to Rank for Information Retrieval. Springer (2011). https://doi.org/10.1007/978-3-642-14267-3

    2011 doi

  17. [25]

    In: Balog, K., Setty, V., Lioma, C., Liu, Y., Zhang, M., Berberich, K

    Macdonald, C., Tonellotto, N.: Declarative experimentation in information retrieval using pyterrier. In: Balog, K., Setty, V., Lioma, C., Liu, Y., Zhang, M., Berberich, K. (eds.) ICTIR ’20: The 2020 ACM SIGIR International Conference on the Theory of Information Retrieval, Vir...

    2020

  18. [26]

    In: Cohn, T., He, Y., Liu, Y

    Nogueira, R.F., Jiang, Z., Pradeep, R., Lin, J.: Document ranking with a pretrained sequence-to-sequence model. In: Cohn, T., He, Y., Liu, Y. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020. Findings of ACL, vol. ...

    2020 doi

  19. [27]

    (Jan 1994)

    Robertson, S., Walker, S., Jones, S., Hancock-Beaulieu, M., Gatford, M.: Okapi at TREC-3. (Jan 1994)

    1994

  20. [28]

    In: Kraaij, W., de Vries, A.P., Clarke, C.L.A., Fuhr, N., Kando, N

    Sakai, T.: Alternatives to bpref. In: Kraaij, W., de Vries, A.P., Clarke, C.L.A., Fuhr, N., Kando, N. (eds.) SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, July 23-...

    2007

  21. [29]

    SIGIR Forum33(1), 6–12 (1999)

    Silverstein, C., Henzinger, M., Marais, H., Moricz, M.: Analysis of a very large web search engine query log. SIGIR Forum33(1), 6–12 (1999)

    1999

  22. [30]

    In: SIGIR

    Soboroff, I.: Dynamic test collections: measuring search effectiveness on the live web. In: SIGIR. pp. 276–283. ACM (2006)

    2006

  23. [31]

    CoRRabs/2312.16098 (2023)

    Tamber, M.S., Pradeep, R., Lin, J.: Scaling down, litting up: Efficient zero-shot listwise reranking with seq2seq encoder-decoder models. CoRRabs/2312.16098 (2023). https://doi.org/10.48550/ARXIV.2312.16098

    2023 doi arXiv

  24. [32]

    In: SIGIR

    Tikhonov, A., Bogatyy, I., Burangulov, P., Ostroumova, L., Koshelev, V., Gusev, G.: Studying page life patterns in dynamical web. In: SIGIR. pp. 905–908. ACM (2013)

    2013

  25. [33]

    In: WSDM

    Tyler, S.K., Teevan, J.: Large scale query log analysis of re-finding. In: WSDM. pp. 191–200. ACM (2010)

    2010

  26. [2020]

    pp. 161–168. ACM (2020).https://doi.org/10.1145/3409256.3409829

    2020

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.