Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Are Recommenders Self-Aware? Label-Free Recommendation Performance Estimation via Model Uncertainty

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims a recommender can estimate its own ranking quality, without labels, from the uncertainty of the list it would generate.

desk verdict A useful heuristic for label-free ranking performance estimation, but the likelihood framing in Eq. (2) is mathematically shaky and should be fixed before publication. read the letter →

arxiv 2507.23208 v1 pith:4LX4554O submitted 2025-07-31 cs.IR cs.LG

Jiayu Li , Ziyi Ye , Guohao Jian , Zhiqiang Guo , Weizhi Ma , Qingyao Ai , Min Zhang This is my paper
classification cs.IRcs.LG
keywords recommendersystemsuncertaintyquantificationlist-wiselabel-freeperformanceestimationTop-Nrecommendationself-awarenessquerypredictionmatrixfactorization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a recommendation model can estimate its own ranking quality before any user sees the list, and answers yes in a specific sense. It proposes LiDu (List Distribution uncertainty), defined as the negative log-probability of the model generating the observed Top-N ranking, computed from per-item score means and variances. Across a synthetic matrix-factorization task and 30 settings from six real-world datasets and five recommenders, LiDu correlates negatively with NDCG@K: the best variant (LiDu-vb) reaches an average Pearson r of 0.222 and win rate 0.665, beating training loss and three QPP baselines. The paper also reports that LiDu tracks users' interest dynamism during training and list diversity at inference. A sympathetic reader would take the paper as establishing an empirical link between list-level self-uncertainty and actual recommendation performance.

What carries the argument

The central object is LiDu, shorthand for probability-based List Distribution uncertainty. It converts each item's predicted score and estimated variance into a Gaussian comparison probability, then aggregates pairwise comparison probabilities into an approximate likelihood for the generated Top-N ranking list; the estimator is the negative log of that list probability. To keep the tail of the list from dominating, the paper adds a step (only compare items sufficiently far apart) and a position bias inspired by NDCG. The variances feeding LiDu come from MC dropout, deep ensembles, or variational Bayesian last layers. This machinery carries the argument because it is the component that makes uncertainty a property of the rank list rather than of isolated predictions.

What would settle it

Synthesize a recommender where item scores share common noise factors, so pairwise comparisons are strongly dependent; compute LiDu and NDCG@K over many train/test splits and check whether the negative correlation persists when the true list likelihood differs sharply from the product approximation. If the correlation becomes positive or vanishes in a regime the paper claims LiDu generalizes to, the central claim would be refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that a recommender's uncertainty about the whole ranking list, rather than about individual scores, behaves as a label-free predictor of its ranking performance. The proposed measure LiDu assumes each item score is Gaussian with mean and variance from uncertainty quantification, computes pairwise probability $P(r_i > r_j)$ via Eq. (1), multiplies these pairwise probabilities to approximate the probability of the observed ranking (Eq. 2), and takes the negative log of that product (Eq. 3). The paper argues theoretically that under a matrix-factorization model trained with negative log-likelihood, optimal variance equals squared prediction error (Eq. 8), so high uncertainty aligns with low accuracy; it then demonstrates on six datasets that LiDu is more strongly correlated with NDCG@K than training loss, SMV, NQC, W-Graph, and point-wise variance. It further reports beyond-accuracy properties: higher LiDu for users with more dynamic interests and for more diversified lists.

Load-bearing premise

Everything rests on Eq. (2) treating the ranking-list probability as a product of independent pairwise comparisons, even though the paper's own Gaussian score model implies those comparisons are correlated; if that approximation is bad in a dataset, LiDu may still rank lists but is no longer the true list likelihood.

Editorial extensions

If this is right

  • A deployed recommender could rank its own output quality before showing it to users, flagging low-confidence lists for intervention.
  • Because LiDu works for inactive users in several datasets, it can serve as a signal for cold-user evaluation and personalized strategy adjustment.
  • LiDu's correlation with user interest dynamism and list diversity gives a training-time and inference-time diagnostic that standard ranking metrics do not provide.
  • The paper's success criteria imply downstream uses such as uncertainty-weighted ensembling, data augmentation from uncertain samples, and uncertainty-aware training losses, all named as future work in the paper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If list-level uncertainty truly tracks performance, then score variance in recommender embeddings encodes information that pointwise scores discard; this suggests a testable extension in which LiDu is computed from learned variance layers on item embeddings, not just user embeddings, to model cold items.
  • The independence approximation in Eq. (2) understates correlated errors across items; an exact joint Gaussian likelihood over the top-K scores might yield a strictly stronger estimator, providing a concrete next step the paper does not take.
  • One could combine LiDu with score-magnitude QPP features such as SMV and NQC in a simple regression, since they rely on different signals; the paper evaluates them separately and does not test such a fusion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes LiDu, a list-level uncertainty measure for Top-N recommendation, defined as the negative log probability (Eq. 3) of a ranking list under a Gaussian score model, where the list probability is computed as a product of pairwise comparison probabilities (Eq. 2). The authors validate LiDu as a label-free performance estimator on a synthetic matrix factorization task and on six real-world datasets with five recommenders, comparing against training loss and three QPP baselines. They report that LiDu correlates more strongly with NDCG than the baselines, and further analyze how LiDu varies with user activeness, interest dynamism, and recommendation diversity. The central empirical claim is that list-level uncertainty is a useful label-free performance estimator; the theoretical framing claims that LiDu is a true negative log likelihood of the ranking distribution.

Significance. If the empirical claim holds, the paper makes a useful contribution to a relatively underexplored problem: label-free performance estimation for Top-N recommendation. The study is reasonably broad, covering 30 model-dataset settings, three uncertainty implementations (MC dropout, deep ensemble, variational Bayesian), and comparisons with classical QPP baselines. The use of ReChorus and the public repository are positive reproducibility features. The main weakness is that the theoretical interpretation of LiDu as a probability is not mathematically justified, and the empirical advantages over baselines, while consistent, are modest in absolute terms. The paper is best seen as presenting a heuristic estimator with empirical support rather than a derived likelihood-based uncertainty measure.

major comments (4)
  1. [Section 3.1, Eqs. (1)-(3)] Equation (2) treats the probability of the ranking list as the product of pairwise probabilities π_{i,j}. Under the Gaussian score model of Eq. (1), and especially when MC dropout is applied to the shared user embedding, pairwise ranking events are dependent: the score perturbations of different items for the same user are correlated, so Var(s_i - s_j) is not generally σ_i^2 + σ_j^2 and P(s_i > s_j and s_i > s_k) ≠ π_{i,j}π_{i,k}. Consequently Eq. (2) is not the true joint probability of the list, and LiDu in Eq. (3) is not the negative log likelihood of any well-defined ranking distribution. Equation (11) adds position-bias exponents and an undefined "step" term, making the quantity a deliberately weighted heuristic rather than a probability. This matters because the paper's motivation and the label 'uncertainty' rely on the likelihood interpretation. The issue is fixable: either derive an exact list probability under the model, or explicitly present LiDu as an approximate/heuristic score and compare it against an exact-list-probability variant to show that the heuristic retains its empirical value.
  2. [Section 3.2.1] The Boltzmann argument is an analogy, not a derivation. Equation (4) asserts Q ∝ -E without justification, and Eq. (5) is the generic Boltzmann distribution with no demonstrated connection to LiDu or to the pairwise product in Eq. (2). The text calls this a 'Boltzmann Hypothesis' and treats it as theoretical support, but no quantitative bridge is provided. This does not invalidate the empirical study, but the paper should soften this to a heuristic motivation or provide a formal connection between ranking loss, energy, and list probability.
  3. [Table 3 and Section 5.1] The central empirical claim is supported by Table 3, but the average differences over the best baselines are small: win rate 0.654 for LiDu-dp vs. 0.629 for the best baseline, and −Pearson's r 0.198 vs. 0.159. Some individual settings clearly go against the trend, such as the ML-1M rows and SimpleX on Yelp. The significance markers (∗,†,‡) are not accompanied by a description of the statistical test, the number of test samples, or a multiple-comparison correction. Please report the number of ranking samples used in each correlation, provide confidence intervals or paired significance tests across users/seeds, and clarify whether the comparisons are performed on paired samples.
  4. [Sections 4.3.4 and 5.2] The hyperparameter selection for the estimators appears to use the evaluation labels. For QPP baselines, the top-k value is tuned over {10, 100, 1000} to the best estimation; for LiDu, N is set to 10 for XING and 100 for other datasets after inspecting performance, and the sensitivity analysis in Section 5.2 motivates L=1000. If these choices are made using the same test NDCG values used for evaluation, the comparison is not fully label-free and may be optimistic. Please specify clearly whether any tuning is performed on a separate validation split, and if not, describe how the dataset-dependent N values were chosen without using test labels.
minor comments (6)
  1. [Eq. (11)] The term s_n = high_bit(n) + n is undefined; the paper should define high_bit and explain why this step function is used. Also, the position-bias exponent is written as p_n = log2(i+1), but the index i is not defined in that context; it should presumably be p_n = log2(n+1).
  2. [Table 3 caption] The caption says 'The negative Pearson's r is reported', but the table shows positive values. Please state explicitly that the absolute value of Pearson's r is reported, or change the column labels accordingly.
  3. [Section 4.3.2] In the definition of Win Rate-δ, the conditioning on e_i > e_j + δ and the choice δ = 5% of the test set size need more explanation; in particular, how ties and near-ties are handled should be stated, and the claim that chance performance is 0.5 should be justified for the conditional event.
  4. [Section 4.3.1] The paper omits point-wise uncertainty baselines after reporting that they show insignificant correlation. For reproducibility and completeness, at least one such baseline (e.g., average prediction variance over top items) should be included in the main table or the supplementary material.
  5. [Section 5.3 / supplementary material] The text states that sARE results are presented in the supplementary information, but no supplementary file is included with the manuscript. Either include the supplementary results or remove the reference to them.
  6. [Figure 1] The caption should clarify that the figure shows Pearson's correlation between LiDu and ranking performance for BPRMF, and should define the 'Ranking Uncertainty' axis; currently the reader must infer the setup from later sections.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: LiDu's correlation with NDCG is empirical, the NLL identity in Eq. (7) is a standard loss property rather than a hidden fit, and the only self-citation is to the public ReChorus implementation toolkit.

full rationale

The paper's central claim is that the list-distribution uncertainty LiDu is empirically correlated with NDCG and other ranking metrics. LiDu is defined by Eqs. (1)-(3) and (11) directly from model score means and variances; no parameter is fitted to NDCG, and the reported correlations are evaluated on held-out test ranking lists against NDCG@K. The theoretical motivation in Eq. (7) derives that an optimally NLL-trained Gaussian variance equals the squared error; this is a mathematical property of the assumed loss, not a hidden circular step, and the actual LiDu implementations (MC dropout, ensemble, variational Bayesian) do not train a variance to match NDCG. The only self-citation, ReChorus [25], is an open-source toolkit used to implement standard baselines and does not carry the argument. The use of NDCG's position discount in Eq. (11) is a heuristic weighting, not an identity with NDCG, and the empirical comparison spans 30 model-dataset settings; it does not reduce to a definitional equivalence. The independence approximation in Eq. (2) is a statistical-correctness concern, not a circularity. Thus no circular step is exhibited.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central empirical claim (LiDu correlates with performance) relies on the Gaussian score assumption and the product-form list probability in Eq. (2), which is an approximation. The Boltzmann analogy and NLL variance derivation are motivational, not load-bearing for the empirical results. Free parameters are mostly standard UQ hyperparameters, with the per-dataset N and L choices being the most consequential.

free parameters (7)
  • N (number of top items in LiDu) = 100 (10 for XING)
    Number of top items included in the ranking probability; set per dataset, with XING using a smaller value because its performance is much higher. Chosen by hand, not from a formal criterion.
  • L (max pairwise items) = 1000 (100 for XING)
    Limits the pairwise comparisons in Eq. (11); fixed per dataset group.
  • MC dropout probability p = 0.2
    Dropout rate used for variance estimation in LiDu-dp; standard choice, not fitted.
  • Number of forward passes T = 50 (20 in MF experiment)
    Number of stochastic passes for MC dropout variance estimation.
  • Ensemble size = 5
    Number of independently initialized models for LiDu-en.
  • Zipf exponent alpha = 5
    Used to sample the synthetic MF training set to mimic long-tailed interaction data; set empirically.
  • Win-rate delta = 5% of test set size
    Threshold to discard close estimator comparisons; chosen, not varied.
assumptions (5)
  • domain assumption Each item score is an independent Gaussian random variable with known mean and variance (Eq. 1).
    LiDu builds on this to define pairwise ranking probabilities; if scores are not independent Gaussians, Eq. (1) does not hold.
  • ad hoc to paper Pairwise ranking events are independent, so the product in Eq. (2) equals the probability of the whole ranking list.
    The paper presents Eq. (2) as the list probability without noting this independence assumption; it is generally false for Gaussian scores, making LiDu an approximate heuristic.
  • ad hoc to paper Boltzmann distribution with energy equal to negative quality describes ranking probability (Eqs. 4-5).
    Used as motivational analogy for the negative correlation; not derived from recommender dynamics.
  • standard math NLL-trained models set variance equal to squared error (Eq. 7).
    Standard result when optimizing Gaussian NLL; used to connect pointwise uncertainty to error, not list-level LiDu.
  • domain assumption Leave-one-out last interaction splitting approximates true recommendation performance.
    Standard evaluation in sequential recommendation literature; the paper relies on it for ground truth labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are Recommenders Self-Aware? Label-Free Recommendation Performance Estimation via Model Uncertainty." pith.science (2026). https://pith.science/paper/4LX4554O

@misc{pith2026250723208,
  author       = {Pith},
  title        = {Pith review of: Are Recommenders Self-Aware? Label-Free Recommendation Performance Estimation via Model Uncertainty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4LX4554O}},
  note         = {Machine review of arXiv:2507.23208}
}
read the original abstract

Can a recommendation model be self-aware? This paper investigates the recommender's self-awareness by quantifying its uncertainty, which provides a label-free estimation of its performance. Such self-assessment can enable more informed understanding and decision-making before the recommender engages with any users. To this end, we propose an intuitive and effective method, probability-based List Distribution uncertainty (LiDu). LiDu measures uncertainty by determining the probability that a recommender will generate a certain ranking list based on the prediction distributions of individual items. We validate LiDu's ability to represent model self-awareness in two settings: (1) with a matrix factorization model on a synthetic dataset, and (2) with popular recommendation algorithms on real-world datasets. Experimental results show that LiDu is more correlated with recommendation performance than a series of label-free performance estimators. Additionally, LiDu provides valuable insights into the dynamic inner states of models throughout training and inference. This work establishes an empirical connection between recommendation uncertainty and performance, framing it as a step towards more transparent and self-evaluating recommender systems.

Figures

Figures reproduced from arXiv: 2507.23208 by the authors.

Figure 1
Figure 1. Empirical results suggest ranking performance of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Results of the matrix factorization experiment on the synthetic data: (a) Uncertainty presents a negative correlation [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The absolute number of negative Pearson’s [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The win rate of LiDu-dp estimating NDCG@1000 under different values of [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Average LiDu-dp of users with (A) different interest [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Uncertainty as Remedy: Mitigating Satisfaction Label Bias in Short Video Multi-Objective Ensemble Ranking

    cs.IR 2026-07 conditional novelty 6.0 of 10

    UAME models each predicted satisfaction score as a Gaussian with learned variance, then uses that variance to up-weight conflicting item pairs in a probabilistic pairwise ranking loss, improving multi-objective short-...

Reference graph

Works this paper leans on

48 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    Fabian Abel, Yashar Deldjoo, Mehdi Elahi, and Daniel Kohlsdorf. 2017. Recsys challenge 2017: Offline and online evaluation. In Proceedings of the eleventh acm conference on recommender systems . 372–373

    work page 2017

  2. [2]

    Negar Arabzadeh, Amin Bigdeli, Morteza Zihayat, and Ebrahim Bagheri. 2021. Query performance prediction through retrieval coherency. In Advances in In- formation Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28–April 1, 2021, Proceedings, Part II 43 . Springer, 193–200

    work page 2021

  3. [3]

    Christine Bauer, Eva Zangerle, and Alan Said. 2024. Exploring the landscape of recommender systems evaluation: Practices and perspectives. ACM Transactions on Recommender Systems 2, 1 (2024), 1–31

    work page 2024

  4. [4]

    Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight uncertainty in neural network. In International conference on machine learning. PMLR, 1613–1622

    work page 2015

  5. [5]

    Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, and Carsten Eickhoff

  6. [6]

    Victor Coscrato and Derek Bridge. 2022. Recommendation uncertainty in implicit feedback recommender systems. In Irish Conference on Artificial Intelligence and Cognitive Science. Springer, 279–291

    work page 2022

  7. [7]

    Victor Coscrato and Derek Bridge. 2023. Estimating and evaluating the uncer- tainty of rating predictions and top-n recommendations in recommender systems. ACM Transactions on Recommender Systems 1, 2 (2023), 1–34

    work page 2023

  8. [8]

    Guglielmo Faggioli, Thibault Formal, Stefano Marchesin, Stéphane Clinchant, Nicola Ferro, and Benjamin Piwowarski. 2023. Query performance prediction for neural IR: Are we there yet?. In European Conference on Information Retrieval . Springer, 232–248

    work page 2023

Show all 48 references
  1. [9]

    Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning. PMLR, 1050–1059

    2016

  2. [10]

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In International conference on machine learning . PMLR, 1321–1330

    2017

  3. [11]

    James Harrison, John Willes, and Jasper Snoek. 2024. Variational Bayesian last layers. arXiv preprint arXiv:2404.11599 (2024)

    2024 arXiv

  4. [12]

    Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval ...

    2020

  5. [13]

    Achim Hekler, Titus J Brinker, and Florian Buettner. 2023. Test time augmen- tation meets post-hoc calibration: uncertainty quantification under real-world conditions. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 37. 14856–14864

    2023

  6. [14]

    Maria Heuss, Daniel Cohen, Masoud Mansoury, Maarten de Rijke, and Carsten Eickhoff. 2023. Predictive uncertainty-based bias mitigation in ranking. In Pro- ceedings of the 32nd ACM International Conference on Information and Knowledge Management. 762–772

    2023

  7. [15]

    Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley

  8. [16]

    Wenbo Hu, Xin Sun, Le Wu, Liang Wang, et al. 2023. Uncertainty calibration for counterfactual propensity estimation in recommendation. arXiv preprint arXiv:2303.12973 (2023)

    2023

  9. [17]

    Jingyuan Huang, Xi Zhu, Minghao Guo, and Yongfeng Zhang. 2025. DeSocial: Blockchain-based Decentralized Social Networks. arXiv preprint arXiv:2505.21388 (2025)

    2025

  10. [18]

    Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS) 20, 4 (2002), 422–446

    2002

  11. [19]

    Junyang Jiang, Deqing Yang, Yanghua Xiao, and Chenlu Shen. 2020. Convolu- tional gaussian embeddings for personalized recommendation with uncertainty. arXiv preprint arXiv:2006.10932 (2020)

    2020 arXiv

  12. [20]

    Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recom- mendation. In 2018 IEEE international conference on data mining (ICDM) . IEEE, 197–206

    2018

  13. [21]

    Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems 30 (2017)

    2017

  14. [22]

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30 (2017)

    2017

  15. [23]

    Lev Davidovich Landau and Evgenii Mikhailovich Lifshitz. 2013. Statistical Physics: Volume 5. Vol. 5. Elsevier

    2013

  16. [24]

    Dong Li, Ruoming Jin, Zhenming Liu, Bin Ren, Jing Gao, and Zhi Liu. 2024. On item-sampling evaluation for recommender system. ACM Transactions on Recommender Systems 2, 1 (2024), 1–36

    2024

  17. [25]

    Jiayu Li, Hanyu Li, Zhiyu He, Weizhi Ma, Peijie Sun, Min Zhang, and Shaoping Ma. 2024. Rechorus2. 0: A modular and task-flexible recommendation library. In Proceedings of the 18th ACM Conference on Recommender Systems . 454–464

    2024

  18. [26]

    Hongyu Lu, Min Zhang, and Shaoping Ma. 2018. Between clicks and satisfaction: Study on multi-phase user preferences and satisfaction for online news read- ing. In The 41st international acm sigir conference on research & development in information retrieval. 435–444

    2018

  19. [27]

    Andrey Malinin and Mark Gales. 2018. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems 31 (2018)

    2018

  20. [28]

    Kelong Mao, Jieming Zhu, Jinpeng Wang, Quanyu Dai, Zhenhua Dong, Xi Xiao, and Xiuqiang He. 2021. SimpleX: A simple and strong baseline for collaborative filtering. In Proceedings of the 30th ACM international conference on information & knowledge management. 1243–1252

    2021

  21. [29]

    Andres Masegosa. 2020. Learning under model misspecification: Applications to variational and ensemble methods. Advances in Neural Information Processing Systems 33 (2020), 5479–5491

    2020

  22. [30]

    Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. 2021. Revisiting the calibration of modern neural networks. Advances in neural information processing systems 34 (2021), 15682–15694

    2021

  23. [31]

    Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme

  24. [32]

    Phyllis A Richmond. 1963. Review of the cranfield project. American Documen- tation (pre-1986) 14, 4 (1963), 307

    1963

  25. [33]

    Murat Sensoy, Lance Kaplan, and Melih Kandemir. 2018. Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems 31 (2018)

    2018

  26. [34]

    Anna Shtok, Oren Kurland, David Carmel, Fiana Raiber, and Gad Markovits. 2012. Predicting query performance by query-drift estimation. ACM Transactions on Information Systems (TOIS) 30, 2 (2012), 1–35

    2012

  27. [35]

    Yongquan Tao and Shengli Wu. 2014. Query performance prediction by consid- ering score magnitude and variance together. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management . 1891–1894

    2014

  28. [36]

    Ellen M Voorhees et al. 1999. The trec-8 question answering track report.. In Trec, Vol. 99. 77–82

    1999

  29. [37]

    Chenxu Wang, Fuli Feng, Yang Zhang, Qifan Wang, Xunhan Hu, and Xiangnan He. 2023. Rethinking missing data: Aleatoric uncertainty-aware recommendation. IEEE Transactions on Big Data 9, 6 (2023), 1607–1619

    2023

  30. [38]

    Chenyang Wang, Zhefan Wang, Yankai Liu, Yang Ge, Weizhi Ma, Min Zhang, Yiqun Liu, Junlan Feng, Chao Deng, and Shaoping Ma. 2022. Target interest distillation for multi-interest recommendation. In Proceedings of the 31st ACM international conference on information & knowledge m...

    2022

  31. [39]

    Jun Yang and Fei Wang. 2020. Auto-ensemble: An adaptive learning rate schedul- ing based deep learning model ensembling. IEEE Access 8 (2020), 217499–217509

    2020

  32. [40]

    Tao Yang, Cuize Han, Chen Luo, Parth Gupta, Jeff M Phillips, and Qingyao Ai

  33. [41]

    Ziyi Ye, Xiaohui Xie, Qingyao Ai, Yiqun Liu, Zhihong Wang, Weihang Su, and Min Zhang. 2024. Relevance Feedback with Brain Signals. ACM Transactions on Information Systems 42, 4 (2024), 1–37

    2024

  34. [42]

    Ziyi Ye, Xiaohui Xie, Yiqun Liu, Zhihong Wang, Xuancheng Li, Jiaji Li, Xuesong Chen, Min Zhang, and Shaoping Ma. 2022. Why Don’t You Click: Understanding Non-Click Results in Web Search with Brain Signals. In Proceedings of the 45th International ACM SIGIR Conference on Resear...

    2022

  35. [43]

    Fan Zhou, Tangjiang Qian, Yuhua Mo, Zhangtao Cheng, Chunjing Xiao, Jin Wu, and Goce Trajcevski. 2023. Uncertainty-aware heterogeneous representation learning in POI recommender systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems 53, 7 (2023), 4522–4535

    2023

  36. [44]

    In Proceedings of the ACM Web Conference 2024

    Mitigating exploitation bias in learning to rank with an uncertainty-aware empirical Bayes approach. In Proceedings of the ACM Web Conference 2024 . 1486– 1496

    2024

  37. [48]

    Feng Zhu, Chaochao Chen, Yan Wang, Guanfeng Liu, and Xiaolin Zheng. 2019. DTCDR: A framework for dual-target cross-domain recommendation. In Pro- ceedings of the 28th ACM International Conference on Information and Knowledge Management. 1533–1542

    2019

  38. [2012]

    arXiv preprint arXiv:1205.2618 (2012)

    BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012)

    2012 arXiv

  39. [2021]

    In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Not all relevance scores are equal: Efficient uncertainty and calibration modeling for deep retrieval models. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval . 654–664

  40. [2024]

    arXiv preprint arXiv:2403.03952 (2024)

    Bridging language and items for retrieval and recommendation. arXiv preprint arXiv:2403.03952 (2024)

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.