Uncertainty as Remedy: Mitigating Satisfaction Label Bias in Short Video Multi-Objective Ensemble Ranking

Jiaqi Yu; Jinfang Gu; Kaiqiao Zhan; Kun Gai; Minzhi Xie; Tiantian He; Xiaoxiao Xu; Yongqi Liu; Zonghe Shao

REVIEW 3 major objections 5 minor 43 references

Uncertainty as Remedy: Mitigating Satisfaction Label Bias in Short Video Multi-Objective Ensemble Ranking

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In short-video ranking, conflicts among behavioral proxy labels create an uncertainty signal that tracks how far those labels sit from true satisfaction, and reweighting training pairs by this uncertainty reduces label bias.

desk verdict A credible industrial uncertainty-weighting recipe with real A/B gains, but the theory that it targets label bias rather than just hard examples is assumed, not shown. read the letter →

arxiv 2607.17092 v1 pith:KULQQTVK submitted 2026-07-19 cs.IR

Zonghe Shao , Tiantian He , Xiaoxiao Xu , Jiaqi Yu , Minzhi Xie , Jinfang Gu , Yongqi Liu , Kaiqiao Zhan

show 1 more author

Kun Gai

This is my paper

classification cs.IR

keywords uncertainty-awarerankingmulti-objectiveensemblesatisfactionlabelbiasprobabilisticpairwiselosssamplereweightingshortvideorecommendationuseralignmentGaussianscoringmodel

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Short-video recommender systems are trained on behavioral signals such as clicks, watch time, and likes, which are fragmented proxies for a user's true satisfaction and often contradict one another. This paper argues that those conflicts produce both predictive uncertainty and a sample-level label bias, and that the two move together: item pairs whose proxy rankings disagree are exactly the pairs whose averaged label is most likely to diverge from what the user actually felt. On that basis the paper builds UAME, which represents each predicted satisfaction score as a Gaussian—mean as score, variance as uncertainty—and inserts an uncertainty-dependent weight into a probabilistic pairwise ranking loss, so high-conflict pairs get larger gradients. A risk decomposition shows equal-weight training minimizes proxy risk rather than true risk, and Proposition 3.2 formalizes the positive association between proxy-bias magnitude and pairwise uncertainty. Offline and online experiments on two large-scale ranking backbones report consistent gains, including better alignment with questionnaire-based satisfaction, with no extra serving latency because only the mean score is used online.

What carries the argument

The engine is the Gaussian scoring variable y_hat_i = mu_i + sigma_i * epsilon_i, which turns a single deterministic ranking score into a distribution. The pairwise win probability becomes the Gaussian CDF Phi((mu_i - mu_j)/sqrt(sigma_i^2 + sigma_j^2)), giving a probabilistic pairwise ranking loss that trains mean and variance jointly. Pairwise comprehensive uncertainty U_ij = sigma_i^2 + sigma_j^2 is max-min normalized and scaled by gamma to form the adaptive sample weight in the weighted loss; an auxiliary pairwise loss forces the predicted variance to track measured pxtr ranking conflict, and a log(1 + sigma_i^2 + sigma_j^2) regularizer stops variance from inflating to game the loss. Onli

What would settle it

Run UAME on logged data where true satisfaction labels are available (e.g., questionnaire responses) but create a synthetic set of proxy labels whose conflicts are random noise independent of true satisfaction, then compare UAME against a uniform-weight control on ranking quality against the true labels. If UAME no longer beats control, the conflict-bias monotonicity assumption is what carried the result; if it still wins, uncertainty is doing something else, such as regularizing variance.

Watch

Extended reading notes

Core claim

The central claim is that satisfaction-label bias in end-to-end multi-objective ensemble ranking is not a separate problem from predictive uncertainty: the same pxtr conflicts that make a pair of videos hard to rank consistently also make the averaged behavioral label for that pair unreliable as a guide to true satisfaction. Formally, the paper decomposes true ranking risk as proxy risk plus a proxy-bias term, bounds the sample-level bias by the mismatch between the true label and the averaged proxy label (Lemma 3.1), and then argues (Proposition 3.2) that this bias magnitude is positively associated in expectation with pairwise uncertainty U_ij = sigma_i^2 + sigma_j^2, through their common

Load-bearing premise

The load-bearing premise is that when behavioral proxy labels conflict, the averaged label is on average farther from the user's true satisfaction (Appendix A.3, Eq. 30), and that model uncertainty faithfully tracks that conflict (Assumption 2); if conflicting proxies are merely independent noise, upweighting high-uncertainty pairs will not reduce true label bias.

Editorial extensions

If this is right

If the central claim is correct, any end-to-end multi-objective ranking model can be made more satisfaction-aligned by adding the uncertainty branch and weighting scheme, without changing the online serving path.
The reported offline results imply up to 5.8% relative GAUC gains on one backbone and 14.2% on another, with the largest gains on long-view and watch-time objectives.
The online A/B results imply the effect transfers to production traffic: statistically significant increases in long-term retention, app stay time, watch time, and engagement metrics in both test groups.
Questionnaire-based evaluation implies the model ranks items closer to what users say they feel, not just closer to behavioral proxies: NDCG@5 and HR@5 improve by 8.7% and 10.6% respectively over a questionnaire-aligned baseline.
Because uncertainty and bias are positively associated, the learned sigma^2 itself becomes a diagnostic: a cheap, always-on indicator of where proxy labels are most untrustworthy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same conflict-to-bias bridge should hold in other multi-objective ranking domains—search, e-commerce, feed recommendations—where click and dwell-time labels disagree; a natural test is to run UAME on a public ranking dataset with known relevance judgments.
Editorial inference: the paper's monotonicity assumption (Eq. 30) is testable in isolation: with logged proxy labels plus a gold-standard satisfaction sample, one can directly estimate the expected absolute gap between true label and averaged proxy label as a function of conflict level and check it is non-decreasing; if it is not, the weighting would need a different justification.
Editorial inference: the auxiliary loss supervises uncertainty with pxtr conflict, which means UAME is implicitly learning to predict a conflict feature; an ablation that removes the auxiliary loss would show how much of the gain comes from the weighting mechanism itself versus from the conflict signal.
Editorial inference: a stronger version of the paper would use the learned variance to calibrate a confidence interval around the final ranking score, not just as a training weight; that extension is left to future work.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a credible industrial recipe—score as Gaussian, variance trained to reflect cross-pxtr conflict, upweight high-variance pairs—and the deployment numbers are consistent. But the paper's theoretical claim that this weighting mitigates true-satisfaction bias is not actually derived. The authors should be pushed to fix that or reposition the paper.

The genuinely new piece is combining uncertainty-weighted pairwise loss with an auxiliary loss that pins the variance to pxtr conflict, in an end-to-end multi-objective ranking model. That combination is not in the cited literature, and the design is clean: the uncertainty is only used during training, so serving latency is unchanged. The experiments are about as good as one can do without releasing data: two internal backbones (EMER, EASQ), eight pxtr metrics, online A/B with p<0.005 on all reported engagement metrics, and a questionnaire-based analysis. The ablation against ANSL (loss-based weighting) is a nice control; it suggests the gain isn't just generic hard-example mining.

The soft spots are real, and they are the ones the stress-test flags. Proposition 3.2 is the load-bearing justification for interpreting the weights as bias-correcting, and its proof sketch in Appendix A.3 relies on Eq. (30)—the assumption that expected true-vs-proxy label deviation is monotone in pxtr conflict. That is the conclusion in different clothes, and it is never justified. Worse, Assumption 2 (uncertainty tracks conflict) is partly manufactured by the auxiliary loss L_aux of Eq. (6), so the Pearson 0.65 correlation in Sec. 4.5.2 partly measures a trained-in alignment rather than an intrinsic connection. If conflicting pxtr labels are just noise, upweighting high-uncertainty pairs does not target bias—it amplifies noise, and the method reduces to a different version of hard-example mining. The offline tables also lack confidence intervals, and no code or data are provided, so nothing is independently checkable from the preprint.

None of that makes the paper unserious. The empirical gains are plausible and the engineering is careful enough that I'd bet the method works. But the title's promise—uncertainty as a remedy for satisfaction label bias—is supported by a rationale, not a proof. A careful referee should ask the authors to either prove the monotonicity condition, or remove the word 'remedy' and present the method as an effective uncertainty-aware weighting heuristic.

Who this is for: industrial recommender-systems researchers and practitioners who want a lightweight, deployable tweak to a multi-objective ranking pipeline. I'd send it to review; it deserves referee time. With a softened theory and preferably a public benchmark or released artifacts, it could be a solid conference paper.

Referee Report

3 major / 5 minor

Summary. The paper proposes UAME, an uncertainty-aware end-to-end multi-objective ensemble ranking framework for short-video recommendation. The model outputs a Gaussian scoring variable per item: the mean estimates satisfaction and the variance quantifies conflict-driven uncertainty. A probabilistic pairwise ranking loss, an auxiliary loss that aligns the predicted variance with pxtr-conflict levels, and an uncertainty-based sample weighting scheme are combined into the training objective. The authors provide a risk-decomposition analysis (Lemma 3.1, Proposition 3.2) to argue that weighting by uncertainty targets sample-level satisfaction-label bias, and they report offline GAUC gains on EMER and EASQ backbones, online A/B improvements with p<0.005, and a questionnaire-based correlation (Pearson 0.65) between learned uncertainty and a constructed label-bias measure.

Significance. The problem is practically important: satisfaction labels in short-video ranking are noisy and conflicting, and existing uncertainty-aware methods mostly adjust scores at inference rather than the training loss. The industrial-scale offline and online validation is a genuine strength, especially the deployment on two strong backbones with no added serving latency. Lemma 3.1 is correctly derived and the probabilistic pairwise loss is a clean formulation. However, the central theoretical claim that uncertainty weighting mitigates satisfaction-label bias is not fully established: it rests on an unproven monotonicity condition (Eq. 30) and an assumption (Assumption 2) that is partly trained into the model by the auxiliary loss. The empirical correlation in Sec. 4.5.2 is suggestive but confounded. If the gap can be closed or reframed with direct validation, the paper would be a solid industrial contribution.

major comments (3)

[Sec. 3.5.2 / Appendix A.3] Proposition 3.2 is not derived from Lemma 3.1 and Assumption 2 alone. The proof in Appendix A.3 introduces Eq. (30), which assumes g_m(c1) <= g_m(c2): the expected mismatch |y*_ij - averaged pxtr label| is non-decreasing in pxtr conflict. This is a substantive assumption about the unobservable true satisfaction label, and it is effectively the conclusion the proposition needs to show. Without Eq. (30), the argument only proves that both |Δ_ij| and U_ij are monotone in conflict if each is individually assumed to be. Since Sec. 3.5.3 uses Proposition 3.2 to justify the uncertainty weighting, this is a load-bearing gap. Please either prove Eq. (30) from a plausible data-generating process or restate it as an explicit assumption and validate it directly with questionnaire data.
[Eq. (6) / Sec. 4.5.2] The reported positive correlation between learned uncertainty and satisfaction label bias is partly confounded. The auxiliary loss L_aux (Eq. 6) explicitly trains sigma^2 to match pxtr-conflict levels, and the bias measure B_ij = |y_q*_ij - (1/K)Σ_k y^(k)_ij| is built from averaged pxtr labels, so high conflict mechanically pushes the average label toward 0.5 and can inflate B_ij even if the true label is independent of conflict. Thus the Pearson/Spearman values in Sec. 4.5.2 may reflect the trained-in alignment between sigma^2 and conflict rather than an intrinsic uncertainty-bias link. Please provide a control: e.g., compute the correlation with L_aux ablated, partial out pxtr conflict, or evaluate on pairs where sigma^2 comes from a model not trained to mimic conflict.
[Table 1 / Sec. 4.2] The offline GAUC results are reported as point estimates with no confidence intervals, standard errors, or significance tests. Some metrics on EASQ are not improved (e.g., pcmtr 0.703 to 0.695; pltr 0.682 to 0.684 is within noise), so the phrase 'consistent gains across a majority of pxtr metrics' is not statistically supported online/offline. The 'statistically significant improvements across all metrics' claim is made only for the online A/B test (Table 2). Please add paired significance tests or confidence intervals for Table 1, or qualify the offline claims accordingly. The online A/B test is the stronger evidence, but the offline table should not overstate its precision.

minor comments (5)

[Sec. 3.3.3] The construction of B+_aux is underspecified: 'item pairs where v_i exhibits significantly higher conflict' has no concrete threshold or definition. Please state how the conflict level is computed and how the threshold is chosen.
[Eq. (8) / Algorithm 1] The max-min normalization uses min(U) and max(U) over the current batch, but Eq. (8) omits the epsilon added in Algorithm 1 to avoid division by zero. Clarify whether epsilon is used in the main text and what happens if all pairs in a batch have equal uncertainty.
[Table 1] The ANSL row appears to have concatenated values ('0.6660.7150.670'), likely a rendering issue. Please fix the table formatting.
[Sec. 4.5.2] The correlation is computed on 3 million samples, but no details are given on how the item-pair samples were selected or whether they overlap with the training set used to fit sigma^2. Reporting a bootstrap confidence interval for the Pearson/Spearman values would strengthen the claim.
[Appendix A.3] The function g_Δ(c) is defined but not actually used in the sketch; the proof would be clearer if the monotonicity of g_Δ were stated explicitly as the needed condition and then tied to Eq. (30).

Circularity Check

2 steps flagged · score 6.0 of 10

The theoretical rationale for uncertainty weighting reduces to an assumed monotonicity (Eq. 30) plus a trained-in conflict correlation; the empirical gains remain independent evidence.

self definitional [Appendix A.3, Eq. (30)]
"We use the following monotonic characterization to formalize the relation between pxtr conflict and the unobservable proxy-label mismatch: g_m(c1) ≤ g_m(c2), ∀ c1 ≤ c2 (30)"

The proof of Proposition 3.2 needs exactly this monotonicity: the expected deviation between the true satisfaction label and the averaged pxtr label must be non-decreasing in pxtr conflict. The paper calls it a 'characterization' but neither derives it from Lemma 3.1 nor from Assumptions 1–2. If conflicting pxtr labels are independent noise, g_m can be flat or decreasing, and the claimed positive association between |Δ_ij| and U_ij fails. Thus the theoretical 'prediction' is the assumption itself, not an independent derivation.
fitted input called prediction [Sec. 3.3.3 Eq. (6); Sec. 3.5.2 Assumption 2]
"we construct a pairwise training set B+_aux, which consists exclusively of item pairs (i,j) where item v_i exhibits significantly higher conflict than item v_j. We adopt a pairwise logistic loss ... Laux = − Σ_{(i,j)∈B+_aux} log(sigmoid(σ_i^2−σ_j^2)) (6) ... Assumption 2. Under the optimization of the PPR loss in equation (4), for an item pair (v_i, v_j), U_ij is positively correlated, in expectation, with its level of pxtrs conflict."

The auxiliary loss explicitly supervises σ^2 to be larger for items with higher pxtr conflict, so the positive correlation asserted in Assumption 2 is a training target, not an independently discovered property. Proposition 3.2 then composes this trained-in correlation with the assumed Eq. (30). The questionnaire correlation (Pearson 0.65, Spearman 0.67) is suggestive external evidence, but B_ij = |y_q*_ij − (1/K)Σ_k y^(k)_ij| shares the averaged pxtr label with the conflict signal used to train σ^2, so part of the observed association is induced by construction.

full rationale

The paper's strongest claimed derivation—that uncertainty-aware weights target satisfaction label bias—is not an independent theoretical result. Proposition 3.2 is stated under Assumptions 1–2, but its proof in Appendix A.3 additionally requires Eq. (30), a monotonicity between pxtr conflict and the expected deviation of the averaged pxtr label from the true satisfaction label. This is exactly the unverified premise that makes the weighting scheme bias-mitigating; it is introduced as a 'monotonic characterization' and not derived. Moreover, L_aux (Eq. 6) explicitly trains σ^2 to rank items by pxtr conflict, so Assumption 2's asserted correlation with conflict is effectively a training target. The empirical questionnaire correlation (Pearson 0.65, Spearman 0.67) is suggestive but partly confounded because the bias measure B_ij shares the averaged pxtr label with the conflict signal used to train σ^2. No code or data are released, so Eq. (30) cannot be independently checked. The offline/online gains and ablations are self-contained and could hold even if the stated rationale were replaced by generic hard-example mining or a probabilistic-loss effect; hence the circularity is partial and concentrated in the theoretical framing, not in the experimental claim. No load-bearing self-citation is present: EMER and EASQ are prior baselines, and the questionnaire source is not used to derive the theory.

Assumptions & free parameters 4 free parameters · 6 assumptions · 2 invented entities

All load-bearing premises are listed. The central mathematical contribution rests on two assumptions that substantially overlap with the conclusion (Assumption 2 and Eq. 30), plus standard technical conditions. The method has three tuned hyperparameters and one unspecified conflict-selection threshold.

free parameters (4)

gamma = 2
Uncertainty weight scaling factor in Eq. (8); tuned on the validation set (Fig. 3c).
alpha = 0.02
Uncertainty regularization coefficient in Eq. (9); tuned on the validation set (Fig. 3a).
beta = 0.1
Auxiliary loss coefficient in Eq. (9); tuned on the validation set (Fig. 3b).
pxtr conflict threshold for B+_aux
Constructing pairs where item i has 'significantly higher' conflict than item j (Sec. 3.3.3) requires an unquantified selection threshold.

assumptions (6)

standard math Assumption 1: the pairwise ranking loss ℓ(p,y) is L-Lipschitz in the soft label y for p∈[ε,1−ε].
Used in Lemma 3.1 to bound |Δ_ij|; log loss is not globally Lipschitz but is Lipschitz on the bounded domain, so this is a standard technical condition.
ad hoc to paper Assumption 2: pairwise uncertainty U_ij=σ_i²+σ_j² is positively correlated in expectation with the level of pxtr conflict.
Sets up Prop. 3.2; the paper says it is 'empirically motivated' but uses it as a premise, so much of the desired conclusion is already contained here.
ad hoc to paper Appendix Eq. (30): expected deviation |y*_ij − averaged pxtr label| is non-decreasing in pxtr conflict c (g_m(c1)≤g_m(c2)).
An unlabeled, unproven monotonicity assumption. It is the bridge from 'conflicting signals' to 'bias against true satisfaction' and is load-bearing for Prop. 3.2.
domain assumption Conditional independence of the Gaussian scoring variables yhat_i and yhat_j for item pairs.
Stated in Sec. 3.3.1 'for tractability'; required for z∼N(μ_i−μ_j, σ_i²+σ_j²).
domain assumption Questionnaire feedback used in EASQ approximates true user satisfaction.
Used in Sec. 4.5.1/4.5.2 to define approximate ground-truth satisfaction and the label-bias correlation; if questionnaires are biased, the validation weakens.
domain assumption Intrinsic conflicts among pxtr labels are the dominant source of aleatoric uncertainty in this setting.
Stated in Sec. 2.2; justifies treating variance as conflict-driven rather than label noise or distribution shift.

invented entities (2)

Gaussian scoring variable yhat_i = μ_i + σ_i ε_i
purpose: Encodes the predicted satisfaction score as a distribution so pairwise comparisons carry confidence.
No direct observation; it is a modeling construct. Its variance is supervised by the auxiliary loss and correlated with questionnaire bias in the same paper, not with an external instrument.
Pairwise comprehensive uncertainty U_ij = σ_i² + σ_j²
purpose: Drives both the adaptive sample weight and the conflict-alignment signal.
A derived quantity of the model; the paper's empirical support for its relation to label bias comes from the same training pipeline that enforces the relation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Uncertainty as Remedy: Mitigating Satisfaction Label Bias in Short Video Multi-Objective Ensemble Ranking." pith.science (2026). https://pith.science/paper/KULQQTVK

@misc{pith2026260717092,
  author       = {Pith},
  title        = {Pith review of: Uncertainty as Remedy: Mitigating Satisfaction Label Bias in Short Video Multi-Objective Ensemble Ranking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KULQQTVK}},
  note         = {Machine review of arXiv:2607.17092}
}

read the original abstract

The core objective of short video recommendation is to model users' unobservable true satisfaction with recommended videos. As the dominant industrial framework, end-to-end multi-objective ensemble ranking models are typically trained with multi-dimensional dense user behavioral signals, such as clicks and watch time. However, these behavioral signals are partial, fragmented, and often mutually conflicting user satisfaction proxies, introducing uncertainty and label bias into satisfaction modeling. Conventional deterministic models overlook this uncertainty, which exacerbates satisfaction label bias and results in suboptimal model convergence. Meanwhile, existing uncertainty-aware methods mostly employ uncertainty for post-hoc ranking adjustments rather than leveraging it as a remedy to mitigate the inherent bias within the core optimization pipeline. This paper proposes UAME, an Uncertainty-Aware end-to-end Multi-objective Ensemble ranking framework for short video recommendation. UAME represents the model's prediction as a Gaussian scoring variable, where the mean denotes the predicted satisfaction score and the variance quantifies predictive uncertainty associated with this score. We further design a probabilistic pairwise ranking loss, and construct an uncertainty-aware sample-level weighting scheme to mitigate the bias. We further provide theoretical analysis suggesting that the weighting scheme helps mitigate satisfaction label bias. Extensive offline and online experiments on a large-scale industrial short video platform demonstrate that UAME consistently improves two state-of-the-art paradigms, EMER and EASQ, and better aligns with questionnaire-based user satisfaction. UAME has been deployed in our production short-video recommendation system and continues to deliver stable, statistically significant gains.

Figures

Figures reproduced from arXiv: 2607.17092 by the authors.

**Figure 1.** Overview of multi-objective ensemble ranking. (a) Stage 1: Point-wise MTL module generates pxtr scores for different [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** Multi-objective pxtrs ranking conflict and corre [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Hyperparameter sensitivity analysis on EMER backbone. (a) Regularization coefficient [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Bucket-level visualization of satisfaction label bias [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 7 linked inside Pith

[1]

Jiangxia Cao, Pengbo Xu, Yin Cheng, Kaiwei Guo, Jian Tang, Shijun Wang, Dewei Leng, Shuang Yang, Zhaojie Liu, Yanan Niu, et al. 2025. Pantheon: Personalized multi-objective ensemble sort via iterative pareto policy optimization. InPro- ceedings of the 34th ACM International Conference on Information and Knowledge Management. 5575–5582

2025
[2]

Sirui Chen, Yuan Wang, Zijing Wen, Zhiyu Li, Changshuo Zhang, Xiao Zhang, Quan Lin, Cheng Zhu, and Jun Xu. 2023. Controllable multi-objective re-ranking with policy hypernetworks. InProceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining. 3855–3864

2023
[3]

Konstantina Christakopoulou, Madeleine Traverse, Trevor Potter, Emma Marriott, Daniel Li, Chris Haulk, Ed H Chi, and Minmin Chen. 2020. Deconfounding user satisfaction estimation from response rate bias. InProceedings of the 14th ACM conference on recommender systems. 450–455

2020
[4]

Chao Du, Zhifeng Gao, Shuo Yuan, Lining Gao, Ziyan Li, Yifan Zeng, Xiaoqiang Zhu, Jian Xu, Kun Gai, and Kuang-Chih Lee. 2021. Exploration in online adver- tising systems with deep uncertainty-aware learning. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2792–2801

2021
[5]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization.Journal of machine learning research12, 7 (2011)

2011
[6]

Feng Gao, Xin Zhou, Yinning Shao, Yue Wu, Jiahua Gao, Yujian Ren, Fengyang Qi, Ruochen Deng, and Jie Liu. 2025. Both Supply and Precision: Sample Debias and Ranking Consistency Joint Learning for Large Scale Pre-Ranking System. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 11672–11680

2025
[7]

Tiantian He, Minzhi Xie, Runtong Li, Xiaoxiao Xu, Jiaqi Yu, Zixiu Wang, Lantao Hu, Han Li, and Kun Gai. 2025. An End-to-End Multi-objective Ensemble Ranking Framework for Video Recommendation.arXiv preprint arXiv:2508.05093(2025)

arXiv 2025
[8]

Maria Heuss, Daniel Cohen, Masoud Mansoury, Maarten de Rijke, and Carsten Eickhoff. 2023. Predictive uncertainty-based bias mitigation in ranking. InPro- ceedings of the 32nd ACM International Conference on Information and Knowledge Management. 762–772

2023

Show all 43 references

[9]

Xin Jiang, Kaiqiang Wang, Yinlong Wang, Fengchang Lv, Taiyang Peng, Shuai Yang, Xianteng Wu, Pengye Zhang, Shuo Yuan, and Yifan Zeng. 2024. Deep Uncertainty-Based Explore for Index Construction and Retrieval in Recommen- dation System. InProceedings of the 33rd ACM Internation...

2024
[10]

Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision?Advances in neural information processing systems30 (2017)

2017
[11]

Alex Kendall, Yarin Gal, and Roberto Cipolla. 2018. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. InProceedings of the IEEE conference on computer vision and pattern recognition. 7482–7491

2018
[12]

Norman Knyazev and Harrie Oosterhuis. 2023. A lightweight method for model- ing confidence in recommendations with learned beta distributions. InProceedings of the 17th ACM conference on recommender systems. 306–317

2023
[13]

Ang Li, Jian Hu, Ke Ding, Xiaolu Zhang, Jun Zhou, Yong He, and Xu Min. 2023. Uncertainty-based heterogeneous privileged knowledge distillation for recom- mendation system. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information R...

2023
[14]

Danwei Li, Zhengyu Zhang, Siyang Yuan, Mingze Gao, Weilin Zhang, Chaofei Yang, Xi Liu, and Jiyan Yang. 2023. Adatt: Adaptive task-to-task fusion network for multitask learning in recommendations. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data M...

2023
[15]

Jiayu Li, Ziyi Ye, Guohao Jian, Zhiqiang Guo, Weizhi Ma, Qingyao Ai, and Min Zhang. 2025. Are Recommenders Self-Aware? Label-Free Recommendation Performance Estimation via Model Uncertainty.arXiv preprint arXiv:2507.23208 (2025)

2025 arXiv
[16]

Na Li, Jiaqi Yu, Minzhi Xie, Tiantian He, Xiaoxiao Xu, Zixiu Wang, Lantao Hu, Yongqi Liu, Han Li, Kaiqiao Zhan, et al. 2026. Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation. InProceedings of the 49th international ACM SIGIR conferenc...

2026
[17]

Yuyuan Li, Yizhao Zhang, Weiming Liu, Xiaohua Feng, Zhongxuan Han, Chaochao Chen, and Chenggang Yan. 2025. Multi-objective unlearning in rec- ommender systems via preference guided pareto exploration.IEEE Transactions on Services Computing(2025)

2025
[18]

Fuhang Liang, Rucong Xu, and Deng Lin. 2025. Adaprl: Adaptive pairwise regression learning with uncertainty estimation for universal regression tasks. arXiv preprint arXiv:2501.05809(2025)

2025 arXiv
[19]

Huafeng Liu, Jingxuan Wen, Liping Jing, and Jian Yu. 2019. Deep generative rank- ing for personalized recommendation. InProceedings of the 13th ACM Conference on Recommender Systems. 34–42

2019
[20]

Taichi Liu, Chen Gao, Zhenyu Wang, Dong Li, Jianye Hao, Depeng Jin, and Yong Li. 2023. Uncertainty-aware consistency learning for cold-start item recommen- dation. InProceedings of the 46th international ACM SIGIR conference on research and development in information retrieval...

2023
[21]

Huanbo Lyu, Daniel Herring, Shiqiao Zhou, Miqing Li, Zheming Zuo, Jelena Ninic, James Andrews, Fabian Spill, and Shuo Wang. 2025. Uncertainty-Aware Dual-Ranking Strategy for Offline Data-Driven Multi-Objective Optimization. arXiv preprint arXiv:2511.06459(2025)

2025 arXiv
[22]

Debabrata Mahapatra, Chaosheng Dong, Yetian Chen, and Michinari Momma
[23]

Yue Meng, Cheng Guo, Yi Cao, Tong Liu, and Bo Zheng. 2025. A generative re- ranking model for list-level multi-objective optimization at taobao. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 4213–4218

2025
[24]

Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari
[25]

Krishna P Neupane, Ervine Zheng, and Qi Yu. 2024. Evidential stochastic differ- ential equations for time-aware sequential recommendation.Advances in Neural Information Processing Systems37 (2024), 67556–67574

2024
[26]

Charul Paliwal, Anirban Majumder, and Sivaramakrishnan Kaveri. 2024. Predic- tive relevance uncertainty for recommendation systems. InProceedings of the ACM Web Conference 2024. 3900–3909

2024
[27]

Coral Scharf, Carmel Domshlak, Avigdor Gal, and Haggai Roitman. 2025. A Rank-Based Approach to Recommender System’s Top-K Queries with Uncertain Scores.Proceedings of the ACM on Management of Data3, 1 (2025), 1–26

2025
[28]

2014.Understanding machine learning: From theory to algorithms

Shai Shalev-Shwartz and Shai Ben-David. 2014.Understanding machine learning: From theory to algorithms. Cambridge university press

2014
[29]

Clemencia Siro, Mohammad Aliannejadi, and Maarten De Rijke. 2023. Under- standing and predicting user satisfaction with conversational recommender systems.ACM Transactions on Information Systems42, 2 (2023), 1–37

2023
[30]

Dusan Stamenkovic, Alexandros Karatzoglou, Ioannis Arapakis, Xin Xin, and Kleomenis Katevas. 2022. Choosing the best of both worlds: Diverse and novel recommendations through multi-objective reinforcement learning. InProceedings of the fifteenth ACM international conference on...

2022
[31]

Jie Tang, Huiji Gao, Liwei He, and Sanjeev Katariya. 2024. Multi-objective learning to rank by model distillation. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5783–5792

2024
[32]

Hongxu Wang, Zhu Sun, Yingpeng Du, Lu Zhang, Tiantian He, and Yew-Soon Ong. 2025. Uncertain Multi-Objective Recommendation via Orthogonal Meta- Learning Enhanced Bayesian Optimization.arXiv preprint arXiv:2502.13180 (2025)

2025 arXiv
[33]

Menghan Wang, Yujie Lin, Guli Lin, Keping Yang, and Xiao-ming Wu. 2020. M2GRL: A multi-task multi-view graph representation learning framework for web-scale recommender systems. InProceedings of the 26th ACM SIGKDD inter- national conference on knowledge discovery & data minin...

2020
[34]

Qichao Wang, Yunhong Lu, Hengyuan Cao, Junyi Zhang, and Min Zhang. 2026. DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12417–12427

2026
[35]

Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, and Xue Liu. 2021. Multi- FR: A multi-objective optimization method for achieving two-sided fairness in E-commerce recommendation.arXiv preprint arXiv:2105.02951(2021)

2021 arXiv
[36]

Boyang Xia, Zhou Yu, Zhiliang Zhu, Hanxiao Sun, Biyun Han, Jun Wang, Run- nan Liu, and Wenwu Ou. 2026. HarmonRank: Ranking-aligned Multi-objective Ensemble for Live-streaming E-commerce Recommendation.arXiv preprint arXiv:2601.02955(2026)

2026
[37]

Zhengrui Xu, Zhe Yang, Zhengxiao Guo, Shukai Liu, Luocheng Lin, Xiaoyan Liu, Yongqi Liu, and Han Li. 2025. UMRE: A Unified Monotonic Transformation for Ranking Ensemble in Recommender Systems.arXiv preprint arXiv:2508.07613 (2025)

2025 arXiv
[38]

Tao Yang, Cuize Han, Chen Luo, Parth Gupta, Jeff M Phillips, and Qingyao Ai
[39]

Tao Yang, Chen Luo, Hanqing Lu, Parth Gupta, Bing Yin, and Qingyao Ai. 2022. Can clicks be both labels and features? unbiased behavior feature collection and uncertainty-aware learning to rank. InProceedings of the 45th international ACM SIGIR conference on research and develo...

2022
[40]

Qihua Zhang, Junning Liu, Yuzhuo Dai, Yiyan Qi, Yifan Yuan, Kunlun Zheng, Fan Huang, and Xianfeng Tan. 2022. Multi-task fusion via reinforcement learning for long-term user satisfaction in recommender systems. InProceedings of the 28th ACM SIGKDD conference on knowledge discov...

2022
[2013]

Learning with noisy labels.Advances in neural information processing systems26 (2013)

2013
[2023]

In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining

Multi-label learning to rank through multi-objective optimization. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining. 4605–4616
[2024]

InProceedings of the ACM Web Conference 2024

Mitigating exploitation bias in learning to rank with an uncertainty-aware empirical Bayes approach. InProceedings of the ACM Web Conference 2024. 1486– 1496

2024

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

Jiangxia Cao, Pengbo Xu, Yin Cheng, Kaiwei Guo, Jian Tang, Shijun Wang, Dewei Leng, Shuang Yang, Zhaojie Liu, Yanan Niu, et al. 2025. Pantheon: Personalized multi-objective ensemble sort via iterative pareto policy optimization. InPro- ceedings of the 34th ACM International Conference on Information and Knowledge Management. 5575–5582

2025

[2] [2]

Sirui Chen, Yuan Wang, Zijing Wen, Zhiyu Li, Changshuo Zhang, Xiao Zhang, Quan Lin, Cheng Zhu, and Jun Xu. 2023. Controllable multi-objective re-ranking with policy hypernetworks. InProceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining. 3855–3864

2023

[3] [3]

Konstantina Christakopoulou, Madeleine Traverse, Trevor Potter, Emma Marriott, Daniel Li, Chris Haulk, Ed H Chi, and Minmin Chen. 2020. Deconfounding user satisfaction estimation from response rate bias. InProceedings of the 14th ACM conference on recommender systems. 450–455

2020

[4] [4]

Chao Du, Zhifeng Gao, Shuo Yuan, Lining Gao, Ziyan Li, Yifan Zeng, Xiaoqiang Zhu, Jian Xu, Kun Gai, and Kuang-Chih Lee. 2021. Exploration in online adver- tising systems with deep uncertainty-aware learning. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2792–2801

2021

[5] [5]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization.Journal of machine learning research12, 7 (2011)

2011

[6] [6]

Feng Gao, Xin Zhou, Yinning Shao, Yue Wu, Jiahua Gao, Yujian Ren, Fengyang Qi, Ruochen Deng, and Jie Liu. 2025. Both Supply and Precision: Sample Debias and Ranking Consistency Joint Learning for Large Scale Pre-Ranking System. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 11672–11680

2025

[7] [7]

Tiantian He, Minzhi Xie, Runtong Li, Xiaoxiao Xu, Jiaqi Yu, Zixiu Wang, Lantao Hu, Han Li, and Kun Gai. 2025. An End-to-End Multi-objective Ensemble Ranking Framework for Video Recommendation.arXiv preprint arXiv:2508.05093(2025)

arXiv 2025

[8] [8]

Maria Heuss, Daniel Cohen, Masoud Mansoury, Maarten de Rijke, and Carsten Eickhoff. 2023. Predictive uncertainty-based bias mitigation in ranking. InPro- ceedings of the 32nd ACM International Conference on Information and Knowledge Management. 762–772

2023