Pith. sign in

REVIEW 3 major objections 4 minor 7 cited by

Uncertainty Quantification for Ranking with Heterogeneous Preferences

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sparse pairwise comparisons can yield entrywise-accurate preference scores and valid ranking confidence intervals for groups and individual users, under a heterogeneous Bradley-Terry-Luce model.

desk verdict Genuinely new indirect-regularization idea and plausible entrywise UQ results for heterogeneous BTL, but the main theorem currently rests on an omitted proof section and the real-data experiment is actually a simulation. read the letter →

arxiv 2509.01847 v1 pith:UVDAJ2OD submitted 2025-09-02 stat.ME math.STstat.TH

Jianqing Fan , Hyukjun Kwon , Xiaonan Zhu This is my paper
classification stat.MEmath.STstat.TH
keywords Bradley–Terry–Lucemodelpreferencelearningcollaborativerankingsieveapproximationnonconvexoptimizationℓ∞errorbounduncertaintyquantificationnuclearnormregularization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to prove that heterogeneous human preferences can be learned from sparse pairwise comparisons with uniform entrywise accuracy, and that the learned score gaps can support valid confidence intervals for rankings, at the group level and even for an individual user. It models each user's preference as a smooth function of low-dimensional item features, which makes the underlying score matrix approximately low-rank, and then "indirectly regularizes" the induced pairwise-choice probability matrix instead of the score matrix itself. The argument combines sieve approximation, a reparameterized nonconvex surrogate of the regularized least-squares problem, leave-one-out gradient descent, and a single Newton-Raphson debiasing step to remove regularization bias. If the theory is right, this is the first entrywise error guarantee and uncertainty-quantification framework for heterogeneous Bradley-Terry-Luce preference learning, and it requires far fewer comparisons per user than learning each user's preferences separately.

What carries the argument

The carrying mechanism is indirect regularization: instead of penalizing the score matrix Θ⋆ or the score-gap matrix M⋆, the estimator solves a nuclear-norm-regularized least-squares problem for the induced choice-probability matrix σ(M⋆). Smoothness of the logistic link and of the nonparametric preference functions transfers approximate low-rank structure to the probability matrix, with sieve rank R. The reparameterized nonconvex surrogate f(X,Y) has a loss whose gradient is linear in X and Y, so it lies in the low-rank subspace and the gap between the convex and nonconvex solutions can be controlled; leave-one-out gradient-descent iterates then yield the entrywise bound. A one-step Newton-

What would settle it

Take a synthetic population where each user's preference over items is an arbitrary permutation (no smooth low-dimensional structure) and items have no shared latent features; run the proposed estimator with increasing d1 and d2 and check whether the entrywise error ∥Θ̂−Θ⋆∥∞ shrinks at the rate predicted by Theorem 4.1. If it does not, or if the sieve approximation error ∥E∥∞ fails to satisfy Assumption 3.1(i) when R grows, the low-rank premise underlying the theorem fails. A less extreme check is to generate preferences from g_i(η_j) with high-frequency, non-smooth oscillations and compare th

Watch

Extended reading notes

Core claim

The paper's central claim is that the score matrix Θ⋆, whose (i,j) entry is user i's score for item j, can be estimated entrywise uniformly from a small random sample of item comparisons, and that the debiased estimator is asymptotically normal for both averaged and individual score gaps. The key move is to regularize the probability matrix σ(M⋆), where M⋆ is the score-gap matrix, rather than the score matrix itself: because σ is smooth and the preference functions are assumed smooth, σ(M⋆) is approximately low-rank with a sieve representation σ(M⋆)=ΛΦ⊤+E whose approximation error is o(1/√max{d1,d2(d2−1)/2}). Theorem 4.1 gives an ℓ∞ error bound of order κ²μR/(min{d1,d2(d2−1)/2}) √(d̄ log d̄/

Load-bearing premise

The load-bearing premise is that real preferences are smooth functions of a low-dimensional item feature vector, so the pairwise-choice probability matrix is close to a low-rank matrix with manageable sieve rank; if preferences are not smooth or items do not share low-dimensional features, the entrywise error bound has no low-rank structure to exploit.

Editorial extensions

If this is right

  • Every entry of the estimated score matrix is simultaneously close to the truth, not just on average, so recommendations and top-K item sets for each user inherit uniform guarantees.
  • A single Newton-Raphson step removes the regularization bias, so score-gap differences and item rankings can be reported with confidence intervals at the aggregated level.
  • Individual-user inference is also available after rank-q spectral projection, so personalized ranking and selection statements can be quantified, not just population averages.
  • The per-user sample requirement depends on the sieve dimension and conditioning parameters rather than directly on the number of items, a major reduction compared with learning each user's preferences in isolation.
  • The same indirect-regularization analysis extends to any binary choice model with a smooth link function, not only the logistic BTL model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same indirect-regularization idea could debias other nonlinear low-rank problems, such as low-rank reward models trained from human feedback, where the likelihood gradient is not aligned with the factorization subspaces; a direct test would be to replace the logistic link with a probit or neural link and check whether entrywise control survives.
  • Because the procedure never estimates or validates the item feature vectors or the sieve basis, a practical diagnostic would be to compare observed entrywise errors against the rate predicted by Assumption 3.1; a data-driven sieve dimension R might make the method less sensitive to misspecification.
  • The sample-splitting step in individual inference is introduced for technical independence; the paper's own simulations suggest splitting costs little, so a theory avoiding the split may be within reach.
  • The ranking confidence intervals are built from simultaneous score-gap intervals and are likely conservative; sharper intervals could come from directly bootstrapping the rank statistic rather than passing through all pairwise gaps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies heterogeneous Bradley–Terry–Luce (BTL) preference learning with user-specific nonparametric preference functions and low-dimensional item features. The underlying score-gap matrix M* is estimated by an 'indirect regularization' scheme: a nuclear-norm penalized least squares problem on the induced probability matrix σ(M*), followed by inverse-link transformation and averaging over score gaps to recover Θ*. The central theoretical claims are an entrywise ℓ∞ error bound for bΘ (Theorem 4.1), debiased asymptotic normality for aggregated and individual score-gap estimates (Theorems 5.1 and 5.3), and simultaneous ranking confidence intervals via Gaussian multiplier bootstrap (Theorem 5.5). The proof strategy relies on a nonconvex surrogate of the regularized problem, a convex–nonconvex equivalence argument, and leave-one-out gradient-descent iterations (Appendices F–H).

Significance. If the theorems are correct, the paper makes a substantive contribution: entrywise error control and uncertainty quantification for heterogeneous BTL preferences have not been established before. The indirect regularization idea is elegant and potentially transferable to other binary choice models. The appendix contains substantial detailed proof material, including the convex–nonconvex equivalence (Lemma G.3) and sieve-based structural arguments. However, the most delicate part of the proof—the leave-one-out analysis of the nonconvex iterates—is omitted, so the central entrywise bound is not currently verifiable. The numerical study is also self-referential: it simulates BTL comparisons from a constructed 'quasi-true' score matrix rather than using real pairwise choices, so it does not validate the key structural assumptions.

major comments (3)
  1. [Section H; Theorem F.1] The proof of Theorem F.1 (and hence Theorem 4.1, Corollary F.2, and the debiasing results) invokes (H.2), Lemma H.1, and Lemma H.2 directly. Section H states that the proofs are omitted 'for brevity' and 'available upon request.' These lemmas are not routine: Lemma H.1 supplies the leave-one-out gradient-descent bounds (H.4)–(H.8), and Lemma H.2 supplies the incoherence and closeness properties used to control ||X_t Y_t^T − L*||∞ and ||bL − X_t Y_t^T||∞. Without a complete Section H, the central entrywise error claim is unverifiable. Please include full proofs, or restructure the claims so that they do not depend on unproved lemmas.
  2. [Section 6; Assumption 3.1] The real-data experiment does not use actual pairwise choices. A 780×40 'quasi-true' score matrix is constructed from discretized watch ratios, and then BTL comparisons are simulated (Section 6). Thus the numerical study can only demonstrate algorithmic self-consistency under the assumed generative model; it cannot validate the load-bearing structural premises of Assumption 3.1 (smoothness of preference functions, low-dimensional item features, small sieve error, spiked/incoherent σ(M*)). The paper should state this limitation explicitly and, ideally, add a diagnostic of approximate low-rankness or analyze a real pairwise-comparison dataset.
  3. [Section 5.3 vs. Section A] Individual-level inference assumes the rank q of the dominant component of M* is known ('For simplicity, this paper assumes that the rank q is known'), but the numerical experiments estimate q by a 10% singular-value threshold (Section A). No theorem shows that this estimator is consistent under Assumptions C.1–C.2, nor that substituting estimated singular vectors into the debiasing procedure preserves the error bounds needed in Theorem 5.3 and Proposition 5.4. The implemented procedure is therefore not covered by the stated guarantees. Either prove consistency of the threshold estimator, or label this part as heuristic and run the UQ simulations with a known q.
minor comments (4)
  1. [Eq. (3.9), Step 2 of Section 3.4] The convex estimator bL is not constrained to [0,1], yet Step 2 applies σ^{−1} entrywise. The analysis assumes bL entries are bounded away from 0 and 1; a clipping step would make the procedure well-defined and should be mentioned.
  2. [Section 5.4] The notation \(\bar L(j,j')\) is used for both orientations in Eq. (5.2) and Theorem 5.5, but it is introduced only informally. Please define it at first use and make the orientation in the simultaneous CIs explicit.
  3. [Section A and Theorem 4.1] Simulations set λ = sqrt(0.5 dbar/pbar), while Theorem 4.1 requires λ = C_λ sqrt(dbar/pbar) for a large constant C_λ. The value 0.5 is not 'large'; please clarify whether the constant can be taken small in practice or justify the choice.
  4. [Section 6] The paragraph 'All proofs are provided in the appendix' in the introduction is contradicted by Section H. Please revise the wording after completing the appendix.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction: the theoretical results are derived from explicit structural assumptions, and the omitted Section H proofs, while a verifiability gap, do not make the derivation equivalent to its inputs.

full rationale

The paper's derivation chain is not circular in the sense of the review criteria. The estimator is defined by solving (3.9), then transforming via cM = sigma^{-1}(bL) and linear averaging to bTheta; the entrywise bound of Theorem 4.1 is proved from Theorem F.1 and Corollary F.2 using Taylor expansion and smoothness of sigma^{-1}, with no constant fitted to the target rate. The regularization parameter is set to a generic constant (lambda = C_lambda sqrt(dbar/pbar)), and Assumption 3.1 is a stated structural condition rather than an output of the estimation procedure. The debiasing results (Theorems 5.1, 5.3, 5.5) follow from Taylor expansions, leave-one-out bounds, and spectral projector representations, again without fitting parameters to achieve normality. The main caveat is that Section H, containing the crucial leave-one-out lemmas H.1 and H.2, says 'we omit them for brevity' and cites Chen et al. (2020b) for 'similar' proofs. Chen et al. (2020b) shares an author (Jianqing Fan) and supplies the proof template, but the target result — entrywise error control for heterogeneous BTL models — is not itself imported from that paper; the citation is for a general technique in matrix completion, and the manuscript states the lemmas it requires. This is an incompleteness/verifiability concern, not a circular reduction of the prediction to its inputs. No fitted parameter is renamed as a prediction, and no uniqueness theorem from the authors' prior work is used to forbid alternatives.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The theory rests on the smooth low-rank latent structure of the preference model, the random graph sampling assumption, and a substantial list of technical conditions on the sieve approximation, condition numbers, incoherence, and signal strength. No new physical entities are introduced. The free parameters are the regularization tuning constant, the unestimated sieve dimension R, and the rank q of the score gap matrix, which is assumed known in the main individual inference theory. The preprint does not supply the omitted proofs in Appendix H, so the ledger cannot be fully certified from the available text.

free parameters (3)
  • Regularization parameter lambda = lambda = C_lambda sqrt(dbar/pbar) with large constant; simulations set lambda = sqrt(0.5 * dbar/pbar)
    Tuning parameter for the nuclear norm penalty; chosen by the authors and not derived from the data or leading to a fitted prediction in the theorems.
  • Sieve dimension R = Not estimated; assumed in Assumption 3.1
    Rank of the sieve representation L* = Lambda Phi^T. The theory requires R to grow slowly, but no data-driven selection rule is given for the main estimator.
  • Rank q of M* for individual inference = Assumed known in Section 5.3; in simulations estimated by thresholding singular values at 10% of the largest
    The individual-level projection and variance formula require the rank q of the score gap matrix; the paper explicitly assumes q known and only uses a heuristic estimator in simulations.
assumptions (6)
  • domain assumption Each item has a low-dimensional latent feature vector eta_j and each user has a smooth nonparametric preference function g_i, so the induced pair-comparison functions are smooth.
    Introduced in Sections 2 and 3.3. Without this, the score matrix and probability matrix have no approximate low-rank structure.
  • domain assumption The sieve approximation error is small and L* has spiked singular values, bounded incoherence, and slowly growing condition number (Assumption 3.1).
    These conditions make nuclear norm regularization effective for estimating sigma(M*) and are stated as assumptions rather than consequences of the model.
  • domain assumption Pairwise comparisons are sampled according to heterogeneous Erdos-Renyi graphs with pmin > c pmax and pmin/pmax bounded away from zero (Assumption 2.2).
    Defines the observation model used throughout the proofs and in simulations.
  • standard math Row sums of the score matrix are zero (Assumption 2.1).
    Identifies the BTL model up to row-wise shifts; standard in ranking models.
  • domain assumption The score gap matrix M* has a low-rank sieve approximation with known rank q and spiked singular values (Assumptions C.1, C.2, F.1-F.3).
    Needed only for individual-level UQ and ranking intervals; it is an additional structural assumption parallel to Assumption 3.1.
  • standard math Sieve approximation theory: smooth functions can be approximated by polynomial, trigonometric, or spline bases with error O(R^{-s}).
    Invoked in Section 3.3 and borrowed from Chen (2007) and related references.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Uncertainty Quantification for Ranking with Heterogeneous Preferences." pith.science (2026). https://pith.science/paper/UVDAJ2OD

@misc{pith2026250901847,
  author       = {Pith},
  title        = {Pith review of: Uncertainty Quantification for Ranking with Heterogeneous Preferences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UVDAJ2OD}},
  note         = {Machine review of arXiv:2509.01847}
}
abstract

This paper studies human preference learning based on partially revealed choice behavior and formulates the problem as a generalized Bradley-Terry-Luce (BTL) ranking model that accounts for heterogeneous preferences. Specifically, we assume that each user is associated with a nonparametric preference function, and each item is characterized by a low-dimensional latent feature vector - their interaction defines the underlying low-rank score matrix. In this formulation, we propose an indirect regularization method for collaboratively learning the score matrix, which ensures entrywise $\ell_\infty$-norm error control - a novel contribution to the heterogeneous preference learning literature. This technique is based on sieve approximation and can be extended to a broader class of binary choice models where a smooth link function is adopted. In addition, by applying a single step of the Newton-Raphson method, we debias the regularized estimator and establish uncertainty quantification for item scores and rankings of items, both for the aggregated and individual preferences. Extensive simulation results from synthetic and real datasets corroborate our theoretical findings.

Figures

Figures reproduced from arXiv: 2509.01847 by the authors.

Figure 1
Figure 1. Histogram of the watch ratios and their assigned baskets and scores. [PITH_FULL_IMAGE:figures/full_fig_p029_1.png] view at source ↗
Figure 2
Figure 2. The left panel corresponds to the first experiment with higher probabilities (0 [PITH_FULL_IMAGE:figures/full_fig_p038_2.png] view at source ↗
Figure 3
Figure 3. Histograms for the aggregated score gap. The first and second rows corre [PITH_FULL_IMAGE:figures/full_fig_p039_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Histograms for the individual score gap. As in Figure [PITH_FULL_IMAGE:figures/full_fig_p039_4.png]
Figure 5
Figure 5. Figure 5: The left panel corresponds to the first experiment with higher probability [PITH_FULL_IMAGE:figures/full_fig_p041_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LLM Evaluation as Tensor Completion: Low Rank Structure and Semiparametric Efficiency

    stat.ME 2026-04 unverdicted novelty 8.0 of 10

    LLM pairwise evaluation is recast as low-rank tensor completion, yielding semiparametric efficient estimators and asymptotic normality for ability functionals via a score-whitening correction for anisotropic operators.

  2. Low Rank for Rank: Uncertainty-Aware Task-Specific LLM Ranking under Sparse Pairwise Comparisons

    stat.ME 2026-05 unverdicted novelty 6.0 of 10

    Low-rank matrix modeling with cross-fitted debiased estimators and multiplier bootstrap yields stable task-specific LLM rankings and asymptotically valid simultaneous confidence sets from sparse pairwise data.

  3. Learning Perturbations to Extrapolate Your LLM

    stat.ML 2026-05 unverdicted novelty 6.0 of 10

    A learnable continuous perturbation framework for LLM token prefixes via latent vector transformations, optimized through unbiased estimating equations, yields gains in out-of-domain performance.

  4. Heterogeneous Judge-Aware Ranking with Sensitivity, Disagreement, and Confidence

    stat.ME 2026-05 unverdicted novelty 6.0 of 10

    HJA ranking separates consensus ranking, judge sensitivity, and residual disagreement as distinct inferential targets with identifiability conditions and an anchored alternating algorithm, yielding better recovery and...

  5. Perturbation is All You Need for Extrapolating Language Models

    stat.ML 2026-05 unverdicted novelty 6.0 of 10

    Perturbing prefixes to semantic neighbors during training creates a hierarchical noise model that improves language model predictions on token sequences outside the training corpus support.

  6. A Judge-Aware Ranking Framework for Evaluating Large Language Models without Ground Truth

    stat.ML 2026-01 conditional novelty 6.0 of 10

    A judge-aware Bradley–Terry model with per-judge discrimination parameters estimates LLM quality and judge reliability from unlabeled pairwise comparisons, with identifiability and asymptotic normality guarantees.

  7. Perturbation is All You Need for Extrapolating Language Models

    stat.ML 2026-05 conditional novelty 5.0 of 10

    Perturbing the prefix before next-token prediction, during both training and inference, improves out-of-distribution language-model generation and yields a conditional extrapolation guarantee.

Reference graph

Works this paper leans on

70 extracted references · 58 canonical work pages · cited by 6 Pith papers

  1. [1]

    Abbe, E., Fan, J., Wang, K., and Zhong, Y. (2020). Entrywise eigenvector analysis of random matrices with low expected rank. Annals of statistics , 48(3):1452

    2020

  2. [2]

    Ahn, S. C. and Horenstein, A. R. (2013). Eigenvalue ratio test for the number of factors. Econometrica , 81(3):1203--1227

    2013

  3. [3]

    Aouad, A., Farias, V., Levi, R., and Segev, D. (2018). The approximability of assortment optimization under ranking preferences. Operations Research , 66(6):1661--1669

    work page 2018

  4. [4]

    N., Glickman, M

    Avery, C. N., Glickman, M. E., Hoxby, C. M., and Metrick, A. (2013). A revealed preference ranking of us colleges and universities. The Quarterly Journal of Economics , 128(1):425--467

    work page 2013

  5. [5]

    and Ng, S

    Bai, J. and Ng, S. (2002). Determining the number of factors in approximate factor models. Econometrica , 70(1):191--221

    2002

  6. [6]

    Baltrunas, L., Makcinskas, T., and Ricci, F. (2010). Group recommendations with rank aggregation and collaborative filtering. In Proceedings of the fourth ACM conference on Recommender systems , pages 119--126

    work page 2010

  7. [7]

    S., Van Handel, R., et al

    Bandeira, A. S., Van Handel, R., et al. (2016). Sharp nonasymptotic bounds on the norm of random matrices with independent entries. The Annals of Probability , 44(4):2479--2506

    work page 2016

  8. [8]

    and Teboulle, M

    Beck, A. and Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences , 2(1):183--202

    work page 2009

Show all 70 references
  1. [9]

    Bradley, R. A. and Terry, M. E. (1952). Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika , 39(3/4):324--345

    1952

  2. [10]

    J., and Shen, Z

    Cai, J.-F., Cand \`e s, E. J., and Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on optimization , 20(4):1956--1982

    2010

  3. [11]

    Cand\` e s, E. J. and Plan, Y. (2010). Matrix completion with noise. Proceedings of the IEEE , 98(6):925--936

    2010

  4. [12]

    Cand\` e s, E. J. and Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational mathematics , 9(6):717

    2009

  5. [13]

    Chatzi, I., Straitouri, E., Thejaswi, S., and Rodriguez, M. (2024). Prediction-powered ranking of large language models. Advances in Neural Information Processing Systems , 37:113096--113133

    2024

  6. [14]

    and Li, X

    Chen, J. and Li, X. (2017). Memory-efficient kernel pca via partial matrix sampling and nonconvex optimization: a model-free analysis of local minima. arXiv preprint arXiv:1711.01742

    2017 arXiv

  7. [15]

    Chen, X. (2007). Large sample sieve estimation of semi-nonparametric models. Handbook of econometrics , 6:5549--5632

    2007

  8. [16]

    Chen, X., Wang, Y., and Zhou, Y. (2020a). Dynamic assortment optimization with changing contextual information. The Journal of Machine Learning Research , 21(1):8918--8961

  9. [17]

    Chen, Y., Chi, Y., Fan, J., Ma, C., and Yan, Y. (2020b). Noisy matrix completion: Understanding statistical guarantees for convex relaxation via nonconvex optimization. SIAM journal on optimization , 30(4):3098--3121

  10. [18]

    Chen, Y., Fan, J., Ma, C., and Wang, K. (2019a). Spectral method and regularized mle are both optimal for top-k ranking. Annals of statistics , 47(4):2204

  11. [19]

    Chen, Y., Fan, J., Ma, C., and Yan, Y. (2019b). Inference and uncertainty quantification for noisy matrix completion. Proceedings of the National Academy of Sciences , 116(46):22931--22937

  12. [20]

    Chen, Y., Li, C., Ouyang, J., and Xu, G. (2023). Statistical inference for noisy incomplete binary matrix. Journal of Machine Learning Research , 24(95):1--66

    2023

  13. [21]

    and Suh, C

    Chen, Y. and Suh, C. (2015). Spectral mle: Top-k rank aggregation from pairwise comparisons. In International Conference on Machine Learning , pages 371--380. PMLR

    2015

  14. [22]

    Chernozhukov, V., Hansen, C., Liao, Y., and Zhu, Y. (2023). Inference for low-rank models. The Annals of statistics , 51(3):1309--1330

    2023

  15. [23]

    Chernozhuokov, V., Chetverikov, D., Kato, K., and Koike, Y. (2022). Improved central limit theorem and bootstrap approximations in high dimensions. The Annals of Statistics , 50(5):2562--2586

    2022

  16. [24]

    Choi, J., Kwon, H., and Liao, Y. (2023). Inference for low-rank models without estimating the rank. arXiv preprint arXiv:2311.16440

    2023 arXiv

  17. [25]

    Choi, J., Kwon, H., and Liao, Y. (2024). Inference for low-rank completion without sample splitting with application to treatment effect estimation. Journal of Econometrics , 240(1):105682

    2024

  18. [26]

    and Yuan, M

    Choi, J. and Yuan, M. (2024). Matrix completion when missing is not at random and its applications in causal panel data models. Journal of the American Statistical Association , pages 1--15

    2024

  19. [27]

    Dhurandhar, A., Nair, R., Singh, M., Daly, E., and Ramamurthy, K. N. (2024). Ranking large language models without ground truth. arXiv preprint arXiv:2402.14860

    2024 arXiv

  20. [28]

    Dwork, C., Kumar, R., Naor, M., and Sivakumar, D. (2001). Rank aggregation methods for the web. In Proceedings of the 10th international conference on World Wide Web , pages 613--622

    2001

  21. [29]

    Fan, J., Ge, J., and Hou, J. (2025a). Covariates-adjusted mixed-membership estimation: A novel network model with optimal guarantees. arXiv preprint arXiv:2502.06671

    arXiv

  22. [30]

    Fan, J., Hou, J., and Yu, M. (2024a). Covariate assisted entity ranking with sparse intrinsic scores. arXiv preprint arXiv:2407.08814

    arXiv

  23. [31]

    Fan, J., Hou, J., and Yu, M. (2024b). Uncertainty quantification of mle for entity ranking with covariates. Journal of Machine Learning Research , 25(358):1--83

  24. [32]

    Fan, J., Liao, Y., and Wang, W. (2016). Projected principal component analysis in factor models. Annals of statistics , 44(1):219

    2016

  25. [33]

    Fan, J., Lou, Z., Wang, W., and Yu, M. (2025b). Ranking inferences based on the top choice of multiway comparisons. Journal of the American Statistical Association , 120:237--250

  26. [34]

    Fan, J., Lou, Z., Wang, W., and Yu, M. (2025c). Spectral ranking inferences based on general multiway comparisons. Operations Research , page to appear

  27. [35]

    Fernandes, C., Siderius, J., and Singal, R. (2025). Peer review market design: Effort-based matching and admission control. Available at SSRN

    2025

  28. [36]

    Gao, C., Li, S., Lei, W., Chen, J., Li, B., Jiang, P., He, X., Mao, J., and Chua, T.-S. (2022). Kuairec: A fully-observed dataset and insights for evaluating recommender systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management , CI...

    2022

  29. [37]

    Gao, C., Shen, Y., and Zhang, A. Y. (2023). Uncertainty quantification in the bradley--terry--luce model. Information and Inference: A Journal of the IMA , 12(2):1073--1140

    2023

  30. [38]

    and Montanari, A

    Javanmard, A. and Montanari, A. (2013). Confidence intervals and hypothesis testing for high-dimensional statistical models. Advances in neural information processing systems , 26

    2013

  31. [39]

    and Montanari, A

    Javanmard, A. and Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. The Journal of Machine Learning Research , 15(1):2869--2909

    2014

  32. [40]

    and Scott, C

    Katz-Samuels, J. and Scott, C. (2018). Nonparametric preference completion. In International Conference on Artificial Intelligence and Statistics , pages 632--641. PMLR

    2018

  33. [41]

    H., Montanari, A., and Oh, S

    Keshavan, R. H., Montanari, A., and Oh, S. (2010). Matrix completion from a few entries. IEEE transactions on information theory , 56(6):2980--2998

    2010

  34. [42]

    B., et al

    Koltchinskii, V., Lounici, K., Tsybakov, A. B., et al. (2011). Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. The Annals of Statistics , 39(5):2302--2329

    2011

  35. [43]

    J., Sun, W

    Lee, S. J., Sun, W. W., and Liu, Y. (2024a). Low-rank contextual reinforcement learning from heterogeneous human feedback. arXiv preprint arXiv:2412.19436

  36. [44]

    J., Sun, W

    Lee, S. J., Sun, W. W., and Liu, Y. (2024b). Low-rank online dynamic assortment with dual contextual information. arXiv preprint arXiv:2404.17592

  37. [45]

    X., and Zhu, W

    Li, H., Simchi-Levi, D., Wu, M. X., and Zhu, W. (2023). Estimating and exploiting the impact of photo layout: A structural approach. Management Science , 69(9):5209--5233

    2023

  38. [46]

    Li, Z., Xu, Q., Jiang, Y., Ma, K., Cao, X., and Huang, Q. (2020). Neural collaborative preference learning with pairwise comparisons. IEEE Transactions on Multimedia , 23:1977--1989

    2020

  39. [47]

    Luce, R. D. (1959). Individual choice behavior , volume 4. Wiley New York

    1959

  40. [48]

    Ma, S., Goldfarb, D., and Chen, L. (2011). Fixed point and bregman iterative methods for matrix rank minimization. Mathematical Programming , 128(1-2):321--353

    2011

  41. [49]

    Maia Polo, F., Xu, R., Weber, L., Silva, M., Bhardwaj, O., Choshen, L., de Oliveira, A., Sun, Y., and Yurochkin, M. (2024). Efficient multi-prompt evaluation of llms. Advances in Neural Information Processing Systems , 37:22483--22512

    2024

  42. [50]

    and Walsh, T

    Mattei, N. and Walsh, T. (2013). Preflib: A library for preferences http://www.preflib.org. In International conference on algorithmic decision theory , pages 259--270. Springer

    2013

  43. [51]

    Mazumder, R., Hastie, T., and Tibshirani, R. (2010). Spectral regularization algorithms for learning large incomplete matrices. Journal of machine learning research , 11(Aug):2287--2322

    2010

  44. [52]

    McFadden, D. (1972). Conditional logit analysis of qualitative choice behavior

    1972

  45. [53]

    K., and Xu, J

    Negahban, S., Oh, S., Thekumparampil, K. K., and Xu, J. (2018). Learning from comparisons and choices. Journal of Machine Learning Research , 19(40):1--95

    2018

  46. [54]

    and Wainwright, M

    Negahban, S. and Wainwright, M. J. (2012). Restricted strong convexity and weighted matrix completion: Optimal bounds with noise. The Journal of Machine Learning Research , 13(1):1665--1697

    2012

  47. [55]

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (2022). Training language models to follow instructions with human feedback. Advances in N eural I nformation P rocessing S ystems , 35:27730--27744

    2022

  48. [56]

    and Boyd, S

    Parikh, N. and Boyd, S. (2014). Proximal algorithms. Foundations and Trends in optimization , 1(3):127--239

    2014

  49. [57]

    Park, C., Liu, M., Kong, D., Zhang, K., and Ozdaglar, A. (2024). Rlhf from heterogeneous feedback via personalization and preference aggregation. arXiv preprint arXiv:2405.00254

    2024 arXiv

  50. [58]

    Park, D., Neeman, J., Zhang, J., Sanghavi, S., and Dhillon, I. (2015). Preference completion: Large-scale collaborative ranking from pairwise comparisons. In International Conference on Machine Learning , pages 1907--1916. PMLR

    2015

  51. [59]

    Plackett, R. L. (1975). The analysis of permutations. Journal of the Royal Statistical Society: Series C (Applied Statistics) , 24(2):193--202

    1975

  52. [60]

    Shah, N. B. (2022). Challenges, experiments, and computational solutions in peer review. Communications of the ACM , 65(6):76--87

    2022

  53. [61]

    Su, B., Zhang, J., Collina, N., Yan, Y., Li, D., Cho, K., Fan, J., Roth, A., and Su, W. J. (2024). Analysis of the icml 2023 ranking data: Can authors' opinions of their own papers assist peer review in machine learning? arXiv preprint arXiv:2408.13430

    2024

  54. [62]

    Su, W. (2021). You are the best reviewer of your own papers: An owner-assisted scoring mechanism. Advances in Neural Information Processing Systems , 34:27929--27939

    2021

  55. [63]

    Tropp, J. A. (2015). An introduction to matrix concentration inequalities. arXiv preprint arXiv:1501.01571

    2015 arXiv

  56. [64]

    and Fan, J

    Wang, B. and Fan, J. (2025). Robust matrix completion with heavy-tailed noise. Journal of the American Statistical Association , 120(550):922--934

    2025

  57. [65]

    Wang, X., Bendersky, M., Metzler, D., and Najork, M. (2016). Learning to rank with selection bias in personal search. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval , pages 115--124

    2016

  58. [66]

    X., Wang, L., and Lu, J

    Wang, Z., Han, Y., Fang, E. X., Wang, L., and Lu, J. (2024). Ranking of large language model with nonparametric prompts. arXiv preprint arXiv:2412.05506

    2024 arXiv

  59. [67]

    Xia, D. (2021). Normal approximation and confidence region of singular subspaces. Electronic Journal of Statistics , 15(2):3798--3851

    2021

  60. [68]

    and Yuan, M

    Xia, D. and Yuan, M. (2021). Statistical inferences of linear forms for noisy matrix completion. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 83(1):58--77

    2021

  61. [69]

    J., and Fan, J

    Yan, Y., Su, W. J., and Fan, J. (2024). Isotonic mechanism for exponential family estimation in machine learning peer review

    2024

  62. [70]

    and Lafferty, J

    Zheng, Q. and Lafferty, J. (2016). Convergence analysis for rectangular matrix completion using burer-monteiro factorization and gradient descent. arXiv preprint arXiv:1605.07051

    2016 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.