Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Fisher Random Walk: Automatic Debiasing Contextual Preference Inference for Large Language Model Evaluation

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A Fisher random walk can debias contextual LLM preference estimates to the semiparametric efficiency bound.

desk verdict Genuinely novel debiasing idea and solid semiparametric framework, but the efficiency guarantee requires L≫n^2 and the paper's own simulations sit outside that regime; still worth a serious referee. read the letter →

arxiv 2509.05852 v1 pith:GYXMUCT6 submitted 2025-09-06 stat.ML cs.LGmath.STstat.TH

Yichi Zhang , Alexander Belloni , Ethan X. Fang , Junwei Lu , Xiaoan Xu This is my paper
classification stat.MLcs.LGmath.STstat.TH MSC 62G0562G2062F12
keywords contextualBradley-Terry-LucesemiparametricefficiencydebiasedmachinelearningFisherrandomwalkgraphLaplacianLLMevaluationpreferenceinferenceReLUnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that contextual pairwise preference comparisons—the kind used to evaluate large language models—can be turned into statistically optimal estimates of the expected score gap between two items over a context domain. The setting is the contextual Bradley-Terry-Luce model, where each item has an unknown context-dependent score and annotators compare pairs of items. The paper builds a debiased estimator that aggregates residuals across the whole comparison graph, weighting edges according to a Fisher-information random walk, and proves it is semiparametric efficient: no regular estimator can have smaller asymptotic variance. The key structural result is that the edge-level balancing weights can be written as differences of node-level potentials, computed through a graph Laplacian pseudoinverse, which cuts computation from O(n^2) nuisance functions to O(n). The practical payoff claimed is that confidence intervals and hypothesis tests for “model i beats model j in domain Ω” remain valid when score functions are learned by flexible black-box learners such as ReLU networks.

What carries the argument

Fisher random walk: a random walk on the comparison graph whose transition probability from node i to node j is proportional to ψ′(θ_i(x)−θ_j(x)), the Fisher information of the logistic comparison model; it supplies the path prior that aggregates edge-level debiasing residuals. Potential representation: the residual-balancing edge weights W_{ij}(x) equal π_i(x)−π_j(x), where π(x|θ,i0,j0) = |A| L†(x|θ)(e_{i0}−e_{j0}) and L is the weighted graph Laplacian built from Fisher information; this reduces the nuisance functions from O(n^2) edges to O(n) nodes. The proof machinery also includes a novel entrywise perturbation inequality for graph Laplacian pseudoinverses and an oracle inequality that t

What would settle it

On synthetic contextual BTL data with known score functions, run the Fisher random walk estimator inside the Theorem 3 rate region (e.g., β* = 2t*) and outside it (e.g., β* = t*/4), with n, p, L chosen accordingly. If the empirical coverage of the 95% intervals stays below nominal, or if the empirical variance ratio exceeds 1 + o_P(1) relative to the lower bound eV_{i0j0}(Ω) inside the admissible region, the efficiency theorem's assumptions are not sufficient.

Watch

Extended reading notes

Core claim

The central claim is that the Fisher random walk debiased estimator of Eq. (14) attains the semiparametric efficiency lower bound for Q_{i0j0}(Ω) = E[I(X∈Ω)(θ*_{i0}(X) − θ*_{j0}(X))]. The estimator debiases the plug-in score-gap estimate by averaging weighted residuals over all edges of the comparison graph, with edge weights chosen so that the underlying random walk has transition probabilities proportional to Fisher information. Theorem 1 recasts these weights as differences of node potentials given by the pseudoinverse of a Fisher-information graph Laplacian, making the debiasing step computationally feasible. Theorems 2 and 5 establish asymptotic normality and semiparametric efficiency:

Load-bearing premise

The plug-in preference score estimator must converge fast enough—at the oracle nonparametric rate tied to effective smoothness β* and dimension t*—for Eq. (20) to hold; if the true score functions are rougher or the context dimension higher, the debiasing residuals do not vanish and the stated normality and efficiency can fail.

Editorial extensions

If this is right

  • Confidence intervals and one-sided p-values for Q_{i0j0}(Ω) remain valid even when items i0 and j0 are never directly compared, provided the comparison graph is connected and satisfies the stated good-graph conditions.
  • The semiparametric efficiency bound is reached with general nonparametric score learners, so practitioners are not tied to linear or twice-differentiable preference score functions.
  • Computational cost drops from estimating O(n^2) edge-level balancing weights to estimating O(n) node-level potentials via graph Laplacian solvers.
  • The same debiasing structure extends to multiple-hypothesis testing across item pairs and domains with familywise error control by Gaussian multiplier bootstrap, and to target-domain inference under distributional shift through cross-fitted importance weighting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct stress test: generate data from the contextual BTL model with known score functions inside and outside the Theorem 3 rate region (e.g., β* = 2t* versus β* = t*/4). If empirical coverage or variance ratios do not match the theory inside the region, the efficiency claim's sufficient conditions are incomplete.
  • The resistance-distance term in the variance formula suggests a practical annotation rule the paper does not spell out: comparisons that shrink the effective resistance between the two target items should reduce the estimator's variance most, so annotation budgets could be allocated along shortest effective-resistance paths.
  • The potential representation is tied only to the graph Laplacian structure, so a similar node-potential debiasing device could plausibly carry over to time-varying comparison graphs or other pairwise exponential-family outcomes, although the paper does not make this claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies contextual preference inference under the contextual Bradley–Terry–Luce model. For a functional Q_{i0j0}(Ω) = E[I(X∈Ω)(θ*_{i0}(X)−θ*_{j0}(X))], it proposes a debiased estimator that aggregates weighted regression residuals over the edges of a comparison graph. The weights are derived from a Fisher-information random walk and, by Theorem 1, admit a potential representation π(x)=|A| L†(x)(e_{i0}−e_{j0}), reducing the nuisance estimation from O(n^2) edge functions to O(n) node potentials. The main theoretical results are: an oracle inequality for a generic plug-in score estimator and for ReLU-DNN estimators (Theorem 3), asymptotic normality of the debiased estimator under Assumption 1 (Theorem 2), a semiparametric efficiency lower bound and a variance equivalence result (Theorems 4 and 5), and extensions to multiple hypotheses and distribution shift (Theorems 6 and 7). Numerical experiments include synthetic contextual BTL data and an MMLU-based comparison of five LLMs with full and partial comparison graphs.

Significance. If the main claims hold, this is a substantial contribution to automatic debiased machine learning for grouped, graph-indexed nuisance parameters. The potential representation is elegant and computationally meaningful, and the graph-Laplacian perturbation bound (Theorem S12) appears to be a novel technical tool with independent value. The paper also includes a genuinely general nuisance-estimation framework: the oracle inequality is not tied to parametric or kernel estimators, and the debiasing construction is validated through Neyman orthogonality rather than through fitted constants. The supplementary material provides detailed proofs of the central theorems, including the oracle inequality, the lower bound, and the Laplacian perturbation analysis. These strengths make the paper worth serious consideration. However, as detailed below, the scope of the theoretical guarantees is narrower than the abstract and the numerical demonstrations suggest, and the manuscript should be revised to state this explicitly and to align the claims with the proven regime.

major comments (3)
  1. [§4.1, Eq. (29) vs Assumption 1, Eq. (20); §6.1] Assumption 1 requires E2(θhat,θ*|A)=o(1/(n^3 p) ∧ 1/(n√(npL))). The ReLU-DNN oracle inequality in Eq. (29) gives, even in the parametric limit β*→∞, only E2 ≲ log^3(npL)·(npL)^{-1}. Substituting this into the first half of Eq. (20) forces (npL)^{-1}=o(1/(n^3 p)), i.e. L≫n^2 for fixed p. The paper does encode this in the second condition of the paragraph after Eq. (29), but the abstract and Section 1.1 present semiparametric efficiency as a general automatic-debiasing result. More importantly, the simulations in §6.1 use n=50 with L≤2000 and n=80 with L=2000, so they do not enter L≫n^2; only the n=20 cases are near the boundary. Thus the empirical coverage rates in Figures 3–4 do not validate Theorem 2/5 in their proven regime. Please qualify the efficiency claim as holding in the large-L regime L≫n^2 (for constant p), and either add simulations with L substantially larger than n^2 or exp
  2. [§6.2] The real-data experiment has n=5 LLMs and a fixed comparison graph. Theorems 2, 5, and 6 are asymptotic in n with np≥40 log n or np≥(log n)^ξ; the conditions Egood in Eq. (25) are graph concentration statements for growing n. With n=5 these conditions are not meaningful, and no separate finite-n/L→∞ theory is provided. The MMLU demonstration is therefore a heuristic application, not a validation of the stated asymptotic guarantees. The paper should state this limitation explicitly and avoid presenting the Hasse diagrams as direct empirical confirmation of the semiparametric efficiency theorem.
  3. [§5.2, Theorem 7, and §6.2] Theorem 7 assumes the density ratio κ is known. In the application, κ is estimated by PCA followed by RuLSIF, with the number of principal components selected by cross-validation. No consistency or rate result is given for this estimated κ, and the analysis in Supplementary S6.2 explicitly does not cover the estimation error. Since distributional shift is one of the paper's stated contributions, the manuscript should either restrict the claim to the known-κ case or supply a theorem for estimated κ under conditions that the PCA/RuLSIF procedure can plausibly satisfy.
minor comments (4)
  1. [Global] Typos and minor wording issues: 'Prefence' in the Section 3 heading; 'Fisher rank walk' in the paragraph after Theorem 2; 'chooinsg' in §4.1; 'algrithms' in §5.2; 'Rank' capitalization in Supplementary S2. These do not affect content.
  2. [§4.2, Theorem 5] The statement of Theorem 5 does not explicitly list Assumption 1 or the L-growth condition needed to combine it with Theorem 2. Since the efficiency conclusion requires both the variance equivalence (31) and the asymptotic normality of the estimator, please state all conditions in Theorem 5 rather than leaving the reader to infer them from the surrounding text.
  3. [§3.1, Eq. (13)] The definition of W_ij in Eq. (13) includes the factor |A| and the indicator I(x∈Ω), while the earlier discussion in Eqs. (8)–(12) uses a different normalization. This is internally consistent, but a sentence explaining the role of the |A| factor in the potential representation would improve readability.
  4. [§5.1, Theorem 6] Condition (iv), (logn)^10/(p^3 L)=o(1), combined with L≥c n^{1/3}(log n)^3, is quite restrictive. Please add a remark on how these conditions interact with the n→∞ rate and whether the multiple-testing extension remains valid in the same L≫n^2 regime used for the single-pair theorem.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the debiasing construction and the efficiency lower bound are derived independently, and the estimator's variance is checked against that bound rather than defined to match it.

full rationale

The central claim is that the Fisher random walk debiased estimator in Eq. (14) attains the semiparametric efficiency lower bound. This is not circular. The debiasing weights are obtained from Theorem 1, which proves an exact identity between the expected net random-walk crossings and the graph-Laplacian pseudoinverse (W_ij = π_i − π_j with π = |A| L†(e_i0 − e_j0)); this is a mathematical derivation from the Markov-chain structure, not a fitted object. The efficiency lower bound is derived independently in Theorem 4 and Section S4.1 by computing the semiparametric efficient influence function under an information-enriched submodel, giving eV_i0j0(Ω) that involves inverse degree-type terms. Theorem 5 then shows that the actual variance V_i0j0(Ω) from Eq. (21), which involves the resistance distance (e_i0−e_j0)^T L† (e_i0−e_j0), is asymptotically equal to eV up to o_P(1) via a new entrywise pseudo-inverse perturbation bound (Theorem S12). Thus the variance is compared with, not defined as, the lower bound. Assumption 1 is a genuine rate condition on the nuisance estimator, and Theorem 3 provides a sufficient oracle inequality using standard external ReLU-DNN approximation and covering-number results; no fitted parameter is relabeled as a prediction. The paper cites its own prior work (Liu et al. 2023; Wang et al. 2024) only as related literature and does not rely on it for the load-bearing efficiency argument. The derivation is self-contained apart from standard external statistical results, so no circular step is present.

Assumptions & free parameters 1 free parameters · 8 assumptions · 0 invented entities

The central derivation has no fitted constants and introduces no new physical or mathematical entities. The main input assumptions are the contextual BTL specification, the rate condition on the nuisance estimator, the comparison graph regularity, and (for the extension) boundedness and knownness of the density ratio. The empirical density-ratio estimate is the only fitted quantity that enters the experiments without a matching theoretical justification.

free parameters (1)
  • number of PCA components for density ratio = 6
    Chosen by ten-fold cross-validation in the MMLU domain-shift experiment (Section 6.2). Not part of the central theoretical claim, but used to estimate the density ratio kappa that the theory treats as known.
assumptions (8)
  • domain assumption Contextual BTL logistic model: P(Y_ij=1|x)=psi(theta_i(x)-theta_j(x)) with psi(t)=1/(1+e^{-t}) (Section 2.1).
    Defines the loss (5), residuals, and Fisher information weights; misspecification would break the efficiency and coverage results.
  • domain assumption Identifiability via mean-zero normalization 1^T theta(x)=0 and bounded scores in [-C,C] (Eqs. (3)-(4)).
    Needed for the pseudoinverse argument in Theorem 1 and for uniform bounds throughout the proofs.
  • domain assumption Data are i.i.d. contexts X_ijell from a reference distribution on a compact set, with L independent comparisons per edge; P(X in Omega) >= c_Omega (Section 2.2).
    Used for all sampling and variance calculations; repeated LLM comparisons on the same question are modeled as conditionally independent Bernoulli draws.
  • domain assumption Comparison graph is Erdos-Renyi with p large enough, or a deterministic A satisfying E_good in Eq. (25).
    Provides spectral concentration and connectedness used in Lemma S8 and Theorems 2 and 5.
  • domain assumption The score estimator btheta achieves Assumption 1, Eq. (20): E_2(btheta,theta*|A)=o(1/(n^3 p) and 1/(n sqrt(npL))).
    This rate condition is the bridge between nuisance estimation and valid debiased inference; if violated, the residual bias terms in Theorem S9 do not vanish.
  • domain assumption theta* is Holder smooth with exponent beta in (0,1] for the efficiency equivalence Theorem 5 and Laplacian perturbation Theorem S12.
    Controls the uniform difference between the graph Laplacian pseudoinverse and the diagonal degree inverse.
  • domain assumption Density ratio kappa=p_t/p_s is known and bounded for Theorem 7.
    The domain-shift inference is proven only for known kappa; the experiments estimate it, which is not covered by the theorem.
  • standard math Standard probabilistic and matrix tools: Wedin pseudoinverse perturbation, Bernstein/Berry-Esseen inequalities, spectral concentration for Erdos-Renyi graphs.
    Used throughout the supplementary proofs without modification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fisher Random Walk: Automatic Debiasing Contextual Preference Inference for Large Language Model Evaluation." pith.science (2026). https://pith.science/paper/GYXMUCT6

@misc{pith2026250905852,
  author       = {Pith},
  title        = {Pith review of: Fisher Random Walk: Automatic Debiasing Contextual Preference Inference for Large Language Model Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GYXMUCT6}},
  note         = {Machine review of arXiv:2509.05852}
}
read the original abstract

Motivated by the need for rigorous and scalable evaluation of large language models, we study contextual preference inference for pairwise comparison functionals of context-dependent preference score functions across domains. Focusing on the contextual Bradley-Terry-Luce model, we develop a semiparametric efficient estimator that automates the debiased estimation through aggregating weighted residual balancing terms across the comparison graph. We show that the efficiency is achieved when the weights are derived from a novel strategy called Fisher random walk. We also propose a computationally feasible method to compute the weights by a potential representation of nuisance weight functions. We show our inference procedure is valid for general score function estimators accommodating the practitioners' need to implement flexible deep learning methods. We extend the procedure to multiple hypothesis testing using a Gaussian multiplier bootstrap that controls familywise error and to distributional shift via a cross-fitted importance-sampling adjustment for target-domain inference. Numerical studies, including language model evaluations under diverse contexts, corroborate the accuracy, efficiency, and practical utility of our method.

Figures

Figures reproduced from arXiv: 2509.05852 by the authors.

Figure 1
Figure 1. In panel (a), one graph path R = {(i0, i1),(i1, i2), . . . ,(iS−1, j0)} connects node i0 and j0. In panel (b), paths in different colors represent distinct random walks sampled from any prior distribution R. all (i, j) ∈ E(A) is not straightforward. In particular, the weights do not admit closed-forms, and it is computationally expensive to compute all weights over the entire comparison graph, which are of order O(n… view at source ↗
Figure 2
Figure 2. In panel 2a, the asymptotic normality region (blue) lies above both the red and blue dashed lines. In panels 2b and 2c, the corresponding regions (blue) lie above the blue dashed lines. suppose L ≍ n ξL for some ξL > 0 and p ≍ n −ξp log n for some ξp ∈ (0, 1], then if 2β ∗ > t∗ and ξL − ξp > 1 + 4t ∗ 2β ∗−t ∗ and ξL + t ∗ 2β ∗ ξp > 2 + 3t ∗ 2β ∗ , chooinsg δ = (npL) −1 , we can verify that the condition in (27) is s… view at source ↗
Figure 3
Figure 3. Simulation results under Setting I. In all panels, the colors of blue, red, and orange represent [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Simulation results under Setting II. The explanations of all panels are similar with Figure [PITH_FULL_IMAGE:figures/full_fig_p020_4.png]
Figure 5
Figure 5. Figure 5: P-values diagrams and the subsequent Hasse diagrams for the significant comparisons of LLMs [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 6
Figure 6. Figure 6: P-values diagrams and the subsequent Hasse diagrams for the significant comparisons of LLMs [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Visualization of the question distributions under different topics through 2-D PCA embeddings. [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: (a) Inference results based on Qij (Ωck). (b) Inference results based on Qij (Ωck | κcm:ck). (c) Inference results based on Qij (Ωcm). (a) (b) (c) [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 9
Figure 9. Figure 9: (a) Inference results based on Qij (Ωck). (b) Inference results based on Qij (Ωck | κmg:ck). (c) Inference results based on Qij (Ωmg). 25 [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Reinforcement Learning from Human Feedback: A Statistical Perspective

    stat.ML 2026-04 accept novelty 2.0 of 10

    A statistical survey of RLHF for LLM alignment that connects preference learning and policy optimization to models like Bradley-Terry-Luce while reviewing methods, extensions, and open challenges.

Reference graph

Works this paper leans on

94 extracted references · 60 canonical work pages · cited by 1 Pith paper

  1. [1]

    Anthropic, A. (2024). The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card\/ 1

    2024

  2. [2]

    Athey, S., G. W. Imbens, and S. Wager (2018). Approximate residual balancing: debiased inference of average treatment effects in high dimensions. Journal of the Royal Statistical Society Series B: Statistical Methodology\/ 80\/ (4), 597--623

    2018

  3. [3]

    Cahyawijaya, N

    Bang, Y., S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung, et al. (2023). A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023\/

    arXiv 2023

  4. [4]

    Baraud, Y. and L. Birg \'e (2014). Estimating composite functions by model selection. In Annales de l'IHP Probabilit \'e s et statistiques , Volume 50, pp.\ 285--314

    2014

  5. [5]

    Bartlett, P. L., A. Montanari, and A. Rakhlin (2021). Deep learning: a statistical viewpoint. Acta numerica\/ 30 , 87--201

    2021

  6. [6]

    Bauer, B. and M. Kohler (2019). On deep learning as a remedy for the curse of dimensionality in nonparametric regression. The Annals of Statistics\/ 47\/ (4), 2261

    2019

  7. [7]

    Adlakha, M

    BehnamGhader, P., V. Adlakha, M. Mosbach, D. Bahdanau, N. Chapados, and S. Reddy (2024). LLM2Vec : Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961\/

    arXiv 2024

  8. [8]

    Chernozhukov, D

    Belloni, A., V. Chernozhukov, D. Chetverikov, C. Hansen, and K. Kato (2018). High-dimensional econometrics and regularized GMM . arXiv preprint arXiv:1806.01888\/

    arXiv 2018

Show all 94 references
  1. [9]

    Bi, X., D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu, et al. (2024). Deepseek LLM : Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954\/

    2024 arXiv

  2. [10]

    Bickel, P. J., C. A. Klaassen, P. J. Bickel, Y. Ritov, J. Klaassen, J. A. Wellner, and Y. Ritov (1993). Efficient and adaptive estimation for semiparametric models , Volume 4. Springer

    1993

  3. [11]

    Liang, and T

    Bommasani, R., P. Liang, and T. Lee (2023). Holistic evaluation of language models. Annals of the New York Academy of Sciences\/ 1525\/ (1), 140--146

    2023

  4. [12]

    Bos, T. and J. Schmidt-Hieber (2022). Convergence rates of deep relu networks for multiclass classification. Electronic Journal of Statistics\/ 16\/ (1), 2724--2773

    2022

  5. [13]

    Bradley, R. A. and M. E. Terry (1952). Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika\/ 39\/ (3/4), 324--345

    1952

  6. [14]

    Buhr, C. R., H. Smith, T. Huppertz, K. Bahr-Hamm, C. Matthias, A. Blaikie, T. Kelsey, S. Kuhn, and J. Eckrich (2023). Chatgpt versus consultants: blinded evaluation on answering otorhinolaryngology case--based questions. JMIR Medical Education\/ 9\/ (1), e49183

    2023

  7. [15]

    Chen, L. H., L. Goldstein, and Q.-M. Shao (2010). Normal approximation by Stein's method . Springer Science & Business Media

    2010

  8. [16]

    Gao, and A

    Chen, P., C. Gao, and A. Y. Zhang (2022). Partial recovery for top-k ranking: optimality of MLE and suboptimality of the spectral method. The Annals of Statistics\/ 50\/ (3), 1618--1652

    2022

  9. [17]

    Chetverikov, M

    Chernozhukov, V., D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey, and J. Robins (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal\/ 21\/ (1), C1--C68

    2018

  10. [18]

    Chetverikov, and K

    Chernozhukov, V., D. Chetverikov, and K. Kato (2013). Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors. Annals of Statistics\/ 41\/ (6), 2786--2819

    2013

  11. [19]

    Newey, W

    Chernozhukov, V., M. Newey, W. K. Newey, R. Singh, and V. Srygkanis (2023). Automatic debiased machine learning for covariate shifts. arXiv preprint arXiv:2307.04527\/

    2023

  12. [20]

    Chernozhukov, V., W. K. Newey, V. Quintas-Martinez, and V. Syrgkanis (2021). Automatic debiased machine learning via R iesz regression. arXiv preprint arXiv:2104.14737\/

    2021 arXiv

  13. [21]

    Chernozhukov, V., W. K. Newey, and R. Singh (2022). Automatic debiased machine learning of causal and structural effects. Econometrica\/ 90\/ (3), 967--1027

    2022

  14. [22]

    Zheng, Y

    Chiang, W.-L., L. Zheng, Y. Sheng, A. N. Angelopoulos, T. Li, D. Li, H. Zhang, B. Zhu, M. Jordan, J. E. Gonzalez, et al. (2024). Chatbot Arena : An open platform for evaluating LLM s by human preference. arXiv preprint arXiv:2403.04132\/

    2024 arXiv

  15. [23]

    Christiano, P. F., J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei (2017). Deep reinforcement learning from human preferences. Advances in neural information processing systems\/ 30

    2017

  16. [24]

    Collins, K. M., A. Q. Jiang, S. Frieder, L. Wong, M. Zilka, U. Bhatt, T. Lukasiewicz, Y. Wu, J. B. Tenenbaum, W. Hart, et al. (2024). Evaluating language models for mathematics through interactions. Proceedings of the National Academy of Sciences\/ 121\/ (24), e2318124121

    2024

  17. [25]

    and N.-B

    Dao, X.-Q. and N.-B. Le (2023). Investigating the effectiveness of ChatGPT in mathematical reasoning and problem solving: Evidence from the vietnamese national high school graduation examination. arXiv preprint arXiv:2306.06331\/

    2023 arXiv

  18. [26]

    De Abreu, N. M. M. (2007). Old and new results on algebraic connectivity of graphs. Linear algebra and its applications\/ 423\/ (1), 53--73

    2007

  19. [27]

    Debreu, G. (1960). Individual choice behavior: A theoretical analysis

    1960

  20. [28]

    Goyal, N

    Didolkar, A., A. Goyal, N. R. Ke, S. Guo, M. Valko, T. Lillicrap, D. Rezende, Y. Bengio, M. Mozer, and S. Arora (2024). Metacognitive capabilities of LLM s: An exploration in mathematical problem solving. arXiv preprint arXiv:2405.12205\/

    2024 arXiv

  21. [29]

    Doyle, P. G. and J. L. Snell (1984). Random walks and electric networks , Volume 22. American Mathematical Soc

    1984

  22. [30]

    Knowles, H.-T

    Erd o s, L., A. Knowles, H.-T. Yau, and J. Yin (2013). Spectral statistics of Erd o s--R \'e nyi graphs I : Local semicircle law

    2013

  23. [31]

    Gu, and W.-X

    Fan, J., Y. Gu, and W.-X. Zhou (2024). How do noise tails impact on deep relu networks? The Annals of Statistics\/ 52\/ (4), 1845--1871

    2024

  24. [32]

    Hou, and M

    Fan, J., J. Hou, and M. Yu (2024). Uncertainty quantification of MLE for entity ranking with covariates. Journal of Machine Learning Research\/ 25\/ (358), 1--83

    2024

  25. [33]

    Ma, and Y

    Fan, J., C. Ma, and Y. Zhong (2020). A selective overview of deep learning. Statistical science: a review journal of the Institute of Mathematical Statistics\/ 36\/ (2), 264

    2020

  26. [34]

    Farrell, M. H., T. Liang, and S. Misra (2021). Deep neural networks for estimation and inference. Econometrica\/ 89\/ (1), 181--213

    2021

  27. [35]

    Foster, D. J. and V. Syrgkanis (2023). Orthogonal statistical learning. The Annals of Statistics\/ 51\/ (3), 879--908

    2023

  28. [36]

    Shen, and A

    Gao, C., Y. Shen, and A. Y. Zhang (2023). Uncertainty quantification in the Bradley--Terry--Luce model. Information and Inference: A Journal of the IMA\/ 12\/ (2), 1073--1140

    2023

  29. [37]

    Ghorbani, B., S. Mei, T. Misiakiewicz, and A. Montanari (2020). Discussion of: `` N onparametric regression using deep neural networks with ReLU activation function''. The Annals of Statistics\/ , 1898--1901

    2020

  30. [38]

    Boyd, and A

    Ghosh, A., S. Boyd, and A. Saberi (2008). Minimizing effective resistance of a graph. SIAM review\/ 50\/ (1), 37--66

    2008

  31. [39]

    Griffiths, D. J. (2023). Introduction to electrodynamics . Cambridge University Press

    2023

  32. [40]

    Resnick, and J

    Halliday, D., R. Resnick, and J. Walker (2013). Fundamentals of physics . John Wiley & Sons

    2013

  33. [41]

    Han, R., R. Ye, C. Tan, and K. Chen (2020). Asymptotic theory of sparse Bradley--Terry model. The Annals of Applied Probability\/ 30\/ (5), 2491--2515

    2020

  34. [42]

    Price, and N

    Hardt, M., E. Price, and N. Srebro (2016). Equality of opportunity in supervised learning. Advances in neural information processing systems\/ 29

    2016

  35. [43]

    Burns, S

    Hendrycks, D., C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt (2020). Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300\/

    2020 arXiv

  36. [44]

    Higham, N. J. (2021). Functions of Matrices: Theory and Computation . Society for Industrial and Applied Mathematics

    2021

  37. [45]

    Horowitz, J. L. and E. Mammen (2007). Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions. The Annals of Statistics\/ , 2589--2619

    2007

  38. [46]

    Ichimura, H. and W. K. Newey (2022). The influence function of semiparametric estimators. Quantitative Economics\/ 13\/ (1), 29--61

    2022

  39. [47]

    Imai, K. and M. Ratkovic (2014). Covariate balancing propensity score. Journal of the Royal Statistical Society Series B: Statistical Methodology\/ 76\/ (1), 243--263

    2014

  40. [48]

    Lepski, and A

    Juditsky, A., O. Lepski, and A. Tsybakov (2009). Nonparametric estimation of composite functions. Annals of Statistics\/ 37\/ (3), 1360--1404

    2009

  41. [49]

    Hido, and M

    Kanamori, T., S. Hido, and M. Sugiyama (2009). A least-squares approach to direct importance estimation. The Journal of Machine Learning Research\/ 10 , 1391--1445

    2009

  42. [50]

    Ohn, and D

    Kim, Y., I. Ohn, and D. Kim (2021). Fast convergence rates of deep neural networks for classification. Neural Networks\/ 138 , 179--197

    2021

  43. [51]

    Klein, D. J. and M. Randić (1993). Resistance distance. Journal of Mathematical Chemistry\/ 12 , 81--95

    1993

  44. [52]

    Kohler, M. and S. Langer (2021). On the rate of convergence of fully connected deep neural network regression estimates. The Annals of Statistics\/ 49\/ (4), 2231--2249

    2021

  45. [53]

    Lei, J. and A. Rinaldo (2015). Consistency of spectral clustering in stochastic block models. The Annals of Statistics\/ 43\/ (1), 215--237

    2015

  46. [54]

    Zhang, Y

    Li, X., T. Zhang, Y. Dubois, R. Taori, I. Gulrajani, C. Guestrin, P. Liang, and T. B. Hashimoto (2023). Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval

    2023

  47. [55]

    Liu, Y., E. X. Fang, and J. Lu (2023). Lagrangian inference for ranking problems. Operations research\/ 71\/ (1), 202--223

    2023

  48. [56]

    Ma, P., R. Ding, S. Wang, S. Han, and D. Zhang (2023). InsightPilot : An LLM -empowered automated data exploration system. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pp.\ 346--352

    2023

  49. [57]

    Mahoney, M. W. (2016). Lecture notes on spectral graph methods. arXiv preprint arXiv:1608.04845\/

    2016 arXiv

  50. [58]

    Maurer, A. and M. Pontil (2009). Empirical bernstein bounds and sample variance penalization. arXiv preprint arXiv:0907.3740\/

    2009 arXiv

  51. [59]

    Ni, A., S. Iyer, D. Radev, V. Stoyanov, W.-t. Yih, S. Wang, and X. V. Lin (2023). Lever: Learning to verify language-to-code generation with execution. In International Conference on Machine Learning , pp.\ 26106--26128. PMLR

    2023

  52. [60]

    OpenAI, R. (2023). GPT-4 technical report. arxiv 2303.08774. View in Article\/ 2\/ (5)

    2023 arXiv

  53. [61]

    Ouyang, L., J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. (2022). Training language models to follow instructions with human feedback. Advances in neural information processing systems\/ 35 , 27730--27744

    2022

  54. [62]

    Morales, J

    Pe \ n a, A., A. Morales, J. Fierrez, I. Serna, J. Ortega-Garcia, I. Puente, J. Cordova, and G. Cordova (2023). Leveraging large language models for topic classification in the domain of public affairs. In International Conference on Document Analysis and Recognition , pp.\ 20...

    2023

  55. [63]

    Reimers, N. and I. Gurevych (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP...

    2019

  56. [64]

    Robins, J. M., M. A. Hernan, and B. Brumback (2000). Marginal structural models and causal inference in epidemiology. Epidemiology\/ 11\/ (5), 550--560

    2000

  57. [65]

    Robins, J. M., A. Rotnitzky, and L. P. Zhao (1995). Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. Journal of the american statistical association\/ 90\/ (429), 106--121

    1995

  58. [66]

    Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with relu activation function. The Annals of Statistics\/ 48\/ (4), 1875--1897

    2020

  59. [67]

    Shen, G., Y. Jiao, Y. Lin, and J. Huang (2022). Approximation with CNN s in Sobolev space: with applications to classification. Advances in neural information processing systems\/ 35 , 2876--2888

    2022

  60. [68]

    and Y.-C

    Simons, G. and Y.-C. Yao (1998). Approximating the inverse of a symmetric positive definite matrix. Linear algebra and its applications\/ 281\/ (1-3), 97--103

    1998

  61. [69]

    Xu, and A

    Singh, R., L. Xu, and A. Gretton (2024). Kernel methods for causal functions: dose, heterogeneous and incremental response curves. Biometrika\/ 111\/ (2), 497--516

    2024

  62. [70]

    Spielman, D. (2012). Spectral graph theory. Combinatorial scientific computing\/ 18 , 18

    2012

  63. [71]

    Spielman, D. A. and N. Srivastava (2008). Graph sparsification by effective resistances. In Proceedings of the fortieth annual ACM symposium on Theory of computing , pp.\ 563--568

    2008

  64. [72]

    Ouyang, J

    Stiennon, N., L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano (2020). Learning to summarize with human feedback. Advances in Neural Information Processing Systems\/ 33 , 3008--3021

    2020

  65. [73]

    Suh, N. and G. Cheng (2024). A survey on statistical theory of deep learning: Approximation, training dynamics, and generative models. Annual Review of Statistics and Its Application\/ 12

    2024

  66. [74]

    Martin, K

    Touvron, H., L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. (2023). Llama 2 : Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288\/

    2023 arXiv

  67. [75]

    Tropp, J. A. et al. (2015). An introduction to matrix concentration inequalities. Foundations and Trends extregistered in Machine Learning\/ 8\/ (1-2), 1--230

    2015

  68. [76]

    Bibaut, N

    van der Laan, L., A. Bibaut, N. Kallus, and A. Luedtke (2025). Automatic debiased machine learning for smooth functionals of nonparametric M -estimands. arXiv preprint arXiv:2501.11868\/

    2025

  69. [77]

    Van Der Vaart, A. and J. Wellner (1996). Weak Convergence and Empirical Processes: With Applications to Statistics . Springer Series in Statistics. Springer

    1996

  70. [78]

    Van der Vaart, A. W. (2000). Asymptotic statistics , Volume 3. Cambridge university press

    2000

  71. [79]

    Vershynin, R. (2018). High-dimensional probability: An introduction with applications in data science , Volume 47. Cambridge university press

    2018

  72. [80]

    He, and G

    Wang, J., X. He, and G. Xu (2020). Debiased inference on treatment effect in a high-dimensional model. Journal of the American Statistical Association\/ 115\/ (529), 442--454

    2020

  73. [81]

    Wang, J., X. Hu, W. Hou, H. Chen, R. Zheng, Y. Wang, L. Yang, H. Huang, W. Ye, X. Geng, et al. (2023). On the robustness of ChatGPT : An adversarial and out-of-distribution perspective. arXiv preprint arXiv:2302.12095\/

    2023 arXiv

  74. [82]

    Wang, Z., Y. Han, E. X. Fang, L. Wang, and J. Lu (2024). Ranking of large language model with nonparametric prompts. arXiv preprint arXiv:2412.05506\/

    2024 arXiv

  75. [83]

    Wedin, P.- . (1973). Perturbation theory for pseudo-inverses. BIT Numerical Mathematics\/ 13 , 217--232

    1973

  76. [84]

    Suzuki, T

    Yamada, M., T. Suzuki, T. Kanamori, H. Hachiya, and M. Sugiyama (2013). Relative density-ratio estimation for robust distribution comparison. Neural computation\/ 25\/ (5), 1324--1370

    2013

  77. [85]

    Yang, K.-C. and F. Menczer (2023). Large language models can rate news outlet credibility. arXiv preprint arXiv:2304.00228\/

    2023 arXiv

  78. [86]

    Zemel, R., Y. Wu, K. Swersky, T. Pitassi, and C. Dwork (2013). Learning fair representations. In International conference on machine learning , pp.\ 325--333. PMLR

    2013

  79. [87]

    Zhang, K., Y. Kang, F. Zhao, and X. Liu (2024). LLM -based medical assistant personalization with short-and long-term memory coordination. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn...

    2024

  80. [88]

    Chiang, Y

    Zheng, L., W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. (2024). Judging LLM -as-a-judge with MT-Bench and Chatbot Arena . Advances in Neural Information Processing Systems\/ 36

    2024

  81. [89]

    Zhou, T.-Y. and X. Huo (2024). Classification of data generated by gaussian mixture models using deep relu networks. Journal of Machine Learning Research\/ 25\/ (190), 1--54

    2024

  82. [90]

    Zhuo, T. Y., Z. Li, Y. Huang, F. Shiri, W. Wang, G. Haffari, and Y.-F. Li (2023). On robustness of prompt-based semantic parsing with large pre-trained language model: An empirical study on codex. arXiv preprint arXiv:2301.12868\/

    2023 arXiv

  83. [91]

    Zubizarreta, J. R. (2015). Stable weights that balance covariates for estimation with incomplete outcome data. Journal of the American Statistical Association\/ 110\/ (511), 910--922

    2015

  84. [92]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  85. [93]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  86. [94]

    Q. @@ O B' ` n nC_hWZs @!Ns > ^ Jy

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    1999

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.