Pith. sign in

REVIEW 3 major objections 4 minor 39 references

Catoni Contextual Bandits are Robust to Heavy-tailed Rewards

T0 review · 3 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Contextual bandits can achieve variance-scale regret under heavy-tailed rewards.

desk verdict Known-variance result is solid; unknown-variance log-R claim doesn't survive contact with Assumption 1. read the letter →

arxiv 2502.02486 v1 pith:HKZBBPV6 submitted 2025-02-04 stat.ML cs.LG

Chenlu Ye , Yujia Jin , Alekh Agarwal , Tong Zhang This is my paper · ORCID
classification stat.MLcs.LG MSC 62G3562L0568Q32
keywords contextualbanditsheavy-tailedrewardsCatoniestimatorvariance-awareregretgeneralfunctionapproximationeluderdimensionunknownvariancerobuststatistics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that contextual bandits with general function approximation can enjoy regret governed by the cumulative variance of the rewards rather than by their worst-case range $R$, even when rewards are heavy-tailed. Its known-variance algorithm, Catoni-OFUL, builds confidence sets from Catoni's robust mean estimator applied to excess-loss terms, and attains regret $\tilde{O}\bigl(\sqrt{\sum_{t} \sigma_t^2 \cdot d_F \ln N_F} + d_F \ln N_F\bigr)$ with $R$ appearing only logarithmically. For unknown variances, the peeling-based Variance-Agnostic Catoni Bandit reaches $\tilde{O}\bigl(d_F \sqrt{\sum_t \sigma_t^2 \ln N_F} + d_F (\ln N_F)^{3/4}(\sqrt{c_\eta}+\sigma_\eta)\bigr)$ under an assumption relating fourth moments to variances. A matching lower bound $\Omega(\sqrt{\mathbb{E}\sum_t \sigma_t^2})$ shows the leading variance term cannot be improved. If correct, these bounds make heavy-tailed or large-range rewards far less costly than earlier range-dependent guarantees suggested.

What carries the argument

The load-bearing object is Catoni's mean estimator, the unique zero of $\sum_i \Psi(\theta(Z_i-x))$ with $\Psi(x)=\log(1+x+x^2/2)$ for $x\ge 0$ and its antisymmetric counterpart for $x<0$. Its concentration inequality, made uniform over the parameter $\theta$, gives an excess-loss estimate whose error depends on the cumulative variance of the noise and only logarithmically on its bound $R$. Around this, the paper builds variance-weighted OFUL confidence sets using eluder dimension as the function-class complexity measure; the unknown-variance algorithm adds a SupLinUCB-style peeling hierarchy in which each level uses a plug-in Catoni variance estimator, replacing per-round variances with weights derived from the uncertainty level.

What would settle it

Run the two-armed bandit from the paper's lower-bound construction with known variances: one arm deterministic and the other heavy-tailed with variance $\sigma^2$ and large range $R$. If any algorithm's regret grows polynomially in $R$ rather than tracking $\sqrt{\sum_t \sigma_t^2}$, the claimed logarithmic-$R$ dependence fails.

Watch

Extended reading notes

Core claim

The central discovery is that robustifying the excess loss, not the reward estimate, is what carries variance-aware regret to general function classes. Over the sample history, the algorithm forms variance-weighted squared losses and replaces the noise cross term with a Catoni estimator whose concentration error scales with the sum of variances rather than with $R$. This keeps the true function in the confidence set with a radius depending only logarithmically on $R$. The same device is extended to unknown per-round variances by layering uncertainty levels and estimating one aggregate variance with another Catoni estimate; the resulting regret matches the known-variance bound up to an extra eluder-dimension and logarithmic factor. The paper also constructs two-armed bandit instances showing that any algorithm must pay $\sqrt{\text{sum of chosen-action variances}}$ in the leading term, so the variance term in the upper bound is optimal.

Load-bearing premise

For the unknown-variance bound, the algorithm's parameters require the learner to know the global bounds $\sigma_\eta$ and $c_\eta$ on the noise variance and its fluctuation; if those constants are truly unavailable, the variance-agnostic guarantee cannot be instantiated.

Editorial extensions

If this is right

  • With known variances, any realizable general function class gets $\tilde{O}\bigl(\sqrt{\sum_t \sigma_t^2 \, d_F \ln N_F} + d_F \ln N_F\bigr)$ regret, so deterministic rewards leave only a complexity term and $R$ is never polynomial in the bound.
  • With unknown variances, the same robustness holds up to an extra factor $d_F$ and a $(\ln N_F)^{3/4}(\sqrt{c_\eta}+\sigma_\eta)$ term, so no separate function class for predicting per-round variance is needed.
  • The lower bound shows that the leading regret term must scale with the variances of the actions the algorithm actually chooses, not with the variances of the optimal policy's actions.
  • For linear function classes, the eluder dimension is logarithmic in the dimension, so the bounds reduce to roughly $d\sqrt{\sum_t \sigma_t^2}$-style guarantees while removing the range dependence present in earlier linear heavy-tailed results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the unknown-variance algorithm's parameters require the global bounds $\sigma_\eta$ and $c_\eta$, the result is best read as removing per-round variance estimation rather than removing all knowledge about variance; an algorithm that must discover even those global constants would need an additional adaptation layer.
  • Editorial inference: the same robust-excess-loss construction may transfer to adversarial corruption and to Markov decision processes with nonlinear function approximation, since those settings also use weighted-regression confidence sets; the paper itself lists these as future directions.
  • Editorial inference: the candidate-set variant of Catoni-OFUL suggests the min-max optimization in the main algorithm is not essential, and a randomized choice from a robust candidate set may be substantially easier to implement in practice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper develops contextual bandit algorithms for general function approximation when rewards are heavy-tailed with bounded conditional variance, aiming for regret that scales with cumulative variance rather than with the worst-case range R. The known-variance algorithm (Catoni-OFUL) uses Catoni's mean estimator to robustify an excess-loss regression oracle and claims a regret bound of ~O(Lf sqrt(sum_t sigma_t^2 * dim(F) log N_F) + Lf dim(F) log N_F), i.e. polylog R. A two-arm lower bound in Theorem 1 shows the leading variance term is minimax optimal. For unknown variances, the paper proposes a peel-based variance-agnostic algorithm (VACB) with a robust plug-in variance estimator and claims a similar variance-based bound with a fourth-moment factor, also polylog R. An alternative candidate-set algorithm is given in Appendix D.

Significance. If the known-variance theorem is correct, it is a substantial step: it removes the polynomial R dependence previously present for general function classes and matches the lower bound. The proof appears detailed and uses a genuine uniform-in-theta Catoni concentration inequality. The lower bound is a clean two-instance argument and does not rely on the upper-bound machinery. The unknown-variance result, by contrast, is not yet established as advertised: the formal theorem contains an unacknowledged factor (sqrt(c_eta) + sigma_eta) that can be polynomial in R under Assumption 1, and the appendix contains an admitted gap in the proof of the variance-estimation lemma. With those items fixed and the assumptions made explicit, this would be a strong paper.

major comments (3)
  1. [Appendix C.1, Lemma 15] The proof of Lemma 15 contains an explicit note: "There is a fixable error. Now, I change the plug-in in Var_hat_t from f_hat_t to f_hat_{t-1}, and change the analysis above, so now the upper and lower bound is reasonable." Since this note states that the printed argument as it stands is not the correct one, and since Lemma 15 is the load-bearing variance-estimation accuracy step for Theorem 5 (Lemma 16, and hence Theorem 3/5), the unknown-variance upper bound is not fully proven as printed. I would ask for a complete rewrite of this part, with consistent notation (Lemma 15 labels the quantity Vart(f_hat_t) while the estimator on the right-hand side is defined with f_hat_{t-1}).
  2. [Theorem 5, Table 1, abstract] The formal regret bound in Theorem 5 contains the factor Lf * dim_{1,T}(F) * (log N)^{3/4} * (sqrt(c_eta) + sigma_eta). Under Assumption 1, c_eta and sigma_eta are unconstrained relative to the reward range R. For the valid noise distribution eta defined by P(eta=+-1)=(1-R^{-2})/2 and P(eta=+-R)=1/(2R^2), we have E[eta]=0, E[eta^2] approx 2, and Var[eta^2] = Theta(R^2), so Assumption 1 holds with bounded sigma_eta and c_eta = Theta(R^2). For this instance the displayed lower-order term is Theta(R), not polylog(R), so the advertised logarithmic R dependence for VACB (abstract, Table 1, informal Theorem 3) does not follow from the stated assumptions. The authors need to either add an explicit condition such as c_eta, sigma_eta = O(polylog(R)) or weaken the claim.
  3. [Algorithm 2, Theorem 5] Algorithm 2 and Theorem 5 require knowledge of sigma_eta and c_eta: gamma = 1/(sigma_eta T^{3/2}) in Theorem 5, and Table 3 parameters (theta_{t,l}^{Var}, theta_t^l, beta_t^l) depend on sigma_eta and c_eta. The paper does not state that the learner must know these global bounds. Without stating this, the "unknown-variance" algorithm cannot be instantiated as described, and the abstract's claim that the method "remove[s] the need for cumbersome variance estimation" is not literally accurate. This is a structural assumption about the algorithm, distinct from the regret-bound claim, and should be stated explicitly in the main text.
minor comments (4)
  1. [Section 5] The conclusion states that the known-variance algorithm "enjoys a variance-based regret bound with only polynomial dependence on R", which contradicts Theorem 2/4's polylog R claim; this should be corrected to "polylogarithmic".
  2. [Section 4.2, proof sketch, Part II] The phrase "For any t in [t]" appears with the same symbol t on both sides; it should be "for any i in [t]" or "for any t in [T]".
  3. [Appendix C.1, Lemmas 12, 15, 16] The definition of theta_t^{Var} is inconsistent across the appendix: Lemma 12 uses (4(2 sigma_eta^2 + c_eta + L_f^2 + 16 rho^2 beta_{t-1})^2)^{-1}, Lemma 15 uses (12(2 sigma_eta^2 + c_eta + L_f^2 + 16 rho^2 beta_{t-1}^2))^{-1}, and Lemma 16 and Table 3 use different forms with 2^{-2l+4}. Please reconcile all occurrences and check that the constants in the proofs match the algorithm specification.
  4. [Table 1] The row for Heavy-OFUL (Huang et al., 2024) is left blank with only a footnote reference; the table would be more readable if the explicit bound were shown directly or stated as identical to AdaOFUL in the table body.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the regret bounds are derived from Catoni concentration inequalities and standard optimistic/peeling analyses, not from self-citations or fitted inputs.

full rationale

The derivation chain is self-contained. The known-variance result (Theorem 2 / Theorem 4) builds confidence sets from the uniform-in-theta Catoni concentration inequality (Lemma 1, formalized as Lemma 23), whose proof reduces to independent results from Wei et al. (2020) and Wagenmaker et al. (2022); no fitted parameter is relabeled as a prediction. The unknown-variance result (Theorem 3 / Theorem 5) estimates only an aggregate variance using the same Catoni mean, and Lemma 5 proves that this estimator is within constants of the true weighted variance; the proof conditions E_t^var and E_t^conv on each other sequentially, which is legitimate mutual conditioning, not circularity. The lower bound (Theorem 1) uses a two-point construction with KL divergence; although the quantity E[sum_t sigma_t^2] is algorithm-dependent, the proof lower-bounds regret on a fixed instance by Omega(sigma sqrt(T)) and then converts to Omega(sqrt(max_i E_i[sum_t sigma_t^2])) using the variance upper bounds of the constructed arms, so it is not a definitional identity. The two flagged concerns are correctness or overclaim issues rather than circularity. First, Theorem 5's formal bound contains an explicit Lf * dim_{1,T}(F) * (log N)^{3/4} * (sqrt(c_eta) + sigma_eta) term, while the abstract and Table 1 suppress this factor; the advertised logarithmic R dependence for VACB is therefore only valid when sigma_eta and c_eta grow polylogarithmically in R, and Assumption 1 alone permits c_eta = Theta(R^2). Second, Appendix C.1 contains an admitted 'fixable error': the text says 'There is a fixable error. Now, I change the plug-in in ^Var_t from ^f_t to ^f_{t-1}', so the printed proof of Theorem 5 is incomplete as stated. These are rigor and interpretation concerns, not circular reductions of the derivation to its inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard realizability and boundedness assumptions, plus an explicit fourth-moment assumption for the unknown-variance result. The algorithm hyperparameters alpha, lambda, upsilon, and gamma are not fitted to data, but the gamma choice reveals an unstated knowledge requirement: the learner must know sigma_eta. No new physical or mathematical entities are introduced; Catoni's estimator and eluder dimension are existing objects.

free parameters (4)
  • alpha = 1/sqrt(T)
    Weight floor in the known-variance algorithm; chosen by hand to avoid division by zero when sigma_t is zero. Not fitted to data.
  • lambda = Theta(1)
    Regularizer in the eluder-dimension denominators; a theoretical choice, not data-dependent.
  • upsilon = O(1/(L_f^12 R^4 T^10))
    Covering resolution; chosen by hand so discretization error is absorbed into the confidence radius beta_t. It affects the log N(F, upsilon) term.
  • gamma = 1/(sigma_eta T^{3/2})
    Peeling threshold in Algorithm 2; requires knowing the global variance bound sigma_eta, an unstated knowledge requirement for the unknown-variance algorithm.
assumptions (5)
  • domain assumption Bounded heavy-tailed noise: |eta_t| <= R, E[eta_t] = 0, E[eta_t^2] <= sigma_t^2.
    Used throughout as the only control on the reward tail; introduced in Section 2.
  • domain assumption Realizability: f* in F and |f(x)| <= L_f for all f in F.
    The confidence sets are built around f*, so the true reward function must belong to the function class; stated in Section 2.
  • domain assumption Fourth-moment ratio bound (Assumption 1): Var[eta_t^2|F_t] <= c_eta Var[eta_t|F_t] and sigma_t^2 <= sigma_eta^2.
    Only needed for the unknown-variance theorem; additionally, the algorithm parameters in Table 3 require these constants to be known.
  • standard math Uniform-in-theta concentration for Catoni's estimator (Lemma 1 and Lemma 23).
    This concentration lemma is the engine of the analysis; it is proved in Appendix E from the existing Lemma 21, not assumed as a black box.
  • standard math Covering-number discretization at scale upsilon.
    Extends finite-class union-bound arguments to infinite function classes through Definition 1; standard in the cited literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Catoni Contextual Bandits are Robust to Heavy-tailed Rewards." pith.science (2026). https://pith.science/paper/HKZBBPV6

@misc{pith2026250202486,
  author       = {Pith},
  title        = {Pith review of: Catoni Contextual Bandits are Robust to Heavy-tailed Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HKZBBPV6}},
  note         = {Machine review of arXiv:2502.02486}
}
abstract

Typical contextual bandit algorithms assume that the rewards at each round lie in some fixed range $[0, R]$, and their regret scales polynomially with this reward range $R$. However, many practical scenarios naturally involve heavy-tailed rewards or rewards where the worst-case range can be substantially larger than the variance. In this paper, we develop an algorithmic approach building on Catoni's estimator from robust statistics, and apply it to contextual bandits with general function approximation. When the variance of the reward at each round is known, we use a variance-weighted regression approach and establish a regret bound that depends only on the cumulative reward variance and logarithmically on the reward range $R$ as well as the number of rounds $T$. For the unknown-variance case, we further propose a careful peeling-based algorithm and remove the need for cumbersome variance estimation. With additional dependence on the fourth moment, our algorithm also enjoys a variance-based bound with logarithmic reward-range dependence. Moreover, we demonstrate the optimality of the leading-order term in our regret bound through a matching lower bound.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 31 canonical work pages

  1. [1]

    Online learning in MDPs with side information

    Abbasi-Yadkori, Y. and Neu, G. (2014). Online learning in mdps with side information. arXiv preprint arXiv:1406.6812

    work page Pith review arXiv 2014

  2. [2]

    Abbasi-Yadkori, Y., P \'a l, D., and Szepesv \'a ri, C. (2011). Improved algorithms for linear stochastic bandits. Advances in neural information processing systems , 24

    2011

  3. [3]

    Agarwal, A., Jin, Y., and Zhang, T. (2023). VO Q L : Towards optimal regret in model-free rl with nonlinear function approximation. In The Thirty Sixth Annual Conference on Learning Theory , pages 987--1063. PMLR

    work page 2023

  4. [4]

    and Catoni, O

    Audibert, J.-Y. and Catoni, O. (2011). Robust linear least squares regression. The Annals of Statistics , 39(5):2766--2794

    work page 2011

  5. [5]

    Bubeck, S., Cesa-Bianchi, N., and Lugosi, G. (2013). Bandits with heavy tail. IEEE Transactions on Information Theory , 59(11):7711--7717

    work page 2013

  6. [6]

    F., Balseiro, S

    Choi, H., Mela, C. F., Balseiro, S. R., and Leary, A. (2020). Online display advertising markets: A literature review and future directions. Information Systems Research , 31(2):556--575

    work page 2020

  7. [7]

    Chu, W., Li, L., Reyzin, L., and Schapire, R. (2011). Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages 208--214. JMLR Workshop and Conference Proceedings

    work page 2011

  8. [8]

    Cont, R. (2001). Empirical properties of asset returns: stylized facts and statistical issues. Quantitative finance , 1(2):223

    work page 2001

Show all 39 references
  1. [9]

    Gentile, C., Wang, Z., and Zhang, T. (2022). Achieving minimax rates in pool-based batch active learning. In International Conference on Machine Learning , pages 7339--7367. PMLR

    2022

  2. [10]

    He, J., Zhao, H., Zhou, D., and Gu, Q. (2023). Nearly minimax optimal reinforcement learning for linear markov decision processes. In International Conference on Machine Learning , pages 12790--12822. PMLR

    2023

  3. [11]

    He, J., Zhou, D., Zhang, T., and Gu, Q. (2022). Nearly optimal algorithms for linear contextual bandits with adversarial corruptions. Advances in neural information processing systems , 35:34614--34625

    2022

  4. [12]

    Huang, J., Zhong, H., Wang, L., and Yang, L. (2024). Tackling heavy-tailed rewards in reinforcement learning with function approximation: Minimax optimal and instance-dependent regret bounds. Advances in Neural Information Processing Systems , 36

    2024

  5. [13]

    Hull, J. (2012). Risk management and financial institutions,+ Web Site , volume 733. John Wiley & Sons

    2012

  6. [14]

    I., Morshed, Z., Shankar, A., Arli, D., and Pentecost, R

    Jebarajakirthy, C., Maseeh, H. I., Morshed, Z., Shankar, A., Arli, D., and Pentecost, R. (2021). Mobile advertising: A systematic literature review and future research agenda. International Journal of Consumer Studies , 45(6):1258--1291

    2021

  7. [15]

    Kim, Y., Yang, I., and Jun, K.-S. (2021). Improved regret analysis for variance-adaptive linear bandits and horizon-free linear mixture mdps. arXiv preprint arXiv:2111.03289

    2021 arXiv

  8. [16]

    and Sun, Q

    Li, X. and Sun, Q. (2024). Variance-aware decision making with linear function approximation under heavy-tailed rewards. Transactions on Machine Learning Research

    2024

  9. [17]

    and Mendelson, S

    Lugosi, G. and Mendelson, S. (2019). Mean estimation and regression under heavy-tailed distributions: A survey. Foundations of Computational Mathematics , 19(5):1145--1190

    2019

  10. [18]

    Medina, A. M. and Yang, S. (2016). No-regret algorithms for heavy-tailed linear bandits. In International Conference on Machine Learning , pages 1642--1650. PMLR

    2016

  11. [19]

    Nair, J., Wierman, A., and Zwart, B. (2013). The fundamentals of heavy-tails: Properties, emergence, and identification. In Proceedings of the ACM SIGMETRICS/international conference on Measurement and modeling of computer systems , pages 387--388

    2013

  12. [20]

    Pacchiano, A. (2024). Second order bounds for contextual bandits with function approximation. arXiv preprint arXiv:2409.16197

    2024 arXiv

  13. [21]

    and Van Roy, B

    Russo, D. and Van Roy, B. (2013). Eluder dimension and the sample complexity of optimistic exploration. Advances in Neural Information Processing Systems , 26

    2013

  14. [22]

    Shao, H., Yu, X., King, I., and Lyu, M. R. (2018). Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems , 31

    2018

  15. [23]

    J., Chen, Y., Simchowitz, M., Du, S., and Jamieson, K

    Wagenmaker, A. J., Chen, Y., Simchowitz, M., Du, S., and Jamieson, K. (2022). First-order regret in reinforcement learning with linear function approximation: A robust estimation approach. In International Conference on Machine Learning , pages 22384--22429. PMLR

    2022

  16. [24]

    Wainwright, M. J. (2019). High-dimensional statistics: A non-asymptotic viewpoint , volume 48. Cambridge University Press

    2019

  17. [25]

    Wang, K., Kallus, N., and Sun, W. (2024a). The central role of the loss function in reinforcement learning. arXiv preprint arXiv:2409.12799

    2024 arXiv

  18. [26]

    Wang, K., Oertell, O., Agarwal, A., Kallus, N., and Sun, W. (2024b). More benefits of being distributional: Second-order bounds for reinforcement learning. arXiv preprint arXiv:2402.07198

    2024 arXiv

  19. [27]

    Wang, Z., Zhou, D., Lui, J., and Sun, W. (2024c). Model-based rl as a minimalist approach to horizon-free and second-order bounds. arXiv preprint arXiv:2408.08994

    2024 arXiv

  20. [28]

    Wei, C.-Y., Luo, H., and Agarwal, A. (2020). Taking a hint: How to leverage loss predictors in contextual bandits? In Conference on Learning Theory , pages 3583--3634. PMLR

    2020

  21. [29]

    Xue, B., Wang, G., Wang, Y., and Zhang, L. (2020). Nearly optimal regret for stochastic linear bandits with heavy-tailed payoffs. arXiv preprint arXiv:2004.13465

    2020 arXiv

  22. [30]

    Ye, C., He, J., Gu, Q., and Zhang, T. (2024a). Towards robust model-based reinforcement learning against adversarial corruption. arXiv preprint arXiv:2402.08991

    2024 arXiv

  23. [31]

    Ye, C., Xiong, W., Gu, Q., and Zhang, T. (2023). Corruption-robust algorithms with uncertainty weighting for nonlinear contextual bandits and markov decision processes. In International Conference on Machine Learning , pages 39834--39863. PMLR

    2023

  24. [32]

    Ye, C., Yang, R., Gu, Q., and Zhang, T. (2024b). Corruption-robust offline reinforcement learning with general function approximation. Advances in Neural Information Processing Systems , 36

    2024

  25. [33]

    Zhang, T. (2023a). Mathematical Analysis of Machine Learning Algorithms . Cambridge University Press. in press, also available as http://tongzhang-ml.org/lt-book.html

    2023

  26. [34]

    Zhang, T. (2023b). Mathematical analysis of machine learning algorithms . Cambridge University Press

    2023

  27. [35]

    Zhang, Z., Yang, J., Ji, X., and Du, S. S. (2021). Improved variance-aware confidence sets for linear bandits and linear mixture mdp. Advances in Neural Information Processing Systems , 34:4342--4355

    2021

  28. [36]

    Zhao, H., He, J., and Gu, Q. (2023a). A nearly optimal and low-switching algorithm for reinforcement learning with general function approximation. arXiv preprint arXiv:2311.15238

    2023

  29. [37]

    Zhao, H., He, J., Zhou, D., Zhang, T., and Gu, Q. (2023b). Variance-dependent regret bounds for linear bandits and reinforcement learning: Adaptivity and computational efficiency. In The Thirty Sixth Annual Conference on Learning Theory , pages 4977--5020. PMLR

    2023

  30. [38]

    and Gu, Q

    Zhou, D. and Gu, Q. (2022). Computationally efficient horizon-free reinforcement learning for linear mixture mdps. arXiv preprint arXiv:2205.11507

    2022 arXiv

  31. [39]

    Zhou, D., Gu, Q., and Szepesvari, C. (2021). Nearly minimax optimal reinforcement learning for linear mixture markov decision processes. In Conference on Learning Theory , pages 4532--4576. PMLR

    2021

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.