Pith. sign in

REVIEW 2 major objections 3 minor 34 references

Decision Making in Hybrid Environments: A Model Aggregation Approach

T0 review · 2 major / 3 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A partition of models and policies turns hybrid online learning into a tunable estimation-vs-decision trade-off, yielding $\sqrt{T}$ regret for linear $Q^\star/V^\star$ MDPs.

desk verdict The AIR^Phi framework is a genuinely new idea, but Theorem 2's proof rests on a false convexity-concavity claim, so the main regret bounds are unproven as written. read the letter →

arxiv 2502.05974 v2 pith:EIVNTPZH submitted 2025-02-09 cs.LG stat.ML

Haolin Liu , Chen-Yu Wei , Julian Zimmert This is my paper · ORCID
classification cs.LGstat.ML MSC 68Q3268W2790C40
keywords hybridonlinedecisionmakingdecision-estimationcoefficientmodelaggregationfixedtransitionadversarialrewardlinearQ-starV-starMDPsbilinearclassesregretbounds
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a general algorithm for sequential decision making in hybrid environments, where the rule by which outcomes are generated stays fixed but rewards can change adversarially over time. Its central claim is that the difficulty of such problems is captured by a partition $\Phi$ of the joint model-policy space, and that regret is bounded by $\log|\Phi|/\eta$ plus $T$ times the supremum of a partition-aware information ratio. This gives a tunable trade-off: coarser partitions pay less estimation cost but may increase decision complexity, and the learner can choose the best balance. In concrete settings the framework produces the first $\sqrt{T}$ regret for linear $Q^\star/V^\star$ MDPs without Bellman completeness, and improves known $T^{2/3}$ bounds for fixed-transition adversarial-reward MDPs to $\sqrt{T}$ or $T^{3/4}$ depending on the information available.

What carries the argument

The central object is the partition-indexed information ratio $\mathrm{AIR}^{\Phi}_{\rho,\eta}(p,\nu)$, which measures the expected regret of playing policy distribution $p$ under model prior $\nu$, penalised by the KL information gained about the true subset $\phi^\star$ of $\mathcal{M}\times\Pi$. The partition $\Phi$ is the lever: coarser partitions hide detail and lower $\log|\Phi|$ at the price of a larger decision term, so the learner trades estimation complexity against decision complexity. A companion object, the $\Phi$-aware convexification $\mathcal{M}(\Phi)$, upper bounds the decision term by a standard decision-estimation coefficient (Lemmas 5 and 6). For model-free bounds, the machinery shifts to a bilinear divergence $D^{\pi}_{\mathrm{bi}}(\phi\|\mathcal{P},R)$ and an optimistic posterior sampling scheme (Algorithm 3) that avoids full model estimation.

What would settle it

Evaluate the decision-estimation coefficient of a concretely specified linear $Q^\star/V^\star$ MDP: if the $\Phi$-aware coefficient exceeds $4\eta dH^2$, Theorem 12 is false. Equivalently, exhibit any fixed-transition MDP class satisfying Definition 14 whose minimax regret grows faster than the $T^{3/4}$ or $T^{5/6}$ rates of Theorem 15; that single example would refute the model-free claim.

Watch

Extended reading notes

Core claim

The load-bearing assertion is Theorem 2 together with Theorem 12: for any $\Phi$-restricted environment, expected regret is at most $\log|\Phi|/\eta + T\sup_{\rho,\nu}\min_p \mathrm{AIR}^{\Phi}_{\rho,\eta}(p,\nu)$, and for linear $Q^\star/V^\star$ MDPs this complexity term is at most $4\eta dH^2$. Consequently, the algorithm attains $O(H\sqrt{dT\log|\mathcal{F}|})$ regret for stochastic linear $Q^\star/V^\star$ MDPs, the first $\sqrt{T}$ guarantee for that class without Bellman completeness. In the hybrid fixed-transition adversarial-reward setting, the framework also yields model-based bounds whose only extra cost over the stochastic case is a logarithmic factor in the number of policies and transitions, and model-free bounds of $\widetilde{O}(T^{3/4})$ (or $T^{5/6}$) under a newly introduced bilinear-class oracle.

Load-bearing premise

The model-free results rest on the existence, for every policy and value function, of a single-step discrepancy function whose expectation exactly equals the bilinear gap and whose magnitude is bounded; if no such discrepancy function exists, the model-free hybrid guarantees in Theorem 15 do not follow.

Editorial extensions

If this is right

  • Fixed-transition adversarial-reward MDPs with convex reward classes have regret within a $\log(|\Pi||\mathcal{P}|)/\eta$ overhead of the fully stochastic bound, so adversarial rewards alone do not make the problem harder.
  • With full-information reward feedback, the estimation penalty in MDPs drops to $\log|\mathcal{A}| + \log|\mathcal{P}|$, decoupling the policy-space size from the per-round estimation cost.
  • Model-free learning in linear $Q^\star/V^\star$ MDPs achieves $O(H\sqrt{dT\log|\mathcal{F}|})$ regret without Bellman completeness, improving the previously known $T^{2/3}$ rate.
  • For the hybrid bilinear class with full-information feedback, the framework yields $\widetilde{O}(T^{3/4})$ regret when policies can estimate themselves and $\widetilde{O}(T^{5/6})$ otherwise.
  • The framework recovers known pure stochastic and pure adversarial bounds as special cases, so it defines a continuum of complexity measures between those two regimes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If these bounds hold, the same partition trade-off should apply to other mixed regimes, such as adversarial transitions with fixed rewards, suggesting interpolation bounds between stochastic and fully adversarial settings.
  • The discrepancy oracle of Definition 14 is the likely bottleneck for extending model-free guarantees to bandit feedback; a natural test is to instantiate it for factored or rich-observation MDPs where single-step estimation is plausible.
  • The $T^{5/6}$ exponent when the estimation policy differs from the evaluated policy leaves an open gap; proving a matching lower bound for that case would separate self-estimating policy classes from those needing active exploration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper proposes a generalization of the DEC/AIR complexity framework to 'Φ-restricted environments', in which the adversary is committed to a fixed subset φ* of the joint model–policy space and then chooses models/comparators from that subset arbitrarily. The main theorem (Theorem 2) states a regret bound of log|Φ|/η plus T times a max-min version of a new complexity measure AIR^Φ. This framework is then applied to three settings: model-based RL in hybrid MDPs with fixed transition and adversarial rewards (Corollaries 9 and 10), model-free RL in stochastic MDPs with a claimed √T regret for linear Q*/V* MDPs (Theorem 12), and model-free RL in hybrid MDPs with full-information feedback under a newly introduced hybrid bilinear class (Theorem 15). The paper also contains a discussion of trade-offs between estimation and decision complexity through the choice of partition Φ.

Significance. If the results hold, the paper would provide a unified complexity measure for hybrid decision making, a first √T regret guarantee for linear Q*/V* MDPs without Bellman completeness, and a model-free no-regret guarantee for fixed-transition adversarial-reward MDPs with function approximation. The conceptual idea of learning over flexible partitions of the model–policy space is interesting and could be of independent interest. The paper is generally well-written and includes detailed appendices; the algebraic derivation in Theorem 12 is coherent under the stated assumptions. However, the central proof of Theorem 2 relies on a convexity–concavity claim that appears to be false, and this gap propagates to the main corollaries; additionally, the advertised √T improvement for low-rank MDPs in Section 4 is asserted without a supporting DEC computation.

major comments (2)
  1. [Appendix A (proof of Theorem 2); also Lemma 17 and Theorem 7] The step preceding the application of Lemma 21 (Sion's minimax theorem) claims that AIR^Φ_{ρ,η}(p,ν) is convex in p and concave in ν, and this is used to replace min_p max_ν with max_ν min_p. The claim is false in general. Consider a single policy π, Φ = {φ1, φ2}, where φ1 = {(M1,π),(M2,π)} with deterministic observations o1 and o2 respectively, and φ2 = {(M3,π)} with deterministic observation o1. Parameterize ν by α = ν(φ1), β = ν(M1|φ1). For ρ uniform on Φ, a direct calculation gives E_o D_KL(ν_φ(·|o)||ρ) = αβ log(αβ) + (1−α) log(1−α) − (αβ+1−α) log(αβ+1−α) + log2. The Hessian of this expression in (α,β) is indefinite (e.g., at α=0.1, β=0.5 its determinant is negative), so AIR^Φ equals a linear term minus (1/η) times a function that is neither convex nor concave in ν. Thus the minimax swap in Theorem 2 is unjustified. Since Theorem 2 underpins Lemma 17 (whose proof also asserts joint concavity of AIR^Φ in (ν,ρ)), Corollaries 9 and 10, and Theorem 12 through Lemma 6, this is a load-bearing proof gap. The same issue affects the analogous swap in the proof of Theorem 7 for InfoAIR^Θ. The authors should either prove the required concavity under an additional assumption that covers the applications (for instance, an assumption that all models within a single φ share the same observation distribution), or revise the analysis to avoid this swap.
  2. [Section 4 (Example)] The example states that Corollaries 9 and 10 'directly improve the best-known regret guarantees from T^{2/3} to √T' for low-rank MDPs with fixed transition and adversarial linear rewards. This claim is not derived: the corollaries bound regret by log(|Π||P|)/η + T · DEC^KL_η(M), and a √T bound requires a quantitative bound of the form DEC^KL_η(M) = O(η · poly(d,H)) for the low-rank class. No such bound is stated or proved in Section 4 or the appendix. Please supply the DEC computation for low-rank MDPs (or a reference and a clear reduction) to support the advertised improvement.
minor comments (3)
  1. [Appendix D.2] The restated guarantee of Algorithm 3 is labelled 'Theorem 10' in the appendix, while the same result is Theorem 15 in the main text; please renumber to avoid confusion.
  2. [Theorem 15 / Definition 14] Theorem 15 states the regret bounds with O(·) notation that hides the dependence on the bound L of the discrepancy functions ℓ^est_h. Since L appears in the detailed statement (Theorem 10 in Appendix D.2) and affects the rates, it should be stated explicitly in the main theorem or in Definition 14.
  3. [Throughout] Some minor typographical issues: 'respectivly' in the example of Section 4, inconsistent use of φ vs θ in the notation of Sections 3.2 and 4, and a repeated use of 'convexified' with a ligature. These are easy to fix and do not affect the mathematics.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the regret bounds are derived from stated complexity definitions and structural assumptions; self-citations are only baselines.

full rationale

The derivation chain is self-contained against the stated objectives. Theorem 2 is a generic online-learning inequality: the expected regret is telescoped against a log posterior ratio, and the per-round term is then bounded by the saddle value of AIR^Phi; AIR^Phi is a supremum over distributions, not a fitted proxy for the realized regret, so the bound does not reduce to its own conclusion. Lemma 5 and Lemma 17 relate AIR^Phi to DEC-type complexity by explicit algebraic manipulations, and Theorem 12 bounds the relevant DEC by 4 eta d H^2 from the linear Q*/V* definitions rather than assuming the bound. Section 6 assumes a hybrid bilinear oracle (Definition 14) and Theorem 15 is a conditional guarantee; the discrepancy functions are structural assumptions, not parameters fitted to produce the stated regret. The few self-citations (Liu et al. 2023, 2024) appear only in baseline comparisons and motivating examples; they are not load-bearing. The possible issue raised about Sion minimax convexity/concavity would be a proof correctness concern, not a circularity, since the theorem's conclusion does not coincide by definition with its assumptions.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central bounds rest on finiteness and realizability assumptions listed above. No physical entity is introduced and no constant is fitted to experimental data; the listed hyperparameters are theoretical tuning variables.

free parameters (2)
  • Learning rate eta in Theorem 2 and Theorem 12 = O(sqrt(log|F| / (d H^2 T)))
    Set to balance log|Phi|/eta against eta d H^2 T. This is theoretical tuning, not a parameter fitted to data.
  • Epoch length tau and rates eta, gamma in Algorithm 3 = tau = sqrt(T), eta = O(T^{-1/4}) or O(T^{-1/3}) depending on est(pi)
    Algorithmic hyperparameters chosen to optimize the stated regret bounds; no empirical fitting is involved.
assumptions (6)
  • domain assumption The model class M, policy class Pi, and partition class Phi are finite
    Stated at the start of Section 2; all regret bounds involve log|Phi|, and continuous extensions are only sketched.
  • domain assumption Cumulative rewards satisfy sum_h R(s_h, a_h) <= 1 almost surely and value functions lie in [0, 1]
    Imported from the DEC literature; keeps regret and KL-divergence bounds scale-independent.
  • domain assumption In the hybrid setting, models decompose as M = R x P and the partition is by transition and comparator
    This defines the hybrid fixed-transition adversarial-reward environment used in Section 4.
  • domain assumption The reward space R is convex
    Used in Corollaries 9 and 10 so that M(Phi_A) = M; for nonconvex R only P x CO(R) is obtained.
  • domain assumption Linear Q*/V* realizability with known features and finite function class F
    Definition 11; the sqrt(T) result is stated for this class and does not apply to arbitrary function approximation.
  • domain assumption Hybrid bilinear discrepancy oracle in Definition 14
    Algorithm 3 and Lemma 20 require ell^est_h and est(pi) for every pi and f; if this oracle is absent, the model-free hybrid bounds fail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decision Making in Hybrid Environments: A Model Aggregation Approach." pith.science (2026). https://pith.science/paper/EIVNTPZH

@misc{pith2026250205974,
  author       = {Pith},
  title        = {Pith review of: Decision Making in Hybrid Environments: A Model Aggregation Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EIVNTPZH}},
  note         = {Machine review of arXiv:2502.05974}
}
read the original abstract

Recent work by Foster et al. (2021, 2022, 2023b) and Xu and Zeevi (2023) developed the framework of decision estimation coefficient (DEC) that characterizes the complexity of general online decision making problems and provides a general algorithm design principle. These works, however, either focus on the pure stochastic regime where the world remains fixed over time, or the pure adversarial regime where the world arbitrarily changes over time. For the hybrid regime where the dynamics of the world is fixed while the reward arbitrarily changes, they only give pessimistic bounds on the decision complexity. In this work, we propose a general extension of DEC that more precisely characterizes this case. Besides applications in special cases, our framework leads to a flexible algorithm design where the learner learns over subsets of the hypothesis set, trading estimation complexity with decision complexity, which could be of independent interest. Our work covers model-based learning and model-free learning in the hybrid regime, with a newly proposed extension of the bilinear classes (Du et al., 2021) to the adversarial-reward case. In addition, our method improves the best-known regret bounds for linear Q*/V* MDPs in the pure stochastic regime.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 24 canonical work pages

  1. [1]

    A definition of continual reinforcement learning

    David Abel, Andr \'e Barreto, Benjamin Van Roy, Doina Precup, Hado P van Hasselt, and Satinder Singh. A definition of continual reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    work page 2024

  2. [2]

    Decision making in changing environments: Robustness, query-based learning, and differential privacy

    Fan Chen and Alexander Rakhlin. Decision making in changing environments: Robustness, query-based learning, and differential privacy. arXiv preprint arXiv:2501.14928, 2025

    arXiv 2025

  3. [3]

    Unified algorithms for rl with decision-estimation coefficients: No-regret, pac, and reward-free learning

    Fan Chen, Song Mei, and Yu Bai. Unified algorithms for rl with decision-estimation coefficients: No-regret, pac, and reward-free learning. arXiv preprint arXiv:2209.11745, 2022

    arXiv 2022

  4. [4]

    Assouad, fano, and le cam with interaction: A unifying lower bound framework and characterization for bandit learnability

    Fan Chen, Dylan J Foster, Yanjun Han, Jian Qian, Alexander Rakhlin, and Yunbei Xu. Assouad, fano, and le cam with interaction: A unifying lower bound framework and characterization for bandit learnability. The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    work page 2024

  5. [5]

    Finding the stochastic shortest path with low regret: The adversarial cost and unknown transition case

    Liyu Chen and Haipeng Luo. Finding the stochastic shortest path with low regret: The adversarial cost and unknown transition case. In International Conference on Machine Learning, pages 1651--1660. PMLR, 2021

    work page 2021

  6. [6]

    Refined regret for adversarial mdps with linear function approximation

    Yan Dai, Haipeng Luo, Chen-Yu Wei, and Julian Zimmert. Refined regret for adversarial mdps with linear function approximation. In International Conference on Machine Learning, 2023

    work page 2023

  7. [7]

    Bilinear classes: A structural framework for provable generalization in rl

    Simon Du, Sham Kakade, Jason Lee, Shachar Lovett, Gaurav Mahajan, Wen Sun, and Ruosong Wang. Bilinear classes: A structural framework for provable generalization in rl. In International Conference on Machine Learning, pages 2826--2836. PMLR, 2021

    2021

  8. [8]

    On the complexity of multi-agent decision making: From learning in games to partial monitoring

    Dean Foster, Dylan J Foster, Noah Golowich, and Alexander Rakhlin. On the complexity of multi-agent decision making: From learning in games to partial monitoring. In The Thirty Sixth Annual Conference on Learning Theory, pages 2678--2792. PMLR, 2023 a

    work page 2023

Show all 34 references
  1. [9]

    The statistical complexity of interactive decision making

    Dylan J Foster, Sham M Kakade, Jian Qian, and Alexander Rakhlin. The statistical complexity of interactive decision making. arXiv preprint arXiv:2112.13487, 2021

    2021 arXiv

  2. [10]

    On the complexity of adversarial decision making

    Dylan J Foster, Alexander Rakhlin, Ayush Sekhari, and Karthik Sridharan. On the complexity of adversarial decision making. Advances in Neural Information Processing Systems, 35: 0 35404--35417, 2022

    2022

  3. [11]

    Tight guarantees for interactive decision making with the decision-estimation coefficient

    Dylan J Foster, Noah Golowich, and Yanjun Han. Tight guarantees for interactive decision making with the decision-estimation coefficient. In The Thirty Sixth Annual Conference on Learning Theory, pages 3969--4043. PMLR, 2023 b

    2023

  4. [12]

    Model-free reinforcement learning with the decision-estimation coefficient

    Dylan J Foster, Noah Golowich, Jian Qian, Alexander Rakhlin, and Ayush Sekhari. Model-free reinforcement learning with the decision-estimation coefficient. Advances in Neural Information Processing Systems, 36, 2023 c

    2023

  5. [13]

    Contextual decision processes with low bellman rank are pac-learnable

    Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, John Langford, and Robert E Schapire. Contextual decision processes with low bellman rank are pac-learnable. In International Conference on Machine Learning, pages 1704--1713. PMLR, 2017

    2017

  6. [14]

    Learning adversarial markov decision processes with bandit feedback and unknown transition

    Chi Jin, Tiancheng Jin, Haipeng Luo, Suvrit Sra, and Tiancheng Yu. Learning adversarial markov decision processes with bandit feedback and unknown transition. In International Conference on Machine Learning, pages 4860--4869. PMLR, 2020

    2020

  7. [15]

    Bellman eluder dimension: New rich classes of rl problems, and sample-efficient algorithms

    Chi Jin, Qinghua Liu, and Sobhan Miryoosefi. Bellman eluder dimension: New rich classes of rl problems, and sample-efficient algorithms. Advances in neural information processing systems, 34: 0 13406--13418, 2021

    2021

  8. [16]

    Improved regret bounds for linear adversarial mdps via linear optimization

    Fang Kong, Xiangcheng Zhang, Baoxiang Wang, and Shuai Li. Improved regret bounds for linear adversarial mdps via linear optimization. Transactions on Machine Learning Research, 2023

    2023

  9. [17]

    Continual learning as computationally constrained reinforcement learning

    Saurabh Kumar, Henrik Marklund, Ashish Rao, Yifan Zhu, Hong Jun Jeon, Yueyang Liu, and Benjamin Van Roy. Continual learning as computationally constrained reinforcement learning. arXiv preprint arXiv:2307.04345, 2023

    2023 arXiv

  10. [18]

    Exploration by optimisation in partial monitoring

    Tor Lattimore and Csaba Szepesv \'a ri. Exploration by optimisation in partial monitoring. In Conference on Learning Theory, pages 2488--2515. PMLR, 2020

    2020

  11. [19]

    Improved algorithm for adversarial linear mixture mdps with bandit feedback and unknown transition

    Long-Fei Li, Peng Zhao, and Zhi-Hua Zhou. Improved algorithm for adversarial linear mixture mdps with bandit feedback and unknown transition. In International Conference on Artificial Intelligence and Statistics, pages 3061--3069. PMLR, 2024

    2024

  12. [20]

    Towards optimal regret in adversarial linear mdps with bandit feedback

    Haolin Liu, Chen-Yu Wei, and Julian Zimmert. Towards optimal regret in adversarial linear mdps with bandit feedback. In The Twelfth International Conference on Learning Representations, 2023

    2023

  13. [21]

    Beating adversarial low-rank mdps with unknown transition and bandit feedback

    Haolin Liu, Zakaria Mhammedi, Chen-Yu Wei, and Julian Zimmert. Beating adversarial low-rank mdps with unknown transition and bandit feedback. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  14. [22]

    Policy optimization in adversarial mdps: Improved exploration via dilated bonuses

    Haipeng Luo, Chen-Yu Wei, and Chung-Wei Lee. Policy optimization in adversarial mdps: Improved exploration via dilated bonuses. Advances in Neural Information Processing Systems, 34: 0 22931--22942, 2021

    2021

  15. [23]

    Online markov decision processes under bandit feedback

    Gergely Neu, Andr \'a s Gy \"o rgy, Csaba Szepesv \'a ri, and Andr \'a s Antos. Online markov decision processes under bandit feedback. IEEE Transactions on Automatic Control, 59 0 (3): 0 676--691, 2013

    2013

  16. [24]

    Online stochastic shortest path with bandit feedback and unknown transition function

    Aviv Rosenberg and Yishay Mansour. Online stochastic shortest path with bandit feedback and unknown transition function. Advances in Neural Information Processing Systems, 32, 2019

    2019

  17. [25]

    Optimistic policy optimization with bandit feedback

    Lior Shani, Yonathan Efroni, Aviv Rosenberg, and Shie Mannor. Optimistic policy optimization with bandit feedback. In International Conference on Machine Learning, pages 8604--8613. PMLR, 2020

    2020

  18. [26]

    Improved regret for efficient online reinforcement learning with linear function approximation

    Uri Sherman, Tomer Koren, and Yishay Mansour. Improved regret for efficient online reinforcement learning with linear function approximation. In International Conference on Machine Learning, pages 31117--31150. PMLR, 2023

    2023

  19. [27]

    On general minimax theorems

    Maurice Sion. On general minimax theorems. 1958

    1958

  20. [28]

    Model-based rl in contextual decision processes: Pac bounds and exponential improvements over model-free approaches

    Wen Sun, Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, and John Langford. Model-based rl in contextual decision processes: Pac bounds and exponential improvements over model-free approaches. In Conference on learning theory, pages 2898--2933. PMLR, 2019

    2019

  21. [29]

    Instance-optimality in interactive decision making: Toward a non-asymptotic theory

    Andrew J Wagenmaker and Dylan J Foster. Instance-optimality in interactive decision making: Toward a non-asymptotic theory. In The Thirty Sixth Annual Conference on Learning Theory, pages 1322--1472. PMLR, 2023

    2023

  22. [30]

    The role of coverage in online reinforcement learning

    T Xie, D Foster, Y Bai, N Jiang, and S Kakade. The role of coverage in online reinforcement learning. In Proceedings of the Eleventh International Conference on Learning Representations, 2023

    2023

  23. [31]

    Bayesian design principles for frequentist sequential learning

    Yunbei Xu and Assaf Zeevi. Bayesian design principles for frequentist sequential learning. In International Conference on Machine Learning, pages 38768--38800. PMLR, 2023

    2023

  24. [32]

    Feel-good thompson sampling for contextual bandits and reinforcement learning

    Tong Zhang. Feel-good thompson sampling for contextual bandits and reinforcement learning. SIAM Journal on Mathematics of Data Science, 4 0 (2): 0 834--857, 2022

    2022

  25. [33]

    Learning adversarial low-rank markov decision processes with unknown transition and full-information feedback

    Canzhe Zhao, Ruofeng Yang, Baoxiang Wang, Xuezhou Zhang, and Shuai Li. Learning adversarial low-rank markov decision processes with unknown transition and full-information feedback. Advances in Neural Information Processing Systems, 36, 2024

    2024

  26. [34]

    Gec: A unified framework for interactive decision making in mdp, pomdp, and beyond

    Han Zhong, Wei Xiong, Sirui Zheng, Liwei Wang, Zhaoran Wang, Zhuoran Yang, and Tong Zhang. Gec: A unified framework for interactive decision making in mdp, pomdp, and beyond. arXiv preprint arXiv:2211.01962, 2022

    2022 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.