Pith. sign in

REVIEW 2 major objections 4 minor 44 references

A Unified Algorithmic Framework for Hybrid Reinforcement Learning in Tabular MDPs with Shifted Transition Dynamics

T0 review · 2 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves that a per-state-action bound on transition shift lets hybrid RL safely exploit biased offline data, with regret and sub-optimality guarantees that match lower bounds.

desk verdict The framework is worth a look, but the main theorems are not proven as written — the count argument in Theorem 1 and the stage-uniformity assumption in Theorem 2 are load-bearing. read the letter →

arxiv 2607.25207 v1 pith:EXJLTYHV submitted 2026-07-28 cs.LG

Zheshun Wu , Renjie Zheng , Jinhang Zuo , Zenglin Xu , Fang Kong This is my paper · ORCID
classification cs.LG
keywords hybridreinforcementlearningtabularMDPsshiftedtransitiondynamicsregretminimizationbestpolicyidentificationbiasboundconcentrabilitymodelselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that offline data from an environment with shifted transition dynamics can still be safely exploited in hybrid tabular reinforcement learning, provided the learner knows a valid per-state-action bound on the shift. It contributes two algorithms: MIN-UCB-VI for regret minimization and MAX-LCB-VI for best policy identification, both based on a hybrid bonus that adds a bias-compensating term to the fused empirical estimate. The claimed guarantees are two-sided: regret O~tilde(H min{sqrt(SAK), K*nu_tilde_max + K sqrt(H/tau*)}) and sub-optimality O~tilde(H min{sqrt(SA/K), H*nu_max + sqrt(S/(N_src/C*_nu + K/A))}), with matching lower bounds. If correct, this means hybrid RL can be made robust to continuous, non-separable transition shifts, unlike prior results that required separability and reachability. A reader should care because such shifts are the norm when offline data comes from simulators or outdated environments.

What carries the argument

The load-bearing object is the hybrid bonus term b^{k,hyb}_h(s,a) = (exploration variance term on pooled counts) + H*nu(s,a)*N^src_{s,a}/(N^src_{s,a}+N^k_{s,a}). The added term exactly compensates for the bias (P_tar - P_src) applied to the value function, keeping the fused estimate optimistic even when the source and target kernels differ by up to nu. The min/max selection between the pure-online and hybrid Q-estimates lets the algorithm adapt to whichever data source is more informative. The linear program defining tau* abstracts the effective coverage of the pooled data, and the concentrability coefficient C*_nu extends single-policy concentrability to shifted dynamics.

What would settle it

Construct a two-state tabular MDP with a known target, a source whose transition differs from the target by delta, and supply the algorithm with a bias bound nu < delta at a single state-action pair. If MIN-UCB-VI's regret then grows at the pure-online rate despite a large offline dataset, or if the claimed regret bound is violated, the bias compensation is not sufficient. Equivalently, set nu = 0 while delta > 0 and check whether the hybrid bonus keeps the fused estimate optimistic; the proof's optimism event will fail.

Watch

Extended reading notes

Core claim

The paper's central claim is that the hybrid estimate, which pools offline and online transition counts and adds the bonus H*nu(s,a)*N^src_{s,a}/(N^src_{s,a}+N^k_{s,a}), remains optimistic (for MIN-UCB-VI) or pessimistic (for MAX-LCB-VI) with respect to the target MDP, so the min/max comparison with the pure-online estimate automatically inherits the better of the two bounds. Quantitatively, the regret bound is O~tilde(H min{sqrt(SAK), K*nu_tilde_max + K sqrt(H/tau*)}), where nu_tilde_max accumulates the bias over the horizon and tau* is the optimal value of a linear program measuring fused-data coverage; the sub-optimality gap of the output policy is O~tilde(H min{sqrt(SA/K), H*nu_max + sqr

Load-bearing premise

The central premise is that the learner is handed a valid bound nu(s,a) with ||P_tar(.|s,a) - P_src(.|s,a)||_1 <= nu(s,a) for every state-action pair, and that this bound feeds the hybrid bonus; if the true shift exceeds nu anywhere, the hybrid estimate can cease to be optimistic (or pessimistic) and the claimed speed-up collapses.

Editorial extensions

If this is right

  • If the transition shift is small (H*nu_tilde_max = o(sqrt(SA/K))), MIN-UCB-VI provably beats pure online value iteration, and the improvement grows with the offline sample size through tau*.
  • MAX-LCB-VI converts biased offline data into a smaller sub-optimality gap whenever the behavior policy covers the optimal policy and nu is small; with zero bias and optimal behavior policy it recovers the standard hybrid RL bound.
  • The matching lower bounds imply that no other algorithm can improve the dependence on nu_max, tau*, or C*_nu in a minimax sense, so the framework leaves little slack.
  • The model-selection variant removes the need to know the bias in advance, preserving a no-worse-than-online guarantee up to extra logarithmic and polynomial factors.
  • When the shift is large, the algorithms fall back to the pure online guarantee, so biased offline data cannot cause negative transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper leaves implicit: if the bias bound nu is estimated from data with a confidence margin, the same optimistic/pessimistic argument should go through, converting the bound from 'given nu' to 'estimated nu'.
  • The joint concentrability C*_nu suggests a data-collection principle for hybrid RL: collect offline data under a policy that covers the optimal target policy, and the shift penalty is paid only where that policy visits, not uniformly.
  • One could test the instance-dependent bound directly: across state-action pairs, plot the per-pair regret against the predicted saving term N^src*Delta*(1 - H*nu_tilde/Delta)^2; if the saving term is not visible in experiments, the bound is not tight.
  • The framework's reliance on a per-state-action TV bound hints that in continuous state spaces, a learned shift model rather than a scalar nu_max would be the natural analogue, but the LP-based tau* would need a new definition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies hybrid RL in finite-horizon tabular MDPs where an online learner has K episodes in a target MDP and a fixed offline dataset of N^src trajectories from a source MDP whose transition kernel may be shifted. It proposes two algorithms: MIN-UCB-VI for regret minimization and MAX-LCB-VI for best-policy identification, both assuming a per-state-action TV bias bound nu(s,a) and (for the second) a single-policy concentrability coefficient C*_nu. The paper claims instance-independent and instance-dependent regret/suboptimality bounds, matching lower bounds, a model-selection extension for unknown bias, and supporting experiments. The algorithmic ideas are natural extensions of the MIN-UCB bandit principle and pessimistic value iteration, but the central upper-bound proofs contain invalid steps, so the main theoretical claims are not established as written.

Significance. If the claimed theorems were correct, the paper would make a meaningful advance: it would remove the separability and reachability assumptions of prior shifted-dynamics hybrid-RL work (Qu et al., 2025), introduce a bias-aware concentrability measure, and provide lower bounds that match up to poly(H) factors. The lower-bound constructions are detailed and the paper is clearly organized. However, the proof gaps affect the two flagship upper bounds (Theorems 1 and 2), and one of the proofs rests on an assumption that is explicitly false in the stated setting. The significance of the paper is therefore conditional on substantial repair of the analysis.

major comments (2)
  1. [Appendix C, Theorem 2 (Eq. (2))] The proof begins with the assertion: 'we assume that for each (s,a), the action is either optimal at all stages or suboptimal at all stages.' This is not part of Theorem 2 and is not implied by the time-homogeneous tabular MDP setting, because Q_h^*(s,a)=r(s,a)+P_{s,a}V_{h+1}^* varies with h through V_{h+1}^*. A concrete valid counterexample is: states {x,g,dead}, actions {a,b}, r(x,a)=0, r(x,b)=0.5, r(g,*)=1, r(dead,*)=0, P(x,a)=g, P(x,b)=dead, with g and dead absorbing. At h=H, b is optimal at x; at h=H-1, a is optimal. Thus both actions are optimal at some stages and suboptimal at others. The proof then uses d_h(s)=V_h^*(s)-V_h^pi(s) >= Delta(s,a) for the executed action, but when the executed action is optimal at stage h, d_h(s)=0 while Delta(s,a)>0, so the clipping inequality fails. The claimed saving term sav(s,a) and the instance-dependent regret bound are therefore not establishe
  2. [Appendix B, Theorem 1 (display after Eq. (16))] The proof asserts without proof that sqrt( sum_{s,a} sum_{n=1}^{N^K_{s,a}} 1/(n+N^src_{s,a}) ) is bounded by the same expression with the inner sum truncated at tau^* - N^src_{s,a}. This comparison is false when N^K_{s,a} > max(0, tau^* - N^src_{s,a}), which is a normal occurrence. For example, take SA=2, N^src=0, H=1, and K episodes all visiting the same state-action pair; then tau^* = K/2 but N^K(s,a)=K. The subsequent bound KH sqrt(H/tau^*) relies on this unproved claim. Consequently the hybrid-branch regret bound in Theorem 1 is not established as written.
minor comments (4)
  1. [Section 3.6] The claim that prior bias knowledge is 'necessary' is argued only by analogy to the multi-armed bandit lower bound of Cheung and Lyu (2024). Since MAB is a special case, the analogy provides motivation, not a proof for the MDP setting. Please phrase this as an informal justification.
  2. [Algorithm 4 and Theorem 3] The two-fold subsampling procedure introduces an unspecified constant 10 in the trimming step, but the proof of Theorem 3 does not mention Algorithm 4. Please clarify whether the theorem applies to Algorithm 3 or to the output of Algorithm 4, and if the latter, specify the constant's role.
  3. [Section 6] The baselines H-MIN-UCB-VI / MIN-UCB-S and H-LCB-VI / LCB-VI-S are not defined in the text. Since the experimental comparison is a claimed validation, each baseline's bonus construction should be specified.
  4. [Abstract and Remark 8] The abstract claims 'matching lower bounds,' while Remark 8 concedes that the instance-dependent bounds are only nearly matching up to a poly(H) factor. Please temper the wording to match the actual results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: bounds are derived from stated bias/concentrability inputs, with no fitted-parameter or self-citation reduction.

full rationale

The derivation chain is self-contained in the circularity sense. The bias bound ν(s,a) and concentrability coefficient C*_ν are assumed inputs (Section 3.6 and Assumption 1); all upper bounds are stated in terms of these inputs, and they are not estimated from the same data used to evaluate the bounds. MIN-UCB-VI and MAX-LCB-VI use external lemmas from Zhang et al. (2021), Chen et al. (2025), Li et al. (2024), Xie et al. (2021), and the MIN-UCB idea from Cheung and Lyu (2024), none of which are self-citations. The model selection section (4.4) applies EXP3.P (Pacchiano et al., 2020) as a meta-algorithm; its regret bound is a standard online model-selection guarantee, not a fitted quantity renamed as a prediction. Lower bounds (Theorems 6–9) construct hard instances whose parameters realize the same LP value τ* and bias/concentrability inputs; this is standard minimax lower-bound methodology, not circularity. A separate, non-circular correctness concern is that the proof of Theorem 2 (Appendix C) introduces an unstated assumption that each (s,a) is either optimal at all stages or suboptimal at all stages; this is a proof gap for time-homogeneous MDPs, but it does not reduce a conclusion to an input or involve self-citation. Accordingly, no step warrants a circularity flag.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claims rest on standard concentration lemmas plus three setting assumptions: a known bias bound nu, single-policy concentrability C*_nu, and an unstated stage-uniformity assumption in the instance-dependent proof. There are no invented physical entities; the new objects are algorithmic or metric-level.

free parameters (2)
  • Bonus constants c1, c2 (and c'_1, c'_2 in the proof)
    Hand-chosen constants from Zhang et al. (2021), required for optimism/pessimism. Experiments set a single bonus scale c=0.01.
  • Subsampling trimming constant 10 in Algorithm 4
    Hand-chosen threshold in the two-fold subsampling procedure for MAX-LCB-VI; no data-driven selection is described.
assumptions (5)
  • domain assumption Finite-horizon episodic tabular MDP with time-homogeneous transitions and known deterministic rewards.
    Section 3.2 defines the setting; all algorithms and proofs use this model.
  • domain assumption A valid per-state-action bias bound nu(s,a) is known to the learner.
    Section 3.6 assumes ||P_tar-P_src||_1 <= nu(s,a) and the hybrid bonus depends on it.
  • domain assumption Single-policy concentrability under bias MDPs (Assumption 1): d^{pi*,P_tar}_h(s,a)/d^{pi_b,P_src}_h(s,a) <= C*_nu.
    Used for all MAX-LCB-VI upper and lower bounds and for the definition of M in lower bounds.
  • ad hoc to paper In the instance-dependent regret proof, each state-action pair is either optimal at all stages or suboptimal at all stages.
    Appendix C states this 'to simplify the analysis'; it is not stated in Theorem 2 and is not generally true for time-homogeneous transitions with step-dependent rewards.
  • ad hoc to paper Unproved count comparison with the LP optimum in the proof of Theorem 1.
    Appendix B asserts sum_{s,a} sum_{n=1}^{N_K} 1/(n+N_src) <= sum_{s,a} sum_{n=1}^{tau*-N_src} 1/(n+N_src) without proof; as written it fails for pairs with N_src > tau*.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Unified Algorithmic Framework for Hybrid Reinforcement Learning in Tabular MDPs with Shifted Transition Dynamics." pith.science (2026). https://pith.science/paper/EXJLTYHV

@misc{pith2026260725207,
  author       = {Pith},
  title        = {Pith review of: A Unified Algorithmic Framework for Hybrid Reinforcement Learning in Tabular MDPs with Shifted Transition Dynamics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EXJLTYHV}},
  note         = {Machine review of arXiv:2607.25207}
}
read the original abstract

This paper investigates a hybrid reinforcement learning setting in tabular Markov Decision Processes (MDPs), where an agent aims to learn an optimal policy by combining online interactions with a target environment and offline data from a source environment. A central challenge is that offline data may be collected from outdated environments with shifted transition dynamics, making naive integration of historical data ineffective. To address this, we propose a unified algorithmic framework featuring two algorithms: MIN-UCB-VI for regret minimization and MAX-LCB-VI for best policy identification. Both algorithms leverage fine-grained bias information to more effectively exploit offline data under general transition shifts. We provide theoretical guarantees for our framework, including both instance-dependent and independent upper bounds on regret and sub-optimality gap. Furthermore, we establish matching lower bounds to demonstrate the optimality of our approach and validate our theoretical findings through extensive experiments.

Figures

Figures reproduced from arXiv: 2607.25207 by the authors.

Figure 1
Figure 1. Cumulative regret of MIN-UCB-VI and baselines under different bias levels and offline sample sizes. 0 2000 4000 6000 8000 10000 Online Episodes K 0 2500 5000 7500 10000 12500 15000 17500 20000 Regret MIN-UCB-VI Pure UCB-VI H-MIN-UCB-VI MIN-UCB-S (a) Bias level 0.5 0 2000 4000 6000 8000 10000 Online Episodes K 0 2500 5000 7500 10000 12500 15000 17500 20000 Regret MIN-UCB-VI Pure UCB-VI H-MIN-UCB-VI MIN-UCB-S (b) Nsrc… view at source ↗
Figure 2
Figure 2. Additional cumulative-regret experiments for MIN-UCB-VI. [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Suboptimality gap of MAX-LCB-VI and baselines under different bias levels, offline sample sizes, and offline data-collection policies. In Figures 1 and 2, the target success probability is fixed at 0.95. We vary the source success probability among 0.75, 0.45, and 0.25, corresponding to bias levels 0.2, 0.5, and 0.7, respectively [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 4 linked inside Pith

  1. [1]

    Prediction, Learning, and Games , publisher=

    Cesa-Bianchi, Nicol\'. Prediction, Learning, and Games , publisher=

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    On gap-dependent bounds for offline reinforcement learning , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    nature , volume=

    Mastering the game of go without human knowledge , author=. nature , volume=. 2017 , publisher=

    2017

  4. [4]

    nature , volume=

    Mastering the game of Go with deep neural networks and tree search , author=. nature , volume=. 2016 , publisher=

    2016

  5. [5]

    arXiv preprint arXiv:2204.05862 , year=

    Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    arXiv

  6. [6]

    International conference on machine learning , pages=

    Minimax regret bounds for reinforcement learning , author=. International conference on machine learning , pages=. 2017 , organization=

    2017

  7. [7]

    2023 , eprint=

    Understanding the Complexity Gains of Single-Task RL with a Curriculum , author=. 2023 , eprint=

    2023

  8. [8]

    The International Journal of Robotics Research , volume=

    Reinforcement learning in robotics: A survey , author=. The International Journal of Robotics Research , volume=. 2013 , publisher=

    2013

Show all 44 references
  1. [9]

    Manning and Stefano Ermon and Chelsea Finn , title =

    Rafael Rafailov and Archit Sharma and Eric Mitchell and Christopher D. Manning and Stefano Ermon and Chelsea Finn , title =. NeurIPS , year =

  2. [10]

    Augmenting Online

    Ruiquan Huang and Donghao Li and Chengshuai Shi and Cong Shen and Jing Yang , booktitle=. Augmenting Online. 2025 , url=

    2025

  3. [11]

    International Conference on Artificial Intelligence and Statistics , pages=

    Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Data , author=. International Conference on Artificial Intelligence and Statistics , pages=. 2025 , organization=

    2025

  4. [12]

    Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence , pages =

    Best Arm Identification with Possibly Biased Offline Data , author =. Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence , pages =. 2025 , editor =

    2025

  5. [13]

    Wang Chi Cheung and Lixing Lyu , title =

  6. [14]

    Algorithmic Learning Theory , pages=

    Episodic reinforcement learning in finite mdps: Minimax lower bounds revisited , author=. Algorithmic Learning Theory , pages=. 2021 , organization=

    2021

  7. [15]

    arXiv preprint arXiv:2312.03764 , year=

    Similarity-based knowledge transfer for cross-domain reinforcement learning , author=. arXiv preprint arXiv:2312.03764 , year=

    arXiv

  8. [16]

    Conference on robot learning , pages=

    Scalable deep reinforcement learning for vision-based robotic manipulation , author=. Conference on robot learning , pages=. 2018 , organization=

    2018

  9. [17]

    Botao Hao and Rahul Jain and Tor Lattimore and Benjamin Van Roy and Zheng Wen , title =

  10. [18]

    Artificial intelligence and statistics , pages=

    Multi-armed bandit problems with history , author=. Artificial intelligence and statistics , pages=. 2012 , organization=

    2012

  11. [19]

    Proceedings of Thirty Third Conference on Learning Theory , pages =

    Provably efficient reinforcement learning with linear function approximation , author =. Proceedings of Thirty Third Conference on Learning Theory , pages =. 2020 , editor =

    2020

  12. [20]

    Advances in Neural Information Processing Systems , volume=

    Reinforcement learning with general value function approximation: Provably efficient approach via bounded eluder dimension , author=. Advances in Neural Information Processing Systems , volume=

  13. [21]

    International conference on machine learning , pages=

    Is pessimism provably efficient for offline rl? , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  14. [22]

    International Conference on Machine Learning , pages=

    Minimax-optimal off-policy evaluation with linear function approximation , author=. International Conference on Machine Learning , pages=. 2020 , organization=

    2020

  15. [23]

    arXiv preprint arXiv:2005.01643 , year=

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems , author=. arXiv preprint arXiv:2005.01643 , year=

    2005 arXiv

  16. [24]

    International conference on machine learning , pages=

    Pessimistic q-learning for offline reinforcement learning: Towards optimal sample complexity , author=. International conference on machine learning , pages=. 2022 , organization=

    2022

  17. [25]

    Advances in Neural Information Processing Systems , volume=

    Bridging offline reinforcement learning and imitation learning: A tale of pessimism , author=. Advances in Neural Information Processing Systems , volume=

  18. [26]

    Malinga Perera and Renata Borovica

    Bastian Oetomo and R. Malinga Perera and Renata Borovica. Cutting to the chase with warm-start contextual bandits , journal =

  19. [27]

    NeurIPS , year =

    Kevin Tan and Wei Fan and Yuting Wei , title =. NeurIPS , year =

  20. [28]

    Kevin Tan and Ziping Xu , title =

  21. [29]

    Yuda Song and Yifei Zhou and Ayush Sekhari and Drew Bagnell and Akshay Krishnamurthy and Wen Sun , title =

  22. [30]

    arXiv preprint arXiv:2210.06718 , year=

    Hybrid rl: Using both offline and online data can make rl efficient , author=. arXiv preprint arXiv:2210.06718 , year=

    arXiv

  23. [31]

    Advances in Neural Information Processing Systems , volume=

    Reward-agnostic fine-tuning: Provable statistical benefits of hybrid reinforcement learning , author=. Advances in Neural Information Processing Systems , volume=

  24. [32]

    International Conference on Machine Learning , pages=

    Leveraging offline data in online reinforcement learning , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  25. [33]

    SIAM journal on computing , volume=

    The nonstochastic multiarmed bandit problem , author=. SIAM journal on computing , volume=. 2002 , publisher=

    2002

  26. [34]

    Advances in Neural Information Processing Systems , volume=

    Model selection in contextual stochastic bandit problems , author=. Advances in Neural Information Processing Systems , volume=

  27. [35]

    Uncertainty in Artificial Intelligence , pages=

    Cross-domain adaptive transfer reinforcement learning based on state-action correspondence , author=. Uncertainty in Artificial Intelligence , pages=. 2022 , organization=

    2022

  28. [36]

    International Joint Conference on Artificial Intelligence , pages=

    Autonomous cross-domain knowledge transfer in lifelong policy gradient reinforcement learning , author=. International Joint Conference on Artificial Intelligence , pages=

  29. [37]

    International Conference on Machine Learning , pages=

    Efficient online reinforcement learning with offline data , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  30. [38]

    Conference on Learning Theory , pages=

    Is reinforcement learning more difficult than bandits? a near-optimal algorithm escaping the curse of horizon , author=. Conference on Learning Theory , pages=. 2021 , organization=

    2021

  31. [39]

    IEEE Transactions on Emerging Topics in Computational Intelligence , volume=

    Reinforcement learning with adaptive policy gradient transfer across heterogeneous problems , author=. IEEE Transactions on Emerging Topics in Computational Intelligence , volume=. 2024 , publisher=

    2024

  32. [40]

    Sharp Gap-Dependent Variance-Aware Regret Bounds for Tabular

    Shulun Chen and Runlong Zhou and Zihan Zhang and Maryam Fazel and Simon Shaolei Du , booktitle=. Sharp Gap-Dependent Variance-Aware Regret Bounds for Tabular. 2025 , url=

    2025

  33. [41]

    The Annals of Statistics , volume=

    Settling the sample complexity of model-based offline reinforcement learning , author=. The Annals of Statistics , volume=. 2024 , publisher=

    2024

  34. [42]

    Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning , url =

    Xie, Tengyang and Jiang, Nan and Wang, Huan and Xiong, Caiming and Bai, Yu , booktitle =. Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning , url =

  35. [43]

    Minimax Regret Bounds for Reinforcement Learning , booktitle =

    Mohammad Gheshlaghi Azar and Ian Osband and R. Minimax Regret Bounds for Reinforcement Learning , booktitle =

  36. [44]

    Advances in Neural Information Processing Systems , volume=

    Non-asymptotic gap-dependent regret bounds for tabular mdps , author=. Advances in Neural Information Processing Systems , volume=

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.