The paper establishes the first tilde O(epsilon^{-1}) upper bounds and matching lower bounds for forward-KL-regularized offline contextual bandits under single-policy concentrability in both tabular and general function approximation settings.
Pessimistic nonlinear least-squares value iteration for offline reinforcement learning.arXiv preprint arXiv:2310.01380
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
representative citing papers
Stationary-weighted FQE achieves finite-sample linear convergence to the projected Bellman fixed point without Bellman completeness by reweighting regressions to the target stationary norm.
Bellman calibration supplies a new reliability criterion and post-hoc recalibration method for value functions in offline RL, with finite-sample guarantees at one-dimensional nonparametric rates that avoid Bellman completeness and realizability assumptions.
citing papers explorer
-
Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability
The paper establishes the first tilde O(epsilon^{-1}) upper bounds and matching lower bounds for forward-KL-regularized offline contextual bandits under single-policy concentrability in both tabular and general function approximation settings.
-
Fitted $Q$ Evaluation Without Bellman Completeness via Stationary Weighting
Stationary-weighted FQE achieves finite-sample linear convergence to the projected Bellman fixed point without Bellman completeness by reweighting regressions to the target stationary norm.
-
Bellman Calibration for $V$-Learning in Offline Reinforcement Learning
Bellman calibration supplies a new reliability criterion and post-hoc recalibration method for value functions in offline RL, with finite-sample guarantees at one-dimensional nonparametric rates that avoid Bellman completeness and realizability assumptions.