Pith. sign in

REVIEW 4 major objections 3 minor 84 references

Constrained Online Decision-Making: A Unified Framework

T0 review · 4 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single unified algorithm, GED-UCB, solves constrained online decision-making problems ranging from safe bandits to calibration, with regret controlled by a generalized eluder dimension.

desk verdict A genuinely useful framework and a plausible new complexity measure, but the main regret theorem is not proven as written and the Gaussian oracle bound is dimensionally wrong. read the letter →

arxiv 2505.07101 v3 pith:XWUO65X3 submitted 2025-05-11 stat.ML cs.LG

Haichen Hu , David Simchi-Levi , Navid Azizan This is my paper · ORCID
classification stat.MLcs.LG MSC 68Q3262L05
keywords constrainedonlinedecision-makinggeneralizedeluderdimensionuppercounterfactualconfidenceboundsofflinedensityestimationoracleutilityregretfeasibilityconstraintscontextualbanditsactivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a wide family of sequential decision problems with per-round feasibility constraints—constrained contextual bandits, stream-based active learning with label budgets, online hypothesis testing with Type I error control, and sequential $L^1$ model calibration—share a single structure and can be solved by one algorithm. The algorithm, GED-UCB, wraps any offline density-estimation oracle into an online policy that is feasible with high probability, and its expected utility regret is controlled by a new complexity measure called the generalized eluder dimension. A sympathetic reader would take the contribution to be a modular reduction: once a density class and an offline estimator are given, the remaining problem of exploration under feasibility constraints is quantified by the divergence-based eluder dimension.

What carries the argument

The load-bearing object is the generalized eluder dimension $\dim_E(\mathcal{G},D,\varepsilon)$: a context-action pair is $\varepsilon$-dependent on a history if every two densities in the class that agree on the history up to squared divergence $\varepsilon^2$ also agree at the new pair, and the dimension is the length of the longest sequence in which each pair is $\varepsilon'$-independent of its predecessors for some $\varepsilon'\ge\varepsilon$. This definition extends the classical eluder dimension, which was stated for reward functions under squared loss, to any metric-like probability divergence such as total variation, Hellinger, or $L^p$ distance. The algorithm's selection rule combines an upper counterfactual confidence bound with a potential-based exploration bonus and a worst-case constraint-width term $\max_{g',g''\in\tilde{\mathcal{C}}_G(t,\delta)}(\tilde{T}_2(g'_{x_t},\pi)-\tilde{T}_2(g''_{x_t},\pi))$; Lemma B.2 uses the eluder dimension to bound the cumulative size of that width term, which is the step that converts the oracle's estimation error into the feasibility component of the regret bound.

What would settle it

Construct a constraint density class with a small generalized eluder dimension and an oracle satisfying the assumed cumulative squared-divergence bound, then run Algorithm 1 and measure the cumulative worst-case $T_2$-width $\sum_t \max_{g',g''\in\tilde{\mathcal{C}}_G(t,\delta)}(\tilde{T}_2(g'_{x_t},\pi_t)-\tilde{T}_2(g''_{x_t},\pi_t))$; if this sum exceeds $\min\{T,\dim_E(\mathcal{G},D,1/T^2)\}+4\sqrt{\dim_E(\mathcal{G},D,1/T^2)T}$ by more than a constant, the stated regret bound does not follow.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that the constrained per-context decision problem—maximize expected utility $\tilde{T}_1(f^*_{x_t},\pi)$ subject to $\tilde{T}_2(g^*_{x_t},\pi)\le\tau$ at every round—can be solved by a UCB-style algorithm that scores each policy by an estimated utility, a counterfactual exploration bonus, and a constraint-uncertainty penalty. Theorem 7.6 is the central quantitative claim: with probability at least $1-\delta$, Algorithm 1 selects feasible policies for every round and incurs expected utility regret bounded by $O(2\beta_T(K+\log(T/K))+2K\beta_T\log T+\alpha_r(1/T+C\min\{T,\dim_E(\mathcal{G},D,1/T^2)\}+4\sqrt{\dim_E(\mathcal{G},D,1/T^2)\,T}))$, with $\beta_t$ set from the offline oracle's estimation error and $\alpha_r=(1-r_0)/(\tau-c_0)$. In other words, the cost of maintaining feasibility is governed by the generalized eluder dimension of the constraint density class rather than by the context space, and any offline density estimator with a known cumulative squared-divergence bound becomes an online algorithm with a regret guarantee.

Load-bearing premise

The load-bearing premise is that the worst-case constraint uncertainty of the stochastic policy chosen at each round is controlled by the generalized eluder dimension of the realized context-action sequence; the proof of Theorem 7.6 assumes this policy-level control rather than deriving it.

Editorial extensions

If this is right

  • Every problem that fits the framework—constrained contextual bandits, active learning with a query budget, online hypothesis testing, and $L^1$ calibration—inherits the same per-round feasibility guarantee and same regret decomposition, so results for one application transfer to the others.
  • The regret bound does not depend on the cardinality of the context space, only on the action count and the generalized eluder dimension of the density class, which is what makes the framework usable for continuous or high-dimensional contexts.
  • Because the algorithm is built on a black-box offline density estimator, any future estimator with a proven cumulative squared-divergence bound immediately yields a new constrained online algorithm without re-deriving the exploration analysis.
  • For the concrete classes studied—linear density models under $L^2$ distance, Gaussian families under Hellinger distance, and exponential families under Hellinger distance—the bounds on the generalized eluder dimension give explicit rates, showing the feasibility-related regret grows like $\sqrt{\dim_E T}$ up to logarithmic factors.
  • The setting where the density classes are parameterized subsets of $\mathbb{R}^d$ and the divergence is Lipschitz in the parameters is covered by a corollary with essentially the same regret, so the finite-class theorem extends to infinite parametric families.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the framework's reduction is tight, then the hard part of safe online learning is not algorithm design but bounding a divergence-based eluder dimension; this suggests a research program of computing that dimension for realistic model classes such as neural density estimators.
  • A natural testable extension is to replace the high-probability feasibility guarantee with an expected violation bound; the same machinery might give smaller regret when occasional small violations are tolerated.
  • The proof's policy-level width step—controlling the worst-case $T_2$-width of the chosen stochastic policy by the eluder dimension of the realized sequence—is the place where the stated theorem could be strengthened or, if it cannot be proven, the bound would need an additional complexity term.
  • The same generalized eluder dimension could likely be applied to constrained Markov decision processes and partially observable constrained settings, though the paper only lists those as future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes a unified framework for online contextual decision-making with stage-wise feasibility constraints. The learner observes a context, selects a stochastic policy over a finite action set, and must keep a constraint functional below a threshold with high probability while maximizing a utility functional. The main contribution is GED-UCB, an algorithm combining an offline density-estimation oracle, a counterfactual upper-confidence-bound selection rule, and a generalized notion of the eluder dimension defined for arbitrary metric-like divergences between conditional densities. The authors claim that GED-UCB achieves high-probability feasibility for all rounds and an expected-utility regret bound of order O(2 β_T (K + log(T/K)) + 2K β_T log T + α_r(1/T + C min{T, dim_E(G,D,1/T^2)} + 4 sqrt(dim_E(G,D,1/T^2) T))), with instantiations for linear density classes, Gaussian families, and exponential families, as well as offline oracles based on least squares and maximum likelihood.

Significance. If the main theorem were established, the paper would make a useful conceptual contribution: the generalized eluder dimension for density classes under metric-like divergences is a natural extension of Russo and Van Roy's eluder dimension, and the oracle-efficient design is appealing because it avoids strong online-oracle assumptions. The paper also demonstrates a broad range of potential applications, including constrained bandits, active learning with label budgets, online hypothesis testing, and sequential calibration. The modular lemma structure and the inclusion of complete proofs in the appendix are strengths. However, the load-bearing steps of the proof of the central regret bound are not currently justified, and the feasibility claim is internally inconsistent with the algorithm's own warm-up phase, so the advertised guarantees are not established as written.

major comments (4)
  1. [Appendix C, proof of Theorem 7.6] The final step controlling the α_r term is unsupported. The proof writes 'we apply lemma B.2 to get E_{x_t∼Q,a_t∼π_t}[max_{g',g''∈C_G(t,δ)}(T2(g'_{x,a_t})−T2(g''_{x,a_t}))] ≤ C min{dim_E(G,D,1/T^2),T} + 4√(dim_E(G,D,1/T^2)T) + 1.' But Lemma B.2 bounds the realized-pair sum ∑_{t=1}^T ω_{G_t}(x_t,a_t), where G_t is defined with radius r_t and ω is the pointwise divergence amplitude at the realized pair (x_t,a_t). The term in Theorem 7.6 is instead a sum over t of expectations over fresh contexts x of the policy-level width max_{g',g''}[tilde T2(g'_x,π_t)−tilde T2(g''_x,π_t)]. No argument is supplied to pass from realized-pair amplitudes to expectations of max-over-action widths under the randomized policy, and no concentration step for the expectation over x is shown. Moreover, the displayed application omits the √r_T factor carried by Lemma B.2; with the confidence set used in Algorithm 1, r_T = EstOff_D(G,T,δ/2T^3), so the bound would need an EstOff-dependent factor. This step is the only control of the α_r term, so Theorem 7.6(2) as stated does not follow from the presented lemmas.
  2. [Algorithm 1 and Theorem 7.6(1)] The claimed feasibility guarantee contradicts the algorithm's warm-up phase. Algorithm 1 says 'for round t = 1,···,K do Choose action a_t regardless of context x_t', and Section 6.2 explicitly states that this initial phase 'may lead to constraint violations (i.e., infeasible actions)'. Theorem 7.6(1) nevertheless claims that with probability at least 1−δ the entire policy sequence {π_t}_{t=1}^T selected in Algorithm 1 is feasible. This is internally inconsistent. Either the feasibility claim must be restricted to t>K, or the warm-up phase must be modified to use only the safe action a0.
  3. [Theorem 9.1] The stated offline oracle bound for least-squares Gaussian regression has the wrong dependence on σ². For two Gaussians with the same variance σ², the squared Hellinger distance behaves as D_H^2 ≈ ((θ*−θ̂)^T φ)^2/(8σ²). The proof's basic inequality gives ||Φθ*−Φθ̂||_2^2 ≤ O(σ²(d+√(d log(1/δ))+log(1/δ))). Combining these two facts yields a sum of Hellinger distances of order O(d+√(d log(1/δ))+log(1/δ)), not O(σ²(d+√(d log(1/δ))+log(1/δ))). The theorem as written is therefore dimensionally inconsistent, and the EstOff bound derived from it needs to be corrected or the Hellinger normalization must be clarified.
  4. [Section 3 and Section 6.2] The regret definition in Section 3 and the statements of Lemmas 7.2, 7.5, and Theorem 7.6 evaluate policies at fresh contexts x∼Q_x, but Algorithm 1 constructs π_t(·|x_t) only for the realized context x_t via the counterfactual simulation loop. The manuscript does not specify how π_t is extended to a mapping X→Δ(A). If the intent is to define π_t(x) by running the same counterfactual argmax rule for every x, that extension is not stated in the algorithm and is potentially infinite-dimensional. This gap affects the interpretation of the expected regret and the validity of the E_{x∼Q} steps in the proofs.
minor comments (3)
  1. [Theorem 7.6 statement] The displayed bound contains the expression T2(g'(x,π_t,y))−T2(g''(x,π_t,y)), where y is not defined; it should be written as max_{g',g''∈C_G(t,δ)} [tilde T2(g'_x,π_t)−tilde T2(g''_x,π_t)].
  2. [Section 5 and Theorem 7.6] Assumption 5.2 defines the Lipschitz constant L_D for the divergence D, but Theorem 7.6 and Lemma 7.2 use the notation L_1 without defining its relation to L_D. These symbols should be reconciled.
  3. [Appendix B, proof of Lemma B.2] In the proof of Lemma B.2, after summing the harmonic-like bound the text writes '≤ Cd + 4√(r_T T)', but the intermediate expression has a factor √d; the correct term should be 4√(r_T d T), matching the statement of the lemma. This appears to be a typographical slip rather than a substantive issue.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the regret bound is derived from independent oracle and eluder-dimension definitions; the noted Lemma B.2 application is a proof gap, not a circular reduction.

full rationale

GED-UCB's regret bound is a theorem derived from independently stated definitions (metric-like divergence, generalized eluder dimension, offline oracle guarantee) rather than from fitting or renaming. The confidence sets are defined by the oracle bound EstOff_D, and the main theorem's β_t and α_r are explicit parameters chosen in the proof; the regret expression contains the algorithm's UCB bonus term, but bounding regret by this bonus is standard UCB accounting, not a self-definitional reduction. Self-citations (Hu et al. 2025; Qian et al. 2024; Simchi-Levi and Xu 2021) appear in related-work and design motivation only; no load-bearing claim rests on a self-cited uniqueness theorem or on an ansatz imported from prior work by the same authors. The generalized eluder dimension is defined directly in Definition 5.5 and bounded in Section 8 for concrete classes, independently of the final regret formula. The genuine weakness is a proof gap in Appendix C: applying Lemma B.2 to the expected policy-level T2-width is not justified by that lemma (which controls realized-pair amplitudes) and drops the sqrt(r_T) factor; this is a correctness risk, not circularity, since it does not make the target result equal to an input by construction.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The framework relies on explicit, strong assumptions: a safe action with known cost, Lipschitz functionals, and a black-box density estimation oracle with known divergence bounds. No parameters are fitted to the target result, and no new physical or metaphysical entities are introduced.

assumptions (6)
  • domain assumption For every context x, there is a safe action a0 with known utility r0 and constraint value c0 ≤ τ (Assumption 3.1).
    The algorithm and the proof of Lemma 7.5 rely on this safe action as a fallback and to define the mixed policy that interpolates between the optimal policy and the safe action.
  • domain assumption The functionals T1 and T2 are Lipschitz continuous with respect to the divergence D with known constant L_D (Assumption 5.2).
    This is used to convert density estimation error in divergence D into error in functional values, which is central to the confidence bounds in Lemmas 7.2 and 7.5.
  • domain assumption The learner has access to an offline density estimation oracle AlgOff with known bounds EstOff_D for classes F and G (Assumption 5.4).
    The algorithm is built on this black-box assumption, and the regret bound is expressed in terms of these oracle bounds. The paper provides examples of such oracles only for specific classes.
  • domain assumption The true densities f* and g* belong to the known model classes F and G (realizability).
    The confidence sets and eluder dimension are defined with respect to F and G, and the regret analysis assumes the true models are inside these classes.
  • standard math The divergence D is metric-like and satisfies D ≤ C for some absolute constant C (Definition 5.1).
    The paper assumes the chosen divergence is symmetric, nonnegative, and satisfies a relaxed triangle inequality, and that all divergences are bounded by C, which is used in Lemma B.2.
  • domain assumption For infinite classes, F and G are parametrized by a compact set Θ ⊂ R^d and the densities are Lipschitz in the parameter (Assumption 7.7).
    This is used in Corollary 7.8 to control the covering number and to extend the finite-class analysis to infinite classes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Constrained Online Decision-Making: A Unified Framework." pith.science (2026). https://pith.science/paper/XWUO65X3

@misc{pith2026250507101,
  author       = {Pith},
  title        = {Pith review of: Constrained Online Decision-Making: A Unified Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XWUO65X3}},
  note         = {Machine review of arXiv:2505.07101}
}
read the original abstract

Contextual online decision-making problems with constraints appear in a wide range of real-world applications, such as adaptive experimental design under safety constraints, personalized recommendation with resource limits, and dynamic pricing under fairness requirements. In this paper, we investigate a general formulation of sequential decision-making with stage-wise feasibility constraints, where at each round, the learner must select an action based on observed context while ensuring that a problem-specific feasibility criterion is satisfied. We propose a unified algorithmic framework that captures many existing constrained learning problems, including constrained bandits, active learning with label budgets, online hypothesis testing with Type I error control, and model calibration. Central to our approach is the concept of upper counterfactual confidence bounds, which enables the design of practically efficient online algorithms with strong theoretical guarantees using any offline conditional density estimation oracle. To handle feasibility constraints in complex environments, we introduce a generalized notion of the eluder dimension, extending it from the classical setting based on square loss to a broader class of metric-like probability divergences. This allows us to capture the complexity of various density function classes and characterize the utility regret incurred due to feasibility constraint uncertainty. Our result offers a principled foundation for constrained sequential decision-making in both theory and practice.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

84 extracted references · 51 canonical work pages

  1. [1]

    Blackwell approachability and no-regret learning are equivalent

    Jacob Abernethy, Peter L Bartlett, and Elad Hazan. Blackwell approachability and no-regret learning are equivalent. In Proceedings of the 24th Annual Conference on Learning Theory, pages 27--46. JMLR Workshop and Conference Proceedings, 2011

    2011

  2. [2]

    Contextual bandit learning with predictable rewards

    Alekh Agarwal, Miroslav Dud \' k, Satyen Kale, John Langford, and Robert Schapire. Contextual bandit learning with predictable rewards. In Artificial Intelligence and Statistics, pages 19--26. PMLR, 2012

    2012

  3. [3]

    Linear stochastic bandits under safety constraints

    Sanae Amani, Mahnoosh Alizadeh, and Christos Thrampoulidis. Linear stochastic bandits under safety constraints. Advances in Neural Information Processing Systems, 32, 2019

    2019

  4. [4]

    Safe reinforcement learning with linear function approximation

    Sanae Amani, Christos Thrampoulidis, and Lin Yang. Safe reinforcement learning with linear function approximation. In International Conference on Machine Learning, pages 243--253. PMLR, 2021 a

    2021

  5. [5]

    Sanae Amani, Christos Thrampoulidis, and Lin F. Yang. Safe reinforcement learning with linear function approximation, 2021 b . URL https://arxiv.org/abs/2106.06239

    work page Pith review arXiv 2021

  6. [6]

    Learning to price with resource constraints: From full information to machine-learned prices

    Ruicheng Ao, Jiashuo Jiang, and David Simchi-Levi. Learning to price with resource constraints: From full information to machine-learned prices. arXiv preprint arXiv:2501.14155, 2025

    arXiv 2025

  7. [7]

    Neural active learning with performance guarantees, 2021

    Pranjal Awasthi, Christoph Dann, Claudio Gentile, Ayush Sekhari, and Zhilei Wang. Neural active learning with performance guarantees, 2021. URL https://arxiv.org/abs/2106.03243

    arXiv 2021

  8. [8]

    An active learning framework for multi-group mean estimation

    Abdellah Aznag, Rachel Cummings, and Adam N Elmachtoub. An active learning framework for multi-group mean estimation. Advances in Neural Information Processing Systems, 36: 0 32602--32635, 2023

    work page 2023

Show all 84 references
  1. [9]

    Bandits with knapsacks

    Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks. Journal of the ACM (JACM), 65 0 (3): 0 1--55, 2018

    2018

  2. [10]

    Using contextual bandits with behavioral constraints for constrained online movie recommendation

    Avinash Balakrishnan, Djallel Bouneffouf, Nicholas Mattei, and Francesca Rossi. Using contextual bandits with behavioral constraints for constrained online movie recommendation. In IJCAI, pages 5802--5804, 2018

    2018

  3. [11]

    Improved algorithms for neural active learning, 2023

    Yikun Ban, Yuheng Zhang, Hanghang Tong, Arindam Banerjee, and Jingrui He. Improved algorithms for neural active learning, 2023. URL https://arxiv.org/abs/2210.00423

    2023 arXiv

  4. [12]

    Neural contextual bandits for personalized recommendation

    Yikun Ban, Yunzhe Qi, and Jingrui He. Neural contextual bandits for personalized recommendation. In Companion Proceedings of the ACM Web Conference 2024, pages 1246--1249, 2024

    2024

  5. [13]

    Sequential experimentation in clinical trials: design and analysis, volume 298

    Jay Bartroff, Tze Leung Lai, and Mei-Chiung Shih. Sequential experimentation in clinical trials: design and analysis, volume 298. Springer Science & Business Media, 2012

    2012

  6. [14]

    Online decision making with high-dimensional covariates

    Hamsa Bastani and Mohsen Bayati. Online decision making with high-dimensional covariates. Operations Research, 68 0 (1): 0 276--294, 2020

    2020

  7. [15]

    Practical adversarial multivalid conformal prediction, 2022

    Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, and Aaron Roth. Practical adversarial multivalid conformal prediction, 2022. URL https://arxiv.org/abs/2206.01067

    2022 arXiv

  8. [16]

    A unifying framework for online optimization with long-term constraints

    Matteo Castiglioni, Andrea Celli, Alberto Marchesi, Giulia Romano, and Nicola Gatti. A unifying framework for online optimization with long-term constraints. Advances in Neural Information Processing Systems, 35: 0 33589--33602, 2022

    2022

  9. [17]

    Safe reinforcement learning for multi-energy management systems with known constraint functions

    Glenn Ceusters, Luis Ramirez Camargo, R \"u diger Franke, Ann Now \'e , and Maarten Messagie. Safe reinforcement learning for multi-energy management systems with known constraint functions. Energy and AI, 12: 0 100227, 2023

    2023

  10. [18]

    Meng-Che Chang and Matthieu R. Bloch. Covert online decision making: From sequential hypothesis testing to stochastic bandits, 2023. URL https://arxiv.org/abs/2311.12176

    2023 arXiv

  11. [19]

    A proof of calibration via blackwell's approachability theorem

    Dean P Foster. A proof of calibration via blackwell's approachability theorem. Games and Economic Behavior, 29 0 (1): 0 73--78, 1999. ISSN 0899-8256. doi:https://doi.org/10.1006/game.1999.0719. URL https://www.sciencedirect.com/science/article/pii/S0899825699907194

    1999

  12. [20]

    Asymptotic calibration

    Dean P Foster and Rakesh V Vohra. Asymptotic calibration. Biometrika, 85 0 (2): 0 379--390, 1998

    1998

  13. [21]

    Beyond ucb: Optimal and efficient contextual bandits with regression oracles

    Dylan Foster and Alexander Rakhlin. Beyond ucb: Optimal and efficient contextual bandits with regression oracles. In International Conference on Machine Learning, pages 3199--3210. PMLR, 2020

    2020

  14. [22]

    The statistical complexity of interactive decision making

    Dylan J Foster, Sham M Kakade, Jian Qian, and Alexander Rakhlin. The statistical complexity of interactive decision making. arXiv preprint arXiv:2112.13487, 2021

    2021 arXiv

  15. [23]

    Matrix theory

    Joel N Franklin. Matrix theory. Courier Corporation, 2012

    2012

  16. [24]

    Exploration vs exploitation vs safety: Risk-aware multi-armed bandits

    Nicolas Galichet, Michele Sebag, and Olivier Teytaud. Exploration vs exploitation vs safety: Risk-aware multi-armed bandits. In Asian conference on machine learning, pages 245--260. PMLR, 2013

    2013

  17. [25]

    Regret-optimal measurement-feedback control

    Gautam Goel and Babak Hassibi. Regret-optimal measurement-feedback control. In Learning for Dynamics and Control, pages 1270--1280. PMLR, 2021 a

    2021

  18. [26]

    Regret-optimal estimation and control, 2021 b

    Gautam Goel and Babak Hassibi. Regret-optimal estimation and control, 2021 b . URL https://arxiv.org/abs/2106.12097

    2021 arXiv

  19. [27]

    Omnipredictors, 2021

    Parikshit Gopalan, Adam Tauman Kalai, Omer Reingold, Vatsal Sharan, and Udi Wieder. Omnipredictors, 2021. URL https://arxiv.org/abs/2109.05389

    2021 arXiv

  20. [28]

    Maximum approximated likelihood estimation, 2019

    Michael Griebel, Florian Heiss, Jens Oettershagen, and Constantin Weiser. Maximum approximated likelihood estimation, 2019

    2019

  21. [29]

    A review of safe reinforcement learning: Methods, theory and applications

    Shangding Gu, Long Yang, Yali Du, Guang Chen, Florian Walter, Jun Wang, and Alois Knoll. A review of safe reinforcement learning: Methods, theory and applications. arXiv preprint arXiv:2205.10330, 2022

    2022 arXiv

  22. [30]

    The star number and eluder dimension: Elementary observations about the dimensions of disagreement

    Steve Hanneke. The star number and eluder dimension: Elementary observations about the dimensions of disagreement. In The Thirty Seventh Annual Conference on Learning Theory, pages 2308--2359. PMLR, 2024

    2024

  23. [31]

    Learning with safety constraints: Sample complexity of reinforcement learning for constrained mdps

    Aria HasanzadeZonuzy, Archana Bura, Dileep Kalathil, and Srinivas Shakkottai. Learning with safety constraints: Sample complexity of reinforcement learning for constrained mdps. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 7667--7674, 2021

    2021

  24. [32]

    The nonstochastic control problem

    Elad Hazan, Sham Kakade, and Karan Singh. The nonstochastic control problem. In Algorithmic Learning Theory, pages 408--421. PMLR, 2020

    2020

  25. [33]

    Contextual online decision making with infinite-dimensional functional regression

    Haichen Hu, Rui Ai, Stephen Bates, and David Simchi-Levi. Contextual online decision making with infinite-dimensional functional regression. arXiv preprint arXiv:2501.18359, 2025

    2025 arXiv

  26. [34]

    Directional optimism for safe linear bandits, 2024

    Spencer Hutchinson, Berkay Turan, and Mahnoosh Alizadeh. Directional optimism for safe linear bandits, 2024. URL https://arxiv.org/abs/2308.15006

    2024 arXiv

  27. [35]

    Safe exploration for optimizing contextual bandits

    Rolf Jagerman, Ilya Markov, and Maarten De Rijke. Safe exploration for optimizing contextual bandits. ACM Transactions on Information Systems (TOIS), 38 0 (3): 0 1--23, 2020

    2020

  28. [36]

    Online learning for equilibrium pricing in markets under incomplete information

    Devansh Jalota, Haoyuan Sun, and Navid Azizan. Online learning for equilibrium pricing in markets under incomplete information. In 2023 62nd IEEE Conference on Decision and Control (CDC), pages 4996--5001. IEEE, 2023

    2023

  29. [37]

    Group sequential methods with applications to clinical trials

    Christopher Jennison and Bruce W Turnbull. Group sequential methods with applications to clinical trials. CRC Press, 1999

    1999

  30. [38]

    How does variance shape the regret in contextual bandits? Advances in Neural Information Processing Systems, 37: 0 83730--83785, 2024

    Zeyu Jia, Jian Qian, Alexander Rakhlin, and Chen-Yu Wei. How does variance shape the regret in contextual bandits? Advances in Neural Information Processing Systems, 37: 0 83730--83785, 2024

    2024

  31. [39]

    Non-monotonic resource utilization in the bandits with knapsacks problem, 2022

    Raunak Kumar and Robert Kleinberg. Non-monotonic resource utilization in the bandits with knapsacks problem, 2022. URL https://arxiv.org/abs/2209.12013

    2022 arXiv

  32. [40]

    Heterogeneous sequential hypothesis testing with active source selection under budget constraints

    Sung-Wen Lan and I-Hsiang Wang. Heterogeneous sequential hypothesis testing with active source selection under budget constraints. In 2021 IEEE International Symposium on Information Theory (ISIT), pages 178--183. IEEE, 2021

    2021

  33. [41]

    Eluder-based regret for stochastic contextual MDP s

    Orin Levy, Asaf Cassel, Alon Cohen, and Yishay Mansour. Eluder-based regret for stochastic contextual MDP s. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st Internati...

    2024

  34. [42]

    Understanding the eluder dimension

    Gene Li, Pritish Kamath, Dylan J Foster, and Nati Srebro. Understanding the eluder dimension. Advances in Neural Information Processing Systems, 35: 0 23737--23750, 2022

    2022

  35. [43]

    A contextual-bandit approach to personalized news article recommendation

    Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661--670, 2010

    2010

  36. [44]

    Online optimal control with linear dynamics and predictions: Algorithms and regret analysis

    Yingying Li, Xin Chen, and Na Li. Online optimal control with linear dynamics and predictions: Algorithms and regret analysis. Advances in Neural Information Processing Systems, 32, 2019

    2019

  37. [45]

    Safe multi-agent reinforcement learning with convergence to generalized nash equilibrium

    Zeyang Li and Navid Azizan. Safe multi-agent reinforcement learning with convergence to generalized nash equilibrium. arXiv preprint arXiv:2411.15036, 2024

    2024 arXiv

  38. [46]

    Cautious regret minimization: Online optimization with long-term budget constraints

    Nikolaos Liakopoulos, Apostolos Destounis, Georgios Paschos, Thrasyvoulos Spyropoulos, and Panayotis Mertikopoulos. Cautious regret minimization: Online optimization with long-term budget constraints. In International Conference on Machine Learning, pages 3944--3952. PMLR, 2019

    2019

  39. [47]

    Trading regret for efficiency: online convex optimization with long term constraints

    Mehrdad Mahdavi, Rong Jin, and Tianbao Yang. Trading regret for efficiency: online convex optimization with long term constraints. The Journal of Machine Learning Research, 13 0 (1): 0 2503--2528, 2012

    2012

  40. [48]

    Sequential multiple hypothesis testing with type i error control

    Alan Malek, Sumeet Katariya, Yinlam Chow, and Mohammad Ghavamzadeh. Sequential multiple hypothesis testing with type i error control. In Artificial Intelligence and Statistics, pages 1468--1476. PMLR, 2017

    2017

  41. [49]

    Online learning with sample path constraints

    Shie Mannor, John N Tsitsiklis, et al. Online learning with sample path constraints. Journal of Machine Learning Research, 10 0 (3), 2009

    2009

  42. [50]

    Safe control with minimal regret

    Andrea Martin, Luca Furieri, Florian D \"o rfler, John Lygeros, and Giancarlo Ferrari-Trecate. Safe control with minimal regret. In Learning for dynamics and control conference, pages 726--738. PMLR, 2022

    2022

  43. [51]

    Regret optimal control for uncertain stochastic systems, 2024

    Andrea Martin, Luca Furieri, Florian Dörfler, John Lygeros, and Giancarlo Ferrari-Trecate. Regret optimal control for uncertain stochastic systems, 2024. URL https://arxiv.org/abs/2304.14835

    2024 arXiv

  44. [52]

    Hard-constrained neural networks with universal approximation guarantees

    Youngjae Min and Navid Azizan. Hard-constrained neural networks with universal approximation guarantees. arXiv preprint arXiv:2410.10807, 2024

    2024

  45. [53]

    Reinforcement learning with convex constraints

    Sobhan Miryoosefi, Kiant \'e Brantley, Hal Daume III, Miro Dudik, and Robert E Schapire. Reinforcement learning with convex constraints. Advances in neural information processing systems, 32, 2019

    2019

  46. [54]

    Safe linear thompson sampling with side information

    Ahmadreza Moradipari, Sanae Amani, Mahnoosh Alizadeh, and Christos Thrampoulidis. Safe linear thompson sampling with side information. IEEE Transactions on Signal Processing, 69: 0 3755--3767, 2021

    2021

  47. [55]

    Active sequential hypothesis testing

    Mohammad Naghshvar and Tara Javidi. Active sequential hypothesis testing. THE ANNALS of STATISTICS, pages 2703--2738, 2013

    2013

  48. [56]

    Model-based reinforcement learning and the eluder dimension

    Ian Osband and Benjamin Van Roy. Model-based reinforcement learning and the eluder dimension. Advances in Neural Information Processing Systems, 27, 2014

    2014

  49. [57]

    Second order bounds for contextual bandits with function approximation

    Aldo Pacchiano. Second order bounds for contextual bandits with function approximation. arXiv preprint arXiv:2409.16197, 2024

    2024 arXiv

  50. [58]

    Stochastic bandits with linear constraints

    Aldo Pacchiano, Mohammad Ghavamzadeh, Peter Bartlett, and Heinrich Jiang. Stochastic bandits with linear constraints. In International conference on artificial intelligence and statistics, pages 2827--2835. PMLR, 2021

    2021

  51. [59]

    Asymptotics of sequential composite hypothesis testing under probabilistic constraints

    Jiachun Pan, Yonglong Li, and Vincent YF Tan. Asymptotics of sequential composite hypothesis testing under probabilistic constraints. IEEE Transactions on Information Theory, 68 0 (8): 0 4998--5012, 2022

    2022

  52. [60]

    Offline oracle-efficient learning for contextual mdps via layerwise exploration-exploitation tradeoff

    Jian Qian, Haichen Hu, and David Simchi-Levi. Offline oracle-efficient learning for contextual mdps via layerwise exploration-exploitation tradeoff. arXiv preprint arXiv:2405.17796, 2024

    2024 arXiv

  53. [61]

    Some aspects of the sequential design of experiments

    Herbert Robbins. Some aspects of the sequential design of experiments . Bulletin of the American Mathematical Society, 58 0 (5): 0 527 -- 535, 1952

    1952

  54. [62]

    Uncertain: Modern topics in uncertainty estimation, 2024

    Aron Roth. Uncertain: Modern topics in uncertainty estimation, 2024

    2024

  55. [63]

    Eluder dimension and the sample complexity of optimistic exploration

    Daniel Russo and Benjamin Van Roy. Eluder dimension and the sample complexity of optimistic exploration. Advances in Neural Information Processing Systems, 26, 2013

    2013

  56. [64]

    Regret-optimal controller for the full-information problem

    Oron Sabag, Gautam Goel, Sahin Lale, and Babak Hassibi. Regret-optimal controller for the full-information problem. In 2021 American Control Conference (ACC), pages 4777--4782, 2021. doi:10.23919/ACC50511.2021.9483023

    2021

  57. [65]

    Regret-optimal lqr control, 2023

    Oron Sabag, Gautam Goel, Sahin Lale, and Babak Hassibi. Regret-optimal lqr control, 2023. URL https://arxiv.org/abs/2105.01244

    2023 arXiv

  58. [66]

    On refined versions of the azuma-hoeffding inequality with applications in information theory

    Igal Sason. On refined versions of the azuma-hoeffding inequality with applications in information theory. arXiv preprint arXiv:1111.1977, 2011

    1977 arXiv

  59. [67]

    Bypassing the monster: A faster and simpler optimal algorithm for contextual bandits under realizability

    David Simchi-Levi and Yunzong Xu. Bypassing the monster: A faster and simpler optimal algorithm for contextual bandits under realizability. Mathematics of Operations Research, 2021

    2021

  60. [68]

    Active learning for streaming data in a contextual bandit framework

    Linqi Song, Jie Xu, and Congduan Li. Active learning for streaming data in a contextual bandit framework. In Proceedings of the 2019 5th International Conference on Computing and Data Engineering, pages 29--35, 2019

    2019

  61. [69]

    Online learning with unknown constraints

    Karthik Sridharan and Seung Won Wilson Yoo. Online learning with unknown constraints. arXiv preprint arXiv:2403.04033, 2024

    2024 arXiv

  62. [70]

    l1-penalization for mixture regression models

    Nicolas Städler, Peter Bühlmann, and Sara van de Geer. l1-penalization for mixture regression models. TEST, 19 0 (2): 0 209–256, June 2010. ISSN 1863-8260. doi:10.1007/s11749-010-0197-z. URL http://dx.doi.org/10.1007/s11749-010-0197-z

    2010 doi

  63. [71]

    Falcon: Fair active learning using multi-armed bandits

    Ki Hyun Tae, Hantian Zhang, Jaeyoung Park, Kexin Rong, and Steven Euijong Whang. Falcon: Fair active learning using multi-armed bandits. arXiv preprint arXiv:2401.12722, 2024

    2024 arXiv

  64. [72]

    Online fair division with contextual bandits

    Arun Verma, Indrajit Saha, Makoto Yokoo, and Bryan Kian Hsiang Low. Online fair division with contextual bandits. arXiv preprint arXiv:2408.12845, 2024

    2024 arXiv

  65. [73]

    Safe reinforcement learning in constrained markov decision processes

    Akifumi Wachi and Yanan Sui. Safe reinforcement learning in constrained markov decision processes. In International Conference on Machine Learning, pages 9797--9806. PMLR, 2020

    2020

  66. [74]

    A survey of constraint formulations in safe reinforcement learning

    Akifumi Wachi, Xun Shen, and Yanan Sui. A survey of constraint formulations in safe reinforcement learning. arXiv preprint arXiv:2402.02025, 2024

    2024 arXiv

  67. [75]

    Optimum character of the sequential probability ratio test

    A Wald and J Wolfowitz. Optimum character of the sequential probability ratio test. The Annals of Mathematical Statistics, 19 0 (3): 0 326--339, 1948

    1948

  68. [76]

    Provably efficient reinforcement learning with general value function approximation

    Ruosong Wang, Ruslan Salakhutdinov, and Lin F Yang. Provably efficient reinforcement learning with general value function approximation. arXiv preprint arXiv:2005.10804, 2020 a

    2005 arXiv

  69. [77]

    Reinforcement learning with general value function approximation: Provably efficient approach via bounded eluder dimension

    Ruosong Wang, Russ R Salakhutdinov, and Lin Yang. Reinforcement learning with general value function approximation: Provably efficient approach via bounded eluder dimension. Advances in Neural Information Processing Systems, 33: 0 6123--6135, 2020 b

    2020

  70. [78]

    Best arm identification with safety constraints

    Zhenlin Wang, Andrew J Wagenmaker, and Kevin Jamieson. Best arm identification with safety constraints. In International Conference on Artificial Intelligence and Statistics, pages 9114--9146. PMLR, 2022

    2022

  71. [79]

    Algorithms with logarithmic or sublinear regret for constrained contextual bandits

    Huasen Wu, Rayadurgam Srikant, Xin Liu, and Chong Jiang. Algorithms with logarithmic or sublinear regret for constrained contextual bandits. Advances in Neural Information Processing Systems, 28, 2015

    2015

  72. [80]

    Upper counterfactual confidence bounds: a new optimism principle for contextual bandits

    Yunbei Xu and Assaf Zeevi. Upper counterfactual confidence bounds: a new optimism principle for contextual bandits. arXiv preprint arXiv:2007.07876, 2020

    2007 arXiv

  73. [81]

    Online convex optimization with stochastic constraints

    Hao Yu, Michael Neely, and Xiaohan Wei. Online convex optimization with stochastic constraints. Advances in Neural Information Processing Systems, 30, 2017

    2017

  74. [82]

    Gec: A unified framework for interactive decision making in mdp, pomdp, and beyond

    Han Zhong, Wei Xiong, Sirui Zheng, Liwei Wang, Zhaoran Wang, Zhuoran Yang, and Tong Zhang. Gec: A unified framework for interactive decision making in mdp, pomdp, and beyond. arXiv preprint arXiv:2211.01962, 2022

    2022 arXiv

  75. [83]

    Spoiled for choice? personalized recommendation for healthcare decisions: A multiarmed bandit approach

    Tongxin Zhou, Yingfei Wang, Lu Yan, and Yong Tan. Spoiled for choice? personalized recommendation for healthcare decisions: A multiarmed bandit approach. Information Systems Research, 34 0 (4): 0 1493--1512, 2023

    2023

  76. [84]

    Kim, Omer Reingold, and Guy N

    Úrsula Hébert-Johnson, Michael P. Kim, Omer Reingold, and Guy N. Rothblum. Calibration for the (computationally-identifiable) masses, 2018. URL https://arxiv.org/abs/1711.08513

    2018 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.