Pith. sign in

REVIEW 2 major objections 5 minor 2 cited by

Offline Learning for Combinatorial Multi-armed Bandits

T0 review · 2 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that logged data alone, run through pessimistic lower confidence bounds and an approximate combinatorial oracle, can identify a combinatorial action whose reward gap matches the lower bound up to log factors.

desk verdict First offline CMAB-T framework with a solid upper bound and a fixable but currently unproven lower bound; deserves refereeing. read the letter →

arxiv 2501.19300 v2 pith:3NNUU2KE submitted 2025-01-31 cs.LG

Xutong Liu , Xiangxiang Dai , Jinhang Zuo , Siwei Wang , Carlee Joe-Wong , John C.S. Lui , Wei Chen This is my paper · ORCID
classification cs.LG
keywords offlinecombinatorialbanditsmulti-armedprobabilisticallytriggeredarmspessimisticlowerconfidencebounddatacoveragecascadingLLMcachinginfluencemaximization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Off-CMAB, the first offline learning framework for combinatorial multi-armed bandits with probabilistically triggered arms (CMAB-T), where the learner only sees a fixed dataset of $n$ logged actions and the feedback those actions triggered. Its central object is the combinatorial lower confidence bound (CLCB) algorithm, which computes a high-probability pessimistic estimate for every base arm and passes those estimates to an approximate combinatorial solver. The paper's central claim is that, under a new infinity-norm triggering-probability-modulated (TPM) data coverage condition, CLCB returns an action whose suboptimality gap is at most $2\alpha B_1 \bar{K}^*_2 \sqrt{2 C^*_\infty \log(2mn/\delta)/n}$, and that this matches a lower bound of $\Omega(\bar{K}^*_2\sqrt{C^*_\infty/n})$ up to logarithms. If correct, this is the first guarantee that offline datasets suffice for hard combinatorial action spaces, with concrete payoffs for learning to rank, LLM caching, and social influence maximization.

What carries the argument

The load-bearing machinery is the pairing of pessimistic base-arm lower confidence bounds with an $\alpha$-approximate oracle under triggering-probability-modulated conditions. The matching condition is 1-norm TPM smoothness, $|r(S;\mu')-r(S;\mu)| \le B_1 \sum_i p_i^{\mu,S}|\mu_i-\mu'_i|$, which says a poorly observed arm can only move the reward by an amount proportional to how often the action triggers it. The dataset-quality conditions are the infinity-norm coverage $\max_i p_i^{\mu,S^*}/p_i^{\mu,D_S} \le C^*_\infty$ and the 1-norm coverage $\sum_i p_i^{\mu,S^*}/p_i^{\mu,D_S} \le C^*_1$, each measuring how much data exists for the arms the optimal action needs. The proof's key step is the decomposition of the gap into uncertainty, oracle, and pessimism gaps, with monotonicity forcing the last two to be non-positive and smoothness converting the first into a sum of per-arm estimation errors.

What would settle it

Recompute the total KL divergence between the two $k$-path instances in Appendix E under the stated data distribution $D_S=(1/C^*_\infty,1-1/C^*_\infty,0,\dots)$. If it equals $n(1-1/C^*_\infty)\,KL(\mathrm{Bern}(1/2-\Delta),\mathrm{Bern}(1/2+\Delta))$, which is larger than the paper's claimed $n\,KL/C^*_\infty$, then the lower-bound proof fails as written; if a corrected distribution or a corrected inequality restores the claimed bound, the near-optimality conclusion survives.

Watch

Extended reading notes

Core claim

The paper's discovery is that pessimism transfers from base arms to combinatorial actions: because the reward is monotone in the base-arm means, replacing the unknown means by lower confidence bounds makes the estimated reward of the optimal action an upper bound on its pessimistic value, so any action the oracle prefers has a pessimistic value at least as good. The suboptimality gap then decomposes into an uncertainty gap over the optimal action, an oracle gap, and a pessimism gap, and monotonicity plus the oracle's approximation guarantee make the last two non-positive. What remains is controlled by the 1-norm TPM smoothness condition and by either the infinity-norm or 1-norm TPM coverage coefficient of the logged data. The paper also constructs a lower-bound instance on the stochastic $k$-path problem and claims no algorithm can beat $\Omega(k\sqrt{C^*_\infty/n})$ under the same coverage condition, so the CLCB bound is order-optimal up to logarithmic factors.

Load-bearing premise

The load-bearing premise is the Appendix E inequality $KL(P^N_{\mu_1}\|P^N_{\mu_2}) \le n\,KL(P_{\mu_1}\|P_{\mu_2})/C^*_\infty$ for the stated data distribution; direct computation with $D_S=(1/C^*_\infty,1-1/C^*_\infty,0,\dots)$ gives a larger total KL because the distinguishing path is drawn with probability $1-1/C^*_\infty$, so the matching lower bound stands only if that step is repaired.

Editorial extensions

If this is right

  • With $n$ logged samples, CLCB finds an action with gap $\tilde{O}(\bar{K}^*_2\sqrt{C^*_\infty/n})$; doubling the dataset halves the gap, with no online exploration.
  • For cascading bandits under uniform logged lists, the specialized algorithm has gap $O(\sqrt{(k/n)\,m\mu_1/\mu_k})$, a direct corollary of the general bound.
  • For LLM caching with empty-cache logging, the cost gap is $\tilde{O}(\sqrt{m/n})$, improving on the previous $\tilde{O}(k\sqrt{m/(C_1 n)})$ guarantee.
  • For influence maximization with node-level feedback, the variance-adaptive LCB algorithm attains a gap $\tilde{O}(\sqrt{V^2 d_{\max}^2 \sigma^2(S^*;G)/(\eta\gamma^3 n)})$ under bounded seed-sampling and activation probabilities.
  • The $k$-path lower bound $\Omega(k\sqrt{C^*_\infty/n})$ asserts that the $\sqrt{C^*_\infty/n}$ dependence is unavoidable for any offline CMAB algorithm.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Appendix E lower-bound calculation is repaired, the framework's claim to order-optimality is much stronger; until then, the upper bound should be read as a standalone guarantee, and the lower bound as conditional on that KL step.
  • The TPM coverage conditions are partial coverage: they only require data for arms the optimal action triggers, not uniform coverage over all actions. This suggests the same pessimistic template should extend to other offline structured-prediction problems where the reward is monotone and Lipschitz in latent parameters.
  • One testable extension is to replace Hoeffding-style intervals everywhere with the variance-adaptive intervals the paper already uses in the influence-maximization section; this could improve the practical gap in cascading bandits and LLM caching at no algorithmic cost.
  • Treating the LLM query-arrival distribution as one vector-valued full-feedback arm is what removes a $\sqrt{m}$ factor; the same trick may apply to other bandit problems with categorical global state observed every round.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper introduces Off-CMAB, an offline learning framework for combinatorial multi-armed bandits with probabilistically triggered arms (CMAB-T). The authors propose the CLCB algorithm, which builds per-arm lower confidence bounds and feeds them to an alpha-approximate combinatorial oracle, and they prove upper bounds on the alpha-approximate suboptimality gap under two new coverage conditions (Conditions 3 and 4). They also state a minimax lower bound for a k-path instance (Theorem 2) to support the claim that the upper bound is optimal up to logarithmic factors. The paper then instantiates the framework for cascading bandits, LLM caching, and influence maximization, with experiments on synthetic and real-world data.

Significance. The upper-bound analysis in Theorem 1 is a genuine contribution: it is the first near-optimal guarantee for offline CMAB-T, and the derivation is self-contained, with explicit coverage coefficients and a clean treatment of nonlinear rewards and probabilistic triggering. The applications to learning to rank, LLM caching, and influence maximization provide useful new results, and the empirical section supports the algorithm's practical value. However, the lower bound of Theorem 2, which is the basis for the 'matching up to a logarithmic factor' claim, is not established by the submitted proof. If the lower bound can be repaired, as appears plausible, the paper would be a strong contribution to offline combinatorial bandits; in its current form the central optimality claim is unsupported.

major comments (2)
  1. [Appendix E (proof of Theorem 2)] The step after Eq. (42) asserts KL(P^N_{mu1} || P^N_{mu2}) <= n KL(P_{mu1} || P_{mu2}) / C*_inf, which is incorrect for the stated data-collecting distribution D_S = (1/C*_inf, 1-1/C*_inf, 0, ..., 0). Because mu1 and mu2 differ only on path 2, and path 2 is drawn with probability 1-1/C*_inf, the true per-sample KL is (1-1/C*_inf) * KL(Bern(1/2-Delta) || Bern(1/2+Delta)). Plugging this into the subsequent display yields at best a bound of order k * min(1, 1/sqrt(n)), independent of C*_inf, rather than k * min(1, sqrt(C*_inf/n)). The construction appears repairable by taking D_S with probability 1/C*_inf on each of paths 1 and 2, but as written Theorem 2 does not follow.
  2. [Theorem 2 and Appendix E] The statement of Theorem 2 asserts a lower bound of k * min(1, sqrt(C*_inf/n)), but the proof's final lines yield only (k/(8e)) * min(1/4, sqrt(C*_inf/(20n))) even after the KL step is corrected. The theorem statement and proof should be reconciled; as stated, the constant factor is not supported.
minor comments (5)
  1. [Appendix B] The final paragraph contains the sentence 'We will incorporate this discussion and examples into the final version of the paper.', which is a self-referential note that does not belong in the manuscript; remove it.
  2. [Appendix D] In the proof of the 1-norm bound, the first line reads '1-nrom TPM data coverage condition'; it should read '1-norm TPM data coverage condition'.
  3. [Section C and Appendix H] The sentence 'we can only obverse the node-level feedback' should say 'observe' instead of 'obverse'.
  4. [Appendix E] The notation P^N_{mu1} (or Pp tensor mu1) is used before it is defined; please define the data-generating distribution of the n-sample dataset explicitly.
  5. [Section 5 and Appendix J] The description of the real-world LLM cache experiment says Fig. 3b plots results over n = 100 rounds, while Appendix J states 10,000 queries; this discrepancy should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the upper bound follows from stated coverage and smoothness conditions via self-contained concentration arguments, and the lower bound is an independent Le Cam construction.

full rationale

Theorem 1's upper bound is derived directly from the CLCB construction: the proof decomposes the alpha-approximate suboptimality gap into uncertainty, oracle, and pessimism gaps; the oracle gap is non-positive by Eq. (1), the pessimism gap is non-positive by monotonicity (Condition 1), and the uncertainty gap is controlled by the 1-norm TPM smoothness condition (Condition 2), per-arm Hoeffding concentration (Lemma 5), counter concentration (Lemma 6), and the coverage conditions. The coefficients C*_infinity and C*_1 are measurable properties of (Darm, DS, S*) and appear as unknowns in the guarantee, not as parameters fitted to the target suboptimality, so Eq. (5) and Eq. (6) are not equivalent to their inputs by construction. Theorem 2 is an independent information-theoretic lower bound via Le Cam's method on two k-path instances; it does not invoke Theorem 1 or any uniqueness claim, so it is not circular. I do flag a non-circular correctness concern: in Appendix E the proof asserts KL(Pp_otimes_mu1 || Pp_otimes_mu2) <= n KL(P_mu1 || P_mu2)/C*_infinity, but with the stated data distribution DS = (1/C*_infinity, 1 - 1/C*_infinity, 0, ...), the two instances differ only on path 2, which has sampling probability 1 - 1/C*_infinity, so the displayed bound appears to understate the true KL; this threatens the claimed matching lower bound but is an error in an independent argument, not a circular step. The applications (cascading bandits, LLM cache, and influence maximization) verify the required conditions rather than renaming a known result, and the stated improvements over Zhu et al. (2023) and Chen et al. (2021) are comparisons of bounds on the same problem. The IM proof borrows a Bernstein-type concentration lemma from Liu et al. (2022), a self-citation, but it is a standard lemma with stated assumptions that do not include the target result, so per the rules it is real evidence and does not raise the circularity score. Section B contains the draft artifact 'We will incorporate this discussion and examples into the final version of the paper'; this indicates an incomplete manuscript, not a circular dependence. The mislabeling of Algorithm 5 versus Algorithm 2 in Theorem 3 is likewise a textual error with no circularity implications. Overall, no prediction or theorem in this paper reduces by construction to its inputs, and no load-bearing step is carried by an unverified self-citation.

Assumptions & free parameters 0 free parameters · 7 assumptions · 0 invented entities

No free parameters are fitted; the algorithm's confidence interval width is determined by δ, m, n. The theoretical results rely on standard concentration inequalities and domain assumptions (monotonicity, TPM smoothness, α-approximate oracle, feedback independent of triggering). The data coverage conditions are new restrictions introduced by this paper; they are the gate for all bounds.

assumptions (7)
  • domain assumption Monotonicity (Condition 1): reward is nondecreasing in each arm mean
    Section 2.1, used in the pessimism gap ≤ 0 step of Theorem 1's proof.
  • domain assumption 1-norm TPM bounded smoothness (Condition 2) with coefficient B1
    Section 2.1; bounds reward changes by triggering-probability-weighted L1 change in means, key to the uncertainty gap bound.
  • domain assumption α-approximate oracle exists (Eq. 1)
    Section 2.2; without it the oracle gap term in the proof cannot be set to zero; many combinatorial rewards are NP-hard to optimize exactly.
  • domain assumption Expected reward r(S;µ) depends on arm distributions only through the mean vector µ
    Section 2.1; used throughout to justify plugging mean estimates into the reward.
  • ad hoc to paper Infinity-norm or 1-norm TPM data coverage (Conditions 3, 4) holds with finite C*
    Introduced in this paper; if an arm of the optimal action is never triggered in the offline data, the coefficient is infinite and the bounds are vacuous.
  • domain assumption Assumption 1 (IM): seed sampling probability q_u ∈ [γ,1-γ] and one-step activation probability p(̄v) ≥ η
    Section C; needed for the variance-adaptive confidence intervals that construct edge LCBs from node-level feedback.
  • standard math Standard concentration inequalities (Hoeffding, multiplicative Chernoff, Le Cam's method)
    Appendix I; used for arm concentration, counter concentration, and the lower bound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Offline Learning for Combinatorial Multi-armed Bandits." pith.science (2026). https://pith.science/paper/3NNUU2KE

@misc{pith2026250119300,
  author       = {Pith},
  title        = {Pith review of: Offline Learning for Combinatorial Multi-armed Bandits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3NNUU2KE}},
  note         = {Machine review of arXiv:2501.19300}
}
read the original abstract

The combinatorial multi-armed bandit (CMAB) is a fundamental sequential decision-making framework, extensively studied over the past decade. However, existing work primarily focuses on the online setting, overlooking the substantial costs of online interactions and the readily available offline datasets. To overcome these limitations, we introduce Off-CMAB, the first offline learning framework for CMAB. Central to our framework is the combinatorial lower confidence bound (CLCB) algorithm, which combines pessimistic reward estimations with combinatorial solvers. To characterize the quality of offline datasets, we propose two novel data coverage conditions and prove that, under these conditions, CLCB achieves a near-optimal suboptimality gap, matching the theoretical lower bound up to a logarithmic factor. We validate Off-CMAB through practical applications, including learning to rank, large language model (LLM) caching, and social influence maximization, showing its ability to handle nonlinear reward functions, general feedback models, and out-of-distribution action samples that excludes optimal or even feasible actions. Extensive experiments on synthetic and real-world datasets further highlight the superior performance of CLCB.

Figures

Figures reproduced from arXiv: 2501.19300 by the authors.

Figure 1
Figure 1. Suboptimality gaps for cascading bandit application. We now present experimental results on both synthetic and real-world datasets. Each experiment was conducted over 20 8 [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Comparison of different offline data generation methods. (a) Synthetic Dataset (b) Real-world Dataset [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Suboptimality gaps for LLM cache application. independent trials. For cascading bandits on the application of learning to rank, in the synthetic setting, item parameters µi are drawn from U[0, 1], and in each round t, a ranked list St of K items is randomly sampled. Fig. 1a shows that compared to CUCB-Offline (Chen et al., 2016), which is an offline adaptation of CUCB for our setting, and EMP (Liu et al., 2021), whi… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Algorithms on another LLM. Real-World Dataset. We use the SciQ dataset (Welbl et al., 2017), which covers a variety of topics, including physics, chemistry, and biology, to evaluate the performance of our proposed CLCB-LLM-C algorithm using OpenAI’s LLMs. The cost is d…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Best Arm Identification with Possibly Biased Offline Data

    cs.LG 2025-05 conditional novelty 6.0 of 10

    LUCB-H adaptively combines offline and online data for best arm identification, matching or beating standard LUCB depending on whether the historical data is helpful or misleading.

  2. A Unified Online-Offline Framework for Co-Branding Campaign Recommendations

    cs.LG 2025-05 conditional novelty 5.0 of 10

    A unified framework for co-branding learns partner success probabilities and market gains online, and allocates sub-brand budgets offline with a 1-1/e approximation guarantee.

Reference graph

Works this paper leans on

33 extracted references · 17 canonical work pages · cited by 2 Pith papers

  1. [1]

    Casper, S., Davies, X., Shi, C., Gilbert, T

    JMLR Workshop and Conference Proceedings, 2012. Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., Wang, T., Marks, S., S ´egerie, C.-R., Car- roll, M., Peng, A., Christoffersen, P. J. K., Damani, M., Slocum, S., Anwar, U., Siththaranjan, A., Nadeau, M., Michaud, E. J., Pfau, J., Kr...

    arXiv 2012

  2. [5]

    org/CorpusID:260316010

    URL https://api.semanticscholar. org/CorpusID:260316010. Cesa-Bianchi, N. and Lugosi, G. Combinatorial bandits. Journal of Computer and System Sciences, 78(5):1404– 1422, 2012. Chang, J. D., Uehara, M., Sreenivas, D., Kidambi, R., and Sun, W. Mitigating covariate shift in im- itation learning via offline data with partial cover- age. InNeural Information ...

    work page 2012

  3. [10]

    org/CorpusID:261065303

    URL https://api.semanticscholar. org/CorpusID:261065303. Choi, H.-j., Udwani, R., and Oh, M.-h. Cascading contex- tual assortment bandits.Advances in Neural Information Processing Systems, 36, 2024. Combes, R., Talebi Mazraeh Shahi, M. S., Proutiere, A., et al. Combinatorial bandits revisited.Advances in neural information processing systems, 28, 2015. Cr...

    arXiv 2024

  4. [12]

    org/CorpusID:235422620

    URL https://api.semanticscholar. org/CorpusID:235422620. Fujimoto, S., Meger, D., and Precup, D. Off- policy deep reinforcement learning without exploration. InInternational Conference on Machine Learning,

    work page

  5. [14]

    org/CorpusID:265067391

    URL https://api.semanticscholar. org/CorpusID:265067391. Gy¨orgy, A., Linder, T., Lugosi, G., and Ottucs´ak, G. The on-line shortest path problem under partial monitoring. Journal of Machine Learning Research, 8(10), 2007. 11 Offline Learning for Combinatorial Multi-armed Bandits Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor- critic: Off-p...

    arXiv 2007

  6. [15]

    org/CorpusID:28202810

    URL https://api.semanticscholar. org/CorpusID:28202810. Han, Y ., Wang, Y ., and Chen, X. Adversarial combinatorial bandits with general non-linear reward functions. In International Conference on Machine Learning, pp. 4030–

    work page

  7. [18]

    org/CorpusID:211011033

    URL https://api.semanticscholar. org/CorpusID:211011033. Kumar, A., Fu, J., Tucker, G., and Levine, S. Stabi- lizing off-policy q-learning via bootstrapping error re- duction. InNeural Information Processing Systems,

    work page

  8. [19]

    org/CorpusID:173990380

    URL https://api.semanticscholar. org/CorpusID:173990380. Kumar, A., Zhou, A., Tucker, G., and Levine, S. Conservative q-learning for offline reinforcement learning.ArXiv, abs/2006.04779, 2020. URL https: //api.semanticscholar.org/CorpusID: 219530894. Kveton, B., Szepesvari, C., Wen, Z., and Ashkan, A. Cas- cading bandits: Learning to rank in the cascade m...

    arXiv 2006

Show all 33 references
  1. [20]

    org/CorpusID:261697361

    URL https://api.semanticscholar. org/CorpusID:261697361. Lange, S., Gabel, T., and Riedmiller, M. Batch reinforce- ment learning. InReinforcement learning: State-of-the- art, pp. 45–73. Springer, 2012. Lattimore, T. and Szepesv´ari, C.Bandit algorithms. Cam- bridge University ...

    2012 arXiv

  2. [21]

    org/CorpusID:219676905

    URL https://api.semanticscholar. org/CorpusID:219676905. Liu, T.-Y . et al. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval, 3(3): 225–331, 2009. Liu, X., Zuo, J., Chen, X., Chen, W., and Lui, J. C. Multi-layered network exploration...

    2009 arXiv

  3. [22]

    org/CorpusID:208617840

    URL https://api.semanticscholar. org/CorpusID:208617840. Narasimhan, H., Parkes, D. C., and Singer, Y . Learnability of influence in networks.Advances in Neural Information Processing Systems, 28, 2015. Neu, G. First-order regret bounds for combinatorial semi- bandits. InConfe...

    2015

  4. [25]

    org/CorpusID:271534421

    URL https://api.semanticscholar. org/CorpusID:271534421. Rashidinejad, P., Zhu, B., Ma, C., Jiao, J., and Russell, S. J. Bridging offline reinforcement learning and imitation learning: A tale of pessimism.IEEE Transactions on Information Theory, 68:8156–8196, 2021. Rashidineja...

    2021 arXiv

  5. [27]

    org/CorpusID:257495837

    URL https://api.semanticscholar. org/CorpusID:257495837. Shi, L., Li, G., Wei, Y ., Chen, Y ., and Chi, Y . Pessimistic q-learning for offline reinforcement learning: Towards optimal sample complexity.ArXiv, abs/2202.13890,

    arXiv

  6. [28]

    org/CorpusID:247159013

    URL https://api.semanticscholar. org/CorpusID:247159013. Singh, B., Kumar, R., and Singh, V . P. Reinforce- ment learning in robotic applications: a comprehensive survey.Artificial Intelligence Review, 55:945 – 990,

  7. [29]

    org/CorpusID:234826156

    URL https://api.semanticscholar. org/CorpusID:234826156. Sun, X., Guo, T., Han, C., and Zhang, H. Greedy algorithms for stochastic monotone k-submodular maximization un- der full-bandit feedback.Journal of Combinatorial Opti- mization, 49(1):1–25, 2025. Szepesvari, C. and Muno...

    2025

  8. [31]

    org/CorpusID:202121359

    URL https://api.semanticscholar. org/CorpusID:202121359. Wang, Q. and Chen, W. Improving regret bounds for com- binatorial semi-bandits with probabilistically triggered arms and its applications. InAdvances in Neural Infor- mation Processing Systems, pp. 1161–1171, 2017. Wang,...

    2017 arXiv

  9. [32]

    org/CorpusID:182952558

    URL https://api.semanticscholar. org/CorpusID:182952558. Xie, T., Cheng, C.-A., Jiang, N., Mineiro, P., and Agarwal, A. Bellman-consistent pessimism for offline reinforce- ment learning. InNeural Information Processing Systems, 2021a. URL https://api.semanticscholar. org/Corpu...

    2021 arXiv

  10. [33]

    batch RL

    URL https://api.semanticscholar. org/CorpusID:259145065. Zhong, Z., Chueng, W. C., and Tan, V . Y . Thompson sam- pling algorithms for cascading bandits.Journal of Ma- chine Learning Research, 22(218):1–66, 2021. Zhu, B., Sheng, Y ., Zheng, L., Barrett, C., Jordan, M., and Jia...

    2012

  11. [159]

    Chen, W., Wang, Y ., Yuan, Y ., and Wang, Q

    PMLR, 2013. Chen, W., Wang, Y ., Yuan, Y ., and Wang, Q. Combinatorial multi-armed bandit and its extension to probabilistically triggered arms.The Journal of Machine Learning Re- search, 17(1):1746–1778, 2016. Chen, W., Sun, X., Zhang, J., and Zhang, Z. Optimiza- tion from st...

    2013

  12. [1375]

    Nguyen-Tang, T., Gupta, S., Nguyen, A

    PMLR, 2015. Nguyen-Tang, T., Gupta, S., Nguyen, A. T., and Venkatesh, S. Offline neural contextual bandits: Pessimism, optimiza- tion and generalization.arXiv preprint arXiv:2111.13807, 2021a. 13 Offline Learning for Combinatorial Multi-armed Bandits Nguyen-Tang, T., Gupta, S....

    2015 arXiv

  13. [1716]

    Chen, X., Zhou, Z., Wang, Z., Wang, C., Wu, Y ., Deng, Q., and Ross, K

    PMLR, 2021. Chen, X., Zhou, Z., Wang, Z., Wang, C., Wu, Y ., Deng, Q., and Ross, K. W. Bail: Best-action imitation learning for batch deep reinforcement learn- ing.ArXiv, abs/1910.12179, 2019. URL https: //api.semanticscholar.org/CorpusID: 204907199. Chen, X., Wang, S., McAule...

    2021 arXiv

  14. [1724]

    Chen, W., Sun, X., Zhang, J., and Zhang, Z

    PMLR, 2020. Chen, W., Sun, X., Zhang, J., and Zhang, Z. Network inference and influence maximization from samples. In 10 Offline Learning for Combinatorial Multi-armed Bandits International Conference on Machine Learning, pp. 1707–

    2020

  15. [2002]

    org/CorpusID:5785954

    URL https://api.semanticscholar. org/CorpusID:5785954. Riedmiller, M. Neural fitted q iteration–first experiences with a data efficient neural reinforcement learning method. InMachine learning: ECML 2005: 16th European con- ference on machine learning, Porto, Portugal, October...

    2005

  16. [2005]

    org/CorpusID:8617488

    URL https://api.semanticscholar. org/CorpusID:8617488. Takemura, K., Ito, S., Hatano, D., Sumita, H., Fukunaga, T., Kakimura, N., and Kawarabayashi, K.-i. Near-optimal regret bounds for combinatorial semi-bandits with linear payoff functions. InProceedings of the AAAI Conferen...

    2021 arXiv

  17. [2015]

    org/CorpusID:742580

    URL https://api.semanticscholar. org/CorpusID:742580. Balkanski, E., Rubinstein, A., and Singer, Y . The power of optimization from samples. InNeural Informa- tion Processing Systems, 2016. URL https://api. semanticscholar.org/CorpusID:15394546. Bang, F. Gptcache: An open-sour...

    2016

  18. [2018]

    org/CorpusID:54457299

    URL https://api.semanticscholar. org/CorpusID:54457299. Gai, Y ., Krishnamachari, B., and Jain, R. Learning mul- tiuser channel allocations in cognitive radio networks: A combinatorial multi-armed bandit formulation. In2010 IEEE Symposium on New Frontiers in Dynamic Spectrum (...

    2010 arXiv

  19. [2019]

    org/CorpusID:212628904

    URL https://api.semanticscholar. org/CorpusID:212628904. Agrawal, S., Avadhanula, V ., Goyal, V ., and Zeevi, A. Mnl- bandit: A dynamic learning approach to assortment selec- tion.Operations Research, 67(5):1453–1485, 2019. Audibert, J.-Y ., Bubeck, S., and Lugosi, G. Regret i...

    2019 arXiv

  20. [2020]

    org/CorpusID:218595964

    URL https://api.semanticscholar. org/CorpusID:218595964. Kiran, B. R., Sobh, I., Talpaert, V ., Mannion, P., Sallab, A. A. A., Yogamani, S. K., and P’erez, P. Deep reinforcement learning for au- tonomous driving: A survey.IEEE Transactions on Intelligent Transportation Systems...

  21. [2021]

    org/CorpusID:248498378

    URL https://api.semanticscholar. org/CorpusID:248498378. Chen, J. and Jiang, N. Information-theoretic considerations in batch reinforcement learning. InInternational Conference on Machine Learning, 2019a. URL https: //api.semanticscholar.org/CorpusID: 141460093. Chen, J. and J...

    2018

  22. [2022]

    org/CorpusID:253420623

    URL https://api.semanticscholar. org/CorpusID:253420623. Qin, L., Chen, S., and Zhu, X. Contextual combinatorial bandit and its application on diversified online recommen- dation. InProceedings of the 2014 SIAM International Conference on Data Mining, pp. 461–469. SIAM, 2014. ...

    2014 arXiv

  23. [2023]

    org/CorpusID:265607979

    URL https://api.semanticscholar. org/CorpusID:265607979. Bouneffouf, D., Rish, I., and Aggarwal, C. Survey on appli- cations of multi-armed and contextual bandits. In2020 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE, 2020. Bubeck, S., Cesa-Bianchi, N., and Ka...

    2020

  24. [2024]

    org/CorpusID:273185502

    URL https://api.semanticscholar. org/CorpusID:273185502. Fourati, F., Aggarwal, V ., Quinn, C., and Alouini, M.-S. Randomized greedy learning for non-monotone stochas- tic submodular maximization under full-bandit feedback. InInternational Conference on Artificial Intelligence...

    2023 arXiv

  25. [4039]

    Hwang, T., Chai, K., and Oh, M.-h

    PMLR, 2021. Hwang, T., Chai, K., and Oh, M.-h. Combinatorial neural bandits. InInternational Conference on Machine Learn- ing, pp. 14203–14236. PMLR, 2023. Ito, S. Hybrid regret bounds for combinatorial semi-bandits and adversarial linear bandits.Advances in Neural Infor- mati...

    2021 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.