Pith. sign in

REVIEW 1 major objections 5 minor 37 references

Fusing Reward and Dueling Feedback in Stochastic Bandits

T0 review · 1 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In a bandit that gets both rewards and pairwise comparisons, the regret of each suboptimal arm can shrink to the smaller of the two feedback costs, and a decomposition algorithm attains this up to a constant factor.

desk verdict New model plus a solid lower bound; the algorithm's near-optimality is real only under an assumption the paper acknowledges but the abstract underplays. read the letter →

arxiv 2504.15812 v1 pith:2QQEJ4B3 submitted 2025-04-22 cs.LG cs.AI

Xuchuang Wang , Qirun Zeng , Jinhang Zuo , Xutong Liu , Mohammad Hajiesmaili , John C.S. Lui , Adam Wierman This is my paper · ORCID
classification cs.LGcs.AI MSC 62L0568Q32
keywords multi-armedbanditsduelingrelativefeedbackrewardregretlowerboundfusioneliminationalgorithmrandomizedexploration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks what happens when an online learner receives both absolute scores and relative comparisons at every round, a setting it calls the dueling-reward multi-armed bandit. It claims that for each arm that is worse than the best arm, the learner need only pay the cheaper of the two learning costs: roughly $\log T / \max\{\Delta_k^{(R)}/\alpha, \Delta_k^{(D)}/(1-\alpha)\}$, where the gaps measure reward and dueling disadvantage and $\alpha$ weights feedback costs. The paper proves a matching lower bound, gives a simple elimination-based fusion algorithm, and then a decomposition-based algorithm whose regret matches the simplified lower bound up to a constant. A sympathetic reader would care because real systems such as LLM alignment collect both scalar ratings and pairwise preferences, and the result says fusing them can be strictly better than using either alone.

What carries the argument

The carrying mechanism is an arm decomposition plus randomized exploration assignment. DecoFusion maintains two approximately decomposed sets $\hat{K}^{(R)}_t$ and $\hat{K}^{(D)}_t$ built from revised empirical log-likelihoods $\hat{I}^{(R)}_{k,t}$ and $\hat{I}^{(D)}_{k,t}$, which measure how much reward and dueling information has distinguished each arm. Each round the algorithm picks one exploration arm, then flips a coin with threshold $\alpha^2/(\alpha^2+(1-\alpha)^2)$ to decide whether to explore it via reward while exploiting the dueling-optimal arm, or explore it via dueling while exploiting the reward-optimal arm. This split compensates for the mismatch between the estimated decomposition and the true one, and it is what converts the lower bound's per-arm minimization into a matching upper bound.

What would settle it

Build a three-arm Bernoulli instance where arm 1 beats arm 2 only slightly ($\Delta_2^{(D)}$ tiny) while arm 3 beats arm 2 overwhelmingly, so that $\ell_2^* = 3$ rather than 1. Run DecoFusion with $\alpha=0.5$ for increasing $T$, estimate $E[R_T]/\log T$, and compare it to the simplified bound $C/\max\{\Delta_2^{(R)}/\alpha, \Delta_2^{(D)}/(1-\alpha)\}$ summed over arms; exceeding the bound by a gap that grows with the advantage of arm 3 would falsify the matching claim.

Watch

Extended reading notes

Core claim

The central discovery is that reward feedback (a numerical score for one arm) and dueling feedback (a winner between two arms) are not redundant in regret terms: their contributions to identifying a suboptimal arm add up, while the regret they incur does not. This leads to a regret lower bound in which each suboptimal arm contributes the minimum of a reward-based cost and a dueling-based cost. The paper's DecoFusion algorithm realizes this bound by partitioning suboptimal arms into a reward-explored group and a dueling-explored group, estimating these groups from empirical log-likelihoods, and randomly assigning one feedback type to exploration and the other to exploitation in each round. When each suboptimal arm is most effectively compared against the optimal arm, DecoFusion achieves $E[R_T] = O(\sum_{k\neq 1} \log T / \max\{\Delta_k^{(R)}/\alpha, \Delta_k^{(D)}/(1-\alpha)\})$, matching the simplified lower bound up to a constant. It also shows that for $\alpha=0$ or $\alpha=1$, where one feedback type is free, regret becomes constant in $T$.

Load-bearing premise

The near-optimal guarantee assumes that for every losing arm, the cheapest way to learn about it by comparison is to duel it against the single best arm; if comparing it against some other arm gives better information per unit of regret, the claimed simplified matching can fail.

Editorial extensions

If this is right

  • If both rewards and comparisons are available, systems like RLHF can be designed to pay only the smaller regret cost for each bad item, rather than the sum or a fixed blend of both costs.
  • When one feedback channel is free ($\alpha=0$ or $\alpha=1$), DecoFusion achieves $T$-independent regret, so exploration can be offloaded entirely to the free channel and exploitation to the other.
  • ElimFusion, which shares one candidate arm set between two elimination routines, still improves over no fusion but pays an extra factor of the number of arms from dueling elimination.
  • The near-optimal constant-factor guarantee requires that the optimal arm be the most effective dueling comparator for every suboptimal arm; outside that case the paper's simplified matching statement is not valid.
  • The regret formula gives a principled way to set the cost weight $\alpha$: it trades off the reward gap and the dueling gap for each arm through the maximum in the denominator.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The decomposition-and-randomize scheme is likely transferable to contextual or linear bandits where a scalar reward model and a preference oracle coexist: the same threshold logic could decide which signal explores which candidate, though the paper does not analyze that setting.
  • The lower-bound structure suggests an information-accounting view: reward and dueling samples are additive evidence for identifying an arm, so a learner should allocate each arm's exploration budget to whichever signal has the better regret-per-information ratio; this is an editorial rephrasing, not a formal theorem of the paper.
  • A concrete testable extension is to modify DecoFusion to duel against the empirically most informative comparator rather than the estimated optimal arm; on instances where $\ell_k^*\neq 1$ such a variant should close the gap to the general lower bound.
  • The constant-regret phenomenon at $\alpha=0$ or $\alpha=1$ suggests that adding a free comparison channel to a purely reward-based bandit can eliminate the logarithmic factor altogether, which is an implication the paper states for its model and that invites experiments in reward-plus-preference systems.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper introduces the dueling-reward multi-armed bandit (DR-MAB), in which each round the learner obtains both a reward for one arm and the outcome of a duel between two arms, and regret is a weighted combination of reward regret and dueling regret. It derives an information-theoretic lower bound (Theorem 2.3) and a simplified lower bound (Corollary 2.4) under the assumption that the most effective dueling competitor of every suboptimal arm is the optimal arm. It then proposes two fusion algorithms: ElimFusion, an elimination method that shares a candidate arm set across feedback types, and DecoFusion, a decomposition method that randomly assigns each arm to reward or dueling exploration. The main theoretical claim is that DecoFusion matches the simplified lower bound up to a constant factor, with additional constant-regret behavior at α=0 and α=1. Experiments compare the algorithms against no-fusion baselines.

Significance. If the results hold, the paper contributes a new model that combines absolute and relative feedback, a lower bound showing the potential of fusion, and algorithms with logarithmic regret that improve over using either feedback alone. The algorithmic idea behind DecoFusion, especially the randomized decision-making that separates exploration and exploitation while balancing two feedback types, is novel and likely to be useful in similar hybrid-feedback settings. The paper is also careful to acknowledge in Section 4.3 that the near-optimality guarantee is conditional on the structural assumption ℓ*_k=1 for all suboptimal arms. The experimental study supports the qualitative claims, although it is limited to synthetic instances. The central caveat is that the matching-to-lower-bound claim is not established for general DR-MAB instances, and the manuscript should make this scope restriction more prominent.

major comments (1)
  1. [Theorem 4.1 and Eq. (5), Section 4.3] The regret bound in Eq. (5) is not well-defined at α=0 and α=1 because it divides by α^2 and (1−α)^2, and the randomization threshold β=α^2/(α^2+(1−α)^2) degenerates at these endpoints. The paper advertises constant (T-independent) regret for α=0 and α=1 as a distinctive feature of DecoFusion, but the stated theorem does not rigorously cover these cases. The authors should either state Theorem 4.1 for α∈(0,1) and provide a separate argument for the endpoints, or adapt the proof with a limiting/separate analysis so that the endpoint claim is formally supported.
minor comments (5)
  1. [Algorithm 2, lines 16-20] The pseudo-code for dueling exploration has a dangling else: the 'if' on line 17 is closed by 'end if' on line 19, leaving the 'else' on line 20 syntactically unattached. The intended logic should be written as an if-else block with a single end if.
  2. [Appendix D.2, Step 3] The notation E[R_T | A_t] and E[R_T | A_t^c] in the regret decomposition is imprecise, since R_T is the cumulative regret over all T rounds and A_t is a per-round event. The intended expressions are sums over t of E[R_t 1{A_t}] and E[R_t 1{A_t^c}], where R_t is the per-round regret; this should be clarified to make the proof easier to follow.
  3. [Abstract and Section 1] The phrase 'only the smaller among the reward and dueling-based regret for each individual arm' is informal and could be misread as applying without the min over dueling competitors. It would be helpful to state in the abstract that the dueling side of the lower bound is the best over competing arms ℓ<k, or to refer the reader to Theorem 2.3.
  4. [Section 3.1] There is a typo on line 4 of the algorithm description: 'warm-up phase (detained in Algorithm 3)' should read 'detailed in Algorithm 3'.
  5. [Appendix D.1 and D.2] In a few places the text writes 'LLM' or 'LLN' where the intended phrase is 'law of large numbers'; for instance, 'maximal LLM' appears in the proof of Lemma 2.2 and should be 'maximal LLN'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the lower and upper bounds are derived independently, and the ell*_k=1 assumption is a stated scope restriction rather than a circular input.

full rationale

The paper's central claims are self-contained rather than circular. The general lower bound (Theorem 2.3, Eq. 2) follows from a change-of-measure argument (Lemma 2.2) using Lai & Robbins and Komiyama et al., minimizing an LP over sampling counts; this derivation does not presuppose any particular algorithm. The simplified lower bound (Corollary 2.4, Eq. 3) is an explicit specialization to the case ell*_k=1, acknowledged in the text: 'by assuming ell*_k = 1...'. The upper bound (Theorem 4.1, Eqs. 5-6) is proved independently through concentration events, a randomized exploration rule with threshold beta = alpha^2/(alpha^2+(1-alpha)^2), and bounds on sampling counts N_{k,T} and M_{k,1,T}; it is not fitted to the lower bound. Eq. (6) and Eq. (3) share only the same stated assumption, which is a scope restriction rather than an input-output identity. No parameter is fitted to data and renamed a prediction, and no load-bearing theorem is imported from the authors' own prior work as an unverified premise. The paper honestly discloses that without ell*_k=1 the upper bound is worse than the general lower bound ('Without this condition, the regret upper bounds in Theorem 4.1 are worse than the general lower bound in (2)'), confirming that the matching claim is conditional rather than definitionally forced. Citations to Komiyama et al. and Honda & Takemura are external algorithmic techniques used as building blocks, which is normal use of prior results and does not constitute circularity. The reader's concern about the ell*_k=1 assumption is a correctness/scope issue, not a circularity issue; the paper never claims the simplified matching holds beyond that assumption.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

No new physical or ontological entities are introduced. The central claim rests on the problem formulation, the ordering consistency between reward and dueling probabilities, the Bernoulli observation model, and the simplifying assumption that arm 1 is the most effective dueling partner for every suboptimal arm. The algorithmic parameters f(K), delta, epsilon, and beta are hand-chosen and affect constants or lower-order terms, but they are not fitted to data.

free parameters (4)
  • Exploration function f(K) = c K^(1+xi) = c > 0 instance-dependent, not specified; xi > 0 small
    DecoFusion's exploration budget. The theorem requires e^{cK - f(K)} to vanish, but no explicit c is given, and the regret bound in Eq. (5) depends on f(K).
  • Confidence parameter delta in ElimFusion = 1 / (K^2 T^2)
    Hand-chosen confidence schedule for elimination. It affects constants in Theorem 3.1 and is not derived from data.
  • Epsilon in Theorem 4.1 = sufficiently small positive constant
    Appears in (1+epsilon) log T and in the additive O(epsilon^{-2}) term. It is a hand-chosen trade-off parameter for the regret analysis.
  • Randomization threshold beta = alpha^2 / (alpha^2 + (1-alpha)^2) = Determined by alpha
    Chosen by derivation in Section 4.2.2 to compensate for the mismatch between constructed and optimal arm decompositions. It is not fitted to data, but it is a hand-set algorithmic parameter.
assumptions (6)
  • domain assumption Reward means and dueling probabilities are consistent in ordering: nu_{k,ell} > 0.5 if mu_k > mu_ell and nu_{k,ell} = 0.5 if mu_k = mu_ell.
    Stated in Section 2. It couples the two feedback types and ensures arm 1 is both the best arm and the Condorcet winner.
  • domain assumption All reward means are distinct and arms are labeled so that mu_1 > mu_2 > ... > mu_K.
    Section 2. Removes ties and fixes the optimal arm without loss of thematic generality.
  • domain assumption Reward and dueling observations are Bernoulli and independent across rounds and arms.
    Section 2. The analysis uses KL divergences for Bernoulli distributions and independence for change-of-measure arguments.
  • domain assumption The most effective dueling arm ell*_k equals the optimal arm 1 for every suboptimal arm.
    Used in Corollary 2.4 and Section 4.3 to simplify the lower bound and obtain the matching upper bound. It is acknowledged as a common assumption from dueling bandit literature, but it restricts the class of instances for which near-optimality is shown.
  • standard math kl(p,q) = Theta((p-q)^2) for Bernoulli distributions.
    Used to convert KL-based bounds into gap-dependent regret expressions such as Eq. (6) and Corollary 2.4.
  • standard math Consistency definition of Lai and Robbins generalizes to DR-MAB.
    Definition 2.1. It is the standard asymptotic framework used to prove regret lower bounds for consistent algorithms.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fusing Reward and Dueling Feedback in Stochastic Bandits." pith.science (2026). https://pith.science/paper/2QQEJ4B3

@misc{pith2026250415812,
  author       = {Pith},
  title        = {Pith review of: Fusing Reward and Dueling Feedback in Stochastic Bandits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2QQEJ4B3}},
  note         = {Machine review of arXiv:2504.15812}
}
read the original abstract

This paper investigates the fusion of absolute (reward) and relative (dueling) feedback in stochastic bandits, where both feedback types are gathered in each decision round. We derive a regret lower bound, demonstrating that an efficient algorithm may incur only the smaller among the reward and dueling-based regret for each individual arm. We propose two fusion approaches: (1) a simple elimination fusion algorithm that leverages both feedback types to explore all arms and unifies collected information by sharing a common candidate arm set, and (2) a decomposition fusion algorithm that selects the more effective feedback to explore the corresponding arms and randomly assigns one feedback type for exploration and the other for exploitation in each round. The elimination fusion experiences a suboptimal multiplicative term of the number of arms in regret due to the intrinsic suboptimality of dueling elimination. In contrast, the decomposition fusion achieves regret matching the lower bound up to a constant under a common assumption. Extensive experiments confirm the efficacy of our algorithms and theoretical results.

Figures

Figures reproduced from arXiv: 2504.15812 by the authors.

Figure 1
Figure 1. Impact of parameter α on regret RT signing algorithms that can effectively leverage relative feedback has become a critical research topic in the online learning community, e.g., reinforcement learning with human feedback (RLHF) (Wang et al., 2023a; Xiong et al., 2024), bandits with human preferences (Ji et al., 2023), dueling bandits (Yan et al., 2022; Saha & Gaillard, 2022), etc. This line of work complements the … view at source ↗
Figure 2
Figure 2. Regret comparison in different settings ˆk (R) t and ˆk (D) t from both feedback types are exactly the optimal arm 1, i.e., PT t=1 E[ ˆk (R) t = ˆk (D) t = 1] = T − O(1), and (ii) under the first claim, the regret of the randomized decision-making policy is upper bounded by the last regret term in (5). While the claim (i) is proved via similar techniques in Honda & Takemura (2010) and Komiyama et al. (2015), the pro… view at source ↗
Figure 3
Figure 3. Regret comparison in different settings (continue) [PITH_FULL_IMAGE:figures/full_fig_p031_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 17 canonical work pages

  1. [1]

    Improved algorithms for linear stochastic bandits

    Abbasi-Yadkori, Y., P \'a l, D., and Szepesv \'a ri, C. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24, 2011

    2011

  2. [2]

    Reducing dueling bandits to cardinal bandits

    Ailon, N., Karnin, Z., and Joachims, T. Reducing dueling bandits to cardinal bandits. In International Conference on Machine Learning, pp.\ 856--864. PMLR, 2014

    2014

  3. [3]

    Using confidence bounds for exploitation-exploration trade-offs

    Auer, P. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3 0 (Nov): 0 397--422, 2002

    2002

  4. [4]

    and Ortner, R

    Auer, P. and Ortner, R. Ucb revisited: Improved regret bounds for the stochastic multi-armed bandit problem. Periodica Mathematica Hungarica, 61 0 (1-2): 0 55--65, 2010

    work page 2010

  5. [5]

    Auer, P., Cesa-Bianchi, N., Freund, Y., and Schapire, R. E. The nonstochastic multiarmed bandit problem. SIAM journal on computing, 32 0 (1): 0 48--77, 2002

    work page 2002

  6. [6]

    Online Learning and Bandits with Queried Hints

    Bhaskara, A., Gollapudi, S., Im, S., Kollias, K., and Munagala, K. Online learning and bandits with queried hints. arXiv preprint arXiv:2211.02703, 2022

    work page Pith review arXiv 2022

  7. [7]

    Bandits games and clustering foundations

    Bubeck, S. Bandits games and clustering foundations. PhD thesis, Universit \'e des Sciences et Technologie de Lille-Lille I, 2010

    work page 2010

  8. [8]

    Regret analysis of stochastic and nonstochastic multi-armed bandit problems

    Bubeck, S., Cesa-Bianchi, N., et al. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning , 5 0 (1): 0 1--122, 2012

    work page 2012

Show all 37 references
  1. [9]

    Kullback-leibler upper confidence bounds for optimal sequential allocation

    Capp \'e , O., Garivier, A., Maillard, O.-A., Munos, R., and Stoltz, G. Kullback-leibler upper confidence bounds for optimal sequential allocation. The Annals of Statistics, pp.\ 1516--1541, 2013

    2013

  2. [10]

    Combinatorial multi-armed bandit: General framework and applications

    Chen, W., Wang, Y., and Yuan, Y. Combinatorial multi-armed bandit: General framework and applications. In International conference on machine learning, pp.\ 151--159. PMLR, 2013

    2013

  3. [11]

    Leveraging initial hints for free in stochastic linear bandits

    Cutkosky, A., Dann, C., Das, A., and Zhang, Q. Leveraging initial hints for free in stochastic linear bandits. In International Conference on Algorithmic Learning Theory, pp.\ 282--318. PMLR, 2022

    2022

  4. [12]

    and Takemura, A

    Honda, J. and Takemura, A. An asymptotically optimal bandit algorithm for bounded support models. In COLT, pp.\ 67--79. Citeseer, 2010

    2010

  5. [13]

    Provable benefits of policy learning from human preferences in contextual bandit problems

    Ji, X., Wang, H., Chen, M., Zhao, T., and Wang, M. Provable benefits of policy learning from human preferences in contextual bandit problems. arXiv preprint arXiv:2307.12975, 2023

    2023 arXiv

  6. [14]

    Regret lower bound and optimal algorithm in dueling bandit problem

    Komiyama, J., Honda, J., Kashima, H., and Nakagawa, H. Regret lower bound and optimal algorithm in dueling bandit problem. In Conference on learning theory, pp.\ 1141--1154. PMLR, 2015

    2015

  7. [15]

    Lai, T. L. and Robbins, H. Asymptotically efficient adaptive allocation rules. Advances in applied mathematics, 6 0 (1): 0 4--22, 1985

    1985

  8. [16]

    and Szepesv \'a ri, C

    Lattimore, T. and Szepesv \'a ri, C. Bandit algorithms. Cambridge University Press, 2020

    2020

  9. [17]

    Li, L., Chu, W., Langford, J., and Schapire, R. E. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pp.\ 661--670, 2010

    2010

  10. [18]

    Fedconpe: Efficient federated conversational bandits with heterogeneous clients

    Li, Z., Liu, M., and Lui, J. Fedconpe: Efficient federated conversational bandits with heterogeneous clients. arXiv preprint arXiv:2405.02881, 2024

    2024 arXiv

  11. [19]

    Predictive bandits

    Lindst hl, S., Proutiere, A., and Johnsson, A. Predictive bandits. In 2020 59th IEEE Conference on Decision and Control (CDC), pp.\ 1170--1176. IEEE, 2020

    2020

  12. [20]

    A modern introduction to online learning

    Orabona, F. A modern introduction to online learning. arXiv preprint arXiv:1912.13213, 2019

    1912 arXiv

  13. [21]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  14. [22]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  15. [23]

    and Gaillard, P

    Saha, A. and Gaillard, P. Versatile dueling bandits: Best-of-both world analyses for learning from relative preferences. In International Conference on Machine Learning, pp.\ 19011--19026. PMLR, 2022

    2022

  16. [24]

    Adversarial dueling bandits

    Saha, A., Koren, T., and Mansour, Y. Adversarial dueling bandits. In International Conference on Machine Learning, pp.\ 9235--9244. PMLR, 2021

    2021

  17. [25]

    Introduction to multi-armed bandits

    Slivkins, A. Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning, 12 0 (1-2): 0 1--286, 2019. ISSN 1935-8237. doi:10.1561/2200000068. URL http://dx.doi.org/10.1561/2200000068

    2019 doi

  18. [26]

    Advancements in dueling bandits

    Sui, Y., Zoghi, M., Hofmann, K., and Yue, Y. Advancements in dueling bandits. In IJCAI, pp.\ 5502--5510, 2018

    2018

  19. [27]

    S., Barto, A

    Sutton, R. S., Barto, A. G., et al. Introduction to reinforcement learning, volume 135. MIT press Cambridge, 1998

    1998

  20. [28]

    Algorithms for reinforcement learning

    Szepesv \'a ri, C. Algorithms for reinforcement learning. Springer nature, 2022

    2022

  21. [29]

    Generic exploration and k-armed voting bandits

    Urvoy, T., Clerot, F., F \'e raud, R., and Naamane, S. Generic exploration and k-armed voting bandits. In International conference on machine learning, pp.\ 91--99. PMLR, 2013

    2013

  22. [30]

    Is rlhf more difficult than standard rl? arXiv preprint arXiv:2306.14111, 2023 a

    Wang, Y., Liu, Q., and Jin, C. Is rlhf more difficult than standard rl? arXiv preprint arXiv:2306.14111, 2023 a

    2023 arXiv

  23. [31]

    Wang, Z., Liu, X., Li, S., and Lui, J. C. Efficient explorative key-term selection strategies for conversational contextual bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.\ 10288--10295, 2023 b

    2023

  24. [32]

    Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint

    Xiong, W., Dong, H., Ye, C., Wang, Z., Zhong, H., Ji, H., Jiang, N., and Zhang, T. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint. In Forty-first International Conference on Machine Learning, 2024

    2024

  25. [33]

    L., Craswell, N., Voorhees, E

    Yan, X., Luo, C., Clarke, C. L., Craswell, N., Voorhees, E. M., and Castells, P. Human preferences as dueling bandits. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, pp.\ 567--577, 2022

    2022

  26. [34]

    The k-armed dueling bandits problem

    Yue, Y., Broder, J., Kleinberg, R., and Joachims, T. The k-armed dueling bandits problem. Journal of Computer and System Sciences, 78 0 (5): 0 1538--1556, 2012

    2012

  27. [35]

    Multi-armed bandit with additional observations

    Yun, D., Proutiere, A., Ahn, S., Shin, J., and Yi, Y. Multi-armed bandit with additional observations. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 2 0 (1): 0 1--22, 2018

    2018

  28. [36]

    Conversational contextual bandit: Algorithm and application

    Zhang, X., Xie, H., Li, H., and CS Lui, J. Conversational contextual bandit: Algorithm and application. In Proceedings of the web conference 2020, pp.\ 662--672, 2020

    2020

  29. [37]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.