REVIEW 5 cited by
Taming Non-stationary Bandits: A Bayesian Approach
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We consider the multi armed bandit problem in non-stationary environments. Based on the Bayesian method, we propose a variant of Thompson Sampling which can be used in both rested and restless bandit scenarios. Applying discounting to the parameters of prior distribution, we describe a way to systematically reduce the effect of past observations. Further, we derive the exact expression for the probability of picking sub-optimal arms. By increasing the exploitative value of Bayes' samples, we also provide an optimistic version of the algorithm. Extensive empirical analysis is conducted under various scenarios to validate the utility of proposed algorithms. A comparison study with various state-of-the-arm algorithms is also included.
Forward citations
Cited by 5 Pith papers
-
Natural Policy Gradient for Average Reward Non-Stationary RL
A natural actor-critic algorithm for non-stationary average-reward MDPs achieves dynamic regret O~(sqrt(|S||A|) Delta_T^{1/6} T^{5/6}).
-
TS-Insight: Visualizing Thompson Sampling for Verification and XAI
A web-based visualization tool exposes Thompson Sampling's posterior draws, evidence counts, and selection history to support per-arm verification and step-level explanation.
-
Thompson Sampling-like Algorithms for Stochastic Rising Bandits
Thompson sampling with Beta or Gaussian priors and forced exploration achieves sublinear regret in stochastic rising rested bandits, with a new instance-complexity index sigma controlling the cost.
-
Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits
fcTS corrects and reweights historical observations via drift models for linear, periodic, and regime-switching non-stationarities in contextual bandits, outperforming forgetting baselines in structured cases.
-
Accelerated learning from recommender systems using multi-armed bandit
A Vrbo team used daily Thompson sampling to rank four recommendation models by click-through rate, but the A/B validation they report is for a previous campaign's winner, not the current one.
Discussion (0). Continue with ORCID to comment.