REVIEW 5 cited by
Local Policy Improvement for Recommender Systems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recommender systems predict what items a user will interact with next, based on their past interactions. The problem is often approached through supervised learning, but recent advancements have shifted towards policy optimization of rewards (e.g., user engagement). One challenge with the latter is policy mismatch: we are only able to train a new policy given data collected from a previously-deployed policy. The conventional way to address this problem is through importance sampling correction, but this comes with practical limitations. We suggest an alternative approach of local policy improvement without off-policy correction. Our method computes and optimizes a lower bound of expected reward of the target policy, which is easy to estimate from data and does not involve density ratios (such as those appearing in importance sampling correction). This local policy improvement paradigm is ideal for recommender systems, as previous policies are typically of decent quality and policies are updated frequently. We provide empirical evidence and practical recipes for applying our technique in a sequential recommendation setting.
Forward citations
Cited by 5 Pith papers
-
Off-Policy Evaluation and Learning for Matching Markets
DiPS and DPR are new OPE estimators for matching markets that exploit the two-stage reward structure to reduce variance while controlling bias.
-
Off-Policy Evaluation and Learning for the Future under Non-Stationarity
A new importance-weighted estimator, OPFV, estimates and optimizes future policy value in non-stationary bandit environments by leveraging recurring time features in historical logs.
-
Optimization of Epsilon-Greedy Exploration
A gradient-based framework tunes epsilon-greedy exploration schedules by minimizing Bayesian regret, matching or beating heuristics in batched recommendation benchmarks.
-
Exponential Reward Weighting for Fine-Tuning Generative Recommenders under Sparse and Noisy Feedback
Exponential reward weighting with a tuned temperature improves offline generative recommenders, and a new theory decomposes its suboptimality into coverage and noise costs that predict the observed inverted-U in performance.
-
Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation
Reward-weighted log-likelihood objectives outperform complex off-policy estimators in large action spaces because their optimization landscapes are much easier to navigate.
Discussion (0). Continue with ORCID to comment.