REVIEW 4 cited by
Off-Policy Evaluation for Large Action Spaces via Embeddings
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Off-policy evaluation (OPE) in contextual bandits has seen rapid adoption in real-world systems, since it enables offline evaluation of new policies using only historic log data. Unfortunately, when the number of actions is large, existing OPE estimators -- most of which are based on inverse propensity score weighting -- degrade severely and can suffer from extreme bias and variance. This foils the use of OPE in many applications from recommender systems to language models. To overcome this issue, we propose a new OPE estimator that leverages marginalized importance weights when action embeddings provide structure in the action space. We characterize the bias, variance, and mean squared error of the proposed estimator and analyze the conditions under which the action embedding provides statistical benefits over conventional estimators. In addition to the theoretical analysis, we find that the empirical performance improvement can be substantial, enabling reliable OPE even when existing estimators collapse due to a large number of actions.
Forward citations
Cited by 4 Pith papers
-
Off-Policy Evaluation and Learning for Matching Markets
DiPS and DPR are new OPE estimators for matching markets that exploit the two-stage reward structure to reduce variance while controlling bias.
-
Off-policy estimation with adaptively collected data: the power of online learning
AIPW estimators with nuisance estimates from no-regret online learning attain near-optimal finite-sample MSE for off-policy evaluation with adaptively collected data.
-
Document Similarity Enhanced IPS Estimation for Unbiased Learning to Rank
IPSsim adds a similarity-to-top-documents term to the IPS weight and achieves small NDCG improvements in large top-n settings on simulated click benchmarks.
-
Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation
Reward-weighted log-likelihood objectives outperform complex off-policy estimators in large action spaces because their optimization landscapes are much easier to navigate.
Discussion (0). Continue with ORCID to comment.