REVIEW 3 major objections 5 minor 22 references
Decision-Making Under Complete Uncertainty: You Will Regret Not Being Greedy
T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper proves that when every product is observed the same number of times, the greedy strategy of picking the highest average rating is minimax-optimal for worst-case regret, with regret shrinking to zero as observations grow.
desk verdict The minimax results for the greedy rule are real, but the advertised asymptotic separation from Thompson Sampling rests on a faulty argument. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the observation matrix $B$, whose columns are constrained to sum to $m$ (equal observations per product), and the strategy $\sigma_g$ that maps $B$ to the product(s) maximizing the observed average $V_B(d)$. The argument runs through the worst-case regret functional $\gamma(\sigma)=\max_S[\max_d V_S(d)-\pi(\sigma,S)]$, the two-state adversarial construction in Proposition 2 (states $S_1$ and $S_2$ that force any tie-breaking probability $p$ to incur regret at least $\frac{1}{8}$), and a Hoeffding concentration bound on the gap between the top two products' empirical sums to prove the limit result. For Thompson Sampling, the machinery is the Beta-Binomial distribution of sampled posterior ratings; the paper integrates $P(X<Y)$ to derive its expected regret.
What would settle it
For a fixed two-product state with rating probabilities $(p_1,p_2)$, compute the expected Thompson Sampling regret exactly for increasing $m$ by summing the closed-form Beta-Binomial expression over all observation matrices; if the expectation decays to $0$ rather than approaching a positive constant, Proposition 5 fails. Separately, run greedy on data where one product has many more reviews than another and record how often it selects the product with the lower true average; a high mis-selection rate would confirm the equal-sample-size limit of the theorem.
Extended reading notes
Core claim
The central discovery is that, in a zero-sum game against Nature where Nature chooses any column-stochastic rating matrix and the decision maker only sees an observation matrix with exactly $m$ independent ratings per product, the greedy strategy $\sigma_g$ is minimax-optimal for worst-case regret $\gamma(\sigma) = \max_S (\max_d V_S(d) - \pi(\sigma,S))$. For two products, two ratings, and $m=1$, the paper proves $\gamma(\sigma_g)=\frac{1}{8}$ (Proposition 1) and proves a matching lower bound for all strategies (Proposition 2), so no decision rule can do better in the worst case. It then shows numerically that $\gamma$ decreases with $m$ up to 20 and proves via a Hoeffding bound (Theorem 4) that $\gamma \to 0$ as $m \to \infty$, with an explicit sample size sufficient for zero regret with probability $1-\delta$. Finally, Proposition 5 claims that for two products and two ratings, Thompson Sampling's regret does not vanish as $m \to \infty$, whereas greedy's does.
Load-bearing premise
The whole theory requires that every product is observed exactly the same number of times $m$; if observation counts differ, the paper's own example shows the greedy rule can pick a product whose true value is far below another, and none of the minimax or convergence guarantees apply.
Editorial extensions
If this is right
- For the binary two-product case with one observation per product, the greedy rule guarantees regret at most $\frac{1}{8}$, and no strategy can guarantee less; a decision maker who only sees one rating per option is assured of capturing at least $\frac{7}{8}$ of the value gap.
- As the number of observations per product grows, greedy's worst-case regret tends to $0$; Theorem 4 gives an explicit $m$ needed to achieve zero regret with probability $1-\delta$.
- The greedy rule's worst-case regret decreases over the range $m=1,\dots,20$ in the binary case, and the empirical study on 1.5 million Google reviews confirms decreasing regret with more reviews across up to 10 products.
- Thompson Sampling, despite being a standard Bayesian bandit method, is claimed to incur positive regret even in the infinite-observation limit, making it worse than greedy in this worst-case one-shot sense.
- For settings with equal observation counts, a quality-control agent can use $m$ and the rating gap to bound the maximum regret of picking the batch with the highest observed average quality.
Reading between the lines
- If the equal-sample-size condition is dropped, the greedy rule can be arbitrarily bad (as the paper's Section 6 example shows), so the practical guidance 'always pick the highest average' should be read as conditional on comparable sample sizes; a confidence-aware variant such as a lower-confidence bound would be the natural extension.
- The minimax optimality result is proven only for the $m=1$, two-product, two-rating case; extending the lower-bound technique to larger $m$ and more products would be needed before claiming the greedy rule is minimax-optimal generally.
- The paper's asymptotic comparison with Thompson Sampling, if correct, would reverse the usual bandit wisdom that greedy starves exploration; a fruitful test is whether the conclusion survives with unequal observation counts, where Thompson Sampling's built-in uncertainty adjustment may matter.
- Because the theorem's zero-regret guarantee is stated with probability $1-\delta$, it is a high-probability statement, not an almost-sure guarantee; a decision maker who needs certainty would still need the explicit sample size formula rather than relying on the limit.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper analyzes a one-shot decision problem in which Nature chooses an arbitrary per-product rating distribution, the decision-maker observes exactly m ratings for each of nd products, and a strategy maps the observation matrix to a product. The authors define worst-case expected regret, prove that for nd = nr = 2 and m = 1 the greedy strategy achieves worst-case regret 1/8 and that no strategy can do better (Propositions 1 and 2), numerically show that this value decreases for m up to 20, state an asymptotic zero-regret theorem (Theorem 4), and claim that greedy outperforms Thompson Sampling even as m goes to infinity (Proposition 5). They also simulate the strategy on Google restaurant reviews data and report that greedy has lower regret than uniform and Thompson Sampling in finite samples.
Significance. The exact minimax constant 1/8 is a clean and correct result, and the equal-observation model is internally coherent. Propositions 1 and 2 are self-contained and their computations check out. The numerical decay and the empirical setup are plausible. However, the advertised asymptotic separation from Thompson Sampling is not established: Proposition 5's proof is invalid in the way identified by the stress-test note, and once that claim is removed the central theoretical contribution is a minimax analysis of greedy under a restrictive equal-sample-size assumption. With a repaired uniform proof for Theorem 4, the worst-case convergence result would still be a useful contribution, but the paper's current abstract substantially overstates the results.
major comments (3)
- [§4.5, Proposition 5] The asymptotic argument is invalid. The proof fixes a generic observation matrix B' with b > d and argues that P(X < Y) > 0 for some a,b,c,d > 0, then takes m to infinity and concludes that Thompson Sampling incurs positive regret. But a,b,c,d are m-dependent random counts: under the state S' with p1 < p2, the matrices that dominate the probability mass have b ≈ m(1-p1), d ≈ m(1-p2), a ≈ m p1, c ≈ m p2, and for these matrices the Beta posteriors concentrate, so P(X < Y) tends to 0 exponentially. The limiting regret is not a single term P(X < Y)(p2 - p1) evaluated at an arbitrary matrix; it is a sum over the full distribution of the observation matrix. A correct analysis shows that the Thompson Sampling regret also converges to 0 as m grows. Therefore the claimed asymptotic separation in the Abstract, in Results Overview item 5, and in the last paragraph of Section 4.5 is unsupported and, as stated, false.
- [§4.4, Theorem 4] The proof is not uniform over states. The final bound nd exp(-m(VS*(d1) - VS*(d2))^2 / (2(nr-1)^2)) depends on the state-specific gap VS*(d1) - VS*(d2), and the sentence 'WLOG we can pick a single state S*' does not justify exchanging the supremum over states with a pointwise argument. The worst-case state can change with m, and gaps arbitrarily close to zero are allowed in the state space. To prove γ(σg) → 0 one must either optimize the product of the gap and the misclassification probability or give a uniform concentration argument; the proof as written only establishes pointwise convergence for each fixed state. The claim may be salvageable, but the current proof is not sufficient for the stated worst-case result.
- [§5, Simulation Result 6] The empirical comparison to Thompson Sampling is currently the only remaining support for the claim that greedy outperforms Thompson Sampling, because the theoretical comparison in Proposition 5 is invalid. The simulation uses a smoothing constant for zero-count Beta parameters that is never specified (footnote 3), and it reports point estimates without confidence intervals or other variability measures, so the finite-sample superiority of greedy over Thompson Sampling is not yet established as a robust finding. The full-data average is also used both to define the ground-truth state and to evaluate regret, which makes the simulation a test of agreement with an empirical proxy rather than with the true state; the paper should state this limitation explicitly and, ideally, add a sensitivity analysis for the smoothing value.
minor comments (5)
- [§4.1, Proposition 1 proof] The displayed formula for π(σg, S') appears to contain a typo; based on the regret formula it should be 2 - p1/2 - p2/2 + (p1 - p2)^2/2, not 2 - p1 + p2/2 + (p1 - p2)^2/2. The regret calculation itself is consistent.
- [Theorem 4 statement] The statement 'the greedy strategy has a regret of 0 with probability 1 - δ' conflates the random event of selecting the best product with the expected regret; the paper should either define a random regret or rephrase the claim as 'picks the best product with probability at least 1 - δ'.
- [§1.1 and Abstract] The claim that greedy is 'optimal' should be qualified as holding for two products, two ratings, and one observation per product; as written, the Abstract and Results Overview overstate the scope of Proposition 2.
- [§4.5, footnote 4] The footnote says Proposition 5 can be generalized to any number of ratings and products by replacing Beta with Dirichlet and taking a union bound, but no proof is given; either provide the proof or remove the assertion.
- [Figure 2 and Appendix A] The figure would benefit from confidence bands or standard errors, since the tables show small differences between greedy and Thompson Sampling in several cells; the tables themselves are helpful and should remain.
Circularity Check
No load-bearing circularity: the minimax regret derivation is self-contained; the only author self-citation is a passing related-work remark.
full rationale
The core derivation chain—Propositions 1, 2, and Theorem 4—starts from the model definitions in Section 2 (states as column-stochastic matrices, observation matrices with m observations per product, greedy as argmax of observed averages) and obtains worst-case regret bounds by direct calculation, Hoeffding inequalities, and a two-state contradiction. No fitted parameter is introduced and then renamed as a prediction; the worst-case regret is maximized over all states, not calibrated to any dataset. The empirical section uses the full Google-reviews averages as ground truth and evaluates regret of strategies on subsamples; this is an evaluation protocol, not a derivation of the theoretical claims. The only self-citation is the passing remark in Related Work that 'Ismail [9] proposes the optimin criterion which coincides with Wald’s maximin criterion in zero-sum games.' That remark is not used in any proof or model assumption, so it is not load-bearing. The paper itself flags the central modeling limitation in Section 6 (equal number of observations per product), and the reviewer-identified flaw in Proposition 5—that a, b, c, d are m-dependent so P(X<Y) cannot be evaluated as a fixed positive constant in the m→∞ limit—is a mathematical correctness issue, not a circularity: it does not reduce the claimed conclusion to its own inputs by definition or by self-citation. Accordingly, the paper has no significant circularity; the score reflects only the presence of a minor, non-load-bearing self-citation.
Assumptions & free parameters
free parameters (1)
- Thompson Sampling zero-count smoothing value =
unspecified (described only as 'small positive values')
assumptions (6)
- domain assumption Ratings for each product are i.i.d. draws from a fixed multinomial distribution; draws are independent across products (Section 2.2).
- standard math Nature is adversarial and the DM's objective is worst-case regret over all column-stochastic states S (Definition 1).
- domain assumption Every product is observed exactly m times, with equal m across products (Section 2.2, footnote 1).
- domain assumption Payoff is the expected rating, with ratings valued at their integer labels 1 through nr (Section 2.4).
- standard math Hoeffding's inequality applies to sums of independent bounded ratings (used in Theorem 4 and Proposition 5).
- ad hoc to paper In the empirical study, the full-data average rating equals the true state value V_S(d) (Section 5, step 1).
Cite this review
Pith. "Pith review of Decision-Making Under Complete Uncertainty: You Will Regret Not Being Greedy." pith.science (2026). https://pith.science/paper/5RBRVZUS
@misc{pith2026250207593,
author = {Pith},
title = {Pith review of: Decision-Making Under Complete Uncertainty: You Will Regret Not Being Greedy},
year = {2026},
howpublished = {\url{https://pith.science/paper/5RBRVZUS}},
note = {Machine review of arXiv:2502.07593}
}
read the original abstract
In this paper, we propose a probabilistic game-theoretic model to study the properties of the worst-case regret of the greedy strategy under complete (Knightian) uncertainty. In a game between a decision-maker (DM) and an adversarial agent (Nature), the DM observes a realization of product ratings for each product. Upon observation, the DM chooses a strategy, which is a function from the set of observations to the set of products. We study the theoretical properties, including the worst-case regret of the greedy strategy that chooses the product with the highest observed average rating. We prove that, with respect to the worst-case regret, the greedy strategy is optimal and that, in the limit, the regret of the greedy strategy converges to zero. We validate the model on data collected from Google reviews for restaurants, showing that the greedy strategy not only performs according to the theoretical findings but also outperforms the uniform strategy and the Thompson Sampling algorithm.
Figures
Reference graph
Works this paper leans on
-
[1]
Daron Anderson and Douglas J. Leith. Expected regret and pseudo-regret are equivalent when the optimal arm is unique. Journal of Machine Learning Research, 23(293):1–12, 2022
work page 2022
-
[2]
Finite-time analysis of the multiarmed bandit problem.Machine Learning, 47:235–256, 05 2002
Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem.Machine Learning, 47:235–256, 05 2002
work page 2002
-
[3]
From opti- mization to regret minimization and back again
Ioannis Avramopoulos, Jennifer Rexford, and Robert Schapire. From opti- mization to regret minimization and back again. 01 2008
work page 2008
-
[4]
Jørgen Bang-Jensen, G. Gutin, and A. Yeo. When the greedy algorithm fails. Discrete Optimization, 1:121–127, 2004
work page 2004
-
[5]
The unreasonable effectiveness of greedy algorithms in multi-armed ban- dit with many arms, 2022
Mohsen Bayati, Nima Hamidi, Ramesh Johari, and Khashayar Khosravi. The unreasonable effectiveness of greedy algorithms in multi-armed ban- dit with many arms, 2022
work page 2022
-
[6]
Learning, Regret Minimization, and Equilibria, page 79–102
Avrim Blum and Yishay Mansour. Learning, Regret Minimization, and Equilibria, page 79–102. Cambridge University Press, 2007
work page 2007
-
[7]
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Greedy Algorithms IV-15; Introduction to Algorithms, 3rd Edition. MIT Press, 2009
work page 2009
-
[8]
Greedy when sure and conservative when uncertain about the opponents
Haobo Fu, Ye Tian, Hongxiang Yu, Weiming Liu, Shuang Wu, Jiechao Xiong, Ying Wen, Kai Li, Junliang Xing, Qiang Fu, and Wei Yang. Greedy when sure and conservative when uncertain about the opponents. In Ka- malika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on...
work page 2022
Show all 22 references
-
[9]
Mehmet S. Ismail. One for all, all for one—von neumann, wald, rawls, and pareto. In Proceedings of the 21st ACM Conference on Economics and Computation, EC ’20, page 763–764, New York, NY, USA, 2020. As- sociation for Computing Machinery
2020
-
[10]
Be greedy in multi-armed bandits, 2021
Matthieu Jedor, Jonathan Louëdec, and Vianney Perchet. Be greedy in multi-armed bandits, 2021
2021
-
[11]
Jiantao Jiao. Analysis of finite-arm i.i.d.-reward bandit; theory of multi- armed bandits and reinforcement learning, Department of Electrical En- gineering and Computer Sciences at University of California Berkeley , United States, 2021
2021
-
[12]
Regret mini- mization with noisy observations, 2022
Mohammad Mahdian, Jieming Mao, and Kangning Wang. Regret mini- mization with noisy observations, 2022. 22
2022
-
[13]
Empirical bernstein bounds and sample variance penalization
Andreas Maurer and Massimiliano Pontil. Empirical bernstein bounds and sample variance penalization. 2009
2009
-
[14]
Algorithmic game theory , University Of Pennsylvania, 2011
Ryan Rogers. Algorithmic game theory , University Of Pennsylvania, 2011
2011
-
[15]
No-Regret Dynamics 230-242; Twenty Lectures on Algorithmic Game Theory
Tim Roughgarden. No-Regret Dynamics 230-242; Twenty Lectures on Algorithmic Game Theory. Cambridge University Press, USA, 1st edition, 2016
2016
-
[16]
A tutorial on thompson sampling
Daniel Russo, Benjamin Van Roy , Abbas Kazerouni, and Ian Osband. A tutorial on thompson sampling. CoRR, abs/1707.02038, 2017
2017 arXiv
-
[17]
Leonard J. Savage. The theory of statistical decision. Journal of the American Statistical Association, 46:55–67, 1951
1951
-
[18]
Learning by convex combination
Karol Szwagrzak. Learning by convex combination. WorkingPaper 16- 2022, Copenhagen Business School [wp], Denmark, 2022
2022
-
[19]
Statistical Decision Functions
Abraham Wald. Statistical Decision Functions. Wiley: New York, 1950
1950
-
[20]
Review on greedy algorithm
Yizhun Wang. Review on greedy algorithm. Theoretical and Natural Science, 14:233–239, 11 2023
2023
-
[21]
Personalized showcases: Generating multi-modal explanations for recom- mendations, 2023
An Yan, Zhankui He, Jiacheng Li, Tianyang Zhang, and Julian McAuley . Personalized showcases: Generating multi-modal explanations for recom- mendations, 2023
2023
-
[22]
Multi-armed bandit with additional observations
Donggyu Yun, Alexandre Proutiere, Sumyeong Ahn, Jinwoo Shin, and Yung Yi. Multi-armed bandit with additional observations. Proc. ACM Meas. Anal. Comput. Syst., 2(1), apr 2018. 23 A Tabulated Results for Google Reviews Experiments Tabulated results are provided in this section ...
2018
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.