{"id":"e65bc43f-3dfc-4602-84a2-136486a43a0d","arxiv_id":"2512.09850","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Using conformal prediction intervals instead of Hoeffding bounds in UCB-style bandit policies can deliver nominal coverage with competitive regret in small-gap settings, at least in simulations and one portfolio backtest.","lead":"This paper replaces the usual statistical bounds in UCB bandit algorithms with conformal prediction intervals and tests the resulting 'Conformal Bandits' in small-gap simulations and a portfolio backtest. It claims these methods keep reward-coverage guarantees while lowering regret when arm differences are tiny, which matters for safe sequential decisions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (10)'s finite-sample coverage claim is unsupported: adaptive arm selection breaks exchangeability, and ACI only provides asymptotic coverage, so the central statistical-validity promise is not delivered.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the finite-sample coverage guarantee of Eq. (10) rests on exchangeability, which adaptive arm selection violates. The paper itself acknowledges this in Remark 4.1 and Section 7.1, admitting that the adopted ACI remedy only provides asymptotic coverage. This is not a hidden flaw but an overclaim of the abstract and Section 4.3, which present finite-sample coverage as a theorem-like property. The empirical coverage in Table 1 may be acceptable in practice, but it does not validate the theoretical promise. Since the central novelty is exactly this statistical validity, the manuscript should be revised to either clearly state that the coverage guarantee is asymptotic (via ACI) or provide a rigorous finite-sample guarantee under weaker exchangeability assumptions. The proposed concrete test would determine whether the exchangeability violation actually causes under-coverage in a simple adaptive setting, or whether the practical impact is negligible. The reader's CONDITIONAL verdict is appropriate: the claims should be restated to match the evidence or supported by further theory, so I recommend no change to the verdict.","tokens_in":25218,"tokens_out":6195,"duration_ms":359967,"concrete_test":"Implement a two-armed bandit where rewards for each arm are iid N(0,1), but the policy is deliberately adversarial to exchangeability: at each round t, pull arm 1 if the previous reward from arm 1 was positive, else pull arm 2. Construct split-CQR intervals (without ACI) for arm 1 from its observed rewards, with a fixed calibration size (e.g., 100 after a burn-in), and evaluate whether a fresh test reward from the arm's true distribution falls inside the interval. Repeat 10,000 times and compute empirical coverage. If coverage is significantly below 1-α (e.g., by more than 2 percentage points), the exchangeability violation directly breaks the finite-sample guarantee; if coverage remains at 1-α, the violation is benign in this instance, and the test should be repeated with heavier-tailed rewards or stronger selection bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central statistical claim, Eq. (10), asserts finite-sample coverage P(Y_{k,t+1} in C^{1-α}_{k,t+1}) ≥ 1-α for every arm and round. This inherits the exchangeability assumption of split conformal prediction. In a bandit, the observations available for arm k are a subsequence of the arm's reward stream selected at data-dependent stopping times; even if the underlying rewards are iid, the observed subsequence is generally not exchangeable because the decision to pull arm k at each round depends on past rewards of all arms. Remark 4.1 explicitly concedes that adaptive arm selection 'may compromise exchangeability over the joint distribution of arm selection and reward.' The paper then adopts Adaptive Conformal Inference [27], but ACI's formal guarantee is asymptotic coverage under distribution shift, not finite-sample exchangeability-based coverage, and its assumptions (e.g., bounded variation of the miscoverage rate) are not verified for bandit feedback. Section 7.1 similarly states ACI gives 'asymptotic coverage' only. Thus the abstract's promise of 'finite-sample prediction coverage' is not met by the provided theorems or assumptions; the coverage shown in Table 1 is an empirical property of ACI in the specific simulations, not a consequence of the finite-sample CP theory. Because the entire motivation for replacing Hoeffding bounds with CP is the finite-sample validity, this is the load-bearing gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a family of 'Conformal Bandits' that replace the Hoeffding-based exploration bonuses of UCB with conformal prediction intervals (specifically Conformalized Quantile Regression, CQR), together with risk-sensitive variants (CP-Bandit, CP-ESI) and a randomized version. It claims that this yields finite-sample prediction coverage for each arm's future reward and superior regret performance in the small-gap regime. The methodology is evaluated in Monte Carlo simulations for Gaussian, Student-t, and skewed-t rewards, and applied to a portfolio allocation problem with HMM-based regime detection.","tokens_in":25533,"tokens_out":6415,"duration_ms":59725,"significance":"If the central guarantee were proven, the paper would make a meaningful contribution by coupling decision-making with finite-sample distribution-free uncertainty quantification. The manuscript has concrete strengths: it provides reproducible code, a clear simulation design, a real-data case study, and an honest discussion of limitations (Remark 4.1, Section 7.1). However, the central finite-sample coverage guarantee is not established, and the regret-efficiency claim rests entirely on simulations. The paper's main theoretical assertion, Eq. (10), is not justified under adaptive arm selection, and the adopted ACI correction provides only asymptotic coverage. As a result, the significance of the contribution is presently much weaker than the abstract and conclusion suggest.","major_comments":[{"comment":"The statement that the intervals satisfy P(Y_{k,t+1} in C^{1-alpha}_{k,t+1}) >= 1-alpha for every arm and round is asserted 'in virtue of CP properties'. This is not valid for bandit feedback. Split conformal/CQR finite-sample coverage requires exchangeability of the observed calibration and test pairs. With an adaptive policy, the sequence of pulls for arm k is a subsequence selected at data-dependent stopping times; even with iid rewards, the resulting observed sequence need not be exchangeable. Remark 4.1 explicitly concedes that adaptivity 'may compromise exchangeability,' and Section 7.1 limits the adopted ACI correction to asymptotic coverage. Thus the abstract's 'finite-sample prediction coverage' is not established by the manuscript. The paper would need either a theorem for conformal inference under adaptive sampling, or clearly re-scoped claims of empirical calibration.","section":"Section 4.3, Eq. (10)"},{"comment":"The statement that all CP-Bandit variants exhibit 'uniformly superior learning efficiency compared to UCB1' in the small-gap regime is an empirical observation from Monte Carlo simulations for three distributions, not a theorem. Section 7.2 acknowledges that formal regret analysis is future work. Moreover, the comparison is limited to T=2000 and a fixed gap of 0.05; no sensitivity analysis over horizon or gap size is provided. The claim should be phrased as finite-horizon empirical performance, or supported by regret bounds.","section":"Section 5, Fig. 3 and Section 7.2"},{"comment":"The adoption of Adaptive Conformal Inference (ACI) is not a substitute for the finite-sample guarantee. ACI's validity is asymptotic under distribution shift and requires conditions (e.g., bounded variation of the miscoverage rate) that are not checked for bandit-dependent selection. The manuscript itself states in Section 7.1 that ACI guarantees asymptotic coverage. Therefore Eq. (10) cannot be read as a finite-sample or even asymptotic coverage theorem for the proposed bandit policy without additional analysis. A concrete test would be to report conditional coverage given arm-selection histories, or at least coverage on a held-out arm's observed pulls.","section":"Sections 4.2-4.3, Remark 4.1, Section 7.1"},{"comment":"Comparing UCB1's exploration intervals to nominal prediction coverage is conceptually mismatched. The UCB1 interval is a confidence bound for the arm mean, not a prediction interval for individual rewards; so its coverage of a future reward is not supposed to be 1-alpha. The claim that 'UCB1 fails to achieve nominal coverage' is therefore a strawman and does not demonstrate a deficiency of UCB as a regret-minimizer. Report, e.g., mean-confidence coverage or compare with an apples-to-apples predictive baseline.","section":"Section 5, Table 1"}],"minor_comments":[{"comment":"Typos and wording: 'probailities' (Section 2), 'ee term' (Section 4.3), 'provids' (Table 3), 'out application' (Section 3.1).","section":"Throughout"},{"comment":"The warm-up is stated to require two observations per arm (one training, one calibration), but CQR with a single training point cannot meaningfully fit a quantile regression. Clarify how the quantile models are initialized and how many samples are actually used before the first interval is formed.","section":"Algorithm 1"},{"comment":"The mean-variance optimization is written with R_j both as part of the objective and as the realized return. Clarify whether W_j uses estimated moments and how the optimization is computed.","section":"Eq. (17)"},{"comment":"The index in the summation reads 't=1,...,t'; it should be i=1,...,t.","section":"Eq. (16)"},{"comment":"Notation conflict: L_{k,t} is used for the lower bound in Eq. (10) but reused as the interval width in Section 5 ('Letting the interval width or length at time t for arm k to be L_{k,t}=U_{k,t}-L_{k,t}'). Use a different symbol for interval width.","section":"Section 5"},{"comment":"The text says all Conformal Bandit policies use CQR with ACI, but the methodology section does not formalize how ACI updates alpha per arm and how it interacts with the quantile regression refitting. Provide the exact update rule used in the experiments.","section":"Section 5"}],"recommendation":"reject","confidential_remarks":"The central finite-sample coverage guarantee is not merely unproven but is contradicted by the adaptive sampling structure of the bandit; this is a load-bearing issue that a minor revision cannot fix. The empirical content, reproducibility, and application are useful, but the paper would need substantial new theory or a fundamental re-scoping as an empirical study without the claimed finite-sample validity. I therefore recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on bandits with statistical validity or small-gap decision problems. The core idea is simple and sensible: replace Hoeffding-based UCB bounds with conformal prediction intervals, with risk-aware variants and an HMM-based regime switch for the portfolio application. What's actually new is the specific algorithms — CP-UCB, CP-Bandit, CP-ESI — and the demonstration that, in their simulations, these beat UCB1 in the hard small-gap regime while maintaining better coverage. That's a genuine, useful contribution, and the Monte Carlo setup (1000 replications, three reward distributions) is reasonable. The writing is clear, and the limitations section is honest.\n\nThe main soft spot is the central coverage claim. Eq. (10) promises finite-sample coverage per arm and per round, but as Remark 4.1 admits, adaptive arm selection compromises exchangeability. The paper then leans on ACI, whose formal guarantee is asymptotic, not finite-sample. Section 7.1 says exactly this. So the abstract's 'finite-sample prediction coverage' is not delivered. This is not a hidden flaw — the authors state it — but it means the headline statistical promise should be restated. The paper's own framing would be much stronger if it said 'empirically calibrated coverage via ACI, asymptotically guaranteed' rather than claiming finite-sample validity throughout.\n\nThe regret claims are also empirical only. There is no theorem, and the paper explicitly defers formal regret analysis. That's fine as a first pass, but the language in Section 5 ('uniformly superior learning efficiency') is stronger than the evidence — which is a set of simulations with a specific gap, σ, and horizon. The portfolio results are a single in-sample historical path from 2018–2025, with no out-of-sample validation, no transaction costs, and comparator hyperparameters unreported for some baselines. These are addressable, not fatal. The minor self-citation ([15]) is peripheral and not a problem.\n\nBottom line: this deserves a serious referee. The idea is useful, the limitations are acknowledged, and the empirical work is reproducible in spirit (code is promised on GitHub). I'd ask for revisions that (1) restate the coverage guarantee as asymptotic or prove a finite-sample alternative, (2) add at least one out-of-sample or robustness check for the portfolio application, and (3) tone down the regret claims to match what simulations can support.","headline":"Honest, useful, and clearly written incremental work that overclaims finite-sample coverage; the regret gains in the small-gap regime are empirical and the statistical-validity guarantee is asymptotic at best, but this is a solid conditional accept rather than a reject.","tokens_in":26053,"tokens_out":934,"would_cite":true,"duration_ms":10226,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62L05","62G15"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes replacing the Hoeffding-based confidence bounds in UCB bandits with conformal prediction intervals, yielding finite-sample coverage and improved regret in the small-gap regime.","keywords":["conformal prediction","multi-armed bandits","small-gap regime","regret minimization","coverage guarantees","adaptive conformal inference","quantile regression","portfolio allocation"],"falsifier":"A Monte Carlo study with K=3 arms, heavy-tailed rewards, and an adaptive policy whose arm-selection probabilities depend on past rewards, where the empirical coverage of the ACI-corrected interval falls below 1-alpha for the most-pulled arm by more than a few percentage points over a horizon of, say, 5,000 rounds, would indicate that the claimed finite-sample validity does not hold under the paper's own design. A counterexample where the conformal upper bound yields worse regret than Hoeffding's bound in a small-gap setting would also falsify the regret claim.","tokens_in":1425,"feed_emoji":"📈","tokens_out":2512,"duration_ms":50770,"temperature":0.7,"pith_summary":"The paper introduces Conformal Bandits, a framework that replaces the Hoeffding-based confidence bounds of UCB1 with conformal prediction intervals built from conformalized quantile regression. It claims that this substitution yields bandit policies that retain the optimism principle of UCB while achieving finite-sample nominal coverage and, in the small-gap regime where arm means differ by about 1/sqrt(n), lower cumulative regret than classical UCB1. The claims are supported by Monte Carlo simulations across Gaussian, Student-t, and skewed reward distributions, and by a portfolio allocation application where a hidden Markov model identifies market regimes and the policy switches between optimistic and defensive arm selection.","feed_headline":"Conformal bandits beat UCB1 in small-gap settings","feed_subtitle":"Swapping Hoeffding bounds for conformal intervals yields nominal coverage and logarithmic regret where UCB1 fails.","key_machinery":"The central object is the arm-specific conformal prediction interval, defined by Eq. (10) as [L_{k,t+1}, U_{k,t+1}] = [ hat{q}_{k,alpha/2}(X_{t+1}) - Q_{1-alpha}(S_k), hat{q}_{k,1-alpha/2}(X_{t+1}) + Q_{1-alpha}(S_k) ], computed via conformalized quantile regression (CQR) and, in practice, adjusted by Adaptive Conformal Inference (ACI) to handle non-exchangeability. The upper bound U drives CP-UCB; the pair (U, L) drives the risk-aware CP-Bandit index (1-lambda)U - lambda|L|; and the ratio U/|L| defines the Exploratory Skewness Index used by CP-ESI. These intervals replace the data-independent Hoeffding exploration bonus with an empirically calibrated measure of predictive uncertainty.","core_discovery":"The paper's central claim is that in the small-gap regime, where classical UCB1 suffers linear regret and fails to achieve nominal coverage, Conformal Bandits built on conformal intervals (Eq. 10) achieve logarithmic regret and coverage close to the nominal level. Specifically, arm-specific conformal intervals give P(Y_{k,t+1} in C^{1-alpha}_{k,t+1}) >= 1-alpha for each arm and round, and the CP-UCB, CP-Bandit, and CP-ESI policies all outperform UCB1 in cumulative regret across the tested heavy-tailed and skewed reward settings. The framework further extends to risk-aware selection through the Exploratory Skewness Index and to non-stationary regimes via hidden Markov model conditioning, with","pith_inferences":["The paper's finite-sample coverage claim is formally proven only for exchangeable data; the adoption of ACI shifts the guarantee to an asymptotic one, and the paper does not provide a theory showing that ACI remains valid when arm selection probabilities depend on past rewards. Thus, the practical validity in the bandit setting is an empirical assertion that could fail in adversarial or rarely-pul","The reported regret advantage over UCB1 is demonstrated only through simulations for specific distributions (Gaussian, Student-t, skew-t) and a fixed gap; whether the advantage is generic or depends on the conformal interval's empirical quantile estimation is left open, especially given the absence of a formal regret bound.","A testable consequence of the risk-aware indices: comparing CP-ESI against mean-variance UCB in a full-information setting could isolate whether asymmetric interval bounds capture drawdown risk better than volatility, which would inform which index is more suitable for downside protection."],"forward_implications":["If correct, Conformal Bandits provide a distribution-free way to obtain nominal finite-sample coverage in sequential decision-making, even when arm rewards are heavy-tailed or skewed.","The framework directly improves regret in the small-gap regime, where classical UCB-type policies exhibit linear regret, making it relevant for applications like portfolio allocation and clinical trials with near-identical interventions.","Risk-aware policy variants (CP-Bandit, CP-ESI) enable downside protection without sacrificing statistical guarantees, and the regime-aware extension via hidden Markov models can adapt to non-stationary environments while preserving coverage.","The approach can be extended beyond UCB-style selection to other bandit families, such as Thompson sampling, as the conformal interval is agnostic to the underlying predictor."],"fun_headline_variants":["Conformal bandits beat UCB1 in tight-arm-gap decisions","Conformal prediction gives bandits finite-sample coverage and better regret","Small reward gaps? Conformal bandits keep regret logarithmic","Conformal bandits: coverage guarantees without sacrificing reward"],"cache_read_input_tokens":27264,"weakest_assumption_plain":"The coverage guarantee holds only if the successive context-reward pairs for each arm are exchangeable; the paper's remedy, ACI, offers only an asymptotic guarantee, and the paper does not prove that ACI remains valid when arm selection is adaptive and some arms are rarely pulled.","fun_headline_variants_meta":{"raw":{"variants":["Conformal bandits beat UCB1 in tight-arm-gap decisions","Conformal prediction gives bandits finite-sample coverage and better regret","Small reward gaps? Conformal bandits keep regret logarithmic","Conformal bandits: coverage guarantees without sacrificing reward"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000323,"raw_usage":{"total_tokens":1658,"prompt_tokens":760,"completion_tokens":898,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":837}},"tokens_in":504,"tokens_out":898,"duration_ms":8352,"temperature":1.0,"reasoning_tokens":837,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T06:30:39.242749+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A Monte Carlo study with K=3 arms, heavy-tailed rewards, and an adaptive policy whose arm-selection probabilities depend on past rewards, where the empirical coverage of the ACI-corrected interval falls below 1-alpha for the most-pulled arm by more than a few percentage points over a horizon of, say, 5,000 rounds, would indicate that the claimed finite-sample validity does not hold under the paper's own design. A counterexample where the conformal upper bound yields worse regret than Hoeffding's bound in a small-gap setting would also falsify the regret claim.","supporting_citations":[],"review_version":1}