{"id":"12f39c2a-4443-49cd-8673-227840957c16","arxiv_id":"2504.16555","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A low-regret online predictor for any GLM yields a valid confidence sequence for the true parameter, giving a unified framework and new sample-size-independent and sparse-model bounds.","lead":"This paper converts the problem of building confidence sets for generalized linear model parameters into a question about regret in an online prediction game: any low-regret sequential predictor yields a valid confidence sequence. The framework recovers existing confidence sets for linear and logistic models and adds new ones, including a bound whose width does not grow with the sample size.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core online-to-confidence-set reduction (Theorem 2.2) is mathematically sound; no load-bearing flaw identified. The disclosed strong-convexity/polar-set limitation in Theorem 4.3 and the abstract's overclaim are practicality/presentation issues, not correctness gaps in the central claim.","rationale":"The reader's weakest_assumption identifies the strong-convexity/polar-set condition in Theorem 4.3, which is indeed the most delicate part of the paper's new n-independent result. I agree that this condition can make the bound practically useless for logistic regression in high dimension, and that misspecifying b breaks the proof. However, the condition is explicitly stated and its limitations are candidly discussed by the authors, so I do not regard it as a hidden correctness gap in the central claim. The central theorem (Theorem 2.2) is a clean and correct application of the sequential likelihood-ratio martingale, and the regret bounds used in the applications are standard and checked. The reader's minor concerns about Eq. (18) and the 'all state-of-the-art' overclaim are valid presentation issues but do not change the mathematical status. Therefore I see no reason to alter the CONDITIONAL verdict: it already reflects the appropriate level of caution about the presentation and the practical reach of the strong-convexity results, without rejecting the sound core reduction framework.","tokens_in":28709,"tokens_out":29266,"duration_ms":252271,"concrete_test":"Run a logistic-regression simulation for Theorem 4.3 with d=5, n=500, covariates satisfying ||X_t||_infty <= 1, and theta* scaled so that max_t |<theta*,X_t>| = b_true. Use the true b = b_true and the true strong-convexity constant m on [-b,b], and compute empirical coverage of the set in (20) over 1000 independent trials. If coverage is within Monte Carlo error of 1-delta, the theorem is internally consistent; then evaluate the width d log(1+2kappa)+2log(1/delta) to confirm whether it is vacuous for moderate d, which would verify the authors' disclosed practical limitation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"No significant objection identified. The central reduction in Theorem 2.2 is correct: the excess likelihood of the true parameter decomposes exactly into the online learner's regret against the reference plus a log-likelihood-ratio martingale, and Ville's inequality (Proposition 2.1) supplies the high-probability control. The F_{t-1}-measurability assumption on the online predictions is the only condition needed for this step, and it is satisfied in both the adaptive and transductive instantiations. The application theorems then follow from the stated deterministic regret bounds; I checked the proofs of Theorem 4.1, Theorem 4.3, and Theorem 4.7, and each step is internally consistent. The most delicate ingredient is Theorem 4.3, where the confidence set is only valid if the unknown parameter lies in the polar set S_{n,b} and if the curvature lower bound m on [-b,b] is known. These assumptions are explicitly stated and their severity (exponentially small m for logistic regression) is acknowledged by the authors in Section 4 and the Discussion. A misspecified b would indeed break the inequality (21) and the first-order optimality step, but this is a disclosed limitation of the n-independent result rather than a hidden flaw in the framework. The side discussion around Eq. (18) and the abstract's 'all state-of-the-art' phrasing are minor presentational issues that do not affect the validity of the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper develops a framework for constructing confidence sets and confidence sequences for GLM parameters by reducing the statistical problem to regret analysis in a sequential probability assignment game. The central reduction (Theorem 2.2) decomposes the excess loss of the true parameter against a data-dependent reference estimator into the online learner's regret plus a sequential log-likelihood-ratio martingale, which is controlled by Ville's inequality. The authors distinguish analytic conversions, where only the existence of a low-regret forecaster is needed, from algorithmic conversions, where the forecaster's predictions are used actively to define the confidence set. The framework is instantiated with EWA and NML forecasters. The resulting applications include confidence sets for adaptive covariates (Theorem 4.1), an n-independent width for transductive strongly convex GLMs (Theorem 4.3), and sparse-model confidence sets obtained from algorithmic conversions with pseudo-labels (Theorems 4.4-4.7). The manuscript also positions the work relative to Abbasi-Yadkori, Pal and Szepesvari (2012), Jun et al. (2017), Kirschner et al. (2025), and related online-to-confidence-set literature.","tokens_in":28989,"tokens_out":18165,"duration_ms":160841,"significance":"Subject to the stated assumptions, the central reduction is correct and the paper is a genuine conceptual contribution: it provides a single template that recovers several known confidence-set constructions and yields new results, including the n-independent transductive width and label-independent sparse confidence sets. I checked the proofs of Propositions 2.1, 2.3, 3.1, and 3.2 and Theorems 2.2, 2.4, 4.1, 4.3, 4.5, 4.6, and 4.7; the martingale decompositions are sound and the Gaussian integral computations are standard. The authors deserve credit for being explicit about the severe limitations of strong-convexity assumptions, in particular the exponentially small m for logistic regression, and about the need for the polar set S_{n,b}. The main issues are local: a technically incorrect regularized-loss comparison around Eq. (18), an overbroad statement in the abstract, and several notational slips. These do not affect the validity of the central reduction and are readily fixable.","major_comments":[],"minor_comments":[{"comment":"The passage around Eq. (18) is not a correct rewrite of Theorem 4.1 for general lambda. Theorem 4.1 is stated with lambda = 1 and with a width for the unregularized loss; after writing \\tilde{\\ell}_t = \\ell_t + \\rho, the exact regularized width is \\tilde{\\beta}_n = \\rho(\\theta_\\star) + \\gamma^{\\rho}_{n,1} + \\log(1/\\delta) at lambda = 1, or, for general lambda, \\tilde{\\beta}_n = (\\rho(\\hat{\\theta}_n) + \\gamma^{\\rho}_{n,\\lambda})/\\lambda + \\rho(\\theta_\\star) - \\rho(\\hat{\\theta}_n) + \\log(1/\\delta). The printed expression with 1/(2\\lambda) prefactors and lambda inside the determinant omits the term (1/\\lambda - 1)\\rho(\\hat{\\theta}_n) and is only valid when lambda = 1 or \\hat{\\theta}_n = 0. Please derive this step explicitly or restrict the display to lambda = 1.","section":"Section 4.1.1, Eq. (18)"},{"comment":"The abstract's claim that the methodology 'recovers all state-of-the-art confidence set constructions within a single framework' overstates the scope. Several constructions in the cited literature rely on additional assumptions such as compact parameter sets or known strong-convexity constants, and the Discussion itself highlights severe limitations of the strong-convexity results. I suggest softening this to 'several' or to 'the constructions covered by our assumptions'.","section":"Abstract"},{"comment":"The theorem's validity depends on the user knowing b and m and on the true parameter lying in the polar set S_{n,b}. These conditions are stated in the text before the theorem, but the 'first to have width independent of n' claim in Section 5 and the abstract do not carry the caveat. Since a misspecified b invalidates the strong-convexity inequality (21) and the first-order optimality step, this caveat deserves more prominence in the theorem statement and the discussion.","section":"Section 4.1.2 and Theorem 4.3"},{"comment":"Notation and typos: Eq. (20) writes the reference estimator as \\hat{\\theta}_{b,n} while the theorem and proof use \\hat{\\theta}_{n,b}; the proof of Theorem 4.3 first writes \\rho(\\theta) = \\gamma^{-2}\\|\\theta-\\theta_\\star\\|^2_{\\Lambda_n} where the subsequent argument uses 1/(2\\gamma^2); Proposition 2.3's proof contains '\\ell_n' and 'q_n' where '\\ell_t' and 'q_t' are meant; in Section 2.1, 'we we will play special attention' should be 'we will pay special attention'; Eq. (12) has a missing parenthesis in the denominator; and in Section 3.2, 'Our of the many possibilities' should be 'Out of the many possibilities'.","section":"Throughout"},{"comment":"The sentence 'We provide concrete examples (as well as a tighter confidence set of a similar shape) in Section 2.3' appears to be a cross-reference error; the concrete examples of convex sets built from d_\\psi are actually given in Section 4.2.","section":"Section 2.3, last paragraph"}],"recommendation":"minor_revision","confidential_remarks":"The overlap with Kirschner et al. (2025) is disclosed and discussed in the manuscript, and I see no novelty or disclosure problem. The central theorem is correct and the paper is well within the scope of the journal. The main revision point is the regularized-loss comparison around Eq. (18), which should be corrected or explicitly restricted to lambda = 1 before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The central reduction in Theorem 2.2 is correct and worth knowing: for any GLM, if you have a sequential predictor with a regret bound, you get a valid confidence sequence centered at any data-dependent reference point. The proof is a clean decomposition into regret plus a log-likelihood-ratio martingale, and Ville's inequality does the rest. I checked the main applications, and the martingale arguments and Gaussian integral bounds hold up. The paper is also honest about the parallel work by Kirschner et al. (2025), which is refreshing.\n\nWhat is actually new: the sample-size-independent confidence set in Theorem 4.3, the sparse EWA-based conversions in Theorems 4.5-4.7, and several constant-factor improvements over Abbasi-Yadkori et al. and Jun et al. The sparse bound in Theorem 4.7 removes the max-label factor from earlier work, which is a genuine improvement. The framework itself is a repackaging of known ideas, but the new instantiations justify the paper.\n\nThe soft spots are real but not load-bearing. The abstract's claim that the methodology recovers \"all state-of-the-art confidence set constructions\" is too strong; it recovers the main GLM ones. The side discussion around Eq. (18) conflates scaled regret with log-loss regret and misstates the regularized-loss representation; it should be corrected, though it does not affect the main theorems. Theorem 4.3 is the most delicate: its n-independent width depends on strong convexity on [-b,b] with known m, and the authors themselves note m can be exponentially small for logistic regression. If b is misspecified, the polar set argument and inequality (21) break. That is a disclosed limitation rather than a hidden flaw, but it means the headline result is less useful than it first appears. The citation pattern looks fair, and the self-citations are to results they actually build on.\n\nWho is this for? People working on sequential decision-making, time-uniform inference, and online-to-confidence conversions. It is a solid contribution that deserves a serious referee. I would send it to peer review, with a request to fix Eq. (18), tone down the abstract, and state more prominently that Theorem 4.3's regime is narrow.","headline":"A sound and honest unification of GLM confidence-sequence construction via regret analysis, with two genuinely new bounds and one overclaim in the abstract that should be fixed.","tokens_in":29624,"tokens_out":1223,"would_cite":true,"duration_ms":13879,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F25","62L12","62J12","68Q87","68Q32"],"pacs":[],"model":"deepseek-v4-flash","headline":"For any generalized linear model, a regret bound for a sequential predictor yields a valid confidence sequence for the true parameter.","keywords":["confidence sequences","generalized linear models","sequential probability assignment","regret analysis","online-to-confidence-set conversions","martingale concentration","Ville's inequality","sparse regression"],"falsifier":"One settling check is simulation-based coverage of the Theorem 2.2 set for a logistic regression with known $\\theta^*$ and a misspecified bound $b$: generate repeated sequences, and if $\\theta^*$ falls outside the set more than $\\delta$ of the time in the regime $|\\langle\\theta^*,X_t\\rangle|\\le b$, the polar-set construction used in the applications is invalid; in the opposite direction, if the set is so large that it contains essentially all parameters, the n-independent width is vacuous for that GLM.","tokens_in":28445,"feed_emoji":"🎯","tokens_out":13350,"duration_ms":112273,"temperature":0.7,"pith_summary":"This paper establishes that confidence-sequence construction for generalized linear models can be reduced to regret analysis in a sequential probability-assignment game. Concretely, for any GLM and any predictor that issues a distribution before seeing each label, the set of parameters whose cumulative negative log-likelihood exceeds the predictor's loss by at most the predictor's regret against a reference parameter plus $\\log(1/\\delta)$ is a valid confidence sequence for the true parameter. Because the covering argument is a martingale inequality, the online algorithm only needs to exist for the bound to hold, although the paper also shows how to use its predictions to center sets adaptively. The authors instantiate the reduction with exponentially weighted and normalized-maximum-likelihood forecasters, recovering earlier confidence-set constructions and producing new ones, including a width that does not grow with the sample size when the log-partition is locally strongly convex.","feed_headline":"Regret bounds yield valid GLM confidence sequences","feed_subtitle":"A martingale identity turns any sequential predictor into a confidence set, with widths that shrink at known optimal rates.","key_machinery":"The load-bearing object is the sequential likelihood-ratio martingale $M_n=\\prod_{t=1}^n p_t(Y_t)/p(Y_t\\mid X_t,\\theta^\\star)$, which is nonnegative and satisfies $\\mathbb{E}[M_t\\mid\\mathcal{F}_{t-1}]=M_{t-1}$; Ville's inequality then gives $\\mathbb{P}[\\exists n:\\log M_n\\ge\\log(1/\\delta)]\\le\\delta$. The conversion identity is $$\\sum_{t=1}^n(\\ell_t(\\$\\theta$^\\star)-\\ell_t(\\theta_n))=\\mathrm{regret}_{p_n}(\\theta_n)+\\sum_{t=1}^n(\\ell_t(\\$\\theta$^\\star)-L_t(p_t)),$$ where the second term is exactly $\\log M_n$. For algorithmic conversions the paper introduces $\\eta$-shifted losses $\\ell_t^{(\\eta)}(\\theta)=\\ell_t(\\eta\\theta+(1-\\eta)\\theta^\\star)$, which keep $\\theta^\\star$ as a fixed point and generate a second martingale, and measures the size of the resulting sets with the symmetrized convexity measure $d_\\psi(z,z')=\\tfrac12\\psi(z)+\\tfrac12\\psi(z')-\\psi((z+z')/2)$.","core_discovery":"The central claim is Theorem 2.2: with $\\ell_t(\\theta)=-\\log p(Y_t\\mid X_t,\\theta)$, let $p_1,\\dots,p_n$ be any predictions chosen before the corresponding labels are revealed, and let $\\theta_n$ be any data-dependent reference parameter. Then $$\\Theta_n=\\left\\{\\$\\theta$\\in\\mathbb{R}^d:\\sum_{t=1}^n(\\ell_t(\\$\\theta$)-\\ell_t(\\theta_n))\\le \\mathrm{regret}_{p_n}(\\theta_n)+\\log(1/\\delta)\\right\\}$$ satisfies $\\mathbb{P}[\\exists n:\\theta^\\star\\notin\\Theta_n]\\le\\delta$. The proof decomposes the excess loss of the true parameter over the reference into the predictor's regret against the reference plus a sequential log-likelihood-ratio sum, which is a nonnegative martingale; Ville's inequality bounds that martingale by $\\log(1/\\delta)$. Thus every regret bound for sequential probability assignment with GLM losses automatically becomes a confidence sequence, and the same reduction, refined with shifted losses, produces sets whose width is governed by regret against $\\theta^\\star$ rather than against a reference estimator.","pith_inferences":["One extension the paper only sketches is that the same martingale argument survives when the exponential-family moment condition is relaxed to an inequality, so the reduction should transfer to sub-exponential families and generalized additive models; checking that transfer would be a natural next test.","The n-independent width of Theorem 4.3 is most useful when the strong-convexity interval is short; for logistic regression the curvature constant $m$ is typically exponentially small in the dimension, so a practical development would be to replace global strong convexity on $[-b,b]$ with curvature measured only on the realized prediction interval of $\\theta^*$.","The analytic/algorithmic distinction suggests a design rule: in analytic mode the user should choose the predictor solely for its regret bound, since it never has to be run, whereas in algorithmic mode the predictor's output defines the pseudo-labels and therefore must be computable; the sparse-GLM result shows when paying that computation removes the label-magnitude factor.","The rank-aware regret bound indicates that confidence widths could adapt to the effective dimension of the covariates, shrinking on sequences where the design matrix is low rank; this is implicit in the determinant bound but not developed as a separate adaptive procedure."],"forward_implications":["With adaptively chosen covariates and an $M$-smooth log-partition, the analytic conversion gives confidence width $\\|\\hat\\theta_n\\|^2/(2\\gamma^2)+\\tfrac12\\log\\det(\\gamma^2M\\Lambda_n+I_d)+\\log(1/\\delta)$, recovering linear-model ellipsoid bounds and removing the uniform lower-curvature assumptions in earlier GLM confidence sets.","With obliviously chosen covariates and a locally $m$-strongly convex log-partition, the confidence set has width $d\\log(1+2M/m)+2\\log(1/\\delta)$, independent of the sample size $n$ and invariant to invertible linear reparametrization of the covariates.","For $s$-sparse parameters, the algorithmic conversion with a sparsity-inducing prior yields width $(4s/m)\\log(2ed\\sqrt{1+MB^2L_\\infty^2n/2}/s)+(4/m)\\log(2\\sqrt e/\\delta)$, which removes the factor $\\max_tY_t^2$ that appears in earlier deterministic-forecaster confidence sets.","Any future regret bound for sequential probability assignment with GLM log losses can be plugged into Theorem 2.2 to yield a valid confidence sequence, so the statistical task reduces to an algorithmic task.","The framework recovers several published confidence-set constructions as special cases of one reduction, including linear, logistic, and general exponential-family settings."],"supporting_citations":[{"why":"Supplies the martingale maximal inequality that converts the sequential likelihood-ratio sum into a high-probability bound.","marker":"Ville (1939)"},{"why":"Introduces online-to-confidence-set conversions, the scheme the paper generalizes into analytic and algorithmic forms.","marker":"Abbasi-Yadkori, Pál and Szepesvári (2012)"},{"why":"Provides the log-loss sequential-prediction framework and the EWA regret calculations used throughout.","marker":"Cesa-Bianchi and Lugosi (2006)"},{"why":"Supplies the no-hypercompression interpretation of Proposition 2.1 and the Shtarkov-sum machinery for normalized maximum likelihood.","marker":"Grünwald (2007)"},{"why":"Defines the Bregman information gain that controls the EWA regret bounds in the applications.","marker":"Chowdhury et al. (2023)"},{"why":"The closest prior GLM confidence-sequence construction whose width the paper tightens by removing extra curvature assumptions.","marker":"Lee, Yun and Jun (2024b)"},{"why":"Earlier GLM bandit confidence sets that rely on uniform strong convexity, which the analytic conversion avoids.","marker":"Jun et al. (2017)"},{"why":"Provides the sparse-regret algorithm whose style the sparsity-inducing EWA prior adapts for the algorithmic sparse confidence sets.","marker":"Gerchinovitz (2013)"},{"why":"Establishes the general equivalence between martingale tail bounds and regret that motivates the reduction.","marker":"Rakhlin and Sridharan (2017)"}],"fun_headline_variants":["Regret bounds directly yield GLM confidence sequences","Online prediction games build valid confidence sets","A martingale identity connects regret to confidence","Unified GLM confidence sets from sequential regret","Low-regret predictors certify GLM parameters online"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The binding assumption is that the user knows an interval on which the log-partition is strongly convex and can guarantee the true parameter's linear predictions stay inside it; when that curvature constant is tiny the width explodes, and a wrong interval invalidates the comparator.","fun_headline_variants_meta":{"raw":{"variants":["Regret bounds directly yield GLM confidence sequences","Online prediction games build valid confidence sets","A martingale identity connects regret to confidence","Unified GLM confidence sets from sequential regret","Low-regret predictors certify GLM parameters online"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000352,"raw_usage":{"total_tokens":1925,"prompt_tokens":959,"completion_tokens":966,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":898}},"tokens_in":575,"tokens_out":966,"duration_ms":9492,"temperature":1.0,"reasoning_tokens":898,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:02:44.901199+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One settling check is simulation-based coverage of the Theorem 2.2 set for a logistic regression with known $\\theta^*$ and a misspecified bound $b$: generate repeated sequences, and if $\\theta^*$ falls outside the set more than $\\delta$ of the time in the regime $|\\langle\\theta^*,X_t\\rangle|\\le b$, the polar-set construction used in the applications is invalid; in the opposite direction, if the set is so large that it contains essentially all parameters, the n-independent width is vacuous for that GLM.","supporting_citations":[{"cited_title":"( 1939 )","cited_arxiv_id":null,"evidence_quote":"Supplies the martingale maximal inequality that converts the sequential likelihood-ratio sum into a high-probability bound."},{"cited_title":"Lugosi , G \\'a bor G","cited_arxiv_id":null,"evidence_quote":"Provides the log-loss sequential-prediction framework and the EWA regret calculations used throughout."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Bregman information gain that controls the EWA regret bounds in the applications."},{"cited_title":", Bhargava , Aniruddha A","cited_arxiv_id":null,"evidence_quote":"Earlier GLM bandit confidence sets that rely on uniform strong convexity, which the analytic conversion avoids."},{"cited_title":"( 2013 )","cited_arxiv_id":null,"evidence_quote":"Provides the sparse-regret algorithm whose style the sparsity-inducing EWA prior adapts for the algorithmic sparse confidence sets."},{"cited_title":"Sridharan , Karthik K","cited_arxiv_id":null,"evidence_quote":"Establishes the general equivalence between martingale tail bounds and regret that motivates the reduction."}],"review_version":1}