{"id":"968d34d7-ae62-457b-95f6-d54523f5ee77","arxiv_id":"2506.20425","paper_version":3,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A new ℓ0 regularized subset selection method for linear mixed models that scales to thousands of predictors, featuring coordinate descent and local search algorithms with convergence guarantees and a finite-sample KL divergence bound.","lead":"The paper introduces a scalable ℓ0-regularized method for subset selection in linear mixed models that handles thousands of predictors efficiently using coordinate descent with a convergence guarantee. This approach could improve sparse modeling for heterogeneous data in fields like personalized medicine by balancing fixed and random effects.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Non-convexity of ℓ0 objective means coordinate descent + local search may miss good solutions; convergence guarantee only reaches stationary points","rationale":"This directly matches the reader's weakest assumption on algorithmic reliability for the non-convex surface. The statistical finite-sample KL bound is secondary if optimization does not deliver good models in practice. Full-text details on the exact coordinate updates and local search implementation would allow tighter scrutiny, but the non-convexity risk remains load-bearing for the central scalability claim regardless.","tokens_in":1696,"tokens_out":377,"duration_ms":37196,"concrete_test":"Generate a synthetic LMM instance with n=1000, p=2000, true support size 10, and known random effects variance; run the full coordinate descent + local search procedure from 20 random initializations and record final objective values and support recovery F1 scores. If the best objective across runs is more than 5% worse than the oracle support objective on average, or if recovery F1 drops below 0.8, the reliability assumption does not hold.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline scalability claim (thousands of predictors in seconds/minutes with excellent performance) requires that the coordinate descent algorithm plus local search reliably reaches high-quality sparse models. The paper states a convergence guarantee for coordinate descent, but on a non-convex ℓ0-regularized objective this only ensures convergence to a coordinate-wise stationary point. The local search step is presented as a practical aid to traverse the surface, yet lacks any guarantee that it escapes poor local minima or that the combination scales without degradation in solution quality as p grows to thousands. If this algorithmic component fails to locate near-optimal supports, both the computational and statistical claims (including the KL bound) rest on unverified behavior rather than guaranteed recovery.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes a scalable ℓ0-regularized method for subset selection in linear mixed models (LMMs) that handles thousands of predictors. It introduces a coordinate descent algorithm with a stated convergence guarantee, augments it with local search to address non-convexity, extends the approach to generalized LMMs via penalized quasi-likelihood, derives a finite-sample bound on the Kullback-Leibler divergence, and reports strong empirical performance on synthetic and real data.","tokens_in":1874,"tokens_out":569,"duration_ms":25238,"significance":"If the algorithmic reliability and statistical bound hold, the work would meaningfully close the scalability gap between sparse linear regression and LMMs, enabling practical sparse modeling for high-dimensional heterogeneous data in areas such as personalized medicine. The explicit convergence guarantee and finite-sample KL bound are positive theoretical contributions that go beyond purely empirical claims.","major_comments":[{"comment":"The convergence guarantee for coordinate descent is stated to apply to the non-convex ℓ0-penalized objective, yet on such surfaces it only reaches coordinate-wise stationary points. Because the central scalability and performance claims (thousands of predictors in seconds to minutes with excellent recovery) rest on the combination of coordinate descent plus local search reliably locating high-quality supports, the manuscript should provide either a stronger guarantee or targeted experiments quantifying how often local search escapes poor stationary points as p grows (e.g., in the synthetic experiments section).","section":"Algorithm and convergence guarantee (abstract and main algorithmic section)"},{"comment":"The finite-sample KL bound is presented as an independent statistical result, but the practical performance claims rely on data-driven choice of the regularization strength λ and other hyperparameters. The manuscript should clarify whether the bound remains valid or informative under such tuning, or whether an additional oracle inequality or stability argument is needed to connect the bound to the reported empirical results.","section":"Statistical bound section"}],"minor_comments":[{"comment":"The extension to generalized LMMs via penalized quasi-likelihood is mentioned but would benefit from an explicit statement of the approximation error or conditions under which the quasi-likelihood remains accurate for the subset selection task.","section":"Generalized LMM extension"},{"comment":"Figure captions and experimental tables should include the exact values of p, n, and number of random effects used in each synthetic setting to allow direct comparison with prior LMM sparse methods.","section":"Experiments"}],"recommendation":"major_revision","confidential_remarks":"The citation list appears light on recent high-dimensional LMM literature; the editor may wish to check whether the novelty claim relative to existing coordinate-descent or local-search approaches for non-convex sparse regression is fully contextualized."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments, which help clarify the scope of our theoretical results and the role of the local search procedure. We address each major comment below, indicating the revisions we plan to incorporate.","responses":[{"response":"We agree that the stated convergence result for coordinate descent establishes convergence to a coordinate-wise stationary point of the non-convex objective, consistent with standard analysis for block coordinate methods on non-convex problems. The local search step is explicitly introduced to move beyond such points by evaluating neighboring supports. To directly address the concern, we will add a targeted set of synthetic experiments that report, for increasing p, the fraction of instances in which local search improves the objective value or support recovery relative to coordinate descent alone. These results will be placed in the synthetic experiments section.","revision_made":"yes","referee_comment":"The convergence guarantee for coordinate descent is stated to apply to the non-convex ℓ0-penalized objective, yet on such surfaces it only reaches coordinate-wise stationary points. Because the central scalability and performance claims (thousands of predictors in seconds to minutes with excellent recovery) rest on the combination of coordinate descent plus local search reliably locating high-quality supports, the manuscript should provide either a stronger guarantee or targeted experiments quantifying how often local search escapes poor stationary points as p grows (e.g., in the synthetic experiments section)."},{"response":"The finite-sample KL bound is derived for the estimator minimizing the penalized objective at a fixed λ. In the reported experiments λ is selected by cross-validation. The bound therefore applies directly only to fixed regularization; its informativeness for the tuned estimator would indeed require an additional stability or oracle argument. We will revise the statistical bound section to state this distinction explicitly and to note that a full oracle inequality connecting the bound to cross-validated λ is left for future work, while preserving the bound as a guarantee for any fixed λ.","revision_made":"partial","referee_comment":"The finite-sample KL bound is presented as an independent statistical result, but the practical performance claims rely on data-driven choice of the regularization strength λ and other hyperparameters. The manuscript should clarify whether the bound remains valid or informative under such tuning, or whether an additional oracle inequality or stability argument is needed to connect the bound to the reported empirical results."}],"tokens_in":1382,"tokens_out":497,"duration_ms":32172,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The central advance here is taking the l0 penalty, already standard for ordinary linear models, and making it computationally feasible inside linear mixed models without restricting to small numbers of predictors. The coordinate descent routine is the main engine, they state a convergence result for it, and they add a local search step to move around the non-convex surface. The extension to generalized LMMs through penalized quasi-likelihood is direct, and the finite-sample KL bound supplies a statistical guarantee that most algorithmic papers skip. These pieces together address the scalability gap the abstract identifies between sparse linear models and sparse LMMs.","headline":"The paper gives a workable l0-regularized coordinate descent method for subset selection in LMMs that scales to thousands of predictors, plus a finite-sample KL bound, though the non-convex solver's ability to find good supports remains the main open question.","tokens_in":2354,"tokens_out":215,"would_cite":false,"duration_ms":33778,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"We develop a coordinate descent algorithm as our main workhorse and provide a guarantee of its convergence. We also develop a local search algorithm to help traverse the nonconvex optimization surface."},{"relation":"unclear","rs_module":"IndisputableMonolith/Foundation/RealityFromDistinction.lean","rs_theorem":"reality_from_one_distinction","paper_passage":"finite-sample bound on the Kullback-Leibler divergence"}],"headline":"Coordinate-descent ℓ0 selection for LMMs lies outside RS forcing chain","alignment":"orthogonal","rationale":"The paper's core machinery (block coordinate descent on a non-convex ℓ0-penalized negative log-likelihood, Sherman-Morrison rank-one updates, local-search swaps, PQL approximation, and finite-sample KL bound) operates entirely within classical statistical estimation and optimization. No reference appears to the recognition cost J, φ-ladder, 8-tick periodicity, or any theorem derived from a single distinction. The domain (stat.ML variable selection) is one on which the RS framework issues no predictions or constraints.","tokens_in":68119,"confidence":"high","tokens_out":305,"duration_ms":9422,"cache_read_input_tokens":38528,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A new ℓ0-regularized method scales subset selection in linear mixed models to thousands of predictors in seconds to minutes.","keywords":["linear mixed models","subset selection","l0 regularization","coordinate descent","high-dimensional data","sparsity","generalized linear mixed models","penalized quasi-likelihood"],"falsifier":"Running the algorithm on a dataset with 2000 predictors and measuring whether runtime stays under a few minutes while the selected model outperforms a dense baseline in cross-validated prediction error would directly test the central scalability and performance claims.","tokens_in":2593,"feed_emoji":"📊","tokens_out":729,"duration_ms":51371,"temperature":0.7,"pith_summary":"Linear mixed models incorporate both fixed and random effects to analyze heterogeneous data such as in personalized medicine, but existing sparse methods cannot handle more than a few hundred candidate predictors. This paper introduces an ℓ0-regularized approach that closes the scalability gap with ordinary linear models. It relies on a coordinate descent algorithm with a convergence guarantee as the main solver and adds a local search step to navigate the non-convex objective surface. The same framework extends to generalized linear mixed models through a penalized quasi-likelihood approximation. A finite-sample bound on Kullback-Leibler divergence and strong empirical results on synthetic and real data complete the contribution.","feed_headline":"New ℓ0 method scales LMM subset selection to thousands of predictors","feed_subtitle":"Coordinate descent plus local search solves the non-convex problem in seconds to minutes with a finite-sample KL bound.","key_machinery":"ℓ0-regularized penalized quasi-likelihood objective for linear mixed models, solved by coordinate descent with convergence guarantee plus local search to handle non-convexity.","core_discovery":"The paper presents a new ℓ0 regularized method for LMM subset selection that can run on datasets containing thousands of predictors in seconds to minutes. On the computational front, a coordinate descent algorithm serves as the main workhorse with a provided convergence guarantee, while a local search algorithm helps traverse the nonconvex optimization surface. Both algorithms extend readily to subset selection in generalized LMMs via a penalized quasi-likelihood approximation. On the statistical front, a finite-sample bound on the Kullback-Leibler divergence of the new method is derived, and experiments demonstrate excellent performance on synthetic and real datasets.","pith_inferences":["Practitioners analyzing clinical or genomic data with individual-level heterogeneity could now perform high-dimensional feature selection without sacrificing the random-effects structure.","The local-search component suggests that direct attack on non-convex ℓ0 problems can be more effective than convex surrogates when paired with a good initialization strategy.","Screening rules or warm-starting from convex solutions might further reduce the already modest runtimes for even larger predictor counts."],"forward_implications":["Subset selection in LMMs becomes practical for wide datasets that previously required dropping random effects or using only convex relaxations.","The coordinate descent procedure converges for the given non-convex objective.","The same computational and statistical framework applies directly to generalized linear mixed models.","A finite-sample KL divergence bound quantifies how well the fitted sparse model approximates the true data-generating process."],"fun_headline_variants":["L0 regularization scales subset selection in LMMs to thousands","Coordinate descent converges for L0-penalized LMM selection","Local search traverses nonconvex surface in L0 LMM optimization","Finite-sample KL bound derived for scalable L0 LMM method"],"cache_read_input_tokens":64,"weakest_assumption_plain":"Coordinate descent combined with local search reliably finds good solutions on the non-convex ℓ0-regularized objective, and the penalized quasi-likelihood approximation stays accurate when extending to generalized linear mixed models.","fun_headline_variants_meta":{"raw":{"variants":["L0 regularization scales subset selection in LMMs to thousands","Coordinate descent converges for L0-penalized LMM selection","Local search traverses nonconvex surface in L0 LMM optimization","Finite-sample KL bound derived for scalable L0 LMM method"]},"model":"grok-4.3","cost_usd":0.016143,"raw_usage":{"total_tokens":6827,"prompt_tokens":686,"num_sources_used":0,"completion_tokens":62,"cost_in_usd_ticks":161428000,"prompt_tokens_details":{"text_tokens":686,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":6079,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":686,"tokens_out":62,"duration_ms":61298,"temperature":1.0,"reasoning_tokens":6079,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-19T08:07:34.786631+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the algorithm on a dataset with 2000 predictors and measuring whether runtime stays under a few minutes while the selected model outperforms a dense baseline in cross-validated prediction error would directly test the central scalability and performance claims.","supporting_citations":[],"review_version":1}