{"id":"ae5680e9-f1e1-431e-a97a-726128abd5bc","arxiv_id":"2502.07655","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"Lasso is convex and fast but biased; SCAD and MCP reduce bias but are non-convex and slower, so method choice depends on context.","lead":"This short note reviews known trade-offs between Lasso and the non-convex penalties SCAD and MCP in high-dimensional regression. It is a readable summary with toy simulations, useful as an introduction but offering no new statistical result.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The empirical claim that SCAD/MCP achieve ground-truth sparsity without sacrificing estimation accuracy is not established, because Figure 3 never pairs the MSE and sparsity curves at a single λ and the non-convex optimizer settings are unreported.","rationale":"The paper is a short teaching-style note; its qualitative statements about bias, oracle properties, and optimization difficulty are broadly consistent with the literature, and the coordinate-descent updates appear standard. The load-bearing part is the specific empirical claim that SCAD/MCP 'can give us an estimator with ground truth sparsity level without sacrificing estimation accuracy.' That claim is the basis for the practical guidance in the abstract and Section 4. My concern is not that the claim is known to be false; it is that the evidence provided in Figure 3 does not establish it. The text compares two conflicting λ values for Lasso, then asserts the non-convex penalties work better without reporting the corresponding λ values or a joint summary. The missing experimental constants (a, initialization, tolerance) matter because for non-convex objectives the output is algorithm-dependent, and Figure 4 already signals unstable convergence. A small, concrete re-analysis would settle whether the claim holds: compute MSE at λ values that yield the ground-truth sparsity level for each method, and check stability under optimizer settings. Given how easy this is, I would not reject the paper; I would keep the CONDITIONAL verdict and require this paired analysis, or an explicit repositioning as a purely qualitative review, before relying on the empirical claim.","tokens_in":5828,"tokens_out":6593,"duration_ms":60100,"concrete_test":"Re-run Model 1 from Sections 2–3 and record, for every λ, both average sparsity and average MSE for each penalty. For each method, select the smallest MSE among λ values whose average sparsity equals 10 within ±1; compare that paired MSE across Lasso, SCAD, and MCP. If no such λ exists for SCAD/MCP, or if their paired MSE is not lower than Lasso's, the central claim fails. As a robustness check, rerun SCAD/MCP with warm starts from the Lasso path, with a = 3.7 and a = 2.01, and with tolerance 1e-8, and confirm the paired result is stable across these settings.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2's central claim, 'the SCAD and the MCP penalties can give us an estimator with ground truth sparsity level without sacrificing estimation accuracy', rests entirely on Figure 3. The figure plots MSE and sparsity level separately against λ for Lasso, SCAD, and MCP. For Lasso the text names two specific λ values (0.12 for minimum MSE, >0.35 for sparsity equal to 10) and uses their mismatch to argue that Lasso cannot satisfy both goals. For SCAD and MCP, however, no λ is identified at which the average sparsity equals the ground truth and the MSE is low; the assertion is made directly from the plotted curves. Without a paired statistic, the curves only show that some intermediate λ may look favorable, not that a single estimator has both properties. This is the first and most direct load-bearing gap. A second confound is the non-convex optimization: coordinate descent for SCAD/MCP can stop at local minima whose location depends on initialization, warm starting, the shape parameter a, and the tolerance. Section 3 shows unstable loss variation (Figure 4) and reports run times (Table 1) but never states a, the initialization, or the convergence tolerance, so the plotted SCAD/MCP behavior may reflect the algorithm's terminal points rather than the penalty family's typical estimator.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This short note compares Lasso with the non-convex SCAD and MCP penalties in high-dimensional penalized likelihood estimation. The paper reviews the penalties, states their known bias/oracle properties, presents a small simulation study (Section 2), derives closed-form coordinate descent updates (Section 3), and reports a real-data comparison on a rat eye eQTL dataset (Section 4). The central empirical assertion is that SCAD and MCP can achieve the ground-truth sparsity level without sacrificing estimation accuracy, while Lasso cannot simultaneously satisfy both goals.","tokens_in":6074,"tokens_out":5524,"duration_ms":50420,"significance":"If the claims were fully supported, the paper would offer a useful, concise comparison with practical guidance on choosing between convex and non-convex penalties. The qualitative statements accord with the existing literature, and the coordinate descent formulas are standard. However, the paper provides no new theory, and its own empirical evidence is too incomplete to establish the central claim: the simulation lacks paired sparsity/MSE statistics, the non-convex optimization settings are unreported, and the real-data comparison relies on in-sample RSS. The manuscript is a reasonable pedagogical note, but the load-bearing quantitative assertions need substantial strengthening before the paper can be accepted as a standalone research contribution.","major_comments":[{"comment":"The claim that 'the SCAD and the MCP penalties can give us an estimator with ground truth sparsity level without sacrificing estimation accuracy' is not established by the displayed results. Figure 3 plots MSE and sparsity level separately against lambda; for Lasso the text explicitly identifies lambda=0.12 (minimum MSE) and lambda>0.35 (sparsity 10), but no lambda is identified for SCAD or MCP at which the average sparsity equals 10 and the MSE is simultaneously low. Because the two criteria are not paired at a single lambda, the curves only suggest that some intermediate lambda may look favorable. Please report a paired summary: for each method and each lambda, the average sparsity and MSE together, or identify a concrete lambda where both objectives are met, with standard errors across the 100 replications.","section":"Section 2, after Figure 3"},{"comment":"The closed-form updates for Lasso, SCAD, and MCP are not correctly normalized with respect to the objective in Eq. (1.1), which includes the factor 1/(2n). With L(beta)=||Y-Xbeta||^2, the coordinate-wise solution for Lasso is beta_j = S(X_j^T r, n lambda)/X_j^T X_j (or, if columns are standardized to unit L2 norm, S(X_j^T r, lambda)). The manuscript writes beta_j = S(X_j^T r, lambda) without stating the column normalization. For the real data, variables are standardized to zero mean and unit standard deviation, so X_j^T X_j is approximately n, making the displayed threshold inconsistent with Eq. (1.1). Please clarify the normalization used in the simulations and correct the formulas, since the lambda grid in Figure 3 depends on this scaling.","section":"Section 3, coordinate descent formulas"},{"comment":"The empirical comparison of convergence is not reproducible because the optimization settings are unreported. The paper never states the SCAD shape parameter a, the MCP shape parameter a, the initialization, whether warm starts were used, the convergence tolerance, the maximum number of iterations, or the lambda grid. For non-convex objectives, coordinate descent can converge to different local minima depending on these choices; the unstable loss variation in Figure 4 may therefore reflect the algorithm's terminal points rather than a property of the penalty family. Please report all of these settings and consider a sensitivity check with multiple starting values or a comparison against a well-tested implementation such as ncvreg.","section":"Section 3, Figure 4 and Table 1"},{"comment":"The real-data comparison uses the residual sum of squares (RSS) to conclude that SCAD 'performs better' than Lasso. Since RSS is an in-sample measure and the two methods have different sparsity levels under the selected lambdas, a lower RSS does not establish better predictive performance or better model selection. Please use out-of-sample prediction error, cross-validated deviance, or another criterion that accounts for model complexity. In addition, the real-data section omits MCP entirely, so it does not support the paper's stated three-way comparison.","section":"Section 4, Figure 5"}],"minor_comments":[{"comment":"The sentence 'the Lasso penalty always introduces a bias on its large components without sacrificing model selection consistency' is ambiguous and potentially misleading: model selection consistency of Lasso is known to require additional conditions such as the irrepresentable condition. Please rephrase to state the precise theoretical result.","section":"Section 2"},{"comment":"The scaling 1/(2n) is introduced for the least-squares case, but L(beta) is defined generally as the negative log-likelihood. Please clarify whether Eq. (1.1) is meant to be generic or specific to the linear model.","section":"Section 1, Eq. (1.1)"},{"comment":"The quantity 'Loss Variation' plotted in Figure 4 is not defined. Specify whether it is |Q(beta^(k)) - Q(beta^(k-1))|, a relative change, or something else.","section":"Section 3, Figure 4"},{"comment":"There are several typographical issues: 'coordinate decent' should be 'coordinate descent', 'right penal' should be 'right panel', 'BardetBiedl' should be 'Bardet-Biedl', and 'Not surprised' should be 'It is not surprising'.","section":"Throughout"},{"comment":"The paper should cite relevant algorithmic references for coordinate descent with SCAD and MCP, such as Breheny and Huang (2011) and Mazumder et al. (2011), and the oracle property results of Fan and Li (2001) and Zhang (2010).","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is more of a pedagogical note than a research paper. Its central empirical claim is not yet supported, and the missing optimization details would need to be addressed in a revision. The fit to a serious statistical journal depends on whether the authors can turn the simulation and real-data sections into reliable evidence, or reframe the paper as an expository review with appropriately hedged claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent teaching note, not a research contribution. It repeats known results about Lasso vs SCAD/MCP, and its own simulation doesn't actually demonstrate the headline claim.\n\nWhat's good: the coordinate descent updates in Section 3 are correctly written, and the figures do show the expected bias-sparsity tension. The paper is easy to read, and for someone who hasn't seen these penalties before, it would be a serviceable introduction. The real-data example uses the standard Scheetz eye data, which is a nice illustration.\n\nThe soft spots are real. The central assertion in Section 2—that SCAD and MCP can achieve ground-truth sparsity without losing accuracy—is read off Figure 3, where MSE and sparsity are plotted against λ as separate curves. For Lasso the text is careful to name specific λ values; for SCAD and MCP it never identifies a single λ where both criteria are met. That's the load-bearing gap. Also, the non-convex optimization details are missing: the shape parameter a, the convergence tolerance, and the initialization are not reported, so the plotted behavior could reflect local minima rather than the typical estimator. The real-data comparison reports training RSS, which tells us nothing about generalization. And the reference list contains several works on 3D pose estimation and other topics that are irrelevant to the content; that looks like citation padding.\n\nNone of this is fatal to the qualitative message, which is standard and correct. But the paper doesn't give the reader a reproducible or rigorous demonstration of its own claim. It reads like an undergraduate project note, not a peer-reviewed research paper.\n\nWho is this for? A student or practitioner looking for a quick overview. Not for a researcher. I would not send it to peer review as a research article; I would suggest the authors either reposition it as an explicit review note or add the missing experimental details and out-of-sample evaluation. If they do that, it could become a decent teaching resource.\n\nFor now: desk reject if it lands on a research journal.","headline":"A correct but unoriginal teaching note whose own experiments don't back the headline claim.","tokens_in":6540,"tokens_out":2802,"would_cite":false,"duration_ms":29342,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62J07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that non-convex penalties such as SCAD and MCP can reach ground-truth sparsity without sacrificing estimation accuracy, while Lasso trades one against the other.","keywords":["penalized likelihood","Lasso","SCAD","MCP","non-convex regularization","coordinate descent","high-dimensional regression","variable selection"],"falsifier":"Run the n=200, p=1000 simulation with SCAD and MCP over a grid of shape parameters (e.g., $a=2.1,3.7,10$) and several random initializations; if some runs stop at visibly higher loss or sparsity levels far from 10, the claimed sparsity-without-accuracy-loss is an optimizer artifact rather than a penalty property. A cleaner check is to solve small instances to certified global optimality and compare the coordinate-descent solution against the global solution.","tokens_in":5607,"feed_emoji":"🎯","tokens_out":6390,"duration_ms":55662,"temperature":0.7,"pith_summary":"This short note tries to establish a practical trade-off between convex and non-convex penalized likelihood methods in high-dimensional regression. Lasso is computationally efficient and convex but biases large coefficients, while SCAD and MCP shrink mainly small coefficients and can match the true sparsity level without losing estimation accuracy. A toy simulation with n=200, p=1000 and ten nonzero coefficients shows Lasso cannot simultaneously reach ground-truth sparsity and low MSE, whereas SCAD and MCP can. The note also reports that coordinate descent for SCAD and MCP converges more slowly and with less stable loss variation, and that on a rat gene-expression data set SCAD gives a sparser model with lower residual sum of squares than Lasso. If the claims hold, practitioners can choose Lasso when fast convex optimization matters and SCAD or MCP when bias on large coefficients matters.","feed_headline":"Non-convex penalties hit true sparsity without Lasso bias","feed_subtitle":"SCAD and MCP trade slower convergence for sparser, less biased estimates in a high-dimensional simulation.","key_machinery":"The machinery is the first derivative of the penalty function $p_\\lambda(t)$: Lasso has $p_\\lambda'(t)=\\lambda$ constant, SCAD has $p_\\lambda'(t)=\\lambda[I(t\\le\\lambda)+((a\\lambda-t)_+/((a-1)\\lambda))I(t>\\lambda)]$, and MCP has $p_\\lambda'(t)=(\\lambda-t/a)_+$. This derivative determines the shrinkage pattern: a constant derivative means equal shrinkage of all coefficients, while a derivative that declines to zero means only small coefficients are penalized. The paper reads the bias, sparsity, and accuracy comparison off this derivative, then uses coordinate descent with closed-form soft-thresholding-type updates for each penalty to test the behavior computationally.","core_discovery":"The central claim is that non-convex penalties are not merely a theoretical alternative to Lasso: in a high-dimensional linear model with n=200, p=1000 and 10 nonzero coefficients, SCAD and MCP produce estimators with ground-truth sparsity level without sacrificing estimation accuracy, while Lasso requires a much larger penalty to reach that sparsity and then suffers high MSE. The mechanism is the derivative of the penalty: Lasso applies a constant penalty to every coefficient regardless of size, shrinking large coefficients and creating bias, whereas SCAD and MCP keep a positive penalty only for small values and let it drop to zero for large values, so large coefficients are left nearly unpenalized. The note derives closed-form coordinate-descent updates for all three penalties and reports that SCAD and MCP converge about three times slower than Lasso with unstable loss variation across iterations. On the rat eye gene-expression data, under cross-validated tuning, SCAD yields a sparser model with lower residual sum of squares than Lasso.","pith_inferences":["A natural extension of the derivative-based explanation is that any penalty whose derivative reaches zero for large coefficients should reproduce the sparsity-without-bias pattern, while any constant-derivative penalty should inherit Lasso's bias.","Because Section 3 omits the shape parameter $a$, convergence tolerance, and initialization, rerunning the simulation across $a$ values and random starts would separate penalty behavior from optimizer behavior.","The real-data conclusion rests on a single data set and a single cross-validation run; repeating the comparison with repeated CV splits or multiple data sets would show whether the sparser SCAD fit is stable.","A practical extension the paper does not test is initializing SCAD or MCP from a Lasso solution, which could reduce the reported convergence time while keeping the favorable sparsity-accuracy profile."],"forward_implications":["In high-dimensional sparse linear models, Lasso forces a trade-off: a penalty level that recovers true sparsity over-shrinks the nonzero coefficients and inflates MSE.","With SCAD or MCP, a single penalty level can deliver both the ground-truth sparsity level and low estimation error in the simulated setting.","Non-convex penalties carry a computational cost: coordinate descent takes roughly three times as long on the toy problem and shows unstable loss variation.","On the rat gene-expression data, SCAD gives a sparser model with lower residual sum of squares than Lasso under cross-validated tuning.","The practical guide is to prefer Lasso when fast convex optimization matters and SCAD or MCP when bias on large coefficients matters.","The choice between convex and non-convex penalties is not about one being universally better; it depends on whether accuracy or computational feasibility is the binding constraint."],"supporting_citations":[{"why":"Defines the Lasso penalty $p_\\lambda(t)=\\lambda|t|$ and the convex regularization framework the paper compares against.","marker":"Tibshirani (1996)"},{"why":"Introduces the SCAD penalty and the oracle-property argument for nonconcave penalized likelihood.","marker":"Fan and Li (2001)"},{"why":"Introduces the MCP penalty and its nearly unbiased variable selection properties.","marker":"Zhang et al. (2010)"},{"why":"Cited for the claim that Lasso introduces bias on large components.","marker":"Zou and Hastie (2005)"},{"why":"Cited for Lasso's model-selection-consistency limitations that motivate non-convex penalties.","marker":"Zhao and Yu (2006)"},{"why":"Supplies the rat eye gene-expression data set used in the real-data comparison.","marker":"Scheetz et al. (2006)"}],"fun_headline_variants":["SCAD and MCP cut Lasso bias, but slow convergence","Non-convex penalties: sparser models, but slower","Lasso's bias fixed by SCAD, MCP at speed cost","True sparsity without Lasso bias? Use SCAD or MCP","Oracle properties vs Lasso: non-convex wins on bias"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the coordinate-descent solutions for SCAD and MCP show the penalties' true behavior rather than a quirk of the optimizer: Section 3 shows unstable loss variation and does not report the shape parameter, the convergence tolerance, or the starting values, so if the optimizer settles in a poor local optimum the claimed sparsity without accuracy loss could be an algorithmic accident.","fun_headline_variants_meta":{"raw":{"variants":["SCAD and MCP cut Lasso bias, but slow convergence","Non-convex penalties: sparser models, but slower","Lasso's bias fixed by SCAD, MCP at speed cost","True sparsity without Lasso bias? Use SCAD or MCP","Oracle properties vs Lasso: non-convex wins on bias"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000168,"raw_usage":{"total_tokens":1208,"prompt_tokens":840,"completion_tokens":368,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":456,"completion_tokens_details":{"reasoning_tokens":278}},"tokens_in":456,"tokens_out":368,"duration_ms":4022,"temperature":1.0,"reasoning_tokens":278,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:00:38.012724+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the n=200, p=1000 simulation with SCAD and MCP over a grid of shape parameters (e.g., $a=2.1,3.7,10$) and several random initializations; if some runs stop at visibly higher loss or sparsity levels far from 10, the claimed sparsity-without-accuracy-loss is an optimizer artifact rather than a penalty property. A cleaner check is to solve small instances to certified global optimality and compare the coordinate-descent solution against the global solution.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the MCP penalty and its nearly unbiased variable selection properties."},{"cited_title":"and Hastie, T","cited_arxiv_id":null,"evidence_quote":"Cited for the claim that Lasso introduces bias on large components."},{"cited_title":"E., Kim, K.-Y","cited_arxiv_id":null,"evidence_quote":"Supplies the rat eye gene-expression data set used in the real-data comparison."}],"review_version":1}