{"id":"e77dbaca-ce41-4946-8e6f-22ce695f69e8","arxiv_id":"2602.15253","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Masked-reconstruction transformers on single-cell RNA show power-law scaling with model size (α≈0.27) only in a data-rich regime; with few cells, scaling disappears.","lead":"This paper measures how well masked transformer models reconstruct hidden gene activity in single-cell RNA data as models grow, and finds power-law improvement only when the training set is large. The result offers foundation-model builders a first rule of thumb: with limited cells, bigger models do not help.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Regime B's flat scaling cannot be attributed to data scarcity while V and D change together; the central comparative claim needs matched-vocabulary controls.","rationale":"The paper's headline is the dual-regime contrast. The Regime A power law is plausible from the data, though the fit is fragile; the novel and strong claim is the 'only when sufficient data' part. That comparative claim requires that Regime B's flat curve is caused by low D, not by the simultaneous change in V. The paper itself flags the confound, but still states the conclusion unqualified. A matched-vocabulary D=10k experiment would directly test whether data scarcity alone reproduces the plateau. The existence of unconverged runs in the Regime B fit is a second, compounding issue; excluding them in a re-analysis is cheap and should be reported. These are addressable, so conditional acceptance is appropriate.","tokens_in":11248,"tokens_out":7906,"duration_ms":72222,"concrete_test":"Run the six canonical Regime A model sizes (XXS–L) on a V=512, D=10,000 subset with the same preprocessing, hyperparameters, and 60k steps. If α remains ≈0.27, the Regime B plateau is attributable to the V=1,024 task, not data scarcity; if α collapses to ≈0, the data-limited interpretation is supported at fixed vocabulary. Report α with bootstrap or profile-likelihood confidence intervals.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines Regime A (V=512, D=200k) and Regime B (V=1024, D=10k), which differ on two axes simultaneously. Section 7.2.2 interprets the flat Regime B fit (α=0.009, R²=0.017) as evidence that 'data scarcity—not model capacity—is the binding constraint,' and Section 10 concludes scaling emerges 'only when sufficient data are available relative to model capacity.' This inference requires that the A/B difference isolates D (or D/P). It does not: the larger vocabulary changes the reconstruction task (more output heads, different per-gene expression statistics after HVG selection). The observed lower absolute MSE in Regime B (Table 2) suggests the tasks are not comparable, so the absence of scaling could reflect task difficulty or D/V ratio, not D alone. The paper acknowledges the confound in Section 9 but still draws the comparative conclusion. Additionally, Regime B fits include runs trained for ≤400 steps (Section 7.2.2) and heterogeneous durations, which could flatten the curve independently of data scarcity. Both issues are acknowledged, but the central claim rests on the Regime B plateau being a genuine data-limited effect.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a first systematic study of neural scaling laws for masked-reconstruction transformers trained on single-cell RNA-seq data. Using CELLxGENE Census data, the authors construct two regimes: a data-rich regime (V=512 genes, D=200,000 cells) and a data-limited regime (V=1,024 genes, D=10,000 cells). They train seven model sizes from 533 to 3.4e8 parameters and fit the parametric scaling law L = a P^{-α} + c to validation MSE. Regime A yields α≈0.27, c≈1.44, R²=0.86 for the canonical 18-run set; Regime B yields α≈0.009, R²=0.02. The authors interpret this contrast as evidence that data scarcity, not model capacity, is the binding constraint, and they convert the Regime A floor into an entropy estimate of ~2.30 bits per masked gene position. The manuscript includes a reproducibility statement, public code repository, and extensive discussion of limitations.","tokens_in":11612,"tokens_out":3691,"duration_ms":37958,"significance":"If the central comparative claim held, this would be a useful first result for the single-cell foundation-model community: it would establish that parameter scaling of pretraining loss can occur in transcriptomic masked reconstruction, and it would quantify a data-limited plateau that argues for dataset curation over blind model scaling. The paper's strengths include a reproducible setup (public code, deterministic splits, fixed seeds), a clear parametric fit for the data-rich regime, and an unusually candid limitations section that identifies the confounded regime comparison and the derived nature of the Gaussian NLL. These strengths are real but do not by themselves establish the headline claim, because the two-regime comparison changes both V and D simultaneously, and the entropy estimate is not an independent measurement.","major_comments":[{"comment":"The central claim that Regime B's flat scaling demonstrates 'data scarcity—not model capacity—is the binding constraint' rests on a confounded design: Regime A (V=512, D=200k) and Regime B (V=1024, D=10k) differ in both vocabulary size and dataset size. The larger vocabulary changes the reconstruction task itself, and Table 2 shows lower absolute MSE in Regime B, so the two tasks are not directly comparable. Section 9 acknowledges this confound, but Sections 7.2.2 and 10 still draw the data-limited conclusion. A matched-vocabulary experiment (e.g., V=512 at D=10k, 50k, 200k) or at least a same-V D-sweep is required before the plateau can be attributed to dataset size rather than task difficulty.","section":"§3.2, §7.2.2, §10"},{"comment":"The Regime B fit (α=0.009, R²=0.017) is computed from runs with heterogeneous training durations, including runs trained for as few as 400 steps that may not have converged. The manuscript itself says this 'further obscures any underlying scaling trend.' As reported, the flat curve is therefore not a clean estimate of the data-limited scaling behavior; it conflates non-convergence, variable compute, and task differences. The authors should rerun Regime B with standardized steps, batch size, and convergence checks across all model sizes and multiple seeds before using this plateau as evidence.","section":"§7.2.2"},{"comment":"The canonical Regime A fit uses 18 runs but only six distinct parameter counts; the three seeds per size contribute replication but do not add independent evidence for the trend across P. No confidence intervals are reported for α or c. Given that the entire data-rich scaling claim rests on this fit, bootstrap or profile-likelihood intervals are needed, and a denser sweep in the 10^5–10^7 range (as the authors themselves suggest in §9) would materially strengthen the result. As it stands, R²=0.86 over six parameter values is suggestive but not definitive.","section":"§6, Table 3"},{"comment":"The 2.30-bit entropy estimate is not an independent result. The Gaussian NLL values are derived post hoc from the best MSE values (provenance label 'derived_from_best_mse'), so the agreement between the two entropy derivations is a mathematical consequence of the Gaussian assumption, not independent validation. The Gaussian residual assumption is also unjustified for log(1+x)-normalized expression counts, which are non-Gaussian and heteroscedastic. The paper acknowledges this in §7.4.3, but the abstract and conclusion present the 2.30-bit figure without the needed caveats. Either remove it from the headline claims or label it explicitly as a conditional calculation, not a measurement.","section":"§7.4, Appendix A"}],"minor_comments":[{"comment":"The abstract reports 'α = 0.234, R² = 0.823' for the data-rich regime, while Table 3 and §7.2.1 report the canonical α = 0.266, R² = 0.858, and the conclusion uses α≈0.27, R²=0.86. These should be reconciled, and the choice of the canonical 18-run fit should be stated in the abstract.","section":"Abstract"},{"comment":"The abstract says parameter counts span 'three orders of magnitude', but the range 533 to 3.4×10^8 spans about 5.8 orders of magnitude. §4.5 correctly says 'approximately five orders of magnitude.' Fix the abstract.","section":"Abstract / §4.5"},{"comment":"Typo: 'adata-richregime' should be 'a data-rich regime'.","section":"Abstract"},{"comment":"Training durations are inconsistent: §5.2 says Regime B uses 20,000–30,000 steps, while §7.2.2 says 20,000–60,000 steps. Clarify which is correct and include per-run step counts in the reproducibility tables.","section":"§5.2 / §7.2.2"},{"comment":"The XXS parameter count is listed as 534 in Table 1 but the abstract/text says 533. Check the count and ensure consistency.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The authors are unusually honest about the limitations, and the Regime A scaling fit is a plausible first observation. However, the headline comparative result—data scarcity as the binding constraint—is currently built on a V/D confound, and the Regime B plateau is further contaminated by non-converged runs. The entropy estimate is circular by the authors' own admission. These are fixable with additional experiments or a substantially reframed manuscript, but the central claim as written is not yet established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead the Kendiukhov paper on scaling laws for masked-reconstruction transformers on scRNA-seq. Worth knowing two things upfront: the Regime A result (power-law scaling with α≈0.27, R²=0.86, floor c≈1.44 over 6 model sizes × 3 seeds) is a plausible and genuinely new finding for this domain; the Regime B plateau (flat scaling) is not yet evidence that data scarcity is the binding constraint, because V and D change together and several runs look unconverged.\n\nWhat the paper does well: it is the first systematic scaling-law study for single-cell masked reconstruction, it ships code and data, and it is refreshingly explicit about its own limitations. Section 9 admits the confounded regime comparison and the fact that the Gaussian NLL values are derived from MSE ('derived_from_best_mse'), so the 2.30-bit entropy estimate is self-consistent rather than independently validated. The canonical Regime A fit is the strongest part; the floor is consistent across two fit variants and the flattening beyond ~20M parameters is internally coherent.\n\nSoft spots, in proportion: the main one is the claim that Regime B demonstrates data scarcity. The stress-test note is right: V=512/D=200k vs V=1024/D=10k differ on two axes, and the paper's own Table 2 shows lower absolute MSE in Regime B, which suggests the tasks are not comparable. The paper acknowledges this but still states in the conclusion that 'data scarcity is the binding constraint'—that inference does not follow from these experiments alone. Also, Regime B runs have heterogeneous training durations including ≤400-step runs; the paper says this may obscure trends, but then still interprets the flat curve as a data-limited effect. So the central comparative claim needs matched-vocabulary controls (e.g., V=512 and V=1024 at multiple D) before it lands.\n\nMinor: no confidence intervals on α and c; the R²=0.86 is lower than typical LM scaling, which is fine but means the exponent is approximate. The entropy estimate is framed as preliminary and the caveats are in Section 7.4.3, so that's a fair presentation.\n\nWho this is for: practitioners building single-cell foundation models who want a first calibration point; also people working on scaling laws in new modalities. It deserves a serious referee—the Regime A result is useful and the limitations are addressable in a revision. I wouldn't desk reject it.\n\nRecommendation: send to peer review, but the revision should either drop or heavily qualify the Regime B conclusion, and ideally add matched datasets.","headline":"Honest first measurement of scaling laws for single-cell transformers, but the data-limited plateau is confounded—the Regime A power law is the solid part.","tokens_in":12031,"tokens_out":2322,"would_cite":true,"duration_ms":19548,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Power-law scaling in single-cell gene-expression transformers emerges only in data-rich regimes, and the asymptotic floor corresponds to about 2.3 bits of entropy per masked gene.","keywords":["scaling laws","single-cell RNA sequencing","masked reconstruction","transformer","power law","data-to-parameter ratio","entropy estimation","foundation models"],"falsifier":"Train the same architecture at V=512 with D=10,000 cells, running each model to convergence with identical hyperparameters and reporting best validation MSE; if this shows clear power-law scaling, then the Regime B plateau is driven by vocabulary size or training noise, not by data scarcity. Alternatively, measure the fitted exponent α for a fixed V=512 at several D values (10k, 50k, 200k) and check whether α increases monotonically with D.","tokens_in":11124,"feed_emoji":"🧬","tokens_out":7462,"duration_ms":63578,"temperature":0.7,"pith_summary":"The paper asks whether the power-law scaling behavior documented for language and vision models carries over to masked-reconstruction transformers trained on single-cell RNA sequencing data. By fitting L = aP^(-α) + c to validation mean squared error across seven model sizes, it finds clear scaling in a data-rich regime (200,000 cells, 512 genes) with α ≈ 0.27 and an irreducible floor c ≈ 1.44, but essentially none in a data-limited regime (10,000 cells, 1,024 genes) with α ≈ 0.009. The contrast is interpreted as evidence that data scarcity, not model capacity, is the binding constraint, and the floor is converted to a preliminary estimate of about 2.30 bits of entropy per masked gene position. A sympathetic reader cares because this gives the single-cell foundation-model community a quantitative way to decide whether adding parameters or adding cells is the better investment, and it provides a first benchmark for how predictable gene expression is from co-expression context.","feed_headline":"Power-law scaling in single-cell RNA emerges only with enough data","feed_subtitle":"With 200,000 cells, loss drops as a power law; with 10,000 it plateaus, so data not model size is the limiter.","key_machinery":"The load-bearing object is the masked-reconstruction transformer: a permutation-invariant encoder that treats a cell's gene expression values as an unordered set, masks 15% of genes, and reconstructs them from the rest using a linear per-gene head, trained with mean squared error. The scaling analysis fits the parametric law L(P) = aP^(-α) + c to the best validation MSE at each parameter count; the exponent α is the scaling rate and c is the irreducible floor. The study's interpretive axis is the data-to-parameter ratio, operationalized as the ratio of cells to genes (D/V), which separates the data-rich regime (≈390 cells per gene) from the data-limited one (≈10).","core_discovery":"In its own terms, the central discovery is that the validation loss of a masked-reconstruction transformer on single-cell transcriptomes obeys a power-law scaling law with an irreducible loss floor in a data-rich setting, and that the same architecture stops scaling once data are scarce. Specifically, in the data-rich regime the fitted exponent is α ≈ 0.27 (R² = 0.86) with floor c ≈ 1.44 MSE, while in the data-limited regime α ≈ 0.009 with R² = 0.017, meaning parameter count explains almost none of the variance. The authors derive from the floor an entropy estimate of approximately 2.30 bits per masked gene position, with the explicit caveat that this is preliminary because the Gaussian NLL","pith_inferences":["A direct test of the data-scarcity interpretation would be to keep the gene vocabulary fixed at 512 and vary only the number of cells (e.g., 10k, 50k, 200k); if the scaling exponent grows with cell count, the plateau in the paper's Regime B is truly data-limited.","The two entropy derivations agree because they both start from the same MSE floor; a model trained from the start to predict a per-gene variance (Gaussian NLL) would give an independent estimate. If that floor differs materially, the homoscedastic Gaussian assumption is inadequate.","The 2.3-bit figure bundles biological noise, technical noise, and information discarded by selecting only highly variable genes; comparing floors across vocabulary sizes or normalization schemes could separate these contributions.","If the data-limited plateau replicates, many current single-cell foundation models trained on modest atlases are likely over-parameterized, and compute-optimal training would shift emphasis to much larger curated datasets rather than larger architectures."],"forward_implications":["On a roughly 200,000-cell, 512-gene dataset, model sizes beyond about 20 million parameters offer no further loss reduction; the fitted curve is flat by that point.","When the training set is small relative to the gene vocabulary, scaling up the model does not improve validation MSE, so data collection is the priority.","The fitted floor c ≈ 1.44 MSE (or ~2.3 bits per masked position) gives a concrete reference value for judging whether a new model or preprocessing pipeline is approaching the limit of predictability.","If the scaling law holds, small-scale runs can be used to extrapolate the eventual loss of larger models on the same data regime, informing resource allocation before expensive training."],"fun_headline_variants":["Scaling laws hold for single-cell transformers only when data is ample","Data-rich single-cell models obey power law; data-poor don't","Cell transcriptomics scaling: data, not size, drives gains","Power-law loss emerges in single-cell RNA with 200k cells","Single-cell transformer scaling hinges on data-to-parameter ratio"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim that the Regime B plateau is caused by data scarcity assumes that the two regimes differ only in data-to-parameter ratio, but they also differ in gene vocabulary (512 vs 1,024) and in training duration, with some runs trained for as few as 400 steps; if the larger vocabulary makes reconstruction harder or if unconverged runs flatten the curve, the plateau might not be about data at all.","fun_headline_variants_meta":{"raw":{"variants":["Scaling laws hold for single-cell transformers only when data is ample","Data-rich single-cell models obey power law; data-poor don't","Cell transcriptomics scaling: data, not size, drives gains","Power-law loss emerges in single-cell RNA with 200k cells","Single-cell transformer scaling hinges on data-to-parameter ratio"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000489,"raw_usage":{"total_tokens":2287,"prompt_tokens":828,"completion_tokens":1459,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":1370}},"tokens_in":572,"tokens_out":1459,"duration_ms":9175,"temperature":1.0,"reasoning_tokens":1370,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T22:53:36.978397+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same architecture at V=512 with D=10,000 cells, running each model to convergence with identical hyperparameters and reporting best validation MSE; if this shows clear power-law scaling, then the Regime B plateau is driven by vocabulary size or training noise, not by data scarcity. Alternatively, measure the fitted exponent α for a fixed V=512 at several D values (10k, 50k, 200k) and check whether α increases monotonically with D.","supporting_citations":[],"review_version":1}