{"id":"87b25c88-ccc5-40fe-98d7-ba1c74e87391","arxiv_id":"2501.04639","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A fast, differentiable emulator for EFTofLSS galaxy power spectra, validated against pybird on simulations and BOSS data, enables gradient-based MCMC inference.","lead":"Effort.jl is a new Julia package that rapidly computes galaxy power spectrum predictions for the Effective Field Theory of Large Scale Structure, and it supports automatic differentiation for gradient-based Bayesian sampling. The authors show it produces cosmological posteriors consistent with the established pybird code, but in minutes instead of hours.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PT-challenge validation is intentionally opaque and BOSS agreement is shown only qualitatively, so the claimed Monte-Carlo-level posterior agreement is not auditable; convergence diagnostics and quantitative posterior statistics are missing.","rationale":"The paper delivers a genuinely useful tool: the architecture, preprocessing, and accuracy checks in Sec. 4.1 are credible, and the released code makes the emulator's sub-percent accuracy independently checkable. The weakest assumption is exactly what the reader identified: the posterior validation chains are assumed converged and the agreement is assumed quantitative, but the evidence is qualitative and the most stringent test is blinded. The paper explicitly states that chains used only 2,000 accepted NUTS steps and that no direct efficiency benchmark against pybird was run; both statements confirm the gaps. These are addressable with additional diagnostics and summary statistics rather than fatal flaws, so CONDITIONAL is the appropriate verdict. My concern matches the reader's weakest_assumption, with the addition that PT-challenge blinding compounds the lack of quantitative comparison.","tokens_in":19521,"tokens_out":1340,"duration_ms":14954,"concrete_test":"Provide a quantitative comparison table for both PT-challenge (with blinded labels if necessary) and BOSS: for each cosmological parameter, report posterior mean and 68% credible interval from pybird, Effort+NUTS, and Effort+MCHMC, along with Gelman-Rubin R-hat and bulk/tail ESS for each chain. If all R-hat values are below 1.01 and all normalized mean differences are below 0.1 sigma, the concern is retired. Also report the maximum emulator-vs-pybird P(k) deviation on a fixed-parameter test grid across the training volume to confirm the sub-percent accuracy claim directly.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim—that Effort.jl reproduces pybird's posteriors with deviations compatible with MonteCarlo noise—rests on Sec. 4.2. Two gaps make this claim load-bearing rather than merely cosmetic. First, the PT-challenge emulator is withheld and the PT-challenge triangle plots have axes removed for blinding, so the most stringent large-volume test cannot be checked quantitatively by the reader; 'excellent agreement' is asserted but not demonstrated numerically. Second, for both PT-challenge and BOSS, agreement is presented only as overlaid contours, with no per-parameter mean/error differences, no Gelman-Rubin R-hat, and no ESS values for the reported 2,000 accepted NUTS steps per chain (Sec. 4.2). Without convergence diagnostics, an un-mixed chain or a small-but-real emulator bias (e.g., 0.1–1% in P(k)) could be masked by the small number of samples, especially given the PT-challenge's enormous volume makes it a stringent test. The internal text itself notes that no direct sampling-efficiency comparison with pybird was made, so the performance claim is also indirect. None of this shows the emulator is wrong; it shows the validation evidence is weaker than the stated conclusion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Effort.jl, a neural-network emulator for the one-loop EFTofLSS galaxy power spectrum multipoles, integrated with automatic differentiation and the Turing.jl probabilistic programming framework. The emulator treats bias and counterterm parameters analytically, uses a physics-motivated rescaling by AsD^2(z) as preprocessing, and implements the Alcock-Paczynski effect and window-mask convolution with custom differentiation rules. The authors validate the emulator by comparing Bayesian posteriors obtained with Effort.jl plus NUTS/MCHMC against pybird with MontePython on the PT-challenge simulations and on BOSS data, reporting sub-percent emulator residuals and wall-clock reductions from days or hours to about ten minutes or one hour on a laptop. The conclusion is that Effort.jl reproduces pybird posteriors with deviations compatible with Monte Carlo noise.","tokens_in":19748,"tokens_out":3220,"duration_ms":33391,"significance":"If the validation holds, Effort.jl is a useful community resource: it is publicly released, differentiable, and fast enough to enable gradient-based sampling for full-shape EFTofLSS analyses of DESI and Euclid data. The paper's technical contributions are real: the preprocessing rescaling demonstrably improves emulator accuracy (Sec. 4.1, Figs. 2 and 3), the custom backward rules for spline interpolation and window convolution solve a genuine AD bottleneck (Sec. 2.4.2), and the symbolic-regression growth factor reduces the preprocessing overhead. The code availability and the careful treatment of observational effects are strengths. The central weakness is that the headline posterior-agreement claim is supported only by overlaid contour plots, with no quantitative discrepancy metrics or convergence diagnostics.","major_comments":[{"comment":"The abstract's central claim—'deviations compatible with MonteCarlo noise'—is not quantitatively demonstrated. The evidence is limited to overlaid triangle plots, without per-parameter posterior means and errors, shifts in units of standard deviation, or any metric such as the maximum difference in contours. Given that the chains use only 2,000 accepted NUTS steps per chain (Sec. 4.2), statistical noise is nontrivial, and a small but real emulator bias (e.g., 0.1–1% in P(k), which is within the reported residual range of Fig. 2) could be masked. The paper should add quantitative posterior comparison statistics and, ideally, a statement of whether the observed differences are within the expected Monte Carlo scatter.","section":"Sec. 4.2, Figs. 4 and 5"},{"comment":"The PT-challenge validation is not auditable in its current form: the dedicated 3-parameter emulator is not released, axis ticks and labels are removed from Fig. 4, and no numerical summary of the posteriors is provided. While the blinding protocol explains the withheld cosmology, it does not prevent reporting blinded summary statistics (for instance, differences in posterior means relative to the posterior width under a fixed labeling convention). Without such numbers, 'excellent agreement' for the largest-volume and most stringent test remains an assertion rather than a demonstrable result.","section":"Sec. 4.2, PT-challenge"},{"comment":"The performance comparison is incomplete. The text reports ESS/s values for Effort.jl (1.2 for NUTS and 5.1 for MCHMC on the PT-challenge; 0.4 and 2.4 on BOSS) and states that pybird required 'several hours' or 'a few days' on a cluster, but also notes that 'a direct comparison of sampling efficiency with pybird was not made.' Without a like-for-like comparison on the same platform, with the same number of chains and the same convergence criteria, the stated orders-of-magnitude improvement is not established. The runtime claims should be either substantiated with a controlled benchmark or softened.","section":"Sec. 4.2, performance claims"},{"comment":"The manuscript does not report any convergence diagnostics for the Effort.jl chains, such as Gelman-Rubin R-hat, effective sample size (ESS), or trace plots. Given that 2,000 accepted NUTS steps per chain may not guarantee convergence for a high-dimensional EFT likelihood, the absence of such diagnostics makes it difficult to rule out the possibility that the agreement with pybird is partly due to chains that have not yet mixed. The authors should report R-hat and ESS for all parameters and state clearly how convergence was assessed.","section":"Sec. 4.2, convergence diagnostics"}],"minor_comments":[{"comment":"The phrase 'via the Metropolis-Hastings sampler' for pybird is imprecise; MontePython may use MH or other samplers, and the paper itself does not specify the exact sampler. Clarify the comparison setup.","section":"Abstract and Sec. 1"},{"comment":"There is a typo in 'conjuction' and in Sec. 2.3 'the dataset as been split' should read 'has been split'.","section":"Sec. 2.1"},{"comment":"The sentence beginning 'This rescaling is particularly effective for extended models' repeats the phrase 'which primarily influence the amplitude of the linear matter power spectrum' twice; remove the duplicate.","section":"Sec. 2.3"},{"comment":"In Eq. (2.11), the definitions of q_parallel and q_perp are dimensionally inconsistent as written: q_parallel is given as DA(z)H(z=0) divided by a reference quantity, which is dimensionless, while q_perp is written as a ratio of Hubble parameters; please check the intended factors of H(z) and H(0) in the notation (see [68, 69]).","section":"Sec. 2.4.2"},{"comment":"The text states that the symbolic growth factor is accurate to 0.1% for 99.87% of the validation dataset, but the maximum error over the remaining 0.13% is not reported; a worst-case error bound would be more informative for downstream inference.","section":"Sec. 4.1, Fig. 3"},{"comment":"The removal of axis ticks and labels in Fig. 4 makes it impossible for the reader to judge the scale of any differences; consider reporting the same information in a blinded table or in a separate figure with a private labeling scheme.","section":"Sec. 4.2, Fig. 4"},{"comment":"The phrase 'we are actively working on ajax-based version' appears to be a typo for 'a JAX-based version'; also, 'see thus advantageous' should read 'find it advantageous'.","section":"Sec. 5"}],"recommendation":"major_revision","confidential_remarks":"The paper's technical core is solid and the emulator is likely to be useful, but the main validation claim is presented in a way that is not quantitatively verifiable. The PT-challenge blinding is a legitimate constraint, but it does not prevent reporting posterior summary statistics or convergence diagnostics; the authors should be pushed to provide them. I do not see evidence of circular reasoning or fabricated results; the concern is purely about the strength and auditability of the evidence. This is fixable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nRead the Effort.jl paper. Bottom line: this is a genuinely useful technical contribution, and the central claim is probably right but currently under-supported in the validation section.\n\nWhat's new: first EFTofLSS power spectrum emulator explicitly designed for and demonstrated with gradient-based samplers (NUTS and MCHMC) through Turing.jl. The physics-based preprocessing—rescaling P11 and counterterms by AsD^2(z) and loop terms by As^2D^4(z), then letting the NN learn residuals—is a thoughtful reduction of dynamic range and demonstrably improves accuracy. The custom backward rules for the AP-effect spline and window convolution, cutting gradient cost by three orders of magnitude, are careful engineering. The accuracy section is quantitative: 95% residual distributions for different rescalings and training sizes, with sub-percent errors. The code is public, which is real evidence.\n\nSoft spots, in order of severity. First, the posterior validation is presented as overlaid triangle plots with no numbers. There is no per-parameter mean/error difference between Effort and pybird, no Gelman-Rubin, no ESS values for the reported 2,000 accepted NUTS steps per chain. For a paper whose abstract says 'deviations compatible with MonteCarlo noise,' that is a load-bearing claim, and it is not auditable from the plots alone. Second, the PT-challenge emulator is withheld and axis labels removed for blinding. That is a legitimate choice, but it makes the most stringent test impossible to check independently; the reader has to take 'excellent agreement' on faith. Third, the speed comparison is indirect: the paper says no direct sampling-efficiency comparison with pybird was made, then claims 'several orders of magnitude.' That overreach should be softened.\n\nNone of this suggests the emulator is wrong. The consistency check against pybird is the right test, and the plots are suggestive. But the validation needs quantitative diagnostics before the claim 'agreement compatible with MonteCarlo noise' can be accepted.\n\nWho's this for: anyone doing full-shape LSS analyses with DESI/Euclid, and people building differentiable emulators. It deserves a serious referee; the issues are addressable in revision. I'd send it to review.\n\nBest.","headline":"A useful, well-engineered differentiable EFTofLSS emulator whose headline posterior-agreement claim still needs quantitative support; worth refereeing.","tokens_in":20316,"tokens_out":2245,"would_cite":true,"duration_ms":20031,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":["98.80.-k","98.65.-r"],"model":"deepseek-v4-flash","headline":"Effort.jl is a neural-network emulator for one-loop galaxy clustering that reproduces pybird's BOSS posteriors while running on a laptop.","keywords":["EFTofLSS","power spectrum emulator","galaxy clustering","Hamiltonian Monte Carlo","automatic differentiation","Bayesian inference","BOSS survey","large-scale structure"],"falsifier":"Compute Gelman-Rubin and effective sample sizes from the saved Effort chains for the PT-challenge and BOSS runs; if any cosmological parameter has $\\hat{R}>1.05$, or if rerunning with ten times more accepted steps shifts the mean by more than the reported Monte Carlo error, the claim that Effort reproduces pybird posteriors is not yet demonstrated.","tokens_in":19305,"feed_emoji":"🌌","tokens_out":9270,"duration_ms":84546,"temperature":0.7,"pith_summary":"Effort.jl is a fast, differentiable surrogate for the one-loop Effective Field Theory of Large-Scale Structure (EFTofLSS) galaxy power spectrum multipoles. The paper's central claim is that it computes each multipole in about 15 microseconds and produces Bayesian posteriors that agree with the standard pybird pipeline on the PT-challenge simulations and BOSS data, with differences at the level of Monte Carlo noise. This matters because full-shape cosmological analyses that normally take days on a cluster can be run in roughly an hour on a laptop, making gradient-based sampling practical for the high-dimensional parameter spaces of upcoming surveys.","feed_headline":"Emulator cuts galaxy-clustering inference from days to one hour","feed_subtitle":"A differentiable surrogate for EFTofLSS reproduces pybird's BOSS posteriors while running on a laptop.","key_machinery":"The central object is the factorization $P_\\ell(k;\\theta)=\\sum_{i,j} b_i b_j P_{ij,\\ell}(k;\\theta)+S_\\ell(k)$: the neural network learns only the cosmology-dependent component kernels $P_{ij,\\ell}$, while bias and counterterm parameters and the stochastic piece $S_\\ell$ are handled analytically, exactly as in the reference EFT code. Around that factorization, the paper assembles three mechanisms: a preprocessing step that rescales outputs by $A(z)$ and $A^2(z)$ to remove the dominant amplitude dependence, with the growth factor either solved as a differentiable ODE or replaced by a symbolic-regression expression; explicit modeling of Alcock-Paczynski distortions with custom backward differentiation that exploits the sparsity of spline Jacobians, cutting gradient cost from about 100 ms to 200 $\\mu$s; and a window-mask convolution implemented as an efficient array contraction with custom differentiation rules. This differentiability is what lets Hamiltonian Monte Carlo samplers explore the posterior, while the analytic bias factorization keeps the neural network small enough to train on a CPU.","core_discovery":"On its own terms, this paper establishes that a carefully designed emulator can replace direct evaluation of the one-loop EFTofLSS power spectrum in full Bayesian inference without changing the inferred cosmology. The key demonstration is that posterior distributions sampled with Effort.jl and Hamiltonian Monte Carlo overlap with pybird posteriors from standard Metropolis-Hastings runs, on both the PT-challenge mock galaxy catalogs and the four BOSS power-spectrum multipoles; the agreement is described as consistent with Monte Carlo noise. The runtime drops to about ten minutes for the PT-challenge and just over an hour for BOSS on a laptop. Accuracy tests also show that physics-based rescaling of the emulator outputs by the amplitude factor $A(z)=A_s D^2(z)$, with loop terms scaled by $A^2(z)$, reduces emulation residuals far more effectively than simply enlarging the training set.","pith_inferences":["Beyond the paper, the same factorization is a natural template for emulating bispectrum loop terms, since Eq. (2.1) separates bias parameters from cosmology-dependent kernels in the same way a bispectrum likelihood would require; the paper does not claim this extension.","Beyond the paper, the symbolic-regression replacement of the growth-factor ODE suggests a general recipe for any differentiable pipeline: replace a continuous preprocessing bottleneck with a closed-form expression, accepting a small accuracy cost for portability; applying this to the AP mapping would remove the remaining interpolation bottleneck.","Beyond the paper, a sharper validation than visual contour overlap would be a quantitative comparison of Effort and pybird posterior means and credible intervals at substantially longer chain lengths, which would turn the Monte-Carlo-noise claim into a measured number.","Beyond the paper, the emulator's error behavior outside the validated range ($k_{\\rm max}$ up to roughly $0.12$-$0.23\\,h/{\\rm Mpc}$) is untested, so a natural stress test is to push to higher $k_{\\rm max}$ on N-body mocks and map where sub-percent accuracy degrades."],"forward_implications":["Full-shape EFTofLSS analyses of BOSS-like data become feasible on a laptop, with wall-clock time of about an hour rather than days.","Gradient-based samplers such as NUTS and microcanonical Hamiltonian Monte Carlo become practical for EFT parameter inference, including scenarios where analytical marginalization of nuisance parameters is not possible.","Emulators with analytic bias handling and physics-based rescaling can reach high accuracy with small networks, lowering training cost and hardware requirements relative to larger black-box models.","Because the emulator is differentiable and fast, it can be combined with other differentiable surrogates for joint multi-probe cosmological analyses.","The modular design allows the same workflow to be retrained on other one-loop EFT codes, extending the speedup to different theory implementations."],"supporting_citations":[{"why":"Supplies the blinded PT-challenge mock galaxy catalogs and the analysis settings used to validate the emulator's posteriors.","marker":"[51]"},{"why":"Supplies the BOSS DR12 galaxy sample that provides the second validation dataset.","marker":"[52]"},{"why":"Supplies the BOSS power spectrum multipoles, window matrices, and priors used in the comparison.","marker":"[82]"},{"why":"Establishes the amplitude-rescaling preprocessing strategy that Effort.jl extends to growth-factor rescaling.","marker":"[14]"},{"why":"Earlier emulator that, like Effort.jl, treats bias parameters analytically and motivates the architecture.","marker":"[20]"},{"why":"Defines the No-U-Turn Sampler used to draw the Effort.jl posterior chains.","marker":"[30]"},{"why":"Defines the microcanonical Hamiltonian Monte Carlo sampler used as the second gradient-based sampler.","marker":"[73]"},{"why":"Provides the window-mask convolution method that Effort.jl implements as a differentiable array contraction.","marker":"[70]"},{"why":"The MontePython sampler used for the pybird reference chains, establishing the runtime baseline.","marker":"[83]"}],"fun_headline_variants":["Effort emulator: BOSS posteriors in an hour, not days","Differentiable emulator matches pybird, runs on a laptop","Effort: differentiable emulator cuts EFT inference time","Emulator makes EFTofLSS inference laptop-fast","Laptop-ready emulator matches pybird EFT posteriors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the short Effort chains (2,000 accepted NUTS steps per chain) and the pybird chains have converged; the paper shows no convergence diagnostics, so if either set has not mixed, the claimed agreement could be an artifact of unfinished sampling.","fun_headline_variants_meta":{"raw":{"variants":["Effort emulator: BOSS posteriors in an hour, not days","Differentiable emulator matches pybird, runs on a laptop","Effort: differentiable emulator cuts EFT inference time","Emulator makes EFTofLSS inference laptop-fast","Laptop-ready emulator matches pybird EFT posteriors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001142,"raw_usage":{"total_tokens":4701,"prompt_tokens":866,"completion_tokens":3835,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":3748}},"tokens_in":482,"tokens_out":3835,"duration_ms":25558,"temperature":1.0,"reasoning_tokens":3748,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:27:26.591526+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute Gelman-Rubin and effective sample sizes from the saved Effort chains for the PT-challenge and BOSS runs; if any cosmological parameter has $\\hat{R}>1.05$, or if rerunning with ten times more accepted steps shifts the mean by more than the reported Monte Carlo error, the claim that Effort reproduces pybird posteriors is not yet demonstrated.","supporting_citations":[{"cited_title":"The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: RSD measurement from the LOS-dependent power spectrum of DR12 BOSS galaxies","cited_arxiv_id":"1509.06386","evidence_quote":"Supplies the BOSS DR12 galaxy sample that provides the second validation dataset."}],"review_version":1}