{"id":"72d8fe20-93d4-481f-8999-de6794d09327","arxiv_id":"2412.03547","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Neural networks trained on simulated rotation curves can infer ultra-light dark matter and baryonic parameters from SPARC dwarf galaxies, with uncertainties comparable to MCMC.","lead":"This paper trains neural networks to estimate dark matter and baryon parameters from galaxy rotation curves, testing them on seven dwarf galaxies from the SPARC catalog. It finds that networks trained with noisy simulated curves match Bayesian MCMC results, and the uncertainty estimates flag parameter combinations the data cannot pin down.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never shows that NN point estimates recover known true parameters; the MCMC agreement uses the same hand-tuned priors and thus is a consistency check, not an accuracy test.","rationale":"The reader's weakest-assumption analysis points to the hand-picked uniform ranges in Table 1, and that is a legitimate concern: the NN can only map rotation curves inside this data-informed training distribution, and parameters outside the ranges are unrecoverable. My stress-test agrees with this, but sharpens it: the more directly load-bearing gap is the absence of any quantitative ground-truth recovery test. The MCMC comparison is a strong internal-consistency check, but because the MCMC uses the same model, the same priors, and the same noise assumptions, it does not establish that the NN returns accurate or unbiased parameter estimates. The low reduced chi-square on observed curves is also partially guaranteed by the construction of the training ranges (Sec. 2.3 step 1). The paper's own caveats (Sec. 5.1 and Sec. 6) acknowledge that heteroscedastic uncertainties are not equivalent to Bayesian posteriors, which makes a coverage test on test data (where true parameters are known) essential. Figure 5 provides a coarse 3-sigma calibration check, but no formal 68% coverage or bias analysis is reported. A concrete test of the kind proposed here would settle whether the central claim holds: if the NN point estimates are unbiased and the 68% intervals cover at the correct rate on test data drawn from the same prior, the method is a faithful posterior emulator; if not, the claim that the NN 'extracts' parameters is unsupported. The reader's CONDITIONAL verdict already captures the need for additional validation, and my concern does not move the verdict; it strengthens the justification for the condition.","tokens_in":22879,"tokens_out":8350,"duration_ms":86535,"concrete_test":"For each of the seven galaxies, use the already-trained neural networks (noisy MSE and heteroscedastic) and generate 10,000 simulated noisy rotation curves from parameters drawn (a) uniformly from the Table 1 ranges and (b) from the emcee posterior samples computed in Sec. 5. Add Gaussian noise with the observed sigma per radius bin, feed the curves to the NNs, and compare predictions to the known true parameters. Compute per-parameter bias, RMSE, and the empirical coverage of the NN 68% (1-sigma) intervals for the heteroscedastic network. Also compute, for the MCMC chains, the fraction of posterior samples within 1% of each Table 1 boundary to test whether the hand-picked ranges truncate the posterior. If coverage is far from 68% for any parameter, or if any posterior is visibly truncated, the central claim that the NNs extract parameters and their uncertainties is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that trained networks can 'extract parameters that describe observations well.' The two supporting lines of evidence are (i) low reduced chi-square when predicted rotation curves are compared to observed ones, and (ii) agreement of point estimates with emcee MCMC. Neither line independently establishes that the NN recovers true parameters. Point (i) is partly guaranteed by construction: Sec. 2.3 step 1 selects the uniform parameter ranges in Table 1 galaxy-by-galaxy by visually matching simulated curves to the observed curve, so the training distribution is already anchored to the target. Point (ii) uses the same model, the same uniform priors, and the same noise model in the MCMC, so a well-trained NN under MSE loss is expected to approximate the posterior mean; agreement with MCMC is a consistency check, not an accuracy validation. What is missing is a quantitative test on simulated rotation curves with known true parameters. The paper reports test-set loss and a few example curves, but no bias, RMSE, or coverage statistics. This matters most for r_s and Upsilon_*, the parameters to which the rotation curves are insensitive: the headline observation that heteroscedastic uncertainties are large for these parameters is compelling, but without a coverage test one cannot tell whether the NN point estimates are merely uncertain or also biased, and whether the reported 1-sigma intervals have the claimed frequentist meaning. Thus the load-bearing gap is the lack of a ground-truth recovery test, which the hand-tuned prior range issue compounds rather than replaces.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains per-galaxy feed-forward neural networks to map rotation curves of seven SPARC dwarf galaxies to five parameters of an ultra-light dark matter (ULDM) model: particle mass m, scale parameter s, transition radius r_t, NFW scale radius r_s, and stellar mass-to-light ratio Upsilon_*. Training data are simulated rotation curves drawn from uniform parameter ranges chosen galaxy by galaxy in Table 1. The authors compare MSE training with and without injected observational noise, and they develop two uncertainty quantification schemes: multiple realizations of the observed curve passed through the network, and a heteroscedastic loss that predicts per-parameter variances. Validation consists of reduced chi-squared values of reconstructed rotation curves and a comparison of point estimates and 1-sigma intervals with emcee MCMC. The paper concludes that the trained networks can extract parameters that describe the observed rotation curves well.","tokens_in":23156,"tokens_out":5059,"duration_ms":50458,"significance":"If the claimed accuracy were established, the method would offer a useful fast emulator-like alternative to MCMC for this five-parameter ULDM model, and the heteroscedastic uncertainty treatment would be an interesting lightweight addition to the astrophysical machine-learning toolbox. The paper's strengths are its explicit internal consistency check against MCMC, its careful treatment of training-set noise, and its honest discussion of the limitations of the heteroscedastic loss (Section 5.1 and the caveats in Section 6). However, the MCMC comparison shares the same forward model, the same uniform priors, and the same Gaussian noise model, so agreement between the neural network and MCMC is a self-consistency test rather than an external accuracy test. The absence of quantitative ground-truth validation on the held-out simulated test set is the main weakness and is directly relevant to the central claim.","major_comments":[{"comment":"The test set of 5x10^4 simulated curves with known target parameters is split but never used to report quantitative recovery accuracy for the five parameters. Figure 2 shows only three example reconstructed curves, and the text reports only loss values and a qualitative 'visually similar' statement. Since the abstract's central claim is that parameters are 'extracted', the paper should report test-set metrics — for example, median bias, RMSE, and per-parameter correlation between predicted and true values — for each of m, s, r_t, r_s, and Upsilon_*. This is especially important for r_s and Upsilon_*, which the paper says have little influence on the rotation curves and for which the network uncertainties are large.","section":"Sections 2.4 and 3.1"},{"comment":"The MCMC comparison uses the same forward model (eqs. 2.1-2.6), the same uniform priors in Table 1, and the same Gaussian noise model as the neural network training. Therefore the agreement shown in Figure 9 and Table 2 is a consistency check between two methods that share the same generative assumptions; it does not establish that the network recovers the true parameter values. A network trained under the same priors is expected to approximate the prior-constrained posterior mean. The paper should explicitly frame the MCMC agreement as a self-consistency test and add a validation against simulated curves with known parameters, reporting coverage of the 1-sigma intervals for both the multiple-realizations and heteroscedastic methods.","section":"Section 5, Eq. (5.1)"},{"comment":"The uniform ranges for each parameter are chosen by visually comparing simulated curves with the observed rotation curve, as stated in step 1 of Section 2.3. This makes the training distribution data-informed. Consequently, the low reduced chi-squared values in Figures 3, 4, 6, and 7 are partly guaranteed by construction, and parameters outside the chosen ranges are unrecoverable by the network. The paper should state in the abstract and Section 6 that the inferred values and uncertainties are conditional on these hand-picked ranges, and ideally test the sensitivity of the results to reasonable variations of the Table 1 ranges, particularly for r_s and Upsilon_*.","section":"Section 2.3, step 1 and Table 1"},{"comment":"The heteroscedastic loss assumes that the five parameters are independent and Gaussian, a point the paper acknowledges. The calibration check in Figure 5 is qualitative: the text says that 'for most samples' the difference lies within three times the predicted uncertainty, but no fraction is quoted and no 1-sigma coverage statistic is computed. As a result, it is unclear whether the reported uncertainties have the claimed frequentist meaning. The authors should report, on the test set, the fraction of true parameter values contained within the predicted 1-sigma interval for each parameter and each uncertainty method, and compare these fractions with the expected 68% for calibrated Gaussian uncertainties.","section":"Section 4.2, Eq. (4.1), and Figure 5"}],"minor_comments":[{"comment":"There are several typographical errors: 'T able 1' in Section 2.3, 'obatined' in Section 4.1, 'discreet' in Section 2.2.1, and 'UGC 5271' in the caption of Figure 6 (should be UGC 5721).","section":"Throughout"},{"comment":"The MSE/multiple-realizations row for UGCA 444 reports 1-sigma uncertainties of about 0.01 kpc for r_t and 0.02 kpc for r_s, which are implausibly small compared with the other galaxies and contradict the paper's statement that the multiple-realizations method under-estimates uncertainties for insensitive parameters. This outlier should either be explained or explicitly flagged as a known failure of that method.","section":"Table 2, UGCA 444 row"},{"comment":"The notation for the heteroscedastic loss would benefit from clarification: the text defines s_ik as the log of the predicted variance, but the equation uses e^{-s_ik} without explicitly connecting the two in the displayed formula. Adding a short sentence defining s_ik in the equation would improve readability.","section":"Section 4.2, Eq. (4.1)"},{"comment":"The authors note in the discussion that the input vector contains only velocities and not the associated radii. This is an important limitation for the generality of the method and should also be mentioned in the abstract, so that readers do not overstate the approach's transferability to other galaxies or to rotation curves with different radial sampling.","section":"Section 6, caveat (c)"}],"recommendation":"major_revision","confidential_remarks":"The main gap is that the central 'extraction' claim is validated only through internal consistency with MCMC under identical priors and forward model, while the held-out test set with known parameters is available but not used for quantitative recovery or coverage statistics. I believe this is fixable within the scope of the paper by adding test-set bias/RMSE and 1-sigma coverage analyses, and by reframing the claims as conditional on the hand-chosen priors. No code or data release is mentioned; a reproducibility statement would strengthen the paper for a methods-oriented venue like JCAP, but this is not a blocker."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: a clean, honest proof-of-concept, not a breakthrough. The authors train per-galaxy NNs to map SPARC rotation curves to five ULDM/baryonic parameters, and the most useful result is that a heteroscedastic loss associates large uncertainties with exactly the parameters (r_s and Upsilon_*) to which dwarf rotation curves are insensitive. That is a genuinely useful diagnostic.\n\nWhat is new here is the specific application: NN-based parameter inference for the soliton-plus-NFW ULDM model, with noise-injection during training and two uncertainty quantification schemes, benchmarked against emcee. The MCMC comparison is done well: for most parameters and galaxies the NN point estimates and uncertainties sit inside the emcee intervals, and the corner plot for UGC 5721 shows the NN captures the main parameter correlations. The finding that noise-injected training improves fits to observed curves is also practically useful. The authors are also candid about caveats: they note heteroscedastic uncertainties are not Bayesian posteriors, runtime exceeds MCMC, and radius information is unused.\n\nThe soft spots are real but not fatal. The abstract says the networks 'extract parameters that describe observations well.' That exact claim is supported. But if 'extract' is read as recovering true parameters, the paper does not demonstrate it. There is no ground-truth test: no simulated galaxies with known parameters, no bias, RMSE, or coverage statistics. The MCMC agreement is a consistency check, not an accuracy test, because the MCMC uses the same forward model, the same noise model, and the same hand-picked uniform priors. Those priors (Table 1) were selected galaxy-by-galaxy by visually matching simulated curves to the observed curve, so the training distribution is anchored to the targets from the start. The heteroscedastic uncertainty behavior is suggestive, but without coverage statistics you cannot tell whether the large error bars are honest or merely broad. The stress-test note puts this exactly right.\n\nNo code or data is provided, which hurts reproducibility for a methods paper. The discarded radius information is acknowledged and is a minor issue in the per-galaxy setup.\n\nBottom line: this paper is a reasonable starting point for anyone wanting a fast approximate fitting tool for this specific ULDM model, and the uncertainty diagnostic is worth remembering. For physical inference from the inferred parameters, the missing recovery test is the load-bearing gap. I would send it to a referee, with that issue clearly flagged.\n\nRecommendation: engage with it, but treat the current version as conditional.","headline":"A clean proof-of-concept that NNs can match MCMC for ULDM rotation-curve fitting, but parameter recovery on known truths is never demonstrated.","tokens_in":23674,"tokens_out":3066,"would_cite":false,"duration_ms":27382,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Trained neural networks can take an observed dwarf-galaxy rotation curve and return dark matter halo and baryon parameters that describe the data as well as MCMC chains do.","keywords":["galactic rotation curves","SPARC galaxies","ultra-light dark matter","neural networks","parameter inference","heteroscedastic loss","MCMC comparison","dwarf galaxies"],"falsifier":"Generate a set of mock galaxies whose true parameters lie deliberately outside the Table 1 ranges, for example $r_t < 1$ kpc or $\\Upsilon_* > 0.8$, feed their rotation curves to the trained networks, and check whether the inferred parameters track the true values or instead collapse to the boundary of the training ranges; the latter would confirm that the reported agreement with MCMC is conditional on the chosen parameter box.","tokens_in":1859,"feed_emoji":"🌌","tokens_out":2829,"duration_ms":69538,"temperature":0.7,"pith_summary":"The paper asks whether a neural network can do what a Bayesian MCMC analysis does for galactic rotation curves: take an observed rotation curve and return the parameters of the dark matter halo and the stellar disk. Using seven dark-matter-dominated dwarf galaxies from the SPARC catalog, the authors generate about half a million simulated rotation curves per galaxy under an ultra-light dark matter (ULDM) core-plus-NFW-halo model, train a separate network for each galaxy, and then feed the observed curves to the trained nets. Their central claim is that the networks extract parameter values that describe the observations well: point estimates mostly fall within the 1-$\\sigma$ intervals of emcee MCMC, and a heteroscedastic loss function yields large uncertainties precisely for the parameters, $r_s$ and $\\Upsilon_*$, to which these rotation curves are insensitive. The paper itself cautions that these learned uncertainties are not equivalent to Bayesian posteriors and that training is still slower than MCMC, so the contribution is a complementary, likelihood-free route to parameter estimation rather than a replacement.","feed_headline":"Neural nets decode rotation curves into dark matter parameters","feed_subtitle":"Trained on simulated ULDM curves, the networks match MCMC estimates and flag unconstrained parameters.","key_machinery":"The load-bearing object is a fully connected neural network with two hidden layers of 200 ReLU neurons, trained per galaxy on $5\\times10^5$ simulated rotation curves generated from a ULDM core-halo density profile, namely a Schive-style soliton core matched continuously to an NFW envelope at a transition radius $r_t$. The input is the vector of observed velocities at the galaxy's $N_{\\rm obs}$ radius bins, and the target is the five-parameter vector $P$. Two additions carry the argument: adding Gaussian noise drawn from the SPARC error bars to each simulated input during training, which the paper equates with Tikhonov regularization and which makes the network cope with non-smooth observed curves; and replacing the mean-squared-error loss with a heteroscedastic loss that learns a Gaussian variance for each parameter alongside its mean, so uncertainties are produced at inference time without writing a likelihood. The MCMC comparison uses the same model and uniform priors, giving the reference posteriors that the neural outputs are tested against.","core_discovery":"On its own terms, the paper establishes that a per-galaxy feed-forward neural network trained on simulated rotation curves can serve as a direct map from observed rotation-curve velocities to the five ULDM-plus-baryon parameters $P=(m, s, r_t, r_s, \\Upsilon_*)$. The key empirical results are that adding Gaussian noise drawn from the observed velocity errors to the training inputs substantially improves the network's fit to real SPARC data, that point estimates from both the multiple-realizations method and the heteroscedastic-loss network lie within the 1-$\\sigma$ MCMC intervals for most parameters and galaxies, and that the heteroscedastic network automatically assigns large uncertainties to $r_s$ and $\\Upsilon_*$, which are the parameters the rotation curves barely constrain. The paper frames this as evidence that neural networks can be a useful complementary approach to likelihood-based inference for this class of problem, while noting that the uncertainty from the heteroscedastic loss need not match the Bayesian posterior and that the approach is not yet faster than MCMC.","pith_inferences":["A natural test the paper does not run is to push mock rotation curves generated from parameters outside the Table 1 ranges through the trained networks; because inference is conditional on the training distribution, such a test would show how much of the reported agreement with MCMC is a consequence of sharing the same hand-picked priors.","The heteroscedastic network's large uncertainties on $r_s$ and $\\Upsilon_*$ suggest the loss function is implicitly learning a sensitivity measure, close to the Fisher information of the rotation curve with respect to each parameter, which could be formalized and compared with the Cramer-Rao bound on mock data.","Because the networks ignore radius values entirely, they cannot distinguish two rotation curves with identical velocities at different radii; feeding shuffled or rescaled radii to the trained networks is a cheap way to quantify how much radial information the current pipeline discards.","The per-galaxy training strategy means the method does not yet produce a single constraint on the fundamental ULDM particle mass $m$ across the sample; pooling galaxies into one network with radii as an additional input is the obvious extension and is explicitly named as future work in the paper."],"forward_implications":["For the seven SPARC dwarf galaxies studied, adding noise to training inputs turns networks whose predicted curves miss the observed rotation curve badly into networks whose reconstructed curves have low reduced $\\chi^2$.","Multiple realizations of the observed rotation curve, pushed through a trained network, produce parameter chains whose medians and correlations (for example, a positive $m$--$s$ correlation and an anticorrelation of $r_t$ with $m$ and $s$) resemble MCMC posteriors for $m$, $s$, and $r_t$.","A heteroscedastic-loss network returns usefully large uncertainties for $r_s$ and $\\Upsilon_*$, the parameters to which dwarf-galaxy rotation curves are insensitive, correcting the overconfidence of the multiple-realizations method.","If these results hold, rotation-curve parameter inference can be done without an explicit likelihood, which matters for dark matter models whose rotation-curve predictions are expensive or hard to evaluate.","The trained networks predict galaxy-specific parameters rather than a single global ULDM particle mass, and the paper identifies pooling multiple galaxies into one network as the natural next step."],"supporting_citations":[{"why":"supplies the observed SPARC rotation curves and the baryonic velocity decompositions for the seven dwarf galaxies that the networks must match.","marker":"[25]"},{"why":"provides the ULDM soliton-core fitting function and core radius relation used to generate the simulated dark matter rotation curves.","marker":"[28]"},{"why":"gives the NFW profile used for the outer halo, with $r_s$ one of the inferred parameters.","marker":"[37]"},{"why":"defines the parent sample of seventeen dark-matter-dominated SPARC dwarf galaxies from which the seven galaxies are drawn.","marker":"[34]"},{"why":"supplies the multiple-realizations method for turning observed errors into parameter chains and the precedent for comparing such chains to MCMC contours.","marker":"[11]"},{"why":"is the recent cosmological parameter-estimation work whose heteroscedastic-loss uncertainties were also similar to MCMC, and whose approach this paper follows.","marker":"[15]"},{"why":"backs the claim that training with noise is equivalent to Tikhonov regularization, justifying the noisy training procedure.","marker":"[53]"}],"fun_headline_variants":["Neural networks map rotation curves to dark matter parameters","AI trained on simulated curves yields dark matter parameters","Rotation curve analysis: neural nets rival Bayesian inference","Dark matter from rotation curves: neural net inference","Neural nets extract ULDM and baryon parameters from SPARC"],"cache_read_input_tokens":25728,"weakest_assumption_plain":"The inference stands or falls on whether the hand-picked uniform parameter ranges in Table 1, chosen by eye to make simulated curves resemble each observed rotation curve, actually cover all plausible parameter values, since a network can never return parameters outside its training distribution.","fun_headline_variants_meta":{"raw":{"variants":["Neural networks map rotation curves to dark matter parameters","AI trained on simulated curves yields dark matter parameters","Rotation curve analysis: neural nets rival Bayesian inference","Dark matter from rotation curves: neural net inference","Neural nets extract ULDM and baryon parameters from SPARC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1415,"prompt_tokens":909,"completion_tokens":506,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":430}},"tokens_in":525,"tokens_out":506,"duration_ms":5763,"temperature":1.0,"reasoning_tokens":430,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:16:42.279603+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a set of mock galaxies whose true parameters lie deliberately outside the Table 1 ranges, for example $r_t < 1$ kpc or $\\Upsilon_* > 0.8$, feed their rotation curves to the trained networks, and check whether the inferred parameters track the true values or instead collapse to the boundary of the training ranges; the latter would confirm that the reported agreement with MCMC is conditional on the chosen parameter box.","supporting_citations":[{"cited_title":"Self-Interacting Superfluid Dark Matter Droplets","cited_arxiv_id":"2201.12418","evidence_quote":"defines the parent sample of seventeen dark-matter-dominated SPARC dwarf galaxies from which the seven galaxies are drawn."}],"review_version":1}