{"id":"2795c82e-2fed-47c8-83da-bf164b3128fc","arxiv_id":"2412.02749","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A deep learning classifier detects simulated stellar wakes from dark matter subhalos as light as 5e7 solar masses, with overdensity and velocity divergence carrying most of the signal.","lead":"This paper trains convolutional neural networks on windtunnel simulations of dark matter subhalos moving through the Milky Way's stellar halo and finds the networks can detect the resulting stellar wakes down to 5e7 solar masses in idealized mock data. It is a feasibility study for a new dark matter probe, not yet a measurement on real stars.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is only established for idealized windtunnel mocks: the binary classifier is trained against a perfectly smooth Maxwellian null class (Sect. 2.2), and no test includes Galactic potential, substructure, tides, or observational errors (Sect. 5), so the reported AOCs (Fig.","rationale":"The reader's weakest assumption—that windtunnel wakes are representative of real Milky Way stellar wakes—is the single most load-bearing concern. The paper's headline numbers (AOC 0.63, 0.77, 1.00) are separations between a wake and a perfectly homogeneous Maxwellian background. Because the negative class is unrealistically clean, the reported FPRs (e.g., 35% for 1e8 Msun at TPR 74%) are optimistic; real halo substructure will populate the same feature space as wakes. The paper honestly lists these limitations in Sect. 5, which is why I do not recommend rejection; the work is a valid proof-of-concept. However, the 'detection in the Milky Way' language in the abstract should be read as 'detection in mock data.' I agree with the conditional verdict and would make the realism test an explicit condition. I also note a reproducibility inconsistency: Table 2 reports 3 z-slices, Sect. 4.1 describes using only the middle slice with two features, and Sect. 5 gives input shape (N,32,32,2); the exact input to the model in Fig. 6 needs clarification. The concrete test I propose—evaluating the stored classifier on realistic subhalo-free halos and on injected low-mass subhalos—would settle whether the low-mass claim transfers.","tokens_in":19619,"tokens_out":14277,"duration_ms":146575,"concrete_test":"Take the final binary classifier and evaluate it on mock samples generated from a realistic Milky Way-like halo simulation (e.g., Auriga or FIRE-2) at 30 kpc with no subhalo injected, using the same 2D binning and features. Measure the false-positive rate at the decision threshold that yields TPR = 74% for the 1e8 Msun case. If the FPR is above ~50% (versus 35% in the idealized test), realistic background structure alone masks the wake signal. Separately, inject a 5e7 Msun Plummer subhalo on a circular orbit into the realistic halo and recompute the AOC; if it falls below 0.55, the low-mass detectability claim does not transfer.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline performance (median AOC 0.63/0.77/1.00 for 5e7/1e8/5e8 Msun, Fig. 6) is computed on mock datasets where the null hypothesis is a perfectly homogeneous, isotropic Maxwellian background with no Galactic potential, no tidal stripping, no initial density clumpiness, and no measurement errors (Sect. 2.2; limitations in Sect. 5). The classifier's task is therefore to separate a wake from white noise, not from realistic stellar-halo structure. In real data, the null class contains streams, shells, velocity anisotropies, and Gaia error covariances; any of these can produce localized overdensity or divergence patterns that mimic a wake. The paper explicitly defers observational error modeling, realistic potentials, and substructure to future work, so the claim that deep learning can detect stellar wakes 'in the Milky Way' is not yet supported. A secondary but real issue is an internal inconsistency about the final model input: Table 2 states 'number of z-slices' final value 3, Sect. 4.1 says training proceeded 'only on data from the middle slice' with two features, and Sect. 5 gives input dimensionality (N, 32, 32, 2); the exact architecture behind Fig. 6 is therefore ambiguous and needs clarification for reproducibility.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper assesses whether supervised deep learning can detect stellar wakes induced by dark-matter subhalos in the Milky Way's stellar halo, using idealized windtunnel N-body simulations. Subhalos with Plummer profiles and masses 5e7, 1e8, and 5e8 solar masses move through a homogeneous background of DM and star particles whose phase-space parameters are matched to conditions at 30 kpc (and, for one test, 50 kpc) from the Galactic center. The simulations are binned into 32x32 pixel maps of four observables (overdensity, mean speed, speed dispersion, velocity divergence) across three z-slices, Gaussian-smoothed, and used to train Harmonic CNNs for binary (subhalo present/absent) and three-class (mass) classification. The headline results are median AOC values of 0.63, 0.77, and 1.00 for the three masses (Fig. 6), a multi-class confusion matrix (Fig. 8), and a cross-distance test showing comparable performance on 50-kpc mocks including with a 30-kpc-trained model (Fig. 9). Feature ablations identify overdensity and velocity divergence as the most informative observables.","tokens_in":19891,"tokens_out":17352,"duration_ms":158232,"significance":"If the claims hold, this is a useful proof of principle: a CNN can extract subhalo-induced wake signatures from density and kinematic maps down to ~5e7 solar masses in idealized conditions, supporting further investment in wake-based probes of the low-mass subhalo mass function. The experimental design is notably careful: train/validation/test splits are separated by simulation seed (Sect. 3.2); every result is repeated over 30 training runs with scatter reported; the data-ablation study in Sect. 4.2 shows performance is data-limited; the mass scaling of the signal (Figs. 3 and 6) is a sensible consistency check; and the derived ML dataset is released on Zenodo. The principal caveat, openly acknowledged in Sect. 5, is that the null hypothesis is a perfectly smooth Maxwellian background with no Galactic potential, tidal stripping, substructure, or observational errors, so the reported AOC values are idealized upper bounds rather than Milky Way detection rates. The paper's value is as a feasibility and methodology study, and it should be framed as such.","major_comments":[{"comment":"The final model input configuration is stated inconsistently in three places, and the architecture that produced the headline results (Fig. 6) is therefore ambiguous. Table 2 lists the final 'number of z-slices' as 3; Sect. 4.1 (last bullet) states that the authors 'proceeded with training only on data from the middle slice of the box'; and Sect. 5 says the input dimensionality is (N, 32, 32, 2) while also stating that 'we used three overdensity images (slices) per sample for training.' Since Sect. 3.1 defines the full feature set as 3 slices x 4 channels = 12 channels, the reader cannot determine whether Fig. 6 corresponds to 3 slices x 4 channels, 3 slices x 2 channels, or the middle slice x 2 channels. Please reconcile these statements and state unambiguously, for each reported result (Figs. 5, 6, 8, 9), the exact input dimensionality and channel list.","section":"Table 2, Sect. 4.1, Sect. 5"},{"comment":"The reported detectability (median AOC 0.63/0.77/1.00 in Fig. 6) is established only against a null class that is a perfectly homogeneous, isotropic Maxwellian background, in a periodic box with no Galactic potential, no tidal stripping, no clumpy halo substructure, and no observational errors (Sect. 2.2; limitations acknowledged in Sect. 5). In real stellar-halo data the null hypothesis is not white noise: streams, shells, velocity anisotropies, and Gaia error covariances can all produce localized overdensity or divergence patterns similar to a wake. As written, the title and abstract claim detection 'in the Milky Way,' which overstates what the mocks demonstrate. I recommend (i) explicitly labeling the Fig. 6 numbers as idealized upper bounds throughout the abstract and conclusions, and (ii) adding a stress test in which the background null is made more realistic (e.g., adding stream-like overdensities or a radial velocity anisotropy to the null class) to verify that the classifier separates wakes from realistic structure rather than from Poisson noise. The central feasibility claim is defensible, but the current framing invites an extrapolation the paper does not yet support.","section":"Sect. 2.2, Sect. 4.2, Sect. 5"}],"minor_comments":[{"comment":"'Area Over the Curve (AOC)' is non-standard terminology; the quantity defined from the ROC curve is the area under the curve (AUC). Please rename accordingly.","section":"Throughout"},{"comment":"The TPR/FPR pairs reported in Sect. 6 at 'optimal threshold' are not derived in the main text; please define the threshold criterion (e.g., Youden's index) and state the operating points explicitly in Sect. 4.","section":"Sect. 4.1, Sect. 6"},{"comment":"Please clarify how the 100 samples per simulation are drawn from the snapshot (random subsampling of particles versus disjoint spatial regions) and state how many independent simulations and samples were used in the 50-kpc case. The reported error bars reflect retraining and reseeding variation over 30 runs; noting the effective number of independent simulations (48 per mass) would help readers gauge the statistical leverage behind the AOC values.","section":"Sect. 3.1, Sect. 4.4"},{"comment":"The conclusions state 'With the amount of training data available (4800 samples)', while Sect. 3.1 specifies 2400 training, 1600 validation, and 800 test samples, so 4800 is the total. Please correct the wording to avoid implying the training set size was 4800.","section":"Abstract, Sect. 6"},{"comment":"There are several typographical artifacts ('succesfully' in the Introduction; 'di fferent', 'a ffected', and 'e ffect' throughout), presumably from a LaTeX source with an unusual hyphenation macro; these should be cleaned up.","section":"Typographical"}],"recommendation":"major_revision","confidential_remarks":"This is a competent feasibility study with a sound ML protocol, and I see no grounds for rejection. The two issues that require attention before publication are the inconsistent description of the final model input (Table 2 versus Sect. 4.1 versus Sect. 5), which blocks reproduction of Fig. 6, and the gap between the idealized-mock results and the 'in the Milky Way' framing of the title and abstract. The authors' own Sect. 5 is transparent about the idealized setup, which works in their favor; my concern is mainly that the headline AOC numbers will be cited without the caveats. I would also suggest asking the authors to state explicitly in the abstract that the results are based on idealized simulations with a smooth background."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a careful, honest ML feasibility study: the binary classifier really does separate wake from no-wake in windtunnel mocks down to 5e7 Msun (median AOC 0.63), with seed-separated train/test splits and 30 training runs. Second, the title's \"in the Milky Way\" oversells it. The null class is a perfectly smooth Maxwellian background with no Galactic potential, tides, substructure, or measurement errors, so the reported AOCs are upper bounds for real Gaia-like data. The authors acknowledge this in Sect. 5, so they are not hiding it, but the framing should be corrected.\n\nWhat is genuinely new: they extend windtunnel wake studies to sub-LMC masses, include self-gravity, show that overdensity plus velocity divergence are the informative features, and demonstrate transfer to a 50 kpc setup with no retraining drop. The multi-class mass discrimination is a nice extra. The simulation setup is reproducible in principle: Pkdgrav3, public libraries, and a Zenodo mention for the ML dataset, though no identifier is given.\n\nThe main soft spot is not the ML methodology, which is handled carefully, but the gap between the mock universe and the Milky Way. The classifier is essentially detecting a wake against white noise; in real data, streams, shells, velocity anisotropies, and Gaia error covariances will produce similar local patterns. So the AOC values are not detection probabilities. That is a real limitation, but not a fatal one, because the paper presents itself as a first step and explicitly defers observational realism to future work.\n\nSecond, there is an internal inconsistency about the input: Table 2 says \"number of z-slices\" final value 3, but Sect. 4.1 says training proceeded only on the middle slice, and Sect. 5 gives input dimensionality (N, 32, 32, 2), which suggests one slice with two features. The exact architecture behind Fig. 6 is therefore ambiguous. This must be fixed before the headline numbers can be reproduced. Third, no code or dataset identifiers are provided; a Zenodo mention without an ID is not enough for reproducibility.\n\nWho is this for? People developing subhalo detection methods, not observers looking for a ready-made pipeline. It deserves a serious referee. I would send it out, but require clarification of the architecture and release of code and dataset identifiers before acceptance.","headline":"A careful, honest ML feasibility study on idealized mocks, but the title overstates readiness for the real Milky Way.","tokens_in":20505,"tokens_out":2326,"would_cite":false,"duration_ms":25287,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A deep-learning classifier can detect the stellar wake of a 5×10^7 solar-mass dark subhalo in idealized Milky Way simulations, with detection power rising steeply with subhalo mass.","keywords":["dark matter subhalos","stellar wakes","dynamical friction","Milky Way stellar halo","deep learning classification","convolutional neural networks","N-body simulations","subhalo mass function"],"falsifier":"Retrain the same binary classifier on mock catalogues that add a Galactic potential, tidal stripping, anisotropic halo densities, and Gaia-like measurement errors to the windtunnel snapshots; if the AOC for the $5\\times10^{7}\\,\\mathrm{M}_{\\odot}$ case falls to 0.5, the idealized wake is not representative. A direct observational check would be to search Gaia data for the predicted trailing overdensity and velocity-divergence dip along candidate subhalo orbits at 30 kpc: a CDM-like subhalo abundance predicts multiple dark subhalos in that volume, so the complete absence of wake-like patterns in a well-characterized survey region would count against the practical detectability claim.","tokens_in":19402,"feed_emoji":"🌌","tokens_out":11167,"duration_ms":104250,"temperature":0.7,"pith_summary":"This paper asks whether the faint wake a dark-matter subhalo leaves in the Milky Way's stellar halo can be found by a deep-learning classifier. The authors simulate dark subhalos of mass $5\\times10^{7}\\,\\mathrm{M}_{\\odot}$, $10^{8}\\,\\mathrm{M}_{\\odot}$, and $5\\times10^{8}\\,\\mathrm{M}_{\\odot}$ moving through a uniform, Maxwellian background of stars and dark matter, and train a convolutional neural network on $32\\times32$ pixel maps of overdensity, mean speed, speed dispersion, and velocity divergence. They report that the network separates subhalo-bearing samples from background-only samples at better than random level for every mass tested, with median Area Over the Curve (AOC) values of 0.63, 0.77, and 1.00, and that overdensity plus velocity divergence carries essentially all the useful signal. If correct, this would establish stellar wakes as a viable, data-driven probe of the low-mass end of the subhalo mass function, a sharp test of the cold dark matter paradigm.","feed_headline":"Neural nets spot dark-subhalo wakes down to 50 million solar masses","feed_subtitle":"Idealized Milky Way mocks show overdensity and velocity divergence can reveal subhalos too faint for other probes.","key_machinery":"The central object is the stellar wake: the overdensity and trailing kinematic disturbance, especially a dip in velocity divergence, that a massive perturber imprints on surrounding stars through gravitational interaction. The machinery that carries the argument is a two-stage pipeline: self-gravitating windtunnel simulations that produce wakes for chosen subhalo masses, and a convolutional neural network whose first layer performs a windowed discrete cosine decomposition, which the authors find well suited to the small-training-data regime. The network classifies $32\\times32\\times2$ images of Gaussian-smoothed overdensity and velocity divergence, and this two-feature combination is shown to be equivalent to using all four phase-space features.","core_discovery":"The paper's central claim is that the gravitational wake of a passing dark subhalo is a detectable phase-space pattern: a convolutional neural network can infer the presence of a subhalo from binned stellar kinematics in mock observations, down to $5\\times10^{7}\\,\\mathrm{M}_{\\odot}$. The claim is established in idealized windtunnel simulations at 30 and 50 kpc from the Galactic center, with classifiers trained and tested on statistically independent mock samples. Detection performance scales strongly with mass: $5\\times10^{8}\\,\\mathrm{M}_{\\odot}$ subhalos are essentially perfectly identified, $10^{8}\\,\\mathrm{M}_{\\odot}$ subhalos give a median AOC of 0.77, and $5\\times10^{7}\\,\\mathrm{M}_{\\odot}$ subhalos give a median AOC of 0.63. The same model trained at 30 kpc generalizes to 50 kpc data, and a multi-class version correctly labels about 97% of the heaviest subhalos. The authors interpret the similar 30 kpc and 50 kpc performance as indicating that the phase-space parameters change too little over 20 kpc to alter detectability.","pith_inferences":["The paper leaves implicit that velocity divergence's dominance implies a survey with only one line-of-sight velocity component may retain much of the signal if that component aligns with the wake; this could be tested by re-running the classifier on mock data with transverse velocity components removed.","Because the 30 kpc and 50 kpc results are nearly identical, we infer the detectability window may extend to larger radii than tested; a run at, say, 80-100 kpc would show where the wake becomes undetectable, though the authors note stellar tracers become scarce there.","The authors attribute the light-subhalo limit to scarce training data; we infer that simulation-based data augmentation, for example generative emulators, is a cheaper route to push the mass threshold below $5\\times10^{7}\\,\\mathrm{M}_{\\odot}$, a claim that could be tested by checking whether augmented training sets raise the low-mass AOC."],"forward_implications":["Subhalos as light as $5\\times10^{7}\\,\\mathrm{M}_{\\odot}$ can in principle be found from their stellar wakes, reaching below the masses where subhalos are expected to be entirely dark, which would open a data-driven route to the low-mass end of the subhalo mass function.","Overdensity and velocity divergence are the observables a survey should prioritize; adding mean speed and speed dispersion to the input does not improve classification once divergence is included.","Smoothing the binned phase-space maps is critical, improving AOC by roughly 25-35%, so survey binning and noise matter as much as network architecture.","The classifier is portable across Galactocentric radii: training at 30 kpc transfers to 50 kpc mock data with little loss, suggesting the learned wake features are not tied to one background density or velocity dispersion.","The reported detection limit is set by the amount of training data rather than by a physical floor; the paper's ablation study shows AOC rises steadily as more independent simulation samples are added."],"supporting_citations":[{"why":"Supplies the dynamical friction picture that motivates why a moving subhalo perturbs the surrounding stellar medium and leaves a wake.","marker":"Chandrasekhar (1943)"},{"why":"Provides the analytic likelihood formalism for wake-based subhalo mass inference and the Plummer perturber convention that this paper extends with self-gravity and machine learning.","marker":"Buschmann et al. (2018)"},{"why":"Demonstrates windtunnel simulations of stellar wakes with self-gravity for LMC-mass perturbers, the setup this paper adapts to smaller subhalo masses.","marker":"Foote et al. (2023)"},{"why":"Prior data-driven study showing dark subhalos leave discernible phase-space imprints in Milky Way-like simulations; the direct predecessor of this ML approach.","marker":"Bazarov et al. (2022)"},{"why":"Source of the Plummer scale-radius relation used to set subhalo sizes for each adopted mass.","marker":"Diemand et al. (2008)"},{"why":"Milky Way mass model used to set the fiducial 225 km/s orbital speed at 30 kpc.","marker":"Karukes et al. (2020)"},{"why":"Provides the stellar halo velocity dispersion of 95 km/s at 30 kpc used in the background initial conditions.","marker":"Deason et al. (2012)"},{"why":"FIRE-2-based subhalo counts and tangential velocities used to motivate the adopted mass range and the 250 km/s velocity estimate at 30 kpc.","marker":"Barry et al. (2023a)"}],"fun_headline_variants":["Deep learning finds dark subhalo wakes down to 50 million suns","AI detects dark matter wakes in Milky Way stellar halo","Neural network spots faint dark subhalo wakes","Machine learning reveals subhalo wakes down to 50 million solar masses","CNN spots dark subhalos via stellar wake signatures"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an idealized windtunnel box, with a homogeneous Maxwellian background, periodic boundaries, no Galactic potential, no tidal stripping, and no observational errors, produces stellar wakes that resemble the wakes real Milky Way subhalos carve into the stellar halo, so that if real halo clumpiness, measurement errors, or the neglected potential erase or distort the wake, the reported AOC values will not transfer to observations.","fun_headline_variants_meta":{"raw":{"variants":["Deep learning finds dark subhalo wakes down to 50 million suns","AI detects dark matter wakes in Milky Way stellar halo","Neural network spots faint dark subhalo wakes","Machine learning reveals subhalo wakes down to 50 million solar masses","CNN spots dark subhalos via stellar wake signatures"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000715,"raw_usage":{"total_tokens":3289,"prompt_tokens":1091,"completion_tokens":2198,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":707,"completion_tokens_details":{"reasoning_tokens":2113}},"tokens_in":707,"tokens_out":2198,"duration_ms":14249,"temperature":1.0,"reasoning_tokens":2113,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:08:58.956125+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same binary classifier on mock catalogues that add a Galactic potential, tidal stripping, anisotropic halo densities, and Gaia-like measurement errors to the windtunnel snapshots; if the AOC for the $5\\times10^{7}\\,\\mathrm{M}_{\\odot}$ case falls to 0.5, the idealized wake is not representative. A direct observational check would be to search Gaia data for the predicted trailing overdensity and velocity-divergence dip along candidate subhalo orbits at 30 kpc: a CDM-like subhalo abundance predicts multiple dark subhalos in that volume, so the complete absence of wake-like patterns in a well-characterized survey region would count against the practical detectability claim.","supporting_citations":[{"cited_title":"R., & Wu, C.-L","cited_arxiv_id":null,"evidence_quote":"Provides the analytic likelihood formalism for wake-based subhalo mass inference and the Plummer perturber convention that this paper extends with self-gravity and machine learning."},{"cited_title":"R., Besla, G., Mocz, P., et al","cited_arxiv_id":null,"evidence_quote":"Demonstrates windtunnel simulations of stellar wakes with self-gravity for LMC-mass perturbers, the setup this paper adapts to smaller subhalo masses."},{"cited_title":"2022, Astronomy and Computing, 41, 100667","cited_arxiv_id":null,"evidence_quote":"Prior data-driven study showing dark subhalos leave discernible phase-space imprints in Milky Way-like simulations; the direct predecessor of this ML approach."},{"cited_title":"2008, Nature, 454, 735","cited_arxiv_id":null,"evidence_quote":"Source of the Plummer scale-radius relation used to set subhalo sizes for each adopted mass."},{"cited_title":"2020, Jour- nal of Cosmology and Astroparticle Physics, 2020, 033","cited_arxiv_id":null,"evidence_quote":"Milky Way mass model used to set the fiducial 225 km/s orbital speed at 30 kpc."},{"cited_title":"J., Belokurov, V ., Evans, N","cited_arxiv_id":null,"evidence_quote":"Provides the stellar halo velocity dispersion of 95 km/s at 30 kpc used in the background initial conditions."}],"review_version":1}