pith. sign in

arxiv: 2606.09576 · v1 · pith:UIWDYRLUnew · submitted 2026-06-08 · 🌌 astro-ph.GA · astro-ph.IM· astro-ph.SR

Characterizing Stellar Streams with Error-Aware Machine Learning

Alexandros Pratsos , Biprateep Dey , Ting S. Li This is my paper

Pith reviewed 2026-06-27 16:02 UTC · model grok-4.3

classification 🌌 astro-ph.GA astro-ph.IMastro-ph.SR
keywords stellar streamsmachine learningobservational uncertaintiesGaiaGD-1weakly-supervised learningMilky WayDESI
0
0 comments X

The pith

Incorporating observational uncertainties into neural network training lets SCREAM identify more stellar stream members than prior ML methods.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents SCREAM, a weakly-supervised machine learning framework that finds member stars of stellar streams by treating them as localized over-densities in feature space. It is the first such method to fold observational uncertainties directly into the neural network training objective, building on techniques from particle physics without assuming specific gravitational potentials or isochrones. Applied to Gaia and DESI data for the GD-1 stream, SCREAM reaches an F1 score of 0.745 against independent labels, outperforming existing ML approaches in both precision and recall while recovering the expected diffuse cocoon and faint main-sequence stars missed by classical algorithms like STREAMFINDER. A sympathetic reader would care because stellar streams are sensitive tracers of the Milky Way's dark matter and assembly history, so better membership lists enable tighter constraints on those processes.

Core claim

SCREAM identifies stream members as localized feature-space over-densities while directly incorporating observational uncertainties into the neural network training objective. Validated against independent labels on the GD-1 stream, the method achieves an F1 score of 0.745, substantially outperforming existing ML methods in precision and recall, and recovers the physically expected diffuse cocoon plus faint main-sequence members that classical physics-based algorithms miss.

What carries the argument

SCREAM, the weakly-supervised neural network framework that builds on CATHODE to locate over-densities while adding observational uncertainties to the training loss.

If this is right

  • Uncertainty-aware training improves both precision and recall compared with standard ML methods for stream selection.
  • The approach recovers diffuse structures and faint members without relying on rigid gravitational potentials or strict isochrone filters.
  • Weakly-supervised over-density detection can be applied to other streams using Gaia and DESI-like catalogs.
  • Direct inclusion of measurement errors in the objective reduces the impact of noisy data on membership classification.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the uncertainty term drives the gains, the same loss modification could improve other astronomical tasks that classify objects from catalogs with varying error bars.
  • The framework might extend to identifying other low-surface-brightness galactic substructures where classical methods struggle with selection biases.
  • Larger future surveys with higher typical uncertainties could see even larger relative gains from this style of training.

Load-bearing premise

That the reported performance gain and recovery of additional members are driven by the explicit inclusion of observational uncertainties rather than by other modeling choices or by biases in the independent validation labels.

What would settle it

Retraining the identical network architecture on the same GD-1 data but with the uncertainty term removed from the objective, then checking whether the F1 score against the independent labels drops substantially.

Figures

Figures reproduced from arXiv: 2606.09576 by Alexandros Pratsos, Biprateep Dey, Ting S. Li.

Figure 1
Figure 1. Figure 1: Schematic representation of the SCREAM methodology. Left (Data Cross-match and Partition￾ing): Kinematic and photometric features (x) and associated measurement uncertainties (σx), are compiled from Gaia and the DESI Legacy Survey. Data is partitioned into a Signal Region (SR) and a background-dominated Sideband Region (SB) based on along-stream proper motion (µϕ1 ). Middle (Generative Modeling of Back￾gro… view at source ↗
Figure 2
Figure 2. Figure 2: Visualization of SCREAM performance relative to the STREAMFINDER (SF) base￾line. The background distribution is shown in gray. Top Left (ϕ2 vs. ϕ1): SCREAM recovers stars along the full spatial extent of GD-1. Many apparent False Positives (orange crosses) located off the narrow central track are confirmed members via independent radial velocity labels. Right (r vs. g − r): Predictions trace the dense main… view at source ↗
read the original abstract

Stellar streams are thin, elongated collections of stars formed by gravitational disruption of orbiting star clusters or dwarf galaxies and are highly sensitive probes of the Milky Way's dark matter distribution and formation history. We present $\texttt{SCREAM}$ ($\textbf{S}$tream $\textbf{C}$ha$\textbf{R}$acterization with $\textbf{E}$rror $\textbf{A}$ware $\textbf{M}$achine Learning), a weakly-supervised framework to identify member stars of stellar streams. Building on the $\texttt{CATHODE}$ method originally developed for particle physics, $\texttt{SCREAM}$ identifies streams as localized feature-space over-densities, avoiding rigid physical priors like assumed gravitational potentials or strict isochrone filtering. Crucially, $\texttt{SCREAM}$ is the first machine learning (ML) framework in this domain to directly incorporate observational uncertainties into the neural network training objective. Using astrometric and photometric data from Gaia Data Release 3 and the Dark Energy Spectroscopic Instrument (DESI) Legacy imaging survey, we demonstrate our algorithm's performance on the prominent GD-1 stream. Validated against independent labels, $\texttt{SCREAM}$ achieves an F1 score of 0.745, substantially outperforming existing ML methods in both precision and recall. Furthermore, $\texttt{SCREAM}$ recovers the physically expected diffuse "cocoon" of GD-1 and faint main-sequence members that classical physics-based algorithms (e.g., $\texttt{STREAMFINDER}$) miss. Our results highlight the transformative potential of uncertainty-aware, weakly-supervised ML to uncover complex galactic structures.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The manuscript introduces SCREAM, a weakly-supervised machine learning framework building on CATHODE for identifying stellar stream members as localized over-densities in feature space without rigid physical priors. It claims to be the first such method to directly incorporate observational uncertainties into the neural network training objective, reports an F1 score of 0.745 on GD-1 validated against independent labels, substantially outperforms existing ML methods, and recovers the expected diffuse cocoon plus faint main-sequence members missed by classical algorithms such as STREAMFINDER.

Significance. If the performance gains and additional member recovery are demonstrably attributable to the explicit uncertainty term rather than other modeling choices, the work would represent a useful advance in applying uncertainty-aware ML to galactic archaeology, enabling more robust identification of stream structures as probes of dark matter and Milky Way history. The emphasis on validation against independent labels is a positive element.

major comments (3)
  1. [Abstract] Abstract: the central claim that SCREAM is the first framework to directly incorporate observational uncertainties into the NN training objective, and that this drives the reported F1=0.745 and recovery of cocoon/faint members, cannot be evaluated because the abstract supplies no equation, loss-function definition, or description of how uncertainties enter the objective (e.g., as an additional term, modified likelihood, or input feature).
  2. [Abstract] Abstract: no ablation or controlled comparison is described that removes or replaces the uncertainty component while holding other elements (architecture, features, weak-supervision details) fixed; without this, the attribution of performance gains specifically to uncertainty incorporation remains untested and is load-bearing for the novelty claim.
  3. [Abstract] Abstract: the validation protocol against independent labels is not specified (how labels were constructed, selection criteria, potential biases, or cross-validation scheme), preventing assessment of whether the F1 score and member-recovery results are robust or could arise from label artifacts.
minor comments (1)
  1. The expansion of the SCREAM acronym uses non-standard bolding; consider conventional formatting for readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their careful review and constructive feedback on the abstract. We address each major comment point-by-point below. We agree that additional detail in the abstract would improve clarity and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that SCREAM is the first framework to directly incorporate observational uncertainties into the NN training objective, and that this drives the reported F1=0.745 and recovery of cocoon/faint members, cannot be evaluated because the abstract supplies no equation, loss-function definition, or description of how uncertainties enter the objective (e.g., as an additional term, modified likelihood, or input feature).

    Authors: We agree that the abstract is too concise to convey the precise mechanism. Section 3.2 of the manuscript defines the modified training objective, in which observational uncertainties enter as an explicit additive term in the loss that penalizes predictions inconsistent with error bars. We will revise the abstract to include a one-sentence description of this term so that the novelty claim can be evaluated from the abstract alone. revision: yes

  2. Referee: [Abstract] Abstract: no ablation or controlled comparison is described that removes or replaces the uncertainty component while holding other elements (architecture, features, weak-supervision details) fixed; without this, the attribution of performance gains specifically to uncertainty incorporation remains untested and is load-bearing for the novelty claim.

    Authors: The referee is correct that no such controlled ablation is described. While the manuscript reports overall gains relative to prior ML methods, it does not isolate the uncertainty term by retraining with the same architecture and supervision but without the uncertainty component. We will add this ablation study to the revised manuscript to strengthen the attribution. revision: yes

  3. Referee: [Abstract] Abstract: the validation protocol against independent labels is not specified (how labels were constructed, selection criteria, potential biases, or cross-validation scheme), preventing assessment of whether the F1 score and member-recovery results are robust or could arise from label artifacts.

    Authors: The validation details (construction of independent spectroscopic labels, selection cuts, and cross-validation scheme) appear in Section 4.3. We acknowledge that the abstract does not summarize these steps. We will add a brief clause to the abstract describing the independent-label validation protocol. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation or validation chain

full rationale

The paper introduces SCREAM as an extension of the external CATHODE method, with performance (F1=0.745 on GD-1) reported against independent validation labels rather than any self-fitted quantity. No equations, procedures, or self-citations are described that reduce the claimed gains or recovered members to quantities defined by the model's own parameters or prior author work. The central result is therefore benchmarked externally and does not reduce to its inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract-only information; the central claim rests on the domain assumption that streams manifest as localized over-densities in astrometric-photometric space and that the CATHODE adaptation transfers without additional physical priors. No free parameters or invented entities are enumerated.

axioms (1)
  • domain assumption Stellar streams appear as localized over-densities in feature space without requiring assumed gravitational potentials or strict isochrone cuts
    The method is described as avoiding rigid physical priors and identifying streams purely via over-densities.

pith-pipeline@v0.9.1-grok · 5821 in / 1330 out tokens · 29473 ms · 2026-06-27T16:02:53.717851+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 21 canonical work pages · 4 internal anchors

  1. [1]

    , keywords =

    doi: 10.1016/j.newar.2024.101713. John Franklin Crenshaw, J. Bryce Kalmbach, Alexander Gagliano, Ziang Yan, Andrew J. Connolly, Alex I. Malz, Samuel J. Schmidt, and The LSST Dark Energy Science Collaboration. Probabilistic Forward Modeling of Galaxy Catalogs with Normalizing Flows. AJ, 168(2):80, August

    work page doi:10.1016/j.newar.2024.101713 2024

  2. [2]

    Arjun Dey, David J

    doi: 10.3847/1538-3881/ad54bf. Arjun Dey, David J. Schlegel, Dustin Lang, Robert Blum, Kaylan Burleigh, Xiaohui Fan, Joseph R. Findlay, Doug Finkbeiner, David Herrera, Stéphanie Juneau, Martin Landriau, Michael Levi, Ian McGreer, Aaron Meisner, Adam D. Myers, John Moustakas, Peter Nugent, Anna Patej, Edward F. Schlafly, Alistair R. Walker, Francisco Valde...

    work page doi:10.3847/1538-3881/ad54bf

  3. [3]

    NICE: Non-linear Independent Components Estimation

    doi: 10.3847/1538-3881/ab089d. Laurent Dinh, David Krueger, and Yoshua Bengio. NICE: Non-linear Independent Components Estimation.arXiv e-prints, art. arXiv:1410.8516, October

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.3847/1538-3881/ab089d

  4. [4]

    NICE: Non-linear Independent Components Estimation

    doi: 10.48550/arXiv.1410.8516. Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. Neural Spline Flows.arXiv e-prints, art. arXiv:1906.04032, June

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1410.8516 1906

  5. [5]

    Gaia Collaboration, A

    doi: 10.48550/arXiv.1906.04032. Gaia Collaboration, A. Vallenari, A. G. A. Brown, T. Prusti, J. H. J. de Bruijne, F. Arenou, C. Babu- siaux, M. Biermann, O. L. Creevey, C. Ducourant, D. W. Evans, L. Eyer, R. Guerra, A. Hutton, C. Jordi, S. A. Klioner, U. L. Lammers, L. Lindegren, X. Luri, F. Mignard, C. Panem, D. Pourbaix, S. Randich, P. Sartoretti, C. So...

    work page doi:10.48550/arxiv.1906.04032 1906

  6. [6]

    doi: 10.1051/0004-6361/202243940. C. J. Grillmair and O. Dionatos. Detection of a 63° Cold Stellar Stream in the Sloan Digital Sky Survey. ApJ, 643(1):L17–L20, May

    work page doi:10.1051/0004-6361/202243940

  7. [7]

    Anna Hallin, Joshua Isaacson, Gregor Kasieczka, Claudius Krause, Benjamin Nachman, Tobias Quadfasel, Matthias Schlaffer, David Shih, and Manuel Sommerhalder

    doi: 10.1086/505111. Anna Hallin, Joshua Isaacson, Gregor Kasieczka, Claudius Krause, Benjamin Nachman, Tobias Quadfasel, Matthias Schlaffer, David Shih, and Manuel Sommerhalder. Classifying anomalies through outer density estimation. Phys. Rev. D, 106(5):055006, September

    work page doi:10.1086/505111

  8. [8]

    Anna Hallin, David Shih, Claudius Krause, and Matthew R

    doi: 10.1103/ PhysRevD.106.055006. Anna Hallin, David Shih, Claudius Krause, and Matthew R. Buckley. Via Machinae 3.0: A search for stellar streams in Gaia with the CATHODE algorithm.arXiv e-prints, art. arXiv:2509.08064, September

    arXiv

  9. [9]

    Dan Hendrycks and Kevin Gimpel

    doi: 10.48550/arXiv.2509.08064. Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus).arXiv preprint arXiv:1606.08415,

    work page doi:10.48550/arxiv.2509.08064

  10. [10]

    Li, Sergey E

    7 Emma Jarvis, Ting S. Li, Sergey E. Koposov, Raymond G. Carlberg, Monica Valluri, Nasser Mohammed, J. Aguilar, S. Ahlen, Carlos Allende Prieto, Leandro Beraldo e Silva, D. Bianchi, D. Brooks, Amanda Byström, T. Claybaugh, A. P. Cooper, A. Cuceu, A. de la Macorra, Arjun Dey, Biprateep Dey, P. Doel, J. E. Forero-Romero, E. Gaztañaga, Oleg Y . Gnedin, Satya...

    Pith/arXiv arXiv

  11. [11]

    Characterizing the GD-1 Stream with DESI DR2 Data: Thin Stream and Hot Cocoon

    doi: 10.48550/arXiv.2604.20958. Kathryn V . Johnston. A Prescription for Building the Milky Way’s Halo from Disrupted Satellites. ApJ, 495(1):297–308, March

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2604.20958

  12. [12]

    Diederik P

    doi: 10.1086/305273. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InInternational Conference on Learning Representations (ICLR),

    work page doi:10.1086/305273

  13. [13]

    Khyati Malhan and Rodrigo A

    doi: 10.1093/mnras/275.2.429. Khyati Malhan and Rodrigo A. Ibata. STREAMFINDER - I. A new algorithm for detecting stellar streams. MNRAS, 477(3):4063–4076, July

    work page doi:10.1093/mnras/275.2.429

  14. [14]

    Cecilia Mateu

    doi: 10.1093/mnras/sty912. Cecilia Mateu. galstreams: A library of Milky Way stellar stream footprints and tracks. MNRAS, 520(4):5225–5258, April

    work page doi:10.1093/mnras/sty912

  15. [15]

    doi: 10.1093/mnras/stad321. Eric M. Metodiev, Benjamin Nachman, and Jesse Thaler. Classification without labels: learning from mixed samples in high energy physics.Journal of High Energy Physics, 2017(10):174, October

    work page doi:10.1093/mnras/stad321 2017

  16. [16]

    Benjamin Nachman and David Shih

    doi: 10.1007/JHEP10(2017)174. Benjamin Nachman and David Shih. Anomaly detection with density estimation. Phys. Rev. D, 101 (7):075042, April

    work page doi:10.1007/jhep10(2017)174 2017

  17. [17]

    Anomaly detection with density estimation , volume=

    doi: 10.1103/PhysRevD.101.075042. Mariel Pettee, Sowmya Thanvantri, Benjamin Nachman, David Shih, Matthew R. Buckley, and Jack H. Collins. Weakly supervised anomaly detection in the Milky Way. MNRAS, 527(3): 8459–8474, January

    work page doi:10.1103/physrevd.101.075042

  18. [18]

    Frank Rosenblatt

    doi: 10.1093/mnras/stad3663. Frank Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain.Psychological review, 65(6):386,

    work page doi:10.1093/mnras/stad3663

  19. [19]

    Maps of Dust IR Emission for Use in Estimation of Reddening and CMBR Foregrounds

    doi: 10.1086/305772. Debajyoti Sengupta, Stephen Mulligan, David Shih, John Andrew Raine, and Tobias Golling. SKY- CURTAINS: model-agnostic search for stellar streams with Gaia data. MNRAS, 536(2):1104–1114, January

    work page internal anchor Pith review doi:10.1086/305772

  20. [20]

    David Shih, Matthew R

    doi: 10.1093/mnras/stae2570. David Shih, Matthew R. Buckley, Lina Necib, and John Tamanas. VIA MACHINAE: Searching for stellar streams using unsupervised machine learning. MNRAS, 509(4):5992–6007, February

    work page doi:10.1093/mnras/stae2570

  21. [21]

    David Shih, Matthew R

    doi: 10.1093/mnras/stab3372. David Shih, Matthew R. Buckley, and Lina Necib. VIA MACHINAE 2.0: Full-sky, model-agnostic search for stellar streams in Gaia DR2. MNRAS, 529(4):4745–4767, April

    work page doi:10.1093/mnras/stab3372

  22. [22]

    Zechang Sun, Joshua S

    doi: 10.1093/ mnras/stae446. Zechang Sun, Joshua S. Speagle, Song Huang, Yuan-Sen Ting, and Zheng Cai. Zephyr : Stitching Heterogeneous Training Data with Normalizing Flows for Photometric Redshift Inference.arXiv e-prints, art. arXiv:2310.20125, October

    arXiv

  23. [23]

    Bingjie Wang, Joel Leja, V

    doi: 10.48550/arXiv.2310.20125. Bingjie Wang, Joel Leja, V . Ashley Villar, and Joshua S. Speagle. SBI ++: Flexible, Ultra-fast Likelihood-free Inference Customized for Astronomical Applications. ApJ, 952(1):L10, July

    work page doi:10.48550/arxiv.2310.20125

  24. [24]

    doi: 10.3847/2041-8213/ace361. 8

    work page doi:10.3847/2041-8213/ace361 2041