Pith. sign in

REVIEW 3 major objections 4 minor 3 cited by

SOAP: A Python Package for Calculating the Properties of Galaxies and Halos Formed in Cosmological Simulations

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SOAP computes the same 250+ halo and galaxy properties for any supported halo finder, making the finders themselves the only variable in comparisons.

desk verdict A genuinely useful software description that deserves review, but the manuscript as submitted lacks a code link, tests, and validation details, so it needs revision before acceptance. read the letter →

arxiv 2507.22669 v1 pith:MUPN2NUN submitted 2025-07-30 astro-ph.IM astro-ph.COastro-ph.GA

Robert McGibbon , John C. Helly , Joop Schaye , Matthieu Schaller , Bert Vandenbroucke This is my paper · ORCID
classification astro-ph.IMastro-ph.COastro-ph.GA
keywords SOAPsphericaloverdensityhalopropertiesgalaxysubhalofinderscosmologicalsimulationsparallelcomputingHDF5
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SOAP is a Python package that takes a subhalo catalogue from a cosmological simulation and computes over 250 halo and galaxy properties for every object, under four different halo definitions. The paper's central claim is that because SOAP places property calculation in a single layer independent of the subhalo finder, researchers can compare catalogues produced by different structure-finding algorithms without the confounding differences in how each finder happens to define and compute those properties. The package is designed to run in parallel and on subvolumes, so it can handle the largest current hydrodynamic simulations, and it has already produced catalogues for the FLAMINGO simulation suite that are used in more than twenty publications. If the paper is right, SOAP gives the community a shared, flexible tool that turns halo-finder comparisons into controlled experiments rather than apples-to-oranges mismatches.

What carries the argument

The load-bearing mechanism is the converter layer plus the standard property-computation core. Each supported halo finder contributes only a script that translates its subhalo catalogue into SOAP's standard format; thereafter, every property is computed by the same code paths regardless of which finder identified the objects. Spherical-overdensity radii are computed by growing spheres until the target density is reached, and all particles within the radius, bound or unbound, are used for SO properties. Properties are defined lazily and can be filtered by user criteria, so only requested quantities are computed, which together with mpi4py-based parallelism and independent subvolume processing is what makes the package efficient on large simulations.

What would settle it

A concrete check would be to run SOAP on an isolated dark-matter halo with a known analytic density profile (for example a Navarro-Frenk-White halo) and compare the spherical-overdensity radii, enclosed masses, and concentrations against exact values; any discrepancy larger than the nominal numerical tolerance would show that the property calculations are not correct. A second, targeted check is to feed SOAP the exact same set of particles through the converters of two different halo finders while keeping the bound-particle lists identical, and confirm that every property that depends only on particle membership is numerically identical; if property values depend on which finder supplied the catalogue, the converters are not actually standardizing the calculations.

Watch

Extended reading notes

Core claim

The central discovery is a working design for separating the identification of bound structures from the calculation of their physical properties. SOAP accepts subhalo catalogues from four widely used finders, converts each into a standard format, and then recomputes properties with identical definitions for every object, including spherical-overdensity radii found by expanding spheres until a target density is reached. With four halo definitions (subhalo particles, spherical overdensity, fixed projected apertures, and fixed physical apertures) and more than 250 lazily computed properties, the output is a single HDF5 catalogue with consistent units, readable by the swiftsimio package. The paper also reports that this design has been exercised on the FLAMINGO simulations, generating catalogues used in over twenty published studies.

Load-bearing premise

The paper assumes that its implementation is numerically correct and physically consistent, yet it presents no validation against analytic solutions, no unit tests, and no repository link or version identifier, so a reader cannot independently check correctness from the manuscript.

Editorial extensions

If this is right

  • Users can compute 250+ properties across four halo definitions with a single tool, making large simulations analyzable by many researchers without each recomputing their own.
  • Halo-finder differences can be isolated: inconsistent property definitions no longer contaminate comparisons, so remaining catalogue differences reflect the structure-identification method itself.
  • The package scales to large simulations via parallel processing and independent subvolume runs, and it handles both dark matter-only and hydrodynamic runs with no parameter changes.
  • Output HDF5 catalogues integrate with swiftsimio for unit handling and are spatially sorted for fast subvolume loading.
  • The FLAMINGO catalogues generated with SOAP are already used in more than 20 publications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same consistency could be used to quantify finder-induced scatter: by running one simulation's snapshots through several finders and comparing the same SOAP-defined properties, cosmological analyses could attach a systematic uncertainty to halo masses and concentrations.
  • If the converter layer is as generic as claimed, extending SOAP to a new halo finder would be a small scripting task, potentially making the package a community standard that new finders are expected to support.
  • The lazy, filterable property system suggests a path toward very large parameter sweeps: because only requested properties are computed, SOAP-based pipelines could produce minimal catalogues for specific observables, reducing storage costs as simulation volumes grow.
  • A natural next step is to apply SOAP to existing simulations with published halo catalogues beyond FLAMINGO and confirm that recomputed properties reproduce the published values for the same objects, which would also serve as the validation the manuscript does not include.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript introduces SOAP, a Python package for computing halo and galaxy properties from cosmological simulations post-processed with a subhalo finder. SOAP is designed to accept input from multiple halo finders, compute over 250 properties under several halo definitions, run in parallel via mpi4py, and output HDF5 catalogues compatible with swiftsimio. The authors report that SOAP has already been used to produce catalogues for the FLAMINGO simulations and that these catalogues have supported more than 20 publications. The paper consists of a summary, a statement of need, and a bulleted overview of features; it does not present validation tests, benchmarks, algorithmic details, a code repository link, or a version number.

Significance. If the implementation is correct and the claims are supported, SOAP would fill a genuine need: a community-shared, multi-finder, parallel tool for consistent halo/galaxy property calculation. The design choices described—lazy property evaluation, per-property filters, automatic PDF documentation, and integration with swiftsimio—are sensible and potentially useful. The strongest parts of the submission are the clear identification of the multi-finder comparison problem and the emphasis on separating genuine structure-finder differences from property-definition differences. However, the paper currently provides no independent evidence of correctness: no source-code link, no tests, no analytic validation, and no performance measurements. The FLAMINGO adoption claim demonstrates usage but not correctness. The central methodological contribution—consistent property calculation across HBT-HERONS, SubFind, VELOCIraptor, and ROCKSTAR—remains unverifiable from the manuscript as submitted.

major comments (3)
  1. [Statement of Need, paragraph 2] The paper claims that SOAP "scales efficiently across multiple compute nodes" and that parallel processing is "a unique feature," but it presents no parallel performance measurements, no scaling study, and no comparison with the cited packages. Since parallel efficiency is one of the two headline practical advantages stated in the abstract, this is a load-bearing claim. Provide at least weak- and strong-scaling tests on a representative snapshot, or explicitly temper the claim to "supports parallel processing."
  2. [Statement of Need, paragraph 3 and Overview of Features] The central claim of consistency across halo finders rests on the SO radius algorithm, which is described only as "constructing expanding spheres until the target density limit is reached." The manuscript does not specify which center is used for the expansion, how the density is estimated from a discrete particle distribution, how periodic boundary conditions are handled, how convergence is decided, or how the result differs between hydrodynamics and dark-matter-only runs. These details are essential because all subsequently computed properties depend on the SO radius. Please specify the algorithm precisely or point to a specific versioned code location where the implementation can be inspected.
  3. [Whole manuscript] No source-code repository, release version, or DOI is provided, and no test suite or validation results are presented. For a software paper, this means the stated features—four halo-finder converters, the lazy property system, and the 250+ property set—cannot be independently checked. Include a repository link and version number, and add at least basic validation (e.g., unit tests, comparisons with analytic cases such as an isothermal sphere or a Hernquist profile, and cross-checks against catalogues from existing codes) before the claims of consistent multi-finder property calculation can be assessed.
minor comments (4)
  1. [Summary and Abstract] There are several spacing typos: "IntroductionSOAP" in the Summary and "aPython" in the Abstract should be "Introduction SOAP" and "a Python."
  2. [Overview of Features, last bullet] The phrase "units that areh-free" is missing a space and should read "units that are h-free."
  3. [References] The CAESAR reference (Narayanan et al., 2023) is formatted as a GitHub repository without a version or access date; consider citing the package documentation or a release DOI if one exists.
  4. [Statement of Need, paragraph 1] The claim that SOAP produces catalogues "used in more than 20 publications" is supported only by a footnote URL; since the number is time-dependent and not part of the technical content, consider moving it to a less prominent place or removing it.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SOAP is a software description with no fitted parameters or derived predictions; self-citations are contextual and not load-bearing.

full rationale

This paper is a software description, not a derivation of a physical result from fitted inputs. There are no equations, no fitted parameters, and no predictions that could reduce to their own inputs by construction. The central claim—that SOAP computes over 250 halo and galaxy properties consistently across multiple halo finders—is an implementation claim; the spherical-overdensity radius is defined by 'constructing expanding spheres until the target density limit is reached,' which is a definition, not a derived result. Self-citations to FLAMINGO and to Forouhar Moreno et al. (2025) are used for motivation and to document prior usage, not as evidence for the correctness of any derived quantity. The cited claim that different halo finders produce significantly different catalogues is an externally checkable empirical statement and does not depend on SOAP's own outputs in a circular way. The absence of a repository link, version number, or validation tests is a reproducibility and correctness risk, but it is not circularity because no scientific claim is being derived from those missing elements.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim, that SOAP computes properties correctly, rests on the correctness of the input catalogues from external subhalo finders and on the accuracy of the software's algorithms and metadata handling. The paper introduces no new physical entities and fits no new parameters to data; the SO overdensity threshold is a user choice, not a fitted constant.

free parameters (1)
  • Spherical overdensity threshold (e.g., 200 times the critical density)
    This is a user-configurable input used to define halo radii; it is not fitted to data in this paper.
assumptions (3)
  • domain assumption Subhalo finder catalogues (HBT-HERONS, SubFind, VELOCIraptor, ROCKSTAR) correctly identify bound structures and provide the particle lists SOAP uses.
    The paper relies on the input catalogues being reliable; it does not validate the finders itself.
  • domain assumption The spherical overdensity algorithm, based on expanding spheres, faithfully recovers the desired density-threshold radii.
    The description in the Statement of Need assumes this standard method works; no test is shown.
  • domain assumption The SWIFT snapshot metadata used for unit handling and spatial sorting is accurate.
    SOAP's output depends on the correctness of the simulation snapshot format.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SOAP: A Python Package for Calculating the Properties of Galaxies and Halos Formed in Cosmological Simulations." pith.science (2026). https://pith.science/paper/MUPN2NUN

@misc{pith2026250722669,
  author       = {Pith},
  title        = {Pith review of: SOAP: A Python Package for Calculating the Properties of Galaxies and Halos Formed in Cosmological Simulations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MUPN2NUN}},
  note         = {Machine review of arXiv:2507.22669}
}
read the original abstract

Modern large scale cosmological hydrodynamic simulations require robust tools capable of analysing their data outputs in a parallel and efficient manner. We introduce SOAP (Spherical Overdensity and Aperture Processor), a Python package designed to compute halo and galaxy properties from SWIFT simulations after being post-processed with a subhalo finder. SOAP takes a subhalo catalogue as input and calculates a wide array of properties for each object. SOAP offers parallel processing capabilities via mpi4py for efficient handling of large datasets, and allows for consistent property calculation across multiple halo finders. SOAP supports various halo definitions, including spherical overdensities and fixed physical apertures, providing flexibility for diverse observational comparisons. The package is compatible with both dark matter-only and full hydrodynamic simulations, producing HDF5 catalogues that are integrated with the swiftsimio package for seamless unit handling.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The One-Loop Power Spectrum of Fast Radio Burst Dispersion Measures

    astro-ph.CO 2026-08 accept novelty 7.0 of 10

    A one-loop EFT power spectrum model for FRB free-electron clustering matches FLAMINGO simulations to k ~ 0.2 h/Mpc, with electron bias b_e ~ 0.92 and near-perfect electron-matter correlation.

  2. Interpreting the stacked kinetic SZ effect I: velocity reconstruction and non-linear velocity effects

    astro-ph.CO 2026-07 conditional novelty 7.0 of 10

    Non-linear velocity terms cancel in real-space linear reconstruction, but redshift-space distortions reintroduce a 10–20% small-scale suppression of the stacked kSZ signal.

  3. The COLIBRE-SKIRT pipeline: Calibration-free dust radiative transfer postprocessing for cosmological simulations

    astro-ph.GA 2026-07 conditional novelty 7.0 of 10

    The COLIBRE-SKIRT pipeline reproduces the observed low-redshift cosmic SED without calibrating the post-processing, using live dust from the simulation and a new 'split & scale' grain-size mapping.

Reference graph

Works this paper leans on

21 extracted references · 4 canonical work pages · cited by 3 Pith papers

  1. [1]

    S., Wechsler, R

    Behroozi, P. S., Wechsler, R. H., & Wu, H.-Y. (2013). The ROCKSTAR phase-space temporal halo finder and the velocity offsets of cluster cores. The Astrophysical Journal, 762(2), 109. https://doi.org/10.1088/0004-637X/762/2/109

    doi:10.1088/0004-637x/762/2/109 2013

  2. [2]

    Borrow, J., & Borrisov, A. (2020). swiftsimio : A Python library for reading SWIFT data. Journal of Open Source Software, 5(52), 2430. https://doi.org/10.21105/joss.02430

    doi:10.21105/joss.02430 2020

  3. [3]

    Collette, A. (2013). Python and HDF5. O'Reilly

    work page 2013

  4. [4]

    Dalcín, L., & Fang, Y.-L. L. (2021). mpi4py: Status update after 12 years of development. Computing in Science & Engineering, 23(4), 47--54. https://doi.org/10.1109/MCSE.2021.3083216

    arXiv 2021

  5. [5]

    Dalcín, L., Paz, R., Kler, P., & Cosimo, A. (2011). Parallel distributed computing using Python . Advances in Water Resources, 34(9), 1124--1139. https://doi.org/10.1016/j.advwatres.2011.04.013

    doi:10.1016/j.advwatres.2011.04.013 2011

  6. [6]

    Dalcín, L., Paz, R., & Storti, M. (2005). MPI for Python . Journal of Parallel and Distributed Computing, 65(9), 1108--1115. https://doi.org/10.1016/j.jpdc.2005.03.010

    doi:10.1016/j.jpdc.2005.03.010 2005

  7. [7]

    Dalcín, L., Paz, R., Storti, M., & D'Elía, J. (2008). MPI for Python : Performance improvements and MPI-2 extensions. Journal of Parallel and Distributed Computing, 68(5), 655--662. https://doi.org/10.1016/j.jpdc.2007.09.005

    doi:10.1016/j.jpdc.2007.09.005 2008

  8. [8]

    Dome, T. (2023). CosmicProfiles : A Python package for radial profiling of finitely sampled dark matter halos and galaxies. Journal of Open Source Software, 8(85), 5008. https://doi.org/10.21105/joss.05008

    work page doi:10.21105/joss.05008 2023

Show all 21 references
  1. [9]

    J., Cañas, R., Poulton, R

    Elahi, P. J., Cañas, R., Poulton, R. J. J., Tobar, R. J., Willis, J. S., Lagos, C. del P., Power, C., & Robotham, A. S. G. (2019). Hunting for galaxies and halos in simulations with VELOCIraptor . Publications of the Astronomical Society of Australia, 36, e021. https://doi.org...

    2019 doi

  2. [10]

    J., Helly, J., McGibbon, R., Schaye, J., Schaller, M., Han, J., & Kugel, R

    Forouhar Moreno, V. J., Helly, J., McGibbon, R., Schaye, J., Schaller, M., Han, J., & Kugel, R. (2025). Assessing subhalo finders in cosmological hydrodynamical simulations . arXiv e-Prints, arXiv:2502.06932. https://arxiv.org/abs/2502.06932

    2025 arXiv

  3. [11]

    J., ZuHone, J

    Goldbaum, N. J., ZuHone, J. A., Turk, M. J., Kowalik, K., & Rosen, A. L. (2018). unyt : Handle, manipulate, and convert data with units in Python . Journal of Open Source Software, 3(28), 809. https://doi.org/10.21105/joss.00809

    2018 doi

  4. [12]

    S., Benitez-Llambay, A., & Helly, J

    Han, J., Cole, S., Frenk, C. S., Benitez-Llambay, A., & Helly, J. (2018). HBT+ : An improved code for finding subhaloes and building merger trees in cosmological simulations. Monthly Notices of the Royal Astronomical Society, 474(1), 604--617. https://doi.org/10.1093/mnras/stx2792

    2018 doi

  5. [13]

    R., Millman, K

    Harris, C. R., Millman, K. J., Walt, S. J. van der, Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., Kerkwijk, M. H. van, Brett, M., Haldane, A., Río, J. F. del, Wiebe, M., Peterson, P., Oliphant, T. E. ...

    2020 doi

  6. [14]

    C., Braspenning, J., Elbers, W., Frenk, C

    Kugel, R., Schaye, J., Schaller, M., Helly, J. C., Braspenning, J., Elbers, W., Frenk, C. S., McCarthy, I. G., Kwan, J., Salcido, J., van Daalen, M. P., Vandenbroucke, B., Bahé, Y. M., Borrow, J., Chaikin, E., Huško, F., Jenkins, A., Lacey, C. G., Nobels, F. S. J., & Vernon, I...

    2023 doi

  7. [15]

    Narayanan et al. (2023). CAESAR. In GitHub repository. https://github.com/dnarayanan/caesar; GitHub

    2023

  8. [16]

    Pontzen, A., & Tremmel, M. (2018). TANGOS: The Agile Numerical Galaxy Organization System . The Astrophysical Journal Supplement Series, 237(2), 23. https://doi.org/10.3847/1538-4365/aac832

    2018 doi

  9. [17]

    W., Ivkovic, M., McAlpine, S., Vandenbroucke, B., Bahé, Y., Chaikin, E., Chalk, A

    Schaller, M., Borrow, J., Draper, P. W., Ivkovic, M., McAlpine, S., Vandenbroucke, B., Bahé, Y., Chaikin, E., Chalk, A. B. G., Chan, T. K., Correa, C., van Daalen, M., Elbers, W., Gonnet, P., Hausammann, L., Helly, J., Huško, F., Kegerreis, J. A., Nobels, F. S. J., Xiang, Z. (...

    2024

  10. [18]

    A., Bower, R

    Schaye, J., Crain, R. A., Bower, R. G., Furlong, M., Schaller, M., Theuns, T., Dalla Vecchia, C., Frenk, C. S., McCarthy, I. G., Helly, J. C., Jenkins, A., Rosas-Guevara, Y. M., White, S. D. M., Baes, M., Booth, C. M., Camps, P., Navarro, J. F., Qu, Y., Rahmati, A., Trayford, ...

    2015 doi

  11. [19]

    C., Braspenning, J., Elbers, W., McCarthy, I

    Schaye, J., Kugel, R., Schaller, M., Helly, J. C., Braspenning, J., Elbers, W., McCarthy, I. G., van Daalen, M. P., Vandenbroucke, B., Frenk, C. S., Kwan, J., Salcido, J., Bahé, Y. M., Borrow, J., Chaikin, E., Hahn, O., Huško, F., Jenkins, A., Lacey, C. G., & Nobels, F. S. J. ...

    2023 doi

  12. [20]

    Springel, V. (2005). The cosmological simulation code GADGET-2 . Monthly Notices of the Royal Astronomical Society, 364(4), 1105--1134. https://doi.org/10.1111/j.1365-2966.2005.09655.x

    2005

  13. [21]

    Springel, V., White, S. D. M., Tormen, G., & Kauffmann, G. (2001). Populating a cluster of galaxies - I . Results at z=0. Monthly Notices of the Royal Astronomical Society, 328(3), 726--750. https://doi.org/10.1046/j.1365-8711.2001.04912.x CSLReferences document

    2001

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.