{"id":"bd044973-bd96-470a-9661-78bba83b0b27","arxiv_id":"2506.02804","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Sorcha is an open-source Python survey simulator that forward-models LSST detection biases for arbitrary solar system small-body populations, with end-to-end validation against JPL Horizons.","lead":"This paper presents Sorcha, an open-source software tool that simulates which small solar system objects the future Rubin Observatory LSST survey will detect, given a synthetic asteroid or trans-Neptunian population. Scientists can use it to correct for the survey's detection biases and compare model populations to real discoveries.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The trailing-loss formula in §7.4 (Eq. 6) is internally inconsistent: with the stated detection coefficients it gives negative losses, which would make fast-moving objects appear brighter and bias NEO/MBA predictions; Section 9 does not test this.","rationale":"The reader's weakest-assumption identification — that the linking filter's flat 95% efficiency and perfect precovery/recovery may misrepresent the real SSP selection function — is a legitimate, explicitly acknowledged limitation. I agree that this is a real risk, but it is a calibration risk that the authors, by their own caveats, cannot retire until LSST operations begin. The concern I raise is different and more immediately checkable: the printed trailing-loss formula in §7.4 is internally inconsistent with the stated coefficients, the nonnegativity claim, and Figure 8. The same module is on the critical path from input populations to predicted discoveries for all fast-moving populations, and the validation in §9 does not cover it. If the code check shows the implementation is correct, the concern reduces to a paper typo and the ACCEPT verdict stands. If the code follows Eq. 6 as printed, the error systematically brightens fast-moving sources and inflates predicted NEO/MBA detection rates, which undermines a core part of the central claim. A conditional verdict is therefore appropriate: accept once the sign convention is resolved and, if needed, the validation is extended to PSF magnitudes and fast-moving objects.","tokens_in":49947,"tokens_out":8756,"duration_ms":102995,"concrete_test":"Run the PPTrailingLoss function from the released Sorcha package with v = 2 deg/day, T_exp = 30 s, and θ = 0.7 arcsec, once with a = 0.42, b = 0 and once with a = 0.67, b = 1.16. If the returned values are positive (approximately +1.0 mag and +0.34 mag, respectively), the code is consistent with Figure 8 and only Eq. 6 in the paper needs a sign correction. If either returned value is negative, the implementation feeds a brighter-than-true PSF magnitude into Eq. 12, and the Section 9 validation should be extended to fast-moving objects and PSF-magnitude comparisons before the discovery-prediction claims are accepted.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing concern is not the acknowledged linking-filter simplification, but a concrete inconsistency in the photometric chain that feeds the detection-efficiency filter. Section 7.4 defines Δm = −1.25 log10((1 + a x²)/(1 + b x)) with a = 0.42, b = 0 for Δm(PSF+detection), and states that both trailing losses are always ≥ 0. For any x > 0, however, (1 + a x²)/(1 + b x) = 1 + 0.42 x² > 1, so log10 > 0 and Δm < 0. Example: v = 2 deg/day, T_exp = 30 s, θ = 0.7 arcsec gives x ≈ 3.57 and Δm ≈ −1.0 mag. Equation 5 then yields m_PSF = m_trailed − 1.0 mag, making a fast NEO appear a full magnitude brighter than its true total flux. Since the fading function in §7.8 evaluates detection probability at m_PSF, this would systematically overpredict detections of fast-moving objects near the survey limit. This contradicts Figure 8 and the text's nonnegativity claim. The Section 9 validation compares only trailed source magnitudes for a TNO and an MBA, so it does not exercise the PSF magnitude, trailing losses, or the detection-efficiency filter; the claim that a stringent test demonstrates correct results is therefore not established for this regime. If the code actually implements the positive loss (i.e., Δm = +1.25 log10(...)), then the paper has a sign error in Eq. 6 that must be corrected; if the code follows Eq. 6 as printed, NEO and fast-MBA discovery predictions are biased.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Sorcha, an open-source Python survey simulator designed for LSST solar system science. The simulator splits into an ephemeris-generation stage that uses ASSIST+REBOUND to propagate orbits and match objects to survey pointings, and a post-processing stage that applies a sequence of modular filters: phase-curve photometry, trailing losses, PSF and trailed-source magnitudes, vignetting, camera footprint, source detection efficiency (fading function), saturation, and a linking filter that emulates the Rubin SSP pipeline. The authors validate the output against JPL Horizons for one TNO (2011 OB60) and one MBA (2010 TU149) over one month, finding sub-mas astrometric agreement and magnitude offsets near 1e-7 mag. The paper also documents HPC benchmarking, utility scripts, and software engineering practices. The central claim is that Sorcha is a comprehensive, scalable, and correct survey simulator for all solar system small-body populations.","tokens_in":50378,"tokens_out":3435,"duration_ms":38926,"significance":"If the technical concerns are resolved, Sorcha fills a clear community need: no existing simulator handles the LSST's scale, cadence, and breadth of populations in a single open-source, configurable package. The paper ships per-function validation notebooks, reproducible benchmarks, and an end-to-end comparison against an independent implementation, which are substantial strengths. The Horizons comparison for astrometry and trailed-source magnitudes is a strong, falsifiable check. The main weakness is that the validation does not exercise the detection-efficiency chain (PSF magnitude, trailing losses, fading function) or fast-moving objects, and the trailing-loss formula as printed is internally inconsistent. The linking filter is also a deliberate simplification, acknowledged by the authors, but its impact on discovery-rate predictions should be stated more explicitly.","major_comments":[{"comment":"The sign convention in Equation (6) is internally inconsistent with the text and with Figure 8. With the stated detection-loss coefficients a=0.42, b=0, the ratio (1 + a x^2)/(1 + b x) = 1 + 0.42 x^2 is always greater than 1 for x > 0, so log10 of that ratio is positive and Equation (6) yields Δm < 0. The paragraph immediately below claims that both trailing losses are 'always greater than or equal to zero.' Consequently, Equation (5) would make the PSF magnitude brighter than the trailed source magnitude, e.g., for v=2 deg/day, T_exp=30 s, and θ=0.7 arcsec, x≈3.57 and Δm≈−1.0 mag, making a fast NEO appear about one magnitude brighter. Since the fading function in Equation (12) is evaluated at m_PSF, this would systematically overpredict detections of fast-moving objects near the survey limit. Section 9 does not test this because it compares only trailed source magnitudes for a slow TNO and an MBA. The authors must either correct the sign in Equation (6) to match a positive loss or, if the code implements a positive loss, fix the printed formula; either way, the nonnegativity claim and Figure 8 need rechecking against the code.","section":"§7.4, Eq. (6)"},{"comment":"The claim that 'a stringent test... demonstrates that Sorcha produces correct results' is too strong given the scope of the validation. The test covers only two objects over one month, both of which are slow-moving (a TNO and an MBA); it checks astrometry and trailed source magnitudes but does not exercise the PSF magnitude, the trailing-loss formulas of Equation (6) (especially the detection-loss branch), the fading-function filter, the saturation limit, or the linking filter. Those components are load-bearing for the simulator's stated purpose of predicting discoveries across all small-body populations, including NEOs. I recommend tempering the conclusion to state that astrometry and trailed-source photometry are validated for the tested regimes, and adding at least one validation case with a fast-moving object (e.g., a synthetic NEO) that checks the PSF magnitude and the detection-efficiency chain.","section":"§9, final paragraph"},{"comment":"The linking filter is a substantial simplification: it counts tracklets and tracks with a flat 95% discovery efficiency and assumes perfect precovery/recovery, rather than modeling the actual SSP linkage geometry, orbit determination, or false-link rejection. This is acknowledged in the text and the filter is configurable, but because the linking filter is the final gate for 'discovered' objects, predicted discovery counts and rates will inherit any mismatch between this counting heuristic and the real SSP selection function. The paper should state this limitation prominently in the abstract or conclusions, not only in the filter description, and should quantify (or at least discuss) how sensitive the science outputs are to the 95% efficiency and to the tracklet/track counting rules.","section":"§7.10"}],"minor_comments":[{"comment":"The text states '≈19 thousand exposures' without specifying the exact number or the pointing database version; please provide a precise count or a reference to the generated dataset so the validation is reproducible.","section":"§9"},{"comment":"The text refers to 'Section 6.12' for the faint object culling filter, but the filter is described in Section 7.11.2; this cross-reference should be corrected.","section":"Author contributions, J. Murtagh"},{"comment":"The sentence describing 'm_trailed.source + Δm(PSF) provides the apparent magnitude of a point-source with the equivalent SNR' appears in both Section 7.4 and Section 7.5; consider consolidating to avoid redundancy.","section":"§7.4 and §7.5"},{"comment":"The notation 'σ2_mPSF' and 'σ2_mtrailed' is inconsistent with the subscript style used elsewhere (e.g., 'σ_mPSF'); using a consistent subscript convention would improve readability.","section":"Equation (8), (9)"},{"comment":"The color term C(i−x) is described in Section 5.2.2 as an offset relative to the main filter, but the sign convention in Equation (1) should be stated explicitly (e.g., whether a positive C makes the object brighter or fainter in filter i).","section":"§5.2.2 and Equation (1)"}],"recommendation":"major_revision","confidential_remarks":"The trailing-loss sign issue in §7.4 is the main correctness concern. If the code already implements a positive loss, the paper's Equation (6) is a typographical sign error and the fix is straightforward; if the code follows the printed equation, NEO and fast-MBA predictions would be systematically biased. Either way, the authors should add a validation case that exercises the PSF magnitude and the detection-efficiency filter for a fast-moving object. The linking filter simplification is acknowledged but should be elevated in prominence because discovery-rate predictions are a central output of the simulator."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read of the Sorcha paper. It's a genuinely useful software contribution: a modular, HPC-scalable survey simulator that ties ASSIST/REBOUND ephemerides to rubin_sim-derived camera/telescope models and a simple linking emulator, with per-module validation notebooks and an open-source release. That combination is new, and the paper is honest about the places where it approximates the real LSST (flat detection efficiency, simplified linking, no stellar crowding). If you work on solar system populations and LSST, you'll likely use this or argue about it.\n\nThe soft spot that matters most is the trailing-loss prescription in §7.4. Equation 6 is typed as Δm = -1.25 log10((1 + a x²)/(1 + b x)), with a=0.42, b=0 for the PSF+detection loss. For any x>0 the ratio is >1, so log10 is positive and Δm is negative. That contradicts the text's claim that both losses are always ≥ 0 and would make fast NEOs appear brighter, systematically overpredicting detections near the limit. The same issue affects the PSF loss at x>1.73. Section 9's validation compares RA/Dec and trailed source magnitudes for one TNO and one MBA; it never exercises Eq. 6 or the PSF magnitude, so it doesn't catch this. If the code implements a positive loss, then the paper has a sign error in the equation; if the code follows the printed equation, the predictions are biased. Either way, authors need to fix it and add a direct test of trailing losses against an independent calculation before this is accepted.\n\nThe other caveats are milder and mostly acknowledged: the linking filter counts tracklets with a flat 95% efficiency, and the detection efficiency is uniform in sky position. Those are reasonable choices before LSST data exists, and they're stated. The end-to-end validation with two objects is thin, though the sub-mas agreement with Horizons is reassuring for the ephemeris side.\n\nI'd send this to peer review, but the referee should require the sign fix and a validation that covers trailing losses. Once that's in, I'd happily cite it and bring it to reading group.","headline":"A valuable, well-engineered LSST survey simulator that deserves referee time, but the trailing-loss equation in §7.4 appears to have a sign error that would bias NEO and fast-MBA predictions.","tokens_in":51024,"tokens_out":3937,"would_cite":true,"duration_ms":36372,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sorcha is a comprehensive, scale-ready survey simulator that forward-models every observational bias onto a user-supplied solar system small-body population.","keywords":["survey simulator","solar system small bodies","observational biases","LSST","detection efficiency","linking filter","ephemeris generation","forward modeling"],"falsifier":"Re-run a set of synthetic populations through both Sorcha and the actual automated linking pipeline on the same pointing history once it is operational, and compare which objects are marked linked. A systematic mismatch in linked or unlinked classifications beyond the stated 95% efficiency would falsify the linking filter's emulation of the survey's selection function.","tokens_in":49757,"feed_emoji":"🔭","tokens_out":9151,"duration_ms":87879,"temperature":0.7,"pith_summary":"Sorcha is a new open-source survey simulator built to predict what the ten-year Legacy Survey of Space and Time will detect and discover among solar system small bodies: asteroids, comets, trans-Neptunian objects, and interstellar objects. The paper argues that earlier simulators cannot handle the survey's scale, about 2 million exposures, billions of observations, and roughly 5 million expected new discoveries, and that Sorcha's two-stage design (ephemeris generation followed by a cascade of observational-bias filters) meets that scale while remaining adaptable to other surveys. The central claim is that, for a given input model population, the simulator reproduces the survey's intertwined detection biases, so output detections and discoveries can be compared directly with real survey results. An end-to-end validation on two objects reports sub-milliarcsecond astrometric agreement and magnitude agreement near $10^{-7}$ mag with independently computed values.","feed_headline":"Simulator forecasts which solar system objects LSST will find","feed_subtitle":"Maps any model small-body population onto exact survey detections, validated to sub-milliarcsecond accuracy.","key_machinery":"The load-bearing mechanism is the two-stage pipeline: an ephemeris-generation stage that propagates every input orbit with a high-accuracy N-body integrator, predicts positions at set times per night, and matches objects to survey pointings via on-sky grids and interpolation; then a post-processing stage that runs each potential detection through a configurable chain of bias filters. The chain's central photometric distinction is between the trailed source magnitude (the object's true apparent brightness, integrating all flux along the trail) and the PSF magnitude (the brightness a stellar-matched detection algorithm would measure); trailing losses connect them through the empirical loss formula $\\Delta m = -1.25 \\log_{10}\\left(\\frac{1+ax^2}{1+bx}\\right)$ with $x = vT_{\\rm exp}/(24\\theta)$. The discovery step is a linking filter that emulates the survey's tracklet-and-track linking by counting detections per night and applying a configurable discovery efficiency, set to 95% for the survey's baseline. The whole chain decides which model objects become reported detections and discoveries.","core_discovery":"On its own terms, the paper's central claim is that Sorcha is a comprehensive, correct, and scalable survey simulator. It takes any user-supplied population of solar system orbits and physical parameters, propagates their positions through the survey's pointing history with a built-in N-body ephemeris integrator, and then applies modular post-processing steps: phase curves, colors, trailing losses, the distinction between trailed-source and PSF magnitudes, astrometric and photometric uncertainties, vignetting, camera footprint, source detection efficiency, saturation, and a linking filter that decides which objects are detected and when they are discovered. The authors assert that 'a stringent test... demonstrates that Sorcha produces correct results', reporting mean RA offsets of $8\\times10^{-4}$ mas and $0.10$ mas, mean Dec offsets of $9\\times10^{-5}$ mas and $0.02$ mas, and trailed-source magnitude offsets of $-10^{-7}$ mag and $-1.9\\times10^{-8}$ mag against independent computations for two test objects. The design goal is that forward-biased model populations can be compared directly to the survey's real discoveries, removing the need to infer the detection biases afterward.","pith_inferences":["Editorial inference: the end-to-end validation covers two objects' astrometry and photometry over roughly a month; it does not exercise the stochastic filters (fading function, footprint gaps, linking). The clearest test of the whole discovery chain would be comparing Sorcha's linked/unlinked output against the real linking pipeline on identical synthetic populations once it is operational.","Editorial inference: because the link between 'detected' and 'discovered' is represented by a flat 95% efficiency plus perfect precovery and recovery, population-level predictions are only as good as that heuristic. If the real pipeline's linking probability varies with magnitude, motion, or tracklet geometry, the ratios of predicted discovery rates between different small-body populations could s","Editorial inference: the same architecture could be pointed at the discovery-linking problem beyond the survey's nominal distance cutoff. With a custom linking module, it could quantify how many distant and interstellar objects a bespoke pipeline might recover, which is a stated motivation but not developed in this paper.","Editorial inference: the modular plugin system for activity and light curves permits a test not run in this paper, forecasting whether rotational variability or cometary outbursts change an object's discoverability, which would be a natural follow-up once the base pipeline is validated."],"forward_implications":["Any user-supplied model population of small-body orbits, real or synthetic, can be propagated through the survey's pointing history and observational biases to yield the specific observations and discoveries the survey would report.","Predicted discoveries can be compared directly against the survey's annual data releases, so model populations can be tested, refined, or ruled out as real observations accumulate.","The two-stage design lets users reuse one expensive ephemeris computation across many runs that vary physical parameters, filters, or biases, making population-model exploration affordable on high-performance computing clusters.","With a different pointing database, camera footprint, and configuration, the same code is intended to simulate other wide-field surveys, not just the Legacy Survey of Space and Time.","Sorcha is designed to run on populations of millions to billions of objects, matching the scale of the survey's expected discoveries."],"supporting_citations":[{"why":"Supplies the N-body integration machinery that powers Sorcha's ephemeris generation for test-particle orbits.","marker":"M. J. Holman et al. (2023)"},{"why":"Describes the survey cadence simulation software whose pointing databases Sorcha ingests and whose footprint and vignetting functions are adapted.","marker":"A. J. Connolly et al. (2014)"},{"why":"Provides the simulated survey pointing databases used as input for Sorcha's validation and benchmark runs.","marker":"P. Yoachim et al. (2023)"},{"why":"Provides the empirical trailing-loss formula and survey sensitivity parameters used to compute PSF magnitudes and signal-to-noise.","marker":"R. L. Jones et al. (2018)"},{"why":"Supplies the fading-function detection efficiency, vignetting treatment, and limiting-magnitude model.","marker":"P. Vereš & S. R. Chesley (2017a)"},{"why":"Supplies the photometric error model, 5-sigma limiting magnitude, saturation limit, and astrometric noise floor used in uncertainty estimation.","marker":"Ž. Ivezić et al. (2019)"},{"why":"Defines the survey's Solar System Processing requirements that the linking filter emulates with tracklet and track counting.","marker":"M. Jurić et al. (2020)"},{"why":"Supplies the phase-function model implementations used for apparent magnitude calculation.","marker":"M. Mommert et al. (2019)"}],"fun_headline_variants":["Sorcha predicts LSST's solar system finds","Sorcha maps orbits to LSST detections","Sorcha simulates solar system populations for LSST","Sorcha maps any solar system population to LSST detections","Sub-milliarcsecond simulator maps LSST discoveries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole discovery prediction depends on one premise: that the survey's automated linking of detections into tracklets and orbits can be represented by simply counting detections per night with a flat 95% discovery efficiency and perfect precovery and recovery, so if the real linking pipeline succeeds or fails in a more complicated way, the predicted discoveries and their dates will be systematically off.","fun_headline_variants_meta":{"raw":{"variants":["Sorcha predicts LSST's solar system finds","Sorcha maps orbits to LSST detections","Sorcha simulates solar system populations for LSST","Sorcha maps any solar system population to LSST detections","Sub-milliarcsecond simulator maps LSST discoveries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001825,"raw_usage":{"total_tokens":7194,"prompt_tokens":978,"completion_tokens":6216,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":6140}},"tokens_in":594,"tokens_out":6216,"duration_ms":39314,"temperature":1.0,"reasoning_tokens":6140,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:15:33.290455+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run a set of synthetic populations through both Sorcha and the actual automated linking pipeline on the same pointing history once it is operational, and compare which objects are marked linked. A systematic mismatch in linked or unlinked classifications beyond the stated 95% efficiency would falsify the linking filter's emulation of the survey's selection function.","supporting_citations":[{"cited_title":"Neilsen, J., et al","cited_arxiv_id":null,"evidence_quote":"Provides the simulated survey pointing databases used as input for Sorcha's validation and benchmark runs."}],"review_version":1}