Pith. sign in

REVIEW 3 major objections 6 minor 19 references

STIPS: The Nancy Grace Roman Space Telescope Imaging Product Simulator

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper presents STIPS, a fast simulator that generates post-pipeline Nancy Grace Roman WFI images of any number of detectors (up to all 18) with point-source flux within about 1–3 percent of Pandeia and photometric recovery at the 0.06…

desk verdict A genuinely useful Roman simulator with an honest write-up, but the headline PSF accuracy is not actually demonstrated off the interpolation grid. read the letter →

arxiv 2411.11978 v1 pith:YWHAJK6W submitted 2024-11-18 astro-ph.IM

STIPS Development Team , Sebastian Gomez , Andrea Bellini , Hanna Al-Kowsi , Tyler Desjardins , Robel Geda , Eunkyu Han , O. Justin Otor
show 4 more authors
Adric Riedel Russell Ryan Isaac Spitzer Brian York
This is my paper · ORCID
classification astro-ph.IM
keywords RomanSpaceTelescopeWide-FieldInstrumentimagesimulationeffectivepointspreadfunctionPandeiasurveyplanningdetectornoiseastronomicalsoftware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

STIPS is a Python package designed to quickly produce realistic, post-pipeline images of the Nancy Grace Roman Space Telescope's Wide-Field Instrument, covering any subset of the 18 detector chips. The paper argues that the simulator is accurate enough for survey planning: point-source fluxes agree with the reference tool Pandeia at the 1–3 percent level for bright non-saturated stars, and a user who fits STIPS's own effective PSF model recovers injected magnitudes to about 0.06 mag and positions to about 0.004 pixels. The package is meant to complement, not replace, high-accuracy exposure time calculators: it trades some physical detail (no saturation, non-linearity, or distortion) for roughly twentyfold speed, enabling rapid exploration of pointings, filters, and source parameters. If the claim holds, it gives the Roman community a practical way to test observing strategies and PSF-dependent science before launch.

What carries the argument

The central object is the effective point spread function (ePSF): the convolution of the instrumental PSF with the detector's pixel-response function, integrated over pixels. STIPS builds a grid of nine WebbPSF PSFs per detector at 4x oversampling, convolves them with the interpixel-capacitance kernel, and uses bilinear interpolation to estimate the ePSF at any requested position; bicubic interpolation is used for the four core pixels when injecting a source. This position-dependent ePSF, injected at subpixel coordinates and scaled by filter zeropoints and background, is what carries the fidelity of both photometry and astrometry in the simulated scenes.

What would settle it

Once Roman is on orbit, compare STIPS predictions to real WFI images of a well-measured star field: fit the STIPS ePSF model to actual sources and check whether recovered magnitudes and positions agree with independent photometry/astrometry, and whether the observed PSF shape across a detector matches the nine-point interpolation. A mismatch in the core shape, focus, or interpixel capacitance beyond the claimed few-percent level would invalidate the simulator's accuracy for real data.

Watch

Extended reading notes

Core claim

The central claim is that STIPS can synthesize post-pipeline Wide-Field Instrument images of Roman that reproduce the key features a planner needs—point spread function shape, extended-source morphology, background, and noise—at speeds that make large survey simulations practical. Point sources are injected as WebbPSF-generated PSFs that have been convolved with the detector pixel response including interpixel capacitance, then interpolated from a nine-point grid per detector to account for the PSF's position dependence; extended sources are drawn from Sersic profiles generated either by Astropy or by the Pandeia engine. Validation against Pandeia shows total flux agreement better than about 1 percent for bright unsaturated sources, degrading above ~10 percent only for very faint sources where background dominates; and in end-to-end tests, fitting the same ePSF model used to create the scenes recovers injected magnitudes with a mean offset of 0.0027 ± 0.0602 mag and positions to 0.004 pixels. The paper is explicit about the trade-off: STIPS omits saturation, non-linearity, and distortion, so it is a rapid scene simulator and PSF-shape explorer rather than a substitute for a high-fidelity exposure time calculator.

Load-bearing premise

The reliability of every simulated scene rests on the assumption that the WebbPSF point spread functions, computed from GSFC optical models at five field points per detector and combined with the 2019 interpixel-capacitance kernel, faithfully represent the true on-orbit Roman ePSF; no real Roman images yet exist to test this.

Editorial extensions

If this is right

  • Survey planning can be done at scale: full 18-SCA scenes with thousands of sources can be simulated in about one-twentieth the time of more exact simulators, making parameter-space exploration feasible.
  • Users can study PSF variation across the WFI: STIPS interpolates the ePSF across each detector, enabling tests of how PSF shape differences affect photometry and astrometry between SCAs.
  • The ~0.06 mag recovery when fitting the STIPS ePSF model demonstrates that, under ideal simulated conditions, the package's own model is self-consistent as a photometric reference.
  • Comparisons against Pandeia show that for bright unsaturated sources STIPS flux is within ~1–3%, making it suitable for exposure time estimates at the planning level, though not as a high-accuracy ETC.
  • The inclusion of Poisson noise, readout noise, flat/dark residuals, cosmic rays, and zodiacal/internal background means the output images carry realistic noise statistics for feasibility studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the on-orbit Roman PSF differs from the pre-launch model, the validation numbers reported here bound only the internal consistency of the simulator, not its accuracy against flight data; a natural extension is to recalibrate the ePSF grid once commissioning images are available.
  • The nine-point bilinear PSF interpolation could be tested against denser PSF grids or against in-flight PSF maps to quantify the residual error introduced by spatial interpolation across a detector.
  • The same scene-generation pipeline could be adapted to other wide-field instruments by swapping the PSF and pixel-response inputs, so the architecture is a template for post-pipeline simulators generally.
  • Because STIPS omits distortion, saturation, and non-linearity, its images should be used to develop and test algorithms that are robust to those effects rather than to validate calibration pipelines that depend on them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents STIPS, an open-source Python simulator for generating post-pipeline images of the Nancy Grace Roman Space Telescope Wide-Field Instrument. The simulation pipeline is described in detail: effective PSFs are produced with WebbPSF on a 4x-oversampled nine-point grid per SCA and interpolated bilinearly; point sources are injected using a hybrid bicubic/bilinear sub-pixel interpolation; extended sources are generated with an Astropy Sersic2D model or Pandeia's SersicDistribution; background and noise sources include zodiacal/internal background, Poisson noise, readout noise, dark and flat residuals, and cosmic rays. Validation in Section 3 compares STIPS to Pandeia for total flux in all eight filters at the SCA center and performs source-recovery tests for photometry and astrometry with three fitting methods. The paper emphasizes speed (~20x faster than the full romanisim simulator) and the ability to simulate any number of SCAs up to the full 18-SCA array.

Significance. The tool fills a practical niche: it is faster than full up-the-ramp simulators and more flexible in field size than Pandeia, and the open-source code is valuable for planning Roman observations. The comparison to Pandeia provides a useful independent check of total flux at the ~1-3% level for bright sources, and the paper is honest about known omissions (saturation, non-linearity, distortion, diffraction-spike truncation). However, the validation is incomplete in two specific ways: the Pandeia comparison and the recovery tests are performed only at the SCA center, which is a grid point where interpolation error vanishes, and the most precise recovery results are obtained by fitting with the same ePSF model used for injection. These issues do not undermine the core usefulness of the simulator, but they need to be addressed before the claimed capabilities are fully established.

major comments (3)
  1. [Section 3.1, Figures 7-8] All validation against Pandeia is performed at the SCA center (2048, 2048), which by construction is one of the nine ePSF grid points used in the bilinear interpolation of Section 2.1; at this location the interpolation error vanishes, so the presented flux comparison does not test the position-dependent ePSF interpolation that the paper advertises in Section 1 ("estimate the shape of the Roman PSF at any location within the WFI"). I request an additional test that compares the interpolated ePSF at one or more non-grid positions (e.g., near a corner or between grid points) against a directly computed WebbPSF ePSF at the same positions, quantifying both flux and shape residuals as a function of location.
  2. [Section 3.2, Figures 9-10] The recovery tests labelled "STIPS ePSF Model" use the same ePSF model to inject and to fit the sources; the resulting photometric accuracy of ~0.06 mag and astrometric accuracy of ~0.4 mas therefore reflect internal consistency of the code rather than agreement with an independent PSF. This should be stated explicitly in the text. To make the test more informative, the authors should also recover sources with an independently generated PSF (e.g., a WebbPSF ePSF computed directly at the source position, bypassing the STIPS interpolation) and report the resulting residuals.
  3. [Section 3.1, text versus Figure 8 caption] The text states that the STIPS fluxes deviate by less than about 1% for bright non-saturated sources, while the caption states that the difference is about 3%. These numbers are inconsistent, and because the Pandeia comparison is the main independent validation in the paper, the discrepancy must be resolved.
minor comments (6)
  1. [Equation (1)] The IPC kernel matrix has inconsistent spacing in several entries (e.g., "1 .62" and "0 .20"), which appears to be a LaTeX typesetting error; please fix.
  2. [Figure 3 caption] "near the code of the ePSF" should be "near the core of the ePSF".
  3. [Section 3.1] "less then" should be "less than".
  4. [Figures 9 and 10 captions] "eSPF" is a typo for "ePSF".
  5. [Section 2.4] The cosmic ray rate is given as "5 hits /cm2/s"; use standard units of hits cm^-2 s^-1.
  6. [Section 4] The GitHub repository is cited by URL without a version or DOI; to support reproducibility, please add a software version (e.g., Zenodo DOI or a specific release tag).

Circularity Check

1 steps flagged · score 4.0 of 10

Photometric/astrometric recovery validation is self-referential: STIPS injects sources with its own ePSF model and then recovers them with the same model, so the ~0.06 mag and ~0.4 mas metrics measure internal consistency rather than independent PSF accuracy.

  1. fitted input called prediction [Section 3.1 (Photometry validation), Figure 9; also Section 3.2 (Astrometry), Figure 10 caption]
    "The third method fits the same ePSF model created by STIPS to the sources in the image. [...] using the model of the Roman PSF to fit the simulated sources produces in the most accurate results, where we are able to recover the input magnitudes with a mean deviation of ~0.06 mag. [...] The most accurate method recovers the positions of sources using the same eSPF model that was used to create them in STIPS."

    The ePSF model is both the generator of the simulated 'truth' image and the fitting template used for recovery. Any systematic error in the model cancels between injection and extraction: the recovered-minus-input residuals test only whether the injection and fitting code paths are mutually consistent, not whether the ePSF matches Roman's true PSF. The quoted precision (~0.06 mag, ~0.004 pixels) is therefore forced to be near zero by construction unless the two code paths are implemented inconsistently. This is an internal-consistency check, and presenting it as a validation of PSF accuracy is self-referential. The independent Pandeia flux comparison in Section 3.1 does not share this circularity, which keeps the overall circularity partial.

full rationale

The central claim that STIPS can generate Roman WFI images is independently grounded: Section 3.1 compares STIPS images with Pandeia reference simulations and reports ~1-3% total-flux agreement for bright non-saturated sources, using an external tool with its own PSF generation. The main circular element is the second validation method in Sections 3.1-3.2, where 1000 sources are injected with the STIPS ePSF model and then recovered with the same STIPS ePSF model. That reduction makes the 0.06 mag photometric and 0.4 mas astrometric figures measures of forward/inverse self-consistency, not independent validation. No other load-bearing circular step was found: the IPC kernel is an external GSFC input, the Pandeia background and Sersic profiles are external references, and the Ryan & MacKenty 2023 citation has author overlap but supplies engineering background spectra rather than the target result. The additional concern that validation is performed only at the SCA-center position (2048,2048), which is one of the nine PSF grid points, is a coverage gap for the off-grid interpolation claim rather than a circularity, so it does not add a separate step. Overall, the paper's central simulator capability retains independent content, but one headline validation metric reduces to an identity check, warranting a partial circularity score of 4.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

Central modeling inputs are imported from prior work (Pandeia, WebbPSF, GSFC engineering data) and from fixed values chosen by hand (cosmic ray rate, readout noise, IPC kernel). No new physical entities are posited. The validation includes a self-referential check in which the same ePSF model that generated the image is used to fit it; the external comparison against Pandeia is the main independent grounding.

free parameters (7)
  • cosmic ray rate = 5 hits/cm2/s
    Fixed value used for cosmic ray injection in Section 2.4; not derived from Roman-specific measurements in this paper.
  • cosmic ray energies = 600 and 5000 electrons
    Two fixed energies represent the injected cosmic ray brightness; chosen by hand in Section 2.4.
  • readout noise standard deviation = 12 counts
    Added to each pixel in Section 2.4; an assumed instrument parameter, not fitted in this paper.
  • flat residual amplitude = ~0.06% increase in noise standard deviation
    Used to add flat field residuals in Section 2.4; a fixed estimate of the expected flat residual.
  • IPC kernel = 3x3 matrix in Eq. 1
    Convolution kernel for interpixel capacitance provided by NASA GSFC on 15 Aug 2019; assumed correct for Roman H4RG-10 detectors.
  • PSF interpolation grid = 9 PSFs per SCA, 4x oversampling
    Bilinear interpolation over a grid of 9 WebbPSF PSFs per detector; an approximation that trades accuracy for speed, Section 2.1.
  • background and zeropoint per filter = Table 1 values in counts/s and AB mag
    Default background and zeropoints imported from Pandeia for one benchmark date in 2023; external inputs, not fitted.
assumptions (4)
  • domain assumption WebbPSF model accuracy
    STIPS assumes the WebbPSF PSFs, based on GSFC optical models at five field points per SCA, accurately represent the Roman on-orbit ePSF (Section 2.1).
  • domain assumption Pandeia as benchmark
    Validation treats Pandeia as ground truth for Roman photometry, though it uses an older WebbPSF version that does not include distortion effects (Section 3.1).
  • domain assumption Fixed background model representativeness
    Default zodiacal background is taken from one date (2023-06-19) and assumed representative for all simulated scenes (Section 2.4).
  • domain assumption Interpolation validity
    Bilinear interpolation over 9 PSFs per SCA is assumed to capture PSF spatial variation adequately, with distortion effects not included (Section 2.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of STIPS: The Nancy Grace Roman Space Telescope Imaging Product Simulator." pith.science (2026). https://pith.science/paper/YWHAJK6W

@misc{pith2026241111978,
  author       = {Pith},
  title        = {Pith review of: STIPS: The Nancy Grace Roman Space Telescope Imaging Product Simulator},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YWHAJK6W}},
  note         = {Machine review of arXiv:2411.11978}
}
read the original abstract

The Space Telescope Imaging Product Simulator (STIPS) is a Python-based package that can be used to simulate scenes from the upcoming \textit{Nancy Grace Roman Space Telescope} (\nancy). STIPS is able to generate post-pipeline astronomical images of any number of sensor chip assembly (SCA) detectors, up to the entire 18-SCA Wide-Field Instrument array on \nancy. STIPS can inject either point spread functions generated with {\tt WebbPSF}, or extended sources in any of the \nancy filters. The output images can include flat field, dark current, and cosmic ray residuals. Additionally, STIPS includes an estimate of Poisson and readout noise, as well as an estimate of the zodiacal background and internal background from the telescope. However, STIPS does not include instrument saturation, non-linearity, or distortion effects. STIPS is provided as an open source repository on GitHub.

Figures

Figures reproduced from arXiv: 2411.11978 by the authors.

Figure 1
Figure 1. Simulated image of the M13 globular cluster generated with STIPS for all 18 SCAs in the WFI. The color image was created using photo editing software by combining the output fits files from three STIPS simulations in F106, F129, and F158 filters. The magnitudes and locations of the sources in the scene were obtained from the PS1/3π catalog (Chambers et al. 2016). enty 2023). STIPS does not include instrument satura￾… view at source ↗
Figure 2
Figure 2. Graphical representation of the bilinear interpolation process that STIPS uses to estimate the shape of a PSF within a single SCA. The distortion of the PSFs shown here are greatly exaggerated for visualization purposes. Left: Grid of nine input PSFs evenly distributed across an SCA. Middle: To estimate the shape of a PSF at a requested location (x, y), STIPS weights the shape of the four closest PSFs by the partial… view at source ↗
Figure 3
Figure 3. Input F129 Roman ePSF displayed using a bi￾linear (top) and bicubic (bottom) interpolation method. The difference between the two methods is greatest near the code of the ePSF. The midpoint of the ePSF is at the center of pixel 22, half of the ePSF size of N = M = 44 pixels. In [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Error resulting from using a bilinear as opposed to a bicubic interpolation method when injecting an ePSF onto an image. The two methods are generally equivalent at the level of ∼ 1%, except in the core of the ePSF, where the difference is closer to ∼ 3% [PITH_FULL_IM…
Figure 5
Figure 5. Figure 5: Left: Extended source with n = 1 created using the Sersic2D model within Astropy, Middle: The same extended source created using the SersicDistribution model within Pandeia. Right: A measure of the error resulting from using the quick approximation as opposed to the pr…
Figure 6
Figure 6. Figure 6: Left: Cross-section along the major-axis of a series of extended sources with varying S´ersic profile n. The dashed lines are sources created using the quick Astropy method, while the solid lines use the precise Pandeia method. Right: Ratio of the total integrated flux…
Figure 7
Figure 7. Figure 7: A simulated image of an 18th mag point source in F129 generated with STIPS (Left) and with Pandeia (Middle). The two results are qualitatively very similar. The Right panel shows a map of the error difference between the two PSFs, where the difference is likely dominat…
Figure 8
Figure 8. Figure 8: Ratio between the total flux in a simulated STIPS image and the equivalent generated using Pandeia. Satu￾rated sources are excluded from this plot. For bright non￾saturated sources, STIPS and Pandeia differ at the ∼ 3% level. domly distributed across SCA01 in F129 and …
Figure 9
Figure 9. Figure 9: Results from using different tools to recover the magnitudes of 1000 point sources injected into an image with STIPS. Left: Source Extractor produces increasingly less accurate results for dimmer sources. Middle: Using an Astropy Gaussian model results in a clear syste…
Figure 10
Figure 10. Figure 10: Results from using different tools to recover the positions of 1000 point sources injected into an image with STIPS. The circles represent sources brighter than 21st mag, and the crosses are sources dimmer than this, which have a reduced accuracy. Left: Using Source E…
Figure 11
Figure 11. Figure 11: Difference between the recovered and input pixel positions of sources using the STIPS ePSF model, as a function of magnitude. For the specific simulation setup described in §3.2, the background becomes noticeable for sources dimmer than 21st mag, and we therefore excl…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 9 canonical work pages

  1. [1]

    Anderson, J., & King, I. R. 2000, PASP, 112, 1360 Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013, A&A, 558, A33 Astropy Collaboration, Price-Whelan, A. M., Sip˝ ocz, B. M., et al. 2018, AJ, 156, 123 Astropy Collaboration, Price-Whelan, A. M., Lim, P. L., et al. 2022, ApJ, 935, 167

    work page 2000

  2. [2]

    1996, A&AS, 117, 393

    Bertin, E., & Arnouts, S. 1996, A&AS, 117, 393

    1996

  3. [3]

    2021, astropy/photutils: 1.1.0, 1.1.0, Zenodo, doi:10.5281/zenodo.4624996

    Bradley, L., Sip˝ ocz, B., Robitaille, T., et al. 2021, astropy/photutils: 1.1.0, 1.1.0, Zenodo, doi:10.5281/zenodo.4624996

    doi:10.5281/zenodo.4624996 2021

  4. [4]

    Bradley, L., Sip˝ ocz, B., Robitaille, T., et al. 2022, astropy/photutils: 1.5.0, 1.5.0, Zenodo, Zenodo, doi:10.5281/zenodo.6825092 4https://github.com/spacetelescope/STScI-STIPS 5https://roman-docs.stsci.edu/simulation-tools-handbook-home/ stips-space-telescope-imaging-product-simulator

    doi:10.5281/zenodo.6825092 2022

  5. [5]

    C., Magnier, E

    Chambers, K. C., Magnier, E. A., Metcalfe, N., et al. 2016, arXiv:1612.05560

    arXiv 2016

  6. [6]

    2024, scipy/scipy: SciPy 1.14.0rc2, v1.14.0rc2, Zenodo, doi:10.5281/zenodo.11702230

    Gommers, R., Virtanen, P., Haberland, M., et al. 2024, scipy/scipy: SciPy 1.14.0rc2, v1.14.0rc2, Zenodo, doi:10.5281/zenodo.11702230

    work page doi:10.5281/zenodo.11702230 2024

  7. [7]

    R., Millman, K

    Harris, C. R., Millman, K. J., van der Walt, S. J., et al. 2020, Nature, 585, 357

    2020

  8. [8]

    Hunter, J. D. 2007, Computing in Science and Engineering, 9, 90

    2007

Show all 19 references
  1. [9]

    D., Sivaramakrishnan, A., Lajoie, C.-P., et al

    Perrin, M. D., Sivaramakrishnan, A., Lajoie, C.-P., et al. 2014, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 9143, Space Telescopes and Instrumentation 2014: Optical, Infrared, and Millimeter Wave, ed. J. Oschmann, Jacobus M., M. Clampi...

    2014

  2. [10]

    2012, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol

    Sivaramakrishnan, A. 2012, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 8442, Space Telescopes and Instrumentation 2012:

    2012

  3. [11]

    M., Pickering, T

    Pontoppidan, K. M., Pickering, T. E., Laidler, V. G., et al. 2016, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 9910, Observatory Operations: Strategies, Processes, and Systems VI, ed. A. B. Peck, R. L. Seaman, & C. R. Benn, 991016

    2016

  4. [12]

    R., Lightsey, P

    Rigby, J. R., Lightsey, P. A., Garc ´ ıa Mar ´ ın, M., et al. 2023, PASP, 135, 048002

    2023

  5. [13]

    Rowe, B. T. P., Jarvis, M., Mandelbaum, R., et al. 2015, Astronomy and Computing, 10, 121

    2015

  6. [14]

    Ryan, R., & MacKenty, J. 2023, The Background Spectra for the Roman/WFI Exposure Time Calculator, Technical Report Roman-STScI-000481, https://www.stsci.edu/files/live/sites/www/files/home/ roman/ documents/ Roman-STScI-000481-BackgroundSpectra.pdf

    2023

  7. [15]

    2015, arXiv e-prints, arXiv:1503.03757 STScI Development Team

    Spergel, D., Gehrels, N., Baltay, C., et al. 2015, arXiv e-prints, arXiv:1503.03757 STScI Development Team. 2018, synphot: Synthetic photometry using Astropy, Astrophysics Source Code Library, record ascl:1811.001, ascl:1811.001 van der Walt, S., Sch¨ onberger, J. L., Nunez-Ig...

    2015 arXiv

  8. [16]

    E., et al

    Virtanen, P., Gommers, R., Oliphant, T. E., et al. 2020, Nature Methods, 17, 261

    2020

  9. [17]

    2024, The Software Citation

    Wagg, T., & Broekgaarden, F. 2024, The Software Citation

    2024

  10. [18]

    Station, Zenodo, doi:10.5281/zenodo.11292917

    doi

  11. [19]

    Wagg, T., & Broekgaarden, F. S. 2024, arXiv e-prints, arXiv:2406.04405

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.