REVIEW 3 major objections 5 minor 41 references
exoALMA VII: Benchmarking Hydrodynamics and Radiative Transfer Codes
T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Forward modeling of planet-driven disk kinematics is robust to the choice of code, a benchmark of five hydrodynamics and two radiative transfer codes shows.
desk verdict A genuinely useful benchmark that does what it claims for the tested configuration, but the abstract's 'any combination' conclusion outruns the single-model evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the planet-driven spiral perturbation in gas density and velocity, which produces a localized velocity kink in 12CO channel maps. The benchmark chain consists of hydrodynamics codes that evolve the disk, radiative transfer codes that convert the density and velocity fields into temperatures and synthetic line cubes, and DISCMINER's folded velocity residual maps, which locate the peak residual velocity associated with the kink. The argument works because the strength of the kink is set by the spiral-arm velocity perturbations, which agree across all five hydrodynamics codes, rather than by the depth of the planet-opened gap, which does not.
What would settle it
Run the same five hydrodynamics and two radiative transfer codes for a substantially different regime, such as a 10 Earth-mass planet or a disk with vertical temperature stratification and finite cooling; if retrieved planet positions scatter by more than a beam or brightness temperatures differ by more than the 1.5 K noise level across code combinations, the claim that any tested combination can be used reliably would be refuted.
Extended reading notes
Core claim
The central claim is that any combination of the tested hydrodynamics and radiative transfer codes can be used reliably to model and interpret planet-driven kinematic perturbations in protoplanetary disks. Across the nine hydrodynamics-plus-radiative-transfer combinations, the disk temperature agrees to within about 3 percent or better between mcfost and RADMC-3D models everywhere in the domain, and synthetic 12CO channel maps show brightness temperature differences within 1.5 K, which is around the typical noise level of exoALMA-quality molecular line images. DISCMINER retrieves the planet's radial location at 105 to 116 au for a true value of 100 au and azimuthal location within about 4 degrees of the true value of -45 degrees, with peak residual velocities between 0.054 and 0.088 km/s. The only notable disagreement is that the Phantom simulation opens a much shallower gap than the grid-based codes, but this does not affect the kinematic retrieval, because the spiral-arm density and velocity perturbations that produce the velocity kinks remain consistent.
Load-bearing premise
The conclusion rests on a single benchmark model being representative: one Jupiter-mass planet on a fixed circular orbit at 100 au in an isothermal disk with no magnetic fields, no cooling, and no migration, a limitation the paper itself acknowledges.
Editorial extensions
If this is right
- Planet location retrieval from kinematic signatures in exoALMA-quality data can proceed with any tested code combination, without needing to re-calibrate the result to a particular hydrodynamics or radiative transfer code.
- Brightness temperature differences between code combinations stay below the typical 1.5 K noise, so synthetic cubes from different codes would be observationally indistinguishable at current sensitivity.
- The shallow gap produced by the SPH code does not translate into a biased planet location, so kinematic planet-finding appears decoupled from gap-depth uncertainties.
- Simulations and radiative transfer calculations from different groups can be combined and compared directly in future forward modeling surveys.
- The known radial bias toward larger radii in DISCMINER retrieval persists across all code combinations, indicating it is a property of the retrieval method rather than of any particular code.
Reading between the lines
- The benchmark covers only one model—a Jupiter-mass planet at 100 au in an isothermal disk with no magnetic fields, cooling, or migration—so the blanket conclusion that any combination works is an extrapolation until other regimes are tested.
- If lower-mass planets or vertically stratified, cooling, or magnetized disks produce larger code-to-code differences, the kinematic retrieval may still be robust, but the exact scatter should be re-measured rather than assumed.
- The Phantom simulation's shallow gap, caused partly by mass loss through the inner boundary, suggests that gap-based planet mass estimates from SPH runs could be biased even though kinematic location retrieval is not.
- Comparing the retrieved peak location with both spiral arms, rather than the global peak that favors the outer arm, might reduce the systematic radial offset and is a testable extension of the DISCMINER workflow.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper benchmarks five hydrodynamics codes (FARGO3D, Idefix, Athena++, PLUTO, and Phantom) and two radiative transfer codes (mcfost and RADMC-3D) on a single planet-disk interaction model: a Jupiter-mass planet at 100 au, isothermal equation of state, kinematic viscosity nu=1e-5, no migration, and no magnetic fields. The authors generate synthetic 12CO J=3-2 cubes from nine code combinations, retrieve the planet's radial and azimuthal location with DISCMINER, and compare the results. They report strong consistency: disk temperatures agree within about 3% between mcfost and RADMC-3D, brightness temperatures along the iso-velocity contour agree within ±1.5 K, retrieved planet locations show only a few percent scatter around the true location, and peak residual velocities vary by roughly 20% among models. They conclude that any combination of the tested hydrodynamics and radiative transfer codes can be used to reliably model and interpret planet-driven kinematic perturbations, while also noting in Section 5 that only a single base hydrodynamics model was considered.
Significance. The benchmark is valuable to the protoplanetary disk community because it is the first systematic, controlled comparison of the specific codes used for forward modeling of planet-driven kinematic structures in exoALMA-era observations. The study is carefully set up: code versions are pinned, the initial conditions are fully specified, and the planet is injected with known mass and orbital parameters, so the retrieval comparison has a well-defined ground truth. The quantitative agreement among grid-based codes and between mcfost and RADMC-3D is a useful reference result, and the paper is commendably explicit about the Phantom mass loss and shallow gap. If the conclusions are appropriately qualified to the tested configuration, this will be a useful citable benchmark for the community. The main weakness is not in the measurements themselves but in the breadth of the summary claim, which goes beyond what a single benchmark model and nine of ten possible code combinations can establish.
major comments (3)
- [Abstract and Section 3] The abstract and Section 5 state that 'any combination of the tested hydrodynamics and radiative transfer codes' can be used reliably, but the Phantom + RADMC-3D combination was not tested; Section 3 explicitly lists it as excluded because no RADMC-3D module reads Phantom outputs. Since both Phantom and RADMC-3D are among the 'tested' codes, the blanket statement is not literally supported by the nine tested combinations. Please rephrase to 'the nine combinations tested here' or provide a justification for why the missing combination cannot affect the conclusion.
- [Section 5 and Figure 5] The universal conclusion overreaches the evidence from a single base model. The Phantom simulation shows only about 10% surface density suppression in the gap versus about 70% in the grid-based codes, and the azimuthal velocity perturbation across the gap is about 1% versus about 6% (Figure 5). The DISCMINER retrieval exercises the spiral-arm velocity kinks, which are consistent, but it does not test the gap-induced azimuthal velocity perturbation as an observable. The paper's own final paragraph acknowledges this limitation. The abstract and summary should therefore be narrowed, for example to 'any of the tested code combinations can be used to model and interpret planet-driven spiral kinematic perturbations in this configuration,' and should state explicitly that consistency for gap-depth-based diagnostics is not established by this benchmark.
- [Section 4 and Abstract] The statement that peak residual velocities vary by 'about 20%' appears inconsistent with the reported minimum, maximum, and mean values (0.054 km/s, 0.088 km/s, and 0.074 km/s). The minimum is about 27% below the mean, and the full range is about 46% of the mean. Please report a standard scatter metric (standard deviation or max absolute deviation from the mean) and adjust the abstract and Section 4 wording accordingly, or explain what '20%' refers to.
minor comments (5)
- [Section 3.2.1] There is a typo in 'hydrodynmics codes'; it should read 'hydrodynamics codes.'
- [Throughout] The notation for carbon monoxide should be consistent: use $^{12}$CO with a superscript in all instances rather than the plain text '12CO'.
- [Section 3.1] In the sentence about the stellar photon count, '10 9 photons' is missing a superscript; it should read $10^9$ photons.
- [Section 4] The radial retrieval is reported as systematically biased to a mean of 109 au versus the true 100 au. This is within one beam, but since the bias appears in all models, a brief discussion of its origin beyond the cited prior work would strengthen the paper.
- [Figure 7] The caption refers to a 'yellow dashed square' marking the velocity kink, but in the figure as rendered the square may be difficult to locate; please ensure the marker is clearly visible in the published version.
Circularity Check
No circularity: the benchmark compares independent code outputs against a fixed ground-truth planet location, with no fitted parameter relabeled as a prediction.
full rationale
The paper's central comparison is a ground-truth benchmark: a planet of known mass and position is embedded in each hydrodynamics code, the resulting disk states are post-processed by two radiative transfer codes, and DISCMINER retrieves the planet location from synthetic 12CO cubes. The true planet location (100 au, -45 deg) is an input to the simulations, not a parameter fitted to the retrieval; the retrieved locations are then compared with that input as a test of code consistency. No equation in the paper defines the predicted quantity in terms of the measured quantity, and no fitted parameter is renamed as a prediction. The temperature comparisons, brightness-temperature differences, and retrieval scatters are all independent outputs of the pipeline. The self-referential element is that several authors are developers of the codes being benchmarked, including DISCMINER, but this does not create circularity because the codes are not tuned or calibrated against the benchmark's target outputs; the retrieval is an independent analysis tool run on synthetic data. The paper's own admission that only a single base hydrodynamics model was considered is a generalization or robustness limitation, not a circularity, and does not affect the internal validity of the benchmark. Therefore the derivation chain is self-contained with respect to its stated comparisons.
Assumptions & free parameters
free parameters (9)
- Disk aspect ratio H/R at planet radius =
0.1 at R_p=1
- Temperature power-law index q =
-0.5
- Surface density power-law index p =
-2.25
- Kinematic viscosity nu =
1e-5
- Planet mass M_p =
1e-3 M* (about one Jupiter mass)
- Planet potential smoothing length epsilon =
3*Delta_r ~ 0.024
- CO freeze-out temperature =
20 K
- Dust-to-gas mass ratio =
0.01
- Stellar temperature and radius =
4000 K, 2 R_sun
assumptions (6)
- domain assumption Isothermal equation of state and vertically isothermal disk
- domain assumption Planet is fixed on a circular orbit with no accretion or migration
- standard math Initial conditions satisfy vertical and radial force balance
- domain assumption Damping zones and boundary conditions adequately mimic an infinite disk
- domain assumption Radiative transfer operates in LTE with thermal broadening only and CO abundance 1e-4 above 20 K
- domain assumption The single benchmark model is representative of exoALMA disks
Cite this review
Pith. "Pith review of exoALMA VII: Benchmarking Hydrodynamics and Radiative Transfer Codes." pith.science (2026). https://pith.science/paper/AA6KIXEV
@misc{pith2026250418643,
author = {Pith},
title = {Pith review of: exoALMA VII: Benchmarking Hydrodynamics and Radiative Transfer Codes},
year = {2026},
howpublished = {\url{https://pith.science/paper/AA6KIXEV}},
note = {Machine review of arXiv:2504.18643}
}
abstract
Forward modeling is often used to interpret substructures observed in protoplanetary disks. To ensure the robustness and consistency of the current forward modeling approach from the community, we conducted a systematic comparison of various hydrodynamics and radiative transfer codes. Using four grid-based hydrodynamics codes (FARGO3D, Idefix, Athena++, PLUTO) and a smoothed particle hydrodynamics code (Phantom), we simulated a protoplanetary disk with an embedded giant planet. We then used two radiative transfer codes (mcfost, RADMC-3D) to calculate disk temperatures and create synthetic 12CO cubes. Finally, we retrieved the location of the planet from the synthetic cubes using DISCMINER. We found strong consistency between the hydrodynamics codes, particularly in the density and velocity perturbations associated with planet-driven spirals. We also found a good agreement between the two radiative transfer codes: the disk temperature in mcfost and RADMC-3D models agrees within $\lesssim 3~\%$ everywhere in the domain. In synthetic $^{12}$CO channel maps, this results in brightness temperature differences within $\pm1.5$ K in all our models. This good agreement ensures consistent retrieval of planet's radial/azimuthal location with only a few % of scatter, with velocity perturbations varying $\lesssim 20~\%$ among the models. Notably, while the planet-opened gap is shallower in the Phantom simulation, we found that this does not impact the planet location retrieval. In summary, our results demonstrate that any combination of the tested hydrodynamics and radiative transfer codes can be used to reliably model and interpret planet-driven kinematic perturbations.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[1]
Andrews, S. M. 2020, ARA&A, 58, 483, doi: 10.1146/annurev-astro-031220-010302
-
[2]
2023, in Astronomical Society of the Pacific Conference Series, Vol
Bae, J., Isella, A., Zhu, Z., et al. 2023, in Astronomical Society of the Pacific Conference Series, Vol. 534, Protostars and Planets VII, ed. S. Inutsuka, Y. Aikawa, T. Muto, K. Tomida, & M. Tamura, 423, doi: 10.48550/arXiv.2210.13314 16 Bae et al
-
[3]
Bae, J., Nelson, R. P., & Hartmann, L. 2016, ApJ, 833, 126, doi: 10.3847/1538-4357/833/2/126
-
[4]
2021, ApJ, 912, 56, doi: 10.3847/1538-4357/abe45e
Bae, J., Teague, R., & Zhu, Z. 2021, ApJ, 912, 56, doi: 10.3847/1538-4357/abe45e
-
[5]
2018, ApJ, 859, 118, doi: 10.3847/1538-4357/aabf8c
Bae, J., & Zhu, Z. 2018, ApJ, 859, 118, doi: 10.3847/1538-4357/aabf8c
-
[6]
2019, ApJL, 884, L41, doi: 10.3847/2041-8213/ab46b0
Bae, J., Zhu, Z., Baruteau, C., et al. 2019, ApJL, 884, L41, doi: 10.3847/2041-8213/ab46b0
-
[7]
Bate, M. R., Bonnell, I. A., & Price, N. M. 1995, MNRAS, 277, 362, doi: 10.1093/mnras/277.2.362 Ben´ ıtez-Llambay, P., & Masset, F. S. 2016, ApJS, 223, 11, doi: 10.3847/0067-0049/223/1/11
-
[8]
2024, ApJ, 976, 49, doi: 10.3847/1538-4357/ad83d0
Chen, K., & Dong, R. 2024, ApJ, 976, 49, doi: 10.3847/1538-4357/ad83d0
Show all 41 references
- [9]
-
[11]
P., Juhasz, A., Pohl, A., et al
Dullemond, C. P., Juhasz, A., Pohl, A., et al. 2012, RADMC-3D: A multi-purpose radiative transfer tool, Astrophysics Source Code Library, record ascl:1202.015
2012
-
[12]
F., et al
Galloway-Sprietsma, M., Bae, J., Izquierdo, A. F., et al. 2025, arXiv e-prints, arXiv:2504.19902. https://arxiv.org/abs/2504.19902
2025 arXiv
-
[13]
F., Testi, L., Facchini, S., Rosotti, G
Izquierdo, A. F., Testi, L., Facchini, S., Rosotti, G. P., & van Dishoeck, E. F. 2021, A&A, 650, A179, doi: 10.1051/0004-6361/202140779
2021 doi
-
[14]
F., Testi, L., Facchini, S., et al
Izquierdo, A. F., Testi, L., Facchini, S., et al. 2023, A&A, 674, A113, doi: 10.1051/0004-6361/202245425
2023 doi
-
[15]
F., Stadler, J., Galloway-Sprietsma, M., et al
Izquierdo, A. F., Stadler, J., Galloway-Sprietsma, M., et al. 2025, arXiv e-prints, arXiv:2504.19986. https://arxiv.org/abs/2504.19986
2025 arXiv
-
[16]
2015, MNRAS, 448, 994, doi: 10.1093/mnras/stv025
Takeuchi, T. 2015, MNRAS, 448, 994, doi: 10.1093/mnras/stv025
2015 doi
-
[17]
Lesur, G. R. J., Baghdadi, S., Wafflard-Fernandez, G., et al. 2023, A&A, 677, A9, doi: 10.1051/0004-6361/202346005
2023 doi
-
[18]
Lin, D. N. C., & Papaloizou, J. C. B. 1993, in Protostars and Planets III, ed. E. H. Levy & J. I. Lunine, 749
1993
-
[19]
Lodato, G., & Price, D. J. 2010, MNRAS, 405, 1212, doi: 10.1111/j.1365-2966.2010.16526.x
2010
-
[20]
Lyra, W., Richert, A. J. W., Boley, A., et al. 2016, ApJ, 817, 102, doi: 10.3847/0004-637X/817/2/102
2016 doi
-
[21]
2000, A&AS, 141, 165, doi: 10.1051/aas:2000116
Masset, F. 2000, A&AS, 141, 165, doi: 10.1051/aas:2000116
2000 doi
-
[22]
2007, ApJS, 170, 228, doi: 10.1086/513316
Mignone, A., Bodo, G., Massaglia, S., et al. 2007, ApJS, 170, 228, doi: 10.1086/513316
2007 doi
- [23]
-
[24]
D., & Klahr, H
Muley, D., Melon Fuksman, J. D., & Klahr, H. 2024, A&A, 687, A213, doi: 10.1051/0004-6361/202449739
2024 doi
-
[25]
I., & Lubow, S
Ogilvie, G. I., & Lubow, S. H. 2002, MNRAS, 330, 950, doi: 10.1046/j.1365-8711.2002.05148.x
2002
-
[26]
2006, A&A, 459, 797, doi: 10.1051/0004-6361:20053275
Pinte, C., M´ enard, F., Duchˆ ene, G., & Bastien, P. 2006, A&A, 459, 797, doi: 10.1051/0004-6361:20053275
2006 doi
-
[27]
2023, in Astronomical Society of the Pacific Conference Series, Vol
Pinte, C., Teague, R., Flaherty, K., et al. 2023, in Astronomical Society of the Pacific Conference Series, Vol. 534, Protostars and Planets VII, ed. S. Inutsuka, Y. Aikawa, T. Muto, K. Tomida, & M. Tamura, 645, doi: 10.48550/arXiv.2203.09528
-
[28]
2009, A&A, 498, 967, doi: 10.1051/0004-6361/200811555
Pinte, C., et al. 2009, A&A, 498, 967, doi: 10.1051/0004-6361/200811555
2009 doi
-
[29]
J., M´ enard, F., et al
Pinte, C., Price, D. J., M´ enard, F., et al. 2018, ApJL, 860, L13, doi: 10.3847/2041-8213/aac6dc
2018 doi
-
[30]
2019, Nature Astronomy, 3, 1109, doi: 10.1038/s41550-019-0852-6
Pinte, C., van der Plas, G., M´ enard, F., et al. 2019, Nature Astronomy, 3, 1109, doi: 10.1038/s41550-019-0852-6
2019 doi
-
[31]
2022, MCFOST: Radiative transfer code, Astrophysics Source Code Library, record ascl:2207.023
Pinte, C., M´ enard, F., Duchˆ ene, G., et al. 2022, MCFOST: Radiative transfer code, Astrophysics Source Code Library, record ascl:2207.023
2022
-
[32]
D., Huang, J., et al
Pinte, C., Ilee, J. D., Huang, J., et al. 2025, arXiv e-prints, arXiv:2504.18717. https://arxiv.org/abs/2504.18717
2025 arXiv
-
[33]
J., Wurster, J., Tricco, T
Price, D. J., Wurster, J., Tricco, T. S., et al. 2018, PASA, 35, e031, doi: 10.1017/pasa.2018.25
2018 doi
-
[34]
Rafikov, R. R. 2002, ApJ, 572, 566, doi: 10.1086/340228
2002 doi
-
[35]
2015, ApJ, 804, 95, doi: 10.1088/0004-637X/804/2/95
Turner, N. 2015, ApJ, 804, 95, doi: 10.1088/0004-637X/804/2/95
2015 doi
-
[36]
P., Longarini, C., Paneque-Carre˜ no, T., et al
Rosotti, G. P., Longarini, C., Paneque-Carre˜ no, T., et al. 2025, arXiv e-prints, arXiv:2504.20012. https://arxiv.org/abs/2504.20012
2025 arXiv
-
[37]
I., & Sunyaev, R
Shakura, N. I., & Sunyaev, R. A. 1973, A&A, 24, 337
1973
-
[38]
M., Tomida, K., White, C
Stone, J. M., Tomida, K., White, C. J., & Felker, K. G. 2020, ApJS, 249, 4, doi: 10.3847/1538-4365/ab929b
2020 doi
-
[39]
2025, arXiv e-prints, arXiv:2504.18688
Teague, R., Benisty, M., Facchini, S., et al. 2025, arXiv e-prints, arXiv:2504.18688. https://arxiv.org/abs/2504.18688
2025 arXiv
-
[40]
C., & Draine, B
Weingartner, J. C., & Draine, B. T. 2001, ApJ, 548, 296, doi: 10.1086/318651
2001 doi
-
[41]
2019, ApJ, 884, 142, doi: 10.3847/1538-4357/ab3fab
Yun, H.-G., Kim, W.-T., Bae, J., & Han, C. 2019, ApJ, 884, 142, doi: 10.3847/1538-4357/ab3fab
2019 doi
-
[42]
2018, ApJL, 869, L47, doi: 10.3847/2041-8213/aaf744
Zhang, S., Zhu, Z., Huang, J., et al. 2018, ApJL, 869, L47, doi: 10.3847/2041-8213/aaf744
2018 doi
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.