Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

The Kepler Peas in a Pod Pattern is Astrophysical

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Kepler's 'peas in a pod' pattern is real, not a detection artifact.

desk verdict The critique of Zhu's SNR-resampling is correct and worth publishing, but the paper's own >10-sigma null-test claim is hostage to an untested width of the radius prior. read the letter →

arxiv 1908.05833 v3 pith:345FYTJS submitted 2019-08-16 astro-ph.EP

Lauren M. Weiss , Erik A. Petigura This is my paper · ORCID
classification astro-ph.EP
keywords Keplermulti-planetsystemspeasinapodplanetradiuscorrelationdetectionbiasbootstrapresamplingtransitsignal-to-noiseperiodratiosplanetarysystemarchitecture
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the Kepler 'peas in a pod' pattern—planets around the same star having similar sizes and regular orbital spacing—is astrophysical. It challenges a recent claim that detection bias produces the pattern, showing that a resampling test based on transit signal-to-noise (SNR) does not generate random, independent planet radii. Because each resampled radius is the observed radius multiplied by $\sqrt{\mathrm{SNR}_{\mathrm{new}}/\mathrm{SNR}_{\mathrm{obs}}}$, the resampled radii inherit the original correlation. When the authors instead sample radii directly from a log-normal distribution and apply a simple SNR $\ge 10$ detection threshold, the null hypothesis of random radii yields Pearson $R=0.023\pm0.044$, far below the observed $R=0.62$, ruling out detection bias at $>10\sigma$ confidence.

What carries the argument

The central object is the radius-SNR identity $R_{p,\mathrm{new}} = R_{p,\mathrm{obs}}\sqrt{\mathrm{SNR}_{\mathrm{new}}/\mathrm{SNR}_{\mathrm{obs}}}$, derived by rearranging Kepler's SNR equation. This identity is what makes SNR resampling invalid for null-hypothesis testing: the observed radius appears as a multiplicative factor, so the resampled radii are not independent of the observed radii. The paper's alternative machinery is a bootstrap null test that draws planet radii directly from a log-normal distribution with $\mu=0$ and $\sigma=1$, redrawing until SNR $\ge 10$, and compares the Pearson-$R$ correlation of adjacent radii across 1000 trials.

What would settle it

Obtain Kepler's empirically measured detection completeness as a function of SNR and orbital period, use it in place of the SNR $\ge 10$ step function, and draw radii from a distribution calibrated to the CKS occurrence rates; if the null then yields Pearson $R\simeq0.62$, the claim that detection bias is ruled out is overturned.

Watch

Extended reading notes

Core claim

The central claim is that a bootstrap test which draws transit SNR and converts it to radius cannot test whether the observed correlation in planet radii is a detection artifact. The identity $R_{p,\mathrm{new}} = R_{p,\mathrm{obs}}\sqrt{\mathrm{SNR}_{\mathrm{new}}/\mathrm{SNR}_{\mathrm{obs}}}$ shows why: the new radius is proportional to the observed radius, so the correlation structure of the observed radii survives resampling. This is demonstrated explicitly by applying the SNR-resampling method to a mock universe in which all planets in a system are identical by construction, and finding that the resampled radii still show a correlation. The paper's own null test resamples radius directly, redrawing until a synthetic planet passes SNR $\ge 10$, and finds essentially no correlation in 1000 trials per system. It also shows that a period-ratio cut based on the product of adjacent ratios being less than 25 induces an artificial anti-correlation, while the CKS data show positively correlated period ratios in compact systems (period ratio < 4).

Load-bearing premise

The $>10\sigma$ rejection assumes that Kepler's detection is completely described by a fixed SNR $\ge 10$ cutoff and that the true underlying radius distribution is a log-normal with $\mu=0$, $\sigma=1$; if either is far from reality, the null distribution of Pearson $R$ could shift enough to change the significance.

Editorial extensions

If this is right

  • The observed radius correlation ($R=0.62$) is not reproduced by a random-radius null convolved with Kepler's detection threshold; the null gives $R=0.023\pm0.044$, so detection bias is excluded at $>10\sigma$.
  • Any future test of whether the peas-in-a-pod pattern is biased must resample the parameter of interest (radius or period ratio), not a derived detection statistic such as SNR.
  • For compact systems with period ratio below 4, adjacent period ratios in CKS multis are correlated, and the apparent randomness argued in the counter-analysis comes from a cut that mathematically forces an anti-correlation.
  • The astrophysical origin of the pattern reinforces forward-modeling studies in which correlated sizes and regular spacings are needed to reproduce the Kepler multis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A concrete extension: re-run the radius-resampling null with Kepler's full measured completeness map instead of a fixed SNR=10 step to test whether the $>10\sigma$ exclusion survives a more realistic sensitivity function.
  • The same radius-resampling logic could be applied to planets detected by the TESS mission, where the detection threshold differs, to see whether peas-in-a-pod is a universal feature of close-in multi-planet architectures.
  • The SNR-resampling artifact is a general caution: any bootstrap that reuses the observed target quantity to generate a synthetic realization will bias the null toward the observed pattern, beyond the planet-radius case.
  • The significance is tied to the log-normal prior with $\sigma=1$; if the true underlying radius distribution is much steeper, the exclusion strength could weaken.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript critiques the resampling method of Zhu (2019, Z20), which draws transit SNR at random and converts to planet radius, and argues that this procedure does not produce random, independent planet radii. The authors show algebraically (Eq. 4) that the resampled radius is proportional to the observed radius, so any correlation in the observed radii is partially preserved and the Z20 null test is unsuitable. They then present their own null test: drawing planet radii directly from a log-normal prior (mu=0, sigma=1) and redrawing until SNR>=10, claiming this rules out detection bias as the source of the peas-in-a-pod radius correlation with >10-sigma confidence. The paper also critiques Z20's period-ratio cut, arguing that the product cut induces a dependence between adjacent period ratios, and concludes that the period ratios of adjacent planets are indeed correlated when both are < 4.

Significance. If the central critique of Z20 is correct, the paper corrects a methodological error in a high-profile claim and strengthens the astrophysical interpretation of the peas-in-a-pod pattern. The derivation of Eq. 4 is elegant and the demonstration that Z20's resampling preserves the observed radius is convincing. The paper also makes a useful point about the period-ratio cut. However, the paper's own headline quantitative claim (>10-sigma rejection of detection bias) depends on untested assumptions about the underlying radius distribution, and the SNR-based test in Section 5 appears to use the same flawed logic the authors attribute to Z20. The significance of the paper is therefore real but would be substantially increased by robustness tests and a coherent SNR analysis.

major comments (4)
  1. [§3.1, Eq. (1) and null-test description] The >10-sigma claim is not robust to the choice of the intrinsic radius distribution. The redraw-until-detected procedure conditions each planet's radius on SNR>=10, and because SNR contains a system-specific factor (Rstar, CDPP6hr, period), the detectable radius floor is common to all planets in a system. With a wide prior (sigma=1), intrinsic radius scatter dominates and keeps the null Pearson-R small, but with a narrower prior (e.g., sigma=0.2-0.3) the common detection floor can induce substantial within-system correlations in the synthetic detected radii. The paper does not vary sigma, does not fit mu and sigma to the observed CKS radius distribution after applying the detection model, and does not compare the synthetic detected radius distribution to the observed one (e.g., with a KS test). As a result, the statement that detection bias is ruled out with >10-sigma confidence is not yet supported; the null distribution of the test statistic is sensitive to the assumed marginal radius distribution.
  2. [§5, SNR correlation test] The 8-sigma test of adjacent SNR correlation is circular with respect to the paper's own critique of Z20. The authors test whether drawing SNRs at random from the observed SNR distribution can reproduce the observed correlation between adjacent transit SNRs, and find it cannot. But this null destroys any correlation that would arise from the common system properties (Rstar, CDPP) entering the SNR expression (Eq. 1) even if the intrinsic radii are independent. A valid null for 'no astrophysical SNR correlation' should draw radii independently from a chosen prior, apply Eq. 1 with the observed stellar parameters, and compute the resulting SNR correlation. As written, the §5 analysis does not demonstrate that the observed SNR correlation is astrophysical; it only shows that the observed SNR correlation is not consistent with randomly permuted SNRs, which is the same flawed approach used in Z20.
  3. [§4, period-ratio analysis] The claim that the observed CKS period ratios are 'inconsistent with random period ratios' is based on comparing the data to a uniform distribution in log period ratio after applying the cut log(Pj)+log(Pj+1)<4. While the demonstration that this cut induces an anti-correlation in a uniform sample is mathematically correct, the uniform distribution is not a realistic null for Kepler's detection and dynamical stability constraints. The actual prior over period ratios is strongly shaped by the requirement of detectability and by stability, so the induced anti-correlation in the uniform null may not correspond to what a realistic random model would predict. To support the conclusion that the observed R=0.15 (rather than the expected -0.5 under the uniform null) is evidence for regularity, the authors should repeat the experiment with a forward model that draws periods from a realistic period distribution and applies the actual detection efficiency, or at least test the sensitivity of the induced correlation to the assumed period prior.
  4. [§3.1, footnote 4] The paper states that discounting undetected planets rather than redrawing 'did not differ significantly' from the redraw procedure, but no quantitative support is given. Since the choice of redraw-until-detected affects the effective radius distribution and hence the null Pearson-R, the paper should either show the comparison or state that this is a claim from previous work.
minor comments (4)
  1. [§4 and Figure 4] The text reports the observed period-ratio correlation after the Z20 cut as R=0.15 with p<10^-5, while the caption of Figure 4 states R=0.15, p=0.04. These values are inconsistent and should be reconciled.
  2. [§2, first paragraph] The statement that in the 10 systems with four or more planets all smaller than 1.8 R_Earth the correlation 'cannot be explained by detection bias' is presented as an argument but is not a statistical test. This is a reasonable motivating example, but the wording overstates its evidential weight.
  3. [§3.1, Eq. (1)] The detection model in Eq. (1) ignores several effects (impact parameter, eccentricity, finite observing window) at the stated level of ~10%. The paper argues these are unimportant because 70% of CKS multis have SNR>20. That argument is reasonable, but a reader may wonder whether the redraw-until-detected procedure is sensitive to the exact threshold behavior at SNR~10; a brief discussion of this sensitivity would strengthen the presentation.
  4. [General] The manuscript is a reply to Zhu (2019) and would benefit from explicit statements of the sample size and the exact definition of Pearson-R (e.g., whether radii are logarithmically transformed) used in the null tests, as these details are left implicit.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central null-hypothesis test is computed from independently drawn radii, and the W18 self-citations are background, not load-bearing.

full rationale

The paper's central quantitative claim is produced by a resampling null test: synthetic planet radii are drawn independently from a log-normal distribution, redrawn until SNR >= 10, and then the Pearson-R statistic is computed over the resulting detected pairs. This is a genuine null construction. No parameter of the null is fitted to the observed correlation; the null distribution (R=0.023±0.044) is computed, not assumed, and is then compared with the observed R=0.62. The conclusion is therefore not encoded in the inputs by construction. The critique of Zhu (2019) follows algebraically from Eq. 4, R_new = R_obs sqrt(SNR_new/SNR_obs), which shows that resampling SNR preserves a multiplicative factor of the observed radius. This is a direct mathematical reduction of the criticized method, not a circular use of the paper's own conclusion. Self-citations to W18 and CKS provide the empirical pattern and the original resampling recipe, but the present paper re-runs the null test with fresh random draws rather than importing the conclusion. Figure 1 is reproduced from W18, yet the pattern is visible in the data and is independently checked by the new null test. Thus self-citation is present but not load-bearing. The skeptic's concern about the choice of the log-normal prior (mu=0, sigma=1) and the fixed SNR>=10 threshold is a model-robustness or correctness issue, not circularity: the prior is an input assumption, not a fitted parameter, and no quantity from the observed correlation is used to tune it. The paper's >10-sigma statement is conditional on its null model; whether that model is the appropriate null is an external statistical question, not a self-referential reduction. By the hard rules, robustness concerns without an exhibited Eq. X = Eq. Y reduction do not constitute circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new physical entities are introduced. The analysis depends on two hand-set statistical inputs (radius prior and SNR threshold) and a simplified sensitivity model, plus one hand-set period-ratio cut.

free parameters (3)
  • Log-normal radius distribution parameters (mu, sigma) = mu = 0, sigma = 1 (chosen by hand)
    Used as the null distribution for planet radii in the bootstrap; not fitted to data, but the shape of this prior affects the null distribution of Pearson-R and hence the claimed significance.
  • SNR detection threshold = 10
    Adopted as the detection cutoff for the synthetic sensitivity model; planets drawn below this are redrawn. Different thresholds would change which planets enter the null sample.
  • Period ratio cut for correlation test = Pj < 4 and Pj+1 < 4
    Used to define the sample for the period-ratio correlation test; the boundary is chosen to avoid the dependent region of the Z20 product cut, but the exact value is not derived from first principles.
assumptions (3)
  • domain assumption The SNR equation (Eq. 1) adequately models Kepler's detection sensitivity for the CKS sample, ignoring impact parameter, eccentricity, and observing-window effects.
    Invoked in Section 3.1; the authors state these effects contribute ~10% uncertainty and argue this is unimportant because most CKS multis have SNR > 20, but this is an assumption about the low-SNR tail.
  • domain assumption Redrawing undetected planets until SNR >= 10 replicates the CKS selection function and preserves the observed multiplicity distribution.
    Section 3.1 states this reproduces the selection criteria for CKS multis; this assumes the observed multiplicities are not themselves biased in a way that the redraw procedure fails to capture.
  • standard math Any planet radius distribution with independent draws is a valid null hypothesis for testing whether neighbor radii are correlated.
    Section 3.1 justifies switching to a log-normal distribution; the statement is true for the null of independence, though the power of the test depends on the chosen distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Kepler Peas in a Pod Pattern is Astrophysical." pith.science (2026). https://pith.science/paper/345FYTJS

@misc{pith2026190805833,
  author       = {Pith},
  title        = {Pith review of: The Kepler Peas in a Pod Pattern is Astrophysical},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/345FYTJS}},
  note         = {Machine review of arXiv:1908.05833}
}
abstract

\Kepler\ planets around a given star have similar sizes to each other and regular orbital spacing, like "peas in a pod." Several studies have tested whether detection bias could produce this apparent pattern by resampling planet radii at random and applying a sensitivity function analogous to that of the Kepler spacecraft. However, \citet{Zhu2019} argues that this pattern is not astrophysical but an artifact of Kepler's discovery efficiency at the detection threshold. To support this claim, their new analysis samples the transit signal-to-noise ratio (SNR) to derive a synthetic population of bootstrapped planet radii. Here, we examine the procedure of sampling transit SNR and demonstrate it is not applicable. Sampling transit SNR does not set up random, independent planet radii, and so it is unsuitable for corroborating (or falsifying) detection bias as the origin of apparent patterns in planet radius. By sampling the planet radii directly and using a simple model for Kepler's sensitivity, we rule out detection bias as the source of the peas-in-a-pod pattern with $>10$-$\sigma$ confidence.

Figures

Figures reproduced from arXiv: 1908.05833 by the authors.

Figure 1
Figure 1. [ Systems from the California-Kepler Sur￾vey with 4 or more transiting planets. Each row corresponds to a planetary sys￾tem, with the star KOI number at the left, and planets represented with their measured semi-major axis (x-axis) and physical radius (point size). The color corresponds to equilibrium temperature. The systems are ranked by stellar mass, for which the errors were typically 5%. In many systems, the pl… view at source ↗
Figure 2
Figure 2. Drawing SNR at random produces correlated planet radii in two samples in which [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Drawing SNR at random produces non-random, non-independent planet radii. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: A comparison of the Pearson-R correlations of the observed CKS period ratios (blue) and draws from a random uniform distribution (gray), after applying various cuts. Left: the gray points are drawn randomly from 0 < log2(P) < 4. There is no correlation between the rand…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 43 citations worldwide. Full citation record

  1. The Radio Properties of Extreme Coronal Line Emitters: Constraints on the Sub-parsec Environment

    astro-ph.HE 2026-07 conditional novelty 6.0 of 10

    About half of low-redshift ECLEs are radio-bright like TDEs/AGN; SED modeling of four shows the ECL gas is clumpy (f_V ~ 10^{-5}-10^{-2}) and spatially distinct from the radio-emitting region.

Reference graph

Works this paper leans on

19 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    J., Koch, D

    Borucki, W. J., Koch, D. G., Basri, G., et al. 2011, ApJ, 736, 19, doi: 10.1088/0004-637X/736/1/19

    doi:10.1088/0004-637x/736/1/19 2011

  2. [2]

    L., Jenkins, J

    Christiansen, J. L., Jenkins, J. M., Caldwell, D. A., et al. 2012, PASP, 124, 1279, doi: 10.1086/668847

    doi:10.1086/668847 2012

  3. [3]

    L., Clarke, B

    Christiansen, J. L., Clarke, B. D., Burke, C. J., et al. 2016, ApJ, 828, 99, doi: 10.3847/0004-637X/828/2/99

    doi:10.3847/0004-637x/828/2/99 2016

  4. [4]

    R., Fabrycky, D

    Ciardi, D. R., Fabrycky, D. C., Ford, E. B., et al. 2013, The Astrophysical Journal, 763, 41, doi: 10.1088/0004-637X/763/1/41

    doi:10.1088/0004-637x/763/1/41 2013

  5. [5]

    J., Kratter, K

    Dupuy, T. J., Kratter, K. M., Kraus, A. L., et al. 2016, ApJ, 817, 80, doi: 10.3847/0004-637X/817/1/80

    doi:10.3847/0004-637x/817/1/80 2016

  6. [6]

    C., Lissauer, J

    Fabrycky, D. C., Lissauer, J. J., Ragozzine, D., et al. 2014, ApJ, 790, 146, doi: 10.1088/0004-637X/790/2/146

    doi:10.1088/0004-637x/790/2/146 2014

  7. [7]

    2013, ApJ, 767, 115, doi: 10.1088/0004-637X/767/2/115

    Fang, J., & Margot, J.-L. 2013, ApJ, 767, 115, doi: 10.1088/0004-637X/767/2/115

    doi:10.1088/0004-637x/767/2/115 2013

  8. [8]

    Y., Ford, E

    He, M. Y., Ford, E. B., & Ragozzine, D. 2019, MNRAS, 490, 4575, doi: 10.1093/mnras/stz2869

    doi:10.1093/mnras/stz2869 2019

Show all 19 references
  1. [9]

    A., Petigura, E

    Johnson, J. A., Petigura, E. A., Fulton, B. J., et al. 2017, AJ, 154, 108, doi: 10.3847/1538-3881/aa80e7

    2017 doi

  2. [10]

    J., Ragozzine, D., Fabrycky, D

    Lissauer, J. J., Ragozzine, D., Fabrycky, D. C., et al. 2011, ApJS, 197, 8, doi: 10.1088/0067-0049/197/1/8

    2011 doi

  3. [11]

    2017, ArXiv e-prints

    Millholland, S., Wang, S., & Laughlin, G. 2017, ArXiv e-prints. https://arxiv.org/abs/1710.11152

    2017 arXiv

  4. [12]

    D., Pascucci, I., Apai, D., & Ciesla, F

    Mulders, G. D., Pascucci, I., Apai, D., & Ciesla, F. J. 2018, AJ, 156, 24, doi: 10.3847/1538-3881/aac5ea

    2018 doi

  5. [13]

    A., Howard, A

    Petigura, E. A., Howard, A. W., Marcy, G. W., et al. 2017, AJ, 154, 107, doi: 10.3847/1538-3881/aa80de The Kepler Peas in a Pod Pattern is Astrophysical 11

    2017 doi

  6. [14]

    F., Bryson, S

    Rowe, J. F., Bryson, S. T., Marcy, G. W., et al. 2014, ApJ, 784, 45, doi: 10.1088/0004-637X/784/1/45

    2014 doi

  7. [15]

    2019, MNRAS, 489, 3162, doi: 10.1093/mnras/stz2350

    Sandford, E., Kipping, D., & Collins, M. 2019, MNRAS, 489, 3162, doi: 10.1093/mnras/stz2350

    2019 doi

  8. [16]

    M., Marcy, G

    Weiss, L. M., Marcy, G. W., Petigura, E. A., et al. 2018, AJ, 155, 48, doi: 10.3847/1538-3881/aa9ff6

    2018 doi

  9. [17]

    2016, Proceedings of the National Academy of Science, 113, 11431, doi: 10.1073/pnas.1604692113

    Xie, J.-W., Dong, S., Zhu, Z., et al. 2016, Proceedings of the National Academy of Science, 113, 11431, doi: 10.1073/pnas.1604692113

    2016 doi

  10. [18]

    2019, arXiv e-prints, arXiv:1907.02074

    Zhu, W. 2019, arXiv e-prints, arXiv:1907.02074. https://arxiv.org/abs/1907.02074

    2019 arXiv

  11. [19]

    K., & Hansen, B

    Zink, J. K., & Hansen, B. M. S. 2019, MNRAS, 487, 246, doi: 10.1093/mnras/stz1246

    2019 doi

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.