{"id":"95f46c96-7876-4b61-8849-c855bfc4985f","arxiv_id":"1908.07514","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"The paper's parametric bootstrap substitutes the sample-size variance 1/(N+1/2) with the standard error of the sample standard deviation, biasing the MMD and invalidating its significance tests.","lead":"This paper proposes a parametric bootstrap for the Mean Measure of Divergence, a distance used on binary skeletal traits in anthropology. The proposed correction, however, uses the spread of sample standard deviations as a variance, which is the wrong quantity and inflates all distances.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The correction S^2 in Eq. (7) is the standard error of the sample standard deviation, not the variance; it is O(N^-2) while the true variance is O(N^-1), so MMDB is positively biased under the null and stMMDB is not a valid N(0,1) test statistic.","rationale":"The reader's weakest assumption identifies exactly the point on which the paper's argument depends: that S_mu_i from Eq. (7) can replace sigma_mu_i in the MMD. My reading of Sec. 2 and Sec. 4 confirms that this is the single load-bearing step, and it fails. The paper itself derives S ~ A/N^beta with beta ~ 1, which makes S^2 an order N smaller than the true variance 1/(N+1/2). The bootstrap procedure never resamples the MMD statistic; it converts the known variance into a much smaller quantity and then treats that quantity as if it corrected the mean of MMD. The numerical examples are consistent with this diagnosis: stMMDB values become enormous because both the numerator and the denominator of Eq. (2) are rescaled by the wrong power of N. I do not see a way to reinterpret Eqs. (4)-(7) that avoids this problem, and the paper provides no null simulation or calibration check that would establish otherwise. My recommendation is therefore to maintain the reader's REJECT verdict; the proposed parametric bootstrap is not a valid correction, and the real-data significance statements, such as alpha < 10^-17, are not trustworthy because the statistic is not calibrated. This is a substantive statistical flaw, not a stylistic or framing issue.","tokens_in":15032,"tokens_out":4456,"duration_ms":49567,"concrete_test":"Simulate the null case: choose P=4 populations, M=13 traits, identical trait probability p=0.3, and moderate trait sample sizes N_mu_i between 20 and 50. For every population pair, compute stMMDB exactly as in Sec. 2 with B=500. Repeat this simulation 1000 times and compare the empirical distribution of stMMDB with the N(0,1) distribution that the original stMMD should follow under the null. If the 95th percentile is close to 1.96, the correction is valid; if, as the O(1/N^2) argument predicts, nearly all values exceed 2, the procedure is not a valid significance test and the concern is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central load-bearing step is Eq. (7): S_mu_i is defined as the standard deviation, across B bootstrap repetitions, of the sample standard deviations sigma*_mu_i computed from N(0, sigma_mu_i) samples. That quantity is the standard error of the estimated standard deviation, not an estimate of sigma_mu_i or of sigma^2_mu_i. Under the paper's own normal model, var(sigma*) is approximately sigma^2/(2(N-1)), so S is approximately sigma/sqrt(2N), and S^2 is approximately 1/(2N^2). The true variance correction in the MMD is sigma^2 = 1/(N+1/2), of order 1/N. Section 4 confirms this: Eq. (9) gives S ~ A/N^beta with beta approximately 1. Consequently, MMDB subtracts S^2_mu_i + S^2_nu_i, a quantity of order N^-2, instead of the dominant sampling noise of order N^-1. Under the null, E[(theta_mu_i - theta_nu_i)^2] = sigma^2_mu_i + sigma^2_nu_i, so E[MMDB] is positive and stMMDB is not approximately N(0,1). The inflated entries in Tables S3, S6, and S9, including stMMDB values above 10^3, and the changed dendrograms, are artifacts of this miscalibration rather than evidence of correction. The paper never checks the null distribution of stMMDB against the known N(0,1) benchmark given in Eq. (2). Thus the central claim is not supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a parametric bootstrap procedure for the Mean Measure of Divergence (MMD), a biodistance measure used in anthropology with non-metric traits. The authors use the Anscombe transformation, which stabilizes the variance so that each trait estimate is approximately N(0, sigma^2) with sigma^2 = 1/(N + 1/2). The proposed bootstrap, described in Section 2, repeatedly draws normal samples of size N, computes the sample standard deviation sigma*, and then takes S, the standard deviation of these sigma* values across B bootstrap repetitions (Eq. 7). The bootstrap-corrected distance MMDB subtracts S^2 terms, and the standardized stMMDB divides by a sigma_B built from S^2. The paper applies the method to artificial data with good and poor statistical representativeness and to real anthropological data, showing that the bootstrap changes dendrograms when the relative error E (Eq. 12) exceeds 10%. The authors conclude that the parametric bootstrap is a powerful tool for small or poorly represented samples.","tokens_in":15387,"tokens_out":3752,"duration_ms":40005,"significance":"If the proposed bootstrap correction were valid, it would provide a practical tool for biodistance analysis with small samples, a common situation in anthropology and paleoanthropology. The paper clearly states the Anscombe-based null distribution, explicitly defines the bootstrap algorithm, and identifies the standard MMD and stMMD formulas. However, the central methodological step, replacing the known variance parameter sigma^2 by the square of the bootstrapped standard deviation S, is internally inconsistent with the paper's own asymptotic analysis in Section 4. Since S scales as O(1/N) while the required variance is O(1/N), the correction removes a term of the wrong order, leaving the proposed stMMDB systematically inflated. The reported extreme stMMDB values and the claim that all real-data distances are significant at alpha <= 10^-17 are artifacts of this miscalibration. The paper provides no simulation or theoretical check of the null distribution of stMMDB, which is essential for a procedure that claims to test statistical representativeness.","major_comments":[{"comment":"Equation (7) defines S as the standard deviation, across B bootstrap repetitions, of the sample standard deviations sigma*_mu i computed from N(0, sigma_mu i) samples. This quantity estimates the standard error of the estimated standard deviation, not the variance parameter sigma^2_mu i. For normal data with true sigma, Var(sigma*) is approximately sigma^2 / (2(N-1)), so S is of order sigma / sqrt(N) ~ 1/N, and S^2 is of order 1/N^2. In contrast, the correction term that must be subtracted in the MMD is sigma^2_mu i = 1/(N_mu i + 1/2), which is of order 1/N. Substituting S^2 for sigma^2 therefore removes a term much smaller than the dominant sampling noise. Under the null, E[(theta_mu i - theta_nu i)^2] = sigma^2_mu i + sigma^2_nu i, so E[MMDB] is not zero but systematically positive, and stMMDB cannot be approximately N(0,1).","section":"Section 2, Eqs. (4)-(7)"},{"comment":"Section 4 itself confirms the problem. Equation (9) reports a least-squares fit S_ls = A / N^beta with beta = 1.0028, and the text states S is of order O(1/N). Thus S^2 is of order O(1/N^2), whereas the true variance of theta is of order O(1/N). This is not a minor mismatch in constants: the bootstrap procedure substitutes a quantity of the wrong asymptotic order into the variance correction. The paper never verifies the null distribution of stMMDB against the known N(0,1) benchmark given in Section 1, and the claimed correction therefore lacks any statistical justification.","section":"Section 4, Eq. (9)"},{"comment":"The bootstrapped distance matrices contain stMMDB values exceeding 1000 in the artificial data and values like 3950.88 in the real-data matrix (Table S9). These enormous values are a direct consequence of dividing by sigma_B, which is constructed from S^2 terms of order N^-2 and is therefore much smaller than the true standard deviation of the MMD. The assertion in Section 7 that all distances after bootstrap are statistically representative at alpha <= 10^-17 is an artifact of this miscalibration, not a meaningful test result. Without a simulation study that checks the empirical null distribution of stMMDB, the procedure's stated property of yielding significance levels is unsupported.","section":"Section 7 and Tables S3, S6, S9"}],"minor_comments":[{"comment":"The text before Eq. (2) refers to the 'standardized MMS biodistance'; this should read 'standardized MMD biodistance'.","section":"Section 1"},{"comment":"There is a typo 'Is is important' in the paragraph after the MMD variance formula; it should be 'It is important'.","section":"Section 1"},{"comment":"The phrase 're-sampling the standard deviation sigma_mu i through N(0, sigma_mu i)' is ambiguous. The procedure actually draws theta-star values from N(0, sigma_mu i) and computes their sample standard deviation; rephrasing would improve clarity.","section":"Section 2"},{"comment":"The asymptotic notation in the line 'sigma* is approximately O(1/sqrt(N)) sigma' is imprecise. The intended statement should be that the standard error of sigma* is O(1/N), not that sigma* itself is O(1/N) as written later.","section":"Section 4"},{"comment":"Tables S3, S6, and S9 are printed in a truncated format (only the upper-left block is shown for S3, for example). Full matrices are needed to reproduce the reported E values and dendrogram comparisons.","section":"Supplementary Material"}],"recommendation":"reject","confidential_remarks":"The central derivation is internally inconsistent with the paper's own Section 4: the quantity S defined by Eq. (7) is a standard error of order O(1/N), and its square cannot serve as the O(1/N) variance correction required by the MMD. This is a load-bearing error that invalidates the proposed procedure and the reported significance claims, so a revision within the manuscript's scope would require changing the core method rather than patching local issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the paper is readable, addresses a real need, and the proposed ingredient is genuinely new to the MMD/bootstrap literature, but the central step is wrong, and the paper's own Section 4 shows it. I would not use this method, and I would not publish it as is, but it deserves a real referee rather than a desk reject.\n\nWhat is good: MMD is standard in bioarchaeology, and small or incomplete skeletal samples are a genuine problem. The authors clearly explain the Anscombe transform and the known null variance sigma^2 = 1/(N + 1/2), give explicit bootstrap steps, and illustrate with artificial and real data. They also cite the relevant transformation and bootstrap literature. The exposition is transparent enough that the flaw is easy to isolate.\n\nThe flaw is in Eq. (7). S_mu_i is the standard deviation, over B bootstrap repetitions, of the sample standard deviations sigma* drawn from N(0, sigma_mu_i). That is the standard error of the estimated SD, not an estimate of sigma_mu_i or sigma^2_mu_i. For normal data, var(sigma*) is approximately sigma^2/(2(N-1)), so S is approximately sigma/sqrt(2N). Since sigma is about 1/sqrt(N), S is O(1/N) and S^2 is O(1/N^2). The MMD variance correction has to be O(1/N). So MMD_B subtracts a negligible quantity, remains positively biased under the null, and stMMD_B is not approximately N(0,1). Section 4 is not a defense; it is the smoking gun: the authors fit S approximately 0.39/N and seem to regard the 1/N scaling as fine. That is exactly the wrong scaling for a variance. The huge stMMD_B entries in Tables S3, S6, and S9, and the claim that all real-data distances are significant at alpha <= 1e-17, are artifacts of this miscalibration. There is also no check of stMMD_B's null distribution against the known N(0,1) benchmark, which would have exposed the problem immediately.\n\nThe simulations demonstrate the procedure's behavior but not its validity, because there is no ground-truth comparison. The real-data application is cited to the authors' own conference abstract, so it is not independent evidence. On the plus side, the algorithm is simple enough to reproduce from the text, though no code is shipped. The citation pattern is otherwise unremarkable and covers the relevant MMD and bootstrap literature.\n\nWho is this for? Anthropologists wanting a turnkey fix for small samples. They should not use this version. A correct approach would either resample the MMD statistic directly or estimate the actual sampling variance of theta_mu_i - theta_nu_i, not the dispersion of a bootstrap SD. I would send the paper to a statistically literate referee and expect rejection, with detailed comments. If the authors rework the method along those lines, there is a salvageable core.","headline":"A serious but invalid proposal: the paper substitutes the standard error of the sample standard deviation for the true variance, which makes the bootstrap correction vanish and inflates every distance.","tokens_in":15907,"tokens_out":5669,"would_cite":false,"duration_ms":61333,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F40","62P10"],"pacs":[],"model":"deepseek-v4-flash","headline":"Replace the theoretical variance in the MMD with a bootstrapped spread so sparse skeletal samples still yield trustworthy population distances.","keywords":["non-metric traits","Mean Measure of Divergence","parametric bootstrap","Anscombe transformation","biodistance","small samples","variance stabilization"],"falsifier":"Simulate two populations with identical true trait probabilities and small $N$, compute $stMMD_B$ many times, and check whether values above $2$ occur in about 5 percent of replicates; a systematic departure would show the substitution is miscalibrated.","tokens_in":14796,"feed_emoji":"🦴","tokens_out":8828,"duration_ms":77152,"temperature":0.7,"pith_summary":"The paper proposes a parametric bootstrap correction for the Mean Measure of Divergence (MMD), a distance statistic anthropologists use to compare populations through present/absent skeletal traits. Because the Anscombe-transformed angle for each trait is nearly normal with known variance $1/(N+1/2)$, the authors resample that normal distribution, measure the spread of the resampled standard deviations, and insert that spread into the MMD in place of the theoretical variance. They argue this makes the MMD usable when samples are small or many traits are unmeasured, and they give a practical rule: when the relative error $E$ of the bootstrapped distances stays below about 10 percent, the method only rescales the original distances, but when $E$ exceeds 10 percent it corrects an unreliable distance ranking. The payoff is a concrete recipe for palaeoanthropologists working with very few skeletons.","feed_headline":"Bootstrap fixes population distances when skeletons are scarce","feed_subtitle":"When a sample is tiny or half-measured, the method corrects the Mean Measure of Divergence and can flip cluster trees.","key_machinery":"The machinery is the parametric bootstrap of Eqs. (4)-(7): for each population-trait pair, draw $N_{\\mu i}$ values from $N(0,\\sigma_{\\mu i})$, compute their sample standard deviation $\\sigma^*_{\\mu i}$, repeat $B$ times, and take $S_{\\mu i}$ to be the standard deviation of the collection of $\\sigma^*$ values. That single number carries the argument: it is plugged into both the bias subtraction in $MMD_B$ and the denominator $\\sigma_B$ of the standardized distance. The distributional input that makes the procedure 'parametric' is the normality of the Anscombe-transformed angle, with variance fixed by the trait count alone.","core_discovery":"The central claim is that the bootstrapped quantity $S_{\\mu i}$, defined as the standard deviation, over $B$ resamples, of the sample standard deviations of synthetic $\\hat\\theta$ draws, is the correct variance correction for the MMD. Replacing $\\sigma_{\\mu i}=1/\\sqrt{N_{\\mu i}+1/2}$ by $S_{\\mu i}$ in the MMD formula yields a bootstrapped distance $MMD_B$ and a standardized version $stMMD_B$ whose significance can be read from $\\chi^2$ or normal cutoffs. The paper shows numerically that $S$ falls as $A/N^\\beta$ with $\\beta \\approx 1$, so small populations and traits with low measurement coverage receive the largest corrections. In well-represented data the bootstrapped distance matrix is nearly a rescaling of the original, preserving cluster and ordination results; in poorly represented data the ranking changes, and the paper treats the bootstrapped matrix as the accurate one.","pith_inferences":["Read as an inference: the paper's own fit $S \\approx A/N$ means $S^2$ is smaller than $\\sigma^2$ by an extra factor of $1/N$, so for moderately large samples the bias subtraction in $MMD_B$ is negligible; the dominant effect of the bootstrap is to shrink the denominator $\\sigma_B$, which inflates $stMMD_B$ and explains the very large distances in the tables.","Read as an inference: a more direct correction would use the known sampling distribution of the sample standard deviation from a normal sample, which could be written in closed form and compared with $S$ to test whether the bootstrap is necessary.","Read as an inference: the $E=10\\%$ threshold is a heuristic on the distance matrix; it could be recalibrated by simulation for a given number of traits and sample sizes, or replaced by a formal test of rank preservation.","Read as an inference: the same resampling logic transfers to other variance-stabilized angular transformations for binomial counts, where the analogous known variance would define the parametric bootstrap."],"forward_implications":["Anthropologists with skeletal samples of a few dozen individuals, or with many unmeasurable traits, can compute $MMD_B$ with $B=500$ and report distances that are comparable across studies.","When the relative error $E \\lesssim 10\\%$, users can trust the original distance ranking and the UPGMA/nMDS results built from it, since bootstrapping only rescales the entries.","When $E > 10\\%$, the original ranking is unreliable and the bootstrapped matrix should be used instead.","Populations with small net size or with information fraction $T_\\mu < 0.5$ receive the largest corrections, so apparent closeness to other groups may disappear after bootstrapping."],"supporting_citations":[{"why":"Introduces the variance-stabilizing inverse-sine transformation that makes the trait angle approximately normal.","marker":"Anscombe (1948)"},{"why":"Supplies the MMD formula, its null variance, and the $\\sigma = 1/\\sqrt{N+1/2}$ law that the bootstrap resamples.","marker":"de Souza, & Houghton (1977)"},{"why":"Provides the bootstrap resampling principle that the parametric procedure adapts.","marker":"Efron (1979)"},{"why":"Compares MMD with Mahalanobis distances and supports the choice of variance-stabilizing transforms for small samples.","marker":"Nikita (2015)"}],"fun_headline_variants":["Parametric bootstrap rescues MMD in sparse samples","Bootstrap variance fix sharpens MMD for tiny datasets","Parametric resampling corrects MMD when data is scarce","New parametric bootstrap tightens MMD for small samples"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method stands or falls on the claim that $S_{\\mu i}$, the spread of bootstrapped sample standard deviations, is the right replacement for the theoretical standard deviation in the MMD formula.","fun_headline_variants_meta":{"raw":{"variants":["Parametric bootstrap rescues MMD in sparse samples","Bootstrap variance fix sharpens MMD for tiny datasets","Parametric resampling corrects MMD when data is scarce","New parametric bootstrap tightens MMD for small samples"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000489,"raw_usage":{"total_tokens":2422,"prompt_tokens":977,"completion_tokens":1445,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":1379}},"tokens_in":593,"tokens_out":1445,"duration_ms":9761,"temperature":1.0,"reasoning_tokens":1379,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:28:44.408457+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate two populations with identical true trait probabilities and small $N$, compute $stMMD_B$ many times, and check whether values above $2$ occur in about 5 percent of replicates; a systematic departure would show the substitution is miscalibrated.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the variance-stabilizing inverse-sine transformation that makes the trait angle approximately normal."}],"review_version":1}