{"id":"6704960f-a6f8-4318-a0c8-2d5cf05dbd13","arxiv_id":"2607.21880","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Multivariate random forest permutation importance is given a conditional sure-screening guarantee and applied to facial-shape GWAS after LASSO pre-screening.","lead":"This paper proves a conditional consistency guarantee for multivariate random-forest permutation importance: if certain moment bounds hold, all truly influential features are retained with probability approaching one. It then applies the method to facial-morphology GWAS data and reports new SNP associations, though the practical thresholding and algorithm details are not covered by the theory.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm undefined: SSE(A) uses V(A)^{-1}, but legal nodes can have size between 5 and q, making V(A) singular; theorem has no well-defined object.","rationale":"The reader's weakest assumption identifies exactly the singular covariance issue: the paper provides no regularization or pseudo-inverse for V(A), and the splitting rule is ill-defined when a node has fewer observations than outcomes. This is the most load-bearing concern because it invalidates the object of the theorem itself. Even if C1–C2 and Condition 1 were accepted as mild and the threshold in (5) were used in practice, the theorem would still not apply to an undefined procedure. The concern is concrete, internal to the paper, and not merely a disagreement with existing consensus. A simple instrumentation test can settle whether real splits with 5 ≤ child size < q occur; if they do not, the concern would be void, but the stopping rule as written makes their occurrence inevitable in the simulations. Since the reader already recommends REJECT, and this concern supports that judgment rather than introducing a new direction, the appropriate verdict adjustment is UNCHANGED.","tokens_in":22214,"tokens_out":8589,"duration_ms":98071,"concrete_test":"Implement the exact splitting rule from Section 2 (SSE with V(A)^{-1}, stop when every candidate split yields a child with fewer than 5 observations) and run Simulation Study 1 (n = 200, q = 10, p = 500). Instrument every candidate split to record child node sizes and check whether V(A) is singular for any child of size between 5 and q−1. If such splits occur, the algorithm cannot compute SSE as written. Alternatively, replace V(A)^{-1} with the Moore–Penrose pseudo-inverse and rerun; if results change materially, the published algorithm was not what was evaluated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central consistency claim concerns the MRF as defined in Section 2, but that algorithm is not well-defined in the settings used for simulations and the GWAS. The splitting rule SSE(A) = Σ (Y_i − Ȳ(A))ᵀ V(A)^{-1} (Y_i − Ȳ(A)) requires the inverse of the within-node outcome covariance matrix V(A). The stopping rule only forbids child nodes with fewer than five observations, so nodes with between 5 and q observations are legal. For q = 10 (Simulation Study 1 and GWAS) or q = 5 (Study 2), a node with n_A ≤ q has covariance matrix of rank at most n_A − 1 < q, hence V(A) is singular and V(A)^{-1} does not exist. For example, a parent with 12 observations split into children of size 5 and 7 must evaluate SSE for children whose covariance matrices are singular. No regularization, pseudo-inverse, shrinkage, or minimum node size > q is stated anywhere. Thus the algorithm as written cannot be executed; it is not a minor implementation detail because the split rule determines the trees and hence every quantity in C1–C2. If the authors' implementation silently replaces the inverse by a generalized inverse or a regularized estimator, then the theorem's assumptions and proof apply to a different algorithm, not to the one described. This gap undermines the abstract's 'first consistency guaranty for MRF' more directly than the unverified moment conditions themselves.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes multivariate random forest (MRF) with permutation-based variable importance (PVIM) as a joint feature-screening tool for multivariate outcomes. It states a theorem (Section 2.2) claiming max_j Var(hat-Lambda_n^{(j)}) = O(R_n) with R_n = max{k_n^{2+alpha}/n, n/m_n}, and a sure-screening conclusion P(D subset of hat-D_n) -> 1 under Condition 1, Conditions C1-C3, and Card(D) R_n / c0^2 -> 0. The proof uses an incomplete U-statistic decomposition to aggregate subsampling of subjects, mtry feature subsampling, and permutation randomness. The paper also reports simulations comparing MRF with CCA, LASSO, and five SIS methods, and applies MRF (after LASSO pre-screening) to a facial-morphology GWAS with 2,342 subjects and 453,273 SNPs. The central claim is the 'first consistency guaranty for MRF'.","tokens_in":22619,"tokens_out":8351,"duration_ms":79839,"significance":"If the theorem were established for a well-defined algorithm, the incomplete-U-statistic variance decomposition and the screening result would fill a recognized gap in the multivariate random forest literature. The paper is also honest about practical limitations: it explicitly states that direct MRF 'breaks down' at ultrahigh dimensionality and that a LASSO pre-screening step is needed. The heterogeneous simulation designs are a constructive contribution. However, as it stands, the theorem is conditional on unverified high-level moment conditions, the algorithm as defined cannot be executed in the paper's own simulation settings because the splitting criterion uses an inverse of a singular covariance matrix, and the practical selection rule differs from the theoretical threshold rule. These issues are load-bearing for the paper's central claim, so the significance cannot be credited in its present form.","major_comments":[{"comment":"The splitting rule uses SSE(A) = sum_i (Y_i - bar-Y(A))^T V(A)^{-1} (Y_i - bar-Y(A)), where V(A) is 'the covariance matrix of the outcome vector estimated using the observations in node A.' The stopping rule only forbids child nodes with fewer than five observations. In Simulation Study 1 and the GWAS, q = 10; in Simulation Study 2, q = 5. Therefore nodes of size between 5 and q are legal, and for such nodes V(A) has rank at most n_A - 1 < q, so V(A) is singular and V(A)^{-1} does not exist. For example, a parent node of size 12 split into children of size 5 and 7 must evaluate SSE(A_L) and SSE(A_R) on singular covariance matrices. No regularization, pseudo-inverse, shrinkage, or minimum-node-size greater than q is stated anywhere. Thus the algorithm as described cannot be run in the paper's own simulations, and the theorem has no well-defined object. This is not a minor implementation d","section":"Section 2, definition of SSE(A)"},{"comment":"Conditions C1 and C2 are high-level moment bounds on the tree prediction errors g_n and g_{nj}. The paper never verifies these conditions for the proposed MRF splitting rule under any concrete data-generating process. Consequently the theorem states: if the variance of the PVIM estimator is O(R_n) and true PVIMs are separated from zero by c0, then thresholding at c0/2 works. The substantive part, namely proving the variance bound for this specific algorithm, is assumed rather than established. The abstract's phrase 'under mild regularity conditions' overstates the status of C1-C2; as written, these are strong, unverified assumptions on an estimator whose construction is itself not fully specified.","section":"Theorem, Section 2.2, Conditions C1-C2"},{"comment":"The theoretical selection rule is hat-D_n = {j : hat-Lambda_n^{(j)} > c0/2}, where c0 is the constant in Condition 1. The empirical rule used in all simulations and in the real data is the max-ratio rule: retain features that fall in the top 5% by PVIM and simultaneously among the top five max-ratio thresholds. These are different rules, and c0 is unknown in practice. The theorem provides no guarantee for the max-ratio rule. Thus the reported individual/overall success rates and the GWAS hits are produced by a procedure for which the consistency theorem is not proved.","section":"Eq. (5) vs. Sections 3.1 and 4"},{"comment":"The real-data pipeline and Simulation Study 3 use a LASSO pre-screening step before MRF; Section 3.3.3 states that direct MRF 'breaks down' at p = 100,000, and Section 4 reduces 453,273 SNPs to about 66,979 (male) and 97,786 (female) by LASSO before MRF is applied. The theorem is stated and proved only for MRF without pre-screening. No theoretical result covers the LASSO+MRF pipeline, so the GWAS findings are not backed by the paper's consistency guarantee. This gap is acknowledged in the practical discussion but not addressed in the theory.","section":"Sections 3.3.3 and 4, LASSO pre-screening"}],"minor_comments":[{"comment":"The statement that 'b1 contains at most k_n^2 m_n^2 non-zero elements' appears to undercount; a more conservative count gives m_n^2 n k_n non-zero terms. Under C3 this still yields the stated O(R_n) rate, but the current line should be corrected.","section":"Appendix A, proof of part (2)"},{"comment":"The 'top five max-ratio thresholds' rule is defined only by reference to Zhao and Fu [48]. A self-contained definition of the ratio and of how the top-five rule interacts with the 5% cutoff would make the empirical procedure reproducible without consulting another paper.","section":"Section 3.1, max-ratio rule"},{"comment":"'Cauchy-Schwartz' should be 'Cauchy-Schwarz' in Appendix A.","section":"Throughout"},{"comment":"The tables reporting individual and overall success rates are visually cramped and hard to read; separating the success rates for each true feature into a clearer layout would improve presentation.","section":"Tables 3, 5, and 7"}],"recommendation":"reject","confidential_remarks":"The paper has an interesting applied component and a constructive simulation framework, but the theoretical contribution as stated is not self-consistent: the splitting rule is undefined in the paper's own settings, the key moment conditions C1-C2 are never verified, and the empirical selection rule differs from the theoretical one. These are load-bearing issues for the abstract's claim of the 'first consistency guaranty for MRF.' The LASSO pre-screening pipeline used in the real data is also outside the theorem. I recommend rejection rather than major revision because closing these gaps would require substantial new theoretical work, not local fixes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper contains a genuine conditional theorem — a variance bound and sure-screening result for multivariate random forest permutation importance via incomplete U-statistics — but the algorithm as written is undefined in exactly the settings the authors run. SSE(A) uses V(A)^{-1}, the inverse of the within-node outcome covariance, while the stopping rule allows terminal nodes as small as five observations and q is 5, 10, or 50 in the simulations and GWAS. Nodes with size between 5 and q are legal, V(A) is singular, and V^{-1} doesn't exist. No regularization, pseudo-inverse, or minimum-node-size > q is stated anywhere. The stress-test note holds up: the theorem applies to a different algorithm than the one that produced the reported results.\n\nWhat is actually new: applying the incomplete-U-statistic decomposition — subject subsampling, feature subsampling, permutation randomness — to MRF-PVIM is new relative to the cited literature. The proof is a standard Chebyshev/U-statistic argument (the key variance formula (12) comes from Mentch-Hooker), but it's carefully assembled, and the theorem is honestly stated as conditional on C1–C3 and Condition 1. Credit also for admitting in Section 3.3.3 that MRF breaks down in ultrahigh dimension, which is why the two-step LASSO+MRF strategy is introduced, and for acknowledging in the Discussion that the max-ratio threshold rule lacks theoretical grounding.\n\nSoft spots, in proportion. The singular covariance issue is load-bearing, not cosmetic: C1–C2 concern the prediction errors of these very trees, so the object the theorem is about isn't well defined. C1–C2 are never verified for this splitting rule, which makes the abstract's 'mild regularity conditions' an overstatement. The theorem guarantees screening for the rule D̂n = {PVIM > c0/2} with known c0; the procedure evaluated throughout is a max-ratio rule (top 5% and top-five ratios), so the guarantee doesn't attach to the evaluated method. The GWAS claims — novel SNPs and interaction hubs — are exploratory, with no replication or multiplicity-controlled null, though the paper presents them strongly. The reader's 'circularity' concern is overstated: the theorem doesn't smuggle its conclusion; it leans on Condition 1, which is standard in screening theory. The custom simulation framework is elaborate, but the near-total failure of five SIS methods (Ps ≈ 0) suggests the design heavily favors interaction-based signal; that weakens the 'rigorous testbed' claim.\n\nThere is a repair path: regularize V(A) (ridge or pseudo-inverse), impose min node size > q or state a regularized splitting rule, verify or weaken C1–C2 for that rule, and either prove the max-ratio rule or stop claiming the theorem covers it. That's a real revision, not a patch.\n\nThis paper is for people working on multivariate screening and random forest theory. The conditional theorem deserves expert scrutiny, and the flaws are identifiable and fixable, so I would send it to peer review — but with the expectation of major revision. As written, reject.","headline":"Real conditional screening theorem for MRF-PVIM, but the algorithm as written is undefined (singular node covariances) and the proved threshold rule isn't the one run; worth refereeing, needs major revision.","tokens_in":23075,"tokens_out":7640,"would_cite":false,"duration_ms":78112,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G20","62H30","62H12"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper establishes the first consistency guarantee for multivariate random forests, proving that permutation-based variable importance retains every truly influential feature with probability tending to one as sample size grows.","keywords":["multivariate random forests","permutation variable importance","sure screening","feature selection","incomplete U-statistics","GWAS","facial morphology","high-dimensional inference"],"falsifier":"Run the algorithm exactly as defined on the paper's Simulation 1 (n=200, q=10 outcomes, stopping at five observations per child): any node with fewer than ten observations yields a singular V(A), so the Mahalanobis split criterion cannot be computed without an unspecified pseudo-inverse or ridge; demonstrating this on a single replicate is enough to show the procedure is not self-contained. Alternatively, estimate the left-hand sides of conditions C1 and C2 from simulated trees and check whether the variance actually scales as O(n) and O(k_n^α); if not, the theorem's rate (6) does not follow.","tokens_in":22130,"feed_emoji":"🌳","tokens_out":4623,"duration_ms":45252,"temperature":0.7,"pith_summary":"This paper tries to put multivariate random forests (MRF) on firm theoretical ground for feature selection. Its central claim is that the permutation-based variable importance measure (PVIM) has the sure screening property: when the sample size grows, with probability approaching one, the procedure retains every feature that truly influences a multivariate outcome. The proof models PVIM as an incomplete U-statistic that averages over three layers of randomness—subsampled subjects, subsampled features at each split, and random permutations of out-of-bag observations. If the claim holds, MRF becomes a joint, interaction-aware alternative to independence screening, and the paper demonstrates its utility on a genome-wide scan of human facial morphology.","feed_headline":"First proof: multivariate random forests keep all true features","feed_subtitle":"A new consistency result backs joint screening that also uncovers interacting facial-morphology genes.","key_machinery":"The central object is the permutation-based variable importance measure (PVIM) of multivariate random forests, defined as the change in out-of-bag prediction error after permuting a single feature, averaged over the ensemble. The proof reshapes PVIM as an incomplete U-statistic averaging over three independent layers of randomness—subsampling of subjects, subsampling of candidate features at each split, and permutation of features on out-of-bag samples—and bounds its variance by decomposing the estimator into a centered per-tree term, a bias term, and a U-statistic term (A_nj, B_nj, U_nj). The splitting rule itself uses the within-node sum of squared Mahalanobis distances, with the inverse o","core_discovery":"The paper establishes the first consistency guarantee for permutation-based variable importance in multivariate random forests. Stated on the paper's own terms: under moment conditions C1–C3 on tree prediction errors, the estimator's maximum variance over all p features is O(R_n), and if Card(D)R_n/c0^2 → 0, then P(D ⊆ D̂n) → 1, where D is the set of truly influential features and D̂n the selected set. The selection rule keeps features whose estimated PVIM exceeds half of a population threshold c0 separating signal from noise. The proof decomposes the PVIM estimator into a centered randomness term, a bias term, and an incomplete U-statistic term, bounding each via combinatorial counting of o","pith_inferences":["The unverified moment conditions C1–C2 are the bridge between the algorithm and the guarantee; one could test them empirically, and if they fail for the Mahalanobis splitting rule, the sure-screening conclusion would not apply to the implemented method.","The two-step LASSO-then-MRF strategy is itself a hybrid that trades joint screening for computational feasibility; a natural extension would be to derive a screening guarantee for the two-step selector rather than only the MRF stage.","The 'hub' interactions (GPD1L in males, GNG4 in females) are the kind of hypothesis a joint method can generate that single-SNP tests cannot; one could validate them with gene-expression or knockdown experiments.","If PVIM variance can be estimated, the max-ratio threshold could be replaced by an inference-based cutoff, connecting this screening result to formal statistical tests for feature importance."],"forward_implications":["With the sure screening property, MRF-based feature selection can serve as a screening step that, as n grows, does not drop any truly influential feature.","The joint, permutation-based importance accounts for multicollinearity and interactions in a way independence screening cannot; the facial-morphology application illustrates this by identifying hub genes such as GPD1L in males and GNG4 in females.","The max-ratio thresholding rule, combined with the theorem, yields parsimonious selections: in simulations MRF recovers all true features with a mean minimum selection size of about 7 out of 500 features.","The two-step LASSO-then-MRF strategy is validated in ultrahigh-dimensional settings, achieving a 90% overall success rate at p=100,000 while keeping false selection rates low.","The theoretical result applies even when the number of features, the set of true features, and the threshold c0 grow with n, covering the high-dimensional regime beyond classical fixed-parameter screening."],"fun_headline_variants":["Multivariate random forests get first consistency proof","Proof: multivariate RF misses no true features","Joint screening keeps all real signals in high dims","Facial morphology genes found via proven MRF","First guarantee: MRF retains all influential features"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The result holds only if conditions C1 and C2—unverified moment bounds on the prediction errors of the individual trees—are satisfied, and the splitting rule as written inverts a within-node covariance matrix that is singular whenever a node has fewer observations than outcome variables, a situation the paper's own stopping rule (minimum five observations) can create when q is ten or fifty.","fun_headline_variants_meta":{"raw":{"variants":["Multivariate random forests get first consistency proof","Proof: multivariate RF misses no true features","Joint screening keeps all real signals in high dims","Facial morphology genes found via proven MRF","First guarantee: MRF retains all influential features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000515,"raw_usage":{"total_tokens":2370,"prompt_tokens":810,"completion_tokens":1560,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":1490}},"tokens_in":554,"tokens_out":1560,"duration_ms":11470,"temperature":1.0,"reasoning_tokens":1490,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T06:25:11.108912+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the algorithm exactly as defined on the paper's Simulation 1 (n=200, q=10 outcomes, stopping at five observations per child): any node with fewer than ten observations yields a singular V(A), so the Mahalanobis split criterion cannot be computed without an unspecified pseudo-inverse or ridge; demonstrating this on a single replicate is enough to show the procedure is not self-contained. Alternatively, estimate the left-hand sides of conditions C1 and C2 from simulated trees and check whether the variance actually scales as O(n) and O(k_n^α); if not, the theorem's rate (6) does not follow.","supporting_citations":[],"review_version":1}