{"id":"95a0b6d3-d801-4614-93f2-6dd875b388a0","arxiv_id":"1909.00659","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GRAF builds oblique random forest ensembles whose hyperplanes are shared across all impure partitions, and reports competitive accuracy on 115 UCI datasets plus a sensitivity score for data approximation.","lead":"This paper introduces Guided Random Forest (GRAF), an ensemble classifier that builds oblique decision trees where each random hyperplane is reused across all impure partitions, not just the region that generated it. The authors report competitive or better accuracy than random forest, gradient boosting, and AdaBoost on 115 UCI datasets, and propose a sensitivity score for approximating datasets with fewer points.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (10) violates the subadditivity property stated in Section 2, so the guided-partitioning purity rationale is unsupported.","rationale":"This is the most load-bearing concern because it is an internal inconsistency, not a disagreement with consensus: the paper explicitly states the subadditivity condition and then uses a function that violates it. Since the greedy selection of the 'most impure' partition and the claim that global partitioning reduces the generalization error bound are the paper's theoretical contributions, their motivation breaks if the property fails. The empirical claim is important but is not fully verified either: no code, no per-dataset significance tests, and the accuracy table is omitted from the submitted text. However, those are addressable reproducibility issues rather than internal contradictions. The reader's conditional verdict is therefore appropriate: the numerical claim should not be rejected outright, but the theoretical framing needs repair or the algorithm's justification must be reformulated.","tokens_in":12603,"tokens_out":9335,"duration_ms":81541,"concrete_test":"Write a script that enumerates all two-class splits of a synthetic dataset with N_1 = 100, N_2 = 10000 and checks Z(parent) >= Z(child0) + Z(child1) using Eq. (10). The split (100,1) -> (50,0) + (50,1) is a counterexample; if the script reproduces it, the subadditivity premise is false. Then either replace Eq. (10) with a provably subadditive impurity such as n_p times the Gini index, or remove the monotonic-purity/global-bound claim from the paper.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2 defines Z with the required property that if Omega_p = Omega_p0 union Omega_p1 then Z(Omega_p) >= Z(Omega_p0) + Z(Omega_p1), and the algorithm greedily selects the partition maximizing Z and extends the split to other partitions. Equation (10) does not satisfy this property. For two classes with N_1 = 100, N_2 = 10000, take Omega_p with counts (100, 1): Z(Omega_p) = 101 * (1 - 1.00000001 / 1.00020001) = 0.020198. Splitting into counts (50, 0) and (50, 1) gives Z values 0 and 51 * (1 - 0.25000001 / 0.25010001) = 0.020396, respectively, so the children sum to 0.020396 > 0.020198. Thus total impurity can increase after a split, contradicting the stated 'overall purity increases' rationale and the claimed generalization-bound reduction that rests on it. The empirical 115-dataset comparison may still be valid, but the theoretical support for global partitioning is missing; the strength/correlation plots in Section 5.2 do not evaluate the bound in Eq. (30) for GRAF or compare it with random forest or gradient boosting.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Guided Random Forest (GRAF), an ensemble classifier in which oblique hyperplane splits are generated greedily for the most impure partition and then extended to other impure partitions, so that the resulting tree uses variable-length partition codes. The authors claim that this global partitioning reduces the generalization error bound, that GRAF outperforms random forest, gradient boosting, and AdaBoost, and that a sample sensitivity score derived from the number of hyperplanes needed to purify a partition can be used for data approximation. The manuscript reports bias/variance experiments on synthetic datasets, strength/correlation curves, a comparison on 115 UCI datasets, and data-approximation experiments on synthetic and UCI data.","tokens_in":12895,"tokens_out":6730,"duration_ms":57539,"significance":"The paper proposes a genuinely different ensemble construction: one oblique hyperplane is applied across multiple impure partitions rather than only to the local partition for which it was generated. The 115-dataset comparison is a substantial empirical effort, and the sensitivity-based data approximation idea is potentially useful. However, the central theoretical motivation is internally inconsistent, the claimed generalization-error-bound reduction is not actually evaluated, and the sensitivity scores depend on an arbitrary ordering. No code, seeds, or error bars are provided, which makes the empirical claims difficult to verify. If the theoretical issues are repaired and the empirical claims are substantiated, GRAF could be a competitive ensemble method with an interesting data-subsetting capability.","major_comments":[{"comment":"The impurity function Z is introduced with the requirement that if Omega_p = Omega_p0 union Omega_p1, then Z(Omega_p) >= Z(Omega_p0) + Z(Omega_p1), and the greedy partition selection in Eqs. (11)-(15) is motivated by an increase in overall purity. Equation (10) does not satisfy this property. For N1=100 and N2=10000, a partition with counts (100,1) has Z = 101*(1 - 1.00000001/1.00020001) = 0.020198, while splitting it into (50,0) and (50,1) gives Z values 0 and 51*(1 - 0.25000001/0.25010001) = 0.020396, respectively, so the children sum to 0.020396 > 0.020198. Thus splitting can increase total impurity, and the subadditivity premise on which the 'global partitioning reduces the generalization error bound' claim rests is false. A corrected impurity definition or a revised theoretical argument is needed.","section":"Section 2, Eq. (10)"},{"comment":"The abstract states that global partitioning reduces the generalization error bound, but the paper never evaluates the bound PE* from Eq. (30) and never compares it with random forest or gradient boosting. Fig. 6 only plots strength and correlation for GRAF on synthetic datasets. A direct computation of the bound, at least on the same datasets, is required to support the central claim that the bound is reduced.","section":"Section 5.2, Eq. (30) and Fig. 6"},{"comment":"Sensitivity scores are computed as theta_{omega x_i} = W(omega)/i after ranking points in a partition arbitrarily. Therefore the scores depend on an arbitrary permutation and are not a well-defined property of the data. The support-vector analogy and the data-approximation experiments in Figs. 7-9 are consequently not reproducible as specified. Please replace the arbitrary rank with an order-invariant definition or provide a justification for the chosen ordering.","section":"Section 6, Eq. (37)"},{"comment":"The claim that GRAF outperforms state-of-the-art bagging and boosting algorithms rests on an accuracy comparison over 115 UCI datasets, but no standard deviations, confidence intervals, statistical significance tests, seeds, or code are reported, and 'comparable' is not defined. The appendix table (Table A1) is also not included in the manuscript, so the result cannot be checked. Reporting these details is necessary before the empirical superiority conclusion can be accepted.","section":"Section 5.3"}],"minor_comments":[{"comment":"The definition splits every impure partition phi by W_omega, but Section 4 states that bit assignment is skipped for partitions not dichotomized at a given height; please reconcile these two descriptions.","section":"Section 2, Eqs. (13)-(14) and Section 4"},{"comment":"The line 'Choose a random hypothesis using P(i)' is not sufficient to reproduce the hyperplane generation; Eq. (4) draws weights uniformly from feature ranges, not conditioned on P(i) as described.","section":"Section 3, Algorithm 1"},{"comment":"The synthetic datasets are described only as generated using Weka with the RandomRBF class; the specific parameters and the number of replicates should be listed for reproducibility.","section":"Section 5.1"},{"comment":"The claim 'outperforms state of the art ... algorithms' is stronger than the result reported in Section 5.3, which says GRAF is better or comparable on a majority of datasets; please align the wording.","section":"Abstract and Section 7"},{"comment":"The method for sampling 'according to their sensitivities (P)' is not explained; it is unclear whether the scores are normalized and used as a probability distribution or converted to ranks.","section":"Section 6, Fig. 8"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the paper has a real empirical algorithm and a potentially useful sensitivity-based approximation idea, but the central theoretical claim is not supported as written; the subadditivity counterexample is an internal inconsistency, not a matter of interpretation. I recommend major revision rather than rejection because the empirical contribution and the sensitivity scheme remain potentially valuable, and the issues could be addressed by replacing the impurity function, adding a direct bound comparison, and supplying reproducible experimental details."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely new: GRAF fits a hyperplane on the most impure partition and then extends it to other partitions, encoding leaves with variable-length codes. That is not in the cited oblique/rotation forest literature, and the sensitivity-based data approximation is a neat application. The empirical scope is also real—115 UCI datasets following the Fernandez-Delgado protocol is a lot of work, and the results are suggestive that GRAF is competitive with random forest, gradient boosting, and AdaBoost.\n\nThe problem is the theory. Section 2 defines the impurity function Z with the property that splitting a partition should not increase total impurity: Z(Omega_p) >= Z(Omega_p0)+Z(Omega_p1). Equation (10) does not satisfy that. With class sizes N1=100, N2=10000, a partition containing (100,1) samples has Z about 0.0202, while splitting into (50,0) and (50,1) gives children with Z sum about 0.0204. So a split can increase impurity, contradicting the greedy rationale and the claimed generalization-bound reduction that rests on it. This is not a nitpick; it is a load-bearing internal contradiction.\n\nSeparately, the abstract claims that global partitioning reduces the generalization error bound, but the paper never actually evaluates the bound in Eq. (30) for GRAF against random forest or gradient boosting. Figure 6 just plots strength and correlation curves for GRAF alone. The empirical comparison also lacks code, seeds, error bars, and significance tests, so the \"comparable or better on a majority\" claim is based on point estimates. And the natural baselines—oblique forests like rotation forest—are absent.\n\nThe sensitivity analysis is interesting but qualitative; the support-vector analogy in Table 1 is suggestive, not rigorous. Still, the sensitivity score could be a useful data-selection heuristic for people working on data summarization.\n\nWho should read this: researchers in ensemble methods, especially oblique decision trees, will want to know the idea. But the theoretical section needs to be fixed or substantially weakened before the narrative can be trusted. I would not cite the theoretical claims as they stand.\n\nRecommendation: this deserves peer review—the empirical breadth and the new mechanism justify referee time—but reviewers should be asked to verify the impurity property, request code and statistical comparisons, and push for a direct bound comparison. As it is, the paper needs major revision, not acceptance.","headline":"A novel twist on oblique forests with a substantial empirical comparison, but the impurity function breaks the paper's own theoretical rationale and the key bound claim is not demonstrated.","tokens_in":13359,"tokens_out":4735,"would_cite":false,"duration_ms":111248,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Guided Random Forest (GRAF) is an ensemble classifier that reuses random hyperplanes across the whole feature space; the paper claims it matches or beats random forest, gradient boosting, and AdaBoost on most of 115 benchmarks and doubles…","keywords":["guided random forest","ensemble learning","oblique decision trees","global partitioning","boosting","data approximation","sensitivity score","generalization error"],"falsifier":"Take the paper's $Z$ from equation (10) on a two-class problem with global counts $N_1=100$, $N_2=10000$, and a partition containing 100 class-1 samples and 1 class-2 sample. The parent $Z$ is approximately 0.0202, while splitting off 50 class-1 samples gives children whose $Z$ values sum to approximately 0.0204, violating the required inequality; any such node falsifies the claim that guided splitting increases overall purity and removes the route to the stated generalization-error bound.","tokens_in":12388,"feed_emoji":"🌲","tokens_out":10553,"duration_ms":106952,"temperature":0.7,"pith_summary":"This paper proposes Guided Random Forest (GRAF), an ensemble classifier that builds each tree by repeatedly drawing random hyperplanes and reusing every hyperplane beyond the partition it was drawn for, so that split decisions are global rather than local. The authors aim to establish that this global partitioning is what tree ensembles lack relative to boosting, and that it lowers an upper bound on generalization error. They report experiments on 115 benchmark datasets in which GRAF is better than or comparable to random forest, gradient boosting, and AdaBoost on most datasets. GRAF also assigns each training point a sensitivity score, and the authors argue that keeping only the highest-sensitivity points approximates the full dataset well enough for downstream tasks. If the claims hold, the paper supplies a single method that is both a competitive ensemble classifier and a data-approximation tool.","feed_headline":"Guided random forests match or beat boosting on most datasets","feed_subtitle":"If right, one algorithm both improves ensemble accuracy and summarizes a dataset by its most confusing points.","key_machinery":"The mechanism is a guided partitioning loop. The impurity function $Z(\\Omega_p)$ in equation (10) scores every current partition using global class counts; GRAF chooses the partition $\\omega = \\arg\\max_{\\Omega_i} Z(\\Omega_i)$, draws a random hyperplane through the centroid of that partition, and then extends the same hyperplane to all other impure partitions. Samples accumulate a variable-length binary code as hyperplanes are added, and a leaf's code maps to the class posterior, with inverse-frequency correction for class imbalance. The sensitivity of a point is the number of hyperplanes needed to put its partition into a pure state, divided by the point's rank inside the partition, and then normalized class-wise; this is what turns the forest into a data-approximation tool.","core_discovery":"The central claim is that reusing a split plane across all impure partitions—rather than letting each tree region learn its own local splits—changes an ensemble of trees from a bagging-style average into something closer to boosting, and that this reduces an upper bound on generalization error. At each step GRAF selects the most impure partition $\\omega$ under the impurity measure $Z$, draws a random hyperplane through it, and applies the same hyperplane to every other impure partition; leaves are described by the variable-length bit sequence each sample accumulates, and the tree returns class posteriors per leaf. On the paper's 115-dataset benchmark, GRAF is better than or comparable to AdaBoost on 94 datasets, random forest on 69, and gradient boosting on 67, including the high-class-count datasets. The paper also claims that the number of hyperplanes needed to purify a region defines a sensitivity score, and that subsampling the highest-sensitivity points can approximate the full data.","pith_inferences":["The bound-reduction argument rests on the subadditivity of $Z$; if that property is not restored, GRAF's empirical success would still stand, but as a heuristic rather than as a consequence of the stated bound.","Because high-sensitivity points overlap with support-vector locations in the paper's experiments, sensitivity scores could plausibly serve as an active-learning acquisition rule, selecting points to label next, although the paper does not test this use.","The paper's own closing remark concedes that the sensitivity scheme does not provide a selection criterion for how many points to keep, which leaves the data-approximation claim underspecified at the point where the user must choose a subset size.","A direct extension would be to define sensitivity from each sample's marginal impurity reduction rather than from leaf size, which would make the approximation claim less sensitive to how the tree is grown."],"forward_implications":["If global partitioning lowers the generalization error bound, then random-forest-style ensembles can be improved by reusing split geometry across regions instead of searching for locally optimal splits in each leaf.","Because GRAF trees output class-posterior estimates rather than raw votes, the ensemble can be combined additively in log space, the same functional form as boosting.","The sensitivity score gives each training point an importance value, so users can subsample by importance rather than uniformly and retain most of the accuracy, as the paper demonstrates on its simulated and benchmark datasets.","GRAF's treatment of each tree as a high-variance instance with weight updates that focus on the most impure partition positions the method between bagging and boosting, giving a single algorithm for both regimes."],"supporting_citations":[{"why":"Supplies the random-forest baseline and the strength-correlation generalization error bound that GRAF is compared against.","marker":"[3]"},{"why":"Supplies the 115-dataset benchmark evaluation protocol used to compare GRAF with other classifiers.","marker":"[2]"},{"why":"Supplies the gradient boosting baseline that GRAF claims to outperform or match.","marker":"[10]"},{"why":"Supplies AdaBoost, both as a baseline and as the weight-update analogy used to relate GRAF to boosting.","marker":"[12]"},{"why":"Supplies the bias-variance decomposition used to study GRAF's behavior.","marker":"[14]"},{"why":"Supplies one of the randomized-tree sources for the strength-correlation bound in equation (30).","marker":"[19]"},{"why":"Provides the benchmark data used in the 115-dataset experiments and the sensitivity approximation study.","marker":"[20]"}],"fun_headline_variants":["Global split planes: random forests that mimic boosting","Match or beat boosting on 94 datasets with global splits","One hyperplane per split, reused: random forests get a boost","Global partitioning turns random forests into boosting-like ensembles"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that splitting the most impure partition always increases overall purity, which requires the impurity measure $Z$ to satisfy $Z(\\Omega_p) \\ge Z(\\Omega_{p0}) + Z(\\Omega_{p1})$; the paper's equation (10) does not guarantee this, so the greedy motivation and the bound argument depend on a property the chosen measure can violate.","fun_headline_variants_meta":{"raw":{"variants":["Global split planes: random forests that mimic boosting","Match or beat boosting on 94 datasets with global splits","One hyperplane per split, reused: random forests get a boost","Global partitioning turns random forests into boosting-like ensembles"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000848,"raw_usage":{"total_tokens":3625,"prompt_tokens":814,"completion_tokens":2811,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":2746}},"tokens_in":430,"tokens_out":2811,"duration_ms":22744,"temperature":1.0,"reasoning_tokens":2746,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:40:54.850914+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the paper's $Z$ from equation (10) on a two-class problem with global counts $N_1=100$, $N_2=10000$, and a partition containing 100 class-1 samples and 1 class-2 sample. The parent $Z$ is approximately 0.0202, while splitting off 50 class-1 samples gives children whose $Z$ values sum to approximately 0.0204, violating the required inequality; any such node falsifies the claim that guided splitting increases overall purity and removes the route to the stated generalization-error bound.","supporting_citations":[{"cited_title":"Do we need hundreds of classifiers to solve real world classifica- tion problems?","cited_arxiv_id":null,"evidence_quote":"Supplies the 115-dataset benchmark evaluation protocol used to compare GRAF with other classifiers."},{"cited_title":"Greedy function approximation: a gradient boost- ing machine,","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient boosting baseline that GRAF claims to outperform or match."},{"cited_title":"A decision-theoretic generalization of on-line learning and an application to boosting,","cited_arxiv_id":null,"evidence_quote":"Supplies AdaBoost, both as a baseline and as the weight-update analogy used to relate GRAF to boosting."},{"cited_title":"Bias plus variance decomposition for zero-one loss functions,","cited_arxiv_id":null,"evidence_quote":"Supplies the bias-variance decomposition used to study GRAF's behavior."},{"cited_title":"Shape quantization and recognition with randomized trees,","cited_arxiv_id":null,"evidence_quote":"Supplies one of the randomized-tree sources for the strength-correlation bound in equation (30)."},{"cited_title":"UCI machine learning repository,","cited_arxiv_id":null,"evidence_quote":"Provides the benchmark data used in the 115-dataset experiments and the sensitivity approximation study."}],"review_version":1}