{"id":"561f995b-f5ee-4c88-95cc-9c2cc0686a59","arxiv_id":"2509.02792","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A radial-basis combiner over base-model outputs, trained with a tunable winner-takes-all diversity weight, is claimed to match loss geometry and beat logit averaging and MoE.","lead":"This paper proposes a structured basis function network, a linear combiner over radial-basis features of multiple base models' outputs, and claims it unifies multi-hypothesis prediction with ensemble learning. The reported experiments, however, contain identical accuracy numbers for different datasets, which makes the empirical claims unreliable without a corrected version.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tables 7–10 list identical MNIST and CIFAR-10 results; the empirical core of the paper is internally inconsistent.","rationale":"I read the method description in good faith: the closed-form ridge solution, the gradient-based updates, and the parametric diversity mechanism are all coherent as a practical combiner. The appendix comparison of basis strategies is a useful empirical note. However, the central empirical claim of the paper is carried by Tables 7–10, and those tables are internally inconsistent: MNIST and CIFAR-10 rows are numerically identical across accuracy, best-configuration, efficiency, and sensitivity tables. This is not a subtle modeling assumption; it is a direct contradiction within the reported evidence. The reader's weakest_assumption focuses on the missing derivation that Phi(Di)*alpha realizes the loss-induced Bregman centroid. That gap is real and would matter even if the experiments were clean. But as printed, the empirical inconsistency is more immediately load-bearing: it makes the headline 's-BFN outperforms logit averaging and MoE' unsupported regardless of whether the theoretical identification is later supplied. I therefore partially agree with the reader: the representational gap is a legitimate concern, but the decisive issue is the self-contradictory experimental tables. The reader's REJECT verdict remains appropriate, so no verdict adjustment is needed.","tokens_in":20035,"tokens_out":3047,"duration_ms":34696,"concrete_test":"Obtain the authors' code/raw outputs, or independently rerun the CIFAR-10 heterogeneous mixed ensemble (M=10, T=3, ηα=0.1, five holdout splits, SimpleCNN/ResNetTiny1/ResNetTiny2) and compare its accuracy and efficiency against the MNIST rows. If the corrected CIFAR-10 values differ from the MNIST values, Tables 7–10 cannot be used as printed and the claimed ranking and trade-off analysis must be recomputed. If the rerun reproduces exact equality, verify that the CIFAR-10 data loader was actually used; exact numerical equality across datasets is strong evidence of a table-generation or dataset-labeling error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is not the unproven Bregman-centroid identification, but the internal inconsistency of the experimental tables that carry the central claim. Section 4.4 claims s-BFN outperforms logit averaging and MoE across MNIST, Fashion-MNIST, and CIFAR-10, and that the results reveal a complexity–capacity–diversity trade-off. However, Table 7 lists exactly the same accuracies for MNIST and CIFAR-10: Heterogeneous sRBF=49.10, MoE=47.96, Base=39.71; Homogeneous sRBF=32.70, MoE=33.45, Base=30.38. Table 8 repeats the same best-configuration rows for MNIST and CIFAR-10; Table 9 gives identical train/eval times, FLOPs, and parameter counts; Table 10 repeats identical sensitivity values for every ε. MNIST and CIFAR-10 are different datasets, and exact equality to two decimals across multiple independent metrics cannot be expected by chance. This indicates a copy/paste or labeling error in the manuscript. Without code or raw results, the error cannot be fixed by inspection. As printed, the tables do not support the paper's headline empirical claim, independent of whether the RBF combiner can be proven to compute the Bregman centroid.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Structured Basis Function Networks (s-BFN), a combiner that maps a concatenation of base-predictor outputs through a radial-basis feature map and a linear readout, with the claimed property that the resulting prediction is the Bregman centroid induced by the task loss (squared loss for regression, cross-entropy for classification). A tunable diversity parameter ε modulates base-learner updates, and the framework admits both a closed-form ridge solution and gradient-based training. Experiments on tabular regression and MNIST/Fashion-MNIST/CIFAR-10 image classification are presented to support claims that s-BFN outperforms logit averaging, Mixture-of-Experts, and arithmetic-mean baselines, and to study a complexity–capacity–diversity trade-off.","tokens_in":20378,"tokens_out":3252,"duration_ms":40775,"significance":"If the central claims held, the paper would offer a useful synthesis of multi-hypothesis prediction and loss-geometry-aware ensembling. The closed-form ridge estimator (Eq. 7) and the gradient updates in Section 3.3.2 are standard and internally consistent, and the explicit diversity-control mechanism is a practical idea. However, the empirical core is compromised by an internal inconsistency in the reported tables, and the theoretical link between the s-BFN output and the Bregman centroid is asserted rather than proven. As printed, the paper does not support its headline empirical claims.","major_comments":[{"comment":"Tables 7, 8, 9, and 10 report exactly identical values for MNIST and CIFAR-10: Table 7 gives the same sRBF/MoE/Base accuracies (49.10/47.96/39.71 and 32.70/33.45/30.38), Table 8 gives identical best-configuration accuracies, Table 9 gives identical train/eval times, FLOPs, and parameters, and Table 10 gives identical sensitivity values for every ε. MNIST and CIFAR-10 are different datasets, so exact equality to two decimals across all metrics cannot occur by chance. This indicates a copy/paste or labeling error. Section 4.4's central claim that 's-BFN achieves the highest accuracy, followed by logit averaging, MoE, and arithmetic-mean' across datasets is therefore unsupported by the printed evidence. This is a load-bearing error: the main empirical contribution cannot be verified without corrected tables or the underlying raw results/code.","section":"Tables 7–10 (Section 4.5)"},{"comment":"The paper asserts that the s-BFN output Φ(Di)α is the Bregman centroid of the base predictions, but no derivation or theorem establishes this. For cross-entropy, the Bregman centroid under KL divergence is a weighted geometric mean of probability vectors (equivalently, softmax of weighted logits). The s-BFN instead applies a radial-basis feature map to concatenated probability vectors and a linear readout; even if α is trained with the task loss, this does not imply that the resulting prediction equals the centroid. The RBF feature map may not be expressive enough, and no convergence or representational result is given. This matters because 'centroidal aggregation' is a central claim of the abstract and introduction. At minimum, the paper should state this as an approximation and provide a concrete test (e.g., synthetic mixtures where the true centroid is known) or a proof under explicit","section":"Section 3.2.1, Figure 1"},{"comment":"Several quantitative claims about the complexity–capacity–diversity trade-off are based on figures that are described but whose numerical values are not reported in tables. For instance, the claims that 'accuracy peaks at moderate M on MNIST' and 'on CIFAR-10, capacity gains materialize only when diversity is sufficiently high' are asserted from plots without accompanying summary statistics or error bars in the text. Given that the only dataset-level tables (Tables 7–10) contain the MNIST/CIFAR-10 duplication, these figure-dependent conclusions are not independently checkable from the manuscript. Please provide per-configuration numerical results or a data/code release.","section":"Section 4.4 and Figures 3–8"}],"minor_comments":[{"comment":"The regularization term in the update is written inconsistently: Eq. (8) uses λ1^j ∇θj Rθj(θj), while Algorithm 1 writes λ1^j/N sgn(θj). Please align the notation and specify the exact regularization used.","section":"Algorithm 1"},{"comment":"The phrase 'the ensemble output must follow the associated centroidal rule' is stated as fact. Since the Bregman centroid is the optimal combiner for a given loss only under a specific aggregation model, please qualify this statement and cite the precise assumptions from Wood et al. (2024).","section":"Section 3.2.1"},{"comment":"There is no code or data availability statement. Given the empirical inconsistencies, providing code or raw output tables would be essential for reproducibility.","section":"Various"},{"comment":"The comparison between GBF and RBF-KMeans is useful, but the description of center initialization says GBF centers are 'uniformly assigned' while RBF-KMeans centers are 'derived from unsupervised clustering.' Please clarify whether GBF centers are fixed or learned, and how this affects the claimed advantages.","section":"Appendix A"}],"recommendation":"reject","confidential_remarks":"The identical MNIST/CIFAR-10 rows in Tables 7–10 are a serious integrity concern, not a minor typo. Even if a resubmission corrects them, the Bregman-centroid identification remains an unproven assertion. The paper would need substantially revised experiments and theory, not just a correction of the tables."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the method is a reasonable incremental extension of the authors' earlier s-BFN work, but the empirical section as printed is not trustworthy. Tables 7, 8, and 10 report identical accuracy values for MNIST and CIFAR-10, which are different datasets. That is a copy/paste or labeling error, and it runs through the core results that support the headline claim that s-BFN beats logit averaging and MoE.\n\nWhat is actually new: the gradient-based training procedure for non-quadratic losses (cross-entropy) and the systematic sweep of the ε diversity parameter on deep ensembles. Those are legitimate extensions, and the paper is honest that the closed-form least-squares estimator was already proposed in Dominguez et al. (2025). The RBF feature map over concatenated base predictions, with a linear head and temperature softmax, is simple and plausible.\n\nWhere it falls down: first, the empirical inconsistency is not a minor typo. Exact equality to two decimals across multiple independent metrics for two different datasets cannot be a coincidence. The paper gives no code or raw results, so the error cannot be fixed by inspection. As printed, the tables do not support the paper's central empirical claim, regardless of the theory.\n\nSecond, the identification of the s-BFN output with the loss-induced Bregman centroid is asserted, not derived. Section 3.2.1 and Figure 1 claim the link, but the RBF map with running-mean centers is not shown to be expressive enough to represent the centroid. If the combiner is just a learned combiner that minimizes the loss, that is fine, but then the 'loss-aware centroidal aggregation' language is overreach. The authors themselves flag a formal characterization of the trade-off as future work, which is an honest limitation.\n\nThird, even if the tables were corrected, the contribution is incremental: the theory comes from Wood et al. (2024), the closed-form estimator from the authors' own prior work, and the new pieces are algorithmic and empirical.\n\nWho this is for: someone working on ensemble combiners or multi-hypothesis prediction might find the gradient-based extension and the diversity-sweep idea worth a look, but only after the authors fix the tables and release code. As printed, the empirical claims should not be cited.\n\nRecommendation: I would not cite this in its current form, and I would not accept it. But it is not incoherent; the method and the new training procedure are real. A serious referee could help if the venue asks for major revisions, including correcting or replacing the affected tables and making the code available. This deserves a serious referee rather than a desk reject, because the underlying method is salvageable and the errors are identifiable.","headline":"The method is a reasonable incremental extension, but the identical MNIST/CIFAR-10 numbers in Tables 7–10 make the empirical core unusable as printed.","tokens_in":20854,"tokens_out":4241,"would_cite":false,"duration_ms":42421,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Structured Basis Function Networks claim to unify multi-hypothesis prediction and ensemble learning by aggregating base predictors at the Bregman centroid of the task loss, with a tunable diversity parameter that controls the bias-variance-","keywords":["Bregman divergences","deep ensembles","multi-hypothesis prediction","structured basis function networks","diversity control","centroidal aggregation","bias-variance-diversity trade-off","loss geometry"],"falsifier":"On a regression problem with M known base predictors, compute the exact Bregman centroid for squared loss - the weighted arithmetic mean sum_j alpha_j f_j / sum_j alpha_j - and compare it with the s-BFN output Phi(D_i) alpha on a held-out set. If the two disagree on a material fraction of inputs (beyond optimization tolerance), the claim that s-BFN implements centroidal aggregation is falsified; likewise for cross-entropy, compare to the KL centroid of the base probability vectors.","tokens_in":19907,"feed_emoji":"🎯","tokens_out":6558,"duration_ms":69033,"temperature":0.7,"pith_summary":"This paper argues that multi-hypothesis prediction and ensemble learning are two halves of one problem: how to combine several plausible outputs in a way that respects the geometry of the loss being optimized. Its central object is the Structured Basis Function Network (s-BFN), a combiner that maps the concatenated predictions of base models through radial-basis features and linearly aggregates them, with the loss chosen by the user (squared loss for regression, cross-entropy for classification). The paper claims this s-BFN output is the Bregman centroid of the base predictions, so that the combiner is aligned with the loss geometry rather than a generic averaging rule. A single scalar parameter epsilon reweights the base learners' updates, interpolating between hard winner-takes-all training and uniform updates, and thereby giving parametric control over diversity. Experiments across regression and image classification report that s-BFN consistently outperforms logit averaging, Mixture-of-Experts, and arithmetic-mean baselines, with the best results at intermediate epsilon and heterogeneous base architectures.","feed_headline":"One combiner unifies multi-hypothesis and ensemble predictions","feed_subtitle":"Tuning a single diversity knob controls the bias-variance-diversity trade-off across regression and classification benchmarks.","key_machinery":"The load-bearing object is the structured basis function network (s-BFN): a radial-basis feature map Phi(D_i; theta) = [phi(||D_i - C_k||/gamma_k)] over the vector of all base predictions, followed by a linear head alpha (and a temperature-scaled softmax in classification). This map is what the paper identifies with the Bregman centroid z_hat = argmin_z sum_j alpha_j B_phi(z_j, z); the feature centers are running statistics of base predictions, and alpha is fit by least squares or gradient descent. The diversity mechanism is the epsilon-modulated update delta_i^(j) = 1-epsilon for the best predictor and epsilon/(M-1) for the rest, which controls how winner-takes-all the training is and thus","core_discovery":"The paper's central claim is that a combiner built as a structured basis function network over base-prediction vectors coincides with the loss-induced Bregman centroid, so that the same framework covers regression and classification by choosing the matching divergence. For squared loss, the combiner is a linear readout of radial-basis features over the concatenated scalars; for cross-entropy, it is a temperature-scaled softmax over radial-basis features of the base models' probability vectors. The combiner weights admit a closed-form ridge solution in the Euclidean case and gradient-based training in general, and a diversity parameter epsilon modulates each base learner's update so that the","pith_inferences":["An extension the paper leaves implicit: if the centroidal identification is taken literally, s-BFN should reproduce the exact Bregman centroid as the number of basis functions and training data grow; one testable prediction is that accuracy gains concentrate where base predictions are well separated into Voronoi cells.","The epsilon mechanism suggests a continuous family of training objectives between hard winner-takes-all and uniform updates; scheduling epsilon over training or per hypothesis could adapt diversity to local data complexity more flexibly than a global constant.","Because the combiner feeds on softmax probabilities rather than logits, its advantage over logit averaging is likely most pronounced when base softmax outputs are miscalibrated; temperature-scaling the base probabilities before the RBF map is a natural intervention the paper does not explore.","The representational claim that RBF features with running-statistic centers can express the loss centroid is the fragile link; a parameter-free diagnostic comparing s-BFN outputs to closed-form centroids on held-out base predictions would directly probe it."],"forward_implications":["If the central claim holds, multi-hypothesis prediction and ensembling share one principled combination rule: the Bregman centroid of the loss, rather than ad hoc averaging or gating.","Practitioners get a direct knob over the bias-variance-diversity trade-off, with evidence that intermediate diversity (epsilon in roughly [0.3, 0.8]) and heterogeneous base architectures give the best accuracy.","The closed-form least-squares solution for squared losses and gradient-based training for arbitrary losses means the same framework can be extended to non-Euclidean losses such as Itakura-Saito divergence.","The reported ranking of combiners - s-BFN above logit averaging, MoE, and arithmetic mean - implies that learned probability-space aggregation can be a robust default for deep ensembles, especially when base models are miscalibrated.","Optimal ensemble size depends on dataset complexity and diversity, so capacity should be scaled jointly with diversity rather than treated as an independent hyperparameter."],"supporting_citations":[{"why":"Supplies the multiple-hypotheses prediction setup with Voronoi tessellations that s-BFN builds on, and the arithmetic-mean baseline it must beat.","marker":"Rupprecht et al., 2017"},{"why":"Provides the Bregman-divergence theory that optimal ensemble combiners should align with loss geometry, the core premise of s-BFN.","marker":"Wood et al., 2024"},{"why":"Defines Bregman divergences, the mathematical object used to induce the centroidal combination rules.","marker":"Bregman, 1967"},{"why":"Introduces the closed-form least-squares s-BFN estimator that this paper extends to gradient-based training and arbitrary losses.","marker":"Dominguez et al., 2025"},{"why":"Supplies the deep-ensemble baseline and the uncertainty-estimation setting that s-BFN is compared against.","marker":"Lakshminarayanan et al., 2017"},{"why":"Motivates the logit-averaging baseline that s-BFN is designed to outperform.","marker":"Hinton et al., 2015"},{"why":"Provides the PAC-Bayesian disagreement framework used in the diagnostics to explain the epsilon trade-off.","marker":"Germain et al., 2015"},{"why":"Introduces the Winner-Takes-All loss and Multiple Choice Learning that the epsilon-diversity mechanism extends.","marker":"Guzmán-rivera et al., 2012"},{"why":"Gives the ambiguity decomposition justifying why diversity can improve ensemble error.","marker":"Krogh and Vedelsby, 1994"}],"fun_headline_variants":["Bregman combiner unifies multi-hypothesis and ensemble learning","Tunable-diversity combiner merges multi-hypothesis ensembles","Controllable diversity unifies multi-hypothesis and ensemble predictions","Bregman centroid ties multi-hypothesis to ensembles","Loss-geometry combiner unites multi-hypothesis and ensembling"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The paper assumes that a radial-basis function whose centers are running statistics of base predictions can actually represent the loss-induced Bregman centroid, so that training the combiner with the task loss gives a truly loss-aware ensemble; if that representational assumption fails, the combiner still minimizes training loss but the centroidal interpretation is unsupported.","fun_headline_variants_meta":{"raw":{"variants":["Bregman combiner unifies multi-hypothesis and ensemble learning","Tunable-diversity combiner merges multi-hypothesis ensembles","Controllable diversity unifies multi-hypothesis and ensemble predictions","Bregman centroid ties multi-hypothesis to ensembles","Loss-geometry combiner unites multi-hypothesis and ensembling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001087,"raw_usage":{"total_tokens":4347,"prompt_tokens":677,"completion_tokens":3670,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":421,"completion_tokens_details":{"reasoning_tokens":3577}},"tokens_in":421,"tokens_out":3670,"duration_ms":30127,"temperature":1.0,"reasoning_tokens":3577,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:23:31.430041+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a regression problem with M known base predictors, compute the exact Bregman centroid for squared loss - the weighted arithmetic mean sum_j alpha_j f_j / sum_j alpha_j - and compare it with the s-BFN output Phi(D_i) alpha on a held-out set. If the two disagree on a material fraction of inputs (beyond optimization tolerance), the claim that s-BFN implements centroidal aggregation is falsified; likewise for cross-entropy, compare to the KL centroid of the base probability vectors.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the multiple-hypotheses prediction setup with Voronoi tessellations that s-BFN builds on, and the arithmetic-mean baseline it must beat."},{"cited_title":"M., Reeve, H","cited_arxiv_id":null,"evidence_quote":"Provides the Bregman-divergence theory that optimal ensemble combiners should align with loss geometry, the core premise of s-BFN."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Bregman divergences, the mathematical object used to induce the centroidal combination rules."},{"cited_title":"R., Shahzad, M., and Hong, X","cited_arxiv_id":null,"evidence_quote":"Introduces the closed-form least-squares s-BFN estimator that this paper extends to gradient-based training and arbitrary losses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the deep-ensemble baseline and the uncertainty-estimation setting that s-BFN is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the PAC-Bayesian disagreement framework used in the diagnostics to explain the epsilon trade-off."},{"cited_title":"and Vedelsby, J","cited_arxiv_id":null,"evidence_quote":"Gives the ambiguity decomposition justifying why diversity can improve ensemble error."}],"review_version":1}