Pith. sign in

REVIEW 4 major objections 6 minor 19 references

Federated Learning with Discriminative Naive Bayes Classifier

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Pith's one-line read This paper proposes a federated Naive Bayes variant that shares only discriminative weight parameters, not probability tables, and reports that it outperforms generative federated baselines on 12 datasets except when the number of clients…

desk verdict A plausible, clearly-written federated extension of weighted Naive Bayes with solid experimental breadth, whose privacy selling point is asserted rather than demonstrated. read the letter →

arxiv 2502.01532 v1 pith:STMOOUSB submitted 2025-02-03 cs.LG

classification cs.LG
keywords federatedlearningNaiveBayesdiscriminativeweightedprivacy-preservingmachineBayesiannetworkclassifiers
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a discriminatively trained Naive Bayes classifier can be federated by sharing only the learned weight vector, with each client keeping its own probability tables local. The proposed scheme, NBw5fed, combines generative estimation of local tables with federated discriminative optimization of a weight vector, and on 12 discrete datasets it achieves higher test accuracy than the non-federated discriminative model and the generative federated baseline in most client-count settings. The intended payoff is privacy: because only opaque weights cross the network, the method is claimed to be more reliable against attacks than sharing conditional probability tables. The main caveat is that the generative federated baseline performs better when the number of clients is very large.

What carries the argument

The central mechanism is the Weighted Naive Bayes (NBw) hybrid, which first estimates the generative probability tables $\theta$ and then learns a weight vector $w$, one weight per parameter, by maximizing conditional log-likelihood. The federated algorithm FedNBw wraps this in a federated averaging loop: clients keep $\theta$ local, synchronize on the global $w$, take a bounded number of L-BFGS-M steps against local data, and the server averages the returned weight vectors. The five-iteration inner limit is the load-bearing tuning choice: it prevents the local discriminative step from overfitting sparse client data, and the paper shows it is what lets the federated weight-sharing variant beat both the unconstrained local model and the generative baseline in most regimes.

What would settle it

Run FedNBw on synthetic datasets where the true class priors and feature-conditional counts are known, give an attacker the sequence of per-round weight vectors (or even the final global weights), and test whether the attacker can recover those counts or identify whether a specific record was in a client's data with better-than-chance accuracy; if any such reconstruction succeeds, the advertised privacy advantage is false. A simpler check is to compare the weight trajectories produced by two datasets that differ only in class priors: if the trajectories diverge systematically, the weights carry information about the very quantities the paper says stay private.

Watch

Extended reading notes

Core claim

The central claim is that the hybrid Weighted Naive Bayes (NBw) model, which attaches a real-valued weight to each generative parameter, can be turned into a practical federated classifier without ever transmitting counts or conditional probabilities. In the FedNBw algorithm each client first computes local probability tables, then repeatedly copies the current global weight vector, runs a limited number of L-BFGS-M optimization steps on its own data, sends the local weight updates to the server, and receives the averaged global weights back. The paper reports that limiting local optimization to five iterations and using the global weights directly, the NBw5fed configuration, gives the best accuracy among the variants tested and generally outperforms both the generative federated baseline and the non-federated discriminative model, with the exception of the 50- and 100-client regimes where the table-sharing baseline wins because it is unaffected by data partitioning. The paper claims this is more privacy-preserving because the shared weights are 'inherently meaningless in revealing sensitive information.'

Load-bearing premise

The load-bearing premise is that the shared weight vector is inherently 'meaningless' and therefore cannot reveal sensitive information about a client's data; the paper assumes this privacy property rather than deriving it from a threat model, an attack analysis, or a formal privacy bound.

Editorial extensions

If this is right

  • Federated discriminative Naive Bayes can be deployed in settings where sharing conditional probability tables is considered too sensitive, since only the weight vector crosses the network.
  • The finding that a five-iteration local optimization limit improves federated accuracy provides a practical default for similar federated discriminative algorithms.
  • In regimes with many clients, the generative table-sharing baseline remains superior, so a deployer may need to choose between the two approaches based on the expected number of clients.
  • The authors identify differential privacy as a natural next step to give formal privacy guarantees on top of the weight-sharing scheme.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The privacy claim is untested: because the paper gives no threat model or reconstruction attack, an attacker who observes the sequence of per-round weight updates might extract more information than a single averaged weight vector would suggest.
  • Even if final weights are opaque, intermediate per-client updates sent to the server each round may carry information about local class distributions; measuring this leakage empirically would settle whether the privacy advantage is real.
  • The same federated weight-sharing idea could extend naturally to semi-naive Bayes or tree-augmented classifiers, since the weights are generic per-parameter scalars rather than NB-specific quantities.
  • The large-client regime result hints at a scaling rule: when each client has very little data, discriminative fine-tuning of shared weights degrades, and directly federating the tables becomes the more robust choice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Federated Weighted Naive Bayes (NBwfed), a federated learning algorithm for discrete discriminative Naive Bayes. In the proposed Algorithm 1, each client learns local conditional probability tables in a generative step and keeps them private, then participates in a federated discriminative step in which only weight vectors of a weighted NB (NBw) model are optimized locally, averaged on the server, and broadcast back. The authors compare NBwfed with non-federated NB and NBw, and with a federated generative NB baseline (NBfed) on 12 OpenML datasets across 5, 10, 20, 50, and 100 clients, using 5-fold cross-validation repeated over 5 seeds. They report that the variant with a 5-iteration L-BFGS-M limit and global weights (NBw5fed) generally achieves the best accuracy, except that NBfed performs better at large client counts. The central advertised advantage of the method is privacy: sharing 'meaningless' weight parameters rather than conditional probability tables is claimed to be more reliable against possible attacks.

Significance. If both the accuracy and privacy claims were substantiated, the paper would make a useful contribution to federated learning for explainable, lightweight classifiers: it demonstrates that a discriminatively trained NB can be federated by exchanging only weights, and it provides broad experimental evidence (12 datasets, 5 client counts, repeated runs) plus released code and OpenML dataset identifiers. The experimental design is commendable in scope and reproducibility, and the comparison with generative federated NB is a sensible baseline. However, the privacy claim is asserted rather than proven, and the statistical reporting is too thin to support the main comparative accuracy statement. The algorithmic description also leaves important implementation details (e.g., zero-count handling and the federated round count) unspecified. The useful, defensible core of the paper is the empirical exploration of a weight-sharing federated NB; the privacy advantage is currently a motivation, not a demonstrated property.

major comments (4)
  1. [§2.3 and Abstract] The paper's central privacy claim—that the shared discriminative weight vectors are 'inherently meaningless in revealing sensitive information'—is asserted without a threat model, an attack analysis, or any formal privacy bound. Each client transmits w_c^{(t)} obtained by L-BFGS-M optimization of the conditional log-likelihood on local data, and the CLL gradient with respect to w_{y,x_j} depends directly on local sufficient statistics such as #(x_j,y) (see §2.2), so the shared weights can encode information about local class-conditional counts. The conclusion (Section 5) lists differential privacy as future work, implicitly conceding that no quantifiable privacy guarantee is currently provided. The authors should either give a formal privacy analysis under an explicit adversary model, or substantially weaken the privacy claims and present the method as a communication-efficient alternative to sharing probability tables, with privacy as a motivating direction for further study.
  2. [§4.2, Table 2, Figures 1–3] The experimental protocol (5-fold CV, 5 seeds, 25 runs) is described, but Table 2 and Figures 1–3 report only mean accuracy, with no standard deviations, confidence intervals, or significance tests. The statement that 'NBw5fed outperforms all other algorithms, except in scenarios with a large number of clients' rests on small mean differences (e.g., Table 2, 5 clients, House Votes: NBw5g 94.85 vs NBw5l 94.29), which may be within run-to-run noise. The authors should report variance and apply paired tests across datasets (e.g., Wilcoxon signed-rank or a sign test on per-dataset means) to support the cross-dataset conclusions.
  3. [§4.2, Algorithm 1] Algorithm 1 depends on the federated round count T, but the experiments never state the value of T; Table 2 reports accuracy at 'the last iteration' without specifying how many iterations were performed. This is a load-bearing configuration choice: the behavior of the 5-iteration-limited variant cannot be fully interpreted without knowing the number of federated rounds, and the communication cost of the method is not quantified. Please report T (and any stopping criterion) for each experimental setting.
  4. [§2.2 and §4.3] The CLL objective for NBw contains terms of the form log θ_{y,x_j}, which are undefined when a local probability table contains zeros. The paper does not state whether Laplace smoothing or another zero-count correction is applied in the generative step or inside the L-BFGS-M optimization. This is particularly relevant because the authors explicitly discuss scenarios 'where most parameters are zero' (§4.3). Without a description of zero-count handling, the algorithm's behavior on small or sparse client partitions is not well-defined and the reported results are not fully reproducible.
minor comments (6)
  1. [§3] There is a naming inconsistency: Algorithm 1 is titled 'FedNBw' while the text, tables, and figures use 'NBwfed'; please unify the notation.
  2. [Algorithm 1, line 7] Line 7 contains a typo: 'Θ_i,y_c,x_c_j' should be 'Θ_c,y_c,x_c_j', and the subscript x_c_j is ambiguous; clarify that it denotes the j-th feature value of instance x_c.
  3. [Table 1] The row for 'House Votes 84' is misaligned: the columns read 'House Votes 84 435 16 2 56', which suggests m=84 though the dataset has 435 instances; correct the table formatting.
  4. [Figures 1–3] In the version of the manuscript provided, the figure text appears as garbled glyph sequences rather than readable labels; ensure that fonts are embedded and legends are legible in the final PDF.
  5. [§2.2] The NBw probability formula writes θ_{y_k}^{w_y} and θ_{y_k,x_j}^{w_y,x_j} in the denominator without clarifying whether these are powers or subscripts; the exponent notation should be defined explicitly.
  6. [References] Reference [12] contains a typo ('Proocedings'); please correct it and verify the conference proceedings venue.

Circularity Check

1 steps flagged · score 4.0 of 10

The accuracy benchmark is self-contained, but the paper's advertised privacy advantage is self-definitional: shared weights are labeled 'meaningless,' and that label is then reported as the privacy result.

  1. self definitional [Section 2.3, final paragraph (restated in Abstract and Section 5)]
    "In contrast, the parameters federated in the NBw model are inherently meaningless in revealing sensitive information. Therefore, this model provides a clear advantage over traditional methods in terms of privacy preservation."

    The claimed privacy advantage is not derived from an attack model, from the CLL gradient equations, or from any information-theoretic bound; it is the predicate 'meaningless in revealing sensitive information' restated as 'advantage in terms of privacy preservation.' Since 'meaningless' is used to mean 'does not reveal sensitive information,' the conclusion is contained in the premise by definition. The paper's own Section 5 lists differential privacy as future work, confirming that no formal privacy guarantee is actually established. This is a self-definitional step rather than an empirical or mathematical derivation. The accuracy results remain independent of this step.

full rationale

The paper contains no self-citations on which the argument depends, and no fitted quantity is renamed as a prediction in the accuracy chain: Table 2 and Figures 1-3 report test accuracy on held-out folds with 25 repetitions, so the central empirical claim that NBw5fed is competitive with generative federated NB is self-contained and externally checkable. The one load-bearing step that reduces to its own input is the privacy claim. The paper repeatedly argues that sharing 'meaningless' parameters is 'more reliable against possible attacks' and 'provides a clear advantage over traditional methods in terms of privacy preservation,' but the term 'meaningless' is doing all the work: a parameter that is 'meaningless in revealing sensitive information' is, by definition, privacy-preserving. No independent argument, threat model, reconstruction attack, or formal bound supports the premise. A separate methodological concern, not circularity, is that the L=5 iteration limit was selected after inspecting the same test table and then the configuration was renamed NBw5fed and compared on the same test data; this is a model-selection bias rather than a reduction by construction. Overall, the accuracy result has independent content, but the paper's advertised privacy advantage is definitional, yielding a moderate circularity score.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities. Its assumptions are the standard NB independence condition, the unspecified behavior of federated averaging, the unsupported privacy claim about shared weights, and the sufficiency of a small number of optimizer iterations.

free parameters (2)
  • L (inner L-BFGS-M iteration limit) = 5
    Chosen as the best configuration after comparing test accuracy in Table 2; used to define the headline NBw5fed method without a separate validation split.
  • T (number of federated rounds) = not specified
    Algorithm 1 takes T as input, but the paper never states its value or stopping criterion; this affects both communication cost and the final accuracy.
assumptions (4)
  • domain assumption Naive Bayes conditional independence assumption
    The whole model factorizes P(y,x) as P(y) times the product of P(x_j|y) (Section 2.1). This is standard for NB but rarely exactly true in data.
  • domain assumption Federated averaging of local weight updates is a meaningful aggregation
    Algorithm 1 lines 19-21 average client weight vectors; no convergence or utility analysis is provided, and local optima may be incomparable.
  • ad hoc to paper Shared weights do not reveal private information
    Section 2.3 declares the weights 'inherently meaningless in revealing sensitive information'. This is the paper's central privacy justification and is asserted without a threat model, formal bound, or attack experiment.
  • domain assumption Five L-BFGS-M iterations with random weight initialization improve the conditional log-likelihood enough to be useful
    The method depends on local optimization improving the shared weights from a random start; the paper tests L=5 empirically but provides no guarantee or sensitivity analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Federated Learning with Discriminative Naive Bayes Classifier." pith.science (2026). https://pith.science/paper/STMOOUSB

@misc{pith2026250201532,
  author       = {Pith},
  title        = {Pith review of: Federated Learning with Discriminative Naive Bayes Classifier},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/STMOOUSB}},
  note         = {Machine review of arXiv:2502.01532}
}
read the original abstract

Federated Learning has emerged as a promising approach to train machine learning models on decentralized data sources while preserving data privacy. This paper proposes a new federated approach for Naive Bayes (NB) classification, assuming discrete variables. Our approach federates a discriminative variant of NB, sharing meaningless parameters instead of conditional probability tables. Therefore, this process is more reliable against possible attacks. We conduct extensive experiments on 12 datasets to validate the efficacy of our approach, comparing federated and non-federated settings. Additionally, we benchmark our method against the generative variant of NB, which serves as a baseline for comparison. Our experimental results demonstrate the effectiveness of our method in achieving accurate classification.

Figures

Figures reproduced from arXiv: 2502.01532 by the authors.

Figure 1
Figure 1. Train accuracy of the algorithms over all the datasets – The federated versions of the algorithms consistently achieve superior results compared to their non-federated counterparts. – The discriminative algorithm (NBw) generally outperforms the generative algorithm (NB), as discussed in Section 2.2. – Without federation, applying an iteration limit to NBw may not be benefi￾cial, although it improves performance in t… view at source ↗
Figure 2
Figure 2. Test accuracy of the algorithms over all the datasets pendence from data partitioning. Nonetheless, NBw 5 f ed demonstrates strong performance even if it only federates the parameter weights rather than the probability tables. This finding is important considering that, in cases with minimal data, where most parameters are zero, it still performs closely to the fully federated NBf ed. – Analysis of the training data… view at source ↗
Figure 3
Figure 3. Mean train and test accuracy erating a discriminative variant of NB that, by sharing meaningless parameters rather than conditional probability tables, offers enhanced privacy protection compared to the generative version. Through comprehensive experiments on 12 discrete datasets, we have demonstrated the effectiveness of our proposed method in achieving accurate classification results. Our results highlight the pot… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 18 canonical work pages

  1. [1]

    Information Fusion58, 82–115 (2020)

    Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., et al.: Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward re- sponsible AI. Information Fusion58, 82–115 (2020)

  2. [2]

    In: Proceedings of the 20th International Conference on Smart Business Technologies

    Giaretta, L., Marchioro, T., Markatos, E., Girdzijauskas, S.: Towards a Realistic Decentralized Naive Bayes with Differential Privacy. In: Proceedings of the 20th International Conference on Smart Business Technologies. pp. 98–121 (2023)

  3. [3]

    In: Proceedings of the 8th International Conference on Knowledge Science, Engineering and Management

    Huai, M., Huang, L., Yang, W., Li, L., Qi, M.: Privacy-Preserving Naive Bayes Classification. In: Proceedings of the 8th International Conference on Knowledge Science, Engineering and Management. pp. 627–638 (2015)

  4. [4]

    Computers & Industrial Engineering149, 106854 (2020)

    Li, L., Fan, Y., Tse, M., Lin, K.Y.: A review of applications in federated learning. Computers & Industrial Engineering149, 106854 (2020)

  5. [5]

    IEEE Transactions on Big Data8, 843–854 (2022)

    Liu, Y., Liu, Y., Liu, Z., Liang, Y., Meng, C., Zhang, J., Zheng, Y.: Federated Forest. IEEE Transactions on Big Data8, 843–854 (2022)

  6. [6]

    In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics

    McMahan, B., Moore, E., Ramage, D., Hampson, S., Agüera y Arcas, B.: Communication-Efficient Learning of Deep Networks from Decentralized Data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. pp. 1273–1282 (2017)

  7. [7]

    In: Proceedings of the 14th International Conference on Computing Communication and Networking Technologies

    Rahman, M.M., Farid, D.M.: Exploring Federated Learning with Naïve Bayes us- ing AVC Information. In: Proceedings of the 14th International Conference on Computing Communication and Networking Technologies. pp. 1–6 (2023)

  8. [8]

    Machine Learning59, 267– 296 (2005)

    Roos, T., Wettig, H., Grünwald, P., Myllymäki, P., Tirri, H.: On Discriminative Bayesian Network Classifiers and Logistic Regression. Machine Learning59, 267– 296 (2005)

Show all 19 references
  1. [9]

    In: Proceed- ings of the 3rd International Conference on Knowledge Discovery and Data Mining

    Rubinstein, D., Hastie, T.J.: Discriminative vs Informative Learning. In: Proceed- ings of the 3rd International Conference on Knowledge Discovery and Data Mining. pp. 49–53 (1997)

  2. [10]

    Generative Learning of Bayesian Network Classifiers

    Santafé, G., Lozano, J.A., Larrañaga, P.: Discriminative vs. Generative Learning of Bayesian Network Classifiers. In: Proceedings of the 9th European Conference on Symbolic and Quantitative Approaches to Reasoning and Uncertainty. pp. 453–464 (2007)

  3. [11]

    The VLDB Journal17, 879–898 (2007)

    Vaidya, J., Kantarcıoğlu, M., Clifton, C.: Privacy-preserving Naïve Bayes classifi- cation. The VLDB Journal17, 879–898 (2007)

  4. [12]

    In: Proocedings of the 2013 IEEE/WIC/ACM International Joint Con- ferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT)

    Vaidya, J., Shafiq, B., Basu, A., Hong, Y.: Differentially Private Naive Bayes Clas- sification. In: Proocedings of the 2013 IEEE/WIC/ACM International Joint Con- ferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT). pp. 571–576 (2013)

  5. [13]

    ACM SIGKDD Explorations Newsletter15, 49–60 (2013)

    Vanschoren, J., van Rijn, J.N., Bischl, B., Torgo, L.: OpenML: Networked Science in Machine Learning. ACM SIGKDD Explorations Newsletter15, 49–60 (2013)

  6. [14]

    In: Encyclopedia of Machine Learning and Data Mining, pp

    Webb, G.I.: Naïve Bayes. In: Encyclopedia of Machine Learning and Data Mining, pp. 713–714. Springer (2010)

  7. [15]

    Information Systems34, 371–380 (2009)

    Yi, X., Zhang, Y.: Privacy-preserving naive Bayes classification on distributed data via semi-trusted mixers. Information Systems34, 371–380 (2009)

  8. [16]

    Journal of Machine Learning Research 14, 1947–1988 (2013)

    Zaidi, N.A., Cerquides, J., Carman, M.J., Webb, G.I.: Alleviating naive Bayes attribute independence assumption by attribute weighting. Journal of Machine Learning Research 14, 1947–1988 (2013)

  9. [17]

    Machine Learning 106, 1289–1329 (2017) Federated Learning with Discriminative Naive Bayes Classifier 13

    Zaidi, N.A., Webb, G.I., Carman, M.J., Petitjean, F., Buntine, W., Hynes, M., De Sterck, H.: Efficient parameter learning of Bayesian network classifiers. Machine Learning 106, 1289–1329 (2017) Federated Learning with Discriminative Naive Bayes Classifier 13

  10. [18]

    Knowledge-Based Systems 216, 106775 (2021)

    Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., Gao, Y.: A survey on federated learning. Knowledge-Based Systems 216, 106775 (2021)

  11. [19]

    ACM Transactions on Mathematical Software 23, 550–560 (1997)

    Zhu, C., Byrd, R.H., Lu, P., Nocedal, J.: Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software 23, 550–560 (1997)

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.