{"id":"af7ee9e0-42e1-421a-ad1a-b27f62d45fd1","arxiv_id":"2504.21739","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MaskedXGBoost trains vertical federated XGBoost with bilateral differential privacy by injecting noise that cancels in the null space of the split-score operation, preserving utility.","lead":"MaskedXGBoost is a protocol for vertical federated XGBoost that adds specially engineered noise to gradients and Hessians, giving differential privacy to both the label-holding and feature-holding parties while keeping split-score calculations nearly undisturbed. It aims for a better utility-privacy tradeoff than standard DP perturbation and faster training than homomorphic encryption.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bilateral DP guarantee covers only B_i and noised vectors; the split operation revealed in Algorithm 3 line 15 is outside Theorems 2–3, so the end-to-end guarantee for PP's features is unsupported.","rationale":"The reader identified the most load-bearing concern correctly: the protocol publishes the splitting operation at every node, and this output is not covered by Theorem 2 or Theorem 3. The central claim is bilateral DP for the VFL XGBoost training, which implies privacy for the full protocol transcript, not just for individual intermediate messages. Since AP can compute the noise it added and observes the chosen split, the split release can be combined with the noise matrices to extract additional information about PP's feature ordering. The paper does not provide a composition argument that includes this split-selection mechanism, so the stated (ε_PP, δ_PP) guarantee is not an end-to-end guarantee. I agree with the reader's assessment that this is fixable (e.g., by adding the split operation to the DP analysis or by explicitly restricting the privacy claim to intermediate messages), but as written the central claim is not fully supported. I also note a secondary, related underspecification: Algorithm 2 does not define the distribution of the coefficients c_i1..c_iW and d_i1..d_iW beyond the sum-of-squares constraint, which is needed to justify the Gaussian covariance in Theorem 2. This does not change the verdict; the primary concern remains the unanalyzed split release. The paper's null-space noise construction and the extensive empirical evaluation are genuine strengths, and the utility analysis in Theorem 1 is a useful contribution, but the privacy guarantee needs to be scoped honestly or extended.","tokens_in":36433,"tokens_out":5918,"duration_ms":64887,"concrete_test":"Extend the attribute-inference attacker in Section 6.6 to condition on the revealed split s*: use the known B_i, c,d, and ⟨g⟩_i,⟨h⟩_i to compute for each candidate split the score PP would have assigned, and compare with s* to reject implausible active sets. If the resulting bit-match accuracy exceeds the upper bound implied by (ε_PP, δ_PP) (e.g., by more than the 0.52 reported at ε_PP=1), the end-to-end guarantee fails. A sharper analytical check: compute the hockey-stick divergence D_{e^{ε_PP}}(M(m) || M(m')) for the joint output M(m)=(B_1,...,B_l, s*) for adjacent splitting vectors m,m' via Monte Carlo; if the divergence's δ exceeds the paper's δ_PP, Theorem 3's scope is insufficient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.4 (Algorithm 3, line 15) requires PP to reveal the selected splitting score and operation (feature and threshold) to AP at every node. Theorems 2 and 3 bound privacy only for the transmissions they analyze: Theorem 3 covers the noise matrices B_i (Section 5.3), and Theorem 2 covers the noised gradient/Hessian vectors (Section 5.2). The revealed split s* is a function of PP's private categorical matrix M and the noised messages ⟨g⟩_i,⟨h⟩_i that AP itself constructed; because AP also knows the coefficients c,d and noise B_i, it can combine s* with its own view to infer the active set/ordering of PP's features. No sequential/parallel composition argument in the paper includes the split-selection mechanism, so the claimed (ε_PP, δ_PP) guarantee does not extend to the full protocol output. This is the central weakness: the bilateral DP claim is scoped to intermediate messages, not the final model, whereas the protocol explicitly publishes the tree structure. Additionally, Algorithm 2 line 2 leaves the distribution of c_i1..c_iW unspecific; if those coefficients are randomized rather than fixed, the noise in Theorem 2 is a scale-mixture and the Gaussian covariance argument in Appendix C needs repair. The split-release gap is the primary load-bearing issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MaskedXGBoost, a vertical federated learning protocol for training XGBoost models with differential privacy for both the active party (AP, holding labels) and the passive party (PP, holding features). The core idea is to inject structured Gaussian noise: PP constructs noise vectors whose active and inactive components lie in the null space of each splitting vector, so they cancel out in the split-score computation, leaving only a smaller disturbing noise to affect utility. AP then applies these noise vectors to its gradients and Hessians before sending them to PP, which computes the split scores and reveals the best split. The paper provides a utility bound (Theorem 1) showing that the score disturbance vanishes as the disturbing-noise scale goes to zero, and gives differential privacy guarantees for AP (Theorem 2) and PP (Theorem 3). Experiments on six datasets show higher utility than DP baselines and lower training time than homomorphic encryption. The central claim is that MaskedXGBoost achieves bilateral differential privacy with better utility and efficiency than existing approaches.","tokens_in":36718,"tokens_out":8651,"duration_ms":80909,"significance":"The null-space noise construction is an elegant and original mechanism-design idea: it decouples privacy noise from the utility-critical projection, which is a genuinely useful contribution to DP mechanism design for tree-based models. If the bilateral DP claim were fully established, the paper would be the first to provide end-to-end DP for both parties in vertical federated XGBoost without heavy cryptography, with a strong utility/efficiency trade-off. The experimental validation is solid (six datasets, multiple baselines) and the theoretical proofs are detailed. However, the privacy analysis has a load-bearing gap: the protocol reveals the selected split operation (and split score) at every node, and this release is not covered by Theorems 2 and 3 nor by any composition argument. The utility analysis (Theorem 1) is sound as a standalone bound, and the empirical results support the utility claims. The paper would be a strong addition to the literature after the privacy accounting is completed.","major_comments":[{"comment":"The protocol requires PP to reveal the selected splitting score and operation (s*) to AP at every node (Algorithm 3, lines 11–16). Theorem 3 bounds the privacy of PP only for the transmitted noise matrices B_i, and Theorem 2 bounds only the noised gradient/Hessian vectors sent by AP. The revealed s* is a function of PP's private splitting vector m_i, AP's private gradients/Hessians, and the noise B_i; since AP knows g, h, and the coefficients c_ik, d_ik, it can compute all the noised aggregates and thus s* becomes a deterministic function of m_i that is not covered by the DP guarantee for B_i. No sequential/parallel composition argument in the paper includes this split-selection mechanism, so the claimed (ε_PP, δ_PP)-DP guarantee does not extend to the full protocol output (the tree structure). This is a load-bearing gap for the bilateral DP claim.","section":"§4.4 (Algorithm 3, line 15) and §5.3"},{"comment":"Algorithm 2 states only that the coefficients c_i1..c_iW and d_i1..d_iW satisfy Σ_k c_ik^2 = C and Σ_k d_ik^2 = C, without specifying their distribution. Theorem 2 and its proof in Appendix C rely on the total noise vector Σ_k c_ik b_ik being Gaussian with covariance matrix (12)–(13). If the coefficients are randomized (e.g., drawn uniformly from the sphere), the noise is a scale mixture of Gaussians and the Gaussian covariance argument in Appendix C no longer holds. The authors need to fix the coefficients (e.g., deterministic values) and re-derive the condition, or provide a DP proof for the actual distribution.","section":"§4.3 (Algorithm 2, line 2) and §5.2 (Theorem 2)"},{"comment":"Even setting aside the split release, the paper does not give a full composition argument for the repeated releases across nodes, trees, and candidates. Theorem 2 is stated for 'each time' AP sends ⟨g⟩_i and ⟨h⟩_i, and Theorem 3 for each noise matrix B_i, but the protocol releases l pairs of vectors per node and repeats this for every node and training round. The experiment section mentions using the advanced composition bound, but the composition count, the adaptivity, and the correlation between the releases of ⟨g⟩_i and ⟨h⟩_i (which share the same B_i) are not analyzed. Without this accounting, the claimed end-to-end (ε_AP, δ_AP)-LDP and (ε_PP, δ_PP)-DP are not demonstrated.","section":"§5.2–§5.3 and §6.2"}],"minor_comments":[{"comment":"The computation 'H ← Σ_{i=1}^n g_i' should be 'H ← Σ_{i=1}^n h_i', as H is defined as the sum of Hessians.","section":"Algorithm 2, line 5"},{"comment":"The phrase 'high performance and interoperability' appears to be a typo for 'high performance and interpretability'.","section":"Section 1, first paragraph"},{"comment":"The table header appears to have the row and column labels interchanged: the first column contains the method names and the remaining columns are datasets; please correct the header.","section":"Table 3"},{"comment":"The claim that 'if either party violates the protocol, it will only affect its own privacy and will not improve its ability to launch a privacy attack against the other party' is asserted without proof; consider providing an argument or softening the claim.","section":"Appendix G"},{"comment":"The figure legends are small and difficult to read; please enlarge the fonts or use a more legible layout.","section":"Figure 9 caption"}],"recommendation":"major_revision","confidential_remarks":"The main contribution is the null-space noise construction and the utility guarantee; the privacy proofs for the intermediate messages are plausible, but the end-to-end privacy claim is not yet established. The split-release issue is likely fixable only by either adding a DP analysis for the revealed split (which may be challenging) or by changing the protocol to hide the split selection from AP, e.g., using secure comparison. Given the strong experimental results and the originality of the mechanism, I see this as a major-revision candidate rather than a reject, but the authors should be prepared for a substantial revision to the privacy accounting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is worth your time. MaskedXGBoost constructs noise that lies partly in the null space of each splitting vector, so the noise cancels in the split-score computation. That is a real mechanism-design novelty, distinct from standard Gaussian perturbation, and it explains the consistent utility gains over FedXGBoost-LDP and XGBoost-LDP in their experiments. The efficiency win over HE is also credible. I found the utility analysis in Theorem 1 believable, and the empirical work on six datasets is more thorough than most papers at this stage.\n\nThe soft spot is exactly where the stress-test lands. Theorem 2 proves LDP for AP's transmitted noised vectors, and Theorem 3 proves DP for PP's noise matrices. But Algorithm 3 line 15 has PP reveal the chosen split and its score to AP at every node. That revealed split is a function of PP's private categorical matrix and of information AP already holds, so it is part of the protocol output. The paper never includes this split-release mechanism in any composition argument. Without that, the claimed (epsilon_PP, delta_PP) guarantee does not cover the final model or the tree structure, which is the thing an honest-but-curious AP actually sees. This is not a nitpick; it is the central privacy claim. The fix is to do an end-to-end privacy analysis that treats the split selection as a mechanism, possibly with composition across nodes and trees, or to explicitly scope the guarantee to a setting where split structure is not considered sensitive.\n\nA second, smaller gap: Algorithm 2 line 2 says AP generates c and d satisfying a norm constraint, but does not specify their distribution. If those coefficients are random, the noise added to g and h is a scale-mixture of Gaussians, and the fixed-covariance argument in Theorem 2 needs repair. If they are deterministic per round, say so. This is fixable but should be stated.\n\nThe attack experiments are nice supporting evidence but do not close the proof gap; empirical resistance is not the same as a DP guarantee. I also noticed the paper's claim of being \"first\" bilateral analysis is plausible but they do not deeply compare against the strongest recent VFL/XGBoost DP work, so the related-work positioning could be sharpened.\n\nWho is this for? People working on privacy-preserving tree boosting or VFL more broadly. The null-space idea has legs beyond this specific protocol. I would not cite it in its current form because the privacy theorem's scope is unsupported, but I would pay attention to a revised version. This deserves peer review, not desk rejection, because the core idea is solid and the flaws are identifiable and fixable. Send it out with a request for major revision.","headline":"Null-space noise trick for VFL XGBoost is genuinely novel and the utility results are strong, but the bilateral DP guarantee only covers intermediate messages, not the published split structure, so the main privacy claim is not yet established.","tokens_in":37212,"tokens_out":1139,"would_cite":false,"duration_ms":13823,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MaskedXGBoost claims a bilateral differential privacy guarantee for vertical federated XGBoost by injecting noise that mostly cancels in the split score.","keywords":["vertical federated learning","XGBoost","gradient boosting decision trees","differential privacy","local differential privacy","noise calibration","split finding","bilateral privacy"],"falsifier":"Train MaskedXGBoost to completion while recording every revealed split, then run the paper's attribute-inference attacker on the union of the final tree structure and the noise matrices; if any splitting vector is recovered noticeably better than the $(\\varepsilon_{\\mathrm{PP}},\\delta_{\\mathrm{PP}})$ bound permits, the per-message guarantee does not compose to the full protocol.","tokens_in":36236,"feed_emoji":"🔐","tokens_out":9790,"duration_ms":89054,"temperature":0.7,"pith_summary":"MaskedXGBoost is a protocol for training XGBoost on vertically partitioned data, where one party holds the labels and gradients and the other holds raw features, without exchanging plaintext. The paper's central claim is that both parties can get a formal differential privacy guarantee at the same time: the label-holding active party's noised gradient and Hessian vectors satisfy local differential privacy, and the feature-holding passive party's noise matrices satisfy differential privacy. The trick is to build the noise so that its dominant components lie in the null space of the inner product $m_i^\\top(\\cdot)$ that XGBoost uses to score a candidate split, leaving only a small disturbing component to degrade accuracy. If the claim holds, vertical federated XGBoost can have provable two-sided privacy with training times 4.82 to 6.72 times faster than homomorphic-encryption baselines and better utility than generic Gaussian-noise local DP.","feed_headline":"Null-space noise gives vertical federated XGBoost bilateral privacy","feed_subtitle":"Most of the noise cancels out of the split score, so both parties get differential privacy at far lower cost than encryption.","key_machinery":"The load-bearing object is the calibrated noise matrix $B_i=[b_{i1},\\ldots,b_{iW}]$ built from three ingredients: active noise, inactive noise, and disturbing noise. For a candidate splitting vector $m_i$, the active and inactive components are constructed so that $m_i^\\top u_{ij}=m_i^\\top v_{ij}=0$, meaning they sit exactly in the null space of the aggregation operation $m_i^\\top(\\cdot)$ that XGBoost uses to compute left-node sums; the disturbing component is ordinary Gaussian noise spread over all coordinates. When the active party forms $\\langle g\\rangle_i=g+\\sum_{k=1}^W c_{ik}b_{ik}$ and the analogous $\\langle h\\rangle_i$, only the $r_{ik}$ terms survive in $m_i^\\top\\langle g\\rangle_i$, so utility loss concentrates around the variance $\\kappa^2=n_{A,i}C\\sigma_2^2$. The same decomposition drives the two-sided privacy accounting: the active party's guarantee depends on total noise energy, while the passive party's guarantee depends on the energy ratio of the null-space components to the disturbing component.","core_discovery":"The discovery is that XGBoost's split evaluation, which is the dot product of a binary splitting vector $m_i$ with the private gradient and Hessian vectors $g$ and $h$, can be masked with Gaussian noise that is mostly invisible to that product. For each candidate split $i$, the passive party draws noise vectors $b_{ij}=u_{ij}+v_{ij}+r_{ij}$: active noise $u_{ij}$ is supported on the active set and sums to zero, inactive noise $v_{ij}$ is supported on the inactive set, and both satisfy $m_i^\\top u_{ij}=m_i^\\top v_{ij}=0$; only the disturbing noise $r_{ij}$ moves the score. The active party weights these vectors by coefficients of total energy $C$ and adds them to $g$ and $h$ to form $\\langle g\\rangle_i$ and $\\langle h\\rangle_i$. Theorem 1 bounds the deviation of the noised split score from the true score by a concentration bound that vanishes with the disturbing-noise scale, Theorem 2 gives $(\\varepsilon_{\\mathrm{AP}},\\delta_{\\mathrm{AP}})$-local differential privacy for the noised messages using the total noise energy, and Theorem 3 gives $(\\varepsilon_{\\mathrm{PP}},\\delta_{\\mathrm{PP}})$-differential privacy for the noise matrices, with leakage controlled mainly by the ratio $\\sigma_1^2/\\sigma_2^2$ and the dataset size. The paper presents this as the first bilateral privacy analysis for vertical federated XGBoost and reports experiments on six datasets supporting the utility and efficiency claims.","pith_inferences":["The null-space construction is a transferable recipe: any learning algorithm whose per-candidate statistic is a linear projection of sensitive data could place most of its DP noise in the kernel of that projection, and MaskedXGBoost is one concrete instance of that pattern.","The theorems are per-message guarantees, not an end-to-end guarantee: the passive party reveals each chosen split operation (feature and threshold) to the active party at every node, and that output is not included in Theorems 2 and 3, so the full training transcript may leak more than the stated budgets account for.","One testable simplification is to reuse a single batch of null-space noise vectors across all candidate splits instead of generating one noise matrix per split, which would cut communication if the privacy accounting can still be performed."],"forward_implications":["For the same privacy budget, MaskedXGBoost reports higher and more stable test AUC than FedXGBoost-LDP and XGBoost-LDP across Credit 1, Adult, Higgs, Bank, Credit 2, and Nomao, approaching the non-private baseline as the budget grows.","Average training time per tree is 4.82 times (Adult) to 6.72 times (Nomao) faster than the HE-XGBoost baseline, because noising and matrix operations replace ciphertext operations.","The passive party's privacy leakage for fixed $\\varepsilon_{\\mathrm{PP}}$ increases with the noise ratio $\\sigma_1^2/\\sigma_2^2$, giving the protocol an explicit knob that trades PP privacy against utility.","The same noising recipe adapts to GBDT by applying it to gradients alone, since GBDT's split score uses only first derivatives.","Communication overhead sits between plain DP methods and HE methods (about 1.2 MB versus 16 KB and 2.1 MB in the paper's illustrative configuration), which is the price of bilateral privacy."],"supporting_citations":[{"why":"supplies the XGBoost split-score objective and tree-learning procedure that the protocol is built on.","marker":"[4]"},{"why":"defines the SecureBoost homomorphic-encryption baseline and the label-leakage risk that motivates masking gradients and Hessians.","marker":"[6]"},{"why":"introduces the splitting-vector reformulation of XGBoost aggregations as $M^\\top g$ and $M^\\top h$, and provides the FedXGBoost-LDP baseline.","marker":"[30]"},{"why":"provides the definition of differential privacy and the sequential and parallel composition lemmas used in the privacy analysis.","marker":"[14]"},{"why":"provides the local differential privacy definition relied on by Theorem 2.","marker":"[16]"},{"why":"supplies the advanced composition theorem used in the experiments to convert per-step budgets into the reported $\\varepsilon_{\\mathrm{AP}}$.","marker":"[15]"},{"why":"defines the FederBoost baseline and the matching privacy notion for the passive party's feature ordering.","marker":"[44]"},{"why":"supplies the gradient-clipping technique used to bound the sensitivity of gradients and Hessians in the LDP analysis.","marker":"[1]"}],"fun_headline_variants":["Null-space noise gives vertical federated XGBoost bilateral DP","Bilateral DP for vertical federated XGBoost via null-space noise","Null-space noise cancels in splits, giving bilateral DP for XGBoost","MaskedXGBoost: bilateral DP with noise that cancels in split scores","Federated XGBoost: null-space noise yields bilateral DP cheaply"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The bilateral guarantee covers only the transmitted noise matrices and noised gradient and Hessian vectors, not the split choices that the passive party reveals at every node, so the end-to-end privacy of the completed tree is not established by Theorems 2 and 3.","fun_headline_variants_meta":{"raw":{"variants":["Null-space noise gives vertical federated XGBoost bilateral DP","Bilateral DP for vertical federated XGBoost via null-space noise","Null-space noise cancels in splits, giving bilateral DP for XGBoost","MaskedXGBoost: bilateral DP with noise that cancels in split scores","Federated XGBoost: null-space noise yields bilateral DP cheaply"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000827,"raw_usage":{"total_tokens":3688,"prompt_tokens":1091,"completion_tokens":2597,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":707,"completion_tokens_details":{"reasoning_tokens":2494}},"tokens_in":707,"tokens_out":2597,"duration_ms":16371,"temperature":1.0,"reasoning_tokens":2494,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:55:01.608984+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MaskedXGBoost to completion while recording every revealed split, then run the paper's attribute-inference attacker on the union of the final tree structure and the noise matrices; if any splitting vector is recovered noticeably better than the $(\\varepsilon_{\\mathrm{PP}},\\delta_{\\mathrm{PP}})$ bound permits, the per-message guarantee does not compose to the full protocol.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the XGBoost split-score objective and tree-learning procedure that the protocol is built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the local differential privacy definition relied on by Theorem 2."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the FederBoost baseline and the matching privacy notion for the passive party's feature ordering."}],"review_version":1}