Online Gradient Boosting Decision Tree: In-Place Updates for Efficient Adding/Deleting Data

Huawei Lin; Jun Woo Chung; Weijie Zhao; Yingjie Lao

REVIEW 3 major objections 5 minor 22 references

Online Gradient Boosting Decision Tree: In-Place Updates for Efficient Adding/Deleting Data

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A trained GBDT can accept new rows or forget old ones in place, keeping the same tree count and accuracy close to retraining from scratch.

desk verdict A genuinely useful GBDT incremental/decremental learning system with broad, convincing experiments, but the theory is heuristic and the lazy-derivative approximation deserves a caveat and better measurement. read the letter →

arxiv 2502.01634 v1 pith:U7W7KYHW submitted 2025-02-03 cs.LG cs.AIcs.CRstat.ML

Huawei Lin , Jun Woo Chung , Yingjie Lao , Weijie Zhao This is my paper · ORCID

classification cs.LGcs.AIcs.CRstat.ML

keywords onlinelearninggradientboostingdecisiontreesincrementaldecrementaldatadeletionsplitcandidatesamplinglazyderivativeupdatemachineunlearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Gradient-boosted decision trees (GBDTs) are usually batch-only: once trained, they cannot absorb new rows or forget old ones without rebuilding the whole ensemble from scratch, which is slow on large data. This paper claims that a well-trained GBDT can instead be updated in place, adding a new batch $D_{in}$ or removing a subset $D_{de}$ while leaving the number of trees and their parameter count unchanged. The intended target is the model a full retrain on $D_{tr}$ plus $D_{in}$ minus $D_{de}$ would produce, and the framework is designed to get close to that model at a cost that scales with the small changed batch rather than the whole dataset. The paper also identifies three interacting controls—the fraction of data being added or removed, the split-candidate sampling rate, and a split-robustness tolerance—and argues that they trade accuracy against update cost in a predictable way. If correct, this makes frequent small updates and targeted data deletion practical for GBDT deployments.

What carries the argument

The load-bearing object is the weighted squared-error gain of a candidate split, $Gain(s)$ from Eq. (5): the squared sum of first derivatives divided by the sum of second derivatives for the left child, plus the same for the right child, minus the parent term. Because this expression uses only per-node sums, the framework stores, for every candidate split, the aggregate $S_{rp}$ (sum of residuals) and $S_{pp}$ (sum of second derivatives) during training and refreshes them from the online batch's rows, avoiding a scan of $D_{tr}$. Around this core sit three optimizations: split-candidate sampling (only a fraction $\alpha$ of the $B$ discretized split points are considered, so the expected gap between the best split and its nearest competitor grows as $1/\alpha$), the adaptive split robustness tolerance (a split is kept when it stays in the top $\sigma$ fraction of candidates, with $\sigma$ around 0.1 recommended), and the lazy derivative update (leaf scores and gradients are recomputed only for leaves reached by changed data). The named robustness conditions—Distance Robust, which bounds the nearest alternative split by a gain-ratio inequality, and Robustness Split, which requires $Gain(s) > Gain(t)/(1-\lambda)$—are what the proofs use to justify keeping old splits.

What would settle it

Take a trained GBDT and remove a batch $D_{de}$ of size 0.1% engineered so that at some node two candidate splits have nearly equal gain, then recompute every split's gain from the full remaining data. If the best split changes even though the Distance Robust inequality predicted it would be kept, the linear-perturbation premise is falsified; alternatively, run an add-then-delete cycle and compare leaf scores to a retrain-from-scratch model, where growing divergence of the lazy derivative update would show accumulated drift.

Watch

Extended reading notes

Core claim

The paper's central claim is that incremental and decremental learning on GBDT can be unified as the same in-place operation: traverse each existing tree from root to leaves, recompute the best split gain using only the added or removed rows, keep the split when it remains optimal or within tolerance, and retrain only the sub-tree rooted at nodes whose best split actually shifts. Deletion is treated as the inverse of addition with the same code path. The argument is carried by the structure of the split gain in Eq. (5), which decomposes into per-node sums of first derivatives (residuals) and second derivatives (hessians); the framework stores those sums for every candidate split during training, so an online update only needs to add or subtract the changed rows' contributions. The paper reports that on public datasets this yields models whose predictions match a retrain-from-scratch model in most cases above roughly 98% functional similarity for small batches, while cutting update time to a small fraction of retraining, with backdoor experiments demonstrating that a trigger learned by incremental learning is erased again by decremental learning.

Load-bearing premise

The central load-bearing premise is a linear-perturbation approximation in the Appendix D proofs: adding or removing a small fraction $\lambda$ of data is treated as shifting split gains approximately linearly, with the nearest alternative split expected to sit at distance $1/\alpha$; the Distance Robust proof relies on this approximation, and the derivation in Eq. (11) contains a sign inconsistency. If this linearization fails, the framework either keeps a stale split (accuracy loss) or retrains too often (speed loss).

Editorial extensions

If this is right

Adding or deleting 0.1%–1% of the training data costs $O(|D'|)$ rather than $O(|D_{tr}|)$, so frequent small updates become practical on large datasets.
Tree count and parameter count stay fixed, so online updates do not inflate model size or inference latency.
Continual batch addition and removal (for example 5% → 100% → 5% of the data) tracks retrain-from-scratch accuracy on the reported datasets.
Backdoor triggers inserted by incremental learning are removed by decremental learning, with attack success rate falling back to clean-model levels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the gain function decomposes into per-node sums, the same in-place strategy may transfer to other additive tree ensembles, such as random forests or arbitrary split-gain trees, though the paper does not claim this.
The robustness-tolerance analysis implies a per-dataset calibration procedure: measure how often split rankings permute on a small holdout and set $\alpha$ and $\sigma$ accordingly, rather than using fixed defaults.
A stress test the paper does not report is repeated add/delete cycling; the lazy derivative update could accumulate drift over many small perturbations, so comparing leaf scores after many cycles against a retrained model would bound this effect.
The membership-inference experiment positions decremental learning as a privacy operation, but the framework provides no certified-unlearning guarantee, so a formal audit bound would be needed before relying on it for regulated data deletion.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The short version: this is a real systems contribution—unified in-place incremental and decremental learning for GBDT with the same model size—and the empirical work is broad enough that it deserves a serious referee. The theoretical wrapping is thinner than claimed, and one load-bearing approximation, the lazy derivative refresh, is only empirically justified.

What's new: prior work handles only one direction, iGBDT for adding, DeltaBoost and the authors' own MUinGBDT for deletion. This paper unifies both in a single framework, stores split statistics to avoid touching the training set, and adds adaptive lazy updates and a split robustness tolerance. The experiments are the strong point: ten public datasets, updates ranging from a single point to 1% of the data, backdoor injection/removal, membership inference, time series, high-dimensional data, and ablations over the main hyper-parameters. CPU speedups for small updates are dramatic, and test accuracy after update tracks retraining in most cases. The code is released.

Where it gets soft. The theory in Appendix D is heuristic. The Distance Robust and Robustness Split proofs rely on unquantified linear approximations, and Eq. (11) has a sign/denominator typo that the surrounding derivation has to correct. The practical conclusion, that smaller α or λ gives more robust splits, is plausible, but the proof does not establish it rigorously.

The larger concern is the lazy derivative update. When a leaf prediction changes but the split is kept, gradients for rows in that leaf are not refreshed before fitting later trees. The authors acknowledge this in Appendix N, but Table 13 does not measure the actual stale-gradient model: it recomputes leaf scores with the latest residuals. So the reported approximation error is not the error of the deployed model. The end-to-end accuracy tables look good, but they are empirical and dataset-limited, and there are no error bars or multiple seeds. A distribution where stale residuals matter could be a real failure mode.

Memory is also a practical issue: storing per-bin statistics costs 185 GB on RCV1 with 47k features, which limits scope.

Bottom line: the core idea is a sound heuristic and the evaluation is substantial. If the authors relabel the theory as heuristics, add variance reporting, and either bound or better characterize the stale-gradient effect, this would be a solid accept. I'd send it to review with a request for major revision on those points.

Read this if you work on GBDT systems, machine unlearning, or online learning for tabular data.

Referee Report

3 major / 5 minor

Summary. The paper proposes an in-place online learning framework for Gradient Boosting Decision Trees (GBDT) that supports both incremental (adding data) and decremental (removing data) learning without changing the number of parameters or trees. The core algorithm (Algorithm 3) traverses existing tree nodes, recomputes the best split using only the online dataset D′ and stored aggregate statistics, keeps splits that remain best, retrains only sub-trees whose best split changed, and updates leaf prediction values. To reduce cost, the authors introduce three optimizations: split-candidate sampling, an adaptive lazy update for derivatives (refreshing residual/hessian values only for retrained sub-trees), and an adaptive split-robustness tolerance. They provide a theoretical analysis (Appendix D) relating the sampling rate α, the online-data fraction λ, and a robustness tolerance σ to the probability of retaining splits, and they report extensive experiments on 10 public datasets plus high-dimensional and time-series data, including backdoor attack and membership-inference tests. The central claim is that adding or deleting a small data fraction can be done in O(|D′|) time rather than O(|Dtr|) retraining, with accuracy close to a full retrain.

Significance. If the central claim holds, the paper addresses a practically important gap: GBDT models are widely deployed, yet standard implementations cannot efficiently add or remove training instances. The proposed framework is one of the first to handle incremental and decremental learning in a unified in-place manner, and the empirical study is unusually broad: it includes large datasets (SUSY, HIGGS, Covtype), high-dimensional datasets (RCV1, News20), continual batch addition/removal, unseen-class expansion, and security-oriented evaluations (backdoor injection/removal and membership inference). The authors also release an open-source implementation, which strengthens reproducibility. The main caveats are that the theoretical justification in Appendix D contains unquantified linear approximations and a sign inconsistency, and that the adaptive lazy update is not covered by that theory; the claimed accuracy-cost trade-off therefore rests primarily on empirical evidence. The results are plausible and potentially useful, but the paper needs revision before the central claims can be considered fully supported.

major comments (3)

[Section 3.2, Appendix N, Table 13] The adaptive lazy update is the load-bearing approximation for the O(|D′|) claim, but the paper never quantifies how much accuracy is lost by using outdated derivatives. Algorithm 3 line 9 updates leaf prediction values whenever any data from D′ reaches a leaf, even if the node's split is unchanged; in standard boosting, changing a leaf score changes the gradient of every training row in that leaf, and those changed gradients should be used when fitting all subsequent trees. The paper instead refreshes derivatives only when a sub-tree is retrained (Appendix N states: 'when a sub-tree requires retraining, the derivatives are updated'), so rows in unchanged-but-updated leaves keep stale residuals. The theoretical results in Appendix D (Distance Robust, Robustness Split) bound split-gain movement under a λ-fraction add/removal; they do not bound the error introduced by stale residuals in later boosting iterations. Table 13, which reports 'approximation error of leaf scores', recomputes leaf values with the latest residual and hessian for both the online-learned and retrained models, so it measures a hypothetical model with fresh leaf scores, not the actual model produced by Algorithm 2. Consequently, Table 13 cannot detect the failure mode of the lazy derivative update. Either add a theorem that bounds the distance between the online-updated model and the retrained model under the lazy update, or provide an experiment that evaluates the actual deployed model (e.g., test error and functional similarity, as in Tables 6 and 8) in settings where splits are unchanged but leaf values are updated, with a breakdown of how many residuals are stale.
[Appendix D, Eqs. (8)-(13)] The proof of Definition 1 (Distance Robust) contains a sign inconsistency. Eq. (10) expands (1 − λ)Gain(s) − Gain(s + NΔ) with a left-child term (1 − NΔ/Nls)(Σg)²/(Σh), but Eq. (11) replaces that factor with (1 + NΔ/Nls) for the left child while keeping (1 − NΔ/Nrs) for the right child. The sign flip is what turns the left-child contribution into a positive term in Eq. (12), and the final inequality in Eq. (13) depends on this flip. The derivation also relies on an unstated linear approximation of the gain under removal (treating removed sums as scaling by λ) and on the assertion E[NΔ] = 1/α without derivation. Since these results are used to justify the split-candidate sampling rate α and the hyper-parameter trade-offs advertised in Section 3.3, the proof must be corrected or the statements demoted to heuristic motivation. Please either fix the algebra and provide the missing assumptions, or explicitly frame the robustness conditions as empirical observations rather than theorems.
[Section 3.4 and Figure 2] The adaptive split robustness tolerance σ is motivated by the empirical observation in Figure 2, which shows that the best split often shifts to the second-best split when |D′| increases. However, the paper does not connect σ to the theoretical conditions in Appendix D; the robustness definitions involve λ and α but not σ. The suggestion that σ ≈ 0.1 is therefore a heuristic, and Figure 10 shows that increasing σ can reduce functional similarity considerably (e.g., the Letter dataset drops by about 5% when σ goes from 0 to 0.5). The abstract claims that the paper 'theoretically show[s] the relationship between the hyper-parameters of the proposed optimizations', but the relationship among α, λ, and σ is only partially established. Please either derive a bound that includes σ or revise the claim to state that the trade-off is demonstrated empirically.

minor comments (5)

[Appendix F, Table 5] The row labeled 'Gain Computing & Split Finding' lists complexity O(αBJσ), which mixes α, B, J, and σ in a way that is not explained in the table footnote. The surrounding text gives the more explicit formula O(J|D′|·Pσ + JαB·(1−Pσ)), and the table should match that expression or define the symbols used.
[Figure captions] The captions for Figures 3 and 5 repeat the phrase 'The impact of tuning data size on the number of retrained nodes for each iteration', but the figures actually plot test accuracy as a function of trained data proportion. These captions should be corrected to describe the accuracy curves.
[Section 2.3, Algorithm 3] In Algorithm 3, line 4 computes the 'best gain' with Eq. (5), but line 5 compares this to the current split s using inequality s′ ≠ s. Since Eq. (5) returns a gain value, the comparison should be between the best split index and s; the wording in the text and algorithm is inconsistent. Clarify whether s′ denotes a gain or a split.
[Section 4.2, Table 3] The speedup values in Table 3 are sometimes below 1 (e.g., SUSY decremental learning at |D′|=1% reports a 0.2× speedup against ThunderGBM on GPU, and several HIGGS rows are below 0.5×). The text claims the method is 'substantially faster than other methods', which is accurate for the incremental/decremental baselines but not for the GPU comparison. Please qualify the claim and discuss when GPU retraining can be faster.
[Appendix D, Eq. (16)] In the Robustness Split proof, the denominator of the second term in Eq. (16) appears to have a missing minus sign: it reads 'Σ_{xi∈rs} hi,k Σ_{xi∈rs∩D′} hi,k' without a subtraction operator. This is likely a typographical error and should be corrected for readability.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-referential benchmark in Appendix N; the central online-GBDT claim is independently validated against retraining and external baselines.

other [Appendix N, Table 13; introduced in Section 4.5]
"Please note that the retrained model has the same structure and split in all nodes of all trees as the model after adding/deleting, and we only update the latest residual and hessian to calculate the latest leaf score."

The 'retrained from scratch' baseline used in the approximation-error metric is defined as the online model's own tree structure with refreshed leaf scores, not as an independently retrained model. Section 4.5 introduces this metric as measuring error 'between the model after addition/deletion and the one retrained from scratch,' but the baseline is constructed from the online model's splits. Consequently, Table 13 isolates only leaf-score staleness and cannot validate the split-retention/retraining decisions that constitute the core of the method. This is a self-referential benchmark, though it is a secondary supporting analysis rather than the central derivation.

full rationale

The central claim—that an in-place, size-preserving GBDT update can closely match retraining on Dtr ∪ Din \ Dde—is tested against independent retraining from scratch and external baselines (XGBoost, LightGBM, CatBoost, ThunderGBM) in Tables 6, 8, and Figure 3, with no fitted constant used to manufacture the reported accuracy. The algorithmic core (Algorithm 3) recomputes split gains from stored sufficient statistics plus the O(|D'|) online data, and retrains subtrees only when the best split changes; this is a direct, self-contained procedure rather than a reduction to its own inputs. The optimizations (split-candidate sampling, robustness tolerance, lazy derivative refresh) are heuristics whose trade-offs are empirically ablated, and the theoretical bounds in Appendix D are approximations but not circular. The paper does cite the authors' own prior MUinGBDT (Lin et al., 2023a) for inherited update concepts, but that self-citation is not load-bearing because the current framework's effectiveness is independently validated against retrained models and external GBDT libraries. The only notable self-referential element is the Appendix N leaf-score metric, which defines its 'retrained' comparator using the online model's own tree structure; this weakens that particular supporting analysis but does not make the central derivation circular.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim relies on stored per-split statistics, linearized gain-change approximations, and a lazy derivative update. The main free parameters are the sampling rate alpha, tolerance sigma, and bin count B, all chosen by hand. No new physical or theoretical entities are introduced.

free parameters (3)

alpha (split sampling rate) = 0.1 (default)
Sampling rate alpha controls the number of split candidates (ceil(alpha*B)); lower alpha stabilizes splits but can reduce accuracy. Chosen by hand; ablations in Appendix O.
sigma (split robustness tolerance) = 0.1 (default)
Robustness tolerance sigma keeps the current split if it remains in the top ceil(sigma*alpha*B) candidates; sigma=0 is exact, sigma=1 never retrains. Recommended value based on Figure 2 observation.
B (max bins for feature discretization) = 1024 (default)
B determines the number of potential split positions per feature. It is a hand-set hyperparameter that affects the granularity of gain computation and the memory footprint of stored split statistics.

assumptions (6)

domain assumption GBDT loss is negative log-likelihood with softmax and Robust LogitBoost updates (Eq. 2-5).
Defines the model family the framework operates on.
domain assumption Deletion data Dde is a subset of Dtr and its per-instance derivatives were stored during training.
Decremental learning subtracts stored gradient sums; without stored derivatives it would need a scan of Dtr.
domain assumption Split gain after add/removal can be obtained by adding/subtracting D' statistics to stored aggregate sums from Dtr (Appendix E).
This is exact if the stored sums are per-candidate-split; it avoids O(Dtr) scans.
ad hoc to paper Gain change is linear in lambda and N_Delta, and E[N_Delta] = 1/alpha for random split sampling.
Underpins Definitions 1 and 2; no error bound is given and Eq. (11) has a sign inconsistency.
ad hoc to paper Derivatives only change for data reaching changed terminal nodes; leaving others untouched preserves closeness to retrain.
This lazy update is the main speed heuristic and can silently diverge if routing changes.
domain assumption Split candidates are sampled uniformly at random at training time.
The E[N_Delta]=1/alpha bound relies on uniform sampling.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Online Gradient Boosting Decision Tree: In-Place Updates for Efficient Adding/Deleting Data." pith.science (2026). https://pith.science/paper/U7W7KYHW

@misc{pith2026250201634,
  author       = {Pith},
  title        = {Pith review of: Online Gradient Boosting Decision Tree: In-Place Updates for Efficient Adding/Deleting Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U7W7KYHW}},
  note         = {Machine review of arXiv:2502.01634}
}

read the original abstract

Gradient Boosting Decision Tree (GBDT) is one of the most popular machine learning models in various applications. However, in the traditional settings, all data should be simultaneously accessed in the training procedure: it does not allow to add or delete any data instances after training. In this paper, we propose an efficient online learning framework for GBDT supporting both incremental and decremental learning. To the best of our knowledge, this is the first work that considers an in-place unified incremental and decremental learning on GBDT. To reduce the learning cost, we present a collection of optimizations for our framework, so that it can add or delete a small fraction of data on the fly. We theoretically show the relationship between the hyper-parameters of the proposed optimizations, which enables trading off accuracy and cost on incremental and decremental learning. The backdoor attack results show that our framework can successfully inject and remove backdoor in a well-trained model using incremental and decremental learning, and the empirical results on public datasets confirm the effectiveness and efficiency of our proposed online learning framework and optimizations.

Figures

Figures reproduced from arXiv: 2502.01634 by the authors.

**Figure 2.** Observation of distance of best split changes. The [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** The impact of tuning data size on the number of [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Feature discretization example. For a feature, all its [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

**Figure 5.** Figure 5: The impact of tuning data size on the number of retrained nodes for each iteration in incremental learning. [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]

**Figure 6.** Figure 6: Different fine-tuning ratio. uses outdated derivatives if the best splits remain unchanged. When the number of add/delete is large enough, almost all nodes in the model will be retrained because their best splits have changed, so the error becomes 0. O. Ablation Study …

**Figure 7.** Figure 7: The impact of sampling rate on time, number of re [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

**Figure 8.** Figure 8: The impact of |D′ | on average learning time in incremental/decremental learning (top/bottom row). 20 40 60 80 100 Iteration 0 200 400 600 800 1000 1200 # Retrained Nodes Adult Add 1 Add 0.1% Add 0.5% Add 1.0% 20 40 60 80 100 Iteration 0 50 100 150 200 250 300 # Retrai…

**Figure 10.** Figure 10: The impact of split robustness tolerance on the learning time, test accuracy, and model functional similarity [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

**Figure 11.** Figure 11: The impact of the # bins and # leaves on the acceleration factor of incremental learning (adding 1 data point). 23 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 16 canonical work pages

[3]

and Yang, J

Cao, Y . and Yang, J. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Secu- rity and Privacy (SP), pp. 463–480, San Jose, CA,

work page 2015
[7]

B., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y ., and He, K

Goyal, P., Doll ´ar, P., Girshick, R. B., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y ., and He, K. Accurate, large minibatch SGD: training imagenet in 1 hour. CoRR, abs/1706.02677,

arXiv
[8]

Similarly, W2C represents the testing instances that are wrongly predicted during retraining but are correctly predicted after decremental learning

C2W refers to the ratio of testing instances that are correctly predicted during retraining but are wrongly predicted after decremental learning. Similarly, W2C represents the testing instances that are wrongly predicted during retraining but are correctly predicted after decremental learning. The W2W column indicates the cases where the two models have d...

work page 2018
[14]

Research on Gender-related Fingerprint Features

Qi, Y ., Li, Y ., Lin, H., Chen, J., and Lei, H. Re- search on gender-related fingerprint features. CoRR, abs/2108.08233, 2021a. Qi, Y ., Lin, H., Li, Y ., and Chen, J. Parameter-free attention in fmri decoding. IEEE Access, 9:48704–48712, 2021b. Rao, H., Shi, X., Rodrigue, A. K., Feng, J., Xia, Y ., Elho- seny, M., Yuan, X., and Gu, L. Feature selection ...

work page Pith review arXiv
[15]

An overview of gradient descent optimization algorithms

Ruder, S. An overview of gradient descent optimization algorithms. CoRR, abs/1609.04747,

arXiv
[17]

K., Chundawat, V

Tarun, A. K., Chundawat, V . S., Mandal, M., and Kankan- halli, M. S. Fast yet effective machine unlearning. CoRR, abs/2111.08947,

arXiv
[18]

Structure aware incremental learning with personalized imitation weights for recommender sys- tems

Wang, Y ., Zhang, Y ., Valkanas, A., Tang, R., Ma, C., Hao, J., and Coates, M. Structure aware incremental learning with personalized imitation weights for recommender sys- tems. In Thirty-Seventh AAAI Conference on Artificial 11 Intelligence, AAAI 2023, Thirty-Fifth Conference on In- novative Applications of Artificial Intelligence, IAAI, pp. 4711–4719, ...

work page 2023
[19]

SecureCut: Federated Gradient Boosting Decision Trees with Efficient Machine Unlearning

Zhang, J., Li, B., Li, J., and Wu, C. Securecut: Federated gradient boosting decision trees with efficient machine unlearning. CoRR, abs/2311.13174,

work page Pith review arXiv

Show all 22 references

[20]

Table 7: Error rate after every on- line learning step

• WebTraffic5: This dataset tracks hourly web requests to a single website over a span of five months. Table 7: Error rate after every on- line learning step. Online Learning StepGlobalTemperatures(×10−3) WebTraffic(×10−3) Initial Train 10% 4.1934 4.0984 Add 10%, Total 20%2.54...

1934
[22]

O. Ablation Study 20 40 60 80 100 Iteration 0 0.5 1 1.5 2 2.5 3 Avg Time (ms/per tree) Pendigits (Incr.) Sampling Rate: 5% Sampling Rate: 10% Sampling Rate: 50% Sampling Rate: 100% 20 40 60 80 100 Iteration 0 0.5 1 1.5 2 2.5 3 Avg Time (ms/per tree) Pendigits (Decr.) Sampling ...

2000
[2001]

Multi-class explainable unlearning for image clas- sification via weight filtering

Poppi, S., Sarto, S., Cornia, M., Baraldi, L., and Cucchiara, R. Multi-class explainable unlearning for image clas- sification via weight filtering. CoRR, abs/2304.02049,

arXiv
[2005]

Mem- bership inference attacks against machine learning mod- els

Shokri, R., Stronati, M., Song, C., and Shmatikov, V . Mem- bership inference attacks against machine learning mod- els. In 2017 IEEE Symposium on Security and Privacy, SP, pp. 3–18, San Jose, CA,

2017
[2007]

T., Huynh, T

Nguyen, T. T., Huynh, T. T., Nguyen, P. L., Liew, A. W., Yin, H., and Nguyen, Q. V . H. A survey of machine unlearning. CoRR, abs/2209.02299,

arXiv
[2009]

Bertsekas, D. P. Incremental gradient, subgradient, and proximal methods for convex optimization: A survey. CoRR, abs/1507.01030,

arXiv
[2010]

and Zhao, W

Li, P. and Zhao, W. Fast abc-boost: A unified framework for selecting the base class in multi-class classification. CoRR, abs/2205.10927, 2022a. Li, P. and Zhao, W. Package for fast abc-boost. CoRR, abs/2207.08770, 2022b. Li, S., Wang, Y ., Li, Y ., and Tan, Y . l-leaks: Membe...

arXiv
[2013]

Incremental support vector learning: Analysis, implementation and applications

Laskov, P., Gehl, C., Kr¨uger, S., and M¨uller, K. Incremental support vector learning: Analysis, implementation and applications. J. Mach. Learn. Res., 7:1909–1936,

1909
[2015]

Membership inference attacks from first prin- ciples

Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., and Tram`er, F. Membership inference attacks from first prin- ciples. In 43rd IEEE Symposium on Security and Privacy, SP, pp. 1897–1914, San Francisco, CA,

1914
[2017]

From N to N+1: multiclass transfer incremental learning

Kuzborskij, I., Orabona, F., and Caputo, B. From N to N+1: multiclass transfer incremental learning. In 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3358–3365, Portland, OR,

2013
[2019]

A., Tram `er, F., Carlini, N., and Pa- pernot, N

Choquette-Choo, C. A., Tram `er, F., Carlini, N., and Pa- pernot, N. Label-only membership inference attacks. In Proceedings of the 38th International Conference on Machine Learning, ICML, volume 139 of Proceedings of Machine Learning Research, pp. 1964–1974, Virtual Event,

1964
[2021]

and Lowd, D

Brophy, J. and Lowd, D. DART: data addition and removal trees. CoRR, abs/2009.05567,

2009 arXiv
[2022]

W., Lao, Y ., and Zhao, W

Lin, H., Chung, J. W., Lao, Y ., and Zhao, W. Machine un- learning in gradient boosting decision trees. In Proceed- ings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 1374–1383, Long Beach, CA, 2023a. Lin, H., Liu, H., Li, Q., and Shen, L....

arXiv
[2023]

V ., Ershov, V ., and Gulin, A

Dorogush, A. V ., Ershov, V ., and Gulin, A. Catboost: gra- dient boosting with categorical features support. CoRR, abs/1810.11363,

arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [3]

and Yang, J

Cao, Y . and Yang, J. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Secu- rity and Privacy (SP), pp. 463–480, San Jose, CA,

work page 2015

[2] [7]

B., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y ., and He, K

Goyal, P., Doll ´ar, P., Girshick, R. B., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y ., and He, K. Accurate, large minibatch SGD: training imagenet in 1 hour. CoRR, abs/1706.02677,

arXiv

[3] [8]

Similarly, W2C represents the testing instances that are wrongly predicted during retraining but are correctly predicted after decremental learning

C2W refers to the ratio of testing instances that are correctly predicted during retraining but are wrongly predicted after decremental learning. Similarly, W2C represents the testing instances that are wrongly predicted during retraining but are correctly predicted after decremental learning. The W2W column indicates the cases where the two models have d...

work page 2018

[4] [14]

Research on Gender-related Fingerprint Features

Qi, Y ., Li, Y ., Lin, H., Chen, J., and Lei, H. Re- search on gender-related fingerprint features. CoRR, abs/2108.08233, 2021a. Qi, Y ., Lin, H., Li, Y ., and Chen, J. Parameter-free attention in fmri decoding. IEEE Access, 9:48704–48712, 2021b. Rao, H., Shi, X., Rodrigue, A. K., Feng, J., Xia, Y ., Elho- seny, M., Yuan, X., and Gu, L. Feature selection ...

work page Pith review arXiv

[5] [15]

An overview of gradient descent optimization algorithms

Ruder, S. An overview of gradient descent optimization algorithms. CoRR, abs/1609.04747,

arXiv

[6] [17]

K., Chundawat, V

Tarun, A. K., Chundawat, V . S., Mandal, M., and Kankan- halli, M. S. Fast yet effective machine unlearning. CoRR, abs/2111.08947,

arXiv

[7] [18]

Structure aware incremental learning with personalized imitation weights for recommender sys- tems

Wang, Y ., Zhang, Y ., Valkanas, A., Tang, R., Ma, C., Hao, J., and Coates, M. Structure aware incremental learning with personalized imitation weights for recommender sys- tems. In Thirty-Seventh AAAI Conference on Artificial 11 Intelligence, AAAI 2023, Thirty-Fifth Conference on In- novative Applications of Artificial Intelligence, IAAI, pp. 4711–4719, ...

work page 2023

[8] [19]

SecureCut: Federated Gradient Boosting Decision Trees with Efficient Machine Unlearning

Zhang, J., Li, B., Li, J., and Wu, C. Securecut: Federated gradient boosting decision trees with efficient machine unlearning. CoRR, abs/2311.13174,

work page Pith review arXiv