Pith. sign in

REVIEW 3 major objections 4 minor 58 references

Targeting Negative Flips in Active Learning using Validation Sets

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Restricting active learning to estimated negative-flip samples improves accuracy, negative flip rate, or both across benchmarks.

desk verdict Useful empirical study with a plausible plug-in, but the negative-flip mechanism is unmeasured and the evaluation has a label-budget confound. read the letter →

arxiv 2411.10896 v1 pith:PWRMKRZU submitted 2024-11-16 cs.LG cs.CV

Ryan Benkert , Mohit Prabhushankar , Ghassan AlRegib This is my paper · ORCID
classification cs.LGcs.CV
keywords negativeflipssoftwareregressionactivelearningvalidationsetacquisitionfunctionmispredictiondetectionenergyscoresubsetselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that active learning should optimize not only accuracy but also negative flips, meaning samples the model predicted correctly before a training update and incorrectly after it. It shows first that negative flip rate and error rate move independently across active learning rounds, so reducing error does not automatically reduce regression. The authors then propose RoSE, a plug-in that restricts any acquisition function to the estimated negative-flip subset of the unlabeled pool. On CIFAR100, CINIC10, and TinyImageNet with seven acquisition functions and two architectures, RoSE matches or improves accuracy, NFR, or both in most rounds. Because validation sets already exist in active learning pipelines, the method adds only a small computational step to a standard workflow.

What carries the argument

RoSE is a two-stage subset estimator followed by an unmodified acquisition step. First, prediction switches between the previous and current model partition the unlabeled pool into the union of positive and negative flips $\mathcal{S}_{PN}$ and the union of both-correct and both-wrong samples $\mathcal{S}_{CW}$, using the identity in Equation 4. Second, an energy-based misprediction detector assigns scores to $\mathcal{S}_{PN}$, and the $k$ samples with the highest scores form the estimated negative-flip subset, with $k = (1-\text{acc}_{\text{val}})|\mathcal{D}_{\text{pool}}|$ read off the validation accuracy. This turns a small labeled validation set into an estimate of the pool's misprediction count and lets an arbitrary acquisition function operate on a smaller, regression-relevant search space.

What would settle it

Take an active learning run where the unlabeled pool labels are available to the experimenter; after each round, compute precision and recall of the estimated negative-flip subset against the true negative flips. If precision and recall are near random, or if replacing the validation-derived $k$ with the true pool misprediction count changes the accuracy and NFR results, the estimation step is not what drives the gains.

Watch

Extended reading notes

Core claim

The central claim is that negative flips are a distinct, addressable failure mode in active learning and that sampling from them can be a win-win. Empirically, restricting an acquisition function to samples that switched prediction between rounds and are then scored as mispredictions improves accuracy in many rounds, for example by about two percent for entropy sampling on TinyImageNet, while also lowering the negative flip rate. The method treats the negative flip subset as a smaller search space for any acquisition function, so the same plug-in works with uncertainty-based, representation-based, and Bayesian acquisition functions. The authors also report that when RoSE lowers accuracy in a particular round, it usually lowers NFR as well, providing a trade-off rather than a pure loss.

Load-bearing premise

The whole method rests on the assumption that the small labeled validation set has the same misprediction rate as the unlabeled pool, so $k=(1-\text{acc}_{\text{val}})|\mathcal{D}_{\text{pool}}|$ counts the pool's true negative flips, and that the energy score ranks negative flips above positive flips within the prediction-switch set.

Editorial extensions

If this is right

  • Wrapping any acquisition function with RoSE yields accuracy and/or NFR gains in most rounds without changing the architecture or training procedure.
  • RoSE's gains grow in later active learning rounds, when the model is more stable; in early rounds it tends to match the baseline.
  • When RoSE does reduce accuracy, as with margin sampling on CINIC10, it simultaneously reduces NFR, allowing users to trade one metric for the other.
  • The gains hold across datasets with different complexity and domain mix, and across both CNN and convmixer architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper does not run is to measure precision and recall of the estimated negative-flip subset against true negative flips; this would separate the contribution of the estimation step from the contribution of the prediction-switch partition alone.
  • The same negative-flip targeting could transfer to other iterative settings such as continual learning or federated learning, where models are updated repeatedly and regression is a known problem; the main obstacle is obtaining a validation set representative of the pool.
  • The method implicitly assumes the validation distribution matches the unlabeled pool; in label-shift or domain-shift scenarios, recalibrating the threshold k with importance weighting would be a natural extension.
  • Combining RoSE with training-time regression-reduction methods such as positive-congruent training could attack negative flips simultaneously through data selection and model optimization, a combination the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies negative flips in active learning: samples that were correctly predicted by the previous model and become mispredicted after retraining on a larger labeled set. The authors make two observations: (i) negative flip rate and accuracy are decoupled, and (ii) restricting the acquisition search pool to negative flips improves accuracy and/or reduces negative flips. They propose RoSE, a plug-in that first restricts the unlabeled pool to prediction-switch samples and then selects a subset of size k=(1-acc_val)|D_pool| using energy-based misprediction scores, where acc_val is the accuracy on a small labeled validation set. Experiments on CIFAR100, CINIC10, and TinyImageNet with seven acquisition functions and two architectures report that RoSE matches or improves the baseline in most rounds, in both accuracy and negative flip rate.

Significance. If the claimed results hold, RoSE would be a useful, model-agnostic plug-in that addresses an underexplored failure mode of active learning, and it would give validation sets a new role beyond early stopping and hyperparameter tuning. The oracle experiment in Section IV-B is a well-designed control that directly supports the premise that sampling from the true negative-flip subset can improve both accuracy and NFR. The paper also provides a broad empirical study across datasets, acquisition functions, and architectures, and the authors release code, which aids reproducibility. However, the significance is conditional on two unresolved issues: the experimental comparison does not hold the label budget equal between RoSE and the baselines, and the proposed estimator is not actually shown to recover negative flips rather than generic mispredictions. These issues directly affect the interpretation of the reported gains.

major comments (3)
  1. [Section VI-A, Tables I-II, Figures 6-8] The label budget is not held equal between RoSE and the baselines. RoSE uses a labeled validation set (1% for CINIC10, 10% for CIFAR100, 5% for TinyImageNet) to compute acc_val and set the threshold k, while the baseline acquisition functions are given no access to these labels. Active learning performance is highly sensitive to the amount of labeled data, so the reported improvements in Table I, Table II, and the learning curves could be partly or entirely due to the extra validation labels rather than to the negative-flip restriction. The authors should either add the validation labels to the baseline training sets, or use the same validation set for all methods (e.g., for early stopping or hyperparameter selection) and explicitly account for its label cost in the comparison.
  2. [Section V-B, Eq. (4)-(5)] The proposed estimator does not specifically target negative flips. Equation (4) defines SPN as all samples whose predicted label changes between rounds, but this includes both-wrong samples whose two wrong predictions differ; such samples are not negative flips. Equation (5) then sets k=(1-acc_val)|D_pool|, which estimates the total number of mispredicted samples in the pool under the validation accuracy, not the number of negative flips. Negative flips are the subset of switches where the old model was correct and the new model is wrong, and their prevalence is not determined by validation accuracy. The paper never measures the overlap between q_phi(D_pool) and the true negative-flip set, so the central mechanism that RoSE 'restricts acquisition functions to negative flips' is unsupported by direct evidence. The authors should report, for at least one dataset and several rounds, the precision and recall of the estimated set against oracle-computed negative flips, and should include an oracle-k baseline that uses the true number of negative flips to separate the effects of subset ordering from subset size.
  3. [Section V-B and Section VI-A, Eq. (5)] The key calibration assumption that the validation misprediction rate equals the pool misprediction rate is untested and load-bearing. Because k is set from acc_val, any distribution shift between the validation set and the unlabeled pool directly biases the subset size; this is a concrete risk for CINIC10, which combines samples from two distinct sources. In addition, the validation fractions (1%, 5%, 10%) are selected per dataset without sensitivity analysis. The authors should verify the calibration by reporting pool accuracy alongside validation accuracy across rounds, and should vary the validation fraction to demonstrate that the method's performance does not hinge on a particular hand-chosen value.
minor comments (4)
  1. [Section VI-C] The formula for the outperformance ratio is stated as Ntotal/NRoSE, but the surrounding text says a value larger than 0.5 indicates RoSE outperforms in the majority of rounds. This only holds for the inverse ratio NRoSE/Ntotal; the formula or the explanation should be corrected.
  2. [Section V-B, Eq. (4)] The set-builder notation in Equation (4) is garbled: 'si = 1 ˜yi new!= ˜yi old = 1' is not a well-formed predicate. The definition of si should be made explicit, for example si = 1 if the predicted label of the new model differs from that of the old model, and the set SPN should be written as {xi in Dpool : si = 1}.
  3. [Section IV-B, Figure 5] The text states that 'the both correct subset has the lowest NFR' and later that the negative-flip subset 'consistently performs among the highest in terms of accuracy while still reducing the NFR.' It would be helpful to include the corresponding numeric values or a table, since the qualitative description alone makes it hard to judge the magnitude of the differences between the subsets.
  4. [Abstract and Section V-B] The abstract says 'integrating a validation set results in a significant performance boost,' but the validation set is only used to compute the scalar acc_val, not as an additional training signal. The wording could be sharpened to avoid implying that the validation labels are added to the training set.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RoSE's derivation is independent of its evaluation metrics.

full rationale

The paper's central chain is: (i) define negative flips via old/new model predictions (Eq. 2); (ii) partition the unlabeled pool into prediction-switch and no-switch sets (Eq. 4), which is an analytic identity from the models' outputs; (iii) estimate the number of mispredicted samples in the switch set as k=(1-acc_val)|Dpool| (Eq. 5), where acc_val is measured on a small validation set; and (iv) select the top-k energy scores (using the external method [52]) and apply an arbitrary acquisition function. No step uses test accuracy or test NFR as an input, so the reported improvements are not forced by construction. The self-citations ([19], [30], etc.) appear in the related-work survey and in the acquisition-function taxonomy, not as premises in the derivation of q_phi; no uniqueness theorem or prior result by the same authors is invoked to rule out alternatives. The threshold k is a hand-set hyperparameter, not a fitted parameter that is later renamed as a prediction. The paper's own Discussion section states that the mechanism 'still requires theoretical justification,' confirming that the empirical gains are presented as observations rather than as definitional equivalences. Any concern that k estimates overall misprediction rate rather than negative-flip prevalence, or that SPN can contain both-wrong samples with different wrong predictions, is a robustness/validity issue for the estimator, not a circularity. The evaluation against baseline acquisition functions on CIFAR100, CINIC10, and TinyImageNet is an external benchmark, and the paper reports cases where RoSE does not improve both metrics, which is inconsistent with a by-construction advantage.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method's load-bearing quantities are the validation-set fraction and the threshold k, both chosen by hand, plus the transfer of an energy score to negative-flip detection. No test-set information is used, so the approach is not circular, but the approximations are unvalidated and the label budget is not held equal to baselines.

free parameters (4)
  • Validation set fraction per dataset = CINIC10: 1%, CIFAR100: 10%, TinyImageNet: 5%
    Chosen per dataset based on size and class complexity in Section VI-A; no sensitivity analysis. Larger validation fractions give RoSE more label information than baselines receive.
  • Negative flip subset size coefficient k = (1 - acc_val) * |D_pool| = k equals misprediction count derived from validation accuracy
    Equation 5; the paper calls phi=acc_val the hyperparameter. This assumes validation accuracy directly sets the number of pool samples estimated as negative flips, with no calibration restricted to prediction-switch samples.
  • Training stopping criterion = Train until 98% training accuracy or 200 epochs
    Section VI-A; this criterion determines the model states between rounds and therefore affects measured negative flips. It is chosen by the authors and not varied.
  • Energy-based misprediction score choice = Energy score from Liu et al. [52]
    Section V-B; only one detector is tested. The choice of score function affects which switch samples are labeled as negative flips.
assumptions (3)
  • ad hoc to paper Validation accuracy is a reliable estimate of the misprediction rate on the unlabeled pool, so k=(1-acc_val)|D_pool| approximates the number of negative flips.
    Section V-B, Equation 5. The authors set the threshold to reflect validation accuracy without a distributional argument connecting validation set and unlabeled pool.
  • domain assumption Among prediction-switch samples, the energy misprediction score ranks negative flips above positive flips.
    Section V-B; an energy-based out-of-distribution detector is transferred to misprediction detection without validating its ranking against actual negative flips.
  • domain assumption Restricting acquisition to estimated negative flips improves accuracy and/or NFR on the test set.
    Supported only by the idealized perfect-knowledge experiment in Section IV-B; the paper admits in Section VII that a theoretical justification is still missing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Targeting Negative Flips in Active Learning using Validation Sets." pith.science (2026). https://pith.science/paper/PWRMKRZU

@misc{pith2026241110896,
  author       = {Pith},
  title        = {Pith review of: Targeting Negative Flips in Active Learning using Validation Sets},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PWRMKRZU}},
  note         = {Machine review of arXiv:2411.10896}
}
read the original abstract

The performance of active learning algorithms can be improved in two ways. The often used and intuitive way is by reducing the overall error rate within the test set. The second way is to ensure that correct predictions are not forgotten when the training set is increased in between rounds. The former is measured by the accuracy of the model and the latter is captured in negative flips between rounds. Negative flips are samples that are correctly predicted when trained with the previous/smaller dataset and incorrectly predicted after additional samples are labeled. In this paper, we discuss improving the performance of active learning algorithms both in terms of prediction accuracy and negative flips. The first observation we make in this paper is that negative flips and overall error rates are decoupled and reducing one does not necessarily imply that the other is reduced. Our observation is important as current active learning algorithms do not consider negative flips directly and implicitly assume the opposite. The second observation is that performing targeted active learning on subsets of the unlabeled pool has a significant impact on the behavior of the active learning algorithm and influences both negative flips and prediction accuracy. We then develop ROSE - a plug-in algorithm that utilizes a small labeled validation set to restrict arbitrary active learning acquisition functions to negative flips within the unlabeled pool. We show that integrating a validation set results in a significant performance boost in terms of accuracy, negative flip rate reduction, or both.

Figures

Figures reproduced from arXiv: 2411.10896 by the authors.

Figure 1
Figure 1. Overview of our method, RoSE, and learning curves of Entropy sampling on CINIC10. a) high-level diagramm of RoSE. b) accuracy and negative flip rate for the CINIC10 dataset. Higher accuracy and lower negative flip rate values are better. gression in active learning which forms the basis of our approach. 2) We develop RoSE, a plug-in algorithm to improve nega￾tive flips and accuracy simultaneously. 3) We perform exte… view at source ↗
Figure 2
Figure 2. Accuracy and NFR curves of three different acquisition functions on the CIFAR10 and CIFAR100 benchmark. The plots consider entropy sampling, [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Class complexity analysis of the NFR. Each plot represents a different [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: NFR on 15 classes in the CIFAR100 dataset. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Random sampling from the restricted unlabeled pool based on the [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Accuracy learning curves on three dataset benchmarks with different acquisition functions. The x-axis shows the labeled training set size. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: NFR learning curves on three dataset benchmarks with different acquisition functions. The x-axis shows the labeled training set size. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Convmixer learning curves for CINIC-10 dataset [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Negative flips on the 2D artificial spiral dataset. Active learning is performed with random sampling with and without restricting the unlabeled [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 33 canonical work pages

  1. [1]

    Positive-Congruent Training: Towards Regression-Free Model Updates

    S. Yan, Y . Xiong, K. Kundu, S. Yang, S. Deng, M. Wang, W. Xia, and S. Soatto, “Positive-congruent training: Towards regression-free model updates,” CoRR, vol. abs/2011.09161, 2020. [Online]. Available: https://arxiv.org/abs/2011.09161

    work page Pith review arXiv 2011

  2. [2]

    CURE-TSR: Challenging unreal and real environments for traffic sign recognition,

    D. Temel, G. Kwon, M. Prabhushankar, and G. AlRegib, “CURE-TSR: Challenging unreal and real environments for traffic sign recognition,” in Neural Information Processing Systems (NeurIPS) Workshop on Machine Learning for Intelligent Transportation Systems , 2017

    work page 2017

  3. [3]

    Active learning with statistical models,

    D. A. Cohn, Z. Ghahramani, and M. I. Jordan, “Active learning with statistical models,” Journal of artificial intelligence research, vol. 4, pp. 129–145, 1996

    work page 1996

  4. [4]

    Tong, Active learning: theory and applications

    S. Tong, Active learning: theory and applications . Stanford University USA, 2001, vol. 1

    work page 2001

  5. [5]

    On learning, representing, and generalizing a task in a humanoid robot,

    S. Calinon, F. Guenter, and A. Billard, “On learning, representing, and generalizing a task in a humanoid robot,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 37, no. 2, pp. 286–298, 2007

    work page 2007

  6. [6]

    Scalable active learning for object detection,

    E. Haussmann, M. Fenzi, K. Chitta, J. Ivanecky, H. Xu, D. Roy, A. Mittel, N. Koumchatzky, C. Farabet, and J. M. Alvarez, “Scalable active learning for object detection,” in 2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020, pp. 1430–1435

    work page 2020

  7. [7]

    Focal: A cost- aware video dataset for active learning,

    K. Kokilepersaud, Y .-Y . Logan, R. Benkert, C. Zhou, M. Prabhushankar, G. AlRegib, E. Corona, K. Singh, and M. Parchami, “Focal: A cost- aware video dataset for active learning,” in 2023 IEEE International Conference on Big Data (BigData) . IEEE, 2023, pp. 1269–1278

    work page 2023

  8. [8]

    Batch mode active learning and its application to medical image classification,

    S. C. Hoi, R. Jin, J. Zhu, and M. R. Lyu, “Batch mode active learning and its application to medical image classification,” inProceedings of the 23rd international conference on Machine learning , 2006, pp. 417–424

    work page 2006

Show all 58 references
  1. [9]

    Patient aware active learning for fine-grained oct classification,

    Y .-y. Logan, R. Benkert, A. Mustafa, G. Kwon, and G. AlRegib, “Patient aware active learning for fine-grained oct classification,” arXiv preprint arXiv:2206.11485, 2022

    2022 arXiv

  2. [10]

    Effective data selection for seismic interpretation through disagreement,

    R. Benkert, M. Prabhushankar, and G. AlRegib, “Effective data selection for seismic interpretation through disagreement,” IEEE Transactions on Geoscience and Remote Sensing , 2024

    2024

  3. [11]

    Clinical trial active learning,

    Z. Fowler, K. P. Kokilepersaud, M. Prabhushankar, and G. AlRegib, “Clinical trial active learning,” in Proceedings of the 14th ACM In- ternational Conference on Bioinformatics, Computational Biology, and Health Informatics, 2023, pp. 1–10

    2023

  4. [12]

    Decal: Deployable clinical active learning,

    Y .-y. Logan, M. Prabhushankar, and G. AlRegib, “Decal: Deployable clinical active learning,” arXiv preprint arXiv:2206.10120 , 2022

    2022 arXiv

  5. [13]

    Generalization and parameter estimation in feedforward nets: Some experiments,

    N. Morgan and H. Bourlard, “Generalization and parameter estimation in feedforward nets: Some experiments,” Advances in neural information processing systems, vol. 2, 1989

    1989

  6. [14]

    SIMILAR: submodular information measures based active learning in realistic scenarios,

    S. Kothawade, N. Beck, K. Killamsetty, and R. K. Iyer, “SIMILAR: submodular information measures based active learning in realistic scenarios,” CoRR, vol. abs/2107.00717, 2021. [Online]. Available: https://arxiv.org/abs/2107.00717

    2021 arXiv

  7. [15]

    Two faces of active learning,

    S. Dasgupta, “Two faces of active learning,” Theoretical computer science, vol. 412, no. 19, pp. 1767–1781, 2011

    2011

  8. [16]

    Active learning literature survey,

    B. Settles, “Active learning literature survey,” 2009

    2009

  9. [17]

    Theory of disagreement-based active learning,

    S. Hanneke et al. , “Theory of disagreement-based active learning,” Foundations and Trends® in Machine Learning , vol. 7, no. 2-3, pp. 131–309, 2014

    2014

  10. [18]

    A new active labeling method for deep learning,

    D. Wang and Y . Shang, “A new active labeling method for deep learning,” in 2014 International joint conference on neural networks (IJCNN). IEEE, 2014, pp. 112–119

    2014

  11. [19]

    Forgetful active learning with switch events: Efficient sampling for out-of-distribution data,

    R. Benkert, M. Prabhushankar, and G. AlRegib, “Forgetful active learning with switch events: Efficient sampling for out-of-distribution data,” in 2022 IEEE International Conference on Image Processing (ICIP). IEEE, oct 2022

    2022

  12. [20]

    Margin-based active learning for structured out- put spaces,

    D. Roth and K. Small, “Margin-based active learning for structured out- put spaces,” in European Conference on Machine Learning . Springer, 2006, pp. 413–424

    2006

  13. [21]

    Less is more: Active learning with support vector machines,

    G. Schohn and D. Cohn, “Less is more: Active learning with support vector machines,” in ICML, vol. 2, no. 4. Citeseer, 2000, p. 6

    2000

  14. [22]

    Transitional un- certainty with layered intermediate predictions,

    R. Benkert, M. Prabhushankar, and G. AlRegib, “Transitional un- certainty with layered intermediate predictions,” arXiv preprint arXiv:2405.17494, 2024

    2024 arXiv

  15. [23]

    Support vector machine active learning with applications to text classification,

    S. Tong and D. Koller, “Support vector machine active learning with applications to text classification,” Journal of machine learning research, vol. 2, no. Nov, pp. 45–66, 2001

    2001

  16. [24]

    The power of ensembles for active learning in image classification,

    W. H. Beluch, T. Genewein, A. N ¨urnberger, and J. M. K ¨ohler, “The power of ensembles for active learning in image classification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    2018

  17. [25]

    Bayesian active learning for classification and preference learning,

    N. Houlsby, F. Husz ´ar, Z. Ghahramani, and M. Lengyel, “Bayesian active learning for classification and preference learning,” arXiv preprint arXiv:1112.5745, 2011

    2011 arXiv

  18. [26]

    Active learning for convolutional neural networks: A core-set approach,

    O. Sener and S. Savarese, “Active learning for convolutional neural networks: A core-set approach,” arXiv preprint arXiv:1708.00489, 2017

    2017 arXiv

  19. [27]

    Deep active learning over the long tail,

    Y . Geifman and R. El-Yaniv, “Deep active learning over the long tail,” CoRR, vol. abs/1711.00941, 2017. [Online]. Available: http://arxiv.org/abs/1711.00941

    2017 arXiv

  20. [28]

    Discriminative active learning,

    D. Gissin and S. Shalev-Shwartz, “Discriminative active learning,” arXiv preprint arXiv:1907.06347, 2019

    1907 arXiv

  21. [29]

    Deep batch active learning by diverse, uncertain gradient lower bounds,

    J. T. Ash, C. Zhang, A. Krishnamurthy, J. Langford, and A. Agar- wal, “Deep batch active learning by diverse, uncertain gradient lower bounds,” arXiv preprint arXiv:1906.03671 , 2019

    1906 arXiv

  22. [30]

    Gaussian switch sampling: A second order approach to active learning,

    R. Benkert, M. Prabhushankar, G. AlRegib, A. Parchami, and E. Corona, “Gaussian switch sampling: A second order approach to active learning,” IEEE Transaction on Artificial Intelligence , 2023

    2023

  23. [31]

    Active learning in bayesian neural networks with balanced entropy learning principle,

    J. O. Woo, “Active learning in bayesian neural networks with balanced entropy learning principle,” arXiv preprint arXiv:2105.14559 , 2021

    2021 arXiv

  24. [32]

    Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning,

    A. Kirsch, J. van Amersfoort, and Y . Gal, “Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning,” CoRR, vol. abs/1906.08158, 2019. [Online]. Available: http://arxiv.org/abs/1906. 08158

    1906 arXiv

  25. [33]

    A simple baseline for batch active learning with stochastic acquisition functions,

    A. Kirsch, S. Farquhar, and Y . Gal, “A simple baseline for batch active learning with stochastic acquisition functions,” CoRR, vol. abs/2106.12059, 2021. [Online]. Available: https://arxiv.org/abs/2106. 12059

    2021 arXiv

  26. [34]

    Lifelong machine learning,

    Z. Chen and B. Liu, “Lifelong machine learning,” Synthesis Lectures on Artificial Intelligence and Machine Learning , vol. 12, no. 3, pp. 1–207, 2018

    2018

  27. [36]

    Online structured laplace approxi- mations for overcoming catastrophic forgetting,

    H. Ritter, A. Botev, and D. Barber, “Online structured laplace approxi- mations for overcoming catastrophic forgetting,” in Advances in Neural Information Processing Systems , 2018, pp. 3738–3748

    2018

  28. [37]

    An empirical study of example forgetting during deep neural network learning,

    M. Toneva, A. Sordoni, R. T. d. Combes, A. Trischler, Y . Bengio, and G. J. Gordon, “An empirical study of example forgetting during deep neural network learning,” arXiv preprint arXiv:1812.05159 , 2018

    2018 arXiv

  29. [38]

    Gdumb: A simple approach that questions our progress in continual learning,

    A. Prabhu, P. H. Torr, and P. K. Dokania, “Gdumb: A simple approach that questions our progress in continual learning,” in European confer- ence on computer vision . Springer, 2020, pp. 524–540

    2020

  30. [39]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 12, pp. 2935– 2947, 2018

    2018

  31. [40]

    Towards open set deep networks,

    A. Bendale and T. E. Boult, “Towards open set deep networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1563–1572

    2016

  32. [41]

    Toward open set recognition,

    W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult, “Toward open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 35, no. 7, pp. 1757–1772, 2013

    2013

  33. [42]

    Open-set recognition with gradient-based representations,

    J. Lee and G. AlRegib, “Open-set recognition with gradient-based representations,” in 2021 IEEE International Conference on Image Processing (ICIP). IEEE, 2021, pp. 469–473

    2021

  34. [43]

    Backprop- agated gradient representations for anomaly detection,

    G. Kwon, M. Prabhushankar, D. Temel, and G. AlRegib, “Backprop- agated gradient representations for anomaly detection,” in European Conference on Computer Vision . Springer, 2020, pp. 206–226

    2020

  35. [44]

    Memory aware synapses: Learning what (not) to forget,

    R. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, and T. Tuytelaars, “Memory aware synapses: Learning what (not) to forget,” inProceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 139–154

    2018

  36. [45]

    Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima,

    G. Shi, J. Chen, W. Zhang, L.-M. Zhan, and X.-M. Wu, “Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima,” Advances in Neural Information Processing Systems , vol. 34, 2021

    2021

  37. [46]

    Explainable seismic neural networks using learning statistics,

    R. Benkert, O. Joseph Aribido, and G. AlRegib, “Explainable seismic neural networks using learning statistics,” in First International Meeting for Applied Geoscience & Energy . Society of Exploration Geophysi- cists, 2021, pp. 1425–1429

    2021

  38. [47]

    Explaining deep models through forgettable learning dynamics,

    R. Benkert, O. J. Aribido, and G. AlRegib, “Explaining deep models through forgettable learning dynamics,” in 2021 IEEE International Conference on Image Processing (ICIP) . IEEE, 2021, pp. 3692–3696

    2021

  39. [48]

    Example forgetting: A novel approach to explain and interpret deep neural networks in seismic interpretation,

    ——, “Example forgetting: A novel approach to explain and interpret deep neural networks in seismic interpretation,” IEEE Transactions on Geoscience and Remote Sensing , 2022

    2022

  40. [49]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, J. Dean et al. , “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , vol. 2, no. 7, 2015

    2015 arXiv

  41. [50]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning,

    Y . Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in international conference on machine learning . PMLR, 2016, pp. 1050–1059

    2016

  42. [51]

    Simple and scalable predictive uncertainty estimation using deep ensembles,

    B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” Advances in neural information processing systems , vol. 30, 2017

    2017

  43. [52]

    Energy-based out-of- distribution detection,

    W. Liu, X. Wang, J. D. Owens, and Y . Li, “Energy-based out-of- distribution detection,” 2021

    2021

  44. [53]

    Deep bayesian active learning with image data,

    Y . Gal, R. Islam, and Z. Ghahramani, “Deep bayesian active learning with image data,” in International Conference on Machine Learning . PMLR, 2017, pp. 1183–1192

    2017

  45. [54]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009

    2009

  46. [56]

    Tiny imagenet visual recognition challenge,

    Y . Le and X. Yang, “Tiny imagenet visual recognition challenge,” CS 231N, vol. 7, no. 7, p. 3, 2015

    2015

  47. [57]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  48. [58]

    Patches are all you need?

    A. Trockman and J. Z. Kolter, “Patches are all you need?” arXiv preprint arXiv:2201.09792, 2022

    2022 arXiv

  49. [2016]

    Available: http://arxiv.org/abs/1612.00796

    [Online]. Available: http://arxiv.org/abs/1612.00796

    arXiv

  50. [2018]

    Available: http://arxiv.org/abs/1810.03505

    [Online]. Available: http://arxiv.org/abs/1810.03505

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.