Pith. sign in

REVIEW 4 major objections 5 minor 130 references

Quantifying the Prediction Uncertainty of Machine Learning Models for Individual Data

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a per-input regret score—the pNML regret—quantifies when a model's prediction can be trusted, with closed-form expressions for linear regression and neural networks.

desk verdict A PhD thesis with genuinely new linear-regression pNML theory, but the neural-network confidence claims rest on an approximate update that is not the true min-max solution; worth a careful referee if Chapter 5 is reframed. read the letter →

arxiv 2412.07520 v1 pith:XLONDL3C submitted 2024-12-10 cs.LG cs.ITmath.IT

Koby Bibas This is my paper · ORCID
classification cs.LGcs.ITmath.IT MSC 62J0568T0794A15
keywords predictivenormalizedmaximumlikelihoodindividualsettingconfidenceestimationout-of-distributiondetectionlinearregressionneuralnetworksactivelearningadversarialrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This thesis tries to establish a single distribution-free measure of prediction confidence that works for a specific test input, not just on average over a test distribution. That measure is the predictive normalized maximum likelihood (pNML) regret, which compares the learner to a 'genie' that knows the true test label but must still use the given hypothesis class. The paper derives closed forms for linear regression and for a single-layer neural network, and then uses the regret as a confidence score for out-of-distribution detection, adversarial defense, and active learning. A sympathetic reader would take the central claim to be that a model can say 'I do not know' about an individual example, with the mathematical backing of a min-max optimality criterion that makes no distributional assumptions.

What carries the argument

The load-bearing object is the pNML learner and its regret. For a hypothesis class $p_\theta(y|x)$ and a training set $D_N$, the pNML probability assignment is $q_{\text{pNML}}(y|x) = p_{\hat\theta(D_N;x,y)}(y|x) / \sum_{y'} p_{\hat\theta(D_N;x,y')}(y'|x)$, and its min-max regret is the logarithm of the normalization factor, $\Gamma = \log \sum_{y'} p_{\hat\theta(D_N;x,y')}(y'|x)$. For linear regression the genie update is solved with recursive least squares, producing the closed form $\Gamma = \log(1 + x^\top(X_N^\top X_N)^{-1}x)$. For neural networks the paper uses a recursive update rule for the weights to approximate the genie, leading to the single-layer regret formula and to the quantity $x^\top g$ that measures alignment with the training data subspace. This single regret quantity carries all three applications: thresholding it detects unexpected inputs, refining an adversarial example toward each hypothesized label yields a defense, and minimizing the worst-case regret over the test set selects which unlabeled sample to query.

What would settle it

Train a small single-layer softmax network on a low-dimensional dataset where exact retraining is feasible, enumerate every possible test label, retrain the last layer from scratch for each label, compute the exact pNML normalization factor, and compare it with formula (5.11); a large discrepancy on inputs with $x^\top g$ near 1 would show that the approximate update does not deliver the claimed regret.

Watch

Extended reading notes

Core claim

The central claim is that the pNML regret, computed from the training set and a single test input, is a per-instance learnability and confidence measure. For linear regression the regret is exact: the pNML predictive distribution is Gaussian with the same mean as the empirical risk minimizer, and its variance is inflated by a factor whose logarithm is Γ = $\log(1 + x^\top(X_N^\top X_N)^{-1}x)$; low regret occurs when the test input lies mostly in the subspace spanned by the large eigenvectors of the training correlation matrix. For a single-layer softmax neural network the paper derives Γ = $\log \sum_{i=1}^C \frac{p_i}{p_i + p_i^{x^\top g}(1-p_i)}$, where $p_i$ is the network's probability for class $i$ and $x^\top g$ measures how aligned the test input is with the training data subspace. Low regret is shown to occur when the test input aligns with large-eigenvalue directions or sits far from the decision boundary. Applied to the last layer of a pretrained deep network, this regret is then used as a confidence score for detecting out-of-distribution samples, as a defense mechanism against adversarial perturbations, and as an active-learning acquisition criterion.

Load-bearing premise

The neural-network regret formula depends on an iterative update rule that the paper itself says does not compute the exact solution for the updated dataset, so the derived regret is not literally the min-max regret for the full network hypothesis class.

Editorial extensions

If this is right

  • A pretrained classifier can gain an out-of-distribution detector without retraining or extra data, simply by computing the pNML regret on the last layer's embeddings.
  • The linear-regression regret gives a per-point generalization measure that explains why over-parameterized models can still generalize when test inputs lie in the 'learnable space' of large training eigenvalues.
  • The luckiness-augmented pNML shifts ridge-regression predictions toward zero outside the training support, which the paper shows improves distribution-shift performance.
  • The active-learning criterion selects samples that minimize worst-case regret on the unlabeled test set, and in the presence of out-of-distribution pool samples it needs fewer labeled examples to reach a given accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the regret depends only on the training set and the single test input, it could serve as a plug-in calibration or misclassification-detection score beyond the OOD task the paper evaluates; the paper does not test that use directly.
  • The neural-network regret formula rests on an iterative update that the paper explicitly says is not the exact retrained solution, so a cheap exactness check on small networks—comparing formula (5.11) against full leave-one-out retraining—would show how large the gap is in practice.
  • The luckiness mechanism suggests that replacing the Gaussian prior with other priors, such as a Laplace prior for $\ell^1$ regularization, would yield analogous shift-toward-zero predictions under distribution shift, extending the same framework beyond ridge regression.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This PhD thesis studies the predictive normalized maximum likelihood (pNML) solution for the individual setting, where no distributional assumption links training and test data. It derives exact pNML regret formulas for ordinary linear regression and for regularized/luckiness-based variants, gives an upper bound for over-parameterized linear regression, and then proposes pNML-based confidence scores for neural network last layers, out-of-distribution detection, adversarial defense, and active learning. The central claim is that the per-instance regret, computed from the training set and the single test input, quantifies learnability and confidence, and that this measure improves OOD detection, adversarial robustness, and active learning. The linear-regression parts are largely self-contained and analytically clean; the neural-network part is where the pNML interpretation breaks down.

Significance. If the central claims held, the thesis would offer a distribution-free, per-instance uncertainty measure for modern ML models, with direct practical value for OOD detection and active learning. The linear-regression derivations in Chapters 2-4 are a genuine strength: the regret formula Γ = log(1 + xᵀ(X_NᵀX_N)⁻¹x) follows from Gaussian integration, the learnable-space eigenvector analysis is insightful, and the LpNML extension is a nontrivial and well-executed idea. The experimental breadth is also substantial, covering multiple architectures, datasets, and metrics. However, the neural-network confidence measure, which is the headline application, is currently not a min-max regret for the stated hypothesis class, so the theoretical backing for the OOD and active-learning claims is missing. The empirical results may still be useful, but they need to be reframed as heuristic and supported by ablations that isolate the source of the gains.

major comments (4)
  1. [Section 5.2, Theorem 5.2.2 / Eq. (5.11)] The claimed pNML regret for a single-layer softmax neural network is not the min-max regret for that hypothesis class. Eq. (5.3) sets θ̂_N = X_N⁺ f⁻¹(Y_N), which is only a closed-form least-squares solution when the activation f is linear and invertible; for a softmax layer, the inverse activation in Eq. (5.7) contains the unknown log-sum-exp term, and applying f⁻¹ to one-hot labels is not well-defined. The recursive update in Eq. (5.5), taken from Zhuang et al., is, as the text itself states one paragraph after Eq. (5.5), an iterative step rather than the exact ERM solution on the augmented dataset. Consequently, the genie probabilities in Lemma 5.2.1 and the sum in Eq. (5.11) do not correspond to the pNML genie for the stated model class, and the min-max guarantee does not follow. The OOD detection application in Tables 5.1-5.4 therefore lacks the claimed theoretical backing. Please either derive the exact pNML for a hypothesis class on which the update is exact, or present Eq. (5.11) explicitly as an approximation/heuristic and remove the optimality claims.
  2. [Section 5.2, Deep neural network adaptation] The L2 normalization of all embeddings is introduced as an empirical observation, not derived from the pNML principle, and the regret in Eq. (5.11) is scale-dependent. Normalizing the test input changes xᵀg and hence Γ, so the reported AUROC gains in Tables 5.1-5.4 may be due to this normalization or to the fine-tuning in Section 5.3 rather than to the pNML score. Please add ablations: (i) pNML score with and without normalization, (ii) normalization applied to baseline scores alone, and (iii) the standalone pNML detector compared with the combined scores. Without such ablations, the empirical improvement over baselines is not attributable to the proposed regret.
  3. [Section 7.2 / Algorithm 2] DIAL is introduced as the solution of the min-max criterion in Eq. (7.5), but the algorithm replaces the posterior p(θ|z^{n-1}) by a uniform distribution over MC-dropout samples and then maximizes the product of two softmax probabilities over that discrete set. No argument is given that this maximization approximates the pNML regret in Eq. (7.5) or that the uniform-weight approximation is adequate. As a result, the active-learning contribution is an interesting heuristic rather than a derived pNML procedure; the min-max framing in Sections 7.2 and 7.5 overstates the theoretical status. Please state the approximation explicitly, provide supporting ablations or approximation bounds, or weaken the claims accordingly.
  4. [Chapter 6, Eqs. (6.5)-(6.8)] The Adversarial pNML scheme uses a refinement stage that performs a targeted adversarial attack, not maximum-likelihood training on the augmented dataset, so the normalization factor K in Eq. (6.8) is not the pNML normalization for any stated hypothesis class. The empirical results are of interest and the adversarial-subspace interpretation is suggestive, but the chapter should not claim the pNML min-max regret interpretation without a derivation. Please either provide a formal link between the refinement step and the pNML genie, or describe the method as a pNML-inspired heuristic and remove the optimality language from the abstract and introduction.
minor comments (5)
  1. [Eq. (2.20)] There appears to be a sign error: with P_Nλ defined as in Eq. (2.18), the normalization factor in q_pNML should involve (1 − xᵀP_Nλx), not (1 + xᵀP_Nλx). As printed, the density does not integrate to 1.
  2. [Section 5.2, after Eq. (5.3)] The symbol f⁻¹(Y_N) is used before the inverse activation is defined in Eq. (5.7), and for the softmax case the definition is ambiguous for one-hot label matrices. Please clarify the domain of f⁻¹ or avoid this notation until the activation is fixed.
  3. [Section 5.2, remarks after Eq. (5.12)] The phrase 'In section 5.2 we show...' appears twice in quick succession and the second occurrence should reference the relevant experiment subsection rather than the same section.
  4. [Section 3.4.1, Eq. (3.27)] The notation in the regret upper bound is dense; in particular, the third term inside the logarithm should be displayed with explicit parentheses so that the reader can see what is inside the cube root. A small derivation sketch in the main text would also improve readability.
  5. [Section 5.3] The text 'Moor-Penrose' should be 'Moore-Penrose', both here and where the same typo appears near Eq. (5.2).

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central derivations are self-contained, with only minor empirical heuristics and non-load-bearing self-citations.

full rationale

The derivation chain is largely self-contained. Chapter 2 obtains the pNML regret for linear regression by direct integration over the RLS genie, and the learnable-space conclusions follow algebraically from the SVD expression; no fitted parameter is renamed as a prediction. Chapter 3 derives an explicit upper bound for a norm-constrained hypothesis class rather than restating the class definition. Chapter 4 derives the LpNML predictive distribution from the luckiness-weighted genie objective. Chapter 5's neural-network regret in Eq. (5.11) is a closed-form function of ERM probabilities and the data-dependent kernel x^Tg; although the paper itself notes in Section 5.2 that the recursive update (5.5) 'does not compute the exact solution' and this is a genuine correctness or approximation risk, the formula is not a tautology and does not reduce to a fitted value. The embedding normalization to unit L2 norms is an empirical heuristic informed by OOD embedding norms, but it is a fixed preprocessing rather than a parameter fitted to test labels. Self-citations (refs [10,11]) are not load-bearing: [11] is cited only as the published version of the Chapter 2 derivation, and [10] appears in a future-work remark. Accordingly, no circular step is identified; the score reflects minor empirical and self-citation concerns rather than circular reasoning.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The central derivations rely on the Gaussian conditional hypothesis class, standard pseudo-inverse updates, and two paper-specific choices: the norm-constrained hypothesis set in Chapter 3 and the luckiness function in Chapter 4. The neural network section adds a third paper-specific assumption: the recursive update (5.5) is treated as the genie despite the text noting it is not exact. There are no invented physical entities such as new particles, forces, or conserved quantities.

free parameters (4)
  • Noise variance sigma^2 = Optimized per dataset on a validation set or leave-one-out
    Used in the regression likelihood and appears in the over-parameterized regret bound (3.27). Tuned in Sections 3.5 and 4.4.3, so the quoted regrets are partially dependent on a fitted scale.
  • Ridge/luckiness regularization lambda (LpNML) = Tuned by leave-one-out per dataset
    Controls the luckiness function w(theta)=exp(-lambda*||theta||^2/(2*sigma^2)) in Eq. 4.19. The LpNML prediction shift in Theorem 4.3.3 depends on lambda.
  • Adversarial refinement strength lambda = Chosen in [0.5*epsilon, epsilon], e.g., 0.03 for CIFAR10, 3/255 for ImageNet
    Sets the targeted attack step in Eq. 6.5. Robustness results in Table 6.1 depend on this hand-selected value.
  • Embedding L2 norm normalization = 1.0
    All embeddings normalized to unit norm after empirically finding that OOD norms are lower. This changes the regret scores in OOD detection (Section 5.2, Deep neural network adaptation).
assumptions (7)
  • domain assumption Gaussian conditional model for regression
    Eqs. 2.3 and 3.15: p_theta(y|x) is Gaussian with variance sigma^2. Even in the individual setting, the hypothesis class is Gaussian, so the pNML integrals are over a Gaussian family.
  • domain assumption Log-loss and min-max regret as the learning objective
    Chapter 1 defines regret via log-loss (Eq. 1.3) and takes pNML as the solution. All uncertainty claims are relative to this objective.
  • standard math Recursive least squares and pseudo-inverse update formulas
    Eqs. 2.10-2.11 and 3.8-3.11 rely on standard Sherman-Morrison and Moore-Penrose inverse update identities.
  • domain assumption Existence of inverse softmax activation and valid one-hot labels
    Eq. 5.7 defines z = f^{-1}(p) = ln p_i + ln sum_j e^{theta_j^T x}. This requires the softmax logits to be recoverable from the probability vector.
  • ad hoc to paper The Zhuang recursive update approximates the ERM retrained on augmented data
    Section 5.2 uses update (5.5) as the genie. The text admits it is only an iterative step, not the exact solution, so this is a paper-specific approximation.
  • ad hoc to paper Norm-constrained hypothesis class for over-parameterized regression
    Eq. 3.18 restricts to ||theta|| <= ||theta*_N|| to avoid non-informative regret. The resulting pNML prediction is asserted to equal the minimum-norm solution.
  • ad hoc to paper Luckiness function shape
    Eq. 4.19 sets w(theta)=exp(-lambda*||theta||^2/(2*sigma^2)) to obtain a ridge genie. Other luckiness functions would give different predictive shifts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantifying the Prediction Uncertainty of Machine Learning Models for Individual Data." pith.science (2026). https://pith.science/paper/XLONDL3C

@misc{pith2026241207520,
  author       = {Pith},
  title        = {Pith review of: Quantifying the Prediction Uncertainty of Machine Learning Models for Individual Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XLONDL3C}},
  note         = {Machine review of arXiv:2412.07520}
}
read the original abstract

Machine learning models have exhibited exceptional results in various domains. The most prevalent approach for learning is the empirical risk minimizer (ERM), which adapts the model's weights to reduce the loss on a training set and subsequently leverages these weights to predict the label for new test data. Nonetheless, ERM makes the assumption that the test distribution is similar to the training distribution, which may not always hold in real-world situations. In contrast, the predictive normalized maximum likelihood (pNML) was proposed as a min-max solution for the individual setting where no assumptions are made on the distribution of the tested input. This study investigates pNML's learnability for linear regression and neural networks, and demonstrates that pNML can improve the performance and robustness of these models on various tasks. Moreover, the pNML provides an accurate confidence measure for its output, showcasing state-of-the-art results for out-of-distribution detection, resistance to adversarial attacks, and active learning.

Figures

Figures reproduced from arXiv: 2412.07520 by the authors.

Figure 2
Figure 2. figure 2.1a (bottom) shows the regret, given by (2.15), for the polynomial model from [PITH_FULL_IMAGE:figures/full_fig_p029_2.png] view at source ↗
Figure 2.1
Figure 2.1. The mean pNML least squares estimator and its associated regret [PITH_FULL_IMAGE:figures/full_fig_p029_2_1.png] view at source ↗
Figure 2
Figure 2. b (bottom) shows the regret (or learnability) of the three pNML learners [PITH_FULL_IMAGE:figures/full_fig_p030_2.png] view at source ↗
Figures from the paper (29 more)
Figure 3.1
Figure 3.1. Figure 3.1: The pNML regret for over-parameterized linear regression [PITH_FULL_IMAGE:figures/full_fig_p040_3_1.png]
Figure 3
Figure 3. Figure 3: shows the logloss and the Cumulative Distribution Function (CDF) as func [PITH_FULL_IMAGE:figures/full_fig_p041_3.png]
Figure 3.2
Figure 3.2. Figure 3.2: The MN and pNML loss for samples with regrets lower than a thresh [PITH_FULL_IMAGE:figures/full_fig_p042_3_2.png]
Figure 3.3
Figure 3.3. Figure 3.3: Double-descent of the test log-loss and regret for UCI datasets [PITH_FULL_IMAGE:figures/full_fig_p044_3_3.png]
Figure 4.1
Figure 4.1. Figure 4.1: Ridge ERM and LpNML prediction histograms [PITH_FULL_IMAGE:figures/full_fig_p047_4_1.png]
Figure 4
Figure 4. Figure 4: shows the Ridge ERM, Bayesian and LpNML predictions. The Bayesian [PITH_FULL_IMAGE:figures/full_fig_p055_4.png]
Figure 4.2
Figure 4.2. Figure 4.2: Polynomial fitting to synthetic data The confidence intervals are shown in figure 4.2 with lighter colors. Both learners have large confidence intervals at the figure’s edges and for 0.1 ≤ t ≤ 0.8, where the training points are scarce. The LpNML has much larger confi…
Figure 4
Figure 4. Figure 4: a shows the histogram of Ridge ERM and LpNML predictions for test sam [PITH_FULL_IMAGE:figures/full_fig_p056_4.png]
Figure 4.3
Figure 4.3. Figure 4.3: LpNML and Bayesian learners for different model degrees [PITH_FULL_IMAGE:figures/full_fig_p057_4_3.png]
Figure 4.4
Figure 4.4. Figure 4.4: PCA for the distribution-shift benchmark [PITH_FULL_IMAGE:figures/full_fig_p059_4_4.png]
Figure 4
Figure 4. Figure 4: a shows the principal component analysis (PCA) with 2 components of the [PITH_FULL_IMAGE:figures/full_fig_p060_4.png]
Figure 5.1
Figure 5.1. Figure 5.1: The pNML regret simulation for a two class predictor [PITH_FULL_IMAGE:figures/full_fig_p068_5_1.png]
Figure 5
Figure 5. Figure 5: b presents the analytical pNML regret. At the upper left and lower right, the [PITH_FULL_IMAGE:figures/full_fig_p068_5.png]
Figure 5.2
Figure 5.2. Figure 5.2: The pNML regret for a Iris dataset In addition, we visualize the regret for overlapping classes. In figure 5.2c, the ERM probability assignment for inseparable class split is shown. The ERM probability is lower than 0.7 for all test feature values [PITH_FULL_IMAGE:f…
Figure 5.3
Figure 5.3. Figure 5.3: CCR vs FPR curves of the compared methods [PITH_FULL_IMAGE:figures/full_fig_p078_5_3.png]
Figure 5
Figure 5. Figure 5: figure 5.3a. Our method provides the highest CCR for all FPR rates. table 5.5 also shows [PITH_FULL_IMAGE:figures/full_fig_p078_5.png]
Figure 5.4
Figure 5.4. Figure 5.4: Confidence histograms for DenseNet100 CIFAR10 based models [PITH_FULL_IMAGE:figures/full_fig_p080_5_4.png]
Figure 5.5
Figure 5.5. Figure 5.5: The pNML performance for the CIFAR10 set of different amount of [PITH_FULL_IMAGE:figures/full_fig_p083_5_5.png]
Figure 6.1
Figure 6.1. Figure 6.1: Label "0" probability manifold right direction. To support that claim we note that the adversarial subspace has a low prob￾ability and is less stable compared to the true data subspace [105]. In other words, while the true hypothesis escapes the adversarial subspace,…
Figure 6
Figure 6. Figure 6: shows the refinement process on top of the trained model label 0 probability [PITH_FULL_IMAGE:figures/full_fig_p090_6.png]
Figure 6
Figure 6. Figure 6: a demonstrates the convergence to local maxima mechanism. [PITH_FULL_IMAGE:figures/full_fig_p091_6.png]
Figure 6.2
Figure 6.2. Figure 6.2: Robustness for different ε values 76 [PITH_FULL_IMAGE:figures/full_fig_p094_6_2.png]
Figure 6.3
Figure 6.3. Figure 6.3: ImageNet accuracy vs the pNML refinement strength [PITH_FULL_IMAGE:figures/full_fig_p095_6_3.png]
Figure 7.1
Figure 7.1. Figure 7.1: Datasets that contain a mix of images with OOD samples [PITH_FULL_IMAGE:figures/full_fig_p104_7_1.png]
Figure 7.2
Figure 7.2. Figure 7.2: Accuracy rate as function of the training set size [PITH_FULL_IMAGE:figures/full_fig_p106_7_2.png]
Figure 7.3
Figure 7.3. Figure 7.3: Active learning performance for EMNIST accuracy rate of 0.95 where DIAL uses 169 samples while BALD attains this accuracy level with 209 samples. To simulate the presence of OOD samples, we added the Fashion MNIST to the unla￾beled pool such that the ratio of Fashion…
Figure 7.4
Figure 7.4. Figure 7.4: Active learning for CIFAR10 7.3.6 CIFAR10 experimental results For the CIFAR10 data-set, we utilized ResNet-18 [46] with acquisition size of 16 samples. We used 1K initial training set size and measured the performance of the active learning strategies up to a traini…
Figure 7
Figure 7. Figure 7: figure 7.4b, DIAL outperforms all other baselines. This can be explained by figure 7.5 of [PITH_FULL_IMAGE:figures/full_fig_p108_7.png]
Figure 7.5
Figure 7.5. Figure 7.5: The amount of chosen OOD samples for CIFAR10 [PITH_FULL_IMAGE:figures/full_fig_p109_7_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

130 extracted references · 61 canonical work pages

  1. [1]

    Tldr: Deep learning-based automated privacy policy annotation with key policy highlights

    Abdulrahman Alabduljabbar, Ahmed Abusnaina, Ülkü Meteriz-Yildiran, and David Mohaisen. Tldr: Deep learning-based automated privacy policy annotation with key policy highlights. In Proceedings of the 20th Workshop on Workshop on Privacy in the Electronic Society, pages 103–118, 2021

    2021

  2. [2]

    A convergence theory for deep learning via over-parameterization

    Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via over-parameterization. InInternational Conference on Machine Learn- ing, pages 242–252. PMLR, 2019

    2019

  3. [3]

    Transductive versions of the lasso and the dantzig selector

    Pierre Alquier and Mohamed Hebiri. Transductive versions of the lasso and the dantzig selector. Journal of Statistical Planning and Inference, 142(9):2485–2500, 2012

    2012

  4. [4]

    Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples

    Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv:1802.00420, 2018

    arXiv 2018

  5. [5]

    Bartlett, Dylan J

    Peter L. Bartlett, Dylan J. Foster, and Matus Telgarsky. Spectrally-normalized mar- gin bounds for neural networks. InNeural Inform. Process. Syst., pages 6240–6249, 2017

    2017

  6. [6]

    Benign overfitting in linear regression

    Peter L Bartlett, Philip M Long, Gábor Lugosi, and Alexander Tsigler. Benign overfitting in linear regression. Proceedings of the National Academy of Sciences , 2020

    2020

  7. [7]

    Hsu, and Partha Mitra

    Mikhail Belkin, Daniel J. Hsu, and Partha Mitra. Overfitting or perfect fitting? risk bounds for classification and regression rules that interpolate. In Neural Inform. Process. Syst., 2018. 96

    2018

  8. [8]

    Reconciling modern machine-learning practice and the classical bias–variance trade-off

    Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32):15849–15854, 2019

    2019

Show all 130 references
  1. [9]

    Generalized inverses: theory and appli- cations, volume 15

    Adi Ben-Israel and Thomas NE Greville. Generalized inverses: theory and appli- cations, volume 15. Springer Science & Business Media, 2003

    2003

  2. [10]

    Deep pnml: Predictive normalized max- imum likelihood for deep neural networks

    Koby Bibas, Yaniv Fogel, and Meir Feder. Deep pnml: Predictive normalized max- imum likelihood for deep neural networks. arXiv preprint arXiv:1904.12286, 2019

    1904 arXiv

  3. [11]

    A new look at an old problem: A univer- sal learning approach to linear regression

    Koby Bibas, Yaniv Fogel, and Meir Feder. A new look at an old problem: A univer- sal learning approach to linear regression. In 2019 IEEE International Symposium on Information Theory (ISIT), pages 2304–2308. IEEE, IEEE, 2019

    2019

  4. [12]

    Learning rotation invariant features for cryogenic electron microscopy image re- construction

    Koby Bibas, Gili Weiss-Dicker, Dana Cohen, Noa Cahan, and Hayit Greenspan. Learning rotation invariant features for cryogenic electron microscopy image re- construction. In International Symposium on Biomedical Imaging (ISBI) , pages 563–566. IEEE, 2021

    2021

  5. [13]

    Evasion attacks against machine learning at test time

    Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndi ´c, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowl- edge discovery in databases, pages 3...

    2013

  6. [14]

    The description length of deep learning models

    Léonard Blier and Yann Ollivier. The description length of deep learning models. In Advances in Neural Information Processing Systems, pages 2216–2226, 2018

    2018

  7. [15]

    On evaluating adversarial robustness

    Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, and Aleksander Madry. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705, 2019

    1902 arXiv

  8. [16]

    Unlabeled data improves adversarial robustness

    Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems, pages 11190–11201, 2019

    2019

  9. [17]

    Transductive inference for estimating values of functions

    Olivier Chapelle, Vladimir Vapnik, and Jason Weston. Transductive inference for estimating values of functions. In Neural Inform. Process. Syst., 2000. 97

    2000

  10. [18]

    Hopskipjumpattack: A query-efficient decision-based attack

    Jianbo Chen, Michael I Jordan, and Martin J Wainwright. Hopskipjumpattack: A query-efficient decision-based attack. In 2020 IEEE Symposium on Security and Privacy (SP), pages 668–685, 2019

    2020

  11. [19]

    Emnist: Extending mnist to handwritten letters

    Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: Extending mnist to handwritten letters. In 2017 international joint conference on neural networks (IJCNN), pages 2921–2926. IEEE, 2017

    2017

  12. [20]

    On transductive regression

    Corinna Cortes and Mehryar Mohri. On transductive regression. In Neural Inform. Process. Syst., pages 305–312, 2007

    2007

  13. [21]

    Laplace redux-effortless bayesian deep learn- ing

    Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortless bayesian deep learn- ing. Advances in Neural Information Processing Systems, 34:20089–20103, 2021

    2021

  14. [22]

    Mathematics for machine learning, chapter 9.3

    Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong. Mathematics for machine learning, chapter 9.3. Cambridge University Press, 2020

    2020

  15. [23]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proc. Conf. Comput. Vision Pattern Recognition, pages 248–255. Ieee, 2009

    2009

  16. [24]

    The mnist database of handwritten digit images for machine learning research [best of the web].IEEE signal processing magazine, 29(6):141–142, 2012

    Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web].IEEE signal processing magazine, 29(6):141–142, 2012

    2012

  17. [25]

    Reducing network agnostophobia

    Akshay Raj Dhamija, Manuel Günther, and Terrance Boult. Reducing network agnostophobia. In Neural Inform. Process. Syst., pages 9157–9168, 2018

    2018

  18. [26]

    UCI machine learning repository, 2017

    Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http: //archive.ics.uci.edu/ml

    2017

  19. [27]

    Revisiting com- plexity and the bias-variance tradeoff

    Raaz Dwivedi, Chandan Singh, , Bin Yu, and Martin Wainwright. Revisiting com- plexity and the bias-variance tradeoff. arXiv preprint arXiv:2006.10189, 2020

    2006 arXiv

  20. [28]

    Wainwright

    Raaz Dwivedi, Chandan Singh, Bin Yu, and Martin J. Wainwright. Revisiting min- imum description length complexity in overparameterized models, 2021

    2021

  21. [29]

    Theory of optimal experiments

    Valerii Vadimovich Fedorov. Theory of optimal experiments. Elsevier, 2013. 98

    2013

  22. [30]

    The use of multiple measurements in taxonomic problems.Annals of eugenics, 7(2):179–188, 1936

    Ronald A Fisher. The use of multiple measurements in taxonomic problems.Annals of eugenics, 7(2):179–188, 1936

    1936

  23. [31]

    On the problem of on-line learning with log-loss.IEEE International Symposium on Information Theory - Proceedings, pages 2995–2999,

    Yaniv Fogel and Meir Feder. On the problem of on-line learning with log-loss.IEEE International Symposium on Information Theory - Proceedings, pages 2995–2999,

  24. [32]

    Universal batch learning with log-loss

    Yaniv Fogel and Meir Feder. Universal batch learning with log-loss. In 2018 IEEE International Symposium on Information Theory (ISIT), pages 21–25. IEEE, 2018

    2018

  25. [33]

    Universal learning of individual data

    Yaniv Fogel and Meir Feder. Universal learning of individual data. In Int. Symp. on Information Theory, 2019

    2019

  26. [34]

    Dropout as a bayesian approximation: Repre- senting model uncertainty in deep learning

    Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Repre- senting model uncertainty in deep learning. In Int. Conf. Mach. Learning , pages 1050–1059, 2016

    2016

  27. [35]

    Deep bayesian active learning with image data

    Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning , pages 1183–

  28. [36]

    Degrees of freedom in deep neural networks

    Tianxiang Gao and Vladimir Jojic. Degrees of freedom in deep neural networks. In Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intel- ligence, 2016

    2016

  29. [37]

    Deep learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016a

  30. [38]

    Explaining and harness- ing adversarial examples

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harness- ing adversarial examples. arXiv preprint arXiv:1412.6572, 2014

    2014 arXiv

  31. [39]

    Machine learning for social science: An agnostic approach

    Justin Grimmer, Margaret E Roberts, and Brandon M Stewart. Machine learning for social science: An agnostic approach. Annual Review of Political Science, 24: 395–419, 2021

    2021

  32. [40]

    The minimum description length principle

    Peter D Grünwald. The minimum description length principle. MIT press, 2007. 99

    2007

  33. [41]

    Lee, Daniel Soudry, and Nati Srebro

    Suriya Gunasekar, Jason D. Lee, Daniel Soudry, and Nati Srebro. Implicit bias of gradient descent on linear convolutional networks. InNeural Inform. Process. Syst., 2018

    2018

  34. [42]

    Coun- tering adversarial images using input transformations

    Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Coun- tering adversarial images using input transformations. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id= SyJ7ClWCb

    2018

  35. [43]

    Friedman

    Tibshirani Hastie and Robert Tibshirani. Friedman. The Elements of Statistical Learning Springer, page 52, 2001

    2001

  36. [44]

    Sur- prises in high-dimensional ridgeless least squares interpolation

    Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan J Tibshirani. Sur- prises in high-dimensional ridgeless least squares interpolation. arXiv preprint arXiv:1903.08560, 2019

    1903 arXiv

  37. [45]

    9.4: Recursive least squares

    Monson H Hayes. 9.4: Recursive least squares. Statistical Digital Signal Processing and Modeling, page 541, 1996

    1996

  38. [46]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. Conf. Comput. Vision Pattern Recognition, pages 770–778, 2016

    2016

  39. [47]

    A baseline for detecting misclassified and out- of-distribution examples in neural networks

    Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out- of-distribution examples in neural networks. Int. Conf. on Learning Representa- tions, 2017

    2017

  40. [48]

    A baseline for detecting misclassified and out-of-distribution examples in neural networks

    Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2017

    2017 arXiv

  41. [49]

    Using self- supervised learning can improve model robustness and uncertainty

    Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self- supervised learning can improve model robustness and uncertainty. In Neural In- form. Process. Syst., 2019

    2019

  42. [50]

    Probabilistic backpropagation for scalable learning of bayesian neural networks

    José Miguel Hernández-Lobato and Ryan Adams. Probabilistic backpropagation for scalable learning of bayesian neural networks. In Int. Conf. Mach. Learning , pages 1861–1869, 2015. 100

    2015

  43. [51]

    Efficient computation of normalized maximum like- lihood coding for gaussian mixtures with its applications to optimal clustering

    So Hirai and Kenji Yamanishi. Efficient computation of normalized maximum like- lihood coding for gaussian mixtures with its applications to optimal clustering. In Int. Symp. on Information Theory, 2011

    2011

  44. [52]

    Hoerl and R Kennard

    A.E. Hoerl and R Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 1970

    1970

  45. [53]

    Bayesian active learning for classification and preference learning

    Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745, 2011

    2011 arXiv

  46. [54]

    Densely connected convolutional networks

    Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proc. Conf. Comput. Vision Pattern Recognition, 2017

    2017

  47. [55]

    Deepal: Deep active learning in python

    Kuan-Hao Huang. Deepal: Deep active learning in python. arXiv preprint arXiv:2111.15258, 2021

    2021 arXiv

  48. [56]

    An introduc- tion to statistical learning, volume 112

    Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. An introduc- tion to statistical learning, volume 112. Springer, 2013

    2013

  49. [57]

    Fantastic generalization measures and where to find them

    Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Ben- gio. Fantastic generalization measures and where to find them. In Int. Conf. on Learning Representations, 2020

    2020

  50. [58]

    On the complexity of linear prediction: Risk bounds, margin bounds, and regularization

    Sham M Kakade, Karthik Sridharan, and Ambuj Tewari. On the complexity of linear prediction: Risk bounds, margin bounds, and regularization. In Advances in neural information processing systems, pages 793–800, 2009

    2009

  51. [59]

    Balancing specialization, generalization, and compression for detection and track- ing

    Dotan Kaufman, Koby Bibas, Eran Borenstein, Michael Chertok, and Tal Hassner. Balancing specialization, generalization, and compression for detection and track- ing. In Proc. British Mach. Vision Conf., 2019

    2019

  52. [60]

    The cifar-10 dataset

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar-10 dataset. online: http://www. cs. toronto. edu/kriz/cifar. html, 55, 2014

    2014

  53. [61]

    Solving least squares problems , vol- ume 15

    Charles L Lawson and Richard J Hanson. Solving least squares problems , vol- ume 15. Siam, 1995. 101

    1995

  54. [62]

    MNIST handwritten digit database

    Yann LeCun and Corinna Cortes. MNIST handwritten digit database. online: http://yann.lecun.com/exdb/mnist/, 2:18, 2010. URL http://yann.lecun.com/ exdb/mnist/

    2010

  55. [63]

    A simple unified framework for detecting out-of-distribution samples and adversarial attacks

    Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Neural Inform. Process. Syst., 2018

    2018

  56. [64]

    Near-optimal linear regression under distribution shift

    Qi Lei, Wei Hu, and Jason Lee. Near-optimal linear regression under distribution shift. In Int. Conf. Mach. Learning, 2021

    2021

  57. [65]

    Measuring the intrinsic dimension of objective landscapes

    Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimension of objective landscapes. In Int. Conf. on Learning Representa- tions, 2018

    2018

  58. [66]

    Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of- distribution image detection in neural networks. In Int. Conf. on Learning Rep- resentations, 2018

    2018

  59. [67]

    Just interpolate: Kernel ridgeless re- gression can generalize

    Tengyuan Liang, Alexander Rakhlin, et al. Just interpolate: Kernel ridgeless re- gression can generalize. Annals of Statistics, 48(3):1329–1347, 2020

    2020

  60. [68]

    Ridge regression: Structure, cross-validation, and sketching

    Sifan Liu and Edgar Dobriban. Ridge regression: Structure, cross-validation, and sketching. In Int. Conf. on Learning Representations, 2020

    2020

  61. [69]

    Energy-based out-of- distribution detection

    Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of- distribution detection. Advances in Neural Information Processing Systems, 2020

    2020

  62. [70]

    Deep learning face at- tributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face at- tributes in the wild. In Proc. Int. Conf. Comput. Vision, December 2015

    2015

  63. [71]

    On the generalization properties of minimum- norm solutions for over-parameterized neural network models

    Chao Ma, Lei Wu, et al. On the generalization properties of minimum- norm solutions for over-parameterized neural network models. arXiv preprint arXiv:1912.06987, 2019

    1912 arXiv

  64. [72]

    Information-based objective functions for active data selection

    David JC MacKay. Information-based objective functions for active data selection. Neural computation, 4(4):590–604, 1992. 102

    1992

  65. [73]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017

    2017 arXiv

  66. [74]

    Universal prediction

    Neri Merhav and Meir Feder. Universal prediction. Trans. on Inform. Theory, 44 (6):2124–2147, 1998

    1998

  67. [75]

    Normalized maximum likelihood with luckiness for multivariate normal distributions

    Kohei Miyaguchi. Normalized maximum likelihood with luckiness for multivariate normal distributions. arXiv preprint arXiv:1708.01861, 2017

    2017 arXiv

  68. [76]

    Harmless interpolation of noisy data in regression

    Vidya Muthukumar, Kailas V odrahalli, Vignesh Subramanian, and Anant Sahai. Harmless interpolation of noisy data in regression. IEEE Journal on Selected Areas in Information Theory, 2020

    2020

  69. [77]

    Optimal regu- larization can mitigate double descent

    Preetum Nakkiran, Prayaag Venkat, Sham Kakade, and Tengyu Ma. Optimal regu- larization can mitigate double descent. arXiv preprint arXiv:2003.01897, 2020

    2003 arXiv

  70. [78]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and An- drew Y Ng. Reading digits in natural images with unsupervised feature learning. In Neural Inform. Process. Syst. Workshops, volume 2011, page 5, 2011

    2011

  71. [79]

    A pac-bayesian approach to spectrally-normalized margin bounds for neural networks

    Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. A pac-bayesian approach to spectrally-normalized margin bounds for neural networks. InInt. Conf. on Learning Representations, 2018

    2018

  72. [80]

    Do deeper convolutional networks perform better? arXiv preprint arXiv:2010.09610, 2020

    Eshaan Nichani, Adityanarayanan Radhakrishnan, and Caroline Uhler. Do deeper convolutional networks perform better? arXiv preprint arXiv:2010.09610, 2020

    2010 arXiv

  73. [81]

    Olson, William La Cava, Patryk Orzechowski, Ryan J

    Randal S. Olson, William La Cava, Patryk Orzechowski, Ryan J. Urbanowicz, and Jason H. Moore. Pmlb: a large benchmark suite for machine learning evaluation and comparison. BioData Mining, 10(1):36, Dec 2017. ISSN 1756-0381. doi: 10.1186/ s13040-017-0154-4. URL https://doi.org/...

    2017 doi

  74. [82]

    Outlier exposure with confidence control for out-of-distribution detection

    Aristotelis-Angelos Papadopoulos, Mohammad Reza Rajati, Nazim Shaikh, and Jiamian Wang. Outlier exposure with confidence control for out-of-distribution detection. Neurocomputing, 2021

    2021

  75. [83]

    Practical black-box attacks against machine learning

    Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In 103 Proceedings of the 2017 ACM on Asia conference on computer and communications security, pages 506–519. ACM, 2017

    2017

  76. [84]

    Evaluation methods in face recognition

    P Jonathon Phillips, Patrick Grother, and Ross Micheals. Evaluation methods in face recognition. In Handbook of face recognition, pages 551–574. Springer, 2011

    2011

  77. [85]

    Ad- versarial robustness through local linearization

    Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvi- jotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Ad- versarial robustness through local linearization. In Advances in Neural Information Processing Systems, pages 13824–13833, 2019

    2019

  78. [86]

    Information-based complexity, feedback and dynamics in convex programming

    Maxim Raginsky and Alexander Rakhlin. Information-based complexity, feedback and dynamics in convex programming. IEEE Transactions on Information Theory, 57(10):7036–7056, 2011

    2011

  79. [87]

    A survey of deep active learning

    Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. A survey of deep active learning. ACM computing surveys (CSUR), 54(9):1–40, 2021

    2021

  80. [88]

    Mdl regression and denoising

    Teemu Roos. Mdl regression and denoising. 2004

    2004

  81. [89]

    Defense-GAN: Protect- ing classifiers against adversarial attacks using generative models

    Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-GAN: Protect- ing classifiers against adversarial attacks using generative models. In International Conference on Learning Representations, 2018. URL https://openreview.net/ forum?id=BkJ3ibb0-

    2018

  82. [90]

    Detecting out-of-distribution exam- ples with Gram matrices

    Chandramouli Shama Sastry and Sageev Oore. Detecting out-of-distribution exam- ples with Gram matrices. In Int. Conf. Mach. Learning, 2020

    2020

  83. [91]

    Toward open set recognition

    Walter J Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance E Boult. Toward open set recognition. Trans. Pattern Anal. Mach. Intell. , 35(7): 1757–1772, 2012

    2012

  84. [92]

    Active learning for convolutional neural networks: A core-set approach

    Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489, 2017

    2017 arXiv

  85. [93]

    On the asymptotic distribution of ridge re- gression estimators using training and test samples

    Nandana Sengupta and Fallaw Sowell. On the asymptotic distribution of ridge re- gression estimators using training and test samples. Econometrics, 8(4):39, 2020. 104

    2020

  86. [94]

    Convolutional neural net- works applied to house numbers digit classification

    Pierre Sermanet, Soumith Chintala, and Yann LeCun. Convolutional neural net- works applied to house numbers digit classification. In Proceedings of the 21st international conference on pattern recognition (ICPR2012) , pages 3288–3291. IEEE, 2012

    2012

  87. [95]

    Minimum norm solutions do not always generalize well for over-parameterized problems.stat, 1050:16, 2018

    Vatsal Shah, Anastasios Kyrillidis, and Sujay Sanghavi. Minimum norm solutions do not always generalize well for over-parameterized problems.stat, 1050:16, 2018

    2018

  88. [96]

    Learn- ability, stability and uniform convergence

    Shai Shalev-Shwartz, Ohad Shamir, Nathan Srebro, and Karthik Sridharan. Learn- ability, stability and uniform convergence. Journal of Machine Learning Research, 11(Oct):2635–2670, 2010

    2010

  89. [97]

    The sample complexity of learning linear predictors with the squared loss

    Ohad Shamir. The sample complexity of learning linear predictors with the squared loss. The Journal of Machine Learning Research, 16(1):3475–3486, 2015

    2015

  90. [98]

    Universal active learning via conditional mutual information minimization

    Shachar Shayovitz and Meir Feder. Universal active learning via conditional mutual information minimization. IEEE Journal on Selected Areas in Information Theory, 2(2):720–734, 2021

    2021

  91. [99]

    Minimax active learning via minimal model capacity

    Shachar Shayovitz and Feder Meir. Minimax active learning via minimal model capacity. Submitted to Machine Learning for Signal Processing Workshop (MLSP), 2019

    2019

  92. [100]

    Universal sequential coding of single messages.Prob- lemy Peredachi Informatsii, 23(3):3–17, 1987

    Yurii Mikhailovich Shtarkov. Universal sequential coding of single messages.Prob- lemy Peredachi Informatsii, 23(3):3–17, 1987

    1987

  93. [101]

    An almost optimal pac algorithm

    Hans U Simon. An almost optimal pac algorithm. In Conference on Learning Theory, pages 1552–1563. PMLR, 2015

    2015

  94. [102]

    Uncertainty aware and explainable diagnosis of retinal disease

    Amitojdeep Singh, Sourya Sengupta, Mohamed Abdul Rasheed, Varadharajan Jayakumar, and Vasudevan Lakshminarayanan. Uncertainty aware and explainable diagnosis of retinal disease. In Medical Imaging 2021: Imaging Informatics for Healthcare, Research, and Applications, 2021

    2021

  95. [103]

    Pixeldefend: Leveraging generative models to understand and defend against adver- sarial examples

    Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adver- sarial examples. In International Conference on Learning Representations , 2018. URL https://openreview.net/forum?id=rJU...

    2018

  96. [104]

    Intriguing properties of neural networks

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013

    2013 arXiv

  97. [105]

    Exploring the space of adversarial images

    Pedro Tabacof and Eduardo Valle. Exploring the space of adversarial images. In 2016 International Joint Conference on Neural Networks (IJCNN), pages 426–433. IEEE, 2016

    2016

  98. [106]

    Hyperparameter-free out-of-distribution detection using cosine similarity

    Engkarat Techapanurak, Masanori Suganuma, and Takayuki Okatani. Hyperparameter-free out-of-distribution detection using cosine similarity. arXiv preprint arXiv:1905.10628, 2020

    1905 arXiv

  99. [107]

    Single point transductive prediction

    Nilesh Tripuraneni and Lester Mackey. Single point transductive prediction. In Int. Conf. Mach. Learning, 2020

    2020

  100. [108]

    Benign overfitting in ridge regression.arXiv preprint arXiv:2009.14286, 2020

    Alexander Tsigler and Peter L Bartlett. Benign overfitting in ridge regression.arXiv preprint arXiv:2009.14286, 2020

    2009 arXiv

  101. [109]

    A theory of the learnable

    Leslie G Valiant. A theory of the learnable. Communications of the ACM, 27(11): 1134–1142, 1984

    1984

  102. [110]

    Principles of risk minimization for learning theory

    Vladimir Vapnik. Principles of risk minimization for learning theory. In Neural Inform. Process. Syst., pages 831–838, 1992

    1992

  103. [112]

    On the uniform convergence of relative frequencies of events to their probabilities

    Vladimir N Vapnik and A Ya Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. InMeasures of complexity, pages 11–30. Springer, 2015

    2015

  104. [113]

    Apoorv Vyas, Nataraj Jammalamadaka, Xia Zhu, Dipankar Das, Bharat Kaul, and Theodore L. Willke. Out-of-distribution detection using an ensemble of self su- pervised leave-out classifiers. In European Conf. Comput. Vision, pages 560–574, 2018. 106

    2018

  105. [114]

    Cosface: Large margin cosine loss for deep face recogni- tion

    Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recogni- tion. In Proc. Conf. Comput. Vision Pattern Recognition, pages 5265–5274, 2018

    2018

  106. [115]

    Cost-effective active learning for deep image classification

    Keze Wang, Dongyu Zhang, Ya Li, Ruimao Zhang, and Liang Lin. Cost-effective active learning for deep image classification. IEEE Transactions on Circuits and Systems for Video Technology, 27(12):2591–2600, 2016

    2016

  107. [116]

    Safety concerns and mitigation approaches regarding the use of deep learning in safety-critical perception tasks

    Oliver Willers, Sebastian Sudholt, Shervin Raafatnia, and Stephanie Abrecht. Safety concerns and mitigation approaches regarding the use of deep learning in safety-critical perception tasks. In International Conference on Computer Safety, Reliability, and Security, pages 336–3...

    2020

  108. [117]

    Stochas- tic variational deep kernel learning

    Andrew G Wilson, Zhiting Hu, Russ R Salakhutdinov, and Eric P Xing. Stochas- tic variational deep kernel learning. Advances in Neural Information Processing Systems, 29, 2016

    2016

  109. [118]

    Zico Kolter

    Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting ad- versarial training. In International Conference on Learning Representations, 2020

    2020

  110. [119]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms

    Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017

    2017 arXiv

  111. [120]

    Turkergaze: Crowdsourcing saliency with webcam based eye tracking

    Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulka- rni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755, 2015

    2015 arXiv

  112. [121]

    Blood-based multi-tissue gene expression inference with bayesian ridge regression

    Wenjian Xu, Xuanshi Liu, Fei Leng, and Wei Li. Blood-based multi-tissue gene expression inference with bayesian ridge regression. Bioinformatics, 36(12):3788– 3794, 2020

    2020

  113. [122]

    Lsun: Con- struction of a large-scale image dataset using deep learning with humans in the loop

    Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Con- struction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015

    2015 arXiv

  114. [123]

    Unsupervised out-of-distribution detection by max- imum classifier discrepancy

    Qing Yu and Kiyoharu Aizawa. Unsupervised out-of-distribution detection by max- imum classifier discrepancy. In Proc. Int. Conf. Comput. Vision, pages 9518–9526, 2019. 107

    2019

  115. [124]

    Wide residual networks

    Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proc. British Mach. Vision Conf., pages 87.1–87.12, 2016

    2016

  116. [125]

    Wide residual networks

    Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proc. British Mach. Vision Conf., 2016

    2016

  117. [126]

    Advances in variational inference

    Cheng Zhang, Judith Bütepage, Hedvig Kjellström, and Stephan Mandt. Advances in variational inference. IEEE transactions on pattern analysis and machine intel- ligence, 41(8):2008–2026, 2018

    2008

  118. [127]

    Understanding deep learning requires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In Int. Conf. on Learning Representations, 2017

    2017

  119. [128]

    Recov- ery guarantees for one-hidden-layer neural networks

    Kai Zhong, Zhao Song, Prateek Jain, Peter L Bartlett, and Inderjit S Dhillon. Recov- ery guarantees for one-hidden-layer neural networks. In Int. Conf. Mach. Learning, pages 4140–4149, 2017

    2017

  120. [129]

    Variants of the greville formula with applications to exact recursive least squares

    Jie Zhou, Yunmin Zhu, X Rong Li, and Zhisheng You. Variants of the greville formula with applications to exact recursive least squares. SIAM journal on matrix analysis and applications, 24(1):150–164, 2002

    2002

  121. [130]

    שאיביואלדרפליישמןבית הספרלתאריםמתקדמיםע

    Huiping Zhuang, Zhiping Lin, and Kar-Ann Toh. Training a multilayer network with low-memory kernel-and-range projection. Journal of the Franklin Institute , 357(1):522–550, 2020. 108 A Appendix for chapter 2 109 A.1 Genie prediction upper bound The ridge regression ERM solutio...

    2020

  122. [2017]

    doi: 10.1109/ISIT.2017.8007079

    ISSN 21578095. doi: 10.1109/ISIT.2017.8007079

    2017

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.