Pith. sign in

REVIEW 3 major objections 6 minor 34 references

Boosting Adversarial Robustness and Generalization with Structural Prior

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Replacing convolutional layers with unrolled elastic dictionary-learning layers — a learnable mix of ℓ1 and ℓ2 reconstruction — boosts adversarial robustness and generalization, beating RobustBench leaders when stacked on standard…

desk verdict Consistent empirical gains from an elastic dictionary layer, but the influence-function theory analyzes a different operator than the network computes, and the leaderboard claims rest on re-implemented baselines. read the letter →

arxiv 2502.00834 v1 pith:LQJYYVVT submitted 2025-02-02 cs.LG cs.CRcs.NE

Zhichao Hou , Weizhi Gao , Hamid Krim , Xiaorui Liu This is my paper · ORCID
classification cs.LGcs.CRcs.NE
keywords adversarialrobustnessdictionarylearningstructuralpriorelasticRISTAinfluencefunctiontrainingrobustgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that adversarial robustness can be pushed past its current plateau not by more data or bigger models, but by building a structural prior — sparse dictionary learning — directly into the network architecture. It first shows that existing dictionary-learning-inspired CNNs give a false sense of security: they survive random noise but collapse to near-zero accuracy under adaptive PGD attacks, because their ℓ2 reconstruction term is highly sensitive to outlying values. To fix this, the authors propose Elastic Dictionary Learning (EDL), which replaces each convolutional layer with an unrolled reweighted iterative shrinkage-thresholding (RISTA) layer that approximately solves a dictionary-learning objective mixing ℓ1 and ℓ2 fidelity with a learnable per-layer balance β. Combined with standard adversarial training methods (PGD-AT, TRADES, PORT, HAT), EDL consistently improves both clean and robust accuracy across backbones, datasets, and attack norms — for instance raising AutoAttack (ℓ∞, 8/255) accuracy on CIFAR-10 ResNet-18 from 53.16% for HAT alone to roughly 59% — and the authors give an influence-function analysis explaining why the elastic mix is more robust than either extreme.

What carries the argument

The load-bearing machinery is the EDL layer: a convolutional layer replaced by $T$ unrolled steps of RISTA (reweighted iterative shrinkage-thresholding), which approximately solves the elastic dictionary-learning objective $\min_z \tfrac{\beta}{2}\|x - A^*(z)\|_2^2 + \tfrac{1-\beta}{2}\|x - A^*(z)\|_1 + \lambda\|z\|_1$, with update $z_{t+1} = T_{\lambda_t}(z_t + \tau_t A((\beta\mathbf{1} + (1-\beta)w_t) \odot (x - A^*(z_t))))$. The reweighting $w_t = 1/(2|x - A^*(z_t)|)$, justified by Lemma 4.1 as a localized convex upper bound on the non-smooth ℓ1 fidelity term, is what downweights pixels with large reconstruction residuals — precisely where adversarial perturbations concentrate — while the learnable per-layer $\beta$ interpolates between ℓ2 fidelity (preserving clean signal) and ℓ1 fidelity (rejecting outliers). The supporting identity is Theorem 4.2's influence-function calculation, which shows the elastic operator $(\beta\mathbf{1} + (1-\beta)w) \odot E(x)$ has sensitivity $(\beta\mathbf{1} + 2(1-\beta)\epsilon w^2) \odot E(\Delta - x)$, a linear blend of the vanilla and robust sensitivities; learning $\beta$ during adversarial training is the mechanism the paper claims adaptively picks the operating point on this spectrum.

What would settle it

Run an adaptive attack that backpropagates through the entire RISTA unrolling — all T steps, the soft-thresholding, the reweighting, and the learned β — on a CIFAR-10 EDLNet trained as described, and compare AutoAttack-ℓ∞ accuracy at 8/255 with the reported roughly 59% for HAT+EDL. If the accuracy collapses toward the near-zero level of the vanilla SDNet baseline, the elastic prior's apparent robustness is an artifact of the attack's approximation rather than a property of the architecture. A second, theory-level check: measure whether the relative influence-function gap between vanilla and elastic operators across layers correlates with the robustness gain those layers provide; absence of such a correlation would indicate the theorem is not carrying the argument.

Watch

Extended reading notes

Core claim

The central claim is that the fidelity term inside a dictionary-learning layer determines whether a learned network is genuinely robust or merely noise-tolerant. Vanilla dictionary learning uses ℓ2 reconstruction, which a small adversarial perturbation can exploit by injecting large outlying residuals; pure ℓ1 reconstruction rejects outliers but loses natural accuracy by suppressing legitimate signal. The paper's discovery is that an elastic objective, $\min_z \frac{\beta}{2}\|x - A^*(z)\|_2^2 + \frac{1-\beta}{2}\|x - A^*(z)\|_1 + \lambda\|z\|_1$ with a layer-wise learnable $\beta$, lets each layer find its own point on the accuracy–robustness trade-off, and that unrolling a reweighted ISTA algorithm for this objective yields a drop-in replacement for convolutional layers. Theorem 4.2 formalizes the mechanism: measured by influence functions, the elastic reconstruction operator has sensitivity $(\beta\mathbf{1} + 2(1-\beta)\epsilon w^2) \odot E(\Delta - x)$, a learnable interpolation between the vanilla sensitivity $E(\Delta - x)$ and the robust outlier-downweighted term $2\epsilon w^2 \odot E(\Delta - x)$, where $w = 1/(2(|E(x)|+\epsilon))$ suppresses large residuals. The paper further claims this structural prior is orthogonal to training-time defenses, so it stacks with PGD-AT, TRADES, PORT, and HAT to produce state-of-the-art results on the RobustBench leaderboard.

Load-bearing premise

The argument rests on the assumption that the influence-function sensitivity of a simplified single-step reconstruction operator — with the soft-thresholding step omitted — predicts how the full unrolled network behaves under strong adaptive attacks; the paper presents no quantitative bound, lemma, or experiment connecting that local sensitivity to PGD or AutoAttack accuracy.

Editorial extensions

If this is right

  • Stacked on the HAT defense, EDL raises AutoAttack (ℓ∞, 8/255) robustness on CIFAR-10 ResNet-18 from 53.16% to roughly 59%, and PORT+EDL likewise beats PORT alone — the structural prior is orthogonal to training-time defenses, not a replacement for them.
  • EDL shrinks robust overfitting: under PGD-based training the gap between best and final robust accuracy drops from 3.11 points (vanilla) to 0.72 points (Elastic DL), and switching on the elastic prior at epoch 150 recovers test robustness that vanilla dictionary learning loses.
  • The improvement transfers across capacity, data, and attack geometry: ResNet10/18/34/50 backbones, CIFAR-10, CIFAR-100, and Tiny-ImageNet, and under ℓ∞, ℓ2, and ℓ1 (SparseFool) budget measurements all show the same qualitative gain.
  • The influence-function result implies the ℓ1-only variant is the special case β = 0, which rejects outliers at the cost of roughly 10% natural accuracy; the learnable β is what lets EDL avoid that sacrifice while keeping the outlier rejection.
  • The added inference cost is modest — about 1–3× a standard ResNet depending on how many layers are replaced — so the robustness gain does not require a large compute multiplier.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence the paper leaves implicit: under larger attack budgets, the learned per-layer β should shift toward 0 (more ℓ1 weighting) in layers where adversarial residuals concentrate; plotting the trained β profile against attack budget would test the influence-function mechanism directly.
  • Because the paper only demonstrates stacking with PGD-AT, TRADES, PORT, and HAT, combining the elastic prior with synthetic-data defenses such as generative-model training data is an untested extension that the orthogonality argument suggests would pay off.
  • The theory covers a single-step, threshold-free operator rather than the full unrolled network, so the discriminating next stress test is an adaptive attack that backpropagates through all T steps including the soft-threshold and β; the reported evaluations cannot fully rule out that part of the gain comes from attack approximation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper argues that existing dictionary-learning-inspired convolutional layers (Vanilla DL) give a false sense of robustness and proposes Elastic Dictionary Learning Networks (EDLNets), which replace convolutional layers with unrolled RISTA iterations solving an elastic l1/l2 dictionary-learning objective (Eq. 12). The central empirical claim is that this architectural prior consistently improves both clean and adversarially robust accuracy when combined with existing adversarial training methods (PGD-AT, TRADES, PORT, HAT), e.g., AutoAttack l-infinity 8/255 accuracy of 59.07% on CIFAR-10 ResNet-18 for HAT+EDL versus 53.16% for HAT alone (Tables 4 and 5). A theoretical robustness analysis via influence functions (Section 4.4) is offered as support. The experiments cover CIFAR-10, CIFAR-100, and Tiny-ImageNet with multiple backbones and attack norms, plus ablations on overfitting, OOD robustness, and running time.

Significance. If the empirical pattern is correct, the paper would make a useful contribution: it demonstrates an architecture-level prior orthogonal to robust training, with modest computational overhead and consistent gains across models and datasets under the external AutoAttack evaluator. The paper deserves credit for using AutoAttack, for reporting a clear and falsifiable architecture change, and for measuring running time. However, the claimed theoretical support and the leaderboard-level claims are not yet at the standard required to establish the result: the influence-function analysis targets a simplified residual operator rather than the deployed layer, and the headline numbers lack seeds, error bars, and an official RobustBench submission.

major comments (3)
  1. [Section 4.4, Theorem 4.2] The influence-function analysis does not analyze the operator implemented in Algorithm 1. With T=1 and z0=A(x), the unthresholded RISTA output is A x + t A(E(x)) after one step, not E(x), and the soft-thresholding T_{lambda_t} is omitted. The operators P_vanilla, P_robust, and P_elastic are reconstruction-residual operators, not the hidden code z^{(l+1)} or the classifier output, and the theorem provides no bound, lemma, or experiment relating the influence function to PGD/AutoAttack accuracy or to the classification margin. As written, Section 4.4 therefore cannot support the paper's theoretical-robustness claims; it should either be repositioned as heuristic intuition or the theorem should be re-derived for the actual EDL layer output and linked to the classifier decision.
  2. [Section 4.1, Lemma 4.1 and the RISTA convergence claim] The localized upper bound U(z,z*) is undefined when any entry of x - A*(z*) is zero, because w = 1/(2|x - A*(z*)|) divides by zero; no epsilon regularization is introduced at that point (epsilon appears only in Theorem 4.2). More importantly, the claimed descent R(z_{t+1}) <= U(z_{t+1}, z_t) <= U(z_t, z_t) = R(z_t) does not follow from the algorithm. Equation (10) minimizes U + lambda ||z||_1, while Lemma 4.1 bounds R by U without the sparsity term, and Eq. (11) is a single proximal-gradient step rather than an exact minimizer of that objective; no step-size condition or proof is supplied to guarantee that U decreases. The asserted convergence guarantee for RISTA is therefore not established and needs a corrected proof.
  3. [Section 5.2, Tables 4-6, and the RobustBench claim] The claim of state-of-the-art performance on RobustBench is not yet substantiated. The EDL variants are not submitted to RobustBench, and the baseline numbers are either re-implemented under the authors' schedule or taken from other papers. No code, random seeds, confidence intervals, or standard deviations are reported, and hyperparameters essential to reproduce the architecture (lambda_t schedule, ISTA step size t_t, number of RISTA iterations T, epsilon in w_t, initialization of beta) are not specified. The tabulated gains are large and also contain internal inconsistencies (HAT+Elastic AutoAttack is 59.07 in Table 4 and 59.10 in Table 5), so the central empirical claim needs multi-seed results, hyperparameter sensitivity, and a reproducible leaderboard evaluation before it can be considered reliable.
minor comments (6)
  1. [Section 5.3 and tables] There are several typos and formatting errors: 'abation' should be 'ablation', Table 4 header 'robsustness' should be 'robustness', Tables 8-10 contain 'RESNWT' for 'ResNet', and Table 5 has run-together numbers such as '36.5322.17'.
  2. [Tables 4 and 5] The HAT+Elastic DL AutoAttack l-infinity/8/255 accuracy is reported as 59.07% in Table 4 and 59.10% in Table 5; these values should be reconciled.
  3. [Appendix D.3.4, Figures 17 and 18] The captions of Figures 17 and 18 refer to 'Figure 12' and 'Figure 13', apparently from an earlier draft; they should be relabeled.
  4. [Section 3.2, Table 1] The preliminary experiment evaluates SDNet18 without adversarial training, so the near-zero PGD accuracy is expected behavior for any standard-trained model. The text should state explicitly that these models are not adversarially trained, so the reader does not interpret near-zero PGD accuracy as a failure specific to dictionary-learning layers.
  5. [Section 5.1] The hyperparameter section does not give concrete values for the RISTA parameters (lambda_t, t_t, T, epsilon in w_t) or the initialization and regularization of the layer-wise balance weights beta^{(l)}; these should be listed to make the experiments reproducible.
  6. [Abstract and Section 1] The abstract's claim of 'surpassing the previous best defense PORT' is not consistent with Table 5, where HAT already outperforms PORT; the claim should be stated as applying to the EDL-enhanced variants rather than to the paper's method versus all prior defenses.

Circularity Check

1 steps flagged · score 3.0 of 10

The influence-function analysis restates the Elastic DL ansatz by construction, but the headline robustness gains are external AutoAttack measurements rather than fitted predictions.

  1. self definitional [Section 4.4, Theorem 4.2 and Algorithm 1]
    "For simplicity, we consider a single-step case for our RISTA algorithm (T = 1) and focus on the analysis of core part rt. ... Let the reconstruction operator is defined as E(·) := (I − A∗ ◦ A)(·), then the Vanilla, Robust, and Elastic DL operators are Pvanilla(x) = E(x), Probust(x) = w ⊙ E(x), and Pelastic(x) = (β1 + (1 − β)w) ⊙ E(x), where w = 1/(2(|E(x)| + ϵ)) ... IF (∆; Pelastic, x) = (β1 + 2(1 − β)ϵw2) ⊙ E(∆ − x)."

    The 'Elastic DL operator' P_elastic is introduced by definition inside Theorem 4.2, not derived from Algorithm 1, whose actual layer output is z_{t+1} = T_{λ_t}(z_t + t·A((β1 + (1−β)w_t) ⊙ (x − A∗(z_t)))). The theorem then differentiates that definitional weighting: because w = 1/(2(|E(x)| + ε)), the resulting IF necessarily carries the factor 2εw², which shrinks as |E(x)| grows. The claimed insight that large residuals are downweighted is exactly the definition of w, so the theoretical robustness analysis restates the ansatz rather than providing independent support. Moreover, no bound or experiment connects this IF to classification margin, end-to-end decision functions, or AutoAttack accuracy.

full rationale

The paper's main empirical claims are self-contained against external benchmarks: the headline numbers such as HAT+Elastic DL reaching 59.07% AutoAttack versus 53.16% for HAT alone (Table 5) are measured with AutoAttack and PGD, not fitted from the EDL objective, and no load-bearing self-citation chain supports them. The RISTA derivation in Lemma 4.1 and Eq. (11) is a standard majorization-minimization argument, so it is not circular. The one notable circular element is Theorem 4.2: the paper defines P_elastic as the weighted residual operator, computes its influence function, and then presents the downward weighting of large residuals—already built into the definition of w—as a derived robustness insight. Because the theorem analyzes a simplified operator rather than the actual thresholded RISTA output, and because no result links the influence function to classification robustness, the theoretical support is self-referential. The central empirical contribution remains independent, so a moderate score of 3 is appropriate rather than a higher score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the dictionary-learning prior, the outlier-downweighting hypothesis, and the unproven link from layer influence functions to end-to-end robustness. It also depends on several unreported hyperparameters (lambda_t, t_t, T, beta initialization, epsilon), which are free in the paper's own equations. No new physical entities are introduced.

free parameters (5)
  • Layer-wise balance weight beta^(l) = learned per layer; values not reported
    Eq. (12) and Algorithm 1 introduce a learnable beta per EDL layer to trade off l2 and l1 fidelity; its learned values are never reported, so the effective trade-off cannot be audited.
  • Sparsity threshold lambda / lambda_t = not specified for main experiments; tuned in preliminary SDNet study
    Appears in Eqs. (7), (8), (12) and in the shrinkage T_lambda_t; no schedule or value is given for the main results, despite being determinative of the sparse code.
  • ISTA step size t_t = not reported
    Algorithm 1 uses t_t (learning rate of the unrolled solver); no value or schedule is given, and the convergence proof (Appendix B.2) does not state the required step-size bound.
  • Number of RISTA iterations T = implied to be 3 from Figure 6, not stated in Algorithm 1
    The unrolled layer depth T controls approximation accuracy and cost; Table 7 varies the number of replaced layers but not T.
  • Stabilizer epsilon in influence weight w = not quantified
    Theorem 4.2 defines w = 1/(2(|E(x)|+epsilon)) to avoid zero residual; the IF for Robust and Elastic DL scales with epsilon, so the qualitative conclusion (Robust DL less sensitive) depends on the unstated value of epsilon.
assumptions (5)
  • domain assumption A natural image signal can be represented as x = A*(z) with a sparse code z over a learned convolutional dictionary A, and every convolutional layer can be replaced by such a dictionary layer.
    Sections 3 and 5 replace all conv layers in ResNet with EDL layers; if this prior is a poor fit for the data distribution, the architecture has no extra robustness mechanism.
  • ad hoc to paper Adversarial perturbations appear as outlying values in the reconstruction residual, so reweighting or downweighting large residuals with l1 or elastic fidelity improves robustness.
    This is the central causal hypothesis of Section 4; the paper provides no empirical test isolating whether the gain comes from outlier downweighting rather than from extra capacity, iterative inference, or the learnable beta.
  • ad hoc to paper The influence function of a single-step, threshold-free reconstruction operator is a faithful proxy for end-to-end adversarial robustness of the trained classifier.
    Theorem 4.2 computes IFs for E(x), w*E(x), and (beta*1+(1-beta)*w)*E(x), but never binds these to classification accuracy under PGD or AutoAttack; no lemma or experiment bridges layer sensitivity to network robustness.
  • standard math The RISTA iteration decreases the elastic objective and the unrolled network can be trained end-to-end by standard backpropagation.
    Appendix B.2 gives a descent argument for a convex surrogate, but the proof omits explicit step-size conditions and treats the loss descent of U(z_{t+1},z_t) as automatic; training sensitivity of the unrolled solver is assumed.
  • domain assumption Standard adversarial training with PGD/AutoAttack on the combined method (e.g., HAT+ElasticDL) converges to a solution that retains robustness, and the reported attacks are sufficient to evaluate the defense (no BPDA/EOT needed).
    Section 5 evaluates with PGD, FGSM, C&W, and AutoAttack but never discusses defense-aware attacks for the non-smooth unrolled layers; if gradients through the iterative solver mislead AutoAttack, the reported robustness could be inflated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Boosting Adversarial Robustness and Generalization with Structural Prior." pith.science (2026). https://pith.science/paper/LQJYYVVT

@misc{pith2026250200834,
  author       = {Pith},
  title        = {Pith review of: Boosting Adversarial Robustness and Generalization with Structural Prior},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LQJYYVVT}},
  note         = {Machine review of arXiv:2502.00834}
}
read the original abstract

This work investigates a novel approach to boost adversarial robustness and generalization by incorporating structural prior into the design of deep learning models. Specifically, our study surprisingly reveals that existing dictionary learning-inspired convolutional neural networks (CNNs) provide a false sense of security against adversarial attacks. To address this, we propose Elastic Dictionary Learning Networks (EDLNets), a novel ResNet architecture that significantly enhances adversarial robustness and generalization. This novel and effective approach is supported by a theoretical robustness analysis using influence functions. Moreover, extensive and reliable experiments demonstrate consistent and significant performance improvement on open robustness leaderboards such as RobustBench, surpassing state-of-the-art baselines. To the best of our knowledge, this is the first work to discover and validate that structural prior can reliably enhance deep learning robustness under strong adaptive attacks, unveiling a promising direction for future research.

Figures

Figures reproduced from arXiv: 2502.00834 by the authors.

Figure 1
Figure 1. Overview of Elastic DL Networks (EDLNets). EDLNets are constructed by replacing the convolutional layers in conventional backbones (e.g., ResNets) with EDL layers that are unrolled with the proposed efficient RISTA algorithm. Each EDL layer introduces a dictionary structural prior, assuming the input signal z (l) is encoded as a sparse code z (l+1) using a few atoms from diction A(l) . The statement (1) indicates th… view at source ↗
Figure 2
Figure 2. Test robust accuracy during the adversarial training. we pretrain the Vanilla DL model for 150 epochs and fine-tune the Elastic DL model starting from 150-th epoch. Our Elastic DL method can achieve the best adversarial robustness. SOTA performance on leaderboard. Furthermore, we validate whether incorporating our structural prior improves over state-of-the-art methods. To achieve this, we select the top-ranking met… view at source ↗
Figure 3
Figure 3. Adversarial robustness under various settings. Our Elastic DL outperforms Vanilla DL across various datasets (CIFAR10 / CIFAR100 / Tiny-ImageNet), backbones (ResNet10 / ResNet18 / ResNet34 / ResNet50) and attacks (PGD / FGSM / CW / AA) [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (13 more)
Figure 5
Figure 5. Figure 5: Embedding differ￾ence. Our Elastic DL shows smaller embedding difference than Vanilla DL [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Algorithm convergence. RISTA algorithm achieves fast convergence within just three steps [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 9
Figure 9. Figure 9: Overview of Elastic DL neural networks in adversarial training. Elastic DL neural networks consist of multiple stacked Elastic DL (EDL) layers. During the forward pass, the input x is fed into the model, generating a series of hidden codes {z (l) } L l=1 through EDL la…
Figure 10
Figure 10. Figure 10: Exploded view of Elastic DL (EDL) layer. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Performance of SDNet18 (Vanilla DL) under random Impulse noise with different levels. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Adversarial training curve of our Elastic DL. During the 100th to 150th epochs, the model experiences a catastrophic robust overfitting problem. By introducing the Elastic DL structural prior at the 150th epoch and fine-tuning, we effectively mitigate overfitting and …
Figure 13
Figure 13. Figure 13: Training curves of baselines. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Comparison of training curves of all methods. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Different adversarial training. Our Elastic DL is orthogonal to existing adversarial training methods and can be combined with them to further improve the performance. D.3.3. DIFFERENT BUDGET MEASUREMENT In addition to ℓ∞-norm attack (PGD-ℓ∞), we also validate the con…
Figure 16
Figure 16. Figure 16: Different attack measurements. Our Elastic DL consistently outperforms Vanilla DL across attacks (PGD-ℓ∞, PGD-ℓ2, SparseFool) evaluated under various metrics (ℓ∞, ℓ2, ℓ0 norms) [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 12
Figure 12. Figure 12: Hidden embedding visualization (Part 1). 22 [PITH_FULL_IMAGE:figures/full_fig_p025_12.png]
Figure 13
Figure 13. Figure 13: Hidden embedding visualization (Part 2). 23 [PITH_FULL_IMAGE:figures/full_fig_p026_13.png]
Figure 19
Figure 19. Figure 19: Reconstruction process. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_19.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 19 canonical work pages

  1. [1]

    • Step 2 (Forward): input x′ as z∗(0) into model to obtain a series of hidden codes for each layer{z(l)}L l=1 by optimizing dictionary learning loss in Eq

    Its overall pipeline can be divided into three main steps as in Figure 9: • Step 1 (Attack): leverage adversarial attack algorithm (e.g., PGD) to generate worst-case perturbation x′. • Step 2 (Forward): input x′ as z∗(0) into model to obtain a series of hidden codes for each layer{z(l)}L l=1 by optimizing dictionary learning loss in Eq. (12). • Step 3 (Ba...

    work page 2024

  2. [2]

    Robustbench: a standardized adversarial robustness benchmark

    Croce, F., Andriushchenko, M., Sehwag, V ., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., and Hein, M. Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670,

    arXiv 2010

  3. [3]

    Improved regularization of convolutional neural networks with cutout

    DeVries, T. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552,

    arXiv

  4. [5]

    J., Shlens, J., and Szegedy, C

    Goodfellow, I. J., Shlens, J., and Szegedy, C. Explain- ing and harnessing adversarial examples. arXiv preprint arXiv:1412.6572,

    arXiv

  5. [9]

    Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks

    Liu, Z., Duan, H., Liang, H., Long, Y ., Snasel, V ., Nicosia, G., Ranjan, R., and Ojha, V . Dynamic label adversarial training for deep learning robustness against adversarial attacks. arXiv preprint arXiv:2408.13102,

    work page Pith review arXiv

  6. [10]

    13 Submission and Formatting Instructions for ICML 2024 B

    Exploded view of Elastic DL (EDL) layer. 13 Submission and Formatting Instructions for ICML 2024 B. Theoretical Proof B.1. Proof of Lemma 4.1 Proof. Since √a ≤ a 2 √ b + √ b 2 and the equlity holds when a = b, by replacemnet as a = (x[i, j, c] − A∗(z)[i, j, c])2 and b = (x[i, j, c] − A∗(z∗)[i, j, c])2, then |x[i, j, c] − A∗(z)[i, j, c]| ≤1 2 · 1 |x[i, j, ...

    work page 2024

  7. [11]

    19 Submission and Formatting Instructions for ICML 2024 D.2

    Performance of SDNet18 (Vanilla DL) under random Impulse noise with different levels. 19 Submission and Formatting Instructions for ICML 2024 D.2. Adversarial Training Curves D.2.1. T RAINING CURVES OF EACH METHOD Training curve of our Elastic DL. From Figure 12, we can observe that during the 100th - 150th epochs, the Vanilla DL model exhibits a severe r...

    work page 2024

  8. [12]

    During the 100th to 150th epochs, the model experiences a catastrophic robust overfitting problem

    Adversarial training curve of our Elastic DL. During the 100th to 150th epochs, the model experiences a catastrophic robust overfitting problem. By introducing the Elastic DL structural prior at the 150th epoch and fine-tuning, we effectively mitigate overfitting and achieve significantly improved robustness and generalization. 20 Submission and Formattin...

    work page 2024

Show all 34 references
  1. [13]

    21 Submission and Formatting Instructions for ICML 2024 D.2.2

    Training curves of baselines. 21 Submission and Formatting Instructions for ICML 2024 D.2.2. C OMPARISON OF ALL METHODS To make a comparison of all the methods, we compare the natural and robust performance in the training and testing dataset through the training curve in Figure

    2024

  2. [14]

    22 Submission and Formatting Instructions for ICML 2024 D.3

    Comparison of training curves of all methods. 22 Submission and Formatting Instructions for ICML 2024 D.3. Ablation Studies D.3.1. U NIVERSALITY Universality across various backbones, datasets and attacks. We conduct ablation studies on different backbones, datasets, and attac...

    2024

  3. [15]

    Online adversarial purification based on self-supervision

    10 Submission and Formatting Instructions for ICML 2024 Shi, C., Holtz, C., and Mishne, G. Online adversarial purification based on self-supervision. arXiv preprint arXiv:2101.09387,

    2024 arXiv

  4. [16]

    Dropout: a simple way to prevent neural networks from overfitting

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958,

    1929

  5. [18]

    Robust sparse coding for face recognition

    Yang, M., Zhang, L., Yang, J., and Zhang, D. Robust sparse coding for face recognition. In CVPR 2011, pp. 625–632. IEEE,

    2011

  6. [19]

    Adversarially robust generalization just requires more unlabeled data

    Zhai, R., Cai, T., He, D., Dan, C., He, K., Hopcroft, J., and Wang, L. Adversarially robust generalization just requires more unlabeled data. arXiv preprint arXiv:1906.00555,

    1906 arXiv

  7. [20]

    mixup: Beyond empirical risk minimization

    Zhang, H. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412,

    arXiv

  8. [21]

    Background subtrac- tion via robust dictionary learning

    Zhao, C., Wang, X., and Cham, W.-K. Background subtrac- tion via robust dictionary learning. EURASIP Journal on Image and Video Processing, 2011:1–12,

    2011

  9. [22]

    Overview of Elastic Dictionary Learning Overview of Elastic DL neural networks

    11 Submission and Formatting Instructions for ICML 2024 A. Overview of Elastic Dictionary Learning Overview of Elastic DL neural networks. Here we plot a figure to show the overall pipeline of incorporating Elastic DL structural prior into adversarial training as in Figure

    2024

  10. [25]

    Proof. For convex objective: f (z) = β 2 ∥x − A∗(z)∥2 2 + 1 − β 2 ∥(w(t))1/2 ⊙ (x − A∗(z)) ∥2 2, we can achieve the optima via the first-order gradient descent: zt+1 = zt − t∇f (zt), or equivalently, zt+1 = arg min z {f (zt) + ⟨z − zt, ∇f (zt)⟩ + 1 2t ∥z − zt∥2}. Then, for the...

    2024

  11. [26]

    have proven highly effective against adaptive adversarial attacks, consistently leading the robustness leaderboard (RobustBench) (Croce et al., 2020). Despite their success, most existing methods rely heavily on extensive synthetic training data generated by advanced models, l...

    2024

  12. [31]

    Adversarial robsustness on Tiny-Imagenet with different backbones. METHOD NATURAL PGD FGSM C&W AA VANILLADL + RESNWT10 49.6 27.17 32.46 37.91 20.20 ELASTICDL + RESNET10 50.12 32.93 39.64 40.10 24.90 VANILLADL + RESNWT18 50.22 31.45 36.46 39.02 30.90 ELASTICDL + RESNET18 50.52 ...

    2024

  13. [32]

    Adversarial robustness on CIFAR10 with different budget measurements. PGD∥ · ∥∞\BUDGET 0 2/255 4/255 8/255 12/255 16/255 32/255 VANILLADL + RESNWT18 83.29 75.86 66.52 45.66 27.5 15.48 2.89 PGD-AT+ EDL - RESNET18 83.57 78.76 71.01 53.29 41.1 34.13 23.84 PGDL2∥ · ∥22 \BUDGET 0 0...

    2024

  14. [33]

    25 Submission and Formatting Instructions for ICML 2024 Here are instances of BIRD, HORSE, AIRPLANE, DEER and DOG: 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 12...

    2024

  15. [34]

    R ECONSTRUCTION PROCESS Image & noise reconstruction

    26 Submission and Formatting Instructions for ICML 2024 D.3.5. R ECONSTRUCTION PROCESS Image & noise reconstruction. In conventional feedforward neural networks, adding a perturbation ϵ to the input can lead the model to make incorrect predictions. However, as illustrated in F...

    2024

  16. [1996]

    B., and Swami, A

    Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., and Swami, A. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications secu- rity, pp. 506–519,

    2017

  17. [2009]

    Crafting papers on machine learning

    Langley, P. Crafting papers on machine learning. In Langley, P. (ed.),Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207–1216, Stan- ford, CA,

    2000

  18. [2010]

    doi: 10.1109/TIT.2010

    ISSN 0018-9448. doi: 10.1109/TIT.2010. 2048473. URL https://doi.org/10.1109/TIT. 2010.2048473. Wright, J., Yang, A. Y ., Ganesh, A., Sastry, S. S., and Ma, Y . Robust face recognition via sparse representation. IEEE transactions on pattern analysis and machine intelligence, 31...

    2010

  19. [2013]

    Towards deep learning models resistant to adver- sarial attacks

    Madry, A. Towards deep learning models resistant to adver- sarial attacks. arXiv preprint arXiv:1706.06083,

    arXiv

  20. [2016]

    Diffusion models for adversarial purifi- cation

    Nie, W., Guo, B., Huang, Y ., Xiao, C., Vahdat, A., and Anandkumar, A. Diffusion models for adversarial purifi- cation. arXiv preprint arXiv:2205.07460,

    arXiv

  21. [2017]

    R., Shintre, S., and Gardner, A

    Feinman, R., Curtin, R. R., Shintre, S., and Gardner, A. B. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410,

    arXiv

  22. [2018]

    Robust learning meets genera- tive models: Can proxy distributions improve adversarial robustness? arXiv preprint arXiv:2104.09425,

    Sehwag, V ., Mahloujifar, S., Handina, T., Dai, S., Xiang, C., Chiang, M., and Mittal, P. Robust learning meets genera- tive models: Can proxy distributions improve adversarial robustness? arXiv preprint arXiv:2104.09425,

    arXiv

  23. [2019]

    H., Genewein, T., Fischer, V ., and Bischoff, B

    Metzen, J. H., Genewein, T., Fischer, V ., and Bischoff, B. On detecting adversarial perturbations. arXiv preprint arXiv:1702.04267,

    arXiv

  24. [2020]

    and Wagner, D

    Carlini, N. and Wagner, D. Towards evaluating the robust- ness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39–57. Ieee,

    2017

  25. [2021]

    On the (statistical) detection of adversarial examples

    Grosse, K., Manoharan, P., Papernot, N., Backes, M., and McDaniel, P. On the (statistical) detection of adversarial examples. arXiv preprint arXiv:1702.06280,

    arXiv

  26. [2022]

    Robust graph neural networks via unbiased aggregation, 2024a

    Hou, Z., Feng, R., Derr, T., and Liu, X. Robust graph neural networks via unbiased aggregation, 2024a. URL https://arxiv.org/abs/2311.14934. 9 Submission and Formatting Instructions for ICML 2024 Hou, Z., Gao, W., Shen, Y ., Wang, F., and Liu, X. Pro- transformer: Robustify tr...

    2024 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.