Pith. sign in

REVIEW 4 major objections 6 minor 34 references

On Hardening DNNs against Noisy Computations

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training with the same additive Gaussian noise analog hardware produces during inference is a stronger defense than quantization-aware training, especially for deep networks.

desk verdict A genuinely novel empirical comparison of constant-scaling QAT versus noisy training for analog-noise robustness, but the central ranking rests on one unvalidated noise model and the paper ships no artifacts. read the letter →

arxiv 2501.14531 v1 pith:A3SE3TGE submitted 2025-01-24 cs.LG

Xiao Wang , Hendrik Borras , Bernhard Klein , Holger Fröning This is my paper · ORCID
classification cs.LG
keywords analognoiserobustnessquantizationquantization-awaretrainingnoisymidpointlevelCIFAR-10deepneuralnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that for deep neural networks running on noisy analog hardware, the most dependable way to keep accuracy high is to train the network in the presence of the same additive Gaussian noise it will encounter during inference. It also claims that quantization-aware training with constant scaling factors improves tolerance to such noise, but at a noticeable cost in clean accuracy, and that this benefit disappears once noisy training is used. If correct, the practical implication is that hardware developers should match training-time noise to inference-time noise rather than rely on quantization as the main robustness mechanism.

What carries the argument

The central object is the midpoint noise level $\mu$, a metric that fits a scaled and shifted logistic function to the accuracy-versus-noise curve and reports the noise level at which accuracy drops halfway; larger $\mu$ means more robust. The mechanism that produces the results is uniform quantization with a constant scaling factor $s$, implemented through quantization-aware training with straight-through gradient estimation, plus a noise-injection module added after each layer that applies forward-path Gaussian noise matching the noise used during inference when noisy training is enabled.

What would settle it

Run the same quantization and noisy-training recipes on a real analog accelerator, such as a resistive-memory or photonic chip, with its native noise measured, and compare which model keeps accuracy at the device's operating noise; if noisy training stops winning under weight-dependent or multiplicative noise, the paper's ranking does not carry over to that hardware.

Watch

Extended reading notes

Core claim

On CIFAR-10, across LeNet-5, VGG-11, and ResNet-18, the authors measure robustness by the midpoint noise level $\mu$, the injected noise standard deviation at which validation accuracy falls halfway from its maximum to random chance. They find that quantized models with dynamic scaling are less robust than full-precision models, while constant scaling factors shift the accuracy-versus-noise curve to the right at the price of lower peak accuracy. Pairing quantization-aware training with noisy training yields almost no additional robustness over noisy training alone, yet quantized noisy-trained models match the robustness of floating-point noisy-trained models, so smaller models can be deployed without losing noise tolerance.

Load-bearing premise

The load-bearing premise is that adding the same strength of additive Gaussian noise to every layer's activations is a faithful proxy for the noise of real analog accelerators.

Editorial extensions

If this is right

  • Quantization-aware training with constant scaling factors raises the midpoint noise level of LeNet-5, VGG-11, and ResNet-18 compared with dynamic scaling or full precision, at the cost of lower peak accuracy.
  • Noisy training with training noise strength equal to inference noise outperforms quantization-aware training alone on all tested architectures, and the gap grows for deeper networks.
  • Adding quantization-aware training on top of noisy training does not meaningfully improve robustness, but quantized noisy-trained models keep the same robustness as floating-point noisy-trained models, making quantized deployment nearly free in robustness once noisy training is used.
  • Deeper architectures are more vulnerable to error amplification under quantization, with VGG-11 collapsing at a scaling factor of 3, while ResNet-18's skip connections reduce error accumulation and improve robustness.
  • Dynamic scaling quantization is consistently the least robust option, performing worse than full-precision models in every architecture tested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the equal-strength matching of training and inference noise may be overly strict; a sweep of training noise levels above and below the inference level could reveal a cheaper or more robust schedule than exact matching, which the paper does not test.
  • Inference: if real device noise is multiplicative or weight-dependent, as memristor drift models suggest, the ranking of defenses found here could change, because quantization's clipped and bounded activations might help more under multiplicative noise than under additive noise.
  • Inference: the result suggests a practical design rule: treat quantization as a compression step that is nearly free in robustness once noisy training is applied, and focus engineering effort on measuring and reproducing the accelerator's noise profile during training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper empirically compares two strategies for making DNNs robust to noisy analog computations: quantization-aware training (QAT) with constant or dynamic activation scaling, and noisy training with Gaussian activation-noise injection. Using the midpoint noise level μ from the authors' prior work, they evaluate LeNet-5, VGG-11, and ResNet-18 on CIFAR-10. They report that constant-scaling QAT improves robustness relative to an fp32 baseline and dynamic scaling, that noisy training leads to substantially higher μ, and that quantization does not further improve already noisy-trained models. The central claims are that QAT with constant scaling enhances robustness and that noisy training is the superior robustness strategy, especially for deeper architectures.

Significance. The questions addressed here are practically relevant: if the results hold, they provide direct guidance for deploying DNNs on analog accelerators and for choosing between quantization-based and noise-injection-based defenses. The paper has several strengths: it uses a quantitative robustness metric, compares three architectures with different depths, sweeps bit widths and scaling factors, and reports both accuracy and robustness so that trade-offs are at least partially visible. The main limitation is that the entire comparison is conducted under one synthetic noise model—zero-mean i.i.d. Gaussian noise added to activations with identical strength at all layers—and the training/inference noise matching for noisy training is fixed to exact equality. The empirical ranking may therefore be an artifact of this specific perturbation and of the matched-noise evaluation protocol. The paper does not release code or trained models, which limits reproducibility of the quantitative claims.

major comments (4)
  1. [Section IV, first paragraph; Section II] The central comparison rests on a single synthetic noise model: zero-mean i.i.d. Gaussian noise added to every layer's activations, with the same standard deviation at all layers and with training noise exactly equal to inference noise. The paper's own Related Work describes alternative hardware noise models—Noisy Machines uses additive Gaussian weight noise, and BayesFT uses multiplicative drift w' = w·e^λ—so the sentence 'As additive noise is often the primary type of noise in accelerators' is an unsupported assertion, not a demonstrated premise. The ranking of QAT versus noisy training could plausibly reverse under weight-dependent, multiplicative, or layer-heterogeneous noise. Please add experiments with at least one weight-noise model (e.g., the BayesFT multiplicative drift) and with a sweep over training-noise intensity (σ_training = c·σ_inference for several c, including c=0 and c=2), or explicitly and prominently restrict the conclusions to the case of matched additive activation noise.
  2. [Section IV-B, Fig. 3] The headline comparison is not apples-to-apples: noisy training is evaluated under the exact noise distribution and intensity used during training (σ_training = σ_inference), whereas the QAT models are given no advantage of being trained under the evaluation noise. This makes the statement 'noisy training emerges as the superior approach' a statement about a specially privileged evaluation condition rather than about general robustness. The paper should state this as a boundary condition and report how the ranking changes when the training noise intensity is not perfectly matched, for example by training with a smaller or larger σ_training and evaluating across the same noise range.
  3. [Section IV-A1, Figs. 2 and 3] The key figures contain no error bars or confidence intervals, even though Section IV-A1 states that each inference is repeated 10 times on different randomly initialized networks and Eq. (2) defines a fit with uncertainty. This matters for the central conclusion: the LeNet-5 differences between the fp32 baseline (μ=0.286) and dynamic scaling (μ=0.242, μ=0.233) are small, and the 'closely matched, almost overlapping' dashed curves in Fig. 3 are used to support the negative claim that quantization does not improve noisy-trained models. Without uncertainty estimates, these comparisons are not quantitatively supported. Please plot confidence bands or report fit uncertainties for every curve in Figs. 2 and 3.
  4. [Table I and Fig. 2(c)] The selection of scaling factors is not systematic. For LeNet-5 the paper sweeps s ∈ {0.5, 2, 8, 64, 512}; for VGG-11 and ResNet-18 the tested sets are different and smaller, and the text does not state a criterion for choosing 'representative' configurations when making claims such as 'constant scaling enhances robustness' or '8-bit outperforms 4-bit'. A fair cross-architecture comparison should use the same candidate scaling-factor grid for all architectures and report the entire Pareto frontier of peak accuracy versus μ, as is done only for LeNet-5 in Fig. 2(c). As written, the comparison reflects the choice of scaling-factor grid as much as the methods themselves.
minor comments (6)
  1. [Abstract and Section I] The abstract says 'both two methods increase tolerance against noise,' but the dynamic-scaling QAT results in Fig. 2 have lower μ than the fp32 baseline; please qualify the claim to constant-scaling QAT and noisy training.
  2. [Section IV-A1] The statement 'Experiments with models quantized to 16-bit did not yield better results than 8-bit' is unsupported by any figure or table; please include the data or remove the claim.
  3. [Table II] Table II lists 'Midpoint noise level μ w/o Noisy Training' values that appear to be the fp32 baselines (e.g., VGG-11 0.154), but the caption does not clarify that these are baseline values rather than QAT values, and the table does not include the best QAT configurations from Table I. Please clarify the table's content or add the QAT rows so the overall comparison is complete.
  4. [Section III-A, Eq. (1)] The symbol δa is used both in the definition 'δa = (amax - amin)/2' and as a free parameter in Eq. (1); please use distinct notation to avoid ambiguity.
  5. [Throughout] There are several typos and small errors, including 'quanitzation,' 'extend,' 'is has been shown,' and 'it is shown previously that that quantization'; please proofread.
  6. [Section IV] No code, configuration files, or trained models are released; please provide at least the Brevitas configuration and hyperparameter details to make the experiments reproducible.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central QAT-versus-noisy-training comparison is an empirical measurement, and the self-cited robustness metric is fully defined in-paper.

full rationale

The central claims—that QAT with constant scaling factors enhances robustness and that noisy training is superior—are empirical comparisons on CIFAR-10 using standard architectures (LeNet-5, VGG-11, ResNet-18). The only self-citation that could raise circularity concerns is the midpoint noise level µ from the authors' prior work [3]. However, the paper defines µ explicitly in Eqs. (1)-(2) as the fitted midpoint of a logistic accuracy-vs-noise curve; it is a measurement device, not a hidden input. The comparison of µ values across methods summarizes directly measured accuracy curves and does not presuppose the ranking it reports. No fitted parameter is renamed as a prediction: the logistic fit is descriptive, and the noisy-training result follows from the measured curves under the explicitly stated assumption that training noise matches inference noise (σ_training = σ_inference). The noise model itself (global i.i.d. additive Gaussian activation noise) is an explicit assumption in Section IV, not a conclusion derived from the experiments. A potential confound is that constant scaling factors change activation magnitudes and therefore the signal-to-noise ratio under fixed-variance injected noise, but the paper does not define robustness in terms of scaling, and the observed effect is empirical rather than definitional. The concern that the noise model may not represent all real analog hardware is a correctness/scope risk, not a circularity. Thus the paper is self-contained for its evaluation; the self-citation is minor and not load-bearing.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new estimators, losses, or physical entities. It relies on existing concepts: uniform quantization (Brevitas), Gaussian activation-noise injection (Walking Noise), and the logistic-fit midpoint metric (also from the authors' prior work). The central findings depend on the chosen scaling factors and bit widths as free parameters, and on the assumption that synthetic Gaussian activation noise reflects real analog hardware.

free parameters (2)
  • Constant activation scaling factor s = Values tested: 0.5, 1, 2, 3, 4, 8, 10, 64, 512
    Chosen by hand to demonstrate a robustness-accuracy trade-off; the conclusion that constant scaling enhances robustness depends on choosing these values.
  • Bit width b = 4, 8, 16
    Models are quantized to 4-bit and 8-bit (16-bit did not improve); the comparison between 4-bit and 8-bit is central to the robustness claim.
assumptions (4)
  • domain assumption Additive Gaussian noise is the primary noise type in analog accelerators.
    Section IV states 'As additive noise is often the primary type of noise in accelerators' and uses Gaussian activation noise for all experiments.
  • domain assumption Injecting noise at the output activations of each layer models combined weight-readout and computation noise.
    Section II attributes this model to Walking Noise [3]; Section IV applies a custom noise module after each layer.
  • domain assumption Fake quantization with STE represents hardware quantization during training.
    Section III-B uses Brevitas fake quantization and the straight-through estimator for backpropagation.
  • domain assumption LeNet-5, VGG-11 and ResNet-18 on CIFAR-10 are representative enough to draw general conclusions about DNN noise robustness.
    Section IV uses only these three architectures and one dataset.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Hardening DNNs against Noisy Computations." pith.science (2026). https://pith.science/paper/A3SE3TGE

@misc{pith2026250114531,
  author       = {Pith},
  title        = {Pith review of: On Hardening DNNs against Noisy Computations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A3SE3TGE}},
  note         = {Machine review of arXiv:2501.14531}
}
read the original abstract

The success of deep learning has sparked significant interest in designing computer hardware optimized for the high computational demands of neural network inference. As further miniaturization of digital CMOS processors becomes increasingly challenging, alternative computing paradigms, such as analog computing, are gaining consideration. Particularly for compute-intensive tasks such as matrix multiplication, analog computing presents a promising alternative due to its potential for significantly higher energy efficiency compared to conventional digital technology. However, analog computations are inherently noisy, which makes it challenging to maintain high accuracy on deep neural networks. This work investigates the effectiveness of training neural networks with quantization to increase the robustness against noise. Experimental results across various network architectures show that quantization-aware training with constant scaling factors enhances robustness. We compare these methods with noisy training, which incorporates a noise injection during training that mimics the noise encountered during inference. While both two methods increase tolerance against noise, noisy training emerges as the superior approach for achieving robust neural network performance, especially in complex neural architectures.

Figures

Figures reproduced from arXiv: 2501.14531 by the authors.

Figure 1
Figure 1. Global noise injection in a quantized neural network. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Robustness of LeNet-5 on CIFAR-10, quantized with different bit widths, using either constant scaling (scaling factors [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Robustness of LeNet-5 and ResNet-18 on CIFAR-10 with noisy training, combined with quantization. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 26 canonical work pages

  1. [1]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville. “Estimating or propagating gradients through stochastic neurons for conditional computation”. In: ArXiv abs/1308.3432 (2013)

    arXiv 2013

  2. [2]

    Precise neural network computation with imprecise analog devices

    Jonathan Binas et al. “Precise deep neural network computation on imprecise low-power analog hardware”. In: ArXiv abs/1606.07786 (2016)

    work page Pith review arXiv 2016

  3. [3]

    Walking Noise: On Layer-Specific Robustness of Neu- ral Architectures against Noisy Computations and Asso- ciated Characteristic Learning Dynamics

    Hendrik Borras, Bernhard Klein, and Holger Fr ¨oning. “Walking Noise: On Layer-Specific Robustness of Neu- ral Architectures against Noisy Computations and Asso- ciated Characteristic Learning Dynamics”. In: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases . ECML- PKDD. 2024

    work page 2024

  4. [4]

    Probabilistic Photonic Computing with Chaotic Light

    Frank Br ¨uckerhoff-Pl¨uckelmann et al. “Probabilistic Photonic Computing with Chaotic Light”. In: CoRR abs/2401.17915 (2024). DOI: 10 . 48550 / arXiv. 2401 . 17915

    work page Pith review arXiv doi:10.48550/arxiv.2401.17915 2024

  5. [5]

    Robust quantization: One model to rule them all

    Brian Chmiel et al. “Robust quantization: One model to rule them all”. In: Advances in Neural Information Processing Systems . V ol. 33. NeurIPS. 2020. DOI: 10. 5555/3495724.3496170

    arXiv 2020

  6. [6]

    Exploring the impact of random tele- graph noise-induced accuracy loss on resistive RAM- based deep neural network

    Yide Du et al. “Exploring the impact of random tele- graph noise-induced accuracy loss on resistive RAM- based deep neural network”. In: IEEE Transactions on Electron Devices 67 (2020)

    work page 2020

  7. [7]

    Relative robustness of quantized neural networks against adversarial attacks

    Kirsty Duncan et al. “Relative robustness of quantized neural networks against adversarial attacks”. In: 2020 International Joint Conference on Neural Networks . IJCNN. IEEE. 2020

    work page 2020

  8. [8]

    Implications of Noise in Resistive Memory on Deep Neural Networks for Image Classification

    Yannick Emonds, Kai Xi, and Holger Fr ¨oning. “Im- plications of Noise in Resistive Memory on Deep Neural Networks for Image Classification”. In: CoRR abs/2401.05820 (2024). DOI: 10.48550/ARXIV.2401. 05820

    work page Pith review arXiv doi:10.48550/arxiv.2401.05820 2024

Show all 34 references
  1. [9]

    A survey of quantization methods for efficient neural network inference

    Amir Gholami et al. “A survey of quantization methods for efficient neural network inference”. In: Low-Power Computer Vision. Chapman and Hall/CRC, 2022

    2022

  2. [10]

    How many bits does it take to quantize your neural network?

    Mirco Giacobbe, Thomas A Henzinger, and Mathias Lechner. “How many bits does it take to quantize your neural network?” In: Tools and Algorithms for the Construction and Analysis of Systems . TACAS. Springer. 2020. DOI: 10.1007/978-3-030-45237-7 5

    2020 doi

  3. [11]

    On the adversarial robustness of quantized neural net- works

    Micah Gorsline, James Smith, and Cory Merkel. “On the adversarial robustness of quantized neural net- works”. In: Great Lakes Symposium on VLSI . 2021

    2021

  4. [12]

    Deep residual learning for image recognition

    Kaiming He et al. “Deep residual learning for image recognition”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition . CVPR. 2016

    2016

  5. [13]

    Neural network compression for noisy storage devices

    Berivan Isik et al. “Neural network compression for noisy storage devices”. In: ACM Transactions on Em- bedded Computing Systems 22 (2023)

    2023

  6. [14]

    Accurate deep neural network infer- ence using computational phase-change memory

    Vinay Joshi et al. “Accurate deep neural network infer- ence using computational phase-change memory”. In: Nature Communications 11 (2020)

    2020

  7. [15]

    Adam: A Method for Stochastic Optimization

    Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: International Confer- ence on Learning Representations . ICLR (2014)

    2014

  8. [16]

    Towards Addressing Noise and Static Variations of Analog Computations Using Effi- cient Retraining

    Bernhard Klein et al. “Towards Addressing Noise and Static Variations of Analog Computations Using Effi- cient Retraining”. In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases - In- ternational Workshops of ECML PKDD 2021, Proceed- ings Part I ...

    2021 doi

  9. [17]

    Quantizing deep convo- lutional networks for efficient inference: A whitepaper

    Raghuraman Krishnamoorthi. “Quantizing deep convo- lutional networks for efficient inference: A whitepaper”. In: ArXiv abs/1806.08342 (2018)

    2018 arXiv

  10. [18]

    Alex Krizhevsky and Geoffrey E. Hinton. Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto, 2009

    2009

  11. [19]

    On the Non-Associativity of Analog Computations

    Lisa Kuhn, Bernhard Klein, and Holger Fr ¨oning. “On the Non-Associativity of Analog Computations”. In: CoRR abs/2309.14292 (2023). DOI: 10.48550/ARXIV. 2309.14292

    2023 doi arXiv

  12. [20]

    Gradient-based learning applied to document recognition

    Yann LeCun et al. “Gradient-based learning applied to document recognition”. In: Proceedings of the IEEE 86 (1998)

    1998

  13. [21]

    Defensive Quan- tization: When Efficiency Meets Robustness

    Ji Lin, Chuang Gan, and Song Han. “Defensive Quan- tization: When Efficiency Meets Robustness”. In: In- ternational Conference on Learning Representations . ICLR. 2019

    2019

  14. [22]

    SGDR: Stochastic Gradient Descent with Warm Restarts

    Ilya Loshchilov and Frank Hutter. “SGDR: Stochastic Gradient Descent with Warm Restarts”. In: Interna- tional Conference on Learning Representations . ICLR (2017)

    2017

  15. [23]

    Mixed-signal computing for deep neural network inference

    Boris Murmann. “Mixed-signal computing for deep neural network inference”. In: IEEE Transactions on V ery Large Scale Integration (VLSI) Systems 29 (2020)

    2020

  16. [24]

    A White Paper on Neural Network Quantization

    Markus Nagel et al. “A White Paper on Neural Network Quantization”. In: ArXiv abs/2106.08295 (2021)

    2021 arXiv

  17. [25]

    XILINX/brevitas

    Alessandro Pappalardo. XILINX/brevitas. 2023. DOI: 10.5281/zenodo.3333552

    2023 doi

  18. [26]

    Model compression via distillation and quantization

    Antonio Polino, Razvan Pascanu, and Dan Alistarh. “Model compression via distillation and quantization”. In: ArXiv abs/1802.05668 (2018)

    2018 arXiv

  19. [27]

    Analog/mixed-signal hardware error modeling for deep learning inference

    Angad S Rekhi et al. “Analog/mixed-signal hardware error modeling for deep learning inference”. In: 56th Annual Design Automation Conference 2019

    2019

  20. [28]

    Resource-Efficient Neural Net- works for Embedded Systems

    Wolfgang Roth et al. “Resource-Efficient Neural Net- works for Embedded Systems”. In: Journal of Machine Learning Research 25.50 (2024), pp. 1–51. URL: http: //jmlr.org/papers/v25/18-566.html

    2024

  21. [29]

    Denoising noisy neural networks: A bayesian approach with compensation

    Yulin Shao, Soung Chang Liew, and Deniz G ¨und¨uz. “Denoising noisy neural networks: A bayesian approach with compensation”. In: IEEE Transactions on Signal Processing (2023)

    2023

  22. [30]

    Deep learning with coherent nanophotonic circuits

    Yichen Shen et al. “Deep learning with coherent nanophotonic circuits”. In: Nature Photonics 11 (2017)

    2017

  23. [31]

    Very deep convolutional networks for large-scale image recogni- tion

    Karen Simonyan and Andrew Zisserman. “Very deep convolutional networks for large-scale image recogni- tion”. In: International Conference on Learning Repre- sentations. ICLR (2014)

    2014

  24. [32]

    Resiliency of deep neural networks under quantiza- tion

    Wonyong Sung, Sungho Shin, and Kyuyeon Hwang. “Resiliency of deep neural networks under quantiza- tion”. In: ArXiv abs/1511.06488 (2015)

    2015 arXiv

  25. [33]

    Improving the robustness of analog deep neural networks through a Bayes-optimized noise injection approach

    Nanyang Ye et al. “Improving the robustness of analog deep neural networks through a Bayes-optimized noise injection approach”. In: Communications Engineering 2 (2023)

    2023

  26. [34]

    Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation

    Chuteng Zhou et al. “Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation”. In: ArXiv abs/2001.04974 (2020)

    2020 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.