Defending Against Intelligent Attackers at Large Scales

Andrew J. Lohn

REVIEW 2 major objections 5 minor 14 references

Defending Against Intelligent Attackers at Large Scales

T0 review · 2 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that with independent intelligent attacks, each additional defense layer multiplies protection, so small defensive gains offset exponential attack growth.

desk verdict A clean scaling argument for defense-in-depth that has a real but fixable error in the speedup section; worth referee time. read the letter →

arxiv 2504.18577 v1 pith:YQW5RHLU submitted 2025-04-22 cs.CR cs.CY

Andrew J. Lohn This is my paper · ORCID

classification cs.CRcs.CY

keywords defenseindepthcyberattackscalingindependentattacksblockadestrategydelaynegativebinomialAIcyberoffenseexponentialtradeoff

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that if AI produces cyberattacks that are both massively scaled and genuinely independent—each attempt differing from the last—defense in depth changes the arithmetic of the offense-defense balance. The core result is that the number of attacks a layered defense can absorb grows exponentially in the number of layers, so a small linear increase in defenses or a modest hardening of each layer compensates for a huge increase in attack volume or speed. The argument extends from defenses that block every attempt to defenses that merely delay while detection happens, and finally to attackers that learn and permanently break each layer. If the model holds, scale favors defenders, and AI's main offensive risk shifts from volume to correlated or shared attack knowledge.

What carries the argument

The workhorse is the defense-in-depth probability model of Equation 1, $L = 1 - (1 - p^n)^N$, where each of $n$ defenses is an independent gate that fails with probability $p$, and an attack succeeds only by passing every gate. Its small-$p^n$ approximation $N \approx L/p^n$ is what produces the exponential trade-off, because each extra defense multiplies the required attack count by a factor $1/p$. The delay variant replaces $p$ with $e^{-\lambda\tau}$, and the learning-attacker model replaces independent repeated attempts with a sum of geometric trials per defense, yielding the negative binomial count $N = n(1-p)/p$. The paper also uses the arithmetic mean-geometric mean inequality and Jensen's inequality to justify treating each defense as having the average failure probability, which it calls the conservative choice for defenders.

What would settle it

Measure the breach probability of a real or simulated $n$-layer system while holding the per-layer failure probability $p$ fixed and forcing $N$ attempts to be independent; if the probability does not fall like $p^n$—for instance, because a shared codebase lets one exploit defeat every layer at once—then the central trade-off fails. The extreme test is a single vulnerability present in all $n$ layers, where the breach probability should be about $p$, not $p^n$.

Watch

Extended reading notes

Core claim

The paper's central claim is that defense in depth has an exponential resilience: when $n$ independent defenses each fail with probability $p$, the number $N$ of independent attacks needed to keep the breach probability near $L$ is approximately $N \approx L/p^n$. Since each added defense divides the required attack count by $p$, linear growth in $n$ neutralizes exponential growth in $N$. The same structure reappears in the delay strategy, where the defensive detection rate $\lambda$ and attack speed $\tau$ enter through $e^{-\lambda\tau}$, and a speedup by a factor $s$ can be compensated by a small linear increase in the number of defenses. For attackers that learn and keep defeated defenses broken, the number of attempts one campaign needs follows a negative binomial distribution, $N = n(1-p)/p$, and the viability condition for many attackers becomes $nd/p - \ln(N_A) > 1$. The upshot is that, under independent-attack assumptions, scale and speed favor defense.

Load-bearing premise

Everything rests on the assumption that attack attempts are independent of one another and that each defense has the same fixed probability $p$ of failing, independent of the other defenses; if attackers share knowledge or if defenses fail in correlated ways, the exponential protection no longer follows.

Editorial extensions

If this is right

A defense composed of about twenty independent layers can absorb millions of independent attacks at a fixed breach probability, and absorbing trillions of attacks requires only a modest further increase in layers.
Sublinear improvements in individual defense hardness (small decreases in $p$) compensate for exponentially more attacks, while any weakening of a defense is equivalent to an exponential increase in the number of attacks.
In delay-based defenses, a millionfold speedup in attack attempts is offset by only a small linear increase in the number of defenses, but the ratio of detection speed to attack speed is itself a sensitive variable.
For attackers who learn and keep broken defenses broken, a campaign needs roughly $n(1-p)/p$ attempts, and defenders can offset exponential increases in the number of attackers by improving any of $n$, detection rate $d$, or hardness $1/p$.
The combined condition $nd/p - \ln(N_A) > 1$ shows that improvements in the number of defenses, detection rate, and per-defense hardness each provide exponential compensation for growth in the number of attackers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same equations imply that the strategic battleground is correlation: if AI attackers discover a single bug shared across all layers, protection collapses from exponential to linear, so defenders should prioritize diversity and independent implementation of layers.
The model predicts an experimental signature: for independent attempts, breach probability should decay as $p^n$, and red-team exercises with forced independence could test this directly.
An attacker's best counter-strategy suggested by the math is not more attempts but targeted diversity aimed at common-mode failures, since independent attempts are exactly the regime where defense wins.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper gives a clear scaling argument for why defense-in-depth can absorb exponentially many independent attacks, but the speedup half has a concrete notational mess in Section 3.1 that needs fixing before the strong claim stands. The Blockade model (N ≈ L/p^n) is a correct rearrangement of the author's 2019 equation, and the exponential-vs-linear tradeoff is exactly what it looks like. The new material is the learning-attacker model in Section 4, which combines blockade and delay and produces a clean relation (nd/p − ln(N_A) > 1). That part is worth reading.

The trouble is in Section 3.1. The paper defines the failure probability p as e^{-λτ} in Eq. 5a, then writes E[n_overcome] = 1/(1-p) = 1/e^{-λτ}. If p = e^{-λτ}, the correct mean is 1/(1-e^{-λτ}), not 1/e^{-λτ}. If p is instead 1-e^{-λτ}, the equality is fine but p has changed meaning without notice. Also, e^{-λτ} is called the 'likelihood of a discovery' when it is actually the probability of no discovery. So Eq. 6a and Eq. 7 are not derived from a coherent model. That said, the qualitative conclusion — linear increases in the number of defenses compensate exponential increases in speed — survives a corrected derivation; the error changes the constant, not the exponential trend. So this is fixable, not fatal.

The independence assumption is strong, and the paper knows it. If attackers share knowledge or defenses are correlated, the exponential tradeoff weakens or vanishes. The paper is upfront that these are simplifications, which is fair for a policy-facing model, but the results are bounds, not predictions. No code or data, though the equations are simple enough to check by hand. The citation pattern is mostly sound; the self-citation to the 2019 model is appropriate. One minor issue: the URL in reference [1] points to the wrong arXiv PDF (2503.11917 instead of 2504.13371).

Overall, this is a transparent, clearly written model paper that deserves a serious referee. The central blockade result is solid, the learning model is a nice addition, and the speedup section is wrong in detail but correctable. Send it to peer review with a request to fix Section 3.1 before acceptance. It will be useful for anyone working on the AI cyber offense-defense balance or cybersecurity policy.

Referee Report

2 major / 5 minor

Summary. The paper presents simple probabilistic models for defense-in-depth against large numbers of independent attacks. In the Blockade model (§2), each of n defenses has failure probability p and N independent attacks lead to breach probability L = 1 - (1 - p^n)^N, yielding the approximations N ≈ L/p^n and p = [1 - (1 - L)^{1/N}]^{1/n}. In the Delay model (§3), attacks must pass n detection opportunities, each with pass probability e^{-λτ}; the paper claims that compounding attack speedup s and attacker count gives N = N_a T s e^{-λτ}/τ and that n must grow only linearly to compensate exponential growth in N or s. Section 4 adds a learning model in which each defense breaks permanently after a geometric number of attempts, with probability of going undetected u, leading to relation nd/p - ln N_A > 1 for viable defenses. The abstract's central conclusion is that small increases in the number or quality of defenses compensate for exponential increases in the number of independent attacks and in attack speedups.

Significance. The Blockade half is a clean, self-contained scaling argument with explicitly stated approximations and a conservative-averaging justification via AM-GM; if the Delay speedup derivation is repaired, the paper would provide a useful and readily checkable pair of scaling laws for debates about AI and cyber offense-defense balance. The learning model in §4 is a new contribution and its asymptotic relation 11 is a falsifiable design rule. However, as submitted the speedup half of the central claim rests on an algebraic inconsistency in §3.1 and is therefore not yet supported.

major comments (2)

[§3.1, Eqs. (5a)–(7)] The derivation of the speedup-compensation formula is internally inconsistent. Eq. (5a) and the surrounding text identify e^{-λτ} with p, the analogue of the per-defense failure probability in Eq. (1). Under that identification, E[n_overcome] = 1/(1-p) equals 1/(1-e^{-λτ}), not 1/e^{-λτ}; the printed equality 1/(1-p)=1/e^{-λτ} is valid only if p=1-e^{-λτ}, which contradicts Eq. (5a). The same sentence also calls e^{-λτ} the 'likelihood of a discovery' at each defense, but e^{-λτ} is the probability of no detection. Because Eq. (6a) and Eq. (7) are built on this step, the claimed exponential compensation for speedups is not derived from a coherent model. Please choose a single definition of p, correct E[n_overcome] accordingly, and re-derive Eqs. (6a)–(7). The corrected result will likely still grow exponentially in n, but the printed equations need to be consistent.
[§4, Eq. (8)] The expected number of attempts required to defeat a defense is stated as (1-p)/p, but this is the expected number of failures before the first success in a geometric distribution, not the expected number of attempts including the successful attempt. For p=0.02, the expected attempts to defeat one defense is 1/p=50, while the paper's N=98 for two defenses counts only the 49 failed attempts per defense and omits the two successful penetrations. This convention matters for Eq. (9), where the exponent n(1-p)/p is the number of detection opportunities; please state explicitly whether the successful attempt is included in the opportunities to detect the attacker, and adjust the formula or the wording accordingly.

minor comments (5)

[§1.1] The sentence "Will AI substantially increase the scale or speed of cyber attack or defense." is a question missing a question mark, and "attack or defense" should likely be "attacks or defenses."
[§1.1] The text says that more aggressive alert thresholds would mean "increasing p," but since p is the probability that an individual defense fails, more aggressive thresholds should decrease p; please correct the direction.
[§3, first paragraph] The notation says "number of attackers (n)" but n is consistently used for the number of defenses in the Blockade section and in Eq. (5); this should be "number of defenses (n)."
[References] Reference [1] lists arXiv:2504.13371 but the URL points to arXiv:2503.11917; please make the identifier and URL consistent.
[Throughout] There are several typos that should be corrected in a final pass: "penentrate" (§1.2), "constat" (§2.2), "harddrives" (§3.2), and "the the" (§3.2).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the scaling claims are algebraic consequences of the stated model equations, and the self-citation to the author's prior work is a non-load-bearing source attribution.

full rationale

The paper's central results (Eqs. 3, 7, and 11) are derived by rearranging the model equations (Eqs. 1, 5b, and 10) using the stated small-value approximations (ln(1-L) ≈ -L, ln(1-p^n) ≈ -p^n, etc.). These are algebraic consequences of the modeling assumptions, not fitted parameters or imported conclusions; no data subset is fitted and then renamed as a prediction. The base model is attributed to the author's previous work [9], but the current paper restates the assumptions and equations explicitly, so the scaling derivation is self-contained and does not require accepting any conclusion from [9] beyond what is written here. The exponential detection form e^{-λτ} is an explicit assumption in Eq. 5a rather than an ansatz smuggled in by citation. No uniqueness theorem is invoked to force a choice. A separate arithmetic inconsistency exists in Sec. 3.1: Eq. 5a identifies p with e^{-λτ}, but E[n_overcome] is then written as 1/(1-p) = 1/e^{-λτ}, which would require p = 1 - e^{-λτ}, and e^{-λτ} is called the 'likelihood of a discovery' when it is the probability of no detection. This is a correctness and consistency concern, not circularity, and is therefore outside the circularity score. With no circular step exhibited, the score is 0.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The model is built on symbolic parameters (p, lambda, tau, d) that are not fitted to data. The central claim depends on the independence and identical-strength assumptions for defenses and on the independence of attacks, plus the smallness approximations. No new entities are postulated.

free parameters (4)

p (Blockade model)
Per-defense failure probability in Eq. 1. Symbolic model input, not fitted to data; the paper uses illustrative values such as 0.43 in figures.
lambda (Delay model)
Per-defense detection rate in Eq. 5a. Symbolic input, not fitted.
tau (Delay model)
Time to overcome one defense in Eq. 5a. Symbolic input, not fitted.
d (learning model)
Per-attempt detection probability in Eqs. 9-10. Symbolic input, not fitted.

assumptions (6)

domain assumption All n defenses must be overcome by each attack attempt (series system).
Used in Eq. 1 and throughout; a single successful attack must defeat every defense.
domain assumption Defense failures are independent across defenses with a common probability p (identical defenses).
Stated in Section 2: 'We treat each defense as having the same value for p as in an average.' The AM-GM inequality is used to justify the equal-p case as conservative for defenders.
domain assumption Attack attempts are statistically independent.
This is the paper's central scenario, introduced in the abstract and Section 1.1: intelligent attackers act independently such that each attempt is different. It is not derived from real attacker behavior.
domain assumption For the delay model, each defense detects at a constant rate lambda and each attack takes constant time tau per defense.
Eq. 5a is taken from prior work [9]; constant rates are a simplification.
domain assumption In the learning model, a defense once breached remains broken for that attacker until detection and repair; detection probability is d per attempt.
Section 4 description of 'attackers that learn'.
standard math Small-L and small-p^n Taylor approximations ln(1-L) ≈ -L and ln(1-p^n) ≈ -p^n.
Used to derive the simplified N ≈ L/p^n and the exponential relation; valid only when L and p^n are small.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Defending Against Intelligent Attackers at Large Scales." pith.science (2026). https://pith.science/paper/YQW5RHLU

@misc{pith2026250418577,
  author       = {Pith},
  title        = {Pith review of: Defending Against Intelligent Attackers at Large Scales},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YQW5RHLU}},
  note         = {Machine review of arXiv:2504.18577}
}

read the original abstract

We investigate the scale of attack and defense mathematically in the context of AI's possible effect on cybersecurity. For a given target today, highly scaled cyber attacks such as from worms or botnets typically all fail or all succeed. Here, we consider the effect of scale if those attack agents were intelligent and creative enough to act independently such that each attack attempt was different from the others or such that attackers could learn from their successes and failures. We find that small increases in the number or quality of defenses can compensate for exponential increases in the number of independent attacks and for exponential speedups.

Figures

Figures reproduced from arXiv: 2504.18577 by the authors.

**Figure 2.** Harder individual defenses can compensate for exponentially increasing numbers of attacks. The probability [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** An exponential increase in the number of attacks or speedups can be compensated for by a linear increase in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The number of defenses and the number of attacks are sensitive to the ratio of defensive and offensive rates [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 10 canonical work pages

[1]

Andrew J. Lohn. The impact of ai on the cyber offense-defense balance and the character of cyber conflict. arXiv:2504.13371, 2025. URL https://arxiv.org/pdf/2503.11917

work page Pith review arXiv 2025
[2]

Considerations Influencing Offense-Defense Dynamics From Artificial Intelligence

Giulio Corsi, Kyle Kilian, and Richard Mallah. Considerations influencing offense-defense dynamics from artificial intelligence. arXiv preprint arXiv:2412.04029, 2024. doi: 10.48550/arXiv.2412.04029

work page Pith review arXiv doi:10.48550/arxiv.2412.04029 2024
[3]

The implications of artificial intelligence in cybersecurity: Shifting the offense-defense balance

Jennifer Tang, Tiffany Saade, and Steve Kelly. The implications of artificial intelligence in cybersecurity: Shifting the offense-defense balance. Technical report, Institute for Security + Technology, 2024. URL https://securityandtechnology.org/virtual-library/reports/ the-implications-of-artificial-intelligence-in-cybersecurity/

work page 2024
[4]

Botnet detection and mitigation: A comprehensive literature review

Saurav Bhattacharya, Anirudh Khanna, and Rajat Dubey. Botnet detection and mitigation: A comprehensive literature review. International Journal of Computer Trends and Technology , 72:77–82, 2024. doi: https: //doi.org/10.14445/22312803/IJCTT-V72I1P113

work page doi:10.14445/22312803/ijctt-v72i1p113 2024
[5]

Bhattacharyya, and Jugal K

Nazrul Hoque, Dhruba K. Bhattacharyya, and Jugal K. Kalita. Botnet in ddos attacks: Trends and challenges. IEEE Communications Surveys & Tutorials, 17(4):2242–2270, 2015. doi: 10.1109/COMST.2015.2457491

arXiv 2015
[6]

Deep learning approaches for detecting ddos attacks: a systematic review

Meenakshi Mittal, Krishan Kumar, and Sunny Behal. Deep learning approaches for detecting ddos attacks: a systematic review. Soft Computing, 27:13039–13075, 2023. doi: https://doi.org/10.1007/s00500-021-06608-1

doi:10.1007/s00500-021-06608-1 2023
[7]

A framework for evaluating emerging cyberattack capabilities of ai

Mikel Rodriguez, Raluca Ada Popa, Four Flynn, Lihao Liang, Allan Dafoe, and Anna Wang. A framework for evaluating emerging cyberattack capabilities of ai. arXiv:2503.11917, 2025. URL https://arxiv.org/pdf/ 2503.11917

arXiv 2025
[8]

Alex Halderman

Zakir Durumeric, Frank Li, James Kasten, Johanna Amann, Jethro Beekman, Mathias Payer, Nicolas Weaver, David Adrian, Vern Paxson, Michael Bailey, and J. Alex Halderman. The matter of heartbleed. Internet Measure Conference, pages 475–488, 2014. doi: https://dl.acm.org/doi/abs/10.1145/2663716.2663755. 8 DEFENDING AGAINST INTELLIGENT ATTACKERS AT LARGE SCAL...

arXiv 2014

Show all 14 references

[9]

Andrew J. Lohn. Defense in depth: The basics of blockade and delay. arXiv: 1910.00111 , 2019. URL https://arxiv.org/abs/1910.00111

1910 arXiv
[10]

Petty, and John A

Christopher Daniel Colvett, Mikel D. Petty, and John A. Bland. Impact of computer users on cyber defense strategies. Systems Engineering, 2023. doi: https://doi.org/10.1002/sys.21737

2023 doi
[11]

Artificial intelligence accelerated cyber operations: A systematic literature review

Pietari Sarjakivi. Artificial intelligence accelerated cyber operations: A systematic literature review. Journal of In- formation Warfare, 24:118–134, 2025. URL https://www.jinfowar.com/journal/volume-24-issue-1/ artificial-intelligence-accelerated-cyber-operations-systematic-...

2025
[12]

Sludge for good: Slowing and imposing costs on cyber attackers

Josiah Dykstra, Kelly Shortridge, Jamie Met, and Douglas Hough. Sludge for good: Slowing and imposing costs on cyber attackers. arXiv:2211.16626, 2022. URL https://arxiv.org/abs/2211.16626

2022 arXiv
[13]

How darktrace finds ’low and slow’ cyber threats

Darktrace. How darktrace finds ’low and slow’ cyber threats. 2018. URL https://www.darktrace.com/es/ blog/flying-under-the-radar-how-darktrace-detects-low-and-slow-cyber-attacks

2018
[14]

Introduction to Probability, Statistics, and Random Processes

Hossein Pishro-Nik. Introduction to Probability, Statistics, and Random Processes. Kappa Research, LLC, 2014. A Appendix A Equation 12 is the same as equation 10. It describes the likelihood of at least one breach in a combined Blockade and Delay strategy. Once an attacker byp...

2014

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Andrew J. Lohn. The impact of ai on the cyber offense-defense balance and the character of cyber conflict. arXiv:2504.13371, 2025. URL https://arxiv.org/pdf/2503.11917

work page Pith review arXiv 2025

[2] [2]

Considerations Influencing Offense-Defense Dynamics From Artificial Intelligence

Giulio Corsi, Kyle Kilian, and Richard Mallah. Considerations influencing offense-defense dynamics from artificial intelligence. arXiv preprint arXiv:2412.04029, 2024. doi: 10.48550/arXiv.2412.04029

work page Pith review arXiv doi:10.48550/arxiv.2412.04029 2024

[3] [3]

The implications of artificial intelligence in cybersecurity: Shifting the offense-defense balance

Jennifer Tang, Tiffany Saade, and Steve Kelly. The implications of artificial intelligence in cybersecurity: Shifting the offense-defense balance. Technical report, Institute for Security + Technology, 2024. URL https://securityandtechnology.org/virtual-library/reports/ the-implications-of-artificial-intelligence-in-cybersecurity/

work page 2024

[4] [4]

Botnet detection and mitigation: A comprehensive literature review

Saurav Bhattacharya, Anirudh Khanna, and Rajat Dubey. Botnet detection and mitigation: A comprehensive literature review. International Journal of Computer Trends and Technology , 72:77–82, 2024. doi: https: //doi.org/10.14445/22312803/IJCTT-V72I1P113

work page doi:10.14445/22312803/ijctt-v72i1p113 2024

[5] [5]

Bhattacharyya, and Jugal K

Nazrul Hoque, Dhruba K. Bhattacharyya, and Jugal K. Kalita. Botnet in ddos attacks: Trends and challenges. IEEE Communications Surveys & Tutorials, 17(4):2242–2270, 2015. doi: 10.1109/COMST.2015.2457491

arXiv 2015

[6] [6]

Deep learning approaches for detecting ddos attacks: a systematic review

Meenakshi Mittal, Krishan Kumar, and Sunny Behal. Deep learning approaches for detecting ddos attacks: a systematic review. Soft Computing, 27:13039–13075, 2023. doi: https://doi.org/10.1007/s00500-021-06608-1

doi:10.1007/s00500-021-06608-1 2023

[7] [7]

A framework for evaluating emerging cyberattack capabilities of ai

Mikel Rodriguez, Raluca Ada Popa, Four Flynn, Lihao Liang, Allan Dafoe, and Anna Wang. A framework for evaluating emerging cyberattack capabilities of ai. arXiv:2503.11917, 2025. URL https://arxiv.org/pdf/ 2503.11917

arXiv 2025

[8] [8]

Alex Halderman

Zakir Durumeric, Frank Li, James Kasten, Johanna Amann, Jethro Beekman, Mathias Payer, Nicolas Weaver, David Adrian, Vern Paxson, Michael Bailey, and J. Alex Halderman. The matter of heartbleed. Internet Measure Conference, pages 475–488, 2014. doi: https://dl.acm.org/doi/abs/10.1145/2663716.2663755. 8 DEFENDING AGAINST INTELLIGENT ATTACKERS AT LARGE SCAL...

arXiv 2014