Optimal Security Response to Network Intrusions in IT Systems

Kim Hammar

REVIEW 3 major objections 4 minor 297 references

Optimal Security Response to Network Intrusions in IT Systems

T0 review · 3 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This thesis claims that automated security response with near-optimal performance can be achieved by learning defender strategies in a simulator from a digital twin's measurements and evaluating them on that twin.

desk verdict A substantial thesis with real structural theorems and a valuable open-source platform, but the headline claim of demonstrating optimal response on an IT infrastructure is only backed by digital-twin experiments, not a physical system. read the letter →

arxiv 2502.02541 v2 pith:2VC7U7VT submitted 2025-02-04 cs.GT cs.CR

Kim Hammar This is my paper · ORCID

classification cs.GTcs.CR MSC 91A8091A1590C4093E35

keywords optimalsecurityresponsedigitaltwinpartiallyobservedstochasticgamePOMDPapproximationthresholdstrategiesintrusiontheory

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis tries to show that optimal security response against network intrusions is not just a theoretical ideal but can be demonstrated on a practical IT infrastructure. Its proposed methodology builds a digital twin of the target system, measures the twin to identify a game-theoretic or partially observed Markov decision model, optimizes defender strategies in a fast simulator using stochastic approximation, and then evaluates and refines those strategies on the twin. Across six instances of the security response problem, the resulting strategies approach the performance of an upper bound that knows the exact intrusion time and outperform rule-based, reinforcement-learning, and change-detection baselines. If this holds, automated response can move from hand-configured rules to data-driven policies that are both scalable and backed by optimality guarantees.

What carries the argument

The load-bearing machinery is the closed loop connecting a digital twin, system identification, and stochastic optimization: the twin both produces the measurements from which the observation model is fitted and serves as the testbed where learned strategies are evaluated. On the theory side, the central objects are the one-sided partially observed stochastic game and the partially observed Markov decision process, for which the thesis proves that optimal defender strategies are threshold strategies and that large games decompose into smaller subgames. These structural theorems enable low-dimensional parameterizations and efficient algorithms such as threshold stochastic approximation, fictitious play, and recursive decomposition, avoiding the exponential blowup of exact dynamic programming.

What would settle it

Run the same methodology against a physical, non-emulated IT infrastructure that the digital twin replicates; if the learned strategy's cumulative reward drops materially relative to its digital-twin evaluation, or falls below the rule-based baselines, the central claim that the loop demonstrates optimal operational security response is refuted.

Watch

Extended reading notes

Core claim

The central claim is that learning defender strategies in a simulator using a model identified from digital-twin measurements, then evaluating them on the digital twin, yields near-optimal security response strategies for an IT infrastructure. The thesis proves structural properties of optimal strategies, notably that optimal stopping rules have threshold structure and that the intrusion response game can be recursively decomposed, and it derives efficient stochastic approximation algorithms that exploit these properties. Experiments on digital twins of infrastructures up to 64 servers, covering intrusion prevention, intrusion response, intrusion tolerance, and defense against advanced persistent threats, show that the learned strategies approach an upper bound that knows the intrusion time and beat rule-based, deep reinforcement learning, and change-detection baselines. The thesis therefore asserts that it solves the previously unsolved problem of demonstrating optimal security response against network intrusions on an IT infrastructure.

Load-bearing premise

The digital twin closely approximates the functionality and timing behavior of the target infrastructure, so a strategy that performs well on the twin is assumed to perform well in operation.

Editorial extensions

If this is right

Automated response policies can be synthesized from measured data rather than expert-authored rules, removing the main practical bottleneck of current rule-based systems.
In the studied scenarios, learned strategies are near-optimal: their cumulative reward approaches an upper bound that knows the exact intrusion time.
Threshold structure makes optimal strategies representable and computable in practice, where exact POMDP dynamic programming does not converge within practical time.
Recursive decomposition keeps the game's state and action spaces from growing exponentially with infrastructure size in the tested settings.
Evaluation on a digital twin, not only simulation, provides evidence that strategies built from identified models retain their performance against realistic emulated attacks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implicit consequence the thesis does not develop: the same loop should transfer to other domains where a high-fidelity emulator can be built and an observation distribution estimated, such as cloud autoscaling or IoT access control; this is an extension, not a claim of the thesis.
The operational claim is only as strong as the digital twin's fidelity, since all experiments run on emulated infrastructure; a physical deployment test would be the natural next step.
The threshold-structure results suggest that cheap detectors with monotone likelihood-ratio observations could be plugged into the framework without re-deriving the theory, a possibility the thesis does not test.
A testable extension would measure how strategy performance degrades when the attacker deviates from the emulated action set; the thesis evaluates dynamic attackers within that set but does not quantify out-of-set robustness.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take on Hammar's thesis. It's a substantial body of work: six published papers building a practical methodology for optimal security response, with new structural theorems (threshold optimality for multiple stopping, recursive decomposition), a family of stochastic approximation algorithms (t-spsa, t-fp, dfp, COL, c-pomcp), and an open-source platform (CSLE) that others can actually use. The theoretical results are real: Theorem 1.1's threshold structure for the POMDP stopping problem is proved in detail, and the decomposition results in Paper 3 are nontrivial. The learning curves are consistent across seeds and the comparisons to strong baselines (PPO, HSVI, Shiryaev) are fair.

The main weakness is the gap between what is claimed and what is shown. The abstract says the work 'demonstrates optimal security response against network intrusions on an IT infrastructure.' What it actually does is run experiments on a digital twin—an emulated environment using containers and netem. The observation model is fitted from that same twin, and the strategies are evaluated on that same twin. So any systematic error in the emulator is present on both sides. That is a real circularity, and it means the operational claim is not supported. It is fair to say the thesis demonstrates optimal response in a high-fidelity simulated environment, not on an IT infrastructure. The authors do list assumptions that limit practical applicability, which is honest, but the abstract overstates the result.

A second, smaller issue: for the multi-stop case (L=2,3), Theorem 1.1.C requires the observation distribution to be TP-2. The thesis only says this is 'reasonable' from smoothed histograms; it does not statistically verify it. Since the algorithm and the performance claims for those instances rely on this assumption, it should have been tested or at least reported with a sensitivity analysis. This is minor relative to the twin issue, but worth flagging.

The theoretical contributions and the platform stand on their own. If you work on security games or POMDP-based response, this is worth citing. The thesis deserves a serious referee; the right outcome would be publication with the central claim tempered to 'digital-twin validation' rather than 'operational demonstration.' I would bring it to a reading group, mainly to discuss the methodology for closing the sim-to-real gap and how one would validate on a physical testbed.

Referee Report

3 major / 4 minor

Summary. This doctoral thesis develops a methodology for automated, game-theoretic security response in IT infrastructures, centered on a digital twin of the target system. The methodology comprises: (i) emulating the infrastructure and collecting measurements; (ii) identifying a partially observed stochastic game or POMDP model from those measurements; (iii) optimizing defender strategies via stochastic approximation, exploiting proven structural properties (notably threshold strategies); and (iv) evaluating the learned strategies on the emulated twin. The thesis compiles six papers covering intrusion prevention, intrusion response, intrusion tolerance, and APT defense, and it introduces the open-source CSLE platform. The main claimed contribution is 'demonstrating optimal security response against network intrusions on an IT infrastructure', supported by experiments showing that threshold-based stochastic approximation algorithms outperform RL, change-detection, and rule-based baselines on a 31- to 64-server emulated infrastructure.

Significance. If the claims are taken at face value, the thesis makes a solid contribution to the security-response literature: it provides structural theorems (threshold optimality for stopping problems and games) that turn otherwise intractable POMDP/OS-POSG computations into low-dimensional stochastic approximation; it presents a fully open-source experimental platform (CSLE) that integrates emulation, system identification, and optimization; and it demonstrates consistently better performance than several strong baselines across multiple response scenarios. The theoretical parts are carefully developed and the learning curves are reproducible across seeds. The main significance is therefore the combination of structural optimality results with an emulation-based methodology. The load-bearing weakness is that the validation is performed entirely on the digital twin used also for system identification, so the evidence supports optimality relative to the twin rather than operational performance on a real IT infrastructure. This limits the strength of the central 'unsolved problem' claim unless the claims are tempered or a physical validation is added.

major comments (3)

[Paper 1, §1.5–1.6; Methodology chapter, 'Digital Twin'] The central claim, stated in the abstract and conclusion, is that the thesis 'demonstrat[es] optimal security response against network intrusions on an IT infrastructure.' The evidence for this claim is obtained in a closed loop: the observation distribution z(ot|st) is estimated from measurements taken on the digital twin (Paper 1, §1.5), and the learned strategies are evaluated on the same twin (Paper 1, §1.6). The thesis asserts that the twin 'closely approximates the functionality and timing behavior of the target infrastructure' but provides no calibration against a physical, non-emulated infrastructure. Consequently, the experiments establish optimality and effectiveness relative to the emulated twin, not operational performance on an actual IT infrastructure. To support the stronger claim, the author should either validate on a physical testbed (or a real operational system), or explicitly limit the claim to 'optimal response in an emulated digital twin' and adjust the abstract and conclusions accordingly.
[Paper 1, §1.4, Thm. 1.1.C and §1.5] Theorem 1.1.C states that, for L ≥ 1, there exists an optimal multi-threshold strategy only if the observation distribution z is totally positive of order 2 (TP-2). The experimental use case uses L = 3 stop actions (Table 1.2), so the 'optimal' label of the t-spsa results for this case rests entirely on the TP-2 condition. In §1.5 the author writes that 'the tp-2 assumption in Thm. 1.1.C is reasonable' based on visual inspection of smoothed histograms (Fig. 1.8), but no formal verification is given. The manuscript should either provide a statistical test of TP-2 on the estimated z, or present the L = 3 experiments as results for a restricted class of threshold strategies (in which case they are not necessarily optimal for the full POMDP), or prove threshold optimality under a weaker and verifiable condition.
[Paper 1, §1.6 (t-spsa convergence) and §1.4 (Eq. 1.8)] The text states that t-spsa converges 'almost surely to a local maximum' of J, while the surrounding discussion and Figure 1.11 suggest global near-optimality ('close approximations of an optimal strategy'). These two statements are not the same. If the parameterized threshold class does not contain the global optimum (e.g., because TP-2 fails, as in the previous comment), convergence to a local maximum within that class does not imply near-optimality for the original problem. The manuscript should clarify what 'optimal' means in the experimental claims: optimal within the threshold class, or optimal for the POMDP subject to an unverified TP-2 assumption. The distinction is load-bearing for the paper's title and abstract.

minor comments (4)

[Abstract] The abstract contains typographical errors: 'security response init infrastructures' should be 'security response in IT infrastructures', and 'anit infrastructure' should be 'an IT infrastructure'. The same issues appear in the Swedish abstract ('it-infrastrukturerienvirtuellmiljö' is missing spaces). These should be corrected.
[Paper 1, Eq. (1.8)] The stochastic threshold strategy in Eq. (1.8) has a removable singularity when σ(θ_l) = 0 or b(1) = 1; the accompanying footnote explains this, but it would be cleaner to define the function with an explicit piecewise case in the main text rather than in a footnote.
[Introduction, Fig. 14] The literature-map figure is dense and many citation labels are hard to read at print resolution. Consider enlarging the figure or providing a searchable table of the categorized references.
[Paper 1, Table 1.4 and Appendix C] The attacker action tables and infrastructure configuration tables are very detailed, but several entries are truncated in the thesis ('...[truncated to fit review budget]' appears in the provided text). The final published thesis should contain the complete configuration listings referenced by the papers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the mathematical derivations are self-contained and the digital-twin evaluation loop is an external-validity limitation, not a definitional reduction.

full rationale

The thesis's derivation chain is self-contained in its mathematical core: the threshold-structure theorems (e.g., Thm. 1.1) are proved in Appendix A from stated assumptions using external results (Nakai, Krishnamurthy), and the stochastic-approximation algorithms' convergence rests on standard Robbins-Monro/SPSA conditions, with empirical comparison against baselines and an upper bound that knows the intrusion time. The main skeptical concern, that the same digital twin is used both to estimate the observation distribution z and to evaluate learned strategies, is a limitation of external validity rather than circularity of derivation. Paper 1, Section 1.5 estimates z from digital-twin measurements, and Section 1.6 evaluates strategies on that same twin; however, no fitted parameter is renamed as the evaluation result. The POMDP reward is specified from the use case, strategies are optimized in the simulator, and the twin evaluation is an independent emulated execution reported separately, with the paper noting that twin performance is close to but slightly worse than simulation performance, confirming a real model gap rather than identity. The assertion that the digital twin 'closely approximates the functionality and timing behavior of the target infrastructure' (Methodology chapter, Digital Twin section) is uncalibrated against a physical infrastructure, which weakens the claim of demonstrating optimality 'on an IT infrastructure,' but this is an unsupported assumption, not a definitional equation. Likewise, the TP-2 condition in Thm. 1.1.C is stated as 'reasonable' from smoothed histograms without formal verification, an unverified assumption rather than a circular step. Self-citations to csle and to the included papers are references to released open-source code and to proofs contained within the same thesis; they are not load-bearing appeals to an unverified external uniqueness theorem. No step in the derivation reduces Eq. X to Eq. Y by construction, and no fitted parameter is presented as a prediction of a closely related quantity. Verdict: no significant circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a fitted POMDP/POSG model whose observation distribution is learned from the same digital twin used for evaluation, on hand-chosen reward constants and intrusion probability, and on several domain assumptions (zero-sum, omniscient attacker, TP-2 observations, digital-twin fidelity). No new physical entities are postulated; the contributions are algorithms, models, and a software platform.

free parameters (3)

Intrusion start probability p = 0.01
Hand-chosen hyperparameter in Paper 1 Table 1.6; governs the Bernoulli intrusion process and shapes the optimal threshold strategy.
Reward constants Rst, Rint, Rsla = 50, -10, 1
Hand-chosen rewards in Paper 1 Table 1.6; the optimal strategy and all reported rewards depend on this subjective objective.
Estimated observation distribution z(ot|st) = Empirical histograms from 21,000 digital-twin samples (Paper 1 Section 1.5)
Fitted to digital-twin measurements via EM; the 'optimal' strategy is optimal for this fitted model, and the TP-2 status is asserted but not formally tested.

assumptions (5)

domain assumption The digital twin closely approximates the target infrastructure's functionality and timing behavior.
Methodology chapter, Digital Twin section; all operational conclusions depend on this.
domain assumption The security response game is zero-sum and the attacker is omniscient.
Assumptions 5 and 6 in the Formalizing chapter; justify worst-case analysis but may not match real attackers.
domain assumption The observation distribution is TP-2 for the threshold theorem with L>1 stops.
Theorem 1.1.C and Appendix A; the paper only notes that the assumption is 'reasonable' from estimated distributions, without a statistical test.
standard math Finite, stationary models and perfect recall hold throughout.
Assumptions 1 and 4 in the background chapter; standard for POMDP and POSG analysis.
domain assumption Clients are implicitly modeled by the observation distribution z, not simulated.
Remark 1.2 in Paper 1 and Remark 6 in the Formalizing chapter; if client behavior changes under attack, the model is misspecified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Optimal Security Response to Network Intrusions in IT Systems." pith.science (2026). https://pith.science/paper/2VC7U7VT

@misc{pith2026250202541,
  author       = {Pith},
  title        = {Pith review of: Optimal Security Response to Network Intrusions in IT Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2VC7U7VT}},
  note         = {Machine review of arXiv:2502.02541}
}

read the original abstract

Cybersecurity is one of the most pressing technological challenges of our time and requires measures from all sectors of society. A key measure is automated security response, which enables automated mitigation and recovery from cyber attacks. Significant strides toward such automation have been made due to the development of rule-based response systems. However, these systems have a critical drawback: they depend on domain experts to configure the rules, a process that is both error-prone and inefficient. Framing security response as an optimal control problem shows promise in addressing this limitation but introduces new challenges. Chief among them is bridging the gap between theoretical optimality and operational performance. Current response systems with theoretical optimality guarantees have only been validated analytically or in simulation, leaving their practical utility unproven. This thesis tackles the aforementioned challenges by developing a practical methodology for optimal security response in IT infrastructures. It encompasses two systems. First, it includes an emulation system that replicates key components of the target infrastructure. We use this system to gather measurements and logs, based on which we identify a game-theoretic model. Second, it includes a simulation system where game-theoretic response strategies are optimized through stochastic approximation to meet a given objective, such as mitigating potential attacks while maintaining operational services. These strategies are then evaluated and refined in the emulation system to close the gap between theoretical and operational performance. We prove structural properties of optimal response strategies and derive efficient algorithms for computing them. This enables us to solve a previously unsolved problem: demonstrating optimal security response against network intrusions on an IT infrastructure.

Figures

Figures reproduced from arXiv: 2502.02541 by the authors.

**Figure 1.** Estimated trends in cyber attacks and network connectivity during 2017-2028. 1Here “control problem” encompasses both control- and game-theoretic problem formulations. 17 [PITH_FULL_IMAGE:figures/full_fig_p023_1.png] view at source ↗

**Figure 2.** Security operations in modern it infrastructures; Security and Event Management (siem) systems aggregate data for a Security Operations Center (soc) where human operators respond to security incidents and potential attacks [PITH_FULL_IMAGE:figures/full_fig_p024_2.png] view at source ↗

**Figure 3.** The actors and systems involved in the security response use case [PITH_FULL_IMAGE:figures/full_fig_p025_3.png] view at source ↗

Figures from the paper (204 more)

**Figure 4.** Figure 4: An example service chain with virtual network functions as microservices. Infrastructure statistics During operation, services on the infrastructure produce real-time data, such as logs, performance metrics, and other statistics, which is critical for monitoring the s…

**Figure 5.** Figure 5: Distributions of security alerts on our testbed during two network intrusions. Client population Clients influence the infrastructure statistics when interacting with services. For example, metrics like traffic rate and cpu utilization may spike during peak client load…

**Figure 6.** Figure 6: Time series of client arrival rates and distributions of alerts on our testbed. Attacker The attacker, much like a client, accesses infrastructure services through the public gateway. However, instead of consuming these services as a client, the attacker aims to exploi…

**Figure 7.** Figure 7: Attacks often involve reconnaissance, brute force, and malicious code execution. Today, it is widely recognized that it infrastructures are unlikely to ever be capable of preventing all attacks (Maloof, 2005). They are simply too complex. Hence, it infrastructures requ…

**Figure 8.** Figure 8: Flow control. Example 1 (Flow control). Flow control is a common response to counter network intrusions. By redirecting suspicious traffic to a honeypot, for example, the defender can isolate and analyze malicious behavior without exposing critical parts of the netw…

**Figure 9.** Figure 9: Access control. Example 2 (Access control). Access control is a traditional mechanism for responding to attacks. By adjusting resource permissions, defenders can prevent attackers from compromising critical assets; see [PITH_FULL_IMAGE:figures/full_fig_p028_9.png]

**Figure 10.** Figure 10: Replication control. Example 3 (Replication control). Replication control in distributed systems can be used to respond to network intrusions by ensuring that multiple replicas of critical services remain available even when some replicas are compromised; see [PITH…

**Figure 11.** Figure 11: Phases and performance metrics of the security response problem. Approach As evidenced by the problem description above, security response is, at its core, a decision-making problem. From choosing the right way to mitigate an attack to configuring access control polic…

**Figure 12.** Figure 12: Our methodology for automated, data-driven, and optimal security response. Emulation system The emulation system creates a digital twin of the target infrastructure consisting of virtual containers and networks. Such a twin allows us to test response strategies under …

**Figure 13.** Figure 13: Security response as a discrete-time dynamical system. Each simulation path s1, s2, . . . , st is associated with security consequences and costs; the aim is to find those defender actions that control the simulation in an optimal manner according to a stipulated obj…

**Figure 14.** Figure 14: A categorization of the literature on automated security response and positioning of this thesis; we present a general methodology for automated security response; unlike other approaches, this methodology is not limited to a specific scenario or formal framework; th…

**Figure 15.** Figure 15: Dynamic programming; (a) policy iteration computes J π = TπJ π (10) and updates π using (9); (b) value iteration implements the recursion Jk+1 = T Jk (8). Under Assumptions 1–3, the recursion Jk+1 = T Jk satisfies limk→∞ Jk = J ⋆ (Thm. 6.3.1, Puterman, 1994). Likewise…

**Figure 16.** Figure 16: The belief state b ∈ ∆(S) (17) is an |S|-dimensional probability vector, which can be represented geometrically as a point in the unit simplex of dimension |S| − 1. 12When the state space is large, (17) generally has to be approximated; one common approximation techn…

**Figure 17.** Figure 17: The control loop of player k in a partially observed stochastic game (19). π1 is a best response against π2 if it maximizes J (π1,π2) (b1) ≜ E(π1,π2) "X T t=1 γ t−1 r(St, At) | b1 # . (23) Similarly, π2 is a best response against π1 if it minimizes J (π1,π2) (b1). Hen…

**Figure 18.** Figure 18: The security response problem; an attacker exploits vulnerabilities of an it infrastructure; the operator of the infrastructure, which we refer to as the defender, monitors the network and executes responses. 21Though several entities may be involved, defining a sing…

**Figure 19.** Figure 19: We use this twin to run attack scenarios and defender responses. Such runs produce system measurements and logs, from which we estimate infrastructure statistics. These statistics allow us to instantiate a mathematical model of the target infrastructure through syste…

**Figure 20.** Figure 20: csle architecture; it is a distributed system with a database and four interfaces: a Python api, a grpc api, a rest api, and a command-line interface. 29The code repository also contains video demonstrations, usage examples, pre-built virtual containers, technical do…

**Figure 21.** Figure 21: Our server rack at kth where we run csle. Server Processors Network ram (gb) 1, r 715 2u two 12-core amd opteron 12×GbE 64. 2, r 715 2u two 12-core amd opteron 12×GbE 64. 3, r 715 2u two 12-core amd opteron 12×GbE 64. 4, r 715 2u two 12-core amd opteron 12×GbE 64. 5, …

**Figure 22.** Figure 22: Time required to deploy a digital twin in function of the infrastructure size; deploying the twin involves creating containers and attaching them to virtual networks; the time measurements were performed for a digital twin with a single virtual network running on a se…

**Figure 23.** Figure 23: A distributed digital twin on csle; physical servers are connected through an ip network, over which virtual networks are created using vxlan tunnels. Network conditions of virtual links are created using the netem module in the linux kernel (Hemminger, 2005). This mo…

**Figure 24.** Figure 24: Monitoring system of a digital twin on csle; emulated devices run monitoring agents that periodically push metrics to an event bus; the data in this bus is consumed by data pipelines that process the data and write to storage systems; the processed data is also used b…

**Figure 25.** Figure 25: Each paper included in this thesis formalizes an aspect of the security response problem, presents analytical results, and evaluates the results through experimentation, all following the methodology from the previous chapter; Paper 1 studies intrusion prevention usin…

**Figure 26.** Figure 26: We prove that the optimal stopping times can be obtained through thresholds, which enables efficient computation of an optimal defender strategy. Based on this insight, we design t-spsa, an efficient stochastic approximation algorithm for estimating the thresholds. W…

**Figure 27.** Figure 27: The fictitious play process between the attacker and the defender in Paper 2; horizontal arrows indicate iterations of fictitious play and vertical arrows indicate the learning of best responses; the process converges to an equilibrium (π ⋆ D, π⋆ A). The paper is publ…

**Figure 28.** Figure 28: Size of the security response game considered in Paper 3 in function of the infrastructure size; the exponential growth exemplifies the curse of dimensionality: more state variables result in a combinatorial explosion of possible states (Bellman, 1957). The paper is p…

**Figure 29.** Figure 29: The intrusion-tolerant control architecture presented in Paper 4; node controllers with strategies π1, . . . , πNt make local recovery decisions; a global system controller with strategy π manages the replication factor Nt. The paper is published as K. Hammar and R. …

**Figure 30.** Figure 30: Conjectural Online Learning (col): Paper 5 formulates the interaction between an attacker and a defender as a game where each player has a probabilistic conjecture about the game model, which may be misspecified in the sense that the true model has probability 0; th…

**Figure 31.** Figure 31: Paper 6 presents Causal-Partially Observed Monte-Carlo Planning (cpomcp), which is an online method for optimal security response; the figure illustrates one time step of c-pomcp during which (i) a particle filter is used to compute a belief state; (ii) a causal grap…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p074_1.png]

**Figure 1.1.** Figure 1.1: Optimal multiple-stopping formulation of intrusion prevention; the horizontal axis represents time; T is the time horizon; the dashed line shows the intrusion start time; the optimal strategy is to prevent the attacker at the time of intrusion. 2The digital twin is c…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p075_1.png]

**Figure 1.2.** Figure 1.2: The it infrastructure and the actors in the intrusion prevention use case. The defender monitors the infrastructure by accessing and analyzing ids statistics and login attempts at the servers. It can take a fixed number of defensive actions to prevent the attacker. …

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p078_1.png]

**Figure 1.3.** Figure 1.3: The Cumulative Distribution Function (cdf) of the intrusion start time I. Consequently, we can define the transition function as f1(∅ | ·, 1) ≜ fl(∅ | ∅, ·) ≜ 1 (1.1a) fl(0 | 0, a) ≜ 1 − p if l − a > 0 (1.1b) fl(1 | 0, a) ≜ p if l − a > 0 (1.1c) fl(1 | 1, a) ≜ 1 if l…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p079_1.png]

**Figure 1.4.** Figure 1.4: State transition diagram of the pomdp: each circle represents a state; an arrow represents a state transition; a label indicates the event that triggers the transition; an episode starts in state s1 = 0 with l1 = L. Remark 1.1 (An intrusion will almost surely occur).…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p081_1.png]

**Figure 1.5.** Figure 1.5: The value of an optimal defender strategy for the example; the dashed red lines indicate alpha-vectors α (1) , α (2) , . . ., where J ⋆ (b) = maxi[1 − b(1), b(1)]T α (i) ; see (Def. 1, Sondik, 1978) for the definition of the alpha vectors; we computed J ⋆ using Heuri…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p082_1.png]

**Figure 1.6.** Figure 1.6: Illustration of Thm. 1.1: there exist L thresholds α ⋆ 1 ≥ α ⋆ 2, . . . , ≥ α ⋆ L in the unit interval [0, 1] and an optimal threshold strategy π ⋆ l that satisfies (1.5)–(1.7) [PITH_FULL_IMAGE:figures/full_fig_p082_1_6.png]

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p083_1.png]

**Figure 1.7.** Figure 1.7: The stochastic threshold strategy in (1.8); σ is the sigmoid function; σ(θl) is the threshold (0.5 in this example); the x-axis indicates the belief state b(1) ∈ [0, 1]; and the y-axis indicates the probability prescribed by πl,θ to the stop action S. 10To avoid divi…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p084_1.png]

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p085_1.png]

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p087_1.png]

**Figure 1.8.** Figure 1.8: Estimated (smoothed) distributions of severe ids alerts ∆x (top row), warning ids alerts ∆y (middle row), and login attempts ∆z (bottom row) based on measurements from the digital twin. Simulating an episode of the POMDP A simulated episode evolves as follows. The ep…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p089_1.png]

**Figure 1.9.** Figure 1.9: Architecture of the neural network used by ppo (Alg. 1, Schulman et al., 2017); the strategy πθ and the value function Jθ are parameterized by a neural network with the actor-critic architecture (Sutton and Barto, 1998); Σ represents a linear sum; σ represents the so…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p090_1.png]

**Figure 1.10.** Figure 1.10: shows the performance of the learned strategies against the three attackers defined in [PITH_FULL_IMAGE:figures/full_fig_p090_1_10.png]

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p091_1.png]

**Figure 1.11.** Figure 1.11: shows a comparison between our stochastic approximation algorithm (t-spsa) and the three baseline algorithms. 0 20 40 60 training time (min) 0 50 100 150 Reward vs Novice 0 20 40 60 training time (min) Reward vs Experienced 0 20 40 60 training time (min) Reward vs E…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p092_1.png]

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p100_1.png]

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p104_1.png]

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p107_2.png]

**Figure 2.1.** Figure 2.1: State transition diagram of a game episode: each disk represents a state; an arrow represents a state transition; a label indicates the conditions for the state transition; a game episode starts in state s1 = 0 with l = L and ends in state sT = ∅. Reward function rl(…

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p110_2.png]

**Figure 2.2.** Figure 2.2: Stopping times of the defender and the attacker in a game episode; τk,j denotes the jth stopping time of player k ∈ {D, A}; the cross shows the time the intrusion is stopped; an intrusion starts when the attacker takes the first stop action (at time τA,1); an episode…

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p111_2.png]

**Figure 2.3.** Figure 2.3: Illustration of Thm. 2.1; the upper part shows L thresholds α˜1 ≥ α˜2, . . . , ≥ α˜L in the unit interval that define a best response π˜D ∈ BD(πA) for the defender (2.8); the lower part shows 2L thresholds β˜0,1, β˜1,1, . . . , β˜0,L, β˜1,L in the unit interval that …

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p112_2.png]

**Figure 2.4.** Figure 2.4: The fictitious play process; horizontal arrows indicate iterations of fictitious play and vertical arrows indicate the learning of best responses; the process converges to an equilibrium (π ⋆ D, π⋆ A) (Thms. 7.2.4-7.2.5, Shoham and Leyton-Brown, 2009). Our fictitious…

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p113_2.png]

**Figure 2.5.** Figure 2.5: The stochastic threshold strategy in (2.10), where σ is the sigmoid function and σ(θ˜(D) l ) is the threshold (0.5 in this example); the x-axis indicates the defender’s belief state b(1) ∈ [0, 1]; and the y-axis indicates the probability prescribed by π˜D,θ˜(D) to th…

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p114_1.png]

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p116_2.png]

**Figure 2.6.** Figure 2.6: Fitted Gaussian mixture models of z (2.4) when no intrusion occurs (st = 0) and during intrusion (st = 1). Running a game episode During a game episode, the state evolves according to the dynamics defined by (2.2), the defender’s belief state evolves according to (22…

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p117_2.png]

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p119_2.png]

**Figure 2.7.** Figure 2.7: shows the learning curves of the strategies obtained during the fictitious play process with t-fp and the baselines introduced above. 0 20 40 60 80 100 # training iterations 0 1 2 3 Approximate exploitability 0 20 40 60 80 100 # training iterations −5 0 5 Defender re…

**Figure 2.8.** Figure 2.8: Comparison between t-fp and two baseline algorithms: nfsp and hsvi; all curves show simulation results; the red curve relates to t-fp; the blue curve to nfsp; the purple curve to hsvi; the left plot shows the approximate exploitability metric (2.13) and the right plo…

**Figure 2.9.** Figure 2.9: shows the estimated value function Jˆ⋆ l : B → R, where Jˆ⋆ l (b) is the expected cumulative discounted reward when the game starts in the belief state b, the defender has l stops remaining, and both players follow equilibrial strategies16 . Jˆ⋆ 7 (b) Jˆ⋆ 1 (b) 1 0 b…

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p121_2.png]

**Figure 2.10.** Figure 2.10: Percentage of blocked attacker and client traffic in the digital twin; the blue curves show results from the equilibrial strategy learned via t-fp; the purple, orange, and black curves relate to baseline strategies. We observe in the upper plot of [PITH_FULL_IMAGE:…

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p122_2.png]

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p124_2.png]

**Figure 2.** Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p127_2.png]

**Figure 2.11.** Figure 2.11: Empirical distributions of selected infrastructure metrics; the red and blue lines show the distributions when no intrusion occurs and during intrusion, respectively. D Attacker Actions The attacker actions and their descriptions are listed in [PITH_FULL_IMAGE:figu…

**Figure 3.** Figure 3: in the introduction chapter. Overlaid on this physical infrastructure is a virtual infrastructure with a tree structure that includes nodes, which collectively offer services to clients. A service is modeled as a workflow, which comprises a set of interdependent nodes.…

**Figure 3.1.** Figure 3.1: Defender actions: (a) migrate a node between two zones; (b) redirect or block traffic flows to a node; (c) shut down a node; and (d) revoke access to a node. 3.4 Formalizing the Intrusion Response Problem We formalize the above use case as an optimization problem whe…

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p134_3.png]

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p137_3.png]

**Figure 3.2.** Figure 3.2: Growth of |S|, |O|, and |Ak| in function of the number of nodes |V|; the curves are computed using |Z| = 10, |O(V)| = 100, and |A(V) D | = |A(V) A | = 10. 3.6 Decomposing the Intrusion Response Game In this section, we present the main contribution of the paper. We s…

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p140_3.png]

**Figure 3.3.** Figure 3.3: Theorem 3.2.A. Theorem 3.2.A is illustrated in [PITH_FULL_IMAGE:figures/full_fig_p140_3_3.png]

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p141_3.png]

**Figure 3.4.** Figure 3.4: Theorem 3.2.B. Theorem 3.2.B is illustrated in [PITH_FULL_IMAGE:figures/full_fig_p141_3_4.png]

**Figure 3.5.** Figure 3.5: Theorem 3.2.C. The argument behind the existence of a switching curve is as follows. On any line segment L(e1, bb(D)) in B (i) D that starts at e1 and ends at the subsimplex joining e2 and e3 (denoted with bb(D) ∈ B(i) D,e1 ), all belief states are totally ordered …

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p143_3.png]

**Figure 3.6.** Figure 3.6: Runtimes of dynamic programming when computing a best response for the attacker in a subgame Γ (i) via Sondik’s value iteration algorithm (Sondik, 1978); we note that even for a minimal observation space O (V) the runtime increases exponentially with the discount fac…

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p145_3.png]

**Figure 3.7.** Figure 3.7: Empirical observation distributions zbO1 , . . . , zbO|V| as estimates of zO1 , . . . , zO|V| in the target infrastructure; Oi is a random variable representing the number of ids alerts related to node i ∈ V, weighted by priority; the x-axes show the node-local obse…

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p148_3.png]

**Figure 3.8.** Figure 3.8: Best response learning via decomposition; (a) shows learning curves in simulation; the curves show the mean and 95% confidence interval for five random seeds; (b) shows the speedup of our approach when computing best responses with different number of parallel proce…

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p149_3.png]

**Figure 3.9.** Figure 3.9: shows the learning curves of the strategies obtained during the dfp execution and the baselines introduced above. The red curves represent the results from the simulator; the blue curves show the results from the digital twin; the green curve gives the performance of…

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p150_3.png]

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p154_3.png]

**Figure 3.** Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p155_3.png]

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p160_4.png]

**Figure 4.1.** Figure 4.1: Two-level feedback control for intrusion tolerance; node controllers with strategies π1, . . . , πNt compute belief states b1, . . . , bNt and make local recovery decisions; a global system controller with strategy π receives belief states and manages the replication…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p162_4.png]

**Figure 4.2.** Figure 4.2: Each node is segmented into two domains: an application domain, which runs a service replica, and a privileged domain, which runs security and control functions. The replicas are coordinated through a replication protocol that relies on digital signatures and guarant…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p164_4.png]

**Figure 4.3.** Figure 4.3: The tolerance architecture: Two-level recovery and replication control with feedback; Nt nodes provide a replicated service to clients; service responses are coordinated through an intrusion-tolerant consensus protocol; local node controllers decide when to recover …

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p168_4.png]

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p169_4.png]

**Figure 4.4.** Figure 4.4: a) disks represent states, arrows represent transitions, labels indicate conditions for transition, self-transitions are not shown; b) the probability that a node is compromised (C) or crashed (∅) if no recoveries occur; curves relate to min[pA,i + pC,i, 1]. 5A cras…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p172_4.png]

**Figure 4.5.** Figure 4.5: Game 4.1 in extensive form; we use the extensive form notation described in (Myerson, 1997); a filled circle denotes a decision node; an unfilled circle denotes a chance node; the first label next to each node indicates the player who makes the decision; N denotes na…

**Figure 4.6.** Figure 4.6: a) the ellipse indicates the place where the equilibrial strategy for the defender is to almost always recover; b) the dashed red lines indicate alpha-vectors α (1) , α (2) , . . ., where Eπ˜ (C) i,t [Ji | bi,1(C)] = maxi[1 − bi,1(C), bi,1(C)]T α (i) (Def. 1, Sondik,…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p175_4.png]

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p176_4.png]

**Figure 4.7.** Figure 4.7: below and [PITH_FULL_IMAGE:figures/full_fig_p176_4_7.png]

**Figure 4.8.** Figure 4.8: Convergence curves of Alg. 4.1 for computing a best response control strategy in Game 4.1 (the local intrusion recovery game); the curves relate to different parametric optimizers: spsa, bo, cem, de, and ppo; the curves show the mean value from evaluations with 20 ra…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p178_4.png]

**Figure 4.9.** Figure 4.9: shows a comparison between the operational cost (4.4) incurred by the equilibrium strategy in Game 4.1 and the periodic recovery strategy used in many state-of-the-art intrusion-tolerant systems (Distler, 2021). 101 102 103 104 105 106 0.5 1 Intrusion cost η Ji T∅ (4…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p179_4.png]

**Figure 4.10.** Figure 4.10: The mttf and the reliability function in Game 4.2 when all controllers are passive; T (F) is a random variable representing the time when Nt < 2f + k + 1 with f = 3 and k = 1 (Prop. 4.1); N1 is the initial number of nodes in the system; hyperparameters are listed in…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p183_4.png]

**Figure 4.11.** Figure 4.11: Illustration of Thm. 4.5. The proofs of Thm. 4.5 and Cor. 4.2 involve a combination of techniques from cmdp theory and lattice programming. We defer the proofs to Appendix C. However, for the coherence of our argument, we outline the main steps here. To prove Thm.…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p184_4.png]

**Figure 4.12.** Figure 4.12: Plot a) shows the compute time to obtain a best response in Game 4.2 via the linear program in Alg. 4.2; the error bars indicate the 95% confidence interval based on 20 runs; smax is the maximum number of nodes; plot b) shows the availability of the equilibrium stra…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p186_4.png]

**Figure 4.13.** Figure 4.13: A description of minbft is available in Appendix F. 3 4 5 6 7 8 9 10 20 40 60 1 client 20 clients Number of nodes (N) # Requests/s [PITH_FULL_IMAGE:figures/full_fig_p186_4_13.png]

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p188_4.png]

**Figure 4.14.** Figure 4.14: Empirical distributions zb1(· | si), . . . , zb10(· | si) as estimates of z1, . . . , z10 (4.2) for the containers in [PITH_FULL_IMAGE:figures/full_fig_p189_4_14.png]

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p190_4.png]

**Figure 4.15.** Figure 4.15: Comparison between our game-theoretic strategies and the baselines; columns represent performance metrics; x-axes indicate values of ∆R; rows relate to the number of initial nodes N1; error bars indicate the 95% confidence interval from evaluations with 20 random s…

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p192_4.png]

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p203_4.png]

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p204_4.png]

**Figure 4.16.** Figure 4.16: Time-space diagrams illustrating the message patterns of the minbft intrusion-tolerant consensus protocol (§4.2, Santos Veronese, 2010) [PITH_FULL_IMAGE:figures/full_fig_p205_4_16.png]

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p206_4.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p208_5.png]

**Figure 5.1.** Figure 5.1: Conjectural Online Learning (col): we formulate the interaction between an attacker and a defender as a non-stationary game; each player has a probabilistic conjecture about the game model, which may be misspecified in the sense that the true model has probability 0;…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p212_5.png]

**Figure 5.2.** Figure 5.2: State transition diagram of the game Γ: disks represent states; arrows represent transitions; labels indicate conditions for transition; the initial state is s1 = 0. Observability The attacker has complete observability. It knows the game state, the defender’s actio…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p214_5.png]

**Figure 5.3.** Figure 5.3: The first term in (5.6) encodes the intrusion cost s p t , which increases with the number of compromised servers st. The second term in (5.6) encodes the cost of the defender’s stop action, which is q − r if an intrusion occurs and q otherwise. 2 4 6 8 10 12 14 16 1…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p215_5.png]

**Figure 5.4.** Figure 5.4: Best response learning when N = 1 using cem (Rubinstein, 1999) and the structural result in Thm. 5.1; the curves show the mean and the 95% confidence interval from evaluations with 20 random seeds; hyperparameters are listed in Appendix F. Remark 5.2 (Finite and stat…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p216_5.png]

**Figure 5.5.** Figure 5.5: shows the value of a pbe when N = 1. Interestingly, the expected cost is the highest when b(1) (belief of compromise) is 0.35 rather than5 0.5. 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 2 4 6 8 alpha vectors J ⋆ D(b) (5.11) b(1) [PITH_FULL_IMAGE:figures/full_fig_p216_5_5.png]

**Figure 5.6.** Figure 5.6: Time required to compute a Perfect Bayesian equilibrium (pbe) of Γ (5.1) with hsvi (Alg. 1, Horák et al., 2017) for different values of N ( [PITH_FULL_IMAGE:figures/full_fig_p216_5_6.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p217_5.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p218_5.png]

**Figure 5.7.** Figure 5.7: Conjectural Online Learning (col); the figure illustrates a time step during which player k updates its conjecture θ (k) t and its strategy πk,t. At time t, player k computes its action as follows7 . (a (k) t ∼ πk,t) ∈ R(k, θ (k) t , bt, J (πt) k , ℓk, h (k) t ) ≜ ar…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p219_5.png]

**Figure 5.8.** Figure 5.8: To manage this complexity for large instantiations of Γθt , we estimate E in (5.12) using Monte-Carlo samples. Note that (5.12) computes the next action as if the conjectures were true, i.e., the action is computed based on (enforced) certainty equivalence (p. 185, B…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p220_5.png]

**Figure 5.9.** Figure 5.9: Illustration of Thm. 5.3; the x-axis indicates the lookahead horizon ℓk; the yaxis indicates the factor of the performance bound in (5.13b); the curves relate to different discount factors γ. After computing (5.12) and executing the corresponding action, player k re…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p224_5.png]

**Figure 5.10.** Figure 5.10: The arrival rate function (5.18) used in Scenario 5.2; the blue curve shows the arrival rate λ(t); the red curve shows the trend of λ(t) without the periodic effects; and the shaded black curve shows the number of arrivals. 20 40 60 80 100 120 140 160 180 200 220 24…

**Figure 5.11.** Figure 5.11: Estimated distributions of the number of clients and the priority-weighted sum of ids alerts ot (5.4) during different arrival rates λ(t) (5.18) based on the apt actions listed in [PITH_FULL_IMAGE:figures/full_fig_p226_5_11.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p227_5.png]

**Figure 5.12.** Figure 5.12: Evaluation results for Scenario 5.1; values indicate the mean; the shaded areas and the error bars indicate the 95% confidence interval based on 20 random seeds; hyperparameters are listed in Appendix F [PITH_FULL_IMAGE:figures/full_fig_p228_5_12.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p229_5.png]

**Figure 5.13.** Figure 5.13: Evaluation results for Scenario 5.2; values indicate the mean; the shaded areas and the error bars indicate the 95% confidence interval based on 20 random seeds; hyperparameters are listed in Appendix F [PITH_FULL_IMAGE:figures/full_fig_p229_5_13.png]

**Figure 5.14.** Figure 5.14: Evaluation results for Scenario 5.3; values indicate the mean; the shaded areas and the error bars indicate the 95% confidence interval based on 20 random seeds; hyperparameters are listed in Appendix F [PITH_FULL_IMAGE:figures/full_fig_p230_5_14.png]

**Figure 5.15.** Figure 5.15: shows the evolution of ρ (D) t . We observe that ρ (D) t starts from a uniform distribution over ΘD and as t → ∞, it concentrates on the set of consistent conjectures Θ⋆ D (5.16a). 1 0.1 t = 1 1 0.1 t = 5 1 0.1 t = 10 1 0.2 t = 15 1 0.2 t = 20 1 0.2 t = 25 1 0.2 t =…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p231_5.png]

**Figure 5.16.** Figure 5.16: shows the evolution of the defender’s conjecture distributions µt and ρ (D) t (5.14). We observe that both distributions converge, which is consistent with Thm. 5.4. The convergence of µt is significantly faster than that of ρ (D) t . We believe this difference is b…

**Figure 5.17.** Figure 5.17: shows the percentage of blocked attacker and client traffic when running the snort idps (Roesch, 1999) and col on the digital twin. We observe that both block some client traffic and fail to block some attacker traffic, which is expected considering the false ids al…

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p232_5.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p233_5.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p235_5.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p248_5.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p249_5.png]

**Figure 5.18.** Figure 5.18: Berk-Nash equilibria of the example instantiation of Γ when p = 1 2 . F Hyperparameters The hyperparameters used for the evaluation in this paper are listed in [PITH_FULL_IMAGE:figures/full_fig_p250_5_18.png]

**Figure 5.** Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p251_5.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p256_6.png]

**Figure 6.1.** Figure 6.1: b. One option is to include the latent variables in the graph ( [PITH_FULL_IMAGE:figures/full_fig_p256_6_1.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p257_6.png]

**Figure 6.3.** Figure 6.3: Determining causal effect identifiability (Def. 6.1) from causal graphs. Do-calculus is complete in that it allows us to derive all identifiable causal effects (Cor. 3.4.2, Pearl, 2009)(Thm. 23, Shpitser and Pearl, 2008). Consequently, one can prove identifiability b…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p259_6.png]

**Figure 6.4.** Figure 6.4: Two causal graphs and the corresponding sets of pomiss; J is the target variable, and all other variables are manipulative. 7Since there is no bidirected edge between Z and Y in GX , they belong to different Ccomponents, which means that their joint probability fact…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p260_6.png]

**Figure 6.5.** Figure 6.5: The cage-2 scenario (cage-2, 2022): a defender aims to protect a networked system against an Advanced Persistent Threat (apt) caused by an attacker while maintaining services for clients; the system configuration is listed in Appendix D [PITH_FULL_IMAGE:figures/ful…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p261_6.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p262_6.png]

**Figure 6.6.** Figure 6.6: Transition diagram of the intrusion state Ii,t (6.5); self-transitions are not shown; disks represent states; arrows represent state transitions; labels indicate conditions for state transition; the initial state is Ii,1 = U. Nodes of the target system provide servic…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p263_6.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p264_6.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p265_6.png]

**Figure 6.7.** Figure 6.7: Causal (summary) graph of (M1) for node i (Ch. 10, Peters et al., 2017); plate notation is used to represent sets of variables (Buntine, 1994). The defender problem in CAGE-2 Given (M1) and the defender objective J (6.9), the problem for the defender can be stated as…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p268_6.png]

**Figure 6.** Figure 6: ) [PITH_FULL_IMAGE:figures/full_fig_p269_6.png]

**Figure 6.8.** Figure 6.8: Causal-Partially Observed Monte-Carlo Planning (c-pomcp, Alg. 6.1); the figure illustrates one time step during which (i) a particle filter is used to compute an approximate belief state bbt (6.14); (ii) a causal graph (Def. 2.2.1, Pearl, 2009) (see [PITH_FULL_IMAGE…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p270_6.png]

**Figure 6.9.** Figure 6.9: Reduction of the size of the search tree by pruning the intervention space |2 Xt | to the set of pomiss P ⋆ G; the x-axis indicates the tree depth T ; curves relate to the factor in (6.16). Unfortunately, computing P⋆ G is generally intractable, as stated below. Prop…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p272_6.png]

**Figure 6.10.** Figure 6.10: Tree search in c-pomcp; a search tree is constructed iteratively where each iteration consists of the four phases above. Tree policy A node at depth k of the tree is associated with a history hk (6.11) and stores two variables: the average objective value Jb(hk) (6.…

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p273_6.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p274_6.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p275_6.png]

**Figure 6.11.** Figure 6.11: Comparing c-pomcp (green curves) with baselines: cardiff-ppo (Vyas et al., 2023) (blue curves) and pomcp (Silver and Veness, 2010) (red curves); rows indicate the evaluation scenario; the curves show the mean value from evaluations with 3 random seeds; shaded areas …

**Figure 6.12.** Figure 6.12: Effect of the pruning of the search tree in c-pomcp. Method Training (min) Search (s) Objective J˜(πD) (6.13) cardiff-ppo[481] 2000 10−4 −429 ± 167 c-pomcp 0 30 −13.32 ± 0.18 pomcp [416] 0 30 −29.51 ± 2.00 [PITH_FULL_IMAGE:figures/full_fig_p278_6_12.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p278_6.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p279_6.png]

**Figure 6.** Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p288_6.png]

**Figure 6.13.** Figure 6.13: Workflow graph GW in cage-2 (cage-2, 2022); circles represent nodes of the target system ( [PITH_FULL_IMAGE:figures/full_fig_p291_6_13.png]

**Figure 1.** Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p294_1.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

297 extracted references · 52 canonical work pages

[1]

Behavioral and Game- Theoretic Security Investments in Interdependent Systems Modeled by At- tack Graphs

Mustafa Abdallah, Parinaz Naghizadeh, Ashish R. Hota, Timothy Ca- son, Saurabh Bagchi, and Shreyas Sundaram. “Behavioral and Game- Theoretic Security Investments in Interdependent Systems Modeled by At- tack Graphs”. In: IEEE Transactions on Control of Network Systems7.4 (2020), pp. 1585–1596.doi: 10.1109/TCNS.2020.2988007

arXiv 2020
[2]

Synchronous Byzantine Agreement with Expected O(1) Rounds, Ex- pected Communication, and Optimal Resilience

Ittai Abraham, Srinivas Devadas, Danny Dolev, Kartik Nayak, and Ling Ren. “Synchronous Byzantine Agreement with Expected O(1) Rounds, Ex- pected Communication, and Optimal Resilience”. In:Financial Cryptogra- phy and Data Security: 23rd International Conference, FC 2019, Frigate Bay, St. Kitts and Nevis, February 18–22, 2019, Revised Selected Papers. St. ...

work page doi:10.1007/978-3-030-32101-7_20 2019
[3]

Causative Cyberat- tacks on Online Learning-Based Automated Demand Response Systems

Samrat Acharya, Yury Dvorkin, and Ramesh Karri. “Causative Cyberat- tacks on Online Learning-Based Automated Demand Response Systems”. In: IEEE Transactions on Smart Grid12.4 (2021), pp. 3548–3559.doi: 10 .1109/TSG.2021.3067896

arXiv 2021
[4]

ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning

Iman Akbari, Ezzeldin Tahoun, Mohammad A. Salahuddin, Noura Limam, and Raouf Boutaba. “ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning”. In:NOMS IEEE/IFIP Network Operations and Management Symposium. 2020, pp. 1–9.doi: 10.1109/NOMS47738.2020.9 110426

arXiv 2020
[5]

Dynkin games

MarieAlario-Nazaret,Jean-PierreLepeltier,andBernard.Marchal.“Dynkin games”. In: Stochastic Differential Systems. Ed. by M. Kohlmann and N. Christopeit. Berlin, Heidelberg: Springer Berlin Heidelberg, 1982, pp. 23–

1982
[6]

Aliprantis and Kim C

Charalambos D. Aliprantis and Kim C. Border. Infinite Dimensional Analysis: a Hitchhiker’s Guide . Berlin; London: Springer, 2006. isbn: 9783540326960 3540326960.doi: 10.1007/3-540-29587-9

doi:10.1007/3-540-29587-9 2006
[7]

Allen B. Danette. Digital Twins and Living Models at NASA. Digital Twin Summit. 2021

2021
[8]

Noncooperative games for control of networked systems

Tansu Alpcan. “Noncooperative games for control of networked systems”. PhD thesis. University of Illinois at Urbana-Champaign, 2006. 291 292 References

2006

Show all 297 references

[9]

A game theoretic analysis of intrusion detection in access control systems

Tansu Alpcan and Tamer Basar. “A game theoretic analysis of intrusion detection in access control systems”. In:2004 43rd IEEE Conference on De- cision and Control (CDC) (IEEE Cat. No.04CH37601). Vol. 2. 2004, 1568– 1573 Vol.2.doi: 10.1109/CDC.2004.1430267

2004 arXiv
[10]

A game theoretic approach to decision and analysis in network intrusion detection

Tansu Alpcan and Tamer Basar. “A game theoretic approach to decision and analysis in network intrusion detection”. In:42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475). Vol. 3. 2003, 2595–2600 Vol.3.doi: 10.1109/CDC.2003.1273013

2003 arXiv
[11]

Tansu Alpcan and Tamer Basar.Network Security: A Decision and Game- Theoretic Approach. 1st. USA: Cambridge University Press, 2010. isbn: 0521119324

2010
[12]

Constrained Markov Decision Processes

Eitan Altman. Constrained Markov Decision Processes. Chapman and Hall, 1999

1999
[13]

A Jamming Game in Wireless Networks with Transmission Cost

Eitan Altman, Konstantin Avrachenkov, and Andrey Garnaev. “A Jamming Game in Wireless Networks with Transmission Cost”. In:NET-COOP. 2007

2007
[14]

Constrained Markov Games: Nash Equilibria

Eitan Altman and Adam Shwartz. “Constrained Markov Games: Nash Equilibria”. In:Advances in Dynamic Games and Applications. Birkhäuser Boston, 2000, pp. 213–221

2000
[15]

Prime: Byzantine ReplicationunderAttack

Yair Amir, Brian Coan, Jonathan Kirsch, and John Lane. “Prime: Byzantine ReplicationunderAttack”.In: IEEE Transactions on Dependable and Secure Computing 8.4 (2011), pp. 564–577.doi: 10.1109/TDSC.2010.70

2011 doi
[16]

A Novel Differential Game Model-Based Intrusion Response Strategy in Fog Computing

Xingshuo An, Fuhong Lin, Shenggang Xu, Li Miao, Chao Gong, and Liran Ma. “A Novel Differential Game Model-Based Intrusion Response Strategy in Fog Computing”. In:Sec. and Commun. Netw.(2018). issn: 1939-0114. doi: 10.1155/2018/1821804

2018 doi
[17]

Anderson, Anant Kharkar, Bobby Filar, David Evans, and Phil Roth

Hyrum S. Anderson, Anant Kharkar, Bobby Filar, David Evans, and Phil Roth. Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning. 2018. arXiv:1801.08917 [cs.CR]

2018 arXiv
[18]

Anderson.Computer Security Threat Monitoring and Surveillance

James P. Anderson.Computer Security Threat Monitoring and Surveillance. Technical Report Contract 79F26400. Prepared for the United States Air Force. Fort Washington, PA: James P. Anderson Co., 1980

1980
[19]

Anderson

Ross J. Anderson. Security Engineering: A Guide to Building Depend- able Distributed Systems. 1st. USA: John Wiley & Sons, Inc., 2001.isbn: 0471389226

2001
[20]

Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation

Alex Andrew, Sam Spillard, Joshua Collyer, and Neil Dhir. “Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation”. In: Proceedings of the ML4Cyber workshop, ICML 2022, Baltimore, USA, July 17-23, 2022. PMLR, 2022

2022
[21]

Playbook Oriented Cyber Response

Andy Applebaum, Shawn Johnson, Michael Limiero, and Michael Smith. “Playbook Oriented Cyber Response”. In: 2018 National Cyber Summit (NCS). 2018, pp. 8–15.doi: 10.1109/NCS.2018.00007. References 293

2018
[22]

Bridging Automated to Autonomous Cyber De- fense: Foundational Analysis of Tabular Q-Learning

Andy Applebaum et al. “Bridging Automated to Autonomous Cyber De- fense: Foundational Analysis of Tabular Q-Learning”. In:Proceedings of the 15th ACM Workshop on Artificial Intelligence and Security. 2022. doi: 10 .1145/3560830.3563732

2022
[23]

Auto- nomicdefense:Thwartingautomatedattacksviareal-timefeedbackcontrol

Derek Armstrong, Sam Carter, Gregory Frazier, and Tiffany Frazier. “Auto- nomicdefense:Thwartingautomatedattacksviareal-timefeedbackcontrol”. In: Complexity 9.2 (2003), pp. 41–48.doi: https://doi.org/10.1002/cp lx.20011. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002 ...

2003 doi
[24]

Arrow and Jerry R

Kenneth J. Arrow and Jerry R. Green.Notes on Expectations Equilibria in Bayesian Settings Institute for Mathematical Studies in the Social Sciences. Working Paper No. 33, Stanford University. 1973

1973
[25]

Robert B. Ash. Real Analysis and Probability. Bibliographie-P. Academic Press, 1972.isbn: 9780120652013

1972
[26]

Optimal control of Markov processes with incomplete state information

Karl Johan Åström. “Optimal control of Markov processes with incomplete state information”. In:Journal of Mathematical Analysis and Applications 10.1 (1965), pp. 174–205.issn: 0022-247X.doi: https://doi.org/10.101 6/0022-247X(65)90154-X

1965
[27]

Adaptive Control (2 ed.)En- glish

Karl Johan Åström and Björn Wittenmark. Adaptive Control (2 ed.)En- glish. Addison-Wesley, 1995.isbn: 0-201-55866-1.doi: 10.2307/1269433

1995 doi
[28]

Distributed Computing: Fundamentals, Simulations and Advanced Topics

Hagit Attiya and Jennifer Welch. Distributed Computing: Fundamentals, Simulations and Advanced Topics. Hoboken, NJ, USA: John Wiley & Sons, Inc., 2004.isbn: 0471453242

2004
[29]

Finite-time Analysis of the Multiarmed Bandit Problem

Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. “Finite-time Analysis of the Multiarmed Bandit Problem”. In:Machine Learning47.2 (May 2002), pp. 235–256.issn: 1573-0565.doi: 10.1023/A:1013689704352

2002 doi
[30]

Design methods for fault-tolerant navigation computers

Algirdas Avižienis. Design methods for fault-tolerant navigation computers. Tech. rep. 1969

1969
[31]

Design of fault-tolerant computers

Algirdas Avižienis. “Design of fault-tolerant computers”. In:Proceedings of the November 14-16, 1967, fall joint computer conference. 1967, pp. 733– 743

1967
[32]

isbn: 978-3-540-39518-8
[33]

Fault-Tolerant Systems

Algirdas Avižienis. “Fault-Tolerant Systems”. In: IEEE Transactions on Computers C-25.12 (1976), pp. 1304–1312.doi: 10.1109/TC.1976.1674 598

1976
[34]

Strategic Defense Against Stealthy Link Flooding Attacks: A Signaling Game Approach

Abdullah Aydeger, Mohammad Hossein Manshaei, Mohammad Ashiqur Rahman, and Kemal Akkaya. “Strategic Defense Against Stealthy Link Flooding Attacks: A Signaling Game Approach”. In: IEEE Transactions on Network Science and Engineering8.1 (2021), pp. 751–764.doi: 10.110 9/TNSE.202...

2021
[35]

Deploying Intrusion-Tolerant SCADAforthePowerGrid

AmyBabay,JohnSchultz,ThomasTantillo,SamuelBeckley,EamonJordan, Kevin Ruddell, Kevin Jordan, and Yair Amir. “Deploying Intrusion-Tolerant SCADAforthePowerGrid”.In: 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 2019, pp. 328–335. d...

2019
[36]

Network-Attack-Resilient Intrusion-Tolerant SCADA for the Power Grid

Amy Babay, Thomas Tantillo, Trevor Aron, Marco Platania, and Yair Amir. “Network-Attack-Resilient Intrusion-Tolerant SCADA for the Power Grid”. In: 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 2018, pp. 255–266.doi: 10.1109/DSN....

2018 doi
[37]

Learning and Information Manipulation: Repeated Hypergames for Cyber-Physical Security

Craig Bakker, Arnab Bhattacharya, Samrat Chatterjee, and Draguna L. Vrabie. “Learning and Information Manipulation: Repeated Hypergames for Cyber-Physical Security”. In:IEEE Control Systems Letters4.2 (2020).doi: 10.1109/LCSYS.2019.2925681

2020
[38]

Commitment Without Regrets: Online Learning in Stackelberg Security Games

Maria-Florina Balcan, Avrim Blum, Nika Haghtalab, and Ariel D. Procaccia. “Commitment Without Regrets: Online Learning in Stackelberg Security Games”.In: Proceedings of the Sixteenth ACM Conference on Economics and Computation. EC ’15. Portland, Oregon, USA: Association for Co...

2015 doi
[39]

Sur les opérations dans les ensembles abstraits et leur appli- cation aux équations intégrales

Stefan Banach. “Sur les opérations dans les ensembles abstraits et leur appli- cation aux équations intégrales”. fre. In:Fundamenta Mathematicae(1922)

1922
[40]

Autonomous Computer Security Game: Techniques, Strategy and Investigation

Tiffany Bao. “Autonomous Computer Security Game: Techniques, Strategy and Investigation”. PhD thesis. Carnegie Mellon University, 2018

2018
[41]

Barlow and Frank Proschan.Mathematical Theory of Reliability

Richard E. Barlow and Frank Proschan.Mathematical Theory of Reliability. John Wiley & Sons, Hoboken, 1965

1965
[42]

Dynamic Noncooperative Game The- ory

Tamer Basar and Geert Jan Olsder. Dynamic Noncooperative Game The- ory. Classics in Applied Mathematics. Society for Industrial and Applied Mathematics, 1999.isbn: 9780898714296

1999
[43]

Reward Shaping for Happier Autonomous Cyber Security Agents

Elizabeth Bates, Vasilios Mavroudis, and Chris Hicks. “Reward Shaping for Happier Autonomous Cyber Security Agents”. In:Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security. AISec ’23. , Copen- hagen, Denmark, 2023, pp. 221–232.isbn: 9798400702600.doi: ...

2023
[44]

Decision Theory: An Introduction to Dynamic Programming and Sequential Decisions

John Bather. Decision Theory: An Introduction to Dynamic Programming and Sequential Decisions. USA: John Wiley and Sons, Inc., 2000. isbn: 0471976490

2000
[45]

Transition-Independent Decentralized Markov Decision Processes

Raphen Becker, Shlomo Zilberstein, Victor Lesser, and Claudia V. Gold- man. “Transition-Independent Decentralized Markov Decision Processes”. In: Joint Conference on Autonomous Agents and Multiagent Systems. Mel- bourne, Australia, 2003.isbn: 1581136838.doi: 10.1145/860575.860...

2003
[46]

A Markovian decision process

Richard Bellman. “A Markovian decision process”. In: Journal of Mathe- matics and Mechanics6.5 (1957), pp. 679–684

1957
[47]

Dynamic Programming

Richard Bellman. Dynamic Programming. Dover Publications, 1957.isbn: 9780486428093

1957
[48]

Dynamic Policy Decision/En- forcement Security Zoning Through Stochastic Games and Meta Learning

Yahuza Bello and Ahmed Refaey Hussein. “Dynamic Policy Decision/En- forcement Security Zoning Through Stochastic Games and Meta Learning”. In: IEEE Transactions on Network and Service Management(2024), pp. 1–

2024
[49]

doi: 10.1109/TNSM.2024.3481662

2024
[50]

Limiting Behavior of Posterior Distributions when the Model is Incorrect

Robert H. Berk. “Limiting Behavior of Posterior Distributions when the Model is Incorrect”. In:The Annals of Mathematical Statistics37.1 (1966), pp. 51–58.doi: 10.1214/aoms/1177699597

1966
[51]

Dota 2 with Large Scale Deep Reinforcement Learning

Christopher Berner et al. “Dota 2 with Large Scale Deep Reinforcement Learning”. In:ArXiv abs/1912.06680 (2019)

2019 arXiv
[52]

Bertsekas

Dimitri P. Bertsekas. Dynamic Programming and Optimal Control. 3rd. Vol. I. Belmont, MA, USA: Athena Scientific, 2005

2005
[53]

Bertsekas

Dimitri P. Bertsekas. Lessons from AlphaZero for Optimal, Model Predic- tive, and Adaptive Control. Athena Scientific optimization and computation series. Athena Scientific, 2022.isbn: 9781886529175

2022
[54]

Bertsekas

Dimitri P. Bertsekas. Reinforcement learning and optimal control. Athena Scientific, 2019

2019
[55]

Bertsekas

Dimitri P. Bertsekas. Rollout, Policy Iteration, and Distributed Reinforce- ment Learning. Athena scientific optimization and computation series. Athena Scientific., 2021.isbn: 9781886529076

2021
[56]

Bertsekas and Steven E

Dimitri P. Bertsekas and Steven E. Shreve. Stochastic Optimal Control: The Discrete Time Case. Mathematics in science and engineering. Academic Press, 1978.isbn: 9780120932603

1978
[57]

Bertsekas and John N

Dimitri P. Bertsekas and John N. Tsitsiklis.Neuro-dynamic programming. Belmont, MA: Athena Scientific, 1996

1996
[58]

State Machine Replication for the Masses with BFT-SMART

Alysson Bessani, João Sousa, and Eduardo E.P. Alchieri. “State Machine Replication for the Masses with BFT-SMART”. In: 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. 2014, pp. 355–362.doi: 10.1109/DSN.2014.43

2014 doi
[59]

The Crutial Way of Critical Infrastructure Protection

Alysson Bessani, Paulo Sousa, Miguel Correia, Nuno Neves, and Paulo Verís- simo. “The Crutial Way of Critical Infrastructure Protection”. In:Security & Privacy, IEEE6 (Jan. 2009), pp. 44–51.doi: 10.1109/MSP.2008.158

2009 doi
[60]

Reinforcement Learning for POMDP: Partitioned Roll- out and Policy Iteration With Application to Autonomous Sequential Repair Problems

Sushmita Bhattacharya, Sahil Badyal, Thomas Wheeler, Stephanie Gil, and Dimitri Bertsekas. “Reinforcement Learning for POMDP: Partitioned Roll- out and Policy Iteration With Application to Autonomous Sequential Repair Problems”. In:IEEE Robotics and Automation Letters5.3 (2020...

2020
[61]

Building secure and reliable network applications

Kenneth P. Birman. “Building secure and reliable network applications”. In: Worldwide Computing and Its Applications. Ed. by Takashi Masuda, Yoshi- fumi Masunaga, and Michiharu Tsukamoto. Berlin, Heidelberg: Springer Berlin Heidelberg, 1997, pp. 15–28.isbn: 978-3-540-69430-4

1997
[62]

Convergence of probability measures

Patrick Billingsley. Convergence of probability measures. Second. Wiley Se- ries in Probability and Statistics: Probability and Statistics. A Wiley- Interscience Publication. New York: John Wiley & Sons Inc., 1999.isbn: 0-471-19745-9

1999
[63]

The process group approach to reliable distributed computing

Kenneth P Birman. “The process group approach to reliable distributed computing”. In:Communications of the ACM36.12 (1993)

1993
[64]

Machine Learning Cyberattack and Defense Strate- gies

John A. Bland, Mikel D. Petty, Tymaine S. Whitaker, Katia P. Maxwell, and Walter Alan Cantrell. “Machine Learning Cyberattack and Defense Strate- gies”. In:Computers & Security92 (2020), p. 101738.issn: 0167-4048.doi: https://doi.org/10.1016/j.cose.2020.101738

2020
[65]

Introduction to Computer Security

Matt Bishop. Introduction to Computer Security. Addison-Wesley Profes- sional, 2004.isbn: 0321247442

2004
[66]

Blackwell and M.A

D.A. Blackwell and M.A. Girshick. Theory of Games and Statistical De- cisions. Dover Books on Mathematics. Dover Publications, 1979. isbn: 9780486638317

1979
[67]

La théorie du jeu et les équations intégrales à noyau symétrique

Emile Borel. La théorie du jeu et les équations intégrales à noyau symétrique. Comptes Rendus de l’Académie des Sciences, 173, pp. 1304-308. English translation by Leonard J. Savage, 1953, Econometrica, 21, pp. 97-100. http://www.jstor.org/stable/1906946. 1921

1953
[68]

Intrusion Response as a Resource Allocation Problem

Michael Bloem, Tansu Alpcan, and Tamer Basar. “Intrusion Response as a Resource Allocation Problem”. In:Proceedings of the 45th IEEE Conference on Decision and Control. 2006, pp. 6283–6288.doi: 10.1109/CDC.2006.37 6981

2006 doi
[69]

Gamifying machine learning for stronger security and AI models

William Blum. Gamifying machine learning for stronger security and AI models. https://www.microsoft.com/security/blog/2021/04/08/gam ifying- machine- learning- for- stronger- security- and- ai- models/. 2021

2021
[70]

Un théorème sur les systèmes de formes linéaires à déterminant symétrique gauche

Emile Borel. Un théorème sur les systèmes de formes linéaires à déterminant symétrique gauche. Comptes Rendus de l’Académie des Sciences, 173, pp. 925-927. 1926

1926
[71]

Sur les jeux où interviennent l’hasard et l’habileté des joueurs

Emile Borel. Sur les jeux où interviennent l’hasard et l’habileté des joueurs. Theorie des Probabilités, Paris: Librairie Scientifique, J. Hermann, pp. 204-
[72]

Asynchronous Consensus and Broadcast Protocols

Gabriel Bracha and Sam Toueg. “Asynchronous Consensus and Broadcast Protocols”. In:J. ACM32.4 (Oct. 1985), pp. 824–840.issn: 0004-5411.doi: 10.1145/4221.214134

1985
[73]

Sur les systèmes de formes linéaires à déterminant symétrique gauche et la théorie générale du jeu.Comptes Rendus de l’Académie des Sciences, 184, pp

Emile Borel. Sur les systèmes de formes linéaires à déterminant symétrique gauche et la théorie générale du jeu.Comptes Rendus de l’Académie des Sciences, 184, pp. 52-53. English translation by Leonard J. Savage, 1953, Econometrica,21,pp.116-117.https://www.jstor.org/stable/19...

1953
[74]

An Artificial Arms Race: Could it Improve Mobile Malware Detectors?

Raphael Bronfman-Nadas, Nur Zincir-Heywood, and John T. Jacobs. “An Artificial Arms Race: Could it Improve Mobile Malware Detectors?” In:2018 Network Traffic Measurement and Analysis Conference (TMA). 2018

2018
[75]

Vivek S. Borkar. Stochastic Approximation. Cambridge University Press, 2008

2008
[76]

George W. Brown. Iterative solution of games by fictitious play. Activity analysis of production and allocation. 1951

1951
[77]

Auctions with untrustworthy bidders

S. Braynov and T. Sandholm. “Auctions with untrustworthy bidders”. In: EEE International Conference on E-Commerce, 2003. CEC 2003. 2003, pp. 363–369.doi: 10.1109/COEC.2003.1210272

2003 arXiv
[78]

Bayesian Games for Threat Predic- tion and Situation Analysis

Joel Brynielsson and Stefan Arnborg. “Bayesian Games for Threat Predic- tion and Situation Analysis”. In:Proceedings of the 7th International Con- ference on Information Fusion. Vol. 2. Stockholm, June 2004

2004
[79]

A robust layered control system for a mobile robot

Rodney Brooks. “A robust layered control system for a mobile robot”. In: IEEE Journal on Robotics and Automation2.1 (1986), pp. 14–23.doi: 10 .1109/JRA.1986.1087032

1986
[80]

Operations for learning with graphical models

Wray L. Buntine. “Operations for learning with graphical models”. In:J. Artif. Int. Res.2.1 (Dec. 1994), pp. 159–225.issn: 1076-9757.doi: 10.161 3/jair.62

1994
[81]

Aspects of adaptive reconfiguration in a scalable intrusion tolerant system

Stephanie Bryant and Feiyi Wang. “Aspects of adaptive reconfiguration in a scalable intrusion tolerant system”. In:Complexity 9.2 (2003).doi: https ://doi.org/10.1002/cplx.20007. eprint: https://onlinelibrary.wile y.com/doi/pdf/10.1002/cplx.20007

2003 doi
[82]

Security and Cooperation in Wireless Networks: Thwarting Malicious and Selfish Behavior in the Age of Ubiquitous Computing

Levente Buttyan and Jean-Pierre Hubaux. Security and Cooperation in Wireless Networks: Thwarting Malicious and Selfish Behavior in the Age of Ubiquitous Computing. USA: Cambridge University Press, 2007. isbn: 0521873711. 298 References

2007
[83]

Security games for vehicular net- works

Sonja Buchegger and Tansu Alpcan. “Security games for vehicular net- works”. In:2008 46th Annual Allerton Conference on Communication, Con- trol, and Computing. 2008, pp. 244–251.doi: 10.1109/ALLERTON.2008.47 97563

2008 doi
[84]

Christian Cachin, Rachid Guerraoui, and Lus Rodrigues.Introduction to Re- liable and Secure Distributed Programming. 2nd. Springer Publishing Com- pany, Incorporated, 2011.isbn: 3642152597

2011
[85]

A framework for building dependable systems

Alan Burns and Andrew M. Lister. “A framework for building dependable systems”. In:The Computer Journal34.2 (1991), pp. 173–181

1991
[86]

CYBERSHIELD: A Competitive Simula- tion Environment for Training AI in Cybersecurity

José Álvaro Fernández Carrasco, Iñigo Amonarriz Pagola, Raúl Orduna Urrutia, and Rodrigo Román. “CYBERSHIELD: A Competitive Simula- tion Environment for Training AI in Cybersecurity”. In:2024 11th Interna- tional Conference on Internet of Things: Systems, Management and Securi...

2024
[87]

Rational Exchange - A Formal Model Based on Game Theory

Levente Buttyán and Hubaux Jean-Pierre. “Rational Exchange - A Formal Model Based on Game Theory”. In:Electronic Commerce. Ed. by Ludger Fiege, Gero Mühl, and Uwe Wilhelm. Berlin, Heidelberg: Springer Berlin Heidelberg, 2001, pp. 114–126.isbn: 978-3-540-45598-1

2001
[88]

Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes

Anthony Cassandra, Michael L. Littman, and Nevin L. Zhang. “Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes”. In:Proceedings of the Thirteenth Conference on Uncer- tainty in Artificial Intelligence. UAI’97. Providence, Rhode...

1997
[89]

TTCP CAGE Challenge 2

cage-2. “TTCP CAGE Challenge 2”. In:AAAI-22 Workshop on Artificial Intelligence for Cyber Security (AICS). https://github.com/cage-chall enge/cage-challenge-2. 2022

2022
[90]

Interdependent Strategic Security Risk Management With Bounded Rationality in the Internet of Things

Juntao Chen and Quanyan Zhu. “Interdependent Strategic Security Risk Management With Bounded Rationality in the Internet of Things”. In:IEEE Transactions on Information Forensics and Security14.11 (2019), pp. 2958–

2019
[91]

Analysis of the cyber attack on the Ukrainian power grid

Defense Use Case. “Analysis of the cyber attack on the Ukrainian power grid”. In: Electricity Information Sharing and Analysis Center (E-ISAC) 388 (2016), pp. 1–29

2016
[92]

On selfish and malicious behaviors in wireless networks - a non- cooperative game theoretic approach

Lin Chen. “On selfish and malicious behaviors in wireless networks - a non- cooperative game theoretic approach”. Theses. Télécom ParisTech, Nov. 2008

2008
[93]

Practical Byzantine Fault Tolerance and Proactive Recovery

Miguel Castro and Barbara Liskov. “Practical Byzantine Fault Tolerance and Proactive Recovery”. In:ACM Trans. Comput. Syst.20.4 (Nov. 2002), pp. 398–461.issn: 0734-2071.doi: 10.1145/571637.571640

2002
[94]

Houghton Mifflin, 1971.isbn: 0395053145

Yuan Chow, Herbert Robbins, and David Siegmund.Great expectations: The theory of optimal stopping. Houghton Mifflin, 1971.isbn: 0395053145

1971
[95]

Attested Append-Only Memory: Making Adversaries Stick to Their Word

Byung-Gon Chun, Petros Maniatis, Scott Shenker, and John Kubiatowicz. “Attested Append-Only Memory: Making Adversaries Stick to Their Word”. In: Proceedings of Twenty-First ACM SIGOPS Symposium on Operating Systems Principles. SOSP ’07. Stevenson, Washington, USA: Association ...

2007
[96]

Security investment under cognitive con- straints: A Gestalt Nash equilibrium approach

Juntao Chen and Quanyan Zhu. “Security investment under cognitive con- straints: A Gestalt Nash equilibrium approach”. In:2018 52nd Annual Con- ference on Information Sciences and Systems (CISS). 2018, pp. 1–6.doi: 10.1109/CISS.2018.8362263

2018
[97]

Upright Cluster Services

Allen Clement, Manos Kapritsos, Sangmin Lee, Yang Wang, Lorenzo Alvisi, Mike Dahlin, and Taylor Riche. “Upright Cluster Services”. In:Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles. SOSP ’09. Big Sky, Montana, USA, 2009, pp. 277–290.isbn: 97816055...

2009
[98]

RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation

Zelei Cheng, Xian Wu, Jiahao Yu, Sabrina Yang, Gang Wang, and Xinyu Xing. RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation. 2024. arXiv:2405.03064 [cs.LG]. References 299

2024 arXiv
[99]

Cormen, Charles E

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms. 4th. The MIT Press, 2022

2022
[100]

Worm-IT – A wormhole-based intrusion-tolerant group communication sys- tem

MiguelCorreia,NunoFerreiraNeves,LauCheukLung,andPauloVeríssimo. “Worm-IT – A wormhole-based intrusion-tolerant group communication sys- tem”. In: Journal of Systems and Software80.2 (2007), pp. 178–197.issn: 0164-1212. doi: https://doi.org/10.1016/j.jss.2006.03.034

2007 doi
[101]

Probability and Stochastics

Erhan Çınlar. Probability and Stochastics. Graduate Texts in Mathematics. Springer New York, 2011.isbn: 9780387878591

2011
[102]

A survey of convergence results on par- ticle filtering methods for practitioners

Dan Crisan and Arnaud Doucet. “A survey of convergence results on par- ticle filtering methods for practitioners”. In:IEEE Transactions on Signal Processing50.3 (2002), pp. 736–746.doi: 10.1109/78.984773

2002 doi
[103]

Network Management Fun- damentals

Alexander Clemm and Inc Cisco Systems. Network Management Fun- damentals. Cisco Press fundamentals series. Cisco Press, 2007. isbn: 9781587201370

2007
[104]

Citus: Distributed PostgreSQL for Data-Intensive Applications

Umur Cubukcu et al. “Citus: Distributed PostgreSQL for Data-Intensive Applications”. In:Proceedings of the 2021 International Conference on Man- agement of Data. SIGMOD ’21. Virtual Event, China, 2021. 300 References

2021
[105]

Formal verification of information flow security for a sim- ple arm-based separation kernel

Mads Dam, Roberto Guanciale, Narges Khakpour, Hamed Nemati, and Oliver Schwarz. “Formal verification of information flow security for a sim- ple arm-based separation kernel”. In:Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security. CCS ’13. Berli...

2013
[106]

HQ Replication: A Hybrid Quorum Protocol for Byzantine Fault Tolerance

James Cowling, Daniel Myers, Barbara Liskov, Rodrigo Rodrigues, and Li- uba Shrira. “HQ Replication: A Hybrid Quorum Protocol for Byzantine Fault Tolerance”. In:Proceedings of the 7th Symposium on Operating Sys- tems Design and Implementation. OSDI ’06. Seattle, Washington: US...

2006
[107]

Stealth Attacks and Protection Schemes for State Estimators in Power Systems

György Dán and Henrik Sandberg. “Stealth Attacks and Protection Schemes for State Estimators in Power Systems”. In:2010 First IEEE International Conference on Smart Grid Communications. 2010, pp. 214–219.doi: 10.11 09/SMARTGRID.2010.5622046

2010
[108]

Understanding fault-tolerant distributed systems

Flavin Cristian. “Understanding fault-tolerant distributed systems”. In: Communications of the ACM34.2 (1991), pp. 56–78

1991
[109]

A Fault Tolerant Optimization Algorithm based on Evolutionary Computation

Francisco Fernandez De Vega. “A Fault Tolerant Optimization Algorithm based on Evolutionary Computation”. In: 2006 International Conference on Dependability of Computer Systems. 2006, pp. 335–342.doi: 10 . 1109 /DEPCOS-RELCOMEX.2006.2

2006
[110]

Autonomous Cybersecurity: Evolving Challenges, Emerging Opportunities, and Future Research Trajectories

Ali Dehghantanha, Abbas Yazdinejad, and Reza M. Parizi. “Autonomous Cybersecurity: Evolving Challenges, Emerging Opportunities, and Future Research Trajectories”. In: Proceedings of the Workshop on Autonomous Cybersecurity. AutonomousCyber ’24. Salt Lake City, UT, USA: Associa...

2024
[111]

Generalized mean estimation in Monte-Carlo tree search

Tuan Dam, Pascal Klink, Carlo D’Eramo, Jan Peters, and Joni Pajarinen. “Generalized mean estimation in Monte-Carlo tree search”. In:Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelli- gence. IJCAI’20. Yokohama, Yokohama, Japan, 2021.isbn: 9780...

2021
[112]

On the complexity of computing Markov perfect equilibrium in general- sum stochastic games

Xiaotie Deng, Ningyuan Li, David Mguni, Jun Wang, and Yaodong Yang. “On the complexity of computing Markov perfect equilibrium in general- sum stochastic games”. In:National Science Review10.1 (Nov. 2022).issn: 2053-714X

2022
[114]

Finite State Markovian Decision Processes

Cyrus Derman. Finite State Markovian Decision Processes. Clinical Engi- neering Series. Academic Press, 1970.isbn: 9780122092503. References 301

1970
[115]

Intrusion tolerance in distributed computing systems

Yves Deswarte, Laurent Blain, and Jean-Charles Fabre. “Intrusion tolerance in distributed computing systems”. In:Proceedings. 1991 IEEE Computer Society Symposium on Research in Security and Privacy. IEEE Computer Society. 1991, pp. 110–110

1991
[116]

Maximum likeli- hood from incomplete data via the EM algorithm

Arhur P. Dempster, Nan M. Laird, and Donald B. Rubin. “Maximum likeli- hood from incomplete data via the EM algorithm”. In:Journal of the Royal Statistical Society: Series B39 (1977), pp. 1–38

1977
[117]

Byzantine Fault-Tolerant State-Machine Replication from a Systems Perspective

Tobias Distler. “Byzantine Fault-Tolerant State-Machine Replication from a Systems Perspective”. In: ACM Comput. Surv. 54.1 (Feb. 2021). issn: 0360-0300. doi: 10.1145/3436728

2021 doi
[118]

An Intrusion-Detection Model

Dorothy E. Denning. “An Intrusion-Detection Model”. In: IEEE Transac- tions on Software EngineeringSE-13.2 (1987), pp. 222–232

1987
[119]

State Transfer for Hypervisor-Based Proactive Recovery of Heterogeneous Replicated Ser- vices

Tobias Distler, Rüdiger Kapitza, and Hans Reiser. “State Transfer for Hypervisor-Based Proactive Recovery of Heterogeneous Replicated Ser- vices”. In:FERS-Mitteilungen 29.1 (2011)

2011
[120]

On the Minimal Synchronism Needed for Distributed Consensus

Danny Dolev, Cynthia Dwork, and Larry Stockmeyer. “On the Minimal Synchronism Needed for Distributed Consensus”. In: J. ACM 34.1 (Jan. 1987), pp. 77–97.issn: 0004-5411.doi: 10.1145/7531.7533

1987
[121]

FlipIt: The Game of “Stealthy Takeover

Marten van Dijk, Ari Juels, Alina Oprea, and Ronald L. Rivest. “FlipIt: The Game of “Stealthy Takeover””. In:Journal of Cryptology4 (Oct. 2013). issn: 1432-1378.doi: 10.1007/s00145-012-9134-5

2013 doi
[122]

Towards Autonomous Cyber Defense: Predictions from a cognitive model

Yinuo Du, Baptiste Prébot, Xiaoli Xi, and Cleotilde Gonzalez. “Towards Autonomous Cyber Defense: Predictions from a cognitive model”. In:Pro- ceedings of the Human Factors and Ergonomics Society Annual Meeting66.1 (2022), pp. 1121–1125.doi: 10.1177/1071181322661504

2022 doi
[123]

Resource-Efficient Byzantine Fault Tolerance

Tobias Distler, Christian Cachin, and Rüdiger Kapitza. “Resource-Efficient Byzantine Fault Tolerance”. In: IEEE Transactions on Computers 65.9 (2016), pp. 2807–2819.doi: 10.1109/TC.2015.2495213

2016
[124]

Optimal Network Security Hardening Using Attack Graph Games

Karel Durkota, Viliam Lisy, Branislav Bošansky, and Christopher Kiek- intveld. “Optimal Network Security Hardening Using Attack Graph Games”. In: Proceedings of the 24th International Conference on Artificial Intelli- gence. Buenos Aires, Argentina, 2015.isbn: 978-1-57735-738-4

2015
[125]

Constraints Sat- isfiability Driven Reinforcement Learning for Autonomous Cyber Defense

Ashutosh Dutta, Ehab Al-Shaer, and Samrat Chatterjee. “Constraints Sat- isfiability Driven Reinforcement Learning for Autonomous Cyber Defense”. In: 2021 International Conference on Autonomous Intelligent Cyber-defence Agents. 2021

2021
[126]

Online and Scalable Unsupervised Network Anomaly Detection Method

Juliette Dromard, Gilles Roudière, and Philippe Owezarski. “Online and Scalable Unsupervised Network Anomaly Detection Method”. In: IEEE Transactions on Network and Service Management14.1 (2017), pp. 34–47. doi: 10.1109/TNSM.2016.2627340

2017
[127]

A game-theoretic version of an optimal stopping prob- lem

Eugene B. Dynkin. “A game-theoretic version of an optimal stopping prob- lem”. In:Dokl. Akad. Nauk SSSR385 (1969), pp. 16–19

1969
[128]

Learning to Play an Adaptive Cyber Deception Game

Yinuo Du, Zimeng Song, Stephanie Milani, Cleotilde Gonzales, and Fei Fang. “Learning to Play an Adaptive Cyber Deception Game”. In:The 13th Work- shop on Optimization and Learning in Multiagent Systems, AAMAS(2022)

2022
[129]

Dynkin Games with Heterogeneous Beliefs

Erik Ekström, Kristoffer Glover, and Marta Leniec. “Dynkin Games with Heterogeneous Beliefs”. In: Journal of Applied Probability1 (2017). issn: 00219002

2017
[130]

Adversarial Reinforcement Learning in a Cyber Security Simulation

Richard Elderman, Leon J. J. Pater, Albert S. Thie, Mădălina M. Drugan, and Marco A Wiering. “Adversarial Reinforcement Learning in a Cyber Security Simulation”. In:ICAART. 2017

2017
[131]

Consensus in the Presence of Partial Synchrony

Cynthia Dwork, Nancy Lynch, and Larry Stockmeyer. “Consensus in the Presence of Partial Synchrony”. In:J. ACM35.2 (Apr. 1988), pp. 288–323. issn: 0004-5411.doi: 10.1145/42282.42283. 302 References

1988
[133]

JavaScript at ten years

Brendan Eich. “JavaScript at ten years”. In:Proceedings of the 10th ACM SIGPLAN International Conference on Functional Programming, ICFP 2005, Tallinn, Estonia, September 26-28, 2005. Ed. by Olivier Danvy and Benjamin C. Pierce. ACM, 2005, p. 129.doi: 10.1145/1086365.1086382

2005
[134]

Ericsson Mobility Report June 2024

Ericsson. Ericsson Mobility Report June 2024. https://www.ericsson.co m/en/reports-and-papers/mobility-report. 2024

2024
[135]

Berk-Nash Equilibrium: A Frame- work for Modeling Agents with Misspecified Models

Ignacio Esponda and Demian Pouzo. “Berk-Nash Equilibrium: A Frame- work for Modeling Agents with Misspecified Models”. In:Econometrica84.3 (2023/10/13/ 2016), pp. 1093–1130

2023
[136]

Estimatesoferrorratesforcodesonburst-noisechannels

EdgarO.Elliott.“Estimatesoferrorratesforcodesonburst-noisechannels”. In: The Bell System Technical Journal42.5 (1963).doi: 10.1002/j.1538- 7305.1963.tb00955.x

1963
[137]

Towards Addressing Spatio-Temporal Aspects in Security Games

Fei Fang. “Towards Addressing Spatio-Temporal Aspects in Security Games”. PhD thesis. University of Southern California, 2016

2016
[138]

Cy- bORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents

Harry Emerson, Liz Bates, Chris Hicks, and Vasilios Mavroudis. Cy- bORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents. 2024. arXiv:2410.16324 [cs.CR]

2024 arXiv
[139]

Ronald A. Fisher. The Design of Experiments. Edinburgh: Oliver and Boyd, 1935. References 303

1935
[140]

Au- tonomous Network Defence using Reinforcement Learning

Myles Foley, Chris Hicks, Kate Highnam, and Vasilios Mavroudis. “Au- tonomous Network Defence using Reinforcement Learning”. In:Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security. ASIA CCS ’22. Nagasaki, Japan: Association for Computing Ma- ...

2022 doi
[141]

Equilibrium in misspecified Markov decision processes

Ignacio Esponda and Demian Pouzo. “Equilibrium in misspecified Markov decision processes”. In: Theoretical Economics 16.2 (2021), pp. 717–757. issn: 1933-6837.doi: 10.3982/te3843

2021 doi
[142]

ADEPTS: Adaptive In- trusion Response Using Attack Graphs in an E-Commerce Environment

Bingrui Foo, Yu-Chun Mao, and Eugene Spafford. “ADEPTS: Adaptive In- trusion Response Using Attack Graphs in an E-Commerce Environment”. In: Proceedings of the 2005 International Conference on Dependable Sys- tems and Networks. DSN ’05. USA, 2005, pp. 508–517.isbn: 0769522823....

2005 doi
[143]

Impossibility of Distributed Consensus with One Faulty Process

Michael J. Fischer, Nancy A. Lynch, and Michael S. Paterson. “Impossibility of Distributed Consensus with One Faulty Process”. In:J. ACM32.2 (Apr. 1985), pp. 374–382.issn: 0004-5411.doi: 10.1145/3149.214121

1985
[144]

Cyber situational awareness - A sys- tematic review of the literature

Ulrik Franke and Joel Brynielsson. “Cyber situational awareness - A sys- tematic review of the literature”. In:Comput. Secur.46 (2014)

2014
[145]

Skynet: a Cyber-Aware Intrusion Tolerant Overseer

Tadeu Freitas, João Soares, Manuel E. Correia, and Rolando Martins. “Skynet: a Cyber-Aware Intrusion Tolerant Overseer”. In:2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S). 2023, pp. 111–116.doi: 10.1109/DSN-...

2023
[146]

Inroads into Autonomous Network Defence using Explained Reinforcement Learning

MylesFoley,MiaWang,ZoeM,ChrisHicks,andVasiliosMavroudis. Inroads into Autonomous Network Defence using Explained Reinforcement Learning
[147]

Levine.The theory of learning in games

Drew Fudenberg and David K. Levine.The theory of learning in games. MIT Press, Cambridge, MA., 1998

1998
[148]

MIT Press, 1991

Drew Fudenberg and Jean Tirole.Game Theory. MIT Press, 1991

1991
[149]

Availabilityin Globally Distributed Storage Systems

Daniel Ford, Francois Labelle, Florentina Popovici, Murray Stokely, Van- AnhTruong,LuizBarroso,CarrieGrimes,andSeanQuinlan.“Availabilityin Globally Distributed Storage Systems”. In:Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation. 2010

2010
[150]

Towards Autonomous Defense of SDN Networks Using MuZero Based Intelligent Agents

Jon Gabirondo-López, Jon Egaña, Jose Miguel-Alonso, and Raul Orduna Urrutia. “Towards Autonomous Defense of SDN Networks Using MuZero Based Intelligent Agents”. In:IEEE Access 9 (2021), pp. 107184–107199. doi: 10.1109/ACCESS.2021.3100706. 304 References

2021
[151]

Casino Rationale: Countering Attacker Deception in Zero-Sum Stackelberg Security Games of Bounded Rationality

RyanGabrys,MarkBilinski,JustinMauger,DanielSilva,andSunnyFugate. “Casino Rationale: Countering Attacker Deception in Zero-Sum Stackelberg Security Games of Bounded Rationality”. In:Decision and Game Theory for Security. Ed. by Fei Fang, Haifeng Xu, and Yezekael Hayel. Cham: Sp...

2023
[152]

Intrusion Detection Systems and Intrusion Preven- tion Systems

Andreas Fuchsberger. “Intrusion Detection Systems and Intrusion Preven- tion Systems”. In:Inf. Secur. Tech. Rep.10.3 (Jan. 2005), pp. 134–139.issn: 1363-4127. doi: 10.1016/j.istr.2005.08.001

2005 doi
[153]

Operational resilience: concepts, design and analysis

Alexander A. Ganin, Emanuele Massaro, Alexander Gutfraind, Nicolas Steen, Jeffrey M. Keisler, Alexander Kott, Rami Mangoubi, and Igor Linkov. “Operational resilience: concepts, design and analysis”. In:Scientific Reports 6.1 (2016), p. 19540.doi: 10.1038/srep19540

2016 doi
[154]

OS diversity for intrusion tolerance: Myth or reality?

Miguel Garcia, Alysson Bessani, Ilir Gashi, Nuno Neves, and Rafael Obel- heiro. “OS diversity for intrusion tolerance: Myth or reality?” In:IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN). 2011

2011
[155]

Fung and Raouf Boutaba

Carol J. Fung and Raouf Boutaba. Intrusion Detection Networks - A Key to Collaborative Security. CRC Press, 2013.isbn: 978-1-466-56412-1

2013
[156]

Scenario-Agnostic Zero-Trust De- fense with Explainable Threshold Policy: A Meta-Learning Approach

Yunfei Ge, Tao Li, and Quanyan Zhu. “Scenario-Agnostic Zero-Trust De- fense with Explainable Threshold Policy: A Meta-Learning Approach”. In: IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)(2023), pp. 1–6

2023
[157]

Next Generation Intrusion Detection: Au- tonomous Reinforcement Learning of Network Attacks

James Cannady Georgia. “Next Generation Intrusion Detection: Au- tonomous Reinforcement Learning of Network Attacks”. In:In Proceedings of the 23rd National Information Systems Secuity Conference. 2000, pp. 1– 12

2000
[158]

Using Cyber Ter- rain in Reinforcement Learning for Penetration Testing

Rohit Gangupantulu, Tyler Cody, Paul Park, Abdul Rahman, Logan Eisen- beiser, Dan Radke, Ryan Clark, and Christopher Redino. “Using Cyber Ter- rain in Reinforcement Learning for Penetration Testing”. In:2022 IEEE In- ternational Conference on Omni-layer Intelligent Systems (CO...

2022
[159]

Sulla determi- nazione empirica delle leggi di probabilità

Valery Ivanovich Glivenko and Francesco Paolo Cantelli. Sulla determi- nazione empirica delle leggi di probabilità. Giorn. Ist. Ital. Attuari (in Ital- ian). 4: 92–99. 1933

1933
[160]

Free Software Foundation

P GNU. Free Software Foundation. Bash (3.2. 48)[Unix shell program]. 2007. References 305

2007
[161]

Lazarus: Automatic Management of Diversity in BFT Systems

Miguel Garcia, Alysson Bessani, and Nuno Neves. “Lazarus: Automatic Management of Diversity in BFT Systems”. In:Proceedings of the 20th In- ternational Middleware Conference. Middleware ’19. Davis, CA, USA: Asso- ciation for Computing Machinery, 2019, pp. 241–254.isbn: 9781450...

2019
[162]

Competition Adds Complexity

Judy Goldsmith and Martin Mundhenk. “Competition Adds Complexity”. In: Advances in Neural Information Processing Systems. Ed. by J. Platt, D. Koller, Y. Singer, and S. Roweis. Vol. 20. Curran Associates, Inc., 2007

2007
[163]

Google Remote Procedure Call

Google. Google Remote Procedure Call. 2022

2022
[164]

Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services

Seth Gilbert and Nancy Lynch. “Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services”. In:SIGACT News 33.2 (June 2002), pp. 51–59.issn: 0163-5700.doi: 10.1145/564585.564601

2002
[165]

A unified framework for simulating Markovian models of highly dependable systems

Ambuj Goyal, Perwez Shahabuddin, Philip Heidelberger, Victor F. Nicola, and Peter W. Glynn. “A unified framework for simulating Markovian models of highly dependable systems”. In:IEEE Transactions on Computers41.1 (1992), pp. 36–51

1992
[166]

Secure or Insure: An Economic Analysis of Security Inter- dependencies and Investment Types

Jens Grossklags. “Secure or Insure: An Economic Analysis of Security Inter- dependencies and Investment Types”. PhD thesis. University of California, Berkeley, 2009

2009
[167]

SBFT: A Scalable and Decentralized Trust Infrastructure

Guy Golan Gueta, Ittai Abraham, Shelly Grossman, Dahlia Malkhi, Benny Pinkas, Michael Reiter, Dragos-Adrian Seredinschi, Orr Tamir, and Alin Tomescu. “SBFT: A Scalable and Decentralized Trust Infrastructure”. In: 2019 49th Annual IEEE/IFIP International Conference on Dependabl...

2019 doi
[168]

Diverse Virtual Replicas for Im- proving Intrusion Tolerance in Cloud

Minzhe Guo and Prabir Bhattacharya. “Diverse Virtual Replicas for Im- proving Intrusion Tolerance in Cloud”. In:Proceedings of the 9th Annual Cyber and Information Security Research Conference. CISR ’14. Oak Ridge, Tennessee, USA, 2014, pp. 41–44.isbn: 9781450328128.doi: 10.11...

2014
[169]

Security of Machine-Learning-Enabled Cyber-Physical Sys- tems: Attack Detection and Mitigation

Sun Guoxin. “Security of Machine-Learning-Enabled Cyber-Physical Sys- tems: Attack Detection and Mitigation”. PhD thesis. The University of Mel- bourne, 2024

2024
[170]

Characterizing intrusion tolerant systems using a state tran- sition model

Katerina Goseva-Popstojanova, Feiyi Wang, Rong Wang, Fengmin Gong, Kalyanaraman Vaidyanathan, Kishor Trivedi, and Balamurugan Muthusamy. “Characterizing intrusion tolerant systems using a state tran- sition model”. In:Proceedings DARPA Information Survivability Conference and ...

2001
[171]

Redundancy Management of Multi- path Routing for Intrusion Tolerance in Heterogeneous Wireless Sensor Net- works

Hamid Al-Hamadi and Ing-Ray Chen. “Redundancy Management of Multi- path Routing for Intrusion Tolerance in Heterogeneous Wireless Sensor Net- works”. In: IEEE Transactions on Network and Service Management10.2 (2013), pp. 189–203.doi: 10.1109/TNSM.2013.043013.120282

2013
[172]

Cyber Security Learning Environment (CSLE)

Kim Hammar. Cyber Security Learning Environment (CSLE). Documenta- tion: https://limmen.dev/csle/, traces: https://github.com/Limmen/c sle/releases/tag/v0.4.0, source code:https://github.com/Limmen/cs le, video demonstration:https://www.youtube.com/watch?v=iE2KPmtIs 2A&. 2023

2023
[173]

Secure or insure? a game-theoretic analysis of information security games

Jens Grossklags, Nicolas Christin, and John Chuang. “Secure or insure? a game-theoretic analysis of information security games”. In:Proceedings of the 17th International Conference on World Wide Web. WWW ’08. Bei- jing, China: Association for Computing Machinery, 2008, pp. 209...

2008
[174]

https://arx iv.org/abs/2402.12499

Kim Hammar, Tao Li, Rolf Stadler, and Quanyan Zhu.Automated Security Response through Online Learning with Adaptive Conjectures. https://arx iv.org/abs/2402.12499. To appear in IEEE Transactions on Information Forensics and Security (TIFS). 2024

2024 arXiv
[175]

A System for Interactive Examination of Learned Security Policies

Kim Hammar and Rolf Stadler. “A System for Interactive Examination of Learned Security Policies”. In:NOMS 2022-2022 IEEE/IFIP Network Op- erations and Management Symposium. 2022, pp. 1–3.doi: 10.1109/NOMS5 4207.2022.9789707

2022
[176]

Protecting the Internet of Vehicles Against Advanced Persistent Threats: A Bayesian Stackelberg Game

Talal Halabi, Omar Abdel Wahab, Ranwa Al Mallah, and Mohammad Zulk- ernine. “Protecting the Internet of Vehicles Against Advanced Persistent Threats: A Bayesian Stackelberg Game”. In:IEEE Transactions on Relia- bility 70.3 (2021), pp. 970–985.doi: 10.1109/TR.2020.3046688. 306 ...

2021
[177]

Digital Twins for Security Automation

Kim Hammar and Rolf Stadler. “Digital Twins for Security Automation”. In: NOMS 2023-2023 IEEE/IFIP Network Operations and Management Sym- posium. 2023, pp. 1–6.doi: 10.1109/NOMS56928.2023.10154288

2023
[178]

Finding Effective Security Strategies through Reinforcement Learning and Self-Play

Kim Hammar and Rolf Stadler. “Finding Effective Security Strategies through Reinforcement Learning and Self-Play”. In:International Confer- ence on Network and Service Management (CNSM 2020). Izmir, Turkey, 2020

2020
[179]

https://arxiv.org/pdf /2407.11070, submitted to IEEE Transactions on Dependable and Secure Computing (TDSC)

Kim Hammar, Neil Dhir, and Rolf Stadler.Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search. https://arxiv.org/pdf /2407.11070, submitted to IEEE Transactions on Dependable and Secure Computing (TDSC). 2024

2024 arXiv
[180]

Intrusion Tolerance as a Two-Level Game

Kim Hammar and Rolf Stadler. “Intrusion Tolerance as a Two-Level Game”. In: Decision and Game Theory for Security. Ed. by Arunesh Sinha, Jie Fu, Quanyan Zhu, and Tao Zhang. Cham: Springer Nature Switzerland, 2025, pp. 3–23.isbn: 978-3-031-74835-6. References 307

2025
[181]

Intrusion Tolerance for Networked Systems through Two-Level Feedback Control

Kim Hammar and Rolf Stadler. “Intrusion Tolerance for Networked Systems through Two-Level Feedback Control”. In:2024 54th Annual IEEE/IFIP In- ternational Conference on Dependable Systems and Networks (DSN). 2024, pp. 338–352.doi: 10.1109/DSN58291.2024.00042

2024
[182]

An Online Framework for Adapting Se- curity Policies in Dynamic IT Environments

Kim Hammar and Rolf Stadler. “An Online Framework for Adapting Se- curity Policies in Dynamic IT Environments”. In:2022 18th International Conference on Network and Service Management (CNSM). 2022.doi: 10.2 3919/CNSM55787.2022.9964838

2022
[183]

Learning Near-Optimal Intrusion Re- sponses Against Dynamic Attackers

Kim Hammar and Rolf Stadler. “Learning Near-Optimal Intrusion Re- sponses Against Dynamic Attackers”. In:IEEE Transactions on Network and Service Management21.1 (2024), pp. 1158–1177.doi: 10.1109/TNSM.2 023.3293413

2024 doi
[184]

Learning Security Strategies through Game Play and Optimal Stopping

Kim Hammar and Rolf Stadler. “Learning Security Strategies through Game Play and Optimal Stopping”. In:Proceedings of the ML4Cyber workshop, ICML 2022, Baltimore, USA, July 17-23, 2022. PMLR, 2022

2022
[185]

Intrusion Prevention through Optimal Stopping

Kim Hammar and Rolf Stadler. “Intrusion Prevention through Optimal Stopping”. In:IEEE Transactions on Network and Service Management19.3 (2022), pp. 2333–2348.doi: 10.1109/TNSM.2022.3176781

2022
[186]

Security defense against long-term and stealthy cyber- attacks

Kookyoung Han, Jin Hyuk Choi, Yunsik Choi, Gene Moo Lee, and An- drew B. Whinston. “Security defense against long-term and stealthy cyber- attacks”. In:Decision Support Systems(2022), p. 113912.issn: 0167-9236. doi: https://doi.org/10.1016/j.dss.2022.113912

2022
[187]

Reinforcement Learning for Autonomous Defence in Software- Defined Networking

Yi Han, Benjamin I. P. Rubinstein, Tamas Abraham, Tansu Alpcan, Olivier De Vel, Sarah Erfani, David Hubczenko, Christopher Leckie, and Paul Montague. “Reinforcement Learning for Autonomous Defence in Software- Defined Networking”. In: Decision and Game Theory for Security. 201...

2018
[188]

Learning Intrusion Prevention Policies through Optimal Stopping

Kim Hammar and Rolf Stadler. “Learning Intrusion Prevention Policies through Optimal Stopping”. In:International Conference on Network and Service Management (CNSM 2021). https://arxiv.org/pdf/2106.07160 .pdf. Izmir, Turkey, 2021

2021 arXiv
[189]

Games with Incomplete Information Played by

John Harsanyi. “Games with Incomplete Information Played by "Bayesian" Players, I-III Part I. The Basic Model”. In:Management Science14.3 (1967), pp. 159–182

1967
[190]

Neuroevolution for Autonomous Cyber Defense

Kade Heckel. “Neuroevolution for Autonomous Cyber Defense”. In:Proceed- ings of the Companion Conference on Genetic and Evolutionary Computa- tion. GECCO ’23 Companion. Lisbon, Portugal: Association for Computing Machinery, 2023, pp. 651–654.isbn: 9798400701207.doi: 10.1145/35...

2023 doi
[191]

Scalable Learning of Intrusion Response through Recursive Decomposition

Kim Hammar and Rolf Stadler. “Scalable Learning of Intrusion Response through Recursive Decomposition”. In:Decision and Game Theory for Se- curity. Ed. by Jie Fu, Tomas Kroupa, and Yezekael Hayel. Cham: Springer Nature Switzerland, 2023, pp. 172–192.isbn: 978-3-031-50670-3

2023
[192]

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Johannes Heinrich and David Silver. “Deep Reinforcement Learning from Self-Play in Imperfect-Information Games”. In: CoRR abs/1603.01121 (2016). arXiv: 1603.01121

2016 arXiv
[193]

Hellerstein, Yixin Diao, Sujay Parekh, and Dawn M

Joseph L. Hellerstein, Yixin Diao, Sujay Parekh, and Dawn M. Tilbury. Feedback Control of Computing Systems. USA: Wiley & Sons, 2004.isbn: 047126637X

2004
[194]

Dynamic Prac- tical Byzantine Fault Tolerance

Xu Hao, Long Yu, Liu Zhiqiang, Liu Zhen, and Gu Dawu. “Dynamic Prac- tical Byzantine Fault Tolerance”. In:2018 IEEE Conference on Communi- cations and Network Security (CNS). 2018, pp. 1–8.doi: 10.1109/CNS.201 8.8433150

2018 doi
[195]

A Generalized Framework for Self-Play Training

Daniel Hernandez, Kevin Denamganaï, Yuan Gao, Peter York, Sam Devlin, Spyridon Samothrakis, and James A. Walker. “A Generalized Framework for Self-Play Training”. In:2019 IEEE Conference on Games (CoG). 2019, pp. 1–8

2019
[196]

Nash equilibria in partial- information games on Markov chains

João P. Hespanha and Maria Prandini. “Nash equilibria in partial- information games on Markov chains”. In: Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228). Vol. 3. 2001. doi: 10.1109/CDC.2001.980562

2001
[197]

Reinforcement Learning from Self-Play in Imperfect- Information Games

Johannes Heinrich. “Reinforcement Learning from Self-Play in Imperfect- Information Games”. PhD thesis. University College London, 2017

2017
[198]

Scalable Algorithms for Solving Stochastic Games with Limited Partial Observability

Karel Horák. “Scalable Algorithms for Solving Stochastic Games with Limited Partial Observability”. PhD thesis. Czech Technical University in Prague, 2019

2019
[199]

Solving Partially Observable Stochas- tic Games with Public Observations

Karel Horák and Branislav Bošanský. “Solving Partially Observable Stochas- tic Games with Public Observations”. In:Proceedings of the AAAI Confer- ence on Artificial Intelligence33.01 (July 2019), pp. 2029–2036.doi: 10.16 09/aaai.v33i01.33012029

2019
[200]

Network emulation with NetEm

Stephen Hemminger. “Network emulation with NetEm”. In: Linux Conf (2005)

2005
[201]

Heuristic Search Value Iteration for One-Sided Partially Observable Stochastic Games

Karel Horák, Branislav Bošanský, and Michal Pěchouček. “Heuristic Search Value Iteration for One-Sided Partially Observable Stochastic Games”. In: Proceedings of the AAAI Conference on Artificial Intelligence(Feb. 2017)

2017
[202]

Optimizing honeypot strategies against dynamic lateral movement using partially observable stochastic games

Karel Horák, Branislav Bošanský, Petr Tomášek, Christopher Kiekintveld, and Charles Kamhoua. “Optimizing honeypot strategies against dynamic lateral movement using partially observable stochastic games”. In:Comput. Secur. 87 (2019).doi: 10.1016/j.cose.2019.101579. References 309

2019
[203]

AdaptiveExperimentalDesignforIntrusionData Collection

Kate Highnam, Zach Hanif, Ellie Van Vogt, Sonali Parbhoo, Sergio Maffeis, andNicholasR.Jennings.“AdaptiveExperimentalDesignforIntrusionData Collection”. In: CAMLIS’23: Conference on Applied Machine Learning for Information Security. https://arxiv.org/pdf/2310.13224. 2023

arXiv
[204]

Linear Programming and Markov Decision Chains

Arie Hordijk and Lodewijk C. M. Kallenberg. “Linear Programming and Markov Decision Chains”. In:Management Science 25.4 (1979), pp. 352–

1979
[205]

Optimal Decision Making Approach for Cyber Security Defense Using Evolutionary Game

Hao Hu, Yuling Liu, Chen Chen, Hongqi Zhang, and Yi Liu. “Optimal Decision Making Approach for Cyber Security Defense Using Evolutionary Game”. In: IEEE Transactions on Network and Service Management17.3 (2020), pp. 1683–1700.doi: 10.1109/TNSM.2020.2995713

2020
[206]

Solving zero-sum one-sided partially observable stochastic games

Karel Horák, Branislav Bošanský, Vojtěch Kovařík, and Christopher Kiek- intveld. “Solving zero-sum one-sided partially observable stochastic games”. In: Artificial Intelligence316 (2023), p. 103838.issn: 0004-3702.doi: http s://doi.org/10.1016/j.artint.2022.103838

2023
[207]

Adaptive Cyber Defense Against Multi-Stage Attacks Using Learning-Based POMDP

Zhisheng Hu, Minghui Zhu, and Peng Liu. “Adaptive Cyber Defense Against Multi-Stage Attacks Using Learning-Based POMDP”. In:ACM Trans. Priv. Secur. (2020). issn: 2471-2566.doi: 10.1145/3418897

2020 doi
[208]

AI-Powered System-Scientific Defense for High-Confidence Cyber-Physical Systems: Modeling, Analysis, and Design

Linan Huang. “AI-Powered System-Scientific Defense for High-Confidence Cyber-Physical Systems: Modeling, Analysis, and Design”. PhD thesis. New York University, 2022

2022
[209]

A Sufficient Condition for the Existence of an Optimal Pol- icy with Respect to the Average Cost Criterion in Markovian Decision Pro- cesses

Arie Hordijk. A Sufficient Condition for the Existence of an Optimal Pol- icy with Respect to the Average Cost Criterion in Markovian Decision Pro- cesses. Mathematisch Centrum. BW, afdeling mathematische besliskunde en systeemtheorie. Mathematisch Centrum, 1971

1971
[211]

A dynamic games approach to proactive defensestrategiesagainstAdvancedPersistentThreatsincyber-physicalsys- tems

Linan Huang and Quanyan Zhu. “A dynamic games approach to proactive defensestrategiesagainstAdvancedPersistentThreatsincyber-physicalsys- tems”. In:Computers & Security89 (2020), p. 101660.issn: 0167-4048.doi: https://doi.org/10.1016/j.cose.2019.101660

2020
[212]

Analysis and Computation of Adaptive Defense Strategies Against Advanced Persistent Threats for Cyber-Physical Systems

Linan Huang and Quanyan Zhu. “Analysis and Computation of Adaptive Defense Strategies Against Advanced Persistent Threats for Cyber-Physical Systems”. In:Decision and Game Theory for Security. Ed. by Linda Bush- nell, Radha Poovendran, and Tamer Başar. Cham: Springer Internati...

2018
[213]

Nash Q-Learning for General-Sum Stochastic Games

Junling Hu and Michael P. Wellman. “Nash Q-Learning for General-Sum Stochastic Games”. In:J. Mach. Learn. Res.4.null (Dec. 2003), pp. 1039–

2003
[214]

Self-cleansing systems for intrusion contain- ment

Yih Huang and Arun Sood. “Self-cleansing systems for intrusion contain- ment”. In:2002 Workshop on Self-Healing, Adaptive and Self-Managed Sys- tems. 2002

2002
[215]

Reinforcement Learning forfeedback-enabledcyberresilience

Yunhan Huang, Linan Huang, and Quanyan Zhu. “Reinforcement Learning forfeedback-enabledcyberresilience”.In: Annual Reviews in Control(2022). issn: 1367-5788.doi: https://doi.org/10.1016/j.arcontrol.2022.01 .001

2022 doi
[216]

Factored Markov Game Theory for Secure Interdependent Infrastructure Networks

Linan Huang, Juntao Chen, and Quanyan Zhu. “Factored Markov Game Theory for Secure Interdependent Infrastructure Networks”. In:Game The- ory for Security and Risk Management: From Theory to Practice. Ed. by Stefan Rass and Stefan Schauer. 2018.isbn: 978-3-319-75268-6.doi: 10.1...

2018
[217]

A Probabilistic Approach to Au- tonomic Security Management

Stefano Iannucci and Sherif Abdelwahed. “A Probabilistic Approach to Au- tonomic Security Management”. In:2016 IEEE International Conference on Autonomic Computing (ICAC). 2016, pp. 157–166.doi: 10.1109/ICAC.20 16.12

2016 doi
[218]

A Performance Evaluation of Deep Reinforcement Learning for Model-Based Intrusion Response

Stefano Iannucci, Ovidiu Daniel Barba, Valeria Cardellini, and Ioana Ban- icescu. “A Performance Evaluation of Deep Reinforcement Learning for Model-Based Intrusion Response”. In:2019 IEEE 4th International Work- shops on Foundations and Applications of Self* Systems (FAS*W)(2...

2019
[219]

An In- trusion Response Approach for Elastic Applications Based on Reinforce- ment Learning

Stefano Iannucci, Emiliano Casalicchio, and Matteo Lucantonio. “An In- trusion Response Approach for Elastic Applications Based on Reinforce- ment Learning”. In:IEEE Symposium Series on Computational Intelligence (SSCI) (2021)

2021
[220]

RADAMS: Resilient and adaptive alert and attention management strategy against Informational Denial-of-Service (IDoS) attacks

Linan Huang and Quanyan Zhu. “RADAMS: Resilient and adaptive alert and attention management strategy against Informational Denial-of-Service (IDoS) attacks”. In:Computers & Security121 (2022), p. 102844.issn: 0167-

2022
[221]

Mesures dans les espaces produits

Cassius T. Ionescu Tulcea. “Mesures dans les espaces produits”. In:Lincei– Rend. Sc. fis. mat. e nat.7 (1949), pp. 208–211

1949
[222]

Convergence of Stochastic Iterative Dynamic Programming Algorithms

Tommi Jaakkola, Michael Jordan, and Satinder Singh. “Convergence of Stochastic Iterative Dynamic Programming Algorithms”. In: Advances in Neural Information Processing Systems. Vol. 6. 1994

1994
[223]

Jaromír Janisch, Tomáš Pevný, and Viliam Lisý.NASimEmu: Network At- tack Simulator & Emulator for Training Agents Generalizing to Novel Sce- narios. 2023. arXiv:2305.17246 [cs.CR]. References 311

2023 arXiv
[224]

Savage, 1953, Econometrica, 21, pp

English translation by Leonard J. Savage, 1953, Econometrica, 21, pp. 101-115. 1924

1953
[225]

PenGym: Pentesting Training Framework for Reinforcement Learning Agents

Nguyen Huynh Phuong Thanh, Zhi Chen, Kento Hasegawa, Kazuhide Fukushima, and Razvan Beuran. “PenGym: Pentesting Training Framework for Reinforcement Learning Agents”. In: Jan. 2024, pp. 498–509.doi: 10.5 220/0012367300003648

2024
[226]

Prospect Theory: An Analysis of Decision under Risk

Daniel Kahneman and Amos Tversky. “Prospect Theory: An Analysis of Decision under Risk”. In: Econometrica 47.2 (1979), pp. 263–291. issn: 00129682, 14680262

1979
[227]

Toward a knowledge graph of cybersecurity countermeasures

Peter E Kaloroumakis and Michael J Smith. “Toward a knowledge graph of cybersecurity countermeasures”. In:The MITRE Corporation11 (2021), p. 2021

2021
[228]

High-Performance IntrusionResponsePlanningonMany-CoreArchitectures

Stefano Iannucci, Qian Chen, and Sherif Abdelwahed. “High-Performance IntrusionResponsePlanningonMany-CoreArchitectures”.In: International Conference on Computer Communication and Networks (ICCCN). 2016

2016
[229]

Mathematical Methods of Organizing and Planning Production

Leonid V. Kantorovich. “Mathematical Methods of Organizing and Planning Production”. In:Manage. Sci.6.4 (July 1960), pp. 366–422.issn: 0025-1909. doi: 10.1287/mnsc.6.4.366

1960 doi
[230]

CheapBFT: Resource-Efficient Byzantine Fault Tolerance

Rüdiger Kapitza, Johannes Behl, Christian Cachin, Tobias Distler, Si- mon Kuhnle, Seyed Vahid Mohammadi, Wolfgang Schröder-Preikschat, and Klaus Stengel. “CheapBFT: Resource-Efficient Byzantine Fault Tolerance”. In: Proceedings of the 7th ACM European Conference on Computer Sy...

2012
[231]

The structure of dynamic programing models

Samuel Karlin. “The structure of dynamic programing models”. In:Naval Research Logistics Quarterly2.4 (Dec. 1955), pp. 285–294.doi: 10.1002/n av.3800020408

1955 doi
[232]

Advanced Persistent Threat Detection Method Research Based on Relevant Algorithms to Artificial Immune Sys- tem

Bin Jia, Zhaowen Lin, and Yan Ma. “Advanced Persistent Threat Detection Method Research Based on Relevant Algorithms to Artificial Immune Sys- tem”. In:Trustworthy Computing and Services. Ed. by Lu Yueming, Wu Xu, and Zhang Xi. Berlin, Heidelberg: Springer Berlin Heidelberg, 2...

2015
[233]

isbn: 978-3-662-47401-3
[234]

The Winner’s Curse and Public Informa- tion in Common Value Auctions

John H. Kagel and Dan Levin. “The Winner’s Curse and Public Informa- tion in Common Value Auctions”. In:The American Economic Review76.5 (1986), pp. 894–920.issn: 00028282

1986
[235]

Graphical Models for Game Theory

Michael Kearns, Michael Littman, and Satinder Singh. “Graphical Models for Game Theory”. In:Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI2001). 2001. 312 References

2001
[236]

The vision of autonomic comput- ing

Jeffrey O. Kephart and David M. Chess. “The vision of autonomic comput- ing”. In:Computer 36.1 (2003), pp. 41–50.doi: 10.1109/MC.2003.1160055

2003 arXiv
[237]

Kamhoua, Cristopher D

Charles A. Kamhoua, Cristopher D. Kiekintveld, Fei Fang, and Quanyan Zhu. Game Theory and Machine Learning for Cyber Security. Wiley, 2021. isbn: 9781119723929

2021
[238]

Toward Intrusion Tolerance as a Service: Confidentiality in Partially Cloud-Based BFT Systems

Maher Khan and Amy Babay. “Toward Intrusion Tolerance as a Service: Confidentiality in Partially Cloud-Based BFT Systems”. In:2021 51st An- nual IEEE/IFIP International Conference on Dependable Systems and Net- works (DSN). 2021, pp. 14–25.doi: 10.1109/DSN48987.2021.00019

2021
[239]

AHybridGameTheoryand Reinforcement Learning Approach for Cyber-Physical Systems Security

JosephKhouryandMohamedElBakerNassar.“AHybridGameTheoryand Reinforcement Learning Approach for Cyber-Physical Systems Security”. In: IEEE/IFIP Network Operations and Management Symposium(2020)

2020
[240]

Computing optimal randomized resource alloca- tions for massive security games

Christopher Kiekintveld, Manish Jain, Jason Tsai, James Pita, Fernando Or- dóñez, and Milind Tambe. “Computing optimal randomized resource alloca- tions for massive security games”. In:Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems ...

2009
[241]

Total positivity, absorption probabilities and applications

Samuel Karlin. “Total positivity, absorption probabilities and applications”. In: Transactions of the American Mathematical Society111 (1964)

1964
[242]

On Expected Constant-Round Proto- cols for Byzantine Agreement

Jonathan Katz and Chiu-Yuen Koo. “On Expected Constant-Round Proto- cols for Byzantine Agreement”. In:Journal of Computer and System Sci- ences 75 (Sept. 2006), pp. 445–462.doi: 10.1007/11818175_27

2006 doi
[243]

Efficient Reinforcement Learning in Factored MDPs

Michael Kearns and Daphne Koller. “Efficient Reinforcement Learning in Factored MDPs”. In:Proceedings of the 16th International Joint Conference on Artificial Intelligence. IJCAI’99. Stockholm, Sweden, 1999

1999
[244]

A Multiple-Choice Secretary Algorithm with Applica- tions to Online Auctions

Robert Kleinberg. “A Multiple-Choice Secretary Algorithm with Applica- tions to Online Auctions”. In:Proceedings of the Sixteenth Annual ACM- SIAM Symposium on Discrete Algorithms. SODA ’05. Vancouver, British Columbia, 2005.isbn: 0898715857

2005
[245]

Cyber Autonomy: Automating the Hacker- Self-healing, self- adaptive, automatic cyber defense systems and their impact to the industry, society and national security

Ryan K L Ko. Cyber Autonomy: Automating the Hacker- Self-healing, self- adaptive, automatic cyber defense systems and their impact to the industry, society and national security. 2020. arXiv:2012.04405 [cs.CR]

2020 arXiv
[246]

Comprehensive Review of Ar- tificial Intelligence and Statistical Approaches in Distributed Denial of Ser- vice Attack and Defense Methods

Bashar Ahmed Khalaf, Salama A. Mostafa, Aida Mustapha, Mazin Abed Mohammed, and Wafaa Mustafa Abduallah. “Comprehensive Review of Ar- tificial Intelligence and Statistical Approaches in Distributed Denial of Ser- vice Attack and Defense Methods”. In:IEEE Access7 (2019), pp. 51...

2019
[247]

Kolmogorov

Andrey N. Kolmogorov. Grundbegriffe der Wahrscheinlichkeitsrechnung. Springer, Berlin, 1933

1933
[248]

Zyzzyva: speculative byzantine fault tolerance

Ramakrishna Kotla, Lorenzo Alvisi, Michael Dahlin, Allen Clement, and Ed- mund L. Wong. “Zyzzyva: speculative byzantine fault tolerance.” In:SOSP. Ed. by Thomas C. Bressoud and M. Frans Kaashoek. ACM, 2007, pp. 45–58. isbn: 978-1-59593-591-5

2007
[249]

Alexander Kott and Igor Linkov.Cyber Resilience of Systems and Networks. 1st. Springer Publishing Company, Incorporated, 2018.isbn: 3319774913

2018
[250]

On Autonomous Agents in a Cyber Defence Environment

Mitchell Kiely, David Bowman, Maxwell Standen, and Christopher Moir. On Autonomous Agents in a Cyber Defence Environment . https://arxiv.org/abs/2309.07388?context=cs. 2023

2023 arXiv
[251]

The Se- cureRing protocols for securing group communication

Kim P. Kihlstrom, Louise E. Moser, and Peter M. Melliar-Smith. “The Se- cureRing protocols for securing group communication”. In:Proceedings of the Thirty-First Hawaii International Conference on System Sciences.Vol.3. 1998, 317–326 vol.3.doi: 10.1109/HICSS.1998.656294

1998
[252]

Sur- vivable SCADA Via Intrusion-Tolerant Replication

Jonathan Kirsch, Stuart Goose, Yair Amir, Dong Wei, and Paul Skare. “Sur- vivable SCADA Via Intrusion-Tolerant Replication”. In:IEEE Transactions on Smart Grid5.1 (2014), pp. 60–70.doi: 10.1109/TSG.2013.2269541

2014
[253]

Kafka : a Distributed Messaging System for Log Processing

Jay Kreps. “Kafka : a Distributed Messaging System for Log Processing”. In: 2011

2011
[254]

Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing

Vikram Krishnamurthy. Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing. Cambridge University Press, 2016. doi: 10.1017/CBO9781316471104

2016 doi
[255]

Bandit Based Monte-Carlo Plan- ning

Levente Kocsis and Csaba Szepesvári. “Bandit Based Monte-Carlo Plan- ning.” In: ECML. Ed. by Johannes Fürnkranz, Tobias Scheffer, and Myra Spiliopoulou. Vol. 4212. Lecture Notes in Computer Science. Springer, 2006, pp. 282–293.isbn: 3-540-45375-X. References 313

2006
[256]

Estimation and simulation of nonhomogeneous Poisson processes having multiple periodic- ities

Michael E. Kuhl, James R. Wilson, and Mary A. Johnson. “Estimation and simulation of nonhomogeneous Poisson processes having multiple periodic- ities”. In:Winter Simulation Conference Proceedings, 1995.1995, pp. 374–

1995
[257]

Harold W. Kuhn. Extensive games and the problem of information. Prince- ton, NJ: Princeton University Press, 1953. Chap. 2, pp. 193–216

1953
[258]

On Information and Suffi- ciency

Solomon Kullback and Richard A. Leibler. “On Information and Suffi- ciency”. In: The Annals of Mathematical Statistics22.1 (1951), pp. 79–86. issn: 00034851

1951
[259]

Math- ematical Modeling of Cyber Resilience

Alexander Kott, Michael J. Weisman, and Joachim Vandekerckhove. “Math- ematical Modeling of Cyber Resilience”. In:MILCOM 2022 - 2022 IEEE Military Communications Conference (MILCOM). IEEE, Nov. 2022.doi: 10.1109/milcom55135.2022.10017731

2022
[260]

Analysis of a Markov decision process model for intrusion tolerance

Patrick Kreidl. “Analysis of a Markov decision process model for intrusion tolerance”. In: 2010 International Conference on Dependable Systems and Networks Workshops (DSN-W). 2010, pp. 156–161.doi: 10.1109/DSNW.20 10.5542603

2010 doi
[261]

Feedback control applied to surviv- ability: a host-based autonomic defense system

Patrick Kreidl and Tiffany M. Frazier. “Feedback control applied to surviv- ability: a host-based autonomic defense system”. In:IEEE Transactions on Reliability 53 (2004), pp. 148–166

2004
[262]

Empirical Study of Inter-Arrival Packet Times and Packet Losses

Takayuki Kushida and Yoshitaka Shibata. “Empirical Study of Inter-Arrival Packet Times and Packet Losses”. In:Proceedings of the 22nd International Conference on Distributed Computing Systems. 2002, pp. 233–240. isbn: 0769515886

2002
[263]

A Byzantine resilient fault tolerant computer for nuclear power plant applications

JaynarayanH.Lala. A Byzantine resilient fault tolerant computer for nuclear power plant applications. United States: IEEE Service Center, 1986.isbn: 0-8186-0703-3

1986
[264]

Multiple stopping time POMDPs: Structural results & application in interactive advertising on social media

Vikram Krishnamurthy, Anup Aprem, and Sujay Bhatt. “Multiple stopping time POMDPs: Structural results & application in interactive advertising on social media”. In:Automatica 95 (2018), pp. 385–398.issn: 0005-1098. doi: https://doi.org/10.1016/j.automatica.2018.06.013

2018 doi
[265]

Reconfiguring a State Machine

Leslie Lamport, Dahlia Malkhi, and Lidong Zhou. “Reconfiguring a State Machine”. In:SIGACT News41.1 (Mar. 2010), pp. 63–73.issn: 0163-5700. doi: 10.1145/1753171.1753191

2010
[266]

The Byzantine Gen- erals Problem

Leslie Lamport, Robert Shostak, and Marshall Pease. “The Byzantine Gen- erals Problem”. In: ACM Trans. Program. Lang. Syst. 4.3 (July 1982), pp. 382–401.issn: 0164-0925.doi: 10.1145/357172.357176

1982
[267]

A Network in a Laptop: Rapid Prototyping for Software-Defined Networks

Bob Lantz, Brandon Heller, and Nick McKeown. “A Network in a Laptop: Rapid Prototyping for Software-Defined Networks”. In:Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks. Monterey, Cal- ifornia, 2010.isbn: 9781450304092.doi: 10.1145/1868447.1868466

2010
[268]

Optimal Thresholds for Intrusion Detection Sys- tems

Aron Laszka, Waseem Abbas, S. Shankar Sastry, Yevgeniy Vorobeychik, and Xenofon Koutsoukos. “Optimal Thresholds for Intrusion Detection Sys- tems”. In: Proceedings of the Symposium and Bootcamp on the Science of Security. Pittsburgh, Pennsylvania, 2016.isbn: 9781450342773.doi:...

2016
[269]

Kumar and Pravin Varaiya.Stochastic systems: estimation, identification and adaptive control

Panganamala R. Kumar and Pravin Varaiya.Stochastic systems: estimation, identification and adaptive control. USA: Prentice-Hall, Inc., 1986. isbn: 013846684X. 314 References

1986
[270]

Thomas Kunz, Christian Fisher, James La Novara-Gsell, Christopher Nguyen, and Li Li.A Multiagent CyberBattleSim for RL Cyber Operation Agents. 2023. arXiv:2304.11052 [cs.CR]

2023 arXiv
[271]

Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach

Mehmet Necip Kurt, Oyetunji Ogundijo, Chong Li, and Xiaodong Wang. “Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach”. In: IEEE Transactions on Smart Grid10.5 (2019), pp. 5174–

2019
[272]

Jukka Lempa and Pekka Matomäki.A Dynkin Game with Asymmetric In- formation. 2010

2010
[273]

Adaptive Hierarchi- cal Intrusion Tolerant Model Based on Autonomic Computing

Bingyang Li, Huiqiang Wang, and Guangsheng Feng. “Adaptive Hierarchi- cal Intrusion Tolerant Model Based on Autonomic Computing”. In:2008 International Conference on Security Technology. 2008, pp. 137–141.doi: 10.1109/SecTech.2008.24

2008 doi
[274]

Neural Fictitious Self-Play for Radar Anti-Jamming Dynamic Game with Imperfect Information

Kang Li, Bo Jiu, Wenqiang Pu, Hongwei Liu, and Xiaojun Peng. “Neural Fictitious Self-Play for Radar Anti-Jamming Dynamic Game with Imperfect Information”. In:IEEE Transactions on Aerospace and Electronic Systems (2022), pp. 1–1.doi: 10.1109/TAES.2022.3175186

2022
[275]

Architectural principles for safety-critical real-time applications

Jaynarayan H. Lala and Richard E. Harper. “Architectural principles for safety-critical real-time applications”. In: Proceedings of the IEEE 82.1 (1994), pp. 25–40.doi: 10.1109/5.259424

1994 doi
[276]

Li Li, Raed Fayad, and Adrian Taylor.CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems. 2021. arXiv:2109.03 331 [cs.CR]

2021
[277]

Conjectural On- line Learning with First-order Beliefs in Asymmetric Information Stochastic Games

Tao Li, Kim Hammar, Rolf Stadler, and Quanyan Zhu. “Conjectural On- line Learning with First-order Beliefs in Asymmetric Information Stochastic Games”. In: 2024 63rd IEEE Conference on Decision and Control (CDC). preprint: https://arxiv.org/pdf/2402.18781.pdf. 2024

2024 arXiv
[278]

The Confluence of Networks, Games, and Learning a Game-Theoretic Framework for Multia- gent Decision Making Over Networks

Tao Li, Guanze Peng, Quanyan Zhu, and Tamer Başar. “The Confluence of Networks, Games, and Learning a Game-Theoretic Framework for Multia- gent Decision Making Over Networks”. In:IEEE Control Systems Magazine 42.4 (2022), pp. 35–67.doi: 10.1109/MCS.2022.3171478

2022
[279]

The role of information structures in game-theoretic multi-agent learning

Tao Li, Yuhan Zhao, and Quanyan Zhu. “The role of information structures in game-theoretic multi-agent learning”. In:Annual Reviews in Control53 (2022), pp. 296–314.issn: 1367-5788.doi: 10.1016/j.arcontrol.2022.03 .003

2022 doi
[280]

Cambridge Univer- sity Press, 2020

Tor Lattimore and Csaba Szepesvári.Bandit algorithms. Cambridge Univer- sity Press, 2020

2020
[281]

A Cybersecurity Insurance Model for Power System Reliability Considering Optimal Defense Resource Allocation

Pikkin Lau, Wei Wei, Lingfeng Wang, Zhaoxi Liu, and Chee-Wooi Ten. “A Cybersecurity Insurance Model for Power System Reliability Considering Optimal Defense Resource Allocation”. In: IEEE Transactions on Smart Grid 11.5 (2020), pp. 4403–4414.doi: 10.1109/TSG.2020.2992782

2020
[282]

Structural causal bandits with non- manipulable variables

Sanghack Lee and Elias Bareinboim. “Structural causal bandits with non- manipulable variables”. In:Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. 2019, pp. 4164–4172. References 315

2019
[283]

Reinforcement Learning Based Approach for Flip Attack De- tection

Hanxiao Liu, Yuchao Li, Jonas Mårtensson, Lihua Xie, and Karl Henrik Johansson. “Reinforcement Learning Based Approach for Flip Attack De- tection”. In: 2020 59th IEEE Conference on Decision and Control (CDC)

2020
[284]

Network defense decision-making based on a stochastic game system and a deep re- current Q-network

Xiaohu Liu, Hengwei Zhang, Shuqin Dong, and Yuchen Zhang. “Network defense decision-making based on a stochastic game system and a deep re- current Q-network”. In:Computers & Security111 (2021), p. 102480.issn: 0167-4048. doi: https://doi.org/10.1016/j.cose.2021.102480

2021
[285]

Active Defense Strategy Selection Method Based on Two-Way Signaling Game

Xiaohu Liu, Hengwei Zhang, Yuchen Zhang, Lulu Shao, and Jihong Han. “Active Defense Strategy Selection Method Based on Two-Way Signaling Game”. In:Security and Communication Networks2019 (Nov. 2019), pp. 1–

2019
[286]

Dynamic Hy- pergames for Synthesis of Deceptive Strategies With Temporal Logic Objec- tives

Lening Li, Haoxiang Ma, Abhishek N. Kulkarni, and Jie Fu. “Dynamic Hy- pergames for Synthesis of Deceptive Strategies With Temporal Logic Objec- tives”. In:IEEE Transactions on Automation Science and Engineering20.1 (2023), pp. 334–345.doi: 10.1109/TASE.2022.3150167

2023
[287]

IEEE P2668-Compliant Multi-Layer IoT-DDoS Defense Sys- tem Using Deep Reinforcement Learning

Yucheng Liu, Kim-Fung Tsang, Chung Kit Wu, Yang Wei, Hao Wang, and Hongxu Zhu. “IEEE P2668-Compliant Multi-Layer IoT-DDoS Defense Sys- tem Using Deep Reinforcement Learning”. In:IEEE Transactions on Con- sumer Electronics(2022), pp. 1–1.doi: 10.1109/TCE.2022.3213872

2022
[291]

Counterfactual Regret Minimization in Sequential Security Games

Viliam Lisy, Trevor Davis, and Michael Bowling. “Counterfactual Regret Minimization in Sequential Security Games”. In:Proceedings of the AAAI Conference on Artificial Intelligence30.1 (Feb. 2016).doi: 10.1609/aaai .v30i1.10051

2016 doi
[292]

Markov Games as a Framework for Multi-Agent Re- inforcement Learning

Michael L. Littman. “Markov Games as a Framework for Multi-Agent Re- inforcement Learning”. In:Proceedings of the Eleventh International Con- ference on International Conference on Machine Learning. ICML’94. New Brunswick, NJ, USA: Morgan Kaufmann Publishers Inc., 1994, pp. 15...

1994
[293]

On the complexity of solving Markov decision problems

Michael L. Littman, Thomas L. Dean, and Leslie Pack Kaelbling. “On the complexity of solving Markov decision problems”. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence. UAI’95. Mon- tréal, Qué, Canada: Morgan Kaufmann Publishers Inc., 1995,...

1995
[298]

doi: 10.1155/2019/1362964

2019 doi
[299]

Deep Reinforcement Learning based Smart Mitigation of DDoS Flooding in Software-Defined Networks

Yandong Liu, Mianxiong Dong, Kaoru Ota, Jianhua Li, and Jun Wu. “Deep Reinforcement Learning based Smart Mitigation of DDoS Flooding in Software-Defined Networks”. In:2018 IEEE 23rd International Workshop on Computer Aided Modeling and Design of Communication Links and Net- wo...

2018
[362]

issn: 00251909, 15265501
[383]

doi: 10.1109/WSC.1995.478761

1995
[2020]

doi: 10.1109/CDC42340.2020.9303818

2020
[2023]

arXiv: 2306.09318 [cs.CR]

arXiv
[2971]

doi: 10.1109/TIFS.2019.2911112

2019
[3974]

296 References

doi: 10.1109/LRA.2020.2978451. 296 References

2020
[4048]

doi: https://doi.org/10.1016/j.cose.2022.102844

2022
[5185]

doi: 10.1109/TSG.2018.2878570

2018

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Behavioral and Game- Theoretic Security Investments in Interdependent Systems Modeled by At- tack Graphs

Mustafa Abdallah, Parinaz Naghizadeh, Ashish R. Hota, Timothy Ca- son, Saurabh Bagchi, and Shreyas Sundaram. “Behavioral and Game- Theoretic Security Investments in Interdependent Systems Modeled by At- tack Graphs”. In: IEEE Transactions on Control of Network Systems7.4 (2020), pp. 1585–1596.doi: 10.1109/TCNS.2020.2988007

arXiv 2020

[2] [2]

Synchronous Byzantine Agreement with Expected O(1) Rounds, Ex- pected Communication, and Optimal Resilience

Ittai Abraham, Srinivas Devadas, Danny Dolev, Kartik Nayak, and Ling Ren. “Synchronous Byzantine Agreement with Expected O(1) Rounds, Ex- pected Communication, and Optimal Resilience”. In:Financial Cryptogra- phy and Data Security: 23rd International Conference, FC 2019, Frigate Bay, St. Kitts and Nevis, February 18–22, 2019, Revised Selected Papers. St. ...

work page doi:10.1007/978-3-030-32101-7_20 2019

[3] [3]

Causative Cyberat- tacks on Online Learning-Based Automated Demand Response Systems

Samrat Acharya, Yury Dvorkin, and Ramesh Karri. “Causative Cyberat- tacks on Online Learning-Based Automated Demand Response Systems”. In: IEEE Transactions on Smart Grid12.4 (2021), pp. 3548–3559.doi: 10 .1109/TSG.2021.3067896

arXiv 2021

[4] [4]

ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning

Iman Akbari, Ezzeldin Tahoun, Mohammad A. Salahuddin, Noura Limam, and Raouf Boutaba. “ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning”. In:NOMS IEEE/IFIP Network Operations and Management Symposium. 2020, pp. 1–9.doi: 10.1109/NOMS47738.2020.9 110426

arXiv 2020

[5] [5]

Dynkin games

MarieAlario-Nazaret,Jean-PierreLepeltier,andBernard.Marchal.“Dynkin games”. In: Stochastic Differential Systems. Ed. by M. Kohlmann and N. Christopeit. Berlin, Heidelberg: Springer Berlin Heidelberg, 1982, pp. 23–

1982

[6] [6]

Aliprantis and Kim C

Charalambos D. Aliprantis and Kim C. Border. Infinite Dimensional Analysis: a Hitchhiker’s Guide . Berlin; London: Springer, 2006. isbn: 9783540326960 3540326960.doi: 10.1007/3-540-29587-9

doi:10.1007/3-540-29587-9 2006

[7] [7]

Allen B. Danette. Digital Twins and Living Models at NASA. Digital Twin Summit. 2021

2021

[8] [8]

Noncooperative games for control of networked systems

Tansu Alpcan. “Noncooperative games for control of networked systems”. PhD thesis. University of Illinois at Urbana-Champaign, 2006. 291 292 References

2006