Pith. sign in

REVIEW 3 major objections 4 minor 4 cited by

Chaos Engineering: A Multi-Vocal Literature Review

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multivocal review of 96 sources proposes the first unified definition and five-component taxonomy of chaos engineering.

desk verdict A solid first multivocal review of chaos engineering with a genuinely useful taxonomy; the source-selection biases are real but manageable and the central synthesis holds up. read the letter →

arxiv 2412.01416 v2 pith:J5BWEUPI submitted 2024-12-02 cs.SE

Joshua Owotogbe , Indika Kumara , Willem-Jan Van Den Heuvel , Damian Andrew Tamburri This is my paper · ORCID
classification cs.SE
keywords chaosengineeringmultivocalliteraturereviewfaultinjectionresiliencetestingdistributedsystemsgreytaxonomycloud-native
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that chaos engineering, despite being born in industry and scattered across blogs, white papers, and research papers, can be captured in one coherent synthesis. It claims to be the first multivocal literature review of the field, combining 49 academic and 47 grey literature sources published between January 2016 and April 2024. From those sources it derives a single working definition, a five-component model of a chaos engineering platform, eleven quality requirements, and a taxonomy that classifies tools by execution environment, automation mode, automation strategy, and deployment stage. A sympathetic reader would care because a fragmented field with inconsistent vocabulary makes tool selection, adoption, and comparison harder than it needs to be.

What carries the argument

The machinery is the multivocal literature review process, a systematic search and thematic coding of 96 academic and grey sources, together with its output artifacts: the unified definition, the five-component platform model, and the four-dimensional tool taxonomy. The thematic coding turns source quotations into keywords, keywords into codes, and codes into themes, and the taxonomy's dimensions do the classifying work that lets the authors compare tools and map adoption practices. The fault-injection conceptual model adapted from the literature underlies the Fault Injection Unit, giving the component architecture a grounding beyond the reviewed sources.

What would settle it

Run an independent search covering equivalent terms such as resilience testing and fault injection without requiring the phrase chaos engineering, include more than the first ten web-search result pages, and tally whether the unified definition and five-component taxonomy still account for every recurring theme; any recurring new component would falsify the claim of completeness.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that chaos engineering can be defined and organized as a coherent discipline despite its fragmented literature. The unified definition is: chaos engineering is a resilience testing practice that intentionally injects controlled faults into software systems in production-like or actual production environments to simulate adverse real-world conditions. The review further identifies five core platform components—Experiment Design, Fault Injection, Observability, Post-Experiment Analysis, and Automation and Integration—and eleven quality requirements, then uses these to compare ten widely used tools. The paper also maps the technical and socio-technical challenges that drive adoption, documents best and bad practices, and points to open issues in culture, skills, and resource constraints.

Load-bearing premise

The load-bearing premise is that the 96 selected sources, screened with criteria that require coverage of all identified chaos engineering phases and limited to the first ten pages of web-search results, fairly represent the whole chaos engineering landscape; if that source set is skewed, the taxonomy and open issues inherit the skew.

Editorial extensions

If this is right

  • Using the taxonomy, an organization can match its infrastructure and risk tolerance to candidate tools by execution environment, automation mode, automation strategy, and deployment stage, instead of choosing by popularity alone.
  • The unified definition gives researchers and practitioners one vocabulary, so future work can compare studies rather than redefining the term each time.
  • The five-component model provides a checklist of what a chaos engineering platform should include, enabling gap analysis of existing tools and structured design of new ones.
  • The compiled best and bad practices translate directly into an adoption playbook: start small, set clear objectives, automate, track results, involve stakeholders, and document findings.
  • The identified open issues—organizational culture, skill gaps, and resource constraints—define concrete research questions for empirical adoption studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: because inclusion criterion IC6 requires articles to cover all identified phases, the synthesis may systematically miss lightweight or single-phase fault-injection practices that do not describe a full pipeline, so the taxonomy's completeness is likely overstated for real-world partial adoptions.
  • My inference: selecting the ten most-starred open-source repositories may under-represent proprietary or enterprise tools, so the taxonomy should be re-tested on a sample chosen by adoption surveys rather than repository popularity.
  • My inference: the paper's own future direction—applying chaos engineering to AI-enabled systems—would stress the five-component model, since perturbations to data quality, model drift, or inference latency do not map cleanly onto infrastructure fault injection.
  • My inference: the unified definition is a synthesis claim that could be checked by asking whether each of the retained definitions reduces to it without loss; the paper presents the result but not that line-by-line reduction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper reports a multivocal literature review (MLR) of chaos engineering, synthesizing 96 academic and grey literature sources published between January 2016 and April 2024. The authors derive a unified definition of chaos engineering, identify core functionalities and architectural components, propose a taxonomy of chaos engineering platforms, compare ten widely used tools, and enumerate adoption challenges, best practices, evaluation approaches, and open research issues.

Significance. If the synthesis is sound, the paper fills a genuine gap: it is the first MLR that explicitly integrates academic and practitioner perspectives on chaos engineering, and it offers a structured vocabulary for describing chaos engineering platforms. The study has notable methodological strengths: it follows the Garousi et al. MLR guidelines, reports inter-rater reliability (Cohen's Kappa 0.826 and 0.857), provides a replication package with search strings and coding artifacts, and includes a clearly written threats-to-validity section. The proposed taxonomy and unified definition are plausible and practically useful for tool selection and future research.

major comments (3)
  1. [§3.3 (IC6) and §4.1 (Table 9)] Inclusion criterion IC6 requires that articles 'cover all identified phases of the chaos engineering pipeline,' but the phases are not defined until the results section, where Table 9 presents the core activities. This creates a circularity: the corpus is selected for sources that already present a full-lifecycle narrative, so the subsequently derived pipeline activities and the taxonomy built on them may reflect the inclusion criterion rather than the full landscape of chaos engineering practice. The authors acknowledge external-validity threats in §8.3.1, but they do not test how much the synthesis depends on this filter. Please either define the pipeline phases a priori in the review protocol, or perform a sensitivity analysis that re-runs the selection without IC6 and compares the resulting definition and taxonomy.
  2. [§3.2.2] The grey-literature search uses only the string '(Chaos AND Engineering)' in Google Search, while the academic search uses ten synonym-rich strings such as 'chaos test,' 'chaos experiment,' 'chaos toolkit,' and 'chaos mesh.' Since grey literature constitutes about half of the corpus (47 of 96 sources), this asymmetry could bias the practitioner perspective toward sources that literally use the term 'chaos engineering' and away from adjacent terminology such as 'fault injection' or 'resilience testing.' The authors note the risk of omitted alternative-terminology literature in §8.3.1, but they do not assess whether expanding the grey-literature queries would materially change the identified themes. Please expand the grey-literature search or provide a sensitivity analysis on the grey-literature subset.
  3. [§4.1] The definitional synthesis contains an internal numerical inconsistency: the text states that '48 definitions were analyzed, 27 from academic sources and 29 from grey literature,' which sums to 56, and that '34 definitions were retained and 22 were excluded,' which also sums to 56. Because the unified definition is a central contribution, these counts need to be corrected or the discrepancy explicitly explained.
minor comments (4)
  1. [§4 heading] The section title 'Chaos Engineering: Definitions, Functionaries, and Elements' appears to contain a typo; 'Functionaries' should likely be 'Functionalities.'
  2. [§3.4.1 / Figure 2] The selection flow in Figure 2 is hard to follow because the N values are not clearly aligned with the individual steps; please annotate each step with the count after that step.
  3. [§6.2 / Table 13] The tool-selection procedure uses GitHub stars 'as of November 2024,' but the literature search was conducted in April 2024; please clarify the timeline and, if possible, report the star counts or repository snapshot dates to improve reproducibility.
  4. [§4.3.4] There is a capitalization inconsistency in the sentence beginning 'Root Cause Analyzer This module Correlates data...'; 'Correlates' should be lowercase and a period or colon should follow 'Root Cause Analyzer.'

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the review's definition and taxonomy are synthesized from its 96 sources through transparent thematic coding; the only mild concern is the undefined IC6 'identified phases' filter, which is a validity limitation rather than a demonstration that the conclusions reduce to the inputs.

full rationale

This paper is a qualitative multivocal literature review, not a derivation or prediction exercise. The proposed unified definition of chaos engineering is explicitly synthesized from 48 collected definitions using a published synthesis method (Gong and Ribiere, Wacker, Suddaby), and the taxonomy dimensions are stated to be 'developed from the core themes identified through our coding and thematic analysis' (Section 6.1). These are standard thematic-synthesis operations: the outputs are summaries and organizations of the source corpus, not quantities fitted to the corpus and then renamed as predictions. There are no fitted parameters, no equations, and no statistical forcing. The self-citations in the methodology (e.g., [27], [100], [160]) are prior methodological works by the same research group; they are used as guides for conducting grey-literature reviews and thematic coding, not as evidence for the substantive chaos-engineering claims, so they are not load-bearing circularity. The paper does not invoke a uniqueness theorem from its own authors, and it does not smuggle in an ansatz via citation; the taxonomy's dimensions are openly derived from the coding process. The nearest thing to a circularity concern is inclusion criterion IC6 in Table 4, which requires that 'Articles must cover all identified phases of the chaos engineering pipeline.' The phases themselves are not defined until Table 9 in the results section, which creates a risk that the corpus was filtered using a lifecycle model that is also presented as a finding. However, the paper does not state that the phases were identified from the final corpus itself; they may have been taken from the known principles formalized by Basiri et al. [17], which the paper cites as the 2016 starting point. Moreover, even if IC6 biased the corpus toward holistic sources, that would be a source-selection and external-validity limitation, not a demonstration that the unified definition or the five-component taxonomy is equivalent to the inclusion criterion by construction. The authors themselves acknowledge in Section 8.3.1 that some relevant literature using alternative terminology may have been omitted and that the taxonomy has not yet undergone formal external validation. Those are honest validity threats, not circular derivations. Overall, the central claims have independent content grounded in the cited primary sources, so the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities apply to a qualitative literature review. The axioms capture the methodological assumptions that the synthesis depends on.

assumptions (3)
  • domain assumption The selected 96 sources are representative of the chaos engineering literature.
    The entire synthesis is built on this source set; if it is biased, the taxonomy and open issues may misrepresent the field. The paper acknowledges external validity threats in Section 8.3.1.
  • domain assumption Thematic analysis with inter-rater consensus yields reliable codes and themes.
    The qualitative coding process is standard for MLRs, but it depends on the authors' interpretation. Inter-rater Kappa values are reported as strong (0.826 and 0.857) in Section 3.4.
  • domain assumption GitHub stars are a valid proxy for tool relevance.
    Section 6.2 selects the ten most starred repositories as of November 2024, citing Borges et al. [23]. This proxy may miss important tools and is acknowledged as a limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chaos Engineering: A Multi-Vocal Literature Review." pith.science (2026). https://pith.science/paper/J5BWEUPI

@misc{pith2026241201416,
  author       = {Pith},
  title        = {Pith review of: Chaos Engineering: A Multi-Vocal Literature Review},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J5BWEUPI}},
  note         = {Machine review of arXiv:2412.01416}
}
read the original abstract

Organizations, particularly medium and large enterprises, typically rely heavily on complex, distributed systems to deliver critical services and products. However, the growing complexity of these systems poses challenges in ensuring service availability, performance, and reliability. Traditional resilience testing methods often fail to capture the intricate interactions and failure modes of modern systems. Chaos Engineering addresses these challenges by proactively testing how systems in production behave under turbulent conditions, allowing developers to uncover and resolve potential issues before they escalate into outages. Though chaos engineering has received growing attention from researchers and practitioners alike, we observed a lack of reviews that synthesize insights from both academic and grey literature. Hence, we conducted a Multivocal Literature Review (MLR) on chaos engineering to address this research gap by systematically analyzing 96 academic and grey literature sources published between January 2016 and April 2024. We first used the chosen sources to derive a unified definition of chaos engineering and to identify key functionalities, components, and adoption drivers. We also developed a taxonomy for chaos engineering platforms and compared the relevant tools using it. Finally, we analyzed the current state of chaos engineering research and identified several open research issues.

Figures

Figures reproduced from arXiv: 2412.01416 by the authors.

Figure 1
Figure 1. Overview of the MLR Process [80] [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The MLR study selection process. Steps are shown sequentially for both academic and grey literature. After final screening, inter-rater reliability was assessed using Cohen’s Kappa [101], yielding 𝜅 = 0.826 for academic and 𝜅 = 0.857 for grey literature, indicating strong reviewer agreement [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. A representative output from our thematic analysis process, showing how quotations from academic and grey literature were distilled into keywords, abstracted into codes, and grouped under broader themes. ( : grey literature; : academic literature ) 4 Chaos Engineering: Definitions, Functionaries, and Elements (RQ1) This section first reviews how chaos engineering is defined across academic and industry sources. Next… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Key Functionalities Provided by Chaos Engineering. against which system behavior is compared during and after failure injection [68, 124, 157]. Establishing these baselines enables reliable detection of performance degradation and validation of resilience mechanisms [7…
Figure 5
Figure 5. Figure 5: Key Components of Chaos Engineering. 4.3.1 Experiment Design Unit. This architectural component transforms high-level test objectives into system-executable configurations [159, 193]. It offers formal interfaces to define hypotheses, select test targets, and validate r…
Figure 6
Figure 6. Figure 6: Conceptual Schema of Fault Execution Engine[77]. 4.3.3 Observability Unit. This component implements the infrastructure for capturing, processing, and presenting telemetry data during chaos experiments [73, 141]. It is composed of two core subsystems: the Data Collecto…
Figure 7
Figure 7. Figure 7: shows the technical and socio-technical challenges we identified from the reviewed literature. These challenges were identified through our coding and thematic analysis (see Section 3.5) and are documented in detail in our online appendix (see Section 3.6). Unanticipat…
Figure 8
Figure 8. Figure 8: A Taxonomy of Chaos Engineering Platforms and Their Capabilities [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Distribution of Academic and Grey Literature Studies over Years and Venue Type. in chaos engineering. Further breakdowns of publication types, venues, and source classifications are available in our online appendix (Section 3.6). 7.3 Publication trends and key Contribu…
Figure 10
Figure 10. Figure 10: Research Type of Academic (AL) and Grey (GL) Liter￾ature Studies. A* A B C Q1 Q2Workshop Unranked Venue Ranking 0 2 4 6 8 10 12 14 16 Number of Sources [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Ecoscape: Fault Tolerance Benchmark for Adaptive Remediation Strategies in Real-Time Edge ML

    cs.PF 2025-07 conditional novelty 6.0 of 10

    Ecoscape provides a configurable chaos-injection benchmark with a weighted SLO violation score for comparing Kubernetes remediation strategies in edge ML inference.

  2. LO2: Microservice API Anomaly Dataset of Logs and Metrics

    cs.SE 2025-04 conditional novelty 6.0 of 10

    LO2 provides over 657,000 labeled log files and 45 million metric files from a production open-source OAuth2 microservice system, for anomaly detection research.

  3. Resilience Evaluation of Kubernetes in Cloud-Edge Environments via Failure Injection

    cs.DC 2025-07 reject novelty 5.0 of 10

    Under network delay and partition faults, cloud-edge Kubernetes deployments show tighter response-time distributions than cloud-only deployments, while cloud deployments stay more stable under bandwidth throttling and...

  4. Designing a Custom Chaos Engineering Framework for Enhanced System Resilience at Softtech

    cs.SE 2025-06 conditional novelty 3.0 of 10

    The paper proposes a compliance-aware, four-phase chaos engineering framework for Softtech using LitmusChaos and a standard monitoring stack, but provides no implementation or validation.

Reference graph

Works this paper leans on

196 extracted references · 78 canonical work pages · cited by 4 Pith papers

  1. [1]

    Kalam Abdul. 2024. Vault chaos engineering. Retrieved April 28, 2024 from https://www.hashicorp.com/blog/vault-chaos-engineering

    2024

  2. [2]

    Koby Aharon. 2024. Introduction to Chaos Engineering in Serverless Architectures. Retrieved April 28, 2024 from https://www.ranthebuilder.cloud/post/ introduction-to-chaos-engineering-serverless

    2024

  3. [3]

    Waseem Ahmed and Yong Wei Wu. 2013. A survey on reliability in distributed systems. J. Comput. System Sci. 79, 8 (2013), 1243–1255

    2013

  4. [4]

    Peace Aisosa. 2023. Principles of Chaos Engineering. Towards AI. https://towardsai.net/p/l/principles-of-chaos-engineering Last accessed: September 17, 2024

    2023

  5. [5]

    Amro Al-Said Ahmad, Lamis F Al-Qora’n, and Ahmad Zayed. 2024. Exploring the impact of chaos engineering with various user loads on cloud native applications: an exploratory empirical study. Computing 106, 8 (2024), 2389–2425

    2024

  6. [6]

    Mohammed M Alabbadi. 2011. Cloud computing for education and learning: Education and learning as a service (ELaaS). In 14th International Conference on Interactive Collaborative Learning (ICL2011) . IEEE, Piestany, Slovakia, 589–594

    2011

  7. [7]

    Peter Alvaro, Kolton Andrus, Chris Sanden, Casey Rosenthal, Ali Basiri, and Lorin Hochstein. 2016. Automating failure testing research at internet scale. In Proceedings of the Seventh ACM Symposium on Cloud Computing . 17–28

    2016

  8. [8]

    Peter Alvaro and Severine Tymon. 2017. Abstracting the Geniuses Away from Failure Testing: Ordinary users need tools that automate the selection of custom-tailored faults to inject. Queue 15, 5 (2017), 29–53

    2017

Show all 196 references
  1. [9]

    Koushik Annapureddy. 2010. Security challenges in hybrid cloud infrastructures. Aalto University 7, 4 (2010), 1–6

    2010

  2. [10]

    Shan Anwar and Balaji Arunachalam. 2019. Automating Resiliency: How To Remain Calm In The Midst Of Chaos. Retrieved April 28, 2024 from https://medium.com/intuit-engineering/automating-resiliency-how-to-remain-calm-in-the-midst-of-chaos-d0d3929243ca

    2019

  3. [11]

    Merishani Arsecularatne and Ruwan Wickramarachchi. 2023. Adoptability of Chaos Engineering with DevOps to Stimulate the Software Delivery Performance. In 2023 International Research Conference on Smart Computing and Systems Engineering (SCSE) , Vol. 6. IEEE, 1–8

    2023

  4. [12]

    Maricela-Georgiana Avram. 2014. Advantages and challenges of adopting cloud computing. Procedia Tech. 12 (2014), 529–534

    2014

  5. [13]

    Kanchan Awasthi, Krunal Padwekar, and Subhas Chandra Misra. 2025. Digital Twin: A Unified Definition, Issues, Challenges, and Opportunities. Encyclopedia of Information Science and Technology, Sixth Edition (2025), 1–18

    2025

  6. [14]

    Azure Readiness. 2023. Intro to Chaos Engineering and Azure Chaos Studio (Preview). Retrieved April 24, 2024 from https://www.007ffflearning.com/post/ intro-to-chaos-engineering-and-azure-chaos-studio-preview

    2023

  7. [15]

    Mekan Bairyev. 2023. Chaos Engineering: Principles and Best Practices. Retrieved April 28, 2024 from https://maddevs.io/blog/chaos-engineering/

    2023

  8. [16]

    R Balasubramanian and M Aramudhan. 2012. Security issues: public vs private vs hybrid cloud computing. International Journal of Computer Applications 55, 13 (2012), 35–41

    2012

  9. [17]

    Ali Basiri, Niosha Behnam, Ruud De Rooij, Lorin Hochstein, Luke Kosewski, Justin Reynolds, and Casey Rosenthal. 2016. Chaos engineering. IEEE Software 33, 3 (2016), 35–41

    2016

  10. [18]

    Ali Basiri, Lorin Hochstein, Nora Jones, and Haley Tucker. 2019. Automating chaos experiments in production. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) . IEEE, Netflix, Los Gatos, CA, 31–40

    2019

  11. [19]

    Martin Bedoya, Sara Palacios, Daniel Diaz-López, Pantaleone Nespoli, Estefania Laverde, and Sebastián Suárez. 2023. Securing cloud-based military systems with Security Chaos Engineering and Artificial Intelligence. In Proc. of the 18th Int. Conf. on A vailability and Security ...

    2023

  12. [20]

    Bello, Lukumon O

    Sururah A. Bello, Lukumon O. Oyedele, Olugbenga O. Akinade, Muhammad Bilal, Juan Manuel Davila Delgado, Lukman A. Akanbi, Anuoluwapo O. Ajayi, and Hakeem A. Owolabi. 2021. Cloud computing in construction industry: Use cases, benefits and challenges. Automation in Construction ...

    2021

  13. [21]

    Sathyapriya Bhaskar. 2022. Chaos engineering: A step toward reliability. Retrieved April 28, 2024 from https://www.virtusa.com/insights/perspectives/chaos- engineering Published by Virtusa

    2022

  14. [22]

    Sam Bocetta. 2019. How to Use Chaos Engineering to Break Things Productively. Retrieved April 28, 2024 from https://www.infoq.com/articles/chaos- engineering-security-networking/

    2019

  15. [23]

    Hudson Borges, Andre Hora, and Marco Tulio Valente. 2016. Understanding the factors that impact the popularity of GitHub repositories. In 2016 IEEE international conference on software maintenance and evolution (ICSME) . IEEE, Brazil, 334–344

    2016

  16. [24]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101

    2006

  17. [25]

    Ralfs Bremmers. 2021. How Implementing Chaos Engineering Can Benefit Your Project. Retrieved April 28, 2024 from https://www.testdevlab.com/blog/how- implementing-chaos-engineering-can-benefit-your-project

    2021

  18. [26]

    Christoph Buck, Christian Olenberger, André Schweizer, Fabiane Völter, and Torsten Eymann. 2021. Never trust, always verify: A multivocal literature review on current knowledge and research gaps of zero-trust. Computers & Security 110 (2021), 102436

    2021

  19. [27]

    Bert-Jan Butijn, Damian A Tamburri, and Willem-Jan van den Heuvel. 2020. Blockchains: a systematic multivocal literature review. ACM Computing Surveys (CSUR) 53, 3 (2020), 1–37

    2020

  20. [28]

    Tammy Butow. 2018. Planning Your Own Chaos Day. Accessed: April 24, 2025. Available at: https://www.gremlin.com/community/tutorials/planning- your-own-chaos-day

    2018

  21. [29]

    Rajkumar Buyya, Chee Shin Yeo, Srikumar Venugopal, James Broberg, and Ivona Brandic. 2009. Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility.Future Generation Computer Systems 25, 6 (2009), 599–616. https://doi.org...

    2009 doi

  22. [30]

    Oliver Byström. 2022. A comparison between on-premise and cloud environments in terms of security: With an emphasis on Software-as-a-Service & Platform-as-a-Service

    2022

  23. [31]

    Rocco Caferra, John D Hey, Andrea Morone, and Marco Santorsola. 2023. Dynamic inconsistency under ambiguity: An experiment. Journal of Risk and Uncertainty 67, 3 (2023), 215–238

    2023

  24. [32]

    Carlos Camacho, Pablo C Cañizares, Luis Llana, and Alberto Núñez. 2022. Chaos as a Software Product Line—a platform for improving open hybrid-cloud systems resiliency. Software: Practice and Experience 52, 7 (2022), 1581–1614

    2022

  25. [33]

    Matteo Camilli, Antonio Guerriero, Andrea Janes, Barbara Russo, and Stefano Russo. 2022. Microservices integrated performance and reliability testing. In Proceedings of the 3rd ACM/IEEE International Conference on Automation of Software Test . ACM, Bolzano, Italy, 29–39

    2022

  26. [34]

    Paul Castro, Vatche Ishakian, Vinod Muthusamy, and Aleksander Slominski. 2017. Serverless programming (function as a service). In 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) . IEEE, Watson Research Center, 2658–2659

    2017

  27. [35]

    Pushpalika Chatterjee. 2023. Cloud-Native Architecture for High-Performance Payment System. (2023)

    2023

  28. [36]

    Guo Chen, Guotao Bai, Chun Zhang, Juan Wang, Kang Ni, and Zhi Chen. 2022. Big data system testing method based on chaos engineering. In 2022 IEEE 12th International Conference on Electronics Information and Emergency Communication (ICEIEC) . IEEE, Beijing, China, 210–215

    2022

  29. [37]

    Hongyang Chen, Pengfei Chen, Guangba Yu, Xiaoyun Li, Zilong He, and Huxing Zhang. 2024. MicroFI: Non-Intrusive and Prioritized Request-Level Fault Injection for Microservice Applications. IEEE Transactions on Dependable and Secure Computing 21, 1 (2024), 1–18

    2024

  30. [38]

    Jessica Chen, Robert M Hierons, and Hasan Ural. 2006. Overcoming observability problems in distributed test architectures. University of Windsor 98, 5 (2006), 177–182

    2006

  31. [39]

    Lianping Chen, Muhammad Ali Babar, and He Zhang. 2010. Towards an evidence-based understanding of electronic data sources. In 14th International Conference on Evaluation and Assessment in Software Engineering (EASE) . BCS Learning & Development, Limerick, Ireland, 1–4

    2010

  32. [40]

    Carlos Colman-Meixner, Chris Develder, Massimo Tornatore, and Biswanath Mukherjee. 2016. A survey on resiliency techniques in cloud computing infrastructures and applications. IEEE Communications Surveys & Tutorials 18, 3 (2016), 2244–2281

    2016

  33. [41]

    Adrian Colyer. 2019. Automating chaos experiments in production. Retrieved April 28, 2024 from https://blog.acolyer.org/2019/07/05/automating-chaos- experiments-in-production/

    2019

  34. [42]

    Domenico Cotroneo, Luigi De Simone, and Roberto Natella. 2022. Thorfi: a novel approach for network fault injection as a service. Journal of Network and Computer Applications 201 (2022), 103334

    2022

  35. [43]

    L Minh Dang, Md Jalil Piran, Dongil Han, Kyungbok Min, and Hyeonjoon Moon. 2019. A survey on internet of things and cloud computing for healthcare. Electronics 8, 7 (2019), 768

    2019

  36. [44]

    Czesław Danilowicz and Ngoc Thanh Nguyen. 2003. Consensus methods for solving inconsistency of replicated data in distributed systems. Distributed and Parallel Databases 14 (2003), 53–69

    2003

  37. [45]

    Gert-Jan de Vreede, Pedro Antunes, Julita Vassileva, Marco Aurélio Gerosa, and Kewen Wu. 2016. Collaboration technology in teams and publishers: Introduction to the special issue. Information Systems Frontiers 18 (2016), 1–6

    2016

  38. [46]

    Panagiotis Dedousis, George Stergiopoulos, George Arampatzis, and Dimitris Gritzalis. 2023. Enhancing Operational Resilience of Critical Infrastructure Processes Through Chaos Engineering. IEEE Access 11 (2023), 106172–106189

    2023

  39. [47]

    Merkebu Zenebe Degefa, Iver Bakken Sperstad, and Hanne Sæle. 2021. Comprehensive classifications and characterizations of power system flexibility resources. Electric Power Systems Research 194 (2021), 107022

    2021

  40. [48]

    Josu Diaz-De-Arcaya, Juan López-De-Armentia, Raúl Miñón, Iker Lasa Ojanguren, and Ana I Torre-Bastida. 2024. Large Language Model Operations (LLMOps): Definition, Challenges, and Lifecycle Management. In 2024 9th Int. Conf. on Smart and Sustainable Technologies . IEEE, 1–4

    2024

  41. [49]

    Ashwin Dua. 2024. What Is Chaos Engineering and What Are Its Benefits? Retrieved April 28, 2024 from https://www.turing.com/blog/chaos-engineering- and-its-benefits

    2024

  42. [50]

    Sindhuja Durai. 2022. Chaos Testing an Application on AWS. Retrieved April 28, 2024 from https://developer.gs.com/blog/posts/chaos-testing-an- application-on-aws

    2022

  43. [51]

    Pranay Dutta and Prashant Dutta. 2019. Comparative study of cloud services offered by Amazon, Microsoft & Google. International Journal of Trend in Scientific Research and Development 3, 3 (2019), 981–985

    2019

  44. [52]

    Kleinner Farias, Alessandro Garcia, and Carlos Lucena. 2012. Evaluating the impact of aspects on inconsistency detection effort: a controlled experiment. In Model Driven Engineering Languages and Systems: 15th Int. Conf., MODELS 2012. Proc. 15 . Springer, Austria, 219–234

    2012

  45. [53]

    Amanda Fawcett. 2020. Chaos engineering 101: Principles, process, and examples. Retrieved April 28, 2024 from https://www.educative.io/blog/chaos- engineering-process-principles

    2020

  46. [54]

    Colin Fidge. 1996. Fundamentals of distributed system observation. IEEE Software 13, 6 (1996), 77–83. Manuscript submitted to ACM 32 Owotogbe et al

    1996

  47. [55]

    Mattia Fogli, Carlo Giannelli, Filippo Poltronieri, Cesare Stefanelli, and Mauro Tortonesi. 2023. Chaos engineering for resilience assessment of digital twins. IEEE Transactions on Industrial Informatics 20, 2 (2023), 1134–1143

    2023

  48. [56]

    Sebastian Frank, Alireza Hakamian, Lion Wagner, Dominik Kesim, Christoph Zorn, Jóakim von Kistowski, and André van Hoorn. 2021. Interactive elicitation of resilience scenarios based on hazard analysis techniques. In European Conf. on Software Architecture . Springer, Sttugard ...

    2021

  49. [57]

    Sebastian Frank, Alireza Hakamian, Denis Zahariev, and André van Hoorn. 2023. Verifying transient behavior specifications in chaos engineering using metric temporal logic and property specification patterns. In 2023 ACM/SPEC Int. Conf. on Performance Engineering . ACM, Sttugar...

    2023

  50. [58]

    Sebastian Frank, M Alireza Hakamian, Lion Wagner, Dominik Kesim, Jóakim von Kistowski, and André van Hoorn. 2021. Scenario-based Resilience Evaluation and Improvement of Microservice Architectures: An Experience Report. In ECSA (Companion). Scopus, Stuttgart, Germany, 1–10

    2021

  51. [59]

    Saurabh Kumar Garg, Steve Versteeg, and Rajkumar Buyya. 2013. A framework for ranking of cloud computing services. Future Generation Computer Systems 29, 4 (2013), 1012–1023

    2013

  52. [60]

    Vahid Garousi and Michael Felderer. 2017. Experience-based guidelines for effective and efficient data extraction in systematic reviews in software engineering. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering . ACM, Anka...

    2017

  53. [61]

    Vahid Garousi, Michael Felderer, and Tuna Hacaloğlu. 2017. Software test maturity assessment and test process improvement: A multivocal literature review. Information and Software Technology 85 (2017), 16–42

    2017

  54. [62]

    Vahid Garousi, Michael Felderer, and Mika V Mäntylä. 2016. The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In Proc. of the 20th int. conf. on evaluation and assessment in software engineering...

    2016

  55. [63]

    Vahid Garousi, Michael Felderer, and Mika V Mäntylä. 2019. Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and software technology 106 (2019), 101–121

    2019

  56. [64]

    Vahid Garousi and Mika V Mäntylä. 2016. When and what to automate in software testing? A multi-vocal literature review. Information and Software Technology 76 (2016), 92–117

    2016

  57. [65]

    O’Reilly Media, Inc

    Justin Garrison and Kris Nova. 2017. Cloud native infrastructure: Patterns for scalable infrastructure and applications in a dynamic environment . " O’Reilly Media, Inc. ", Tokyo

    2017

  58. [66]

    Bernd Gastermann, Markus Stopper, Anja Kossik, and Branko Katalinic. 2015. Secure implementation of an on-premises cloud storage service for small and medium-sized enterprises. Procedia Engineering 100 (2015), 574–583

    2015

  59. [67]

    Neharika Gianchandani, Dushyant Anoop Sahni, and Ramanpreet Singh. 2022. Is chaos engineering exclusive to Netflix? Well, no, it’s for you too! Retrieved April 28, 2024 from https://www.nagarro.com/en/blog/chaos-engineering-best-practices

    2022

  60. [68]

    Navdeep Singh Gill. 2021. Chaos Engineering Principles, Tools and Best Practices. Retrieved April 28, 2024 from https://www.xenonstack.com/insights/chaos- engineering

    2021

  61. [69]

    Navdeep Singh Gill. 2022. Chaos Engineering For Cloud Native - A Definitive Guide. Retrieved April 28, 2024 from https://www.xenonstack.com/blog/chaos- engineering-for-cloud-native Accessed: 2024-07-28

    2022

  62. [70]

    Cheng Gong and Vincent Ribiere. 2021. Developing a unified definition of digital transformation. Technovation 102 (2021), 102217

    2021

  63. [71]

    Eugene Gorelik. 2013. Cloud computing models. Ph. D. Dissertation. Massachusetts Institute of Technology

    2013

  64. [72]

    Sumit Goyal. 2014. Public vs private vs hybrid vs community-cloud computing: a critical review. International Journal of Computer Network and Information Security 6, 3 (2014), 20–29

    2014

  65. [73]

    Simon Green. 2023. SRE’s Guide to Chaos Engineering: Embrace the Chaos for Resilience. Retrieved April 28, 2024 from https://www.linkedin.com/pulse/sres- guide-chaos-engineering-embrace-resilience-simon-green/

    2023

  66. [74]

    Orabi Habeh, Firas Thekrallah, Said A Salloum, and Khaled Shaalan. 2021. Knowledge sharing challenges and solutions within software development team: a systematic review. Recent Advances in Intelligent Systems and Smart Applications 8, 9 (2021), 121–141

    2021

  67. [75]

    Greg Hawkins. 2020. The Abyss of Ignorable: a Route into Chaos Testing from Starling Bank. Retrieved April 28, 2024 from https://www.infoq.com/ articles/chaos-testing-starling-bank/

    2020

  68. [76]

    Lorin Hochstein and Casey Rosenthal. 2016. Netflix Chaos Monkey Upgraded. Accessed: April 24, 2025. Available at: https://netflixtechblog.com/netflix- chaos-monkey-upgraded-1d679429be5d

    2016

  69. [77]

    Mei-Chen Hsueh, Timothy K Tsai, and Ravishankar K Iyer. 1997. Fault injection techniques and tools. Computer 30, 4 (1997), 75–82

    1997

  70. [78]

    Hiroki Ikeuchi, Jiawen Ge, Yoichi Matsuo, and Keishiro Watanabe. 2020. A framework for automatic failure recovery in ict systems by deep reinforcement learning. In 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS) . IEEE, Tokyo Japan, 1310–1315

    2020

  71. [79]

    Gremlin Inc. 2023. Chaos Engineering: the history, principles, and practice. Retrieved April 28, 2024 from https://www.gremlin.com/community/tutorials/ chaos-engineering-the-history-principles-and-practice

    2023

  72. [80]

    Chadni Islam, Muhammad Ali Babar, and Surya Nepal. 2019. A multi-vocal review of security orchestration. Comput. Surveys 52, 2 (2019), 1–45

    2019

  73. [81]

    Yashpalsinh Jadeja and Kirit Modi. 2012. Cloud computing-concepts, architecture and challenges. In 2012 international conference on computing, electronics and electrical technologies (ICCEET) . IEEE, Nagercoil, India, 877–880

    2012

  74. [82]

    Madhuri Jakkaraju. 2020. 5 steps to getting your app chaos ready. Retrieved April 28, 2024 from https://www.capitalone.com/tech/software-engineering/is- your-app-chaos-engineering-ready/

    2020

  75. [83]

    Hugo Jernberg, Per Runeson, and Emelie Engström. 2020. Getting Started with Chaos Engineering-design of an implementation framework in practice. In Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) . ACM, Lund Swe...

    2020

  76. [84]

    Zhenlan Ji, Pingchuan Ma, and Shuai Wang. 2023. Perfce: Performance debugging on databases with chaos engineering-enhanced causality analysis. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, Bangalore India, 1454–1466

    2023

  77. [85]

    Ruturaj Kadikar. 2023. Building Resilience with Chaos Engineering and Litmus. Retrieved April 28, 2024 from https://www.infracloud.io/blogs/building- resilience-chaos-engineering-litmus/

    2023

  78. [86]

    Ayham Kassab, Etienne Rivière, Guillaume Rosinosky, Ramin Sadre, and Viet Hoang Tran. 2022. C2B2: a Cloud-native Chaos Benchmarking suite for the Hyperledger Fabric Blockchain. In 2022 18th European Dependable Computing Conference (EDCC) . IEEE, Icteam Belgium, 89–96

    2022

  79. [87]

    Nikos Katirtzis. 2022. Chaos Engineering at Expedia Group. Retrieved April 28, 2024 from https://medium.com/expedia-group-tech/chaos-engineering-at- expedia-group-e51a0288ee2

    2022

  80. [88]

    David Kavaler, Asher Trockman, Bogdan Vasilescu, and Vladimir Filkov. 2019. Tool choice matters: JavaScript quality assurance tools and usage outcomes in GitHub projects. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, University of Califo...

    2019

  81. [89]

    Dominik Kesim, André van Hoorn, Sebastian Frank, and Matthias Häussler. 2020. Identifying and prioritizing chaos experiments by using established risk analysis techniques. In 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) . IEEE, Sttugard Ge...

    2020

  82. [90]

    Rohit Khankhoje. 2022. Beyond Coding: A Comprehensive Study of Low-Code, No-Code and Traditional Automation. Journal of Artificial Intelligence & Cloud Computing. SRC/JAICC-160. DOI: doi. org/10.47363/JAICC/2022 (1) 148 (2022), 2–5

    2022 doi

  83. [91]

    Barbara Kitchenham. 2004. Procedures for performing systematic reviews. Keele, UK, Keele University 33, 2004 (2004), 1–26. Manuscript submitted to ACM Chaos Engineering: A Multi-Vocal Literature Review 33

    2004

  84. [92]

    Barbara Kitchenham, Stuart Charters, et al. 2007. Guidelines for performing systematic literature reviews in software engineering

    2007

  85. [93]

    Floriment Klinaku, Martina Rapp, Jörg Henss, and Stephan Rhode. 2022. Beauty and the beast: A case study on performance prototyping of data-intensive cloud applications. In Companion of the 2022 ACM/SPEC International Conference on Performance Engineering . ACM, Stuttgart, Ger...

    2022

  86. [94]

    Charalambos Konstantinou, George Stergiopoulos, Masood Parvania, and Paulo Esteves-Verissimo. 2021. Chaos engineering for enhanced resilience of cyber-physical systems. In 2021 Resilience Week (RWS). IEEE, Saudi Arabia, 1–10

    2021

  87. [95]

    Mikael Koskinen, Tommi Mikkonen, and Pekka Abrahamsson. 2019. Containers in software development: A systematic mapping study. In International conference on product-focused software process improvement . Springer, University of Helsinki, Finland, 176–191

    2019

  88. [96]

    Nikola Kostic. 2024. Chaos Engineering: Definition, Principles, Best Practices. Retrieved April 28, 2024 from https://phoenixnap.com/blog/chaos-engineering

    2024

  89. [97]

    Narayanan Krishnamurthy. 2021. Chaos Is Good! — In Tech. Retrieved April 28, 2024 from https://eng.lifion.com/chaos-is-good-in-tech-2c487fce102f

    2021

  90. [98]

    Andreas Krivas and Rafael Portela. 2020. Comparing Chaos Engineering Tools for Kubernetes Workloads. Retrieved April 28, 2024 from https: //blog.container-solutions.com/comparing-chaos-engineering-tools

    2020

  91. [99]

    Santosh Kumar and RH Goudar. 2012. Cloud computing-research issues, challenges, architecture, platforms and applications: a survey. International Journal of Future Computer and Communication 1, 4 (2012), 356

    2012

  92. [100]

    Indika Kumara, Martín Garriga, Angel Urbano Romeu, Dario Di Nucci, Fabio Palomba, Damian Andrew Tamburri, and Willem-Jan van den Heuvel. 2021. The do’s and don’ts of infrastructure code: A systematic gray literature review. Information and Software Technology 137 (2021), 106593

    2021

  93. [101]

    J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159–174

    1977

  94. [102]

    Nuno Laranjeiro, João Agnelo, and Jorge Bernardino. 2021. A systematic review on software robustness assessment. ACM CSUR 54, 4 (2021), 1–65

    2021

  95. [103]

    Doug Lardo. 2019. Controlled Chaos with Fault Injection Testing. Retrieved April 28, 2024 from https://technology.riotgames.com/news/controlled-chaos- fault-injection-testing

    2019

  96. [104]

    Andy Le. 2022. Chaos Engineering in Accounting team. Retrieved April 28, 2024 from https://engineering.zalopay.vn/how-we-apply-chaos-engineering/

    2022

  97. [105]

    Rakesh Kumar Lenka, Sarthak Padhi, and Kabita Manjari Nayak. 2018. Fault injection techniques-a brief review. In 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN) . IEEE, Greater Noida, India, 832–837

    2018

  98. [106]

    Chunxiao Li, Anand Raghunathan, and Niraj K Jha. 2011. A trusted virtual machine in an untrusted management environment. IEEE Transactions on services computing 5, 4 (2011), 472–483

    2011

  99. [107]

    Yuqian Lu, Xun Xu, and Jenny Xu. 2014. Development of a hybrid manufacturing cloud. Journal of manufacturing systems 33, 4 (2014), 551–566

    2014

  100. [108]

    Fuchen Ma, Yuanliang Chen, Yuanhang Zhou, Jingxuan Sun, Zhuo Su, Jiaguang Jiang, and Huizhong Li. 2023. Phoenix: Detect and locate resilience issues in blockchain via context-sensitive chaos. In Proc. of the 2023 ACM SIGSAC Conf. on Computer and Communications Security . ACM, ...

    2023

  101. [109]

    Sehrish Malik, Moeen Ali Naqvi, and Leon Moonen. 2023. CHESS: A Framework for Evaluation of Self-adaptive Systems based on Chaos Engineering. In 2023 IEEE/ACM 18th Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) . IEEE, Oslo Norway, 195–201

    2023

  102. [110]

    Neelanjan Manna. 2021. Part-2: A Beginner’s Practical Guide to Containerisation and Chaos Engineering with LitmusChaos 2.0. Retrieved April 28, 2024 from https://medium.com/litmus-chaos/a-beginners-practical-guide-to-containerisation-and-chaos-engineering-with-litmuschaos-2-0-...

    2021

  103. [111]

    Christopher S Meiklejohn, Andrea Estrada, Yiwen Song, Heather Miller, and Rohan Padhye. 2021. Service-level fault injection testing. In Proceedings of the ACM Symposium on Cloud Computing . ACM, PA, United States, 388–402

    2021

  104. [112]

    Nihar Ajay Mhatre, Mugdha Shailendra Kulkarni, and Fatima Ali. 2024. The Role of Chaos Engineering in DevOps for Software Robustness. In Applied Intelligence and Computing, Mukesh Saraswat and Rajani Kumari (Eds.). Symbiosis Centre for Information Technology, Symbiosis Interna...

    2024 doi

  105. [113]

    Microsoft. 2024. What is an endpoint? Microsoft. https://www.microsoft.com/en-us/security/business/security-101/what-is-an-endpoint?msockid= 399306b6d8e86cfc287812c0d9446d18 Last accessed: October 18, 2024

    2024

  106. [114]

    Samuel Migirditch, John Asplund, and William Curran. 2022. Chaos engineering: stress-testing algorithms to facilitate resilient strategic military planning. In Proceedings of the Genetic and Evolutionary Computation Conference Companion . 2160–2167

    2022

  107. [115]

    Russ Miles. 2019. Chaos Engineering with Humans in the Loop. Retrieved April 28, 2024 from https://medium.com/chaos-toolkit/chaos-engineering-with- humans-in-the-loop-f4854900b1eb

    2019

  108. [116]

    Russ Miles. 2019. Learning Chaos engineering: discovering and overcoming system weaknesses through experimentation . O’Reilly Media, USA

    2019

  109. [117]

    Subhra Mondal and Prateek Sachan. 2020. Observability in the realm of Chaos Engineering. Retrieved April 28, 2024 from https://medium.com/ @nabtechblog/observability-in-the-realm-of-chaos-engineering-99089226ca51

    2020

  110. [118]

    Sophia Monroe. 2019. Investigate Methodologies for Intentionally Introducing Failures to Improve System Resilience and Fault Tolerance. International Journal of Artificial Intelligence and Machine Learning in Engineering 405 (2019), 405–418

    2019

  111. [119]

    Matthew Moon. 2022. Causing Chaos. Retrieved April 28, 2024 from https://medium.com/justeattakeaway-tech/causing-chaos-3ab9bb5a7235

    2022

  112. [120]

    Mallory Mooney. 2023. Security-focused chaos engineering experiments for the cloud. Retrieved April 28, 2024 from https://www.datadoghq.com/blog/chaos- engineering-for-security/

    2023

  113. [121]

    Andrea Morichetta, Nikolaus Spring, Philipp Raith, and Schahram Dustdar. 2023. Intent-based management for the distributed computing continuum. In 2023 IEEE International Conference on Service-Oriented System Engineering (SOSE) . IEEE, 239–249

    2023

  114. [122]

    Brad Myers, Scott E Hudson, and Randy Pausch. 2000. Past, present, and future of user interface software tools. ACM Transactions on Computer-Human Interaction (TOCHI) 7, 1 (2000), 3–28

    2000

  115. [123]

    Lavan Nallainathan. 2023. Mitigating Downtime and Increasing Reliability: Strategies for Managing Complexity in the Cloud. https://techcommunity. microsoft.com/t5/azure-architecture-blog/mitigating-downtime-and-increasing-reliability-strategies-for/ba-p/3810399 Last accessed: ...

    2023

  116. [124]

    Moeen Ali Naqvi, Sehrish Malik, Merve Astekin, and Leon Moonen. 2022. On evaluating self-adaptive and self-healing systems using chaos engineering. In 2022 IEEE international conference on autonomic computing and self-organizing systems (ACSOS) . IEEE, CA, USA, 1–10

    2022

  117. [125]

    Roberto Natella, Domenico Cotroneo, and Henrique S Madeira. 2016. Assessing dependability with software fault injection: A survey. ACM Computing Surveys (CSUR) 48, 3 (2016), 1–55

    2016

  118. [126]

    National Australia Bank. 2020. Observability in the realm of Chaos Engineering. Retrieved April 23, 2024 from https://medium.com/@nabtechblog/ observability-in-the-realm-of-chaos-engineering-99089226ca51 Medium

    2020

  119. [127]

    Fotis Nikolaidis, Antony Chazapis, Manolis Marazakis, and Angelos Bilas. 2021. Frisbee: automated testing of Cloud-native applications in Kubernetes. arXiv preprint arXiv:2109.10727 abs/2109.10727, 6 (2021), 1–14

    2021 arXiv

  120. [128]

    Fotis Nikolaidis, Antony Chazapis, Manolis Marazakis, and Angelos Bilas. 2023. Event-Driven Chaos Testing for Containerized Applications. InInternational Conference on High Performance Computing . Springer, Rethimno Greece, 144–157

    2023

  121. [129]

    Jesús Gil Nombela. 2023. Chaos Engineering: The Art of introduce Intentional Failures. Retrieved April 28, 2024 from https://www.santanderconsumergs. com/news/https-impulsate-between-tech-chaos-engineering Manuscript submitted to ACM 34 Owotogbe et al

    2023

  122. [130]

    Guruprasad Nookala. 2023. Serverless Data Architecture: Advantages, Drawbacks, and Best Practices. Journal of Computing and Information Technology 3, 1 (2023)

    2023

  123. [131]

    Santeri Paavolainen. 2016. Observed A vailability of Cloud Services. Master’s thesis. University of Helsinki

    2016

  124. [132]

    Palani and J

    R. Palani and J. Gupta. 2023. Adopting Chaos Engineering. LTIMindtree. https://www.ltimindtree.com/wp-content/uploads/2023/09/Adopting-Chaos- Engineering-WP.pdf Retrieved April 28, 2024

    2023

  125. [133]

    Ragupathi Palani and Joydeep Gupta. 2023. Adopting Chaos Engineering. Retrieved April 28, 2024 from https://www.ltimindtree.com/wp-content/uploads/ 2023/09/Adopting-Chaos-Engineering-WP.pdf © LTIMindtree | Privileged and Confidential

    2023

  126. [134]

    Sumin Park, Zelalem Mihret Belay, and Doo-Hwan Bae. 2019. A simulation-based behavior analysis for mci response system of systems. In Proc. of the 2019 IEEE/ACM 7th Intl. Workshop on SESoS and 13th WDES . IEEE, South Korea, 2–9

    2019

  127. [135]

    Brian Parsons. 2021. Using Chaos Engineering to Improve the Resiliency of Transportation Cyber Physical Systems. In INCOSE Americas Sector 14th Annual North-Central and Great Lakes Regional Conference . International Council on Systems Engineering, North-Central and Great Lakes Region

    2021

  128. [136]

    Viral Patel. 2022. What Is Chaos Engineering and Why You Should Break More Things On Purpose. Retrieved April 28, 2024 from https://www.contino.io/ insights/chaos-engineering

    2022

  129. [137]

    Riccardo Patriarca, Andrea Falegnami, Francesco Costantino, and Federico Bilotta. 2018. Resilience engineering for socio-technical risk analysis: Application in neuro-surgery. Reliability Engineering & System Safety 180 (2018), 321–335

    2018

  130. [138]

    Siani Pearson. 2013. Privacy, security and trust in cloud computing . Springer, Bristol, UK

    2013

  131. [139]

    Tony Pierce, Jason Schanck, Alex Groeger, Raed Salih, and Michael R Clark. 2021. Chaos engineering experiments in middleware systems using targeted network degradation and automatic fault injection. In Open Architecture/Open Business Model Net-Centric Systems and Defense Trans...

    2021

  132. [140]

    Filippo Poltronieri, Mauro Tortonesi, and Cesare Stefanelli. 2022. A chaos engineering approach for improving the resiliency of it services configurations. In NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium . IEEE, Ferrara Italy, 1–6

    2022

  133. [141]

    Lucas Eduardo Gulka Pulcinelli, Diego Frazatto Pedroso, and Sarita Mazzini Bruschi. 2023. Conceptual and comparative analysis of application metrics in microservices. In 2023 International Symposium on Computer Architecture and High Performance Computing Workshops . IEEE, Carl...

    2023

  134. [142]

    Akond Rahman, Dibyendu Brinto Bose, Farhat Lamia Barsha, and Rahul Pandita. 2023. Defect Categorization in Compilers: A Multi-vocal Literature Review. Comput. Surveys 56, 4 (2023), 1–42

    2023

  135. [143]

    Arokia Paul Rajan. 2020. A review on serverless architectures-function as a service (FaaS) in cloud computing. TELKOMNIKA (Telecommunication Computing Electronics and Control) 18, 1 (2020), 530–537

    2020

  136. [144]

    Andreas Riege. 2005. Three-dozen knowledge-sharing barriers managers must consider. Journal of knowledge management 9, 3 (2005), 18–35

    2005

  137. [145]

    Bhaskar Prasad Rimal, Eunmi Choi, and Ian Lumb. 2009. A taxonomy and survey of cloud computing systems. In 2009 fifth international joint conference on INC, IMS and IDC . IEEE, Kookmin University in Seoul, South Korea, 44–51

    2009

  138. [146]

    Luis F Rivera, Norha M Villegas, Gabriel Tamura, Hausi A Muller, Ian Watts, Eric Erpenbach, and Xiaotong Shwartz. 2023. Using Digital Twins for Software Change Risk Assessment. In Proc. of the 33rd Annual CASCON: Intl. Conf. on Computer Science and Software Engineering . ACM, ...

    2023

  139. [147]

    Yury Niño Roa. 2022. Chaos Engineering and Observability with Visual Metaphors. Retrieved April 28, 2024 from https://www.infoq.com/articles/chaos- engineering-observability-visual-metaphors/ Reviewed by Ben Linders

    2022

  140. [148]

    Seyed Reza Rouholamini, Meghdad Mirabi, Razieh Farazkish, and Amir Sahafi. 2020. Proactive self-healing techniques for cloud computing: A systematic review. Concurrency and Computation: Practice and Experience 23, 3 (2020), e8246

    2020

  141. [149]

    Johnny Saldaña. 2021. The coding manual for qualitative researchers. an international journal 12, 2 (2021), 169–170

    2021

  142. [150]

    Manish Saraswat and RC Tripathi. 2020. Cloud computing: Comparison and analysis of cloud service providers-AWs, Microsoft and Google. In2020 9th international conference system modeling and advancement in research trends (SMART) . IEEE, UP, India, 281–285

    2020

  143. [151]

    Takieddine Sbiai, Gorik Van Steenberge, Adrian Hornsby, and Milosz Kosmider. 2023. Lessons from Amazon Search’s Chaos Engineering Journey. Retrieved April 28, 2024 from https://community.aws/content/2gBghy9s00swu4qCxKUf1b8fDiP/amazon-search-chaos-engineering-journey?lang=en

    2023

  144. [152]

    Kathan Shah. 2021. Chaos Experimentation, an open-source framework built on top of Envoy Proxy. Retrieved April 28, 2024 from https://eng.lyft.com/chaos- experimentation-an-open-source-framework-built-on-top-of-envoy-proxy-df87519ed681

    2021

  145. [153]

    Prateek Sharma, Lucas Chaufournier, Prashant Shenoy, and YC Tay. 2016. Containers and virtual machines at scale: A comparative study. In Proceedings of the 17th international middleware conference . Springer, Amherst, USA, 1–13

    2016

  146. [154]

    Pareek Chandra Shekhar. 2024. Chaos Testing: A Proactive Framework for System Resilience in Distributed Architectures. International Journal of Science and Research (IJSR) 13, 11 (2024), 851–855. https://doi.org/10.21275/SR241110081650 Fully Refereed, Open Access, Double Blind...

    2024 doi

  147. [155]

    Scheila Farias Silveira. 2023. Fault Tolerance in Microservices: Ensuring Service Resilience and High Availability. https://ubiminds.com/en-us/fault-tolerance/ Last accessed: April 28, 2024

    2023

  148. [156]

    Jesper Simonsson, Long Zhang, Brice Morin, Benoit Baudry, and Martin Monperrus. 2021. Observability and chaos engineering on system calls for containerized applications in docker. Future Generation Computer Systems 122 (2021), 117–129

    2021

  149. [157]

    Gautam Siwach, Adinarayana Haridas, and Nagaraj Chinni. 2022. Evaluating operational readiness using chaos engineering simulations on kubernetes architecture in big data. In 2022 International Conference on Smart Applications, Communications and Networking (SmartNets) . IEEE, ...

    2022

  150. [158]

    James E Smith and Ravi Nair. 2005. The architecture of virtual machines. Computer 38, 5 (2005), 32–38

    2005

  151. [159]

    Jacopo Soldani and Antonio Brogi. 2021. Automated generation of configurable cloud-native chaos testbeds. In Dependable Computing-EDCC 2021 Workshops: DREAMS, DSOGRI, SERENE 2021, Munich, Germany, September 13, 2021, Proceedings 17 . Springer, Pisa, Italy, 101–108

    2021

  152. [160]

    Jacopo Soldani, Damian Andrew Tamburri, and Willem-Jan Van Den Heuvel. 2018. The pains and gains of microservices: A systematic grey literature review. Journal of Systems and Software 146 (2018), 215–232

    2018

  153. [161]

    Shiv Sondhi, Sherif Saad, Kevin Shi, Mohammad Mamun, and Issa Traore. 2021. Chaos engineering for understanding consensus algorithms performance in permissioned blockchains. In Proc. of the 2021 IEEE Intl. Conf. on DASC, PiCom, CBDCom, and CyberSciTech . IEEE, Canada, 51–59

    2021

  154. [162]

    Chi-hoon Song and Young-woo Sohn. 2022. The influence of dependability in cloud computing adoption. The Journal of Supercomputing 78, 10 (2022), 12159–12201

    2022

  155. [163]

    Tiago Boldt Sousa, Hugo Sereno Ferreira, Filipe Figueiredo Correia, and Ademar Aguiar. 2018. Engineering software for the cloud: External monitoring and failure injection. In Proceedings of the 23rd European conference on pattern languages of programs . 1–8

    2018

  156. [164]

    Richard Starr, Animesh Kundu, and Haresh Nandwani. 2022. Automating and Scaling Chaos Engineering using AWS Fault Injection Simulator. Retrieved April 28, 2024 from https://aws.amazon.com/blogs/industries/automating-and-scaling-chaos-engineering-using-aws-fault-injection-simulator/

    2022

  157. [165]

    Lauren Stewart. 2024. Sampling Bias in Research: How to Avoid it. https://atlasti.com/research-hub/sampling-bias Last accessed: April 28, 2024

    2024

  158. [166]

    Roy Suddaby. 2010. Editor’s comments: Construct clarity in theories of management and organization. , 346–357 pages. Manuscript submitted to ACM Chaos Engineering: A Multi-Vocal Literature Review 35

    2010

  159. [167]

    Davide Taibi, Nabil El Ioini, Claus Pahl, and Jan Raphael Schmid Niederkofler. 2020. Patterns for serverless functions (function-as-a-service): A multivocal literature review. In Proceedings of the 10th International Conference on Cloud Computing and Services Science 6, 4 (202...

    2020

  160. [168]

    John E Thomas, Daniel A Eisenberg, Thomas P Seager, and Erik Fisher. 2019. A resilience engineering approach to integrating human and socio-technical system capacities and processes for national infrastructure resilience. Journal of Homeland Security and Emergency Management 1...

    2019

  161. [169]

    Thomas Thüm, Christian Kästner, Fabian Benduhn, Jens Meinicke, Gunter Saake, and Thomas Leich. 2014. FeatureIDE: An extensible framework for feature-oriented software development. Science of Computer Programming 79 (2014), 70–85

    2014

  162. [170]

    Ian Tivey, Brett Aukburg, Paul Jones, Jim Oulton, and Ming Zheng. 2019. Implementing Chaos Engineering for Financial Services. Retrieved April 28, 2024 from https://www.synechron.com/sites/default/files/2022-04/Implementing-chaos-engineering-continuous-compliance-for-financial...

    2019

  163. [171]

    Martin Tomka. 2024. Resilience and Chaos Engineering. Retrieved April 28, 2024 from https://devblogs.microsoft.com/dotnet/resilience-and-chaos- engineering/

    2024

  164. [172]

    Kennedy A Torkura, Muhammad IH Sukmana, Feng Cheng, and Christoph Meinel. 2020. Cloudstrike: Chaos engineering for security and resiliency in cloud infrastructure. IEEE Access 8 (2020), 123044–123060

    2020

  165. [173]

    Kennedy A Torkura, Muhammad IH Sukmana, Feng Cheng, and Christoph Meinel. 2021. Continuous auditing and threat detection in multi-cloud infrastructure. Computers & Security 102 (2021), 102124

    2021

  166. [174]

    Haley Tucker, Lorin Hochstein, Nora Jones, Ali Basiri, and Casey Rosenthal. 2018. The business case for chaos engineering. IEEE Cloud Computing 5, 3 (2018), 45–54

    2018

  167. [175]

    Tuomas Väisänen. 2023. Security review of Cloud Application architectures . Master’s thesis. Aalto University, Espoo, Finland. https://aaltodoc.aalto.fi/items/ 9ff107f8-4ca4-46c3-8f86-ba0723e973c0 Thesis submitted for examination for the degree of Master of Science in Technology

    2023

  168. [176]

    Erwin van Eyk and Alexandru Iosup. 2018. Addressing performance challenges in serverless computing. Proc. ICT. Open 4, 6 (2018), 1–2

    2018

  169. [177]

    Ángel Jesús Varela-Vaca and Antonia M Reina Quintero. 2021. Smart contract languages: A multivocal mapping study. ACM Computing Surveys (CSUR) 54, 1 (2021), 1–38

    2021

  170. [178]

    Roberto Verdecchia, Ivana Malavolta, and Patricia Lago. 2019. Guidelines for architecting android apps: A mixed-method empirical study. In 2019 IEEE International Conference on Software Architecture (ICSA) . IEEE, Amsterdam The Netherlands, 141–150

    2019

  171. [179]

    Bryant Vinisky. 2024. Improving Database Resilience with Observability and Chaos Testing. Retrieved April 28, 2024 from https://newrelic.com/blog/how- to-relic/improving-database-resilience-with-observability-and-chaos-testing

    2024

  172. [180]

    Bich Vu, J Darby Mitchell, Katherine Stowell, Mark Rabe, Orton Huang, Robert Lychev, and Martine Kalke. 2022. Mission resilience experimentation and evaluation testbed. In MILCOM 2022-2022 IEEE Military Communications Conference (MILCOM) . IEEE, United States, 173–179

    2022

  173. [181]

    Jessica Wachtel. 2023. EBay Explores Chaos Fault Testing at the Application Level. Retrieved April 28, 2024 from https://thenewstack.io/ebay-explores- chaos-fault-testing-at-the-application-level/

    2023

  174. [182]

    John G Wacker. 2004. A theory of formal conceptual definitions: developing theory-building measurement instruments. Journal of Operations Management 22, 6 (2004), 629–650

    2004

  175. [183]

    Guiping Wang, Shuyu Chen, and Jun Liu. 2015. An environment-aware anomaly detection framework of cloud platform for improving its dependability. In Proceedings of the Int. Conference on Parallel Processing Techniques and App. Committee of The Congress in Computer Science, Chon...

    2015

  176. [184]

    Sylwia Werbińska-Wojciechowska and Klaudia Winiarska. 2023. Maintenance performance in the age of Industry 4.0: A bibliometric performance analysis and a systematic literature review. Sensors 23, 3 (2023), 1409

    2023

  177. [185]

    Shanika Wickramasinghe. 2023. Chaos Engineering: Benefits, Best Practices, and Challenges. Retrieved April 28, 2024 from https://www.splunk.com/en_ us/blog/learn/chaos-engineering.html

    2023

  178. [186]

    Roel Wieringa, Neil Maiden, Nancy Mead, and Colette Rolland. 2006. Requirements engineering paper classification and evaluation criteria: a proposal and a discussion. Requirements engineering 11 (2006), 102–107

    2006

  179. [187]

    Jill Willard and James Hutson. 2024. Fail Fast, Fail Small: Designing Resilient Systems for the Future of Software Engineering. SSRG International Journal of Recent Engineering Science 11 (2024), 51–58

    2024

  180. [188]

    Benjamin Wilms. 2018. Chaos Engineering – withstanding turbulent conditions in production. Accessed: April 24, 2025. Available at: https://www.codecentric.de/en/knowledge-hub/blog/chaos-engineering

    2018

  181. [189]

    Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, Anders Wesslén, et al. 2012. Experimentation in software engineering . Vol. 236. Springer, Karlskrona, Sweden

    2012

  182. [190]

    Zhaojun Wu. 2021. Securing Online Gaming: Combine Chaos Engineering with DevOps Practices. Retrieved April 28, 2024 from https://www.pingcap. com/blog/securing-online-gaming-combine-chaos-engineering-with-devops-practices/

    2021

  183. [191]

    Guangba Yu, Pengfei Chen, Hongyang Chen, Zijie Guan, Zicheng Huang, Tianjun Jing, Xinmeng Sun, and Xiaoyun Li. 2021. Microrank: End-to-end latency issue localization with extended spectrum analysis in microservice environments. In Proceedings of the Web Conference 2021 . ACM, ...

    2021

  184. [192]

    Jun Zhang, Robert Ferydouni, Aldrin Montana, Daniel Bittman, and Peter Alvaro. 2021. 3milebeach: A tracer with teeth. In Proceedings of the ACM Symposium on Cloud Computing . Scopus, Stockholm Sweden, 458–472

    2021

  185. [193]

    Long Zhang, Brice Morin, Benoit Baudry, and Martin Monperrus. 2021. Maximizing error injection realism for chaos engineering with system calls. IEEE Transactions on Dependable and Secure Computing 19, 4 (2021), 2695–2708

    2021

  186. [194]

    Long Zhang, Brice Morin, Philipp Haller, Benoit Baudry, and Martin Monperrus. 2019. A chaos engineering system for live analysis and falsification of exception-handling in the JVM. IEEE Transactions on Software Engineering 47, 11 (2019), 2534–2548

    2019

  187. [195]

    Long Zhang, Javier Ron, Benoit Baudry, and Martin Monperrus. 2023. Chaos engineering of ethereum blockchain clients. Distributed Ledger Technologies: Research and Practice 2, 3 (2023), 1–18

    2023

  188. [196]

    Minqi Zhou, Rong Zhang, Dadan Zeng, and Weining Qian. 2010. Services in the cloud computing era: A survey. In 2010 4th international universal communication Symposium. IEEE, Beijing, China, 40–46. Manuscript submitted to ACM

    2010

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.