Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

Chaos Engineering in the Wild: Findings from GitHub

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes an empirical picture of chaos engineering tool adoption by mining 971 GitHub repositories, showing that use is concentrated in Toxiproxy and Chaos Mesh, that network and instance-termination faults dominate…

desk verdict Useful descriptive dataset and honest reporting, but the abstract's 'consistent growth since 2016' adoptions claim is not actually supported because Figure 2 plots repository creation dates, not when the tool was adopted. read the letter →

arxiv 2505.13654 v1 pith:2O7KHAGE submitted 2025-05-19 cs.SE

Joshua Owotogbe , Indika Kumara , Dario Di Nucci , Damian Andrew Tamburri , Willem-Jan van den Heuvel This is my paper · ORCID
classification cs.SE
keywords chaosengineeringfaultinjectionGitHubrepositoryminingtooladoptionresiliencetestingcloud-nativesystemsToxiproxyMesh
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show, from public evidence, that chaos engineering has become a recognizable, tool-mediated practice in open source: a small set of tools accounts for most usage, new tool creation peaked around 2018 and then gave way to integration and refinement, and practitioners overwhelmingly inject infrastructure-level faults rather than application-level ones. A sympathetic reader should care because these findings give concrete grounding for deciding which resilience-testing tools, research directions, and fault categories deserve attention. The study also exposes a gap: research and teaching repositories are a minority, while industry-driven development repositories dominate the most active projects.

What carries the argument

The load-bearing mechanism is a mining-and-filtering pipeline: select ten chaos engineering tools by literature presence and GitHub popularity; query GitHub code search for tool-specific integration signatures in configuration, code, and build files; retrieve repository metadata through the GitHub API; manually remove false positives, reducing 5,845 candidate repositories to 971; then classify each repository by purpose and owner origin and count concrete fault-type usages against a four-category taxonomy. This pipeline converts general claims about adoption into countable, auditable evidence.

What would settle it

Re-run the authors' tool queries by creation year and divide by the total number of GitHub repositories created in that year; if the normalized share of CE-using repositories is flat or declining after 2016, the paper's 'consistent growth' conclusion is an artifact of platform growth.

Watch

Extended reading notes

Core claim

The central discovery is an empirical map of chaos engineering tool usage drawn from 971 manually verified GitHub repositories across ten popular tools. The study claims that Toxiproxy and Chaos Mesh are the most widely used tools, with steady growth since 2016; that the ecosystem's release activity peaked in 2018 and then shifted toward refinement and integration, led by Chaos Mesh and LitmusChaos; that development-oriented repositories dominate at 58.0%, with industry ownership at 43.2%; and that tested fault scenarios are concentrated in network disruptions (40.9%) and instance termination (32.7%), with application-level faults at only 3.0%.

Load-bearing premise

The claim that chaos engineering tool adoption has grown since 2016 rests on the assumption that rising counts of CE-using repositories reflect rising adoption, not simply the overall growth of GitHub; the paper does not normalize for the number of repositories created each year.

Editorial extensions

If this is right

  • If the counts are representative, Toxiproxy and Chaos Mesh are de facto standards for network-level and Kubernetes-native chaos, and new tools should target integration or specific gaps rather than general fault injection.
  • If release activity peaked in 2018, the field's center of gravity has moved from creating tools to maintaining and embedding them; release frequency and commit recency, not star counts alone, now separate active from dormant tools.
  • If 81% of high-commit repositories are development-focused, industrial use is the main driver of deep, sustained adoption, while teaching and learning repositories are typically short-lived experiments rather than production usage.
  • If application-level faults are only 3.0% of observed fault scenarios, there is a concrete, measurable niche for tools and research on exceptions, return-value modification, and method-call delay.
  • Adoption-trend conclusions should be read as provisional until normalized by overall GitHub repository growth, because the paper's own validity section states that raw increases may merely track platform growth.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 3.0% application-fault figure likely understates real usage: fault injections in application code are often written as custom scripts or test frameworks, which the tool-name-based search would miss; a signature-based search for exception injection and method-call-delay patterns could test whether the gap is real or an artifact.
  • Because teaching and learning repositories are numerous but low-commit, the pipeline probably captures a large population of tutorial users; an implication the authors do not draw is that educational content is a primary adoption channel, so improving tutorials may matter more than adding features.
  • The observed growth since 2016 could be tested directly by re-running the same queries per year and dividing by the total number of GitHub repositories created that year; if the normalized fraction is flat, the headline should be rewritten as 'chaos tooling grew with GitHub,' not 'chaos tooling grew.'
  • The findings suggest a research opportunity: since industry-owned repositories dominate while academic research repositories are only 5.7%, direct interviews or surveys with maintainers of Toxiproxy- and Chaos Mesh-using repositories could reveal why tools are chosen and why application-level faults are skipped; repository mining alone cannot answer that.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This manuscript reports a mining study of 971 GitHub repositories that incorporate one of ten popular chaos engineering (CE) tools. The authors select the ten tools from a compiled list of 41 using GitHub star counts and the number of importing repositories, query GitHub code search, remove false positives by manual inspection, and classify the remaining repositories by purpose (development, teaching, learning, research, unclassified) and by ownership origin. They then report descriptive statistics for five research questions: adoption trends over time (RQ1), tool release and commit activity (RQ2), purpose and ownership (RQ3), activity and popularity metrics (RQ4), and fault types exercised (RQ5). Headline findings are that Toxiproxy and Chaos Mesh are the most frequently used tools with consistent growth since 2016; that new tool releases peaked in 2018; that 58.0% of repositories serve development purposes; and that network (40.9%) and instance-termination (32.7%) faults dominate the injected scenarios.

Significance. The study is a useful descriptive contribution to the empirical software-engineering literature on tool adoption, following the established template of Sánchez et al. (2022). Its genuine strengths include a publicly available replication package on Zenodo, a manual verification step over roughly a thousand repositories, and an explicit validity section that acknowledges the main threats. The classification results (purpose, ownership, fault-type distribution) are grounded in the published dataset and will interest both practitioners and researchers. However, the headline adoption-growth claim is not supported by the data as analyzed: Figure 2 plots repository creation dates while the text and abstract interpret the curves as adoption, and the growth is not normalized by overall GitHub growth. These are load-bearing problems for the paper's central claim, though they appear fixable within scope by re-dating adoption events and reframing the claims.

major comments (3)
  1. [§4 (RQ1), Fig. 2, Table 2; Abstract] The claim that Toxiproxy and Chaos Mesh show 'consistent growth since 2016' and 'increasing adoption' is not supported by the presented data. Figure 2's caption states that the plot shows 'Yearly repository creation trends,' so the x-axis records when each GitHub repository was created, not when the CE tool was adopted. The paper itself notes in §4 (RQ1) that 'many repositories adopted these tools long after their initial creation,' and Table 2 dates Toxiproxy's first commit to 2014-09-07 and Chaos Mesh's to 2019-09-04, yet Figure 2a includes Toxiproxy repositories created in 2011 and Chaos Mesh entries before 2019. A repository created in 2011 that added Toxiproxy in 2024 is counted at 2011, making recent adoption invisible and early adoption overstated. The abstract's phrase 'reflecting increasing adoption' therefore goes beyond what the analysis can establish; the authors should either re-date the analysis (e.g., to the first commit referencing the tool) or explicitly reword the RQ1 conclusion and abstract to refer to repository creation rather than adoption.
  2. [§4 (RQ1), §6 (Conclusion Validity)] The growth curves are not normalized against the growth of GitHub as a whole. The authors acknowledge this in Section 6, stating that 'an observed increase in the number of projects using CE tools might reflect the general increase in GitHub activity over time, rather than a genuine increase in interest or adoption of CE practices.' This caveat is correctly placed, but it is not a mere validity footnote: the unnormalized trend is the basis of the abstract's headline 'consistent growth since 2016' claim. Moreover, normalizing by repository-creation counts would still not fix the deeper problem that Figure 2 measures creation dates rather than adoption dates (see previous comment). Both corrections are needed before the adoption-growth finding can stand as stated.
  3. [§3 (Tool Selection), §4 (RQ1), Abstract] The finding that 'Toxiproxy and Chaos Mesh are the most frequently used' (§4, Abstract) is partly determined by the tool-selection criterion. Section 3 states that the ten tools were chosen using 'the number of GitHub repositories that import the tools' and their star counts, and the same import-count measure then underlies the prevalence ranking reported in Table 1 and RQ1. The ranking is not fully tautological because the final counts come after false-positive filtering, which could reorder the tools, but the paper currently presents the prevalence result as an independent discovery ('The analysis reveals that...') without discussing its dependence on the selection procedure. The authors should acknowledge this circularity explicitly and frame the prevalence ranking as a property of the chosen sampling frame, or demonstrate that the post-filtering ranking is robust to reasonable variations of the selection rule.
minor comments (7)
  1. [§6 (Internal Validity)] The statement that the authors 'excluded 1,248 false positives, leaving 971 verified repositories in the dataset' is numerically inconsistent with Table 1, which reports 5,845 candidate repositories and 971 retained, implying 4,874 exclusions; the source of the 1,248 figure should be clarified or corrected.
  2. [Table 1] The last column header, '# Repos w/ FP)', appears malformed and its meaning is unclear; the caption should state explicitly that this column reports the final number of repositories retained after false-positive removal.
  3. [Figures 4 and 6] In the manuscript as submitted, the labels of Figures 4 and 6 render as unreadable 'uni000000...' character sequences rather than readable category names (e.g., 'Teaching,' 'Industry'); the encoding or figure generation must be fixed before the paper can be evaluated visually.
  4. [Abstract and §4 (RQ1)] The phrase 'consistent growth since 2016' conflicts with Figure 2a, which shows Toxiproxy repositories from 2011 onward and Chaos Mesh only from 2019; the wording should be aligned with whatever variable the corrected figure actually plots.
  5. [Throughout] The tool name is written inconsistently as both 'Kube-monkey' (text, Table 1) and 'Kube-Monkey' (Figure 2b, Figure 8); one canonical spelling should be used throughout.
  6. [§4 (RQ5)] The RQ5 summary box contains the typo 'Chaos enginnering'; this should be corrected.
  7. [§3 (Manual Identification)] The two-reviewer classification procedure is described, but no inter-rater agreement statistic (e.g., Cohen's kappa) is reported for the 971-repository classification, which would strengthen the internal-validity argument.

Circularity Check

1 steps flagged · score 6.0 of 10

The 'most used tools' headline largely restates the tool-selection criterion; the purpose, ownership, and fault-type analyses remain independent.

  1. self definitional [Section 3 'Chaos Engineering Tool Search' (tool selection) and Section 4 'RQ1' (adoption finding)]
    "We used the number of GitHub repositories that import the tools (Sánchez et al, 2022; Ustunboyacioglu et al, 2024) and their GitHub star counts (Borges et al, 2016) as criteria to identify the top ten tools. ... Among the tools, Toxiproxy stood out as the most widely adopted (243 repositories), followed by Chaos Mesh (225), Chaos Monkey (159)."

    The set of analyzed tools was selected using the number of GitHub repositories that import each tool as a criterion, and the RQ1 finding then reports, for that same set, which tools have the most repositories in the collected dataset. The abstract's claim that 'Toxiproxy and Chaos Mesh are the most frequently used' is therefore a re-description of the popularity measure used to choose the tools, not an independent discovery. The reduction is partial: the final counts come from tool-specific search queries plus manual false-positive filtering, so the ordering could shift (e.g., Chaos Mesh had more raw hits, Toxiproxy more after filtering), and the purpose, ownership, activity, and fault-type analyses are not predetermined by the selection step.

full rationale

The paper is a transparent observational mining study rather than a formal derivation, so most of its chain is self-contained. Its main independent contributions—repository purpose, ownership, activity levels, and fault-injection categories—come from manual classification and code inspection and do not reduce to the tool-selection step. I flag one partial circularity: the abstract's first finding, that Toxiproxy and Chaos Mesh are the most frequently used, restates the popularity criterion (number of importing repositories plus stars) used to select the ten tools in the first place. Because the final counts are obtained through tool-specific search queries and manual false-positive removal, the ranking is not perfectly forced, so this is partial rather than total circularity. Separately, the paper's own Conclusion Validity paragraph notes a non-circular validity threat: repository-creation-date trends were not normalized against overall GitHub growth, and the Figure 2 caption 'Yearly repository creation trends' shows that the 'growth since 2016' language refers to repository creation, not adoption timing. That is a validity limitation, not a circularity. The citations to the authors' prior multivocal review (Owotogbe et al. 2024) are used only to seed the tool list and are not load-bearing as an argument.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The study relies on methodological assumptions rather than numeric free parameters. The main assumptions are that GitHub metrics reflect popularity, that search query matches indicate tool usage, and that the accessible GitHub API sample is representative. No invented entities or fitted parameters appear.

assumptions (3)
  • domain assumption GitHub stars and repository counts are reliable indicators of tool popularity and adoption.
    Used to select the top 10 tools in Section 3, following the approach of earlier mining studies.
  • domain assumption The presence of tool-specific strings, such as 'chaosmonkey' in YAML files, indicates actual usage of the tool in the repository.
    Basis for the repository search queries and for the manual false-positive filtering in Section 3.
  • domain assumption Repositories accessible via the GitHub API, which excludes private repos, files over 384 KB, and results beyond API caps, are representative of all CE-tool-using repositories.
    Search limitations are described in Section 3 and acknowledged in Section 6.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chaos Engineering in the Wild: Findings from GitHub." pith.science (2026). https://pith.science/paper/2O7KHAGE

@misc{pith2026250513654,
  author       = {Pith},
  title        = {Pith review of: Chaos Engineering in the Wild: Findings from GitHub},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2O7KHAGE}},
  note         = {Machine review of arXiv:2505.13654}
}
read the original abstract

Chaos engineering aims to improve the resilience of software systems by intentionally injecting faults to identify and address system weaknesses that cause outages in production environments. Although many tools for chaos engineering exist, their practical adoption is not yet explored. This study examines 971 GitHub repositories that incorporate 10 popular chaos engineering tools to identify patterns and trends in their use. The analysis reveals that Toxiproxy and Chaos Mesh are the most frequently used, showing consistent growth since 2016 and reflecting increasing adoption in cloud-native development. The release of new chaos engineering tools peaked in 2018, followed by a shift toward refinement and integration, with Chaos Mesh and LitmusChaos leading in ongoing development activity. Software development is the most frequent application (58.0%), followed by unclassified purposes (16.2%), teaching (10.3%), learning (9.9%), and research (5.7%). Development-focused repositories tend to have higher activity, particularly for Toxiproxy and Chaos Mesh, highlighting their industrial relevance. Fault injection scenarios mainly address network disruptions (40.9%) and instance termination (32.7%), while application-level faults remain underrepresented (3.0%), highlighting for future exploration.

Figures

Figures reproduced from arXiv: 2505.13654 by the authors.

Figure 1
Figure 1. shows the key steps in our research design, including CE tool search, repository mining, manual identification, and the analysis steps used to address the research questions, which are described in Section 4 We consider five research questions (RQs), partially inspired by previous work (S´anchez et al, 2022; Ustunboyacioglu et al, 2024). Chaos Engineering Tool Search Start Literature Search (Previous Compilation) Gi… view at source ↗
Figure 2
Figure 2. Yearly repository creation trends for the top 10 chaos engineering tools only four repositories in 2022. Examining repository commit histories reveals that many repositories adopted these tools long after their initial creation. This delay between repository setup and tool usage is consistent with observations made in previous studies (S´anchez et al, 2022; Ustunboyacioglu et al, 2024). Overall, this distribution in… view at source ↗
Figure 3
Figure 3. Number of CE tools released per year RQ2: How have the top 10 Chaos Engineering tools evolved over time in terms of GitHub activity? This section analyzes the evolution of CE tools, focusing on the release trends of the most widely adopted tools [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Classification of repositories by type [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Repository type distribution for each tool [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Classification of repositories by origin [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Repository origin distribution for each tool [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Commit distribution across repositories using each selected CE tool. Each bar chart represents a tool and shows the number of associated repositories falling into the specified commit-count ranges [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Commit distribution across repositories by activity type (Development, Research, Teaching, Learning, and Unclassified). Each bar chart represents an activity type and shows the number of associated repositories falling into the specified commit-count ranges. the approa…
Figure 10
Figure 10. Figure 10: Distribution of forks, contributors, stars, and watchers in the selected tools. RQ4: Most repositories used for development features have more than 500 com￾mits. Tools such as Toxiproxy and Chaos Mesh are the most adopted, reflecting their importance in large-scale, p…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Resilience Evaluation of Kubernetes in Cloud-Edge Environments via Failure Injection

    cs.DC 2025-07 reject novelty 5.0 of 10

    Under network delay and partition faults, cloud-edge Kubernetes deployments show tighter response-time distributions than cloud-only deployments, while cloud deployments stay more stable under bandwidth throttling and...

Reference graph

Works this paper leans on

72 extracted references · 64 canonical work pages · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all ...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION ...

  3. [3]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

  4. [4]

    write newline

    " write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION n.separate 't := "" #0 'numnames := t empty not t #-1 #1 subs...

  5. [5]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

  6. [6]

    write newline

    " write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION n.separate 't := "" #0 'numnames := t empty not t #-1 #1 subs...

  7. [7]

    Computing 106(8):2389--2425

    Al-Said Ahmad A, Al-Qora’n LF, Zayed A (2024) Exploring the impact of chaos engineering with various user loads on cloud native applications: an exploratory empirical study. Computing 106(8):2389--2425

    work page 2024

  8. [8]

    Author U (n.d.) Application-level fault-injection. In: Chaos Engineering: Building Confidence in System Behavior through Fault Injection, Manning Publications, chap 8, ://livebook.manning.com/book/chaos-engineering/chapter-8/v-5/5, accessed on 21 January 2025

    work page 2025

Show all 72 references
  1. [9]

    ://maddevs.io/blog/chaos-engineering/

    Bairyev M (2023) Chaos engineering: Principles and best practices. ://maddevs.io/blog/chaos-engineering/

    2023

  2. [10]

    IEEE Software 33(3):35--41

    Basiri A, Behnam N, De Rooij R, Hochstein L, Kosewski L, Reynolds J, Rosenthal C (2016) Chaos engineering. IEEE Software 33(3):35--41

    2016

  3. [11]

    O'Reilly Media, Inc., Califonia

    Basiri A, Blohowiak A, Hochstein L, Jones N, Rosenthal C (2017) Chaos Engineering: Building Confidence in System Behavior Through Experiments. O'Reilly Media, Inc., Califonia

    2017

  4. [12]

    ://github.com/powerfulseal/powerfulseal, accessed: 2025-01-28

    Bloomberg, Contributors P (2025) Powerfulseal: A testing tool for kubernetes clusters. ://github.com/powerfulseal/powerfulseal, accessed: 2025-01-28

    2025

  5. [13]

    In: 2016 IEEE international conference on software maintenance and evolution (ICSME), IEEE, pp 334--344

    Borges H, Hora A, Valente MT (2016) Understanding the factors that impact the popularity of github repositories. In: 2016 IEEE international conference on software maintenance and evolution (ICSME), IEEE, pp 334--344

    2016

  6. [14]

    Software: Practice and Experience 52(7):1581--1614

    Camacho C, Ca \ n izares PC, Llana L, N \'u \ n ez A (2022) Chaos as a software product line—a platform for improving open hybrid-cloud systems resiliency. Software: Practice and Experience 52(7):1581--1614

    2022

  7. [15]

    In: 2022 IEEE 12th International Conference on Electronics Information and Emergency Communication (ICEIEC), IEEE, Beijing, China, pp 210--215

    Chen G, Bai G, Zhang C, Wang J, Ni K, Chen Z (2022) Big data system testing method based on chaos engineering. In: 2022 IEEE 12th International Conference on Electronics Information and Emergency Communication (ICEIEC), IEEE, Beijing, China, pp 210--215

    2022

  8. [16]

    https://www.cncf.io/reports/cncf-annual-report-2018/, accessed: 2025-05-07

    Cloud Native Computing Foundation (2018) CNCF Annual Report 2018 . https://www.cncf.io/reports/cncf-annual-report-2018/, accessed: 2025-05-07

    2018

  9. [17]

    CNCF Website, ://www.cncf.io/projects/chaosmesh/, this grey literature page presents Chaos Mesh, a chaos engineering platform for Kubernetes

    Cloud Native Computing Foundation (2025 a ) Chaos mesh. CNCF Website, ://www.cncf.io/projects/chaosmesh/, this grey literature page presents Chaos Mesh, a chaos engineering platform for Kubernetes. The project was accepted into the CNCF Sandbox on July 14, 2020, and advanced t...

    2025

  10. [18]

    Cloud Native Computing Foundation (2025 b ) Chaosblade. CNCF Website, ://www.cncf.io/projects/chaosblade/, this grey literature page introduces ChaosBlade, a powerful and user-friendly chaos engineering experiment toolkit open-sourced by Alibaba, and accepted into the CNCF San...

    2025

  11. [19]

    CNCF Website, ://www.cncf.io/projects/litmus/, this grey literature page describes Litmus, a Kubernetes-native chaos engineering platform designed for SREs and developers

    Cloud Native Computing Foundation (2025 c ) Litmus. CNCF Website, ://www.cncf.io/projects/litmus/, this grey literature page describes Litmus, a Kubernetes-native chaos engineering platform designed for SREs and developers. It was accepted into the CNCF Sandbox on June 25, 202...

    2025

  12. [20]

    Cloud Native Computing Foundation (2025 d ) Who we are. CNCF Website, ://www.cncf.io/about/who-we-are/, this grey literature page outlines the mission, structure, and community initiatives of the Cloud Native Computing Foundation, which supports projects like Kubernetes, Prome...

    2025

  13. [21]

    ://github.com/chaosblade-io/chaosblade, accessed: 2025-01-28

    Community C (2025 a ) Chaosblade: An easy to use and powerful chaos engineering toolkit. ://github.com/chaosblade-io/chaosblade, accessed: 2025-01-28

    2025

  14. [22]

    ://github.com/chaos-mesh/chaos-mesh, accessed: 2025-01-28

    Community CM (2025 b ) Chaos mesh: A chaos engineering platform for kubernetes. ://github.com/chaos-mesh/chaos-mesh, accessed: 2025-01-28

    2025

  15. [23]

    ://github.com/chaostoolkit/chaostoolkit, accessed: 2025-01-28

    Community CT (2025 c ) Chaos toolkit: Chaos engineering toolkit & orchestration for developers. ://github.com/chaostoolkit/chaostoolkit, accessed: 2025-01-28

    2025

  16. [24]

    ://github.com/litmuschaos/litmus-go, accessed: 2025-01-28

    Community L (2025 d ) Litmus-go: Chaos experiments in go for kubernetes and cloud-native systems. ://github.com/litmuschaos/litmus-go, accessed: 2025-01-28

    2025

  17. [25]

    In: Proceedings of the 13th International Conference on Mining Software Repositories, pp 137--141

    Cosentino V, Luis J, Cabot J (2016) Findings from github: methods, datasets and limitations. In: Proceedings of the 13th International Conference on Mining Software Repositories, pp 137--141

    2016

  18. [26]

    Proceedings of the IEEE 112(1):12--46

    Deng S, Zhao H, Huang B, Zhang C, Chen F, Deng Y, Yin J, Dustdar S, Zomaya AY (2024) Cloud-native computing: A survey from the perspective of services. Proceedings of the IEEE 112(1):12--46

    2024

  19. [27]

    ://www.educative.io/blog/chaos-engineering-process-principles

    Fawcett A (2020) Chaos engineering 101: Principles, process, and examples. ://www.educative.io/blog/chaos-engineering-process-principles

    2020

  20. [28]

    IEEE Transactions on Industrial Informatics 20(2):1134--1143

    Fogli M, Giannelli C, Poltronieri F, Stefanelli C, Tortonesi M (2023) Chaos engineering for resilience assessment of digital twins. IEEE Transactions on Industrial Informatics 20(2):1134--1143

    2023

  21. [29]

    In: Proceedings of the 17th International conference on mining software repositories, pp 431--442

    Gonzalez D, Zimmermann T, Nagappan N (2020) The state of the ml-universe: 10 years of artificial intelligence & machine learning software development on github. In: Proceedings of the 17th International conference on mining software repositories, pp 431--442

    2020

  22. [30]

    ://www.linkedin.com/pulse/sres-guide-chaos-engineering-embrace-resilience-simon-green/

    Green S (2023) Sre’s guide to chaos engineering: Embrace the chaos for resilience. ://www.linkedin.com/pulse/sres-guide-chaos-engineering-embrace-resilience-simon-green/

    2023

  23. [31]

    Gremlin (2023) Comparing chaos engineering tools. Gremlin Community Tutorials, ://www.gremlin.com/community/tutorials/chaos-engineering-tools-comparison, this grey paper explores and compares various Chaos Engineering tools, including LitmusChaos, AWS FIS, Azure Chaos Studio, ...

    2023

  24. [32]

    In: 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS), pp 57--66, doi:10.1109/ICDCS.2016.11

    Heorhiadi V, Rajagopalan S, Jamjoom H, Reiter MK, Sekar V (2016) Gremlin: Systematic resilience testing of microservices. In: 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS), pp 57--66, doi:10.1109/ICDCS.2016.11

    2016 doi

  25. [33]

    IBM (2024) What is chaos engineering? ://www.ibm.com/topics/chaos-engineering

    2024

  26. [34]

    ://www.gremlin.com/community/tutorials/chaos-engineering-the-history-principles-and-practice

    Inc G (2023) Chaos engineering: the history, principles, and practice. ://www.gremlin.com/community/tutorials/chaos-engineering-the-history-principles-and-practice

    2023

  27. [35]

    ://www.gremlin.com/, accessed: 2025-01-29

    Inc G (2025 a ) Introducing gremlin for aws: Reliability testing and chaos engineering tools. ://www.gremlin.com/, accessed: 2025-01-29

    2025

  28. [36]

    ://speedscale.com/, accessed: 2025-01-29

    Inc S (2025 b ) Run realistic local environments without real cloud expenses. ://speedscale.com/, accessed: 2025-01-29

    2025

  29. [37]

    In: Practical Oracle Cloud Infrastructure: Infrastructure as a Service, Autonomous Database, Managed Kubernetes, and Serverless, Springer, pp 487--551

    Jak \'o bczyk MT (2020) Cloud-native architecture. In: Practical Oracle Cloud Infrastructure: Infrastructure as a Service, Autonomous Database, Managed Kubernetes, and Serverless, Springer, pp 487--551

    2020

  30. [38]

    In: Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp 1--10

    Jernberg H, Runeson P, Engstr \"o m E (2020) Getting started with chaos engineering-design of an implementation framework in practice. In: Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp 1--10

    2020

  31. [39]

    Wiley Interdisciplinary Reviews: Climate Change 10(1):e552

    Jones L (2019) Resilience isn't the same for all: Comparing subjective and objective approaches to resilience measurement. Wiley Interdisciplinary Reviews: Climate Change 10(1):e552

    2019

  32. [40]

    AWS DevOps Blog, ://aws.amazon.com/blogs/devops/chaos-engineering-on-amazon-eks-using-aws-fault-injection-simulator/

    Kahil O, Arenhage D (2021) Chaos engineering on amazon eks using aws fault injection simulator. AWS DevOps Blog, ://aws.amazon.com/blogs/devops/chaos-engineering-on-amazon-eks-using-aws-fault-injection-simulator/

    2021

  33. [41]

    In: Proceedings of the 11th working conference on mining software repositories, pp 92--101

    Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2014) The promises and perils of mining github. In: Proceedings of the 11th working conference on mining software repositories, pp 92--101

    2014

  34. [42]

    Empirical Software Engineering 21:2035--2071

    Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2016) An in-depth study of the promises and perils of mining github. Empirical Software Engineering 21:2035--2071

    2016

  35. [43]

    Information Processing & Management 60(6):103477

    Kang D, Kang T, Jang J (2023) Papers with code or without code? impact of github repository usability on the diffusion of machine learning research. Information Processing & Management 60(6):103477

    2023

  36. [44]

    In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pp 476--487, doi:10.1109/ICSE.2019.00060

    Kavaler D, Trockman A, Vasilescu B, Filkov V (2019 a ) Tool choice matters: Javascript quality assurance tools and usage outcomes in github projects. In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pp 476--487, doi:10.1109/ICSE.2019.00060

    2019

  37. [45]

    In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), IEEE, pp 476--487

    Kavaler D, Trockman A, Vasilescu B, Filkov V (2019 b ) Tool choice matters: Javascript quality assurance tools and usage outcomes in github projects. In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), IEEE, pp 476--487

    2019

  38. [46]

    Journal of Systems and Software 126:1--16

    Kratzke N, Quint PC (2017) Understanding cloud-native applications after 10 years of cloud computing-a systematic mapping study. Journal of Systems and Software 126:1--16

    2017

  39. [47]

    ://github.com/alexei-led/pumba, accessed: 2025-01-28

    Ledenev A, Contributors P (2025) Pumba: Chaos testing, network emulation, and stress testing tool for containers. ://github.com/alexei-led/pumba, accessed: 2025-01-28

    2025

  40. [48]

    A Rain Capital Research Note

    Lewis J, Wang C (2019) Chaos engineering: New approaches to security. A Rain Capital Research Note

    2019

  41. [49]

    ://github.com/linki/chaoskube, accessed: 2025-01-28

    Linki, Contributors C (2025) Chaoskube: Periodically kills random pods in your kubernetes cluster. ://github.com/linki/chaoskube, accessed: 2025-01-28

    2025

  42. [50]

    In: 2023 IEEE/ACM 18th Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS), IEEE, pp 195--201

    Malik S, Naqvi MA, Moonen L (2023) Chess: A framework for evaluation of self-adaptive systems based on chaos engineering. In: 2023 IEEE/ACM 18th Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS), IEEE, pp 195--201

    2023

  43. [51]

    GitHub repository, ://github.com/Netflix/SimianArmy, available at: https://github.com/Netflix/SimianArmy

    Netflix (2021) Simian army: Tools for cloud resiliency testing. GitHub repository, ://github.com/Netflix/SimianArmy, available at: https://github.com/Netflix/SimianArmy

    2021

  44. [52]

    ://github.com/Netflix/chaosmonkey, accessed: 2025-01-28

    Netflix (2025) Chaos monkey: A resiliency tool that helps applications tolerate random instance failures. ://github.com/Netflix/chaosmonkey, accessed: 2025-01-28

    2025

  45. [53]

    ://www.santanderconsumergs.com/news/https-impulsate-between-tech-chaos-engineering

    Nombela JG (2023) Chaos engineering: The art of introduce intentional failures. ://www.santanderconsumergs.com/news/https-impulsate-between-tech-chaos-engineering

    2023

  46. [54]

    Olaogun D (2024) Top 7 kubernetes chaos engineering tools. Speedscale Blog, ://speedscale.com/blog/kubernetes-chaos-engineering-tools/, this grey literature article provides a comprehensive comparison of chaos engineering tools for Kubernetes, including Speedscale, AWS Fault I...

    2024

  47. [55]

    GitHub repository, ://github.com/Optum/ChaoSlingr, available at: https://github.com/Optum/ChaoSlingr

    Optum (2019) Chaoslingr: Introducing security into chaos testing. GitHub repository, ://github.com/Optum/ChaoSlingr, available at: https://github.com/Optum/ChaoSlingr

    2019

  48. [56]

    GitHub repository, ://github.com/osrg/namazu, available at: https://github.com/osrg/namazu

    OSRG (2021) Namazu: Programmable fuzzy scheduler for testing distributed systems. GitHub repository, ://github.com/osrg/namazu, available at: https://github.com/osrg/namazu

    2021

  49. [57]

    arXiv preprint arXiv:241201416

    Owotogbe J, Kumara I, Heuvel WJVD, Tamburri DA (2024) Chaos engineering: A multi-vocal literature review. arXiv preprint arXiv:241201416

    2024

  50. [58]

    ://www.contino.io/insights/chaos-engineering

    Patel V (2022) What is chaos engineering and why you should break more things on purpose. ://www.contino.io/insights/chaos-engineering

    2022

  51. [59]

    Pawlikowski M (2025) Application-level fault injection. In: Chaos Engineering: Building Confidence in System Behavior through Fault Injection, O'Reilly Media, Inc., chap 8, ://www.oreilly.com/library/view/chaos-engineering/9781617297755/OEBPS/Text/08.htm, accessed on 21 January 2025

    2025

  52. [60]

    PhD thesis, Instituto Politecnico do Porto (Portugal)

    Ribeiro HAdF (2024) Chaos engineering: bring order to chaos. PhD thesis, Instituto Politecnico do Porto (Portugal)

    2024

  53. [61]

    ://www.infoq.com/articles/chaos-engineering-observability-visual-metaphors/, reviewed by Ben Linders

    Roa YN (2022) Chaos engineering and observability with visual metaphors. ://www.infoq.com/articles/chaos-engineering-observability-visual-metaphors/, reviewed by Ben Linders

    2022

  54. [62]

    Empirical Software Engineering 27(6):132

    S \'a nchez AB, Delgado-P \'e rez P, Medina-Bulo I, Segura S (2022) Mutation testing in the wild: findings from github. Empirical Software Engineering 27(6):132

    2022

  55. [63]

    ://github.com/Shopify/toxiproxy, accessed: 2025-01-28

    Shopify (2025) Toxiproxy: A tcp proxy to simulate network and system conditions for chaos and resiliency testing. ://github.com/Shopify/toxiproxy, accessed: 2025-01-28

    2025

  56. [64]

    In: 2022 International Conference on Smart Applications, Communications and Networking (SmartNets), IEEE, Hyderabad India, pp 1--7

    Siwach G, Haridas A, Chinni N (2022) Evaluating operational readiness using chaos engineering simulations on kubernetes architecture in big data. In: 2022 International Conference on Smart Applications, Communications and Networking (SmartNets), IEEE, Hyderabad India, pp 1--7

    2022

  57. [65]

    ://github.com/asobti/kube-monkey, accessed: 2025-01-28

    Sobti A, Contributors KM (2025) Kube-monkey: An implementation of netflix's chaos monkey for kubernetes clusters. ://github.com/asobti/kube-monkey, accessed: 2025-01-28

    2025

  58. [66]

    Sondhi S, Saad S, Shi K, Mamun M, Traore I (2021) Chaos engineering for understanding consensus algorithms performance in permissioned blockchains. In: 2021 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Co...

    2021

  59. [67]

    GitHub repository, ://github.com/ThalesGroup/chaos-engine, available at: https://github.com/ThalesGroup/chaos-engine

    ThalesGroup (2024) Chaos engine: A tool for testing resiliency in cloud applications. GitHub repository, ://github.com/ThalesGroup/chaos-engine, available at: https://github.com/ThalesGroup/chaos-engine

    2024

  60. [68]

    IEEE Access 8:123044--123060

    Torkura KA, Sukmana MI, Cheng F, Meinel C (2020) Cloudstrike: Chaos engineering for security and resiliency in cloud infrastructure. IEEE Access 8:123044--123060

    2020

  61. [69]

    arXiv preprint arXiv:220408108

    Tutko A, Henley AZ, Mockus A (2022) How are software repositories mined? a systematic literature review of workflows, methodologies, reproducibility, and tools. arXiv preprint arXiv:220408108

    2022

  62. [70]

    In: Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering, pp 120--129

    Ustunboyacioglu I, Kumara I, Di Nucci D, Tamburri DA, Van Den Heuvel WJ (2024) Data quality assessment in the wild: Findings from github. In: Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering, pp 120--129

    2024

  63. [71]

    Journal of Systems and Software 183:111117

    Wattanakriengkrai S, Chinthanet B, Hata H, Kula RG, Treude C, Guo J, Matsumoto K (2022) Github repositories with links to academic papers: Public access, traceability, and evolution. Journal of Systems and Software 183:111117

    2022

  64. [72]

    arXiv preprint arXiv:240700125

    Yu G, Tan G, Huang H, Zhang Z, Chen P, Natella R, Zheng Z (2024) A survey on failure analysis and fault injection in ai systems. arXiv preprint arXiv:240700125

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.