Unsafe at any AUC: Unlearned Lessons from Sociotechnical Disasters for Responsible AI

Abigail Z. Jacobs; Andrew Smart; Joshua A. Kroll; R. Stuart Geiger

Unsafe at any AUC: Unlearned Lessons from Sociotechnical Disasters for Responsible AI

T0 review · 0 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Model benchmarks cannot certify AI safety; safety lives in the whole sociotechnical system.

desk verdict A solid sociotechnical synthesis that deserves a serious referee—the critique of component-level safety is well supported, the positive organizational-governance agenda is more prescription than proof, and the paper mostly admits that itself. read the letter →

arxiv 2607.14353 v1 pith:Y72N2NNU submitted 2026-07-15 cs.CY cs.AIcs.SYeess.SY

Joshua A. Kroll , Andrew Smart , R. Stuart Geiger , Abigail Z. Jacobs This is my paper

classification cs.CYcs.AIcs.SYeess.SY

keywords accidentsresponsibleAIsafetysociotechnicalsystemsorganizationsdisastersriskperceptionnormalizationofdeviance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that current responsible-AI practice—improving model performance, benchmarks, alignment, documentation, and audits—confuses reliability with safety. Drawing on decades of disaster research, it shows that catastrophic failures like Challenger, Chernobyl, the Boeing 737 MAX, and Silicon Valley Bank share recurring organizational causes: poor risk perception, misaligned incentives, permanent rush, suppression of bad news, and displacement of blame onto operators. The same patterns appear in today's AI industry, so safety cannot be guaranteed by any model metric or component-level fix. If the paper is right, responsible AI must shift its primary focus from technical artifacts to organizational cultures, traceable processes, and governance structures with real authority to act on risk.

What carries the argument

The analytic mechanism is the systems-safety lens, which distinguishes component reliability from system-level emergent behavior. The load-bearing principle is 'safe components do not imply safe systems': failures arise from interactions among correctly functioning parts, so verification, benchmarking, and reliability assurance are necessary but never sufficient for safety. This principle drives the paper's taxonomy of six unlearned lessons, which it uses to connect disaster case studies to current AI practices and to motivate sociotechnical interventions such as safety cultures, curmudgeons, traceability, and structural incentives.

What would settle it

An empirical study showing that real-world AI harms are fully explained by model performance metrics, with no residual variation attributable to organizational safety culture, would falsify the central claim. More directly, a demonstrated counterexample of a high-AUC, component-focused system with zero recorded harms and no organizational safety practices would undermine the thesis that safety requires system-level governance.

Watch

Extended reading notes

Core claim

The paper's central claim is that safety is an emergent property of a composed sociotechnical system, not a property of individual components—models, datasets, or their performance scores. At any level of performance, or for any tradeoff between failure modes ('at any AUC'), a system that causes harm in its context of use is unsafe. Safety therefore exists only for the whole assemblage of technology, people, organizations, incentives, and environment; without a context of use, claims about safe component performance are 'technical artifice.' The paper distills six 'unlearned lessons' from industrial disasters and maps each to common AI practices, arguing that the same organizational dynamics

Load-bearing premise

The load-bearing premise is that the organizational failure mechanisms documented in other safety-critical industries—and the accuracy of the retold disaster accounts—transfer to modern AI development and deployment; if that analogy fails, the taxonomy is illustrative rather than evidence-bearing.

Editorial extensions

If this is right

Safety claims about AI must be contextual: a system is safe only within a defined context of use, not in the abstract.
Benchmarks, audits, red-teaming, and documentation are useful only when embedded in organizational processes that act on them; as standalone rituals they can create the illusion of safety.
Organizations deploying AI should institutionalize protected dissent, traceable decision records, pre-mortems, blameless postmortems, and leaders with authority to block launches.
AI safety research should broaden from component-level fixes to system-level interventions based on accident-modeling methods.
Regulation should aim to strengthen organizational risk perception and control, not merely mandate technical tests or disclosures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the paper is right, model cards and benchmark scores are the wrong primary audit objects; governance practices—like whether a safety review can veto a release—would carry most of the predictive signal for real-world harm.
A testable extension: compare deployments matched on model performance but differing in safety-culture practices; the paper predicts the organizational dimension, not AUC, explains downstream harm rates.
The same logic implies that procurement standards and regulatory frameworks should require assurance cases with explicit claims, evidence, and skeptical review rather than metric dashboards.
A further consequence is that incident reporting and whistleblower protections, long used in aviation and medicine, may be more consequential for AI safety than any technical alignment method.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a well-argued position paper, and I agree with the reader that it should go to peer review. But the strongest headline claim—the line about safety existing only for the composed assemblage—is a philosophical stance, not something the paper demonstrates empirically. That is fine for the genre, but don't cite it as evidence.

What is actually new: the six "unlearned lessons" taxonomy and the SRE-as-precedent discussion. The taxonomy does useful packaging work, pulling together Vaughan, Perrow, and Leveson into categories that map cleanly onto AI practice: benchmarks treated as safety, moral crumple zones, no-bad-news cultures, incentive-driven drift. The SRE example is a concrete anchor showing that a software discipline already institutionalizes system-level risk thinking, which helps the "AI is not exceptional" argument. The paper is also genuinely honest: it flags that STAMP's transfer to AI is open, that documentation practices are unvalidated, and that participation can be pseudo-participation. That honesty is real credit.

The soft spot is the one the stress-test identifies. The case studies support the negative claim—component reliability does not entail system safety—much better than the positive claim that organizational and governance structures are the correct primary locus for AI safety. The disaster cases come from mature, heavily regulated industries, and the analogy to fast-moving AI firms is plausible but not established. However, the paper repeatedly frames its positive contribution as a research agenda and names its open questions, so this is a scope-of-claim limitation rather than an internal contradiction. It should be read as a call for research, not a completed empirical argument. Self-citation is noticeable but not circular; the central claim does not depend on the authors' prior outputs.

Who this is for: people in FAccT, AI governance, and responsible-AI practice, plus anyone tempted to treat a benchmark score as a safety certificate. It will not persuade the existential-risk crowd, but that is not the target.

Recommendation: send it to peer review. It deserves referee time. My own verdict would be accept with minor revisions, mainly asking the authors to sharpen the distinction in the conclusion between the well-supported negative claim and the programmatic positive one.

Referee Report

0 major / 5 minor

Summary. This position paper argues that AI safety is a property of full sociotechnical systems rather than of individual models or technical components. Drawing on safety science and a set of well-known disasters—Challenger, Chernobyl, USS Scorpion, Three Mile Island, Boeing 737 MAX, Silicon Valley Bank, the Uber autonomous-vehicle crash, and others—the authors identify six “unlearned lessons” (poor risk perception; hazardous incentives; permanent rush cultures; suppression of bad news; safe components not implying safe systems; and the “kick the can” displacement of responsibility onto operators). They then translate these lessons into a set of organizational and governance-oriented practices: authentic safety culture, curmudgeons and critics, traceable processes, psychological safety and diversity, meaningful external participation, treating failure as normal, and structural incentives for self-regulation. The paper uses Site Reliability Engineering as a precedent, critiques component-level responsible-AI tools such as documentation and benchmarks, and proposes a research agenda centered on applying systems-safety methods such as STAMP to AI systems. The central normative claim is that reliability or benchmark performance is not safety and that safety can only be assessed in context of the composed sociotechnical system.

Significance. If the paper's position is accepted, it would usefully reframe responsible-AI evaluation away from model metrics and toward organizational and governance structures. The paper's strengths are its careful synthesis of an existing safety-science literature, its concrete taxonomy of recurring failure etiologies, and its repeated honesty about open empirical questions: §4.4 notes that documentation practices require validation, §5.3 calls governance-practice validation a critical open problem, and §5.4 explicitly asks whether STAMP can translate to AI systems. These disclaimers substantially disarm the otherwise obvious objection that the positive program is unproven: the paper presents itself as a grounded research agenda rather than a completed empirical demonstration. It also makes a valuable intervention by distinguishing safety from reliability and by arguing that component-level tools such as AUC, benchmarks, and alignment techniques cannot substitute for system-level analysis. The paper is synthetic rather than experimental, but it is a significant contribution to the responsible-AI literature.

minor comments (5)

[§3.5] The Chernobyl account is internally inconsistent and should be clarified. The text says operators made “seemingly insignificant changes to components of the reactor’s control rods with inadequate records” but then states that investigation “did not uncover specific component failures or operator deviation from procedure.” Chernobyl is one of the paper's recurring anchor cases, and the standard accident literature (including INSAG-7) does identify operator deviations in addition to design flaws. Please rewrite this passage to align with the cited sources or replace the Chernobyl example with a less contested instance of component-level correctness failing to imply system safety.
[§6] The concluding sentence “System behavior safety exists only for this composed assemblage” is a strong (and defensible) conceptual claim, but it could be misread as an empirically established result. Since §4.4 and §5.3 explicitly acknowledge that the proposed organizational practices are not yet validated, consider adding one sentence in §6 that distinguishes the conceptual redefinition of safety from the empirical research agenda the paper proposes.
[§5.3] Minor technical errors: “ISO 420001” should be “ISO/IEC 42001,” and “International Organizations for Standardization” should be “International Organization for Standardization.”
[§5.1] Typo: “Preparadness” should be “Preparedness” in the list of themes from the SRE literature.
[References] The Khlaaf (2023) reference lists the arXiv identifier “2606.29390,” which is inconsistent with a 2023 publication year. Please verify the identifier and date.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the sociotechnical-safety thesis is supported by external safety-science literature and case evidence; self-citations are background, not load-bearing.

full rationale

The paper is an argumentative synthesis, not a derivation with fitted parameters, equations, or predictions. Its central thesis—component reliability (e.g., AUC) does not entail system safety, which is an emergent property of the whole sociotechnical assemblage—is supported by independent external evidence: Vaughan's Challenger study, Perrow's normal accidents, Leveson's STAMP literature, Elish's moral crumple zone, and the Obermeyer, Boeing, SVB, and Uber case accounts. The paper explicitly adopts a broad stipulative definition of 'AI system' that includes organizational processes (Section 1), so the conclusion that organizational factors matter is partly built into the framing; but the load-bearing, non-circular step is the demonstration that failures occur even when components work to specification (Section 3.5, Chernobyl, cybersecurity incidents), which comes from outside the paper and does not presuppose the conclusion. There are several self-citations (Kroll 2020/2021; Geiger et al. 2018/2024; Smart & Kasirzadeh 2024; Jatho & Kroll 2022; Rismani et al. 2023; Abdu & Jacobs 2026), but none is the sole support for the central claim: they are used for background concepts such as traceability, accountability, documentation practice, and prior STAMP applications, and the main argument rests on the independently citable safety-science canon. The paper also explicitly flags its own open questions ('can the utility of STAMP translate to AI systems?', Section 5.4; documentation 'is, in fact, an empirical question requiring validation', Section 4.4), so it does not present its proposed interventions as already validated. The skeptic's complaint—that the positive thesis about organizational/governance locus outruns the evidence—is a scope-of-claim or evidence-weight concern, which the review rules classify as correctness risk, not circularity. Accordingly, no circular step meets the quoted-reduction standard; score 2 reflects minor non-load-bearing self-citation only.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numeric free parameters are present; the paper is qualitative. The load-bearing axioms are the transferability of industrial accident lessons to AI, the emergent-property view of safety, the trustworthiness of the chosen historical accounts, and the validity of the paper's own taxonomy. No new entities are postulated.

assumptions (4)

domain assumption Accident case studies (Challenger, Chernobyl, SVB, Boeing) provide transferable causal knowledge about organizational failure that applies to AI systems.
The paper builds its six lessons on these cases and assumes the same mechanisms operate in AI development; stated in §2 and §3 intro ('Our analysis maps known challenges of systems safety to the assessment of real world AI systems').
domain assumption Safety is an emergent system-level property not decomposable into component reliability.
Adopted from Leveson and Perrow (§2, §3.5); the paper treats this as the foundational premise rather than deriving it from first principles.
domain assumption The cited retrospective accounts of disaster causation (Vaughan on Challenger, Plokhy on Chernobyl, Washington Post on SVB) are accurate and uncontested.
The paper relies on these accounts without engaging competing historical explanations; cited in §3.1–3.6 and elsewhere.
ad hoc to paper The six-part 'unlearned lessons' taxonomy is a valid and useful partition of failure etiologies.
The taxonomy is constructed for this paper, not derived from a systematic review; the paper itself admits 'While not exhaustive' (§3).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Unsafe at any AUC: Unlearned Lessons from Sociotechnical Disasters for Responsible AI." pith.science (2026). https://pith.science/paper/Y72N2NNU

@misc{pith2026260714353,
  author       = {Pith},
  title        = {Pith review of: Unsafe at any AUC: Unlearned Lessons from Sociotechnical Disasters for Responsible AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y72N2NNU}},
  note         = {Machine review of arXiv:2607.14353}
}

read the original abstract

As automated decision-making and data-driven technologies pervade society and are used to manage consequential outcomes, understanding the technology's capabilities, limitations, and attendant risks in context requires analysis of full sociotechnical systems. Sociotechnical analysis of risks in highly complex systems provides clear lessons for the design and evaluation of AI systems, transcending a technical focus on reliable or "responsibly designed" components to understand risks at a systems level. Human-made catastrophes have been studied for decades because of the severity of these events: consider Chernobyl, Three Mile Island, Fukushima-Daiichi, Bhopal, the Challenger disaster. A common misconception is that these kinds of events are freak accidents, resulting from the inherently unforeseeable interactions in complex systems. Closer examination reveals that the risks and hazards were well-known beforehand but not acted upon due to social structural, political and economic factors. We outline several areas where the development and use of AI can benefit from learning these unlearned lessons: improved risk perception, communication, and analysis at the organizational level; traceability of requirements and responsibilities; and holistic approaches to responsibility and safety that include social and organizational dynamics as first-order engineering concerns. For each area, we offer concrete unlearned lessons and exemplify how they led to failure in prior accidents as well as examples of how these lessons remain unlearned for modern computing systems, particularly AI.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

249 extracted references · 8 canonical work pages

[1]

UCLA Law Review , volume=

The Public Harms of Private Surveillance , author=. UCLA Law Review , volume=. 2025 , URL=

2025
[2]

First Monday , doi =

Ahmed, Shazeda and Ja. First Monday , doi =
[3]

Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =

Ali, Sanna J and Christin, Ang. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =

2023
[4]

Communications of the ACM , volume =

Computational complexity and information asymmetry in financial products , author =. Communications of the ACM , volume =. 2011 , publisher =

2011
[5]

Responsive

Ayres, Ian and Braithwaite, John , year =. Responsive
[6]

Analysis, Design and Evaluation of Man--Machine Systems , pages =

Ironies of automation , author =. Analysis, Design and Evaluation of Man--Machine Systems , pages =. 1983 , publisher =

1983
[7]

, urldate =

Gebru, Timnit and Torres, Émile P. , urldate =. The. doi:10.5210/fm.v29i4.13636 , shorttitle =

doi:10.5210/fm.v29i4.13636
[8]

2020 , publisher =

Can artificial intelligence transform higher education? , author =. 2020 , publisher =

2020

Show all 249 references

[9]

arXiv preprint arXiv:2501.17805 , year =

Bengio, Yoshua and Mindermann, S. arXiv preprint arXiv:2501.17805 , year =

arXiv
[10]

doi:10.48550/arXiv.2511.19863 , year =

Bengio, Yoshua and Clare, Stephen and Prunkl, Carina and Andriushchenko, Maksym and Bucknall, Ben and Fox, Philip and Maslej, Nestor and McGlynn, Conor and Murray, Malcolm and Rismani, Shalaleh and others , journal =. doi:10.48550/arXiv.2511.19863 , year =

doi
[11]

2016 , URL =

Site reliability engineering: How Google runs production systems , author =. 2016 , URL =

2016
[12]

Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , pages =

Robot rights? Let's talk about human welfare instead , author =. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , pages =
[13]

2021 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages =

Large image datasets: A pyrrhic win for computer vision? , author =. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages =. 2021 , organization =

2021
[14]

1975 , publisher =

The mythical man-month: essays on software engineering , author =. 1975 , publisher =

1975
[15]

2018 , publisher =

Artificial unintelligence: How computers misunderstand the world , author =. 2018 , publisher =

2018
[16]

Big data & society , volume =

How the machine ‘thinks’: Understanding opacity in machine learning algorithms , author =. Big data & society , volume =. 2016 , publisher =

2016
[17]

Annual Review of Sociology , volume =

The Society of Algorithms , author =. Annual Review of Sociology , volume =. 2020 , publisher =

2020
[18]

1981 , publisher =

Checkland, Peter , title =. 1981 , publisher =

1981
[19]

2016 , publisher =

Man-made catastrophes and risk information concealment , author =. 2016 , publisher =

2016
[20]

Journal of International Business Studies , volume =

Firm self-regulation through international certifiable standards: Determinants of symbolic versus substantive implementation , author =. Journal of International Business Studies , volume =. 2006 , publisher =

2006
[21]

Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages =

Reviewable automated decision-making: A framework for accountable algorithmic systems , author =. Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages =. 2021 , doi =

2021
[22]

Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =

Understanding accountability in algorithmic supply chains , author =. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =. 2023 , doi =

2023
[23]

2019 , publisher =

Between truth and power: The legal constructions of informational capitalism , author =. 2019 , publisher =

2019
[24]

Proceedings of the 2022 ACM conference on fairness, accountability, and transparency , pages=

Accountability in an algorithmic society: relationality, responsibility, and robustness in machine learning , author=. Proceedings of the 2022 ACM conference on fairness, accountability, and transparency , pages=. 2022 , doi =

2022
[25]

2020 , location =

Design justice: Community-led practices to build the worlds we need , author =. 2020 , location =

2020
[26]

2002 , publisher =

Dekker, Sidney and Woods, David D , journal =. 2002 , publisher =

2002
[27]

2016 , publisher =

Drift into failure: From hunting broken components to understanding complex systems , author =. 2016 , publisher =

2016
[28]

Government Information Quarterly , volume =

Integral system safety for machine learning in the public sector: An empirical account , author =. Government Information Quarterly , volume =. 2024 , publisher =

2024
[29]

Oxford Handbook of

System Safety and Artificial Intelligence , author =. Oxford Handbook of. 2022 , doi =

2022
[30]

Toward Sociotechnical

Dobbe, Roel and Wolters, Anouk , journal =. Toward Sociotechnical. 2024 , publisher =

2024
[31]

2025 , doi =

Dobbe, Roel , journal =. 2025 , doi =

2025
[32]

Industry and Innovation , volume =

Learning from a drastic failure: the case of the Airbus A380 program , author =. Industry and Innovation , volume =. 2014 , publisher =

2014
[33]

ACM SIGSOFT Software Engineering Notes , volume =

The Ariane 5 software failure , author =. ACM SIGSOFT Software Engineering Notes , volume =. 1997 , publisher =

1997
[34]

International Conference on Computer Safety, Reliability, and Security , pages =

Byzantine fault tolerance, from theory to reality , author =. International Conference on Computer Safety, Reliability, and Security , pages =. 2003 , organization =

2003
[35]

American Journal of Sociology , volume=

Legal ambiguity and symbolic structures: Organizational mediation of civil rights law , author=. American Journal of Sociology , volume=. 1992 , publisher=

1992
[36]

American journal of Sociology , volume =

The endogeneity of legal regulation: Grievance procedures as rational myth , author =. American journal of Sociology , volume =. 1999 , publisher =

1999
[37]

Explaining Compliance: Business Responses to Regulation , pages=

To comply or not to comply—That isn’t the question: How organizations construct the meaning of compliance , author=. Explaining Compliance: Business Responses to Regulation , pages=. 2011 , publisher=

2011
[38]

Praise the machine! Punish the human! The contradictory history of accountability in automated aviation , author =
[39]

Engaging Science, Technology, and Society , volume = 5, year =

Moral crumple zones: Cautionary tales in human-robot interaction , author =. Engaging Science, Technology, and Society , volume = 5, year =
[40]

Repairing innovation: A study of integrating

Elish, Madeleine Clare and Watkins, Elizabeth Anne , institution =. Repairing innovation: A study of integrating
[41]

1996 , publisher =

Tyranny of the bottom line: Why corporations make good people do bad things , author =. 1996 , publisher =

1996
[42]

1963 , publisher =

Engines of culture: Philanthropy and art museums , author =. 1963 , publisher =

1963
[43]

Handbook of ethics, values, and technological design: sources, theory, values and application domains , pages =

Design for values and operator roles in sociotechnical systems , author =. Handbook of ethics, values, and technological design: sources, theory, values and application domains , pages =. 2015 , publisher =

2015
[44]

2019 , publisher =

Power without knowledge: a critique of technocracy , author =. 2019 , publisher =

2019
[45]

Atul Gawande , publisher =
[46]

Communications of the ACM , volume =

Datasheets for datasets , author =. Communications of the ACM , volume =. 2021 , publisher =

2021
[47]

International Journal of Communication , author =

Making Algorithms Public: Reimagining Auditing from Matters of Fact to Matters of Concern , volume =. International Journal of Communication , author =. 2024 , pages =

2024
[48]

Berkeley Tech

Through the handoff lens: competing visions of autonomous futures , author =. Berkeley Tech. LJ , volume =. 2020 , publisher =

2020
[49]

To Be High-Risk, or Not To Be—Semantic Specifications and Implications of the

Golpayegani, Delaram and Pandit, Harshvardhan J and Lewis, Dave , booktitle =. To Be High-Risk, or Not To Be—Semantic Specifications and Implications of the. 2023 , doi =

2023
[50]

2021 , publisher =

The dawn of everything: A new history of humanity , author =. 2021 , publisher =

2021
[51]

Safety science , volume =

The nature of safety culture: a review of theory and research , author =. Safety science , volume =. 2000 , publisher =

2000
[52]

Law & policy , volume =

Industry self-regulation: an institutional perspective , author =. Law & policy , volume =. 1997 , publisher =

1997
[53]

Proceedings Fourth Safety Congress , pages =

Standardization of safeguards , author =. Proceedings Fourth Safety Congress , pages =. 1915 , doi =

1915
[54]

STAT News , author =
[55]

Benjamin, Ruha , date =
[56]

2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) , pages =

Enhanced well-being assessment as basis for the practical implementation of ethical and rights-based normative principles for AI , author =. 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) , pages =. 2020 , organization =

2020
[57]

Proceedings of the ACM on Human-Computer Interaction , volume =

Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and Desiderata , author =. Proceedings of the ACM on Human-Computer Interaction , volume =. 2022 , publisher =

2022
[58]

2021 , doi=

Hendrycks, Dan and Carlini, Nicholas and Schulman, John and Steinhardt, Jacob , journal =. 2021 , doi=

2021
[59]

2020 , publisher =

Herkert, Joseph and Borenstein, Jason and Miller, Keith , journal =. 2020 , publisher =

2020
[60]

Ethics and Information Technology , volume =

ChatGPT is bullshit , author =. Ethics and Information Technology , volume =. 2024 , publisher =

2024
[61]

Jama , volume =

Deep learning—a technology with the potential to transform health care , author =. Jama , volume =. 2018 , publisher =

2018
[62]

Accountability: Power, ethos and the technologies of managing , volume =

The ‘awful idea of accountability’: inscribing people into the measurement of objects , author =. Accountability: Power, ethos and the technologies of managing , volume =. 1996 , publisher =

1996
[63]

Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency , pages =

Towards accountability for machine learning datasets: Practices from software engineering and infrastructure , author =. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency , pages =. 2021 , doi =

2021
[64]

2022 , journal =

Artificial Intelligence: Too Fragile to Fight? , author =. 2022 , journal =

2022
[65]

Concrete Safety for

Jatho, Edgar W and Mailloux, Logan O and Williams, Eugene D and McClure, Patrick and Kroll, Joshua A , journal =. Concrete Safety for. 2023 , doi =

2023
[66]

Journal of business ethics , volume=

The social license to operate , author=. Journal of business ethics , volume=. 2016 , publisher=

2016
[67]

Toward Comprehensive Risk Assessments and Assurance of

Khlaaf, Heidy , institution =. Toward Comprehensive Risk Assessments and Assurance of. 2023 , url =

2023
[68]

arXiv preprint arXiv:2506.11135 , year =

Large Language Models and Emergence: A Complex Systems Perspective , author =. arXiv preprint arXiv:2506.11135 , year =

arXiv
[69]

Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences , volume =

The fallacy of inscrutability , author =. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences , volume =. 2018 , publisher =

2018
[70]

2020 , publisher =

Accountability in computer systems , author =. 2020 , publisher =

2020
[71]

2021 Conference on Fairness, Accountability, and Transparency , pages =

Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems , author =. 2021 Conference on Fairness, Accountability, and Transparency , pages =. 2021 , doi =

2021
[72]

Responsible

Kroll, Joshua A , journal =. Responsible
[73]

2022 , institution =

Understanding, Assessing, and Mitigating Safety Risks in Artificial Intelligence Systems , author =. 2022 , institution =

2022
[74]

Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =

The Misuse of AUC: What High Impact Risk Assessment Gets Wrong , author =. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =

2023
[75]

Where are the missing masses? The sociology of a few mundane artifacts , shorttitle =

Latour, Bruno , editor =. Where are the missing masses? The sociology of a few mundane artifacts , shorttitle =. Shaping Technology/Building Society: Studies in Sociotechnical Change , publisher =
[76]

ACM Computing Surveys (CSUR) , volume =

A survey of DevOps concepts and challenges , author =. ACM Computing Surveys (CSUR) , volume =. 2019 , publisher =

2019
[77]

An investigation of the

Leveson, Nancy and Turner, Clark S , journal =. An investigation of the. 1993 , publisher =

1993
[78]

Safety science , volume =

A new accident model for engineering safer systems , author =. Safety science , volume =. 2004 , publisher =

2004
[79]

Journal of spacecraft and Rockets , volume =

Role of software in spacecraft accidents , author =. Journal of spacecraft and Rockets , volume =. 2004 , doi=

2004
[80]

Organization studies , volume =

Moving beyond normal accidents and high reliability organizations: A systems approach to safety in complex systems , author =. Organization studies , volume =. 2009 , publisher =

2009
[81]

Safety Science , volume =

Applying systems thinking to analyze and learn from events , author =. Safety Science , volume =. 2011 , publisher =

2011
[82]

doi:10.7551/mitpress/8179.003.0019 , booktitle =

Leveson, Nancy , date =. doi:10.7551/mitpress/8179.003.0019 , booktitle =

doi
[83]

2016 , publisher =

Engineering a safer world: Systems thinking applied to safety , author =. 2016 , publisher =

2016
[84]

Leveson, Nancy , institution =. Safety
[85]

2023 , publisher =

An Introduction to System Safety Engineering , author =. 2023 , publisher =

2023
[86]

Transactions on Machine Learning Research , issn =

Holistic Evaluation of Language Models , author =. Transactions on Machine Learning Research , issn =. 2023 , url =

2023
[87]

1996 , publisher =

Ariane 5 flight 501 failure report by the inquiry board , author =. 1996 , publisher =

1996
[88]

Regulation & Governance , volume =

Certification systems for machine learning: Lessons from sustainability , author =. Regulation & Governance , volume =. 2022 , publisher =

2022
[89]

McDonald, Allan J and Hansen, James R , year =
[90]

arXiv preprint arXiv:2011.08512 , year =

Preventing repeated real world AI failures by cataloging incidents: The AI incident database , author =. arXiv preprint arXiv:2011.08512 , year =

2011 arXiv
[91]

Social Research: An International Quarterly , volume =

Owning ethics: Corporate logics, silicon valley, and the institutionalization of ethics , author =. Social Research: An International Quarterly , volume =. 2019 , publisher =

2019
[92]

Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages =

Algorithmic impact assessments and accountability: The co-construction of impacts , author =. Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages =

2021
[93]

American Journal of Sociology , volume =

Institutionalized organizations: Formal structure as myth and ceremony , author =. American Journal of Sociology , volume =. 1977 , publisher =

1977
[94]

Proceedings of the conference on fairness, accountability, and transparency , pages =

Model cards for model reporting , author =. Proceedings of the conference on fairness, accountability, and transparency , pages =. 2019 , doi =

2019
[95]

arXiv preprint arXiv:1805.03677 , primaryClass=

The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards , author=. arXiv preprint arXiv:1805.03677 , primaryClass=. 2018 , eprint=

2018 arXiv
[96]

Philosophy & Technology , volume =

Decolonial AI: Decolonial theory as sociotechnical foresight in artificial intelligence , author =. Philosophy & Technology , volume =. 2020 , publisher =

2020
[97]

2013 , publisher =

To save everything, click here: The folly of technological solutionism , author =. 2013 , publisher =

2013
[98]

, author =

Machine behaviour is old wine in new bottles. , author =. Nature , volume =. 2019 , publisher =

2019
[99]

Report of the Data & Society Research Institute , howpublished =

Assembling accountability: algorithmic impact assessment for the public interest , author =. Report of the Data & Society Research Institute , howpublished =
[100]

California Law Review , volume =

Saving governance-by-design , author =. California Law Review , volume =. 2018 , publisher =

2018
[101]

Berkeley Technology Law Journal , volume =

Procurement as policy: Administrative process for machine learning , author =. Berkeley Technology Law Journal , volume =. 2019 , doi =

2019
[102]

Proceedings of the ACM on Human-Computer Interaction , volume =

This thing called fairness: Disciplinary confusion realizing a value in technology , author =. Proceedings of the ACM on Human-Computer Interaction , volume =. 2019 , publisher =

2019
[103]

The Oxford handbook of ethics of AI , volume =

The concept of handoff as a model for ethical analysis and design , author =. The Oxford handbook of ethics of AI , volume =. 2020 , publisher =

2020
[104]

1965 , publisher =

Unsafe at Any Speed: The Designed-In Dangers of the American Automobile , author =. 1965 , publisher =

1965
[105]

International Organization , volume =

Uncertainty, risk, and the financial crisis of 2008 , author =. International Organization , volume =. 2014 , publisher =

2008
[106]

Structured Programming

Dijkstra, Edsger. Structured Programming. Software Engineering Techniques. 1970

1970
[107]

2015 , publisher=

Simple rules: How to thrive in a complex world , author=. 2015 , publisher=

2015
[108]

Science and engineering ethics , volume =

Accountability in a computerized society , author =. Science and engineering ethics , volume =. 1996 , publisher =

1996
[109]

2023 , url =

Artificial Intelligence Risk Management Framework , institution =. 2023 , url =

2023
[110]

2018 , publisher =

Algorithms of Oppression: How search engines reinforce racism , author =. 2018 , publisher =

2018
[111]

USENIX ;login: , month = jul, day = 29, year = 2024, url =

Laura Nolan , title =. USENIX ;login: , month = jul, day = 29, year = 2024, url =

2024
[112]

Proceedings of the 24th Annual International Conference on Digital Government Research , pages =

Diagnosing and Addressing Emergent Harms in the Design Process of Public AI and Algorithmic Systems , author =. Proceedings of the 24th Annual International Conference on Digital Government Research , pages =. 2023 , doi =

2023
[113]

Science , volume =

Dissecting racial bias in an algorithm used to manage the health of populations , author =. Science , volume =. 2019 , publisher =

2019
[114]

Wall street as community of fate: Toward financial industry self-regulation , author =. U. Pa. L. Rev. , volume =. 2010 , publisher =

2010
[115]

Research in transportation economics , volume =

Analyzing aviation safety: Problems, challenges, opportunities , author =. Research in transportation economics , volume =. 2013 , publisher =

2013
[116]

2015 , publisher =

The black box society: The secret algorithms that control money and information , author =. 2015 , publisher =

2015
[117]

Proceedings of the conference on fairness, accountability, and transparency , pages =

Problem formulation and fairness , author =. Proceedings of the conference on fairness, accountability, and transparency , pages =. 2019 , doi =

2019
[118]

1984 , publisher =

Normal accidents: Living with high risk technologies , author =. 1984 , publisher =

1984
[119]

2018 , publisher =

Chernobyl: The history of a nuclear catastrophe , author =. 2018 , publisher =

2018
[120]

2020 , publisher =

Trust in numbers , author =. 2020 , publisher =

2020
[121]

1999 , publisher =

The Audit Society: Rituals of Verification , author =. 1999 , publisher =

1999
[122]

International Journal of Auditing , volume =

The audit society—Second thoughts , author =. International Journal of Auditing , volume =. 2000 , publisher =

2000
[123]

Machine Bias , author =
[124]

Advances in Neural Information Processing Systems , volume =

Failing loudly: An empirical study of methods for detecting dataset shift , author =. Advances in Neural Information Processing Systems , volume =
[125]

Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society , pages =

Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products , author =. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society , pages =

2019
[126]

Raji, Inioluwa Deborah and Smart, Andrew and White, Rebecca N and Mitchell, Margaret and Gebru, Timnit and Hutchinson, Ben and Smith-Loud, Jamila and Theron, Daniel and Barnes, Parker , booktitle =
[127]

Concrete Problems in

Inioluwa Deborah Raji and Roel Dobbe , year=. Concrete Problems in. 2401.10899 , journal=

arXiv
[128]

Bender and Alex Hanna and Amandalynne Paullada , booktitle=

Inioluwa Deborah Raji and Emily Denton and Emily M. Bender and Alex Hanna and Amandalynne Paullada , booktitle=. 2021 , url=

2021
[129]

From Algorithmic Audits to Actual Accountability: Overcoming Practical Roadblocks on the Path to Meaningful Audit Interventions for AI Governance , booktitle =

Raji, Inioluwa Deborah , year =. From Algorithmic Audits to Actual Accountability: Overcoming Practical Roadblocks on the Path to Meaningful Audit Interventions for AI Governance , booktitle =
[130]

2022 , doi =

Raji, Inioluwa Deborah and Kumar, I Elizabeth and Horowitz, Aaron and Selbst, Andrew , booktitle =. 2022 , doi =

2022
[131]

arXiv preprint arXiv:2305.05733 , year =

Algorithms as social-ecological-technological systems: an environmental justice lens on algorithmic audits , author =. arXiv preprint arXiv:2305.05733 , year =

arXiv
[132]

Strategy Science , volume =

Enabling technologies and the role of private firms: A machine learning matching analysis , author =. Strategy Science , volume =. 2021 , publisher =

2021
[133]

2023 , doi =

Rismani, Shalaleh and Shelby, Renee and Smart, Andrew and Delos Santos, Renelito and Moon, AJung and Rostamzadeh, Negar , booktitle =. 2023 , doi =

2023
[134]

2023 , doi =

Rismani, Shalaleh and Shelby, Renee and Smart, Andrew and Jatho, Edgar and Kroll, Joshua and Moon, AJung and Rostamzadeh, Negar , booktitle =. 2023 , doi =

2023
[135]

California Management Review , volume =

Managing high reliability organizations , author =. California Management Review , volume =. 1990 , publisher =

1990
[136]

Ethical Theory and Moral Practice , volume =

Agency laundering and information technologies , author =. Ethical Theory and Moral Practice , volume =. 2019 , publisher =

2019
[137]

Rushby, John , TITLE =
[138]

Technology and Culture , volume =

After innovation, turn to maintenance , author =. Technology and Culture , volume =. 2018 , publisher =

2018
[139]

Ruster, Lorenn P and Oliva-Altamirano, Paola and Daniell, Katherine A , booktitle =
[140]

Der Transparenz-Imperativ , pages =

The morality of transparency: clarity versus emptiness , author =. Der Transparenz-Imperativ , pages =. 2019 , publisher =

2019
[141]

Journal of Legal Anthropology , volume =

Good people doing bad things: Compliance regimes in organisations , author =. Journal of Legal Anthropology , volume =. 2021 , publisher =

2021
[142]

Zygon , volume =

Schradle, Nathan , title =. Zygon , volume =. doi:10.1111/zygo.12637 , url =. https://onlinelibrary.wiley.com/doi/pdf/10.1111/zygo.12637 , abstract =

doi
[143]

SE4ML: software engineering for machine learning (NeurIPS 2014 Workshop) , volume =

Machine learning: The high interest credit card of technical debt , author =. SE4ML: software engineering for machine learning (NeurIPS 2014 Workshop) , volume =. 2014 , organization =

2014
[144]

2018 , journal =

Winner's curse? On pace, progress, and empirical rigor , author =. 2018 , journal =

2018
[145]

Conference on Fairness, Accountability, and Transparency , pages =

Fairness and abstraction in sociotechnical systems , author =. Conference on Fairness, Accountability, and Transparency , pages =. 2019 , organization =

2019
[146]

2020 , publisher =

Selbst, Andrew D , journal =. 2020 , publisher =

2020
[147]

arXiv preprint arXiv:1809.02104 , year =

Are adversarial examples inevitable? , author =. arXiv preprint arXiv:1809.02104 , year =

arXiv
[148]

2023 , isbn =

Shelby, Renee and Rismani, Shalaleh and Henne, Kathryn and Moon, AJung and Rostamzadeh, Negar and Nicholas, Paul and Yilla-Akbari, N'Mah and Gallegos, Jess and Smart, Andrew and Garcia, Emilio and Virk, Gurleen , title =. 2023 , isbn =. doi:10.1145/3600211.3604673 , booktitle =

2023
[149]

2020 , publisher=

Halfaker, Aaron and Geiger, R Stuart , journal=. 2020 , publisher=

2020
[150]

Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society , pages =

Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction , author =. Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society , pages =. 2023 , doi =

2023
[151]

Shen, Hua and Knearem, Tiffany and Ghosh, Reshmi and Alkiek, Kenan and Krishna, Kundan and Liu, Yachuan and Ma, Ziqiao and Petridis, Savvas and Peng, Yi-Hao and Qiwei, Li and others , journal =
[152]

2009 , publisher =

Silbey, Susan S , journal =. 2009 , publisher =

2009
[153]

Sloane, Mona and Zakrzewski, Janina , booktitle =
[154]

TechReg Chronicle , year =
[155]

2022 , publisher =

Sloane, Mona and Moss, Emanuel and Chowdhury, Rumman , journal =. 2022 , publisher =

2022
[156]

Nature Machine Intelligence , volume =

Introducing contextual transparency for automated decision systems , author =. Nature Machine Intelligence , volume =. 2023 , publisher =

2023
[157]

AI & SOCIETY , pages =

Beyond model interpretability: Socio-structural explanations in machine learning , author =. AI & SOCIETY , pages =. 2024 , publisher =

2024
[158]

Proceedings of the ACM on Human-Computer Interaction , volume =

Becoming interdisciplinary: Fostering critical engagement with disaster data , author =. Proceedings of the ACM on Human-Computer Interaction , volume =. 2021 , publisher =

2021
[159]

European review , volume =

Improving ratings: audit in the British University system , author =. European review , volume =. 1997 , publisher =

1997
[160]

Proceedings of the AAAI Conference on Artificial Intelligence , volume =

Energy and policy considerations for modern deep learning research , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =
[161]

2020 , publisher =

The Political Constitution of the Corporation: A Management Studies View , author =. 2020 , publisher =

2020
[162]

Taplin, Jonathan , year =
[163]

Vaughan, Diane , year =
[164]

Proceedings of the 2018 chi conference on human factors in computing systems , pages =

Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making , author =. Proceedings of the 2018 chi conference on human factors in computing systems , pages =

2018
[165]

2021 , publisher =

Veale, Michael and Zuiderveen Borgesius, Frederik , journal =. 2021 , publisher =

2021
[166]

2015 , publisher =

Seeing like a rover: How robots, teams, and images craft knowledge of Mars , author =. 2015 , publisher =

2015
[167]

2020 , publisher =

The innovation delusion: How our obsession with the new has disrupted the work that matters most , author =. 2020 , publisher =

2020
[168]

2021 , publisher =

Industry unbound: The inside story of privacy, data, and corporate power , author =. 2021 , publisher =

2021
[169]

Chernobyl Operators Blamed , journal =

Daniel Weisskopf , year =. Chernobyl Operators Blamed , journal =
[170]

2018 , publisher =

Effective Project Management: Guidance and Checklists for Engineering and Construction , author =. 2018 , publisher =

2018
[171]

2025 , isbn =

Winecoff, Amy and Bogen, Miranda , title =. 2025 , isbn =. doi:10.1145/3706598.3713814 , booktitle =

2025
[172]

Bogen, Miranda and Winecoff, Amy , journal =
[173]

Winner, Langdon , year =. The
[174]

2020 , howpublished =

Skewed Grading Algorithms Fuel Backlash Beyond the Classroom , author =. 2020 , howpublished =

2020
[175]

Business Insider , url =

Wolverton, Troy , year =. Business Insider , url =
[176]

arXiv preprint arXiv:2309.07045 , year =

Safetybench: Evaluating the safety of large language models with multiple choice questions , author =. arXiv preprint arXiv:2309.07045 , year =

arXiv
[177]

2019 , publisher =

The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power , author =. 2019 , publisher =

2019
[178]

and Menn,Joseph , year =

Gilbert,Daniel and Frankel,Todd C. and Menn,Joseph , year =. Silicon Valley Bank’s risk model flashed red. So its executives changed it , journal =
[179]

United States Navy , year = 2025, url =

2025
[180]

Human factors , volume=

Does team training improve team performance? A meta-analysis , author=. Human factors , volume=. 2008 , publisher=

2008
[181]

, author=

The influence of shared mental models on team process and performance. , author=. Journal of applied psychology , volume=. 2000 , publisher=

2000
[182]

Floridi, Luciano and Cowls, Josh , journal =. A. 2019 , month =

2019
[183]

Tackling

Leslie, David , journal =. Tackling. 2020 , month =

2020
[184]

Banks, David and Melo,. Data. Harvard Data Science Review , number =. 2024 , month =. doi:10.1162/99608f92.740dcedb , publisher =

2024 doi
[185]

Participatory

Venkatachalapathy, Rajesh , journal =. Participatory. 2025 , publisher =

2025
[186]

2024 , month =

Leslie, David and Perini, Antonella Maia , journal =. 2024 , month =

2024
[187]

Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =

Septiandri, Ali Akbar and Constantinides, Marios and Tahaei, Mohammad and Quercia, Daniele , title =. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =. 2023 , isbn =. doi:10.1145/3593013.3593985 , abstract =

2023
[188]

Bloomfield, Robin and Rushby, John , journal=. Where. 2025 , note=

2025
[189]

Assurance of

Bloomfield, Robin and Rushby, John , institution=. Assurance of. 2024 , url=

2024
[190]

Measuring What Matters: Connecting

Rismani, Shalaleh and Shelby, Renee and Davis, Leah and Rostamzadeh, Negar and Moon, AJung , booktitle=. Measuring What Matters: Connecting. 2025 , doi=

2025
[191]

R.E Melchers , keywords =. On the. Reliability Engineering & System Safety , volume =. 2001 , issn =. doi:10.1016/S0951-8320(00)00096-X , url =

2001 doi
[192]

Big Data & Society , volume=

Legitimacy practices, algorithms, and the new bureaucratic quantification , author=. Big Data & Society , volume=. doi:10.1177/20539517261442385 , year=

doi
[193]

Power to the People? Opportunities and Challenges for Participatory

Birhane, Abeba and Isaac, William and Prabhakaran, Vinodkumar and Diaz, Mark and Elish, Madeleine Clare and Gabriel, Iason and Mohamed, Shakir , urldate =. Power to the People? Opportunities and Challenges for Participatory. Proceedings of the 2nd. doi:10.1145/3551624.3555290 ...
[194]

Stakeholder Participation for Responsible

Kallina, Emma and Bohné, Thomas and Singh, Jatinder , urldate =. Stakeholder Participation for Responsible. Proceedings of the 2025. doi:10.1145/3715275.3732069 , series =

2025
[195]

Enrolling Citizens: A Primer on Archetypes of Democratic Engagement with

Hu, Wanheng and Singh, Ranjit , urldate =. Enrolling Citizens: A Primer on Archetypes of Democratic Engagement with
[196]

Are We Asking the Right Questions?: Designing for Community Stakeholders’ Interactions with

Haque,. Are We Asking the Right Questions?: Designing for Community Stakeholders’ Interactions with. Proceedings of the 2024. doi:10.1145/3613904.3642738 , series =

2024
[197]

Emerging Practices in Participatory

Saxena, Devansh and Kahn, Zoe and Moon, Erina Seh-Young and Chambers, Lauren Marietta and Jackson, Corey and Lee, Min Kyung and Eslami, Motahhare and Guha, Shion and Erete, Sheena and Irani, Lilly and Mulligan, Deirdre and Zimmerman, John , urldate =. Emerging Practices in Par...
[198]

The Design of Pseudo-Participation , isbn =

Palacin, Victoria and Nelimarkka, Matti and Reynolds-Cuéllar, Pedro and Becker, Christoph , urldate =. The Design of Pseudo-Participation , isbn =. Proceedings of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise - Volume 2 , publisher =. doi:10.1145/3...

2020
[199]

Subcommittee on Consumer Protection, Product Safety, and Data Security of the Committee on Commerce, Science, and Transportation , author =

Protecting Kids Online: Testimony from a Facebook Whistleblower , url =. Subcommittee on Consumer Protection, Product Safety, and Data Security of the Committee on Commerce, Science, and Transportation , author =
[200]

, urldate =

Arnstein, Sherry R. , urldate =. A Ladder Of Citizen Participation , volume =. doi:10.1080/01944366908977225 , abstract =

doi
[201]

ACM Transactions on Information Systems (TOIS) , volume=

Bias in computer systems , author=. ACM Transactions on Information Systems (TOIS) , volume=. 1996 , publisher=

1996
[202]

interactions , volume=

Value-sensitive design , author=. interactions , volume=. 1996 , publisher=

1996
[203]

In SIGCHI Conference on Human Factors in Computing Systems , year =

Steve Harrison and Deborah Tatar and Phoebe Sengers , title =. In SIGCHI Conference on Human Factors in Computing Systems , year =
[204]

Participatory Design: Principles and Practices , year =
[205]

European Journal of Information Systems , volume=

User participation in decision support systems development: influencing system outcomes , author=. European Journal of Information Systems , volume=. 2004 , publisher=

2004
[206]

and Hughes, J

Bentley, R. and Hughes, J. A. and Randall, D. and Rodden, T. and Sawyer, P. and Shapiro, D. and Sommerville, I. , title =. 1992 , isbn =. doi:10.1145/143457.143470 , booktitle =

1992
[207]

Proceedings of the ACM on Human-Computer Interaction , volume=

WeBuildAI: Participatory framework for algorithmic governance , author=. Proceedings of the ACM on Human-Computer Interaction , volume=. 2019 , publisher=

2019
[208]

Proceedings of the ACM on Human-Computer Interaction , volume=

Value-sensitive algorithm design: method, case study, and lessons , author=. Proceedings of the ACM on Human-Computer Interaction , volume=. 2018 , publisher=

2018
[209]

Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems , pages=

Towards algorithmic experience: Initial efforts for social media contexts , author=. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems , pages=. 2018 , organization=

2018
[210]

Proceedings of the ACM on Human-Computer Interaction , volume=

The principles and limits of algorithm-in-the-loop decision making , author=. Proceedings of the ACM on Human-Computer Interaction , volume=. 2019 , publisher=

2019
[211]

Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing , pages=

The Changing Contours of Participation in Data-driven, Algorithmic Ecosystems: Challenges, Tactics, and an Agenda , author=. Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing , pages=. 2018 , organization=

2018
[212]

Explainer: A sociotechnical approach to

Chen, Brian J and Metcalf, Jacob , journal=. Explainer: A sociotechnical approach to. 2024 , URL=

2024
[213]

Voluntary aviation safety information-sharing process: Preliminary audit of distributed

Chidester, Thomas R , institution=. Voluntary aviation safety information-sharing process: Preliminary audit of distributed. 2007 , URL=

2007
[214]

Reconciling trade secrets and

Keller, Perry and Aplin, Tanya , year=. Reconciling trade secrets and. doi:10.1017/S0008197325101086 , journal=

doi
[215]

Annals Health L

Physicians and safe harbor legal immunity , author=. Annals Health L. , volume=. 2012 , publisher=

2012
[216]

Safety monitoring in the vaccine adverse event reporting system

Shimabukuro, Tom T and Nguyen, Michael and Martin, David and DeStefano, Frank , journal=. Safety monitoring in the vaccine adverse event reporting system. 2015 , publisher=

2015
[217]

Lessons from complex systems science for

Kolt, Noam and Shur-Ofry, Michal and Cohen, Reuven , journal=. Lessons from complex systems science for. 2025 , publisher=

2025
[218]

Minds and Machines , volume=

A sociotechnical system perspective on AI , author=. Minds and Machines , volume=. 2024 , publisher=

2024
[219]

, title =

Boysen, Philip G. , title =. 2013 , publisher =. http://www.ochsnerjournal.org/content/13/3/400.full.pdf , journal =

2013
[220]

The 35th European Safety and Reliability Conference (ESREL2025) and the 33rd Society for Risk Analysis Europe Conference (SRA-E 2025) , year=

From rigid orderliness to barely controlled chaos: sociotechnical risk and AI in aviation , author=. The 35th European Safety and Reliability Conference (ESREL2025) and the 33rd Society for Risk Analysis Europe Conference (SRA-E 2025) , year=

2025
[221]

Patterns , volume=

A normative framework for artificial intelligence as a sociotechnical system in healthcare , author=. Patterns , volume=. 2023 , publisher=

2023
[222]

Nuclear Technology , volume=

A sociotechnical readiness level framework for the development of advanced nuclear technologies , author=. Nuclear Technology , volume=. 2024 , publisher=

2024
[223]

2020 , organization=

Andrus, McKane and Dean, Sarah and Gilbert, Thomas Krendl and Lambert, Nathan and Zick, Tom , booktitle=. 2020 , organization=

2020
[224]

What does explainable

Doran, Derek and Schulz, Sarah and Besold, Tarek R , journal=. What does explainable. 2017 , doi=

2017
[225]

Artificial intelligence , volume=

Explanation in artificial intelligence: Insights from the social sciences , author=. Artificial intelligence , volume=. 2019 , publisher=

2019
[226]

Proceedings of the 58th annual meeting of the association for computational linguistics , doi =

Language (technology) is power: A critical survey of ``bias''' in NLP , author=. Proceedings of the 58th annual meeting of the association for computational linguistics , doi =
[227]

Annual review of statistics and its application , volume=

Algorithmic fairness: Choices, assumptions, and definitions , author=. Annual review of statistics and its application , volume=. 2021 , publisher=

2021
[228]

Proceedings of the 2019 CHI Conference on Human Factors in computing Systems , pages=

Improving fairness in machine learning systems: What do industry practitioners need? , author=. Proceedings of the 2019 CHI Conference on Human Factors in computing Systems , pages=. 2019 , doi =

2019
[229]

Co-designing checklists to understand organizational challenges and opportunities around fairness in

Madaio, Michael A and Stark, Luke and Wortman Vaughan, Jennifer and Wallach, Hanna , booktitle=. Co-designing checklists to understand organizational challenges and opportunities around fairness in. 2020 , doi=

2020
[230]

What does it mean to be a responsible

Rismani, Shalaleh and Moon, AJung , booktitle=. What does it mean to be a responsible
[231]

Where responsible

Rakova, Bogdana and Yang, Jingying and Cramer, Henriette and Chowdhury, Rumman , journal=. Where responsible. 2021 , publisher=

2021
[232]

Operationalising

Morley, Jessica and Kinsey, Libby and Elhalal, Anat and Garcia, Francesca and Ziosi, Marta and Floridi, Luciano , journal=. Operationalising. 2023 , publisher=

2023
[233]

IEEE Cloud Computing , volume=

Emergent failures: Rethinking cloud reliability at scale , author=. IEEE Cloud Computing , volume=. 2018 , publisher=

2018
[234]

no blame

Balancing “no blame” with accountability in patient safety , author=. N Engl J Med , volume=. 2009 , doi =

2009
[235]

European Journal of Social Psychology , volume =

Nemeth, Charlan and Brown, Keith and Rogers, John , title =. European Journal of Social Psychology , volume =. doi:10.1002/ejsp.58 , url =. https://onlinelibrary.wiley.com/doi/pdf/10.1002/ejsp.58 , year =

doi
[236]

Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , issn =

Buolamwini, Joy and Gebru, Timnit , urldate =. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , issn =. Proceedings of the 1st Conference on Fairness, Accountability and Transparency , publisher =
[237]

Performing a Project Premortem , issn =

Klein, Gary , date =. Performing a Project Premortem , issn =
[238]

Power, Safety, and Learning in Racially Diverse Groups , volume =

Foldy, Erica and Rivard, Peter and Buckley, Tamara , doi =. Power, Safety, and Learning in Racially Diverse Groups , volume =
[239]

Does Diversity Pay?: Race, Gender, and the Business Case for Diversity , volume =

Herring, Cedric , urldate =. Does Diversity Pay?: Race, Gender, and the Business Case for Diversity , volume =. doi:10.1177/000312240907400203 , shorttitle =

doi
[240]

, urldate =

Hong, Lu and Page, Scott E. , urldate =. Groups of diverse problem solvers can outperform groups of high-ability problem solvers , volume =. doi:10.1073/pnas.0403723101 , abstract =

doi
[241]

, editoratype =

Turner, Barry A. , editoratype =. Man-Made Disasters , isbn =
[242]

IEEE Softw

Basiri, Ali and Behnam, Niosha and de Rooij, Ruud and Hochstein, Lorin and Kosewski, Luke and Reynolds, Justin and Rosenthal, Casey , title =. IEEE Softw. , month = may, pages =. 2016 , issue_date =. doi:10.1109/MS.2016.60 , abstract =

2016 doi
[243]

Principle 1: Preoccupation with Failure , booktitle =

Weick, Karl and Sutcliffe, Kathleen , publisher =. Principle 1: Preoccupation with Failure , booktitle =. 2015 , keywords =. doi:10.1002/9781119175834.ch03 , eprint =

2015 doi
[244]

The Fearless Organization: Creating Psychological Safety in the Workplace for Learning, Innovation, and Growth , isbn =

Edmondson, Amy , address =. The Fearless Organization: Creating Psychological Safety in the Workplace for Learning, Innovation, and Growth , isbn =
[245]

Psychological Safety and Learning Behavior in Work Teams , volume =

Edmondson, Amy , urldate =. Psychological Safety and Learning Behavior in Work Teams , volume =. doi:10.2307/2666999 , pages =

doi
[246]

Stuart and Varoquaux, Nelle and Mazel-Cabasse, Charlotte and Holdgraf, Chris , title =

Geiger, R. Stuart and Varoquaux, Nelle and Mazel-Cabasse, Charlotte and Holdgraf, Chris , title =. 2018 , issue_date =. doi:10.1007/s10606-018-9333-1 , journal =

2018 doi
[247]

Superintelligence: Paths, Dangers, Strategies , isbn =

Bostrom, Nick , date =. Superintelligence: Paths, Dangers, Strategies , isbn =
[248]

Human Compatible: Artificial Intelligence and the Problem of Control , isbn =

Russell, Stuart , date =. Human Compatible: Artificial Intelligence and the Problem of Control , isbn =
[249]

Artificial Intelligence as a positive and negative factor in global risk , isbn =

Yudkowsky, Eliezer , editor =. Artificial Intelligence as a positive and negative factor in global risk , isbn =. doi:10.1093/oso/9780198570509.003.0021 , booktitle =

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

UCLA Law Review , volume=

The Public Harms of Private Surveillance , author=. UCLA Law Review , volume=. 2025 , URL=

2025

[2] [2]

First Monday , doi =

Ahmed, Shazeda and Ja. First Monday , doi =

[3] [3]

Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =

Ali, Sanna J and Christin, Ang. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =

2023

[4] [4]

Communications of the ACM , volume =

Computational complexity and information asymmetry in financial products , author =. Communications of the ACM , volume =. 2011 , publisher =

2011

[5] [5]

Responsive

Ayres, Ian and Braithwaite, John , year =. Responsive

[6] [6]

Analysis, Design and Evaluation of Man--Machine Systems , pages =

Ironies of automation , author =. Analysis, Design and Evaluation of Man--Machine Systems , pages =. 1983 , publisher =

1983

[7] [7]

, urldate =

Gebru, Timnit and Torres, Émile P. , urldate =. The. doi:10.5210/fm.v29i4.13636 , shorttitle =

doi:10.5210/fm.v29i4.13636

[8] [8]

2020 , publisher =

Can artificial intelligence transform higher education? , author =. 2020 , publisher =

2020