Pith. sign in

REVIEW 4 major objections 5 minor 44 references

From Hazard Identification to Controller Design: Proactive and LLM-Supported Safety Engineering for ML-Powered Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that routine hazard analysis, assisted by large language models, can anticipate a wide range of harms in ML-powered applications and guide the design of system-level controllers before release.

desk verdict A useful, honest position paper that makes LLM-supported STPA concrete for everyday ML apps, but the 'demonstrates' wording outstrips the single self-reported example behind it. read the letter →

arxiv 2502.07974 v1 pith:HEQ73ZSY submitted 2025-02-11 cs.SE cs.AIcs.LG

Yining Hong , Christopher S. Timperley , Christian Kästner This is my paper · ORCID
classification cs.SEcs.AIcs.LG
keywords hazardanalysisSTPAlargelanguagemodelsML-poweredsystemscontrollerdesignresponsibleengineeringproactiveriskidentificationsafety
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper claims that the harms of ML-powered software are largely anticipatable if developers follow a systematic hazard-analysis process, and that large language models can make that process cheap enough and accessible enough to use routinely. The authors walk a trail-recommendation chatbot through the System-Theoretic Process Analysis (STPA) and show it surfaces harms beyond model errors, such as ecological damage, consent gaps, and monitoring failures. They further report that LLMs, with human oversight, can generate stakeholder lists, losses, hazards, and controller designs at low cost and in minutes. A sympathetic reader would take away that 'unintended consequences' are often preventable through structured, sociotechnical thinking rather than model-focused fixes alone.

What carries the argument

The carrying mechanism is the four-step STPA workflow: define the purpose by listing stakeholders, values, losses, hazards, and constraints; model the control structure of controllers and controlled processes; identify unsafe control actions by checking each control action against six failure modes (absent, incorrect, mistimed, wrong sequence, prolonged, prematurely stopped); and identify loss scenarios that explain why unsafe actions occur. Within this scaffold, LLMs act as breadth generators, producing candidate stakeholders, losses, hazards, controllers, and unsafe-action checklists that developers then filter and judge. The process works backward from harms to constraints to controllers, which is what lets it suggest system-level mitigations such as dashboards, alert procedures, and consent interfaces rather than only model-level patches.

What would settle it

Run the same LLM-supported STPA procedure on several real ML-powered applications and have expert safety engineers independently perform a full STPA on the same systems, then compare the hazard sets. If a substantial fraction of expert-identified hazards never appear in the LLM-assisted output for any application, or if developers misclassify severe hazards as minor, the claim that LLM-supported analysis is effective for routine use would be falsified.

Watch

Extended reading notes

Core claim

The central discovery is that applying STPA — a safety framework normally reserved for aviation and other safety-critical domains — to an ordinary ML-powered application yields a large inventory of concrete, mitigable hazards and corresponding controller designs. On a simple customized GPT that recommends hiking trails, the process identified twenty stakeholders, 145 losses, and over a thousand hazards, which were merged into fifty distinct system-level hazards including recommending trails in ecologically sensitive areas and lacking a consent-withdrawal mechanism. For each hazard, LLM-assisted brainstorming produced ten to fifteen possible controller designs, leading to a redesigned control structure with feedback loops and hierarchical monitoring. The authors' key empirical claim is that LLMs can substantially reduce the effort of STPA — generating candidate content in minutes for under two dollars — while developers still make the final judgments about relevance and severity.

Load-bearing premise

The claim rests on the assumption that LLM-generated candidate lists, once filtered by developers, are complete and accurate enough that the process does not miss more critical hazards than it finds, and that developers without safety-engineering training can judge which candidates are relevant and severe.

Editorial extensions

If this is right

  • If adopted routinely, ML-powered applications would ship with system-level controllers — dashboards, alert procedures, consent interfaces — rather than relying only on model fixes.
  • The cost and expertise barriers that keep hazard analysis out of agile workflows shrink, since LLM support can produce candidate material in minutes for under a few dollars.
  • Developers without safety-engineering training could conduct a first-pass analysis, letting scarce safety experts focus on review and high-risk decisions.
  • The same process extends to non-safety harms like fairness, usability, and ecological impact, broadening what counts as responsible engineering.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be a benchmark comparing LLM-assisted STPA against expert-only STPA on the same system to measure recall of critical hazards.
  • If LLM-supported STPA becomes routine, the bottleneck may shift from generating candidates to judging them; tools that help developers prioritize and triage hundreds of hazards would be the natural next step.
  • The running example suggests that a catalog of common system-level controllers could be embedded in the process, making the LLM prompts less necessary and the results more reproducible.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This position paper argues that system-theoretic hazard analysis, specifically STPA, should be routinely applied to ML-powered software products, and that LLMs can make the process accessible to developers without extensive safety-engineering expertise. The authors present a running example—an LLM-based trail recommendation assistant—walking through the STPA steps: identifying 20 stakeholders, 145 losses, and 1,159 hazards merged into 50 system-level hazards, then designing controllers for three selected hazards. They report that LLMs helped generate broad candidate lists and reduced effort, but acknowledge in Section V that the process can become unmanageable and that developers may perform superficial analyses. The paper advocates for a lightweight, human-oversight-in-the-loop LLM-supported STPA process as a routine step in responsible engineering.

Significance. The paper addresses an important and timely problem: ML-powered applications are often released without proactive hazard analysis, and existing safety-engineering methods are expert-heavy and time-consuming. If the claims were substantiated, the proposed approach could broaden the adoption of STPA beyond safety-critical domains and expand the range of harms considered (e.g., fairness, usability, consent, ecological impact). The paper is honest about its scope as a position paper and includes a concrete, reproducible running example with supplementary materials. It also candidly discusses threats to validity in Section V, including the risk of over-reliance on LLMs and superficial analysis by non-experts. The main weakness is that the evidence is a single uncontrolled case study: the counts show throughput, not completeness or quality, and no baseline or independent evaluation is provided.

major comments (4)
  1. [Section II and Section IV-A] The paper states in Section II that 'This paper demonstrates that hazard analysis is effective in anticipating a wide range of hazards' and 'shows that LLMs are promising in supporting humans conducting STPA by reducing effort.' The evidence provided is a single running example with no human-only baseline, no expert STPA comparison, no gold-standard set of known hazards, and no inter-rater reliability. The reported numbers (20 stakeholders, 145 losses, 1,159 hazards, 50 merged hazards) measure throughput, not whether critical hazards were missed. To support 'demonstrates,' the paper needs a comparative evaluation or a clearly specified set of testable hypotheses with a falsifiability criterion; otherwise, these claims should be reframed as an illustrative feasibility study. The paper's own Section V concession that 'there is a risk that these developers may perform a more superficial analysis' reinforces that the current evidence does not establish effectiveness at the claimed level.
  2. [Section IV-B] The selection of three hazards (H4, H39, H48) for controller design is described as 'Having assessed the risks and costs,' but the criteria and the assessor are not specified. This selection materially affects the demonstration: if the chosen hazards are those with the most obvious or cheapest mitigations, the example may overstate the general usefulness of the approach. The authors should either provide a transparent scoring rubric (e.g., likelihood, severity, cost) or explicitly state that the selection was a subjective, illustrative choice and discuss how selection bias could affect the conclusions.
  3. [Section IV-A and Section V] The claim that LLMs reduce effort is not operationalized. Section IV-A mentions that analyzing all losses took 'a few minutes for under USD 2' for one step, but no time or cost data are reported for the full process, and no comparison is made to an unaided analysis. Section V admits that the analysis 'can easily grow to include even more potential controllers and associated issues' and 'can feel like a bureaucratic paper-heavy compliance activity.' Without measuring total effort or comparing to a non-LLM baseline, the 'reducing effort' claim is not supported. The paper should report effort data for the complete walkthrough or temper the claim to a qualitative observation.
  4. [Section IV-C and Section V] The evaluation is self-referential: the authors use LLM outputs to generate candidate hazards and controllers, then draw conclusions about LLM usefulness from that same process. While this is not a formal circularity, it leaves open the question of whether the approach would work for the target population—developers without safety-engineering training—since the authors themselves are experienced researchers and are able to filter, merge, and judge severity. The paper explicitly flags in Section V that 'developers may perform a more superficial analysis,' but Section II's broad claim about making STPA accessible should be conditioned on this unresolved risk. A user study with the intended audience is needed before the accessibility claim can be sustained.
minor comments (5)
  1. [Abstract and Section II] The abstract says 'demonstrating that many seemingly unanticipated issues can, in fact, be anticipated,' while Section II uses 'demonstrates' and 'shows.' Given the position-paper framing and the single-case evidence, 'illustrates' and 'suggests' would be more accurate and would preempt the appearance of overclaiming.
  2. [Section II, paragraph 3] There is a typo: 'brainstroming' should be 'brainstorming'.
  3. [Section IV-A, footnote 1] The supplementary material is hosted on a Google Spreadsheet with a shared link; this is not a permanent archival venue. The authors should consider depositing the prompts and outputs in a versioned repository (e.g., Zenodo or a GitHub release) to support reproducibility.
  4. [Figure 2 caption] The caption says 'The bold arrow represents the control action analyzed in Section IV-C,' but the figure is somewhat hard to read in monochrome; using a distinct line style or color and enlarging labels would improve clarity.
  5. [Section III, Table I] The mapping table is useful, but the term 'allocation harms' is not defined before use; a brief parenthetical explanation would help readers unfamiliar with ML harm taxonomies.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are empirical and positional, and the only self-citation is not load-bearing.

full rationale

This is a position paper with an illustrative running example rather than a derivation chain, so the circularity patterns that require equations or fitted parameters do not apply. The paper's claims that hazard analysis can anticipate harms and that LLMs can assist are supported by a walkthrough in which the authors use GPT-4o to generate stakeholders, losses, hazards, and controller designs and then apply human judgment to merge and select (Section IV). There is no quantity that is fitted to data and then predicted, no definition that presupposes the conclusion, and no invocation of the authors' own prior theorems to force a choice. The one self-citation (Kästner [39]) is used only to support the uncontroversial statement that system-level mitigations receive less attention, and it is not load-bearing for the paper's main argument. The absence of a baseline or gold standard is a real threat to the strength of the word 'demonstrates,' but that is an empirical validity concern, not circularity. Accordingly, no circular step is identified and the score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities. The argument rests on domain assumptions about the transferability of STPA to everyday ML apps and about the quality of LLM output under human oversight. The paper openly labels these as open questions in Section V, which is honest but leaves the central position conditional.

assumptions (3)
  • domain assumption STPA, with broadened loss categories, transfers effectively to non-safety-critical ML-powered applications.
    Invoked throughout Section IV and the Introduction; the paper's demonstration relies on this transfer rather than proving it.
  • domain assumption LLM-generated stakeholders, losses, hazards, and controller designs are useful raw material when a human filters them.
    Sections IV-A, IV-B, and IV-C rely on GPT-4o output quality; no systematic evaluation or baseline is provided.
  • domain assumption Developers can provide adequate human oversight and judgment without deep safety-engineering expertise.
    Section V discusses the risk of superficial analysis; the proposed process depends on this oversight being sufficient.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Hazard Identification to Controller Design: Proactive and LLM-Supported Safety Engineering for ML-Powered Systems." pith.science (2026). https://pith.science/paper/HEQ73ZSY

@misc{pith2026250207974,
  author       = {Pith},
  title        = {Pith review of: From Hazard Identification to Controller Design: Proactive and LLM-Supported Safety Engineering for ML-Powered Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HEQ73ZSY}},
  note         = {Machine review of arXiv:2502.07974}
}
read the original abstract

Machine learning (ML) components are increasingly integrated into software products, yet their complexity and inherent uncertainty often lead to unintended and hazardous consequences, both for individuals and society at large. Despite these risks, practitioners seldom adopt proactive approaches to anticipate and mitigate hazards before they occur. Traditional safety engineering approaches, such as Failure Mode and Effects Analysis (FMEA) and System Theoretic Process Analysis (STPA), offer systematic frameworks for early risk identification but are rarely adopted. This position paper advocates for integrating hazard analysis into the development of any ML-powered software product and calls for greater support to make this process accessible to developers. By using large language models (LLMs) to partially automate a modified STPA process with human oversight at critical steps, we expect to address two key challenges: the heavy dependency on highly experienced safety engineering experts, and the time-consuming, labor-intensive nature of traditional hazard analysis, which often impedes its integration into real-world development workflows. We illustrate our approach with a running example, demonstrating that many seemingly unanticipated issues can, in fact, be anticipated.

Figures

Figures reproduced from arXiv: 2502.07974 by the authors.

Figure 1
Figure 1. An overview of the STPA method, based on the textual description and figures in the STPA handbook [18]. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The control structure of the system: The bold arrow represents the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 31 canonical work pages

  1. [1]

    A framework for understanding sources of harm throughout the machine learning life cycle,

    H. Suresh and J. Guttag, “A framework for understanding sources of harm throughout the machine learning life cycle,” in Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, 2021, pp. 1–9

    work page 2021

  2. [2]

    When machine unlearning jeopardizes privacy,

    M. Chen, Z. Zhang, T. Wang, M. Backes, M. Humbert, and Y . Zhang, “When machine unlearning jeopardizes privacy,” in Proceedings of the 2021 ACM SIGSAC conference on computer and communications security, 2021, pp. 896–911

    2021

  3. [3]

    Bias and unfairness in machine learning models: a systematic literature review

    T. P. Pagano, R. B. Loureiro, F. V . N. Lisboa, G. O. R. Cruz, R. M. Peixoto, G. A. d. S. Guimar ˜aes, L. L. d. Santos, M. M. Araujo, M. Cruz, E. L. S. de Oliveira et al., “Bias and unfairness in machine learning mod- els: a systematic literature review,” arXiv preprint arXiv:2202.08176 , 2022

    work page Pith review arXiv 2022

  4. [4]

    A survey on bias and fairness in machine learning,

    N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM computing surveys (CSUR), vol. 54, no. 6, pp. 1–35, 2021

    2021

  5. [5]

    Aha!: Facilitating ai impact assessment by generating examples of harms,

    Z. Buc ¸inca, C. M. Pham, M. Jakesch, M. T. Ribeiro, A. Olteanu, and S. Amershi, “Aha!: Facilitating ai impact assessment by generating examples of harms,” arXiv preprint arXiv:2306.03280 , 2023

    arXiv 2023

  6. [6]

    Farsight: Fostering responsible ai awareness during ai application prototyping,

    Z. J. Wang, C. Kulkarni, L. Wilcox, M. Terry, and M. Madaio, “Farsight: Fostering responsible ai awareness during ai application prototyping,” in Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024, pp. 1–40

    work page 2024

  7. [7]

    Responsible AI impact assessment template,

    Microsoft, “Responsible AI impact assessment template,” 2022, accessed: 2024-11-07. [Online]. Available: https://blogs.microsoft.com/ wp-content/uploads/prod/sites/5/2022/06/Microsoft-RAI-Impact-Asses sment-Template.pdf

    work page 2022

  8. [8]

    System Safety Engineering for Social and Ethical ML Risks: A Case Study

    E. W. Jatho III, L. O. Mailloux, S. Rismani, E. D. Williams, and J. A. Kroll, “System safety engineering for social and ethical ml risks: A case study,” arXiv preprint arXiv:2211.04602 , 2022

    work page Pith review arXiv 2022

Show all 44 references
  1. [9]

    A hazard analysis framework for code synthesis large language models,

    H. Khlaaf, P. Mishkin, J. Achiam, G. Krueger, and M. Brundage, “A hazard analysis framework for code synthesis large language models,” arXiv preprint arXiv:2207.14157 , 2022

    2022 arXiv

  2. [10]

    System safety and artificial intelligence,

    R. Dobbe, “System safety and artificial intelligence,” in 2022 ACM Conference on Fairness, Accountability, and Transparency , ser. FAccT ’22. ACM, Jun. 2022. [Online]. Available: http: //dx.doi.org/10.1145/3531146.3533215

    2022

  3. [11]

    Exploring opportunities in usable hazard analysis processes for ai engineering,

    N. Martelaro, C. J. Smith, and T. Zilovic, “Exploring opportunities in usable hazard analysis processes for ai engineering,” 2022

    2022

  4. [12]

    Beyond the ml model: Applying safety engineering frameworks to text-to-image development,

    S. Rismani, R. Shelby, A. Smart, R. Delos Santos, A. Moon, and N. Rostamzadeh, “Beyond the ml model: Applying safety engineering frameworks to text-to-image development,” in Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society , 2023, pp. 70–83

    2023

  5. [13]

    From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ml,

    S. Rismani, R. Shelby, A. Smart, E. Jatho, J. Kroll, A. Moon, and N. Rostamzadeh, “From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ml,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , 2023...

    2023

  6. [14]

    From silos to systems: Process-oriented hazard analysis for ai systems,

    S. Rismani, R. Dobbe, and A. Moon, “From silos to systems: Process-oriented hazard analysis for ai systems,” arXiv preprint arXiv:2410.22526, 2024

    2024 arXiv

  7. [15]

    How do ai systems fail socially?: an engi- neering risk analysis approach,

    S. Rismani and A. Moon, “How do ai systems fail socially?: an engi- neering risk analysis approach,” in 2021 IEEE International Symposium on Ethics in Engineering, Science and Technology (ETHICS) . IEEE, 2021, pp. 1–8

    2021

  8. [16]

    Safety engineering for autonomous vehicles,

    R. Adler, P. Feth, and D. Schneider, “Safety engineering for autonomous vehicles,” in 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshop (DSN-W) . IEEE, 2016, pp. 200–205

    2016

  9. [17]

    Using stpa in compliance with iso 26262 for develop- ing a safe architecture for fully automated vehicles,

    A. Abdulkhaleq, S. Wagner, D. Lammering, H. Boehmert, and P. Blueher, “Using stpa in compliance with iso 26262 for develop- ing a safe architecture for fully automated vehicles,” arXiv preprint arXiv:1703.03657, 2017

    2017 arXiv

  10. [18]

    and Thomas, John P., STPA handbook, MIT Part- nership for Systems Approaches to Safety and Security (PSASS), Cambridge, Massachusetts, U.S., 2018

    Leveson, Nancy G. and Thomas, John P., STPA handbook, MIT Part- nership for Systems Approaches to Safety and Security (PSASS), Cambridge, Massachusetts, U.S., 2018

    2018

  11. [19]

    On the dangers of stochastic parrots: Can language models be too big?

    E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the dangers of stochastic parrots: Can language models be too big?” in Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, 2021, pp. 610–623

    2021

  12. [20]

    Ethical and social risks of harm from language models,

    L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh et al., “Ethical and social risks of harm from language models,” arXiv preprint arXiv:2112.04359, 2021

    2021 arXiv

  13. [21]

    Learning fair representations,

    R. Zemel, Y . Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning fair representations,” in International conference on machine learning . PMLR, 2013, pp. 325–333

    2013

  14. [22]

    Privacy-preserving deep learning,

    R. Shokri and V . Shmatikov, “Privacy-preserving deep learning,” in Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, 2015, pp. 1310–1321

    2015

  15. [23]

    Responsible ai tools and practices,

    Microsoft, “Responsible ai tools and practices,” 2024, accessed: 2024-11-09. [Online]. Available: https://www.microsoft.com/en-us/ai/t ools-practices

    2024

  16. [24]

    Anticipating impacts: Using large-scale scenario writing to explore diverse implications of generative ai in the news environment,

    K. Kieslich, N. Diakopoulos, and N. Helberger, “Anticipating impacts: Using large-scale scenario writing to explore diverse implications of generative ai in the news environment,” 2023

    2023

  17. [25]

    Co- designing an ai impact assessment report template with ai practitioners and ai compliance experts,

    E. Bogucka, M. Constantinides, S. ˇS´cepanovi´c, and D. Quercia, “Co- designing an ai impact assessment report template with ai practitioners and ai compliance experts,” inProceedings of the AAAI/ACM Conference on AI, Ethics, and Society , vol. 7, 2024, pp. 168–180

    2024

  18. [26]

    N. J. Bahr, System Safety Engineering and Risk Assessment: A Practical Approach, 2nd ed. Boca Raton, FL: CRC Press, 2014

    2014

  19. [27]

    A new accident model for engineering safer systems,

    N. Leveson, “A new accident model for engineering safer systems,” Safety science, vol. 42, no. 4, pp. 237–270, 2004

    2004

  20. [28]

    Guidelines and methods for conducting the safety assessment process on civil airborne systems and equipment,

    S. International, “Guidelines and methods for conducting the safety assessment process on civil airborne systems and equipment,” SAE International, Tech. Rep. ARP4761A, December 2023. [Online]. Available: https://www.sae.org/standards/content/arp4761a/

    2023

  21. [29]

    N. G. Leveson, Safeware: system safety and computers . New York, NY , USA: Association for Computing Machinery, 1995

    1995

  22. [30]

    N. R. Storey, Safety Critical Computer Systems. USA: Addison-Wesley Longman Publishing Co., Inc., 1996

    1996

  23. [31]

    N. G. Leveson, Engineering a Safer World: Systems Thinking Applied to Safety. The MIT Press, 01 2012

    2012

  24. [32]

    [Online]

    OpenAI, “Gpt-4,” 2023, accessed: 2024-11-04. [Online]. Available: https://openai.com/research/gpt-4

    2023

  25. [33]

    Chatgpt assistant: Alltrails,

    ——, “Chatgpt assistant: Alltrails,” https://chatgpt.com/g/g-KpF6lTka3 -alltrails, accessed: 2024-10-25

    2024

  26. [34]

    Who validates the validators? aligning llm-assisted evaluation of llm outputs with human preferences,

    S. Shankar, J. Zamfirescu-Pereira, B. Hartmann, A. Parameswaran, and I. Arawjo, “Who validates the validators? aligning llm-assisted evaluation of llm outputs with human preferences,” in Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, 20...

    2024

  27. [35]

    Howard and D

    M. Howard and D. LeBlanc, Writing Secure Code, 2nd ed. Redmond, W A: Microsoft Press, 2003

    2003

  28. [36]

    Howard and S

    M. Howard and S. Lipner, The Security Development Lifecycle: SDL, a Process for Developing Demonstrably More Secure Software . Red- mond, W A: Microsoft Press, 2006

    2006

  29. [37]

    Automated machine learning (automl): an overview of opportunities for application and research,

    V . K. Singh and K. Joshi, “Automated machine learning (automl): an overview of opportunities for application and research,” Journal of Information Technology Case and Application Research , vol. 24, no. 2, pp. 75–85, 2022

    2022

  30. [38]

    Trust in automl: exploring information needs for es- tablishing trust in automated machine learning systems,

    J. Drozdal, J. Weisz, D. Wang, G. Dass, B. Yao, C. Zhao, M. Muller, L. Ju, and H. Su, “Trust in automl: exploring information needs for es- tablishing trust in automated machine learning systems,” in Proceedings of the 25th international conference on intelligent user interfac...

    2020

  31. [39]

    K ¨astner, Machine Learning in Production: From Models to Products

    C. K ¨astner, Machine Learning in Production: From Models to Products. Cambridge, MA: The MIT Press, 2025

    2025

  32. [40]

    Adopting devops in the real world: A theory, a model, and a case study,

    W. P. Luz, G. Pinto, and R. Bonif ´acio, “Adopting devops in the real world: A theory, a model, and a case study,” Journal of Systems and Software, vol. 157, p. 110384, 2019

    2019

  33. [41]

    K. E. Wiegers, Creating a software engineering culture . Pearson Education, 1996

    1996

  34. [42]

    Where responsible ai meets reality: Practitioner perspectives on enablers for shifting or- ganizational practices,

    B. Rakova, J. Yang, H. Cramer, and R. Chowdhury, “Where responsible ai meets reality: Practitioner perspectives on enablers for shifting or- ganizational practices,” Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW1, pp. 1–23, 2021

    2021

  35. [43]

    The sociological imagination and social responsibility,

    R. J. Hironimus-Wendt and L. E. Wallace, “The sociological imagination and social responsibility,” Teaching Sociology, vol. 37, no. 1, pp. 76–88, 2009

    2009

  36. [44]

    E. H. Schein, Organizational culture and leadership . John Wiley & Sons, 2010, vol. 2

    2010

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.