pith. sign in

arxiv: 2605.16281 · v1 · pith:YVGPG6MCnew · submitted 2026-04-10 · 💻 cs.CY · cs.AI

From Reactive to Proactive: A Multi-Regulatory Empirical Analysis of 480 AI Incidents and a Data-Driven Governance Compliance Framework

Ummara Mumtaz , Summaya Mumtaz This is my paper

Pith reviewed 2026-05-21 09:11 UTC · model grok-4.3

classification 💻 cs.CY cs.AI
keywords AI governanceincident databaseEU AI ActNIST AI RMFGDPRproactive frameworkpost-deployment accountabilitycompliance lifecycle
0
0 comments X

The pith

Analysis of 480 AI incidents uncovers major gaps in post-deployment accountability under EU AI Act, NIST, and GDPR, prompting a new proactive governance framework.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines 480 real-world AI incidents to see how they align with post-deployment rules in three key frameworks. It finds that these frameworks have significant shortcomings in holding systems accountable after they are released. From this analysis, the authors develop the Proactive AI Governance Compliance Framework, or PAGCF, which uses a four-phase process to check compliance earlier in the AI lifecycle. This approach includes different levels of oversight based on risk and uses internal monitoring to gauge readiness. Readers interested in AI safety would care because it points to concrete ways current rules fall short and suggests a structured method to improve them before problems occur.

Core claim

The results reveal substantial governance gaps across these frameworks, indicating persistent weaknesses in post-deployment accountability. The PAGCF is a four-phase lifecycle methodology designed to shift governance from reactive incident response toward pre-deployment compliance assurance, including risk-stratified governance tiers and internal monitoring as a proxy for proactive governance capacity.

What carries the argument

The Proactive AI Governance Compliance Framework (PAGCF), a four-phase lifecycle methodology that shifts from reactive responses to pre-deployment compliance checks using risk-stratified tiers and internal monitoring.

If this is right

  • Substantial gaps exist in post-deployment accountability for AI systems under current frameworks.
  • Adopting the PAGCF can help organizations move to pre-deployment compliance assurance.
  • Risk-stratified governance tiers allow for tailored approaches based on system risk levels.
  • An implementation checklist links directly to specific regulatory provisions in the EU AI Act, NIST, and GDPR.
  • Projected impact analysis indicates that internal monitoring can serve as a measure of proactive governance capacity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the framework is adopted, it may reduce the frequency of high-stakes AI incidents by catching issues earlier.
  • Future work could test the PAGCF in actual organizational settings to measure its effectiveness.
  • This analysis highlights the need for better data collection on AI incidents to improve representativeness.
  • Connections to other regulatory efforts could lead to more unified global AI governance standards.

Load-bearing premise

The assumption that the 480 incidents are representative of high-stakes AI deployments and that alignment with regulatory articles can be judged reliably without detailed coding criteria, plus that internal monitoring is a valid proxy for proactive capacity.

What would settle it

Re-evaluating the 480 incidents with explicit coding criteria or analyzing a larger, more recent set of incidents to check if the governance gaps persist or change in magnitude.

Figures

Figures reproduced from arXiv: 2605.16281 by Summaya Mumtaz, Ummara Mumtaz.

Figure 1
Figure 1. Figure 1: summarizes compliance patterns across the three governance frameworks based on the full sample of 480 AI incidents. Overall, the findings indicate substantial governance gaps, although the extent and form of these gaps differ across frameworks [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Non-compliance rate by regulatory provision. Bar color indicates framework; sample sizes vary by provision applicability. Several important patterns emerge from the provision-level analysis. Post-market monitoring under EU AI Act Article 72 shows the highest non-compliance rate, at 77.1%, indicating that ongoing oversight of deployed AI systems is the most neglected governance obligation across the framewo… view at source ↗
Figure 3
Figure 3. Figure 3: Non-compliance heatmap by risk category and framework. Darker colors indicate higher non-compliance rates. Discrimination and bias, as well as privacy and security incidents, which are among the categories most directly addressed by existing governance frameworks, show distinct patterns of compliance. Safety and failure incidents, the largest category in the dataset, exhibit particularly high non-complianc… view at source ↗
read the original abstract

Artificial intelligence systems are increasingly deployed in high-stakes domains, yet it remains unclear whether existing governance frameworks ensure accountability after deployment. This study makes two contributions. First, it presents a cross-regulatory empirical analysis of 480 real-world AI incidents from the AI Incident Database (AIID), evaluating their alignment with post-deployment provisions in three major governance frameworks: the EU AI Act (Articles 72-73), the NIST AI Risk Management Framework (MANAGE and GOVERN functions), and the General Data Protection Regulation (GDPR Articles 22, 33-35). The results reveal substantial governance gaps across these frameworks, indicating persistent weaknesses in post-deployment accountability. Second, based on these findings, the study proposes the Proactive AI Governance Compliance Framework (PAGCF), a four-phase lifecycle methodology designed to shift governance from reactive incident response toward pre-deployment compliance assurance. The framework includes risk-stratified governance tiers, an implementation checklist linked to specific regulatory provisions, and a projected impact analysis that uses internal monitoring as a proxy for proactive governance capacity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper conducts a cross-regulatory empirical analysis of 480 AI incidents from the AI Incident Database, mapping them to post-deployment provisions in the EU AI Act (Articles 72-73), NIST AI RMF (MANAGE and GOVERN functions), and GDPR (Articles 22, 33-35). It reports substantial governance gaps in post-deployment accountability and proposes the Proactive AI Governance Compliance Framework (PAGCF), a four-phase lifecycle methodology with risk-stratified tiers, an implementation checklist, and internal monitoring as a proxy for proactive governance capacity.

Significance. If the gap findings hold after methodological clarification, the work provides useful empirical grounding for debates on AI accountability by leveraging real-world incident data across multiple frameworks. The PAGCF offers a concrete, lifecycle-based proposal that could inform organizational practices and policy development in high-stakes AI domains. The multi-framework comparison is a constructive element that identifies shared weaknesses.

major comments (2)
  1. [Empirical analysis and projected impact analysis] The classification of the 480 incidents for alignment with specific regulatory articles (EU AI Act 72-73, NIST MANAGE/GOVERN, GDPR 22/33-35) is presented without any disclosed coding protocol, decision rules, inter-rater reliability statistics, or examples of borderline cases. This directly undermines the central empirical claim of 'substantial governance gaps' because the gap identification rests on these judgments; without transparency, the results could reflect coder discretion rather than objective misalignment (see the description of the empirical analysis and projected impact analysis).
  2. [PAGCF framework and projected impact analysis] The claim that internal monitoring serves as a valid proxy for proactive governance capacity is introduced as an axiom for the PAGCF's projected impact but lacks supporting validation, either from the incident data or external benchmarks. This assumption is load-bearing for the framework's justification as a shift from reactive to proactive governance.
minor comments (1)
  1. [Abstract and Methods] The abstract and methods description would benefit from explicit statement of the time period covered by the 480 incidents and the precise selection criteria applied from the AIID to allow assessment of sample representativeness.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which help clarify areas where our manuscript can be strengthened. We address each major comment below and indicate the revisions planned for the next version.

read point-by-point responses

  1. Referee: [Empirical analysis and projected impact analysis] The classification of the 480 incidents for alignment with specific regulatory articles (EU AI Act 72-73, NIST MANAGE/GOVERN, GDPR 22/33-35) is presented without any disclosed coding protocol, decision rules, inter-rater reliability statistics, or examples of borderline cases. This directly undermines the central empirical claim of 'substantial governance gaps' because the gap identification rests on these judgments; without transparency, the results could reflect coder discretion rather than objective misalignment (see the description of the empirical analysis and projected impact analysis).

    Authors: We agree that greater transparency in the classification process is necessary to support the empirical claims. The current manuscript provides only a high-level description of the mapping to regulatory provisions without detailing the protocol, decision rules, or reliability checks. In the revised version, we will add a dedicated methods subsection that specifies the coding protocol, explicit decision rules for alignment with each article or function, examples of borderline cases and their resolution, and any steps taken to ensure consistency across coders. This addition will allow readers to evaluate the objectivity of the gap identification independently. revision: yes

  2. Referee: [PAGCF framework and projected impact analysis] The claim that internal monitoring serves as a valid proxy for proactive governance capacity is introduced as an axiom for the PAGCF's projected impact but lacks supporting validation, either from the incident data or external benchmarks. This assumption is load-bearing for the framework's justification as a shift from reactive to proactive governance.

    Authors: We recognize that the proxy role of internal monitoring is presented conceptually without direct empirical validation from the 480-incident dataset or cited external benchmarks. The PAGCF is derived from the observed post-deployment gaps, and the proxy is intended to operationalize proactive capacity. In the revision, we will expand the projected impact analysis to include supporting references from the AI governance literature on monitoring practices, clarify the logical basis for the proxy, and explicitly note its status as a proposed mechanism rather than a validated result. We will also discuss potential avenues for future empirical testing of the proxy. revision: yes

Circularity Check

0 steps flagged

No significant circularity: empirical mapping to external regulations and data-driven proposal remain independent

full rationale

The paper's derivation proceeds from an external dataset (480 incidents from the AI Incident Database) to alignment judgments against fixed external regulatory texts (EU AI Act Articles 72-73, NIST MANAGE/GOVERN, GDPR 22/33-35), identification of gaps, and then a constructive proposal of the PAGCF as a response. No equations, fitted parameters, or self-referential definitions are present. The PAGCF is not equivalent to the input data by construction; it is a new four-phase methodology offered as a remedy. Absence of disclosed coding criteria affects reliability but does not create circularity, as the core claims rest on external sources rather than reducing to self-definition or self-citation chains. This is a standard empirical-to-proposal structure with independent content.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 1 invented entities

The central claims rest on the representativeness of the AIID sample and the validity of the alignment evaluation and proxy measure, none of which receive independent empirical support beyond the abstract description.

free parameters (1)
  • Risk-stratification thresholds
    The framework defines risk-stratified governance tiers whose specific cutoffs or criteria are not specified in the abstract.
axioms (2)
  • domain assumption The AI Incident Database supplies a representative sample of real-world AI incidents suitable for evaluating post-deployment governance gaps.
    The entire empirical analysis depends on this assumption about the 480 incidents.
  • ad hoc to paper Internal monitoring can serve as a reliable proxy for proactive governance capacity.
    This proxy is invoked to support the projected impact analysis of the PAGCF.
invented entities (1)
  • Proactive AI Governance Compliance Framework (PAGCF) no independent evidence
    purpose: To provide a four-phase lifecycle methodology that shifts AI governance from reactive to pre-deployment compliance.
    New framework introduced by the authors based on the gap analysis.

pith-pipeline@v0.9.0 · 5721 in / 1590 out tokens · 53781 ms · 2026-05-21T09:11:08.632258+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [1]

    In response, governance frameworks have emphasized post-deployment oversight through mechanisms such as post-market monitoring, incident reporting, and regulatory accountability

    Introduction Artificial intelligence systems are increasingly deployed across high-stakes domains, including healthcare, criminal justice, finance, transportation, and public administration. In response, governance frameworks have emphasized post-deployment oversight through mechanisms such as post-market monitoring, incident reporting, and regulatory acc...

    work page 2024

  2. [2]

    Literature Review and Regulatory Background The governance of AI systems has become a central challenge in technology policy. Although existing scholarship has examined AI governance principles (Jobin, Ienca, & Vayena, 2019; Hagendorff, 2020), regulatory design (Smuha, 2021), and risk-based approaches (Novelli, Taddeo, & Floridi, 2023), empirical evidence...

    work page doi:10.1007/s11023-020-09517-8 2019