REVIEW 3 major objections 5 minor 6 references
Unlocking the Black Box: Analysing the EU Artificial Intelligence Act's Framework for Explainability in AI
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper argues that the lack of explainability of AI is the first obstacle that industry, regulators, and supervisors must overcome, and that the EU AI Act's explainability principle becomes enforceable only if co-regulation and…
desk verdict A competent, well-referenced doctrinal synthesis of XAI and the EU AI Act that argues explainability is central to enforcement, but the standardization bridge between principle and practice is assumed rather than demonstrated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is the legal principle of explainability—the requirement that AI decisions be comprehensible to the relevant humans—paired with the operational mechanism of co-regulation through standardisation under the New Legislative Framework (NLF), the EU's product-regulation toolbox. The paper's argument works by treating the black box problem (known inputs and outputs but opaque input-to-output transformation) as the obstacle, explainability as the remedy, and the NLF as the vehicle that can specify the remedy in technical standards. In this design, the AI Act sets the obligation, standardisation bodies specify the XAI methods, conformity assessment verifies compliance before market entry, and market surveillance authorities enforce the standards ex post. The paper also relies on a user-centric requirement: explanations must be tailored to different audiences, from laypersons to expert overseers, and balanced against privacy, trade secrets, and the transparency-performance trade-off.
What would settle it
The central claim would be falsified by observing a real high-risk AI system that is certified as explainable under the AI Act yet cannot produce an explanation that a non-expert can use to contest an adverse decision. Concretely, one could follow the first harmonised XAI standard adopted under the Act and test it on certified systems: if no such standard emerges, or if certified systems still fail auditable explanation in field tests, the co-regulation premise in Section 5 is undercut.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that the lack of explainability of AI is one of the first obstacles that the industry, regulators, and supervisors must overcome to mitigate AI risks. The paper establishes that AI opacity arises from complex architectures, non-linearity, high-dimensional data, and learnt representations, and that this opacity is the common denominator of most criticisms of AI. It then argues that explainability goes beyond transparency by revealing the cognitive processes underlying decisions, and that this distinction matters legally because disclosure alone cannot enable contestation or supervision. The constructive claim follows: the EU AI Act's explainability principle should be turned into enforceable obligations by specifying which XAI methods and standards count as compliant, using the EU's co-regulatory standardisation toolbox, and backing it with ex ante conformity assessment, registration, ex post market surveillance, effective penalties, and a supranational coordination mechanism.
Load-bearing premise
The load-bearing premise is that co-regulation and standard setting can turn the broad principle of explainability into concrete, auditable technical standards; the paper's whole enforcement proposal collapses if standardisation bodies cannot specify workable XAI requirements—and the paper itself notes that no agreed objective evaluation method for XAI exists.
Editorial extensions
If this is right
- High-risk AI systems covered by the AI Act would face a gatekeeping requirement: before entering the EU market, they must undergo conformity assessment that verifies explainability alongside quality and risk management, and they must be registered in an EU database.
- Mere transparency will not satisfy the law; the explainability obligation will require explanations that render the decision process comprehensible to the relevant user group, which makes user-centric explanation design a legal compliance task.
- The enforceability of the explainability duty depends on standardisation bodies producing concrete, auditable XAI standards; absent such standards, market surveillance authorities have no benchmark against which to check AI systems.
- Oversight authorities need effective powers (audits, inspections, reporting requirements) and adequate resources, plus a supranational coordination mechanism for cross-border AI deployments.
- Because XAI involves trade-offs with performance, cost, privacy, and trade secrets, the AI Act's framework implies that explainability requirements must be calibrated to risk category and to the intended audience.
Reading between the lines
- Editorial inference: If no agreed objective method for evaluating XAI exists, as the paper itself acknowledges, then co-regulation transfers the hard problem to standardisation bodies rather than solving it; the true test of the paper's framework is whether those bodies can produce a standard that auditors can apply to real models.
- Editorial inference: The paper's distinction between transparency and explainability implies a measurable design requirement: explanation interfaces should be evaluated by how well the intended user group understands and contests decisions, not by technical fidelity of the explanation.
- Editorial inference: The same logic extends beyond the EU: any jurisdiction that adopts an AI explainability duty without an auditable standard-setting mechanism will likely face the same enforcement gap the paper identifies in the EU context.
- Editorial inference: The cost and inequity of XAI flagged in the paper suggests a possible market failure: if explainability standards are expensive to meet, small and medium-sized providers may be priced out, so the framework may need tiered or subsidised compliance pathways.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper, a legal-policy analysis published as an accepted author manuscript in Law, Innovation and Technology, argues that the lack of explainability in AI systems ('the black box problem') is one of the first obstacles that industry, regulators, and supervisors must overcome to mitigate AI-related risks. It situates this argument within the EU AI Act, describing the Act's risk-based framework and its legislative history up to the December 2023 political agreement. The paper then surveys XAI techniques (feature attribution, counterfactuals, sensitivity analysis, etc.), reviews implementation challenges (evaluation, cost, trade-offs, ethics, privacy, trade secrecy, human-AI interaction), and proposes that explainability be integrated into EU law through the New Legislative Framework's co-regulation model, with CENELEC and other standardisation bodies specifying XAI standards and national market surveillance authorities enforcing them. The paper closes with broader implications for trust and accountability.
Significance. If the claims hold, the paper offers a useful synthesis of the legal and technical dimensions of explainability in EU AI regulation, bridging a gap between XAI research and EU law scholarship. Its strength lies in the clear distinction between transparency and explainability (§3), the comprehensive identification of implementation obstacles (§4), and the explicit policy proposal to use co-regulation and standardisation (§5). The paper is not empirical; its argument is normative and policy-oriented, and it does not claim to provide new technical results. The main value is in framing explainability as an enforceable legal obligation rather than a mere ethical aspiration, and in highlighting the standardisation–evaluation nexus. However, the significance is tempered by the underdeveloped operational mechanism: the paper does not show how the proposed standardisation route overcomes the acknowledged lack of objective XAI evaluation methods. For a law and technology readership, the paper is a useful roadmap but leaves the central enforcement question open.
major comments (3)
- [§5 and §4]
- [Abstract and §1]
- [§5, Footnote 70]
minor comments (5)
- [§1, first paragraph]
- [Footnote 34]
- [§1]
- [§5, paragraph 5]
- [§3]
Circularity Check
No circularity: the paper is a legal-policy analysis with no fitted quantities or derivations, and its self-citations are illustrative, not load-bearing.
full rationale
The paper does not derive a quantitative or formal result; it advances a normative argument that explainability is a prerequisite for effective AI regulation and analyzes how the EU AI Act could integrate XAI. There are no equations, fitted parameters, or 'predictions' in the scientific sense, so the core circularity patterns (self-definitional, fitted-input-called-prediction, uniqueness imported from authors, ansatz smuggled via citation, renaming known result) do not apply. The author's self-citations (e.g., Pavlidis 2021 on digital finance regulation, Pavlidis 2023 and 2019 on AML supervision and cross-border confiscation) appear in footnotes as supporting examples or analogies for institutional design, not as the foundation of the central claim. The central argument rests on external legal texts, EU documents, and the broader literature on XAI and AI governance. Section 4's concession that 'there is a lack of agreement on how to perform an objective evaluation of XAI methods' is an acknowledged open problem, not a hidden input into the conclusion. Section 5's reliance on co-regulation and standardization is a policy proposal informed by the New Legislative Framework and cited external scholarship; it is not circular merely because the paper argues that such governance will gain traction. Thus there is no load-bearing self-citation or construction by which the paper's conclusion is equivalent to its inputs. The paper is self-contained as a legal analysis, and any weakness in the operational bridge between XAI evaluation and standardization is a matter of evidentiary support or normative plausibility, not circularity.
Assumptions & free parameters
assumptions (3)
- domain assumption Explainability is a prerequisite for accountability, fairness, public trust, and effective regulation and supervision.
- domain assumption The shift from soft law to hard law for AI regulation is timely and appropriate.
- domain assumption Standardization under the New Legislative Framework can provide concrete XAI techniques and requirements.
Cite this review
Pith. "Pith review of Unlocking the Black Box: Analysing the EU Artificial Intelligence Act's Framework for Explainability in AI." pith.science (2026). https://pith.science/paper/JIG5JC7T
@misc{pith2026250214868,
author = {Pith},
title = {Pith review of: Unlocking the Black Box: Analysing the EU Artificial Intelligence Act's Framework for Explainability in AI},
year = {2026},
howpublished = {\url{https://pith.science/paper/JIG5JC7T}},
note = {Machine review of arXiv:2502.14868}
}
read the original abstract
The lack of explainability of Artificial Intelligence (AI) is one of the first obstacles that the industry and regulators must overcome to mitigate the risks associated with the technology. The need for eXplainable AI (XAI) is evident in fields where accountability, ethics and fairness are critical, such as healthcare, credit scoring, policing and the criminal justice system. At the EU level, the notion of explainability is one of the fundamental principles that underpin the AI Act, though the exact XAI techniques and requirements are still to be determined and tested in practice. This paper explores various approaches and techniques that promise to advance XAI, as well as the challenges of implementing the principle of explainability in AI governance and policies. Finally, the paper examines the integration of XAI into EU law, emphasising the issues of standard setting, oversight, and enforcement.
Reference graph
Works this paper leans on
-
[1]
AI has already started to streamline mundane tasks, advance critical domains of scientific research and disrupt professions and in dustries
The Imperative of Explainability: Navigating the Landscape of AI’s Impact and Risks Artificial intelligence (AI) has emerged as a fascinating and influential force in today’s technological and business worlds. AI has already started to streamline mundane tasks, advance critical domains of scientific research and disrupt professions and in dustries. Whethe...
2023
-
[2]
The EU AI Act: Searching for Rules and Clarity EU policymakers have consistently stressed the importance of fostering trustworthy, responsible and ‘human -centric’ innovation in the field of AI. 16 From the 2016 EU Global Strategy for Foreign and Security Policy17 to the 2021 European Commission’s plan for AI, 18 the EU has recognised the need to adapt cu...
arXiv 2018
-
[3]
Unravelling the Magic: Approaches to and Techniques for XAI The principle of explainability and the field of XAI are often put forward as the solution to the above -mentioned ‘black box problem’ . Although XAI lacks a commonly agreed definition, it can be broadly described as AI techniques that enable human users to comprehend, trust and manage AI effecti...
arXiv 2022
-
[4]
Challenges and Considerations regarding XAI Implementation While the importance of explainability is widely acknowledged and despite years of research in the field, implementing explainability has proved challenging due to issues of complexity and scalability. As AI models grow in size and complexity, so do the practical obstacles to producing XAI models ...
arXiv 2021
-
[5]
Feature Importance Measure for Non-linear Learning Algorithms
Integrating AI Explainability into EU Law Given these challenges, it is worth examining how to harmoniously meld the principle of AI explainability with the current legal landscape of the EU. As already mentioned, the EU must first determine which AI systems qualify as high risk and thus fall under the obligation of explainability. The EU must also specif...
work page Pith review arXiv 2016
-
[6]
Fostering Trust and Accountability: The Broader Implications of Explainability in AI The concept of explainability is not just a legal term and a software/technical challenge. It is a principle with far -reaching societal implications, as it can foster trust and empower the public to engage with AI technologies more actively. Furthermore, explainability i...
work page 2019
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.