REVIEW 4 major objections 4 minor 34 references
A Rapid Review Regarding the Concept of Legal Requirements in Requirements Engineering
T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Legal requirements in software engineering have no shared definition, and the common claims about them rest on weak evidence, a rapid review argues.
desk verdict A candid, small-scale rapid review that documents conceptual disarray about legal requirements, but overreaches when it claims 'no robust empirical evidence' on the strength of an open-access-only sample. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a rapid review design: a Boolean keyword search in a literature database, restricted to open-access English papers, then reduced to the 21 papers that actually define or characterize LRs, and finally coded thematically across three research questions (what LRs are, what characteristics they have, how they affect engineers). The thematic table organizing 31 themes under these three questions is the paper's main conceptual device, augmented by a 'classical' selection of additional literature.
What would settle it
A broader systematic review that searches all 602 initially retrieved papers (not just the open-access subset) for explicit definitions of legal requirements; if many definitions with a shared core are found, the claim of conceptual confusion would be weakened. Alternatively, a large representative survey of requirements engineers asking whether they implement LRs reluctantly and minimally; a clear majority reporting diligent implementation would undercut the folklore claim.
Extended reading notes
Core claim
On its own terms, the paper establishes that among a systematically sampled set of 21 open-access papers plus subjectively chosen classical literature, there is no consistent definition of legal requirements. A normative understanding is common but rarely operationalized, and LRs are variously framed as functional or non-functional requirements. The review catalogs a recurring set of characteristics: ambiguity and complexity, knowledge gaps among engineers, change and overlap, high effort, minimal and reluctant implementation, prioritization driven by penalty risk, late consideration, and validation by regulators or supervisors. Its critical claim is that none of these recurring characteriza
Load-bearing premise
The conclusions depend on the assumption that the 21 open-access English-language papers from one literature database, plus the author's subjectively chosen classical literature, fairly represent how requirements engineering research thinks about legal requirements.
Editorial extensions
If this is right
- If there is no shared conceptual framework for LRs, then tools and methods for legal compliance in RE rest on unstable foundations; a common terminology would be a necessary first step toward stability.
- The recurring claims about vagueness, complexity, effort, and reluctance should be treated as hypotheses, not established facts, until tested by empirical studies such as industry surveys.
- The unresolved functional versus non-functional classification of LRs has practical consequences for how engineers model, prioritize, and trace legal requirements.
- Laws themselves can be measured directly, for example through readability metrics and cross-reference analysis, giving empirical content to claims about legal ambiguity and complexity.
- Because this is a rapid review with a deliberately small sample, it does not settle the questions it raises; it reframes them as research gaps for the community.
Reading between the lines
- The absence of empirical evidence may reflect publication incentives: LRs often appear only as motivation for a proposed tool or technique, so the recurrence of the same characterizations across papers might indicate citation chains rather than independent observation.
- A natural test would be a preregistered survey of requirements engineers in regulated industries asking whether they implement LRs reluctantly and minimally; if most report diligent, value-driven implementation, the 'folklore' claim would be weakened.
- The paper's hint about deontic versus constitutive norms suggests one constructive path forward: borrowing more rigorously from legal theory categories to give LRs a firmer conceptual footing.
- The stabilization hypothesis—that LRs stabilize over time and recur in similar forms—could be tested with longitudinal case studies of how specific legal requirements migrate into software systems across projects and years.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper conducts a rapid review of the concept of legal requirements (LRs) in requirements engineering (RE) research. Based on a Scopus search restricted to open-access English-language papers, 21 papers were included after excluding those that did not offer a definition or characterization of LRs. The author augments this sample with subjectively selected 'classical' literature, including his own prior work. The review identifies themes under three research questions: what LRs are (RQ.1), their characteristics (RQ.2), and their effect on engineers' work beyond elicitation (RQ.3). The paper reports that LRs are often treated normatively, sometimes as functional and sometimes as non-functional, and are characterized as ambiguous, complex, changing, overlapping, and reluctantly implemented. The main conclusions are that there is enduring conceptual confusion about LRs and that the recurring claims about them are not backed by robust empirical evidence.
Significance. If the conclusions are accepted, the paper provides a useful critical synthesis of how a core but poorly defined concept in RE research is used, and it identifies a potential gap: many claims about LRs rest on anecdote or opinion rather than empirical data. The review is transparent about its scope and limitations, and it offers concrete quotes from the sampled papers, which supports the observation of heterogeneous conceptualizations. However, the strength of the claimed empirical gap is not warranted by the sampling method, which excludes closed-access venues by design. The paper also raises a plausible and practically important call for empirical studies of how software engineers perceive and handle legal requirements. The contribution is modest but potentially valuable for RE researchers and practitioners seeking a starting point for a more rigorous conceptual foundation.
major comments (4)
- [Section 4] The claim 'none of what is present in Table 1 is backed by robust empirical evidence' overreaches relative to the sampling frame. Section 2 deliberately restricts the corpus to open-access papers (602 hits reduced to 154 open access, then to 21 after exclusions). This design biases against finding empirical evidence, as many empirical RE studies appear in closed-venue journals and conferences. Moreover, the sample itself contains at least three papers with empirical components: [7] is an empirical perceptions study, [9] includes an evaluation of design patterns, and [3] is published in Empirical Software Engineering. The author should either restrict the conclusion explicitly to the sampled papers or conduct an audit of the 154 open-access papers for empirical content before generalizing to the field.
- [Section 2] The augmentation of the systematic sample with 'subjectively selected' classical literature is a source of potential selection bias and mild circularity. The author's own prior work [15,26,27] is included and then used to support themes such as overlaps and non-negotiability. This can inflate support for themes the author already believes in. The paper should specify the criteria for selecting the classical literature and ideally show that the core themes identified from the 21-paper sample do not depend on these additional works. Without this, the conclusion of conceptual confusion may partly reflect the author's own framing.
- [Section 3 / Table 1] The thematic coding is not described with a formal protocol. The paper says a thematic approach is used for feasibility, but does not explain how themes were derived, how disagreements were handled, or whether any inter-rater reliability was assessed. For a review that draws critical conclusions about the field, the lack of a transparent coding scheme makes it difficult for readers to verify the mapping from papers to themes. This is a methodological limitation that should be acknowledged and at least partially mitigated by providing example quotes for each theme and clarifying the inclusion/exclusion decision rules for each theme.
- [Section 3, 'Reluctance' theme] In Table 1, the Theme 5 'Reluctance' is associated with references [5,9,10]. However, in the text, the reluctance theme is supported only by quotations from [10] and [9], while [5] is used for the 'timing' theme. This appears to be a citation mismatch. If [5] does indeed support reluctance, a specific quotation or explanation should be provided; otherwise, the table entry should be corrected. Such inconsistencies undermine the reliability of the theme enumeration as presented.
minor comments (4)
- [Section 2] The search string and inclusion criteria are described only vaguely: 'from all fields of papers written in English' and the keyword search. Please specify the exact Scopus query, including whether 'legal requirement' was searched as an exact phrase, and list the exclusion criteria (e.g., non-English, non-article) more explicitly for reproducibility.
- [Section 2, footnote] The footnote about 68 papers being mislabeled as open access is important for transparency. Consider moving this information into the main text so that readers understand the full reduction from 154 to 21 papers, as it affects the credibility of the sample description.
- [Abstract] The phrase 'These characterizations supposedly correlate with knowledge gaps' is vague. Consider rephrasing to clarify whether the correlation is claimed in the literature, inferred by the author, or hypothesized, as this distinction is central to the review's critical argument.
- [Section 3] Some quotations include page numbers (e.g., [10, p. 2322]) while others do not (e.g., [6, p. 3] is given but many others are not). For a rapid review, this is acceptable, but consistency would be helpful if the authors want to align with systematic review conventions.
Circularity Check
No significant circularity; self-citations are peripheral, non-load-bearing corroboration.
full rationale
The paper is a rapid literature review, not a derivation or prediction exercise. The central observations (conceptual confusion, normative/functional/non-functional framings, ambiguity/complexity themes, lack of empirical evidence) are drawn from a systematically sampled set of 21 Scopus-indexed open-access papers and quoted passages such as [3,4,9,10,30,33]. The conclusion that 'none of what is present in Table 1 is backed by robust empirical evidence' is an evaluation of that sample, not a quantity fitted from it. The author's own prior work [15,26,27] appears in the 'classical' augmentation and in asides: e.g., 'overlaps were frequently recognized in the systematically sampled literature, which aligns with the non-sampled literature [26,27]'. This self-citation is corroborative only; the overlap theme is already supported by sampled papers [9,10,21,29] in Table 1. Other self-citations ([26] for perceived sensibility of CRA requirements, [27] for 'checkbox exercises' and compliance-oriented risk analysis) illustrate or contextualize themes but are not the basis for the paper's main claims. No definition is stipulated in terms of a conclusion, no parameter is fitted and renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. The sampling limitation (open-access-only, small n) is a validity concern, not a circularity. Hence no circular step is present; at most there is a minor self-citation that is not load-bearing.
Assumptions & free parameters
assumptions (3)
- domain assumption The 21 open-access Scopus papers plus the author's subjectively selected additions adequately represent how legal requirements are conceptualized in RE research.
- domain assumption Categorizing statements from sampled papers into the themes in Table 1 is a valid way to answer the research questions.
- ad hoc to paper The absence of empirical evidence in the sampled papers is evidence that robust empirical evidence is lacking in the wider literature.
Cite this review
Pith. "Pith review of A Rapid Review Regarding the Concept of Legal Requirements in Requirements Engineering." pith.science (2026). https://pith.science/paper/CJ5NRCXM
@misc{pith2026250906012,
author = {Pith},
title = {Pith review of: A Rapid Review Regarding the Concept of Legal Requirements in Requirements Engineering},
year = {2026},
howpublished = {\url{https://pith.science/paper/CJ5NRCXM}},
note = {Machine review of arXiv:2509.06012}
}
read the original abstract
Out of a personal puzzlement, recent peer review comments, and demonstrable confusion in the existing literature, the paper presents a rapid review of the concept of legal requirements (LRs) in requirements engineering (RE) research. According to reviewing results, a normative understanding of LRs has often been present, although proper definitions and conceptual operationalizations are lacking. Some papers also see LRs as functional and others as non-functional requirements. Legal requirements are often characterized as being vague and complex, requiring a lot of effort to elicit, implement, and validate. These characterizations supposedly correlate with knowledge gaps among requirements engineers. LRs are also seen to often change and overlap. They may be also prioritized. According to the literature, they seem to be also reluctantly implemented, often providing only a minimal baseline for other requirements. With these and other observations, the review raises critical arguments about apparent knowledge gaps, including a lack of empirical evidence backing the observations and enduring conceptual confusion.
Reference graph
Works this paper leans on
-
[3]
Empirical Software Engineering pp
Azeem, M.I., Abualhaija, S.: A Multi-Solution Study on GDPR AI-Enabled Com- pleteness Checking of DPAs. Empirical Software Engineering pp. 1–31 (2024)
work page 2024
-
[10]
Electronic Markets 32, 2311–2331 (2022)
Dickhaut, E., Li, M.M., Janson, A., Leimeister, J.M.: The Role of Design Patterns in the Development and Legal Assessment of Lawful Technologies. Electronic Markets 32, 2311–2331 (2022)
work page 2022
-
[30]
SN Computer Science 3, 1–25 (2022)
Soavi, M., Zeni, N., Mylopoulos, J., Mich, L.: From Legal Contracts to Formal Specifications: A Systematic Literature Review. SN Computer Science 3, 1–25 (2022)
work page 2022
-
[7]
Canedo, E.D., Calazans, A.T.S., Masson, E.T.S., Costa, P.H.T., , Lima, F.: Per- ceptions of ICT Practitioners Regarding Software Privacy. Entropy 22, 1–23 (2020)
work page 2020
-
[9]
European Journal of Information Systems 33(4), 441–468 (2024)
Dickhaut, E., Janson, A., S¨ ollner, M., Leimeister, J.M.: Lawfulness by design – Development and Evaluation of Lawful Design Patterns to Consider Legal Rquire- ments. European Journal of Information Systems 33(4), 441–468 (2024)
work page 2024
-
[5]
Neurocomputing 483, 386–397 (2022)
Billhardt, H., Santos, J.A., Fern´ andez, A., Moreno, M., Ossowski, S., Rodr ´ ıguez, J.A.: Streamlining Advanced Taxi Assignment Strategies Based on Legal Analysis. Neurocomputing 483, 386–397 (2022)
work page 2022
-
[1]
Abualhaija, S., Ceci, M., Briand, L.: Legal Requirements Analysis (2024), archived manuscript, available online:https://doi.org/10.48550/arXiv.2311.13871
work page Pith review arXiv doi:10.48550/arxiv.2311.13871 2024
-
[2]
ACM Com- puting Surveys 54(5), 1–38 (21)
Aljeraisy, A., Barati, M., Rana, O., Perera, C.: Privacy Laws and Privacy by De- sign Schemes for the Internet of Things: A Developer’s Perspective. ACM Com- puting Surveys 54(5), 1–38 (21)
Show all 34 references
-
[4]
In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI 2024)
Bertrand, A., Eagan, J.R., Maxwell, W., Brand, J.: AI Is Entering Regulated Territory: Understanding the Supervisors’ Perspective for Model Justifiability in Financial Crime Detection. In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI 2024)....
2024
-
[6]
Data & Knowledge Engineering 159 (2025)
Buschhaus, C., Butting, A., Michael, J., Nitsch, V., P¨ utz, S., Rumpe, B., Stell- macher, C., Theis, S.: Overcoming the Hurdle of Legal Expertise: A Reusable Model for Smartwatch Privacy Policies. Data & Knowledge Engineering 159 (2025)
2025
-
[8]
Information 11, 1–27 (2020)
Diamantopoulou, V., Androutsopoulou, A., Gritzalis, S., Charalabidis, Y.: Pre- serving Digital Privacy in e-Participation Environments: Towards GDPR Com- pliance. Information 11, 1–27 (2020)
2020
-
[11]
The Journal of Systems and Software 148, 170–179 (2019)
Fern´ andez, D.M., Passoth, J.H.: Empirical Software Engineering: From Discipline to Interdiscipline. The Journal of Systems and Software 148, 170–179 (2019)
2019
-
[12]
In: Proceedings of the 33rd International Conference on Software Engineering and Knowledge Engineering (SEKE 2021)
G´ omez-Mart ´ ınez, E., Marroyo, M., Acu˜ na, S.T.: Towards the Integration of the GDPR in the Unified Software Development Process. In: Proceedings of the 33rd International Conference on Software Engineering and Knowledge Engineering (SEKE 2021). pp. 199–204. KSI Research I...
2021
-
[13]
Journal of Clinical Epidemiology 129, 74–85 (2021)
Hamela, C., Michauda, A., Thukua, M., Skidmorea, B., Stevensa, A., Nussbaumer- Streitc, B., Garritty, C.: Defining Rapid Reviews: A Systematic Scoping Review and Thematic Analysis of Definitions and Defining Characteristics of Rapid Re- views. Journal of Clinical Epidemiology ...
2021
-
[14]
Resources, Conservation & Recycling 209, 107752 (2024)
Hansen, S., R ¨ther, T., Mennenga, M., Helbig, C., Ohnem¨ uller, G., Vysoudil, F., Wolf, C., Rosemann, B., Pav´ on, S., Michaelis, A., Vietor, T., D¨ opper, F., Her- rmann, C., Danzer, M.A.: A Structured Approach for the Compliance Analysis of Battery Systems With Regard to th...
2024
-
[15]
In: Proceedings of the 27th IEEE International Requirements Engineering Conference (RE 2019)
Hjerppe, K., Ruohonen, J., Lepp¨ anen, V.: The General Data Protection Regula- tion: Requirements, Architectures, and Constraints. In: Proceedings of the 27th IEEE International Requirements Engineering Conference (RE 2019). pp. 265–
2019
-
[16]
Information and Software Technology 51, 7–15 (2009)
Kitchenham, B., Brereton, O.P., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic Literature Reviews in Software Engineering – A Systematic Literature Review. Information and Software Technology 51, 7–15 (2009)
2009
-
[17]
Information and Software Technology 178, 107622 (2025)
Kosenkov, O., Elahidoost, P., Gorschek, T., Fischbach, J., Mendez, D., Unterkalm- steiner, M., Fucci, D., Mohanani, R.: Systematic Mapping Study on Requirements Engineering for Regulatory Compliance of Software Systems. Information and Software Technology 178, 107622 (2025)
2025
-
[18]
BMC Medical Informatics and Decision Making 16, 1–19 (2016)
Kuchinke, W., Krauth, C., Bergmann, R., Karakoyun, T., Woollard, A., Schluen- der, I., Braasch, B., Eckert, M., Ohmann, C.: Legal Assessment Tool (LAT): An Interactive Tool to Address Privacy and Data Protection Issues for Data Sharing. BMC Medical Informatics and Decision Mak...
2016
-
[19]
Data & Knowledge Engineering 134, 1–35 (2021)
Maass, W., Storey, V.C.: Pairing Conceptual Modeling With Machine Learning. Data & Knowledge Engineering 134, 1–35 (2021)
2021
-
[20]
IEEE Security & Privacy 2(2), 80–83 (2004)
McGraw, G.: Software Security. IEEE Security & Privacy 2(2), 80–83 (2004)
2004
-
[21]
Ethics and Information Technology 22, 223–238 (2020)
Muravyeva, E., Janssen, J., Specht, M., Custers, B.: Exploring Solutions to the Privacy Paradox in the Context of e-Assessment: Informed Consent Revisited. Ethics and Information Technology 22, 223–238 (2020)
2020
-
[22]
In: Forth- coming in the Proceedings of the IEEE 33rd International Requirements Engi- neering Conference Workshops (REW 2025)
Negri-Ribalta, C., Noel, R., Sergeeva, A., Gabriele, L.: Towards Evidence-Based Conceptual Modeling for International Data Protection Requirements. In: Forth- coming in the Proceedings of the IEEE 33rd International Requirements Engi- neering Conference Workshops (REW 2025). I...
2025
-
[23]
Computers & Security 117, 102697 (2022)
Olukoya, O.: Assessing Frameworks for Eliciting Privacy & Security Requirements from Laws and Regulations. Computers & Security 117, 102697 (2022)
2022
-
[24]
Sustain- ability 8, 1–15 (2016)
Pigosso, D.C.A., Ferraz, M., Teixeira, C.E., Rozenfeld, H.: The Deployment of Product-Related Environmental Legislation into Product Requirements. Sustain- ability 8, 1–15 (2016)
2016
-
[25]
In: Proceedings of the Eighth International Conference on eDemocracy & eGovernment (ICEDEG 2021)
Ruohonen, J.: Assessing the Readability of Policy Documents on the Digital Sin- gle Market of the European Union. In: Proceedings of the Eighth International Conference on eDemocracy & eGovernment (ICEDEG 2021). pp. 205–209. IEEE, Quito (online) (2021)
2021
-
[26]
In: Forthcoming in the Proceedings of the IEEE 33rd International Requirements Engineering Conference Workshops (REW 2025)
Ruohonen, J., Hjerppe, K., Kang, E.Y.: A Mapping Analysis of Requirements Between the CRA and the GDPR. In: Forthcoming in the Proceedings of the IEEE 33rd International Requirements Engineering Conference Workshops (REW 2025). IEEE, Valencia (2025)
2025
- [27]
-
[28]
Require- ments Engineering 22, 215–237 (2017)
Sannier, N., Adedjouma, M., Sabetzadeh, M., Briand, L.: An Automated Frame- work for Detection and Resolution of Cross References in Legal Texts. Require- ments Engineering 22, 215–237 (2017)
2017
-
[29]
Data & Policy 4, 1–21 (2022)
Simonofski, A., Tombal, T., De Terwangne, C., Willem, P., Frenay, B., Janssen, M.: Balancing Fraud Analytics With Legal Requirements: Governance Practices and Trade-Offs in Public Administrations. Data & Policy 4, 1–21 (2022)
2022
-
[31]
In: Proceedings of the IEEE/ACM 1st Workshop on AI Engineering – Software Engineering for AI (W AIN 2021)
Song, Q., Engstr¨ om, E., Runeson, P.: Concepts in Testing of Autonomous Systems: Academic Literature and Industry Practice. In: Proceedings of the IEEE/ACM 1st Workshop on AI Engineering – Software Engineering for AI (W AIN 2021). pp. 74–81. Madrid (2021)
2021
-
[32]
JMIR Human Factors 9(2), 1–25 (2022)
Yeng, K., Fauzi, M.A., Sun, L., Yang, B.: Assessing the Legal Aspects of Informa- tion Security Requirements for Health Care in 3 Countries: Scoping Review and Framework Development. JMIR Human Factors 9(2), 1–25 (2022)
2022
-
[33]
Journal of Cyber- security 11(1), 1–20 (2025)
Zafar, A.: Reconciling Blockchain Technology and Data Protection Laws: Regula- tory Challenges, Technical Solutions, and Practical Pathways. Journal of Cyber- security 11(1), 1–20 (2025)
2025
-
[275]
IEEE, Jeju Island (2019)
2019
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.