{"id":"a3da4f66-0d4e-41ed-b12a-3a210c78e63a","arxiv_id":"2412.17329","paper_version":2,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A systematic review shows the safe-and-secure-defaults design paradigm has spread across computing domains and gained new principles like off-by-default and zero trust, with IoT insecurity as a major driver.","lead":"This paper reviews 148 peer-reviewed papers to map how the design principle of safe and secure defaults has been used, extended, and violated in computing. It finds the principle has grown from access control to IoT, networking, and zero trust, and is now written into EU regulations.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sample bias in ACM/IEEE-only search may undercut the review's 'continuously discussed, used, and developed further' and 'adopted in numerous domains' claims, but the paper's own hedging and internal checks make this a conditional rather than fatal concern.","rationale":"The reader's weakest_assumption is the ACM/IEEE-only sample, and that is precisely the load-bearing concern I would raise. The paper itself is transparent about this limitation and explicitly notes that social science and organizational security may be missing. The conclusions are exploratory and hedged, so the concern does not rise to a rejection. However, the strongest claim—'extensively discussed, used, and developed further since the late 1990s'—is only supported for the sampled venues. A concrete re-search test would settle whether the sample is representative enough. I agree with the reader's verdict of ACCEPT but would add this test as a conditional verification step.","tokens_in":28498,"tokens_out":1327,"duration_ms":12519,"concrete_test":"Re-run the identical Boolean search (safe AND default) OR (secure AND default) in a broad multidisciplinary database (e.g., Scopus or Web of Science) and in HCI-focused venues (CHI, SOUPS, USENIX Security), applying the same four exclusion criteria. If the resulting set includes substantial new papers introducing distinct principles (e.g., from privacy, behavioral science, or safety engineering) or contradicting the claimed acceleration trend from mid-2010s, the review's conclusions would require re-scoping. If the additional set only adds confirmatory examples, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that safe-and-secure-defaults is a widely adopted, evolving design doctrine since the late 1990s—rests on a literature sample restricted to ACM's and IEEE's electronic libraries (Section 2). This restriction is flagged by the author as a limitation (Section 4.2), but it is still load-bearing because the review explicitly draws conclusions about trends (Fig. 2), domains (Table 2), design principles (Fig. 5), and problems (Fig. 6) from this sample. The strongest internal check is the author's own acknowledgment that social science and organizational security are missing (Section 4.3), which suggests the sample may under-represent research on defaults in human-computer interaction, behavioral economics, privacy, and safety engineering. However, the paper's conclusions are explicitly scoped as exploratory and hedged ('tentatively concluded', 'apparently', 'seems to be emerging'). The 148-paper sample covers a wide range of domains and venues, and the qualitative thematic analysis does not depend on statistical representativeness. The claim is therefore not invalidated, but its generalizability beyond ACM/IEEE venues is untested. A direct test: run the same search protocol in additional databases (e.g., Scopus, Web of Science, or ACM/IEEE plus major HCI venues such as CHI, SOUPS, USENIX Security) and compare whether the identified principles and missing domains change materially.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a systematization of knowledge (SoK) on the design paradigm of safe and secure defaults, conducted as a systematic mapping study / scoping review. The search protocol is the Boolean query (safe AND default) OR (secure AND default) run in the ACM and IEEE electronic libraries, with four exclusion criteria (primary studies only; substantive discussion of safe or secure defaults; peer-reviewed journal or conference papers; non-financial topics), yielding n=148 papers. The review maps publication trends (growth from the late 1990s, acceleration from the mid-2010s attributed partly to IoT insecurity), contextual domains (28 categories in Table 2, from access control and operating systems to IoT, cyber-physical systems, and work safety), motivations (dominated by insecure or unsafe defaults and human factors), design principles (including newer off-by-default, overriding/fallback, leak-prevention, and zero-trust principles), and four problem areas (break-glass emergencies, prediction failure, developer workarounds, trade-offs). The discussion closes with six summary points, an explicit limitation section (single-author coding; uncertain generalizability of the ACM/IEEE-only sample), and four future-research directions, including the EU Cyber Resilience Act.","tokens_in":28777,"tokens_out":22048,"duration_ms":192882,"significance":"If the review's claims hold, this is the first systematic map of the fail-safe-defaults paradigm and a useful reference for researchers, educators, and regulators. Strengths worth naming: the search string and exclusion criteria are reported verbatim; sampled and non-sampled literature are cleanly separated throughout; quotations are given with page numbers, making the thematic claims checkable; the author candidly classifies much human-factor reasoning in the sampled literature as folklore rather than evidence; and Section 4.2 states the single-coder and generalizability limitations explicitly. I find no circularity: the self-citations serve background or methodological points only. The main contributions are the catalog of emerging principles (off-by-default, overriding and fallback, leak prevention, zero trust), the documentation of the IoT-driven acceleration, and the identified gaps (organizational security, social science, regulation), which give the paper an agenda-setting function despite its exploratory design.","major_comments":[{"comment":"Section 2 restricts the search to the ACM and IEEE electronic libraries, Section 4.2 concedes that the n=148 sample cannot be generalized to an unknown theoretical population, and Section 4.3 admits that social-science and organizational-security work is absent from the sample. Despite these concessions, the abstract states that 'the paradigm has been extensively discussed, used, and developed further since the late 1990s,' and Section 4.1 makes unqualified assertions such as 'the domains in which the paradigm has been discussed and applied have considerably expanded over the years' (point 1) and 'the design paradigm has been discussed and developed with many security design principles' (point 5). These are claims about the paradigm as a whole, whereas the evidence base is a single sample drawn from two publisher libraries; major security venues such as USENIX Security, SOUPS, and NDSS, as well as behavioral, privacy, and safety-engineering outlets, are outside the sample. I regard this as a load-bearing scope mismatch rather than a fatal flaw: the fix is to either run the same protocol in additional databases (e.g., Scopus, Web of Science) and report whether the principles and gaps change, or explicitly qualify all headline claims and conclusions with 'in the reviewed ACM/IEEE-indexed literature,' including a visible caveat in the abstract.","section":"Abstract; §2; §4.1; §4.2"},{"comment":"Fig. 1 reports raw query counts (ACM 76/35; IEEE 35/503) and qualified counts (20/12/22/94) but does not report how many papers were excluded under each of the four inclusion/exclusion criteria, and the date of the searches is not stated anywhere in Section 2. Without this standard screening accounting, readers cannot assess how the exclusion criteria shaped the sample, which is central to a paper that claims systematization. Please add a per-criterion screening table with counts at each stage, state the search date, and clarify whether the ACM and IEEE queries matched metadata or full text.","section":"§2; Fig. 1"},{"comment":"Section 4.2 openly acknowledges that the review was conducted by a single author and that inter-rater reliability measures cannot be provided; this honesty is to the paper's credit. Nevertheless, Figs. 4–6 and Table 2 are the substantive contribution, and the paper currently offers no way to audit the coding: there is no initial category list, no operationalized merging rule beyond 'collating and merging of overlapping... categories,' and no worked example of how borderline papers were assigned (the 'opt-in' category in Fig. 4 is one such case). I recommend adding a codebook with the category definitions and one or two worked coding examples as an appendix or supplementary material so that the thematic constructs can be judged without a second coder.","section":"§2; §4.2; Figs. 4–6; Table 2"}],"minor_comments":[{"comment":"Table 2 lists 147 paper references by my count, whereas Fig. 1 reports a sample size of n=148; please verify the bookkeeping and add the missing reference or correct the count.","section":"Table 2; Fig. 1"},{"comment":"Section 3.4 states that 'three papers built upon the emerging zero trust principle' and quotes [98], [102], and [112], but Fig. 5 lists [98], [100], and [102] for zero trust; please align the text and the figure.","section":"§3.4; Fig. 5"},{"comment":"Section 2 flags paper [10] as likely containing tortured phrases, yet Section 3.3 cites [10] for the substantive claim that email is 'still unencrypted by default'; please verify that citation or replace it with a source of clear provenance.","section":"§2; §3.3"},{"comment":"Several reference entries contain errors: [44] gives the year 2019 for SACMAT 2007, [101] reports the symposium acronym as 'MOMS' where NOMS is presumably intended, [98] reads 'Scince' and [69] reads 'Proceeding sof,' [154] duplicates 'Workshop on,' [125] has '2the IEEE 15th Intl,' and [91] and [142] contain stray commas; a reference-list cleanup is in order.","section":"References"},{"comment":"Figure 1 spells 'detaults' twice, Fig. 4 has 'Acess controls,' Section 2 uses the unusual phrase 'overcasting' where 'overarching' seems intended, and Section 1 reads 'The Saltzer's and Schroeder's paper'; a light copy-editing pass over the figures and prose is needed.","section":"Figs. 1, 4; §1; §2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a transparent and honest scoping review whose main risk is that the abstract and Section 4.1 conclusions outrun the ACM/IEEE-only sample; I believe this is repairable by rescoping or by extending the search, not by re-analysis. The single-author coding is a genuine weakness but is disclosed, and a codebook appendix would largely mitigate it. I found no integrity concerns: the self-citations (e.g., [5], [15], [16], [18], [178]) are background or methodological, and the author's explicit flagging of the suspected tortured-phrase paper [10] is a point in the paper's favor. For the journal, this is a good-fit contribution; if the authors choose the rescoping route, the contribution becomes a map of the ACM/IEEE literature specifically, which I still consider publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: this is a useful and honest first systematic map of the safe-and-secure-defaults design paradigm. The obvious concern, the ACM/IEEE-only sample, is real, but the author's hedging and the scoping-review framing keep it from invalidating the core contribution.\n\nWhat's new and good: prior reviews like van den Berghe et al. touch this paradigm only in passing; this is the first dedicated SoK. The methodology is transparent: explicit search string, inclusion/exclusion criteria, and a clear diagram showing n=148 papers. The thematic synthesis is the main contribution. It catalogs domains (IoT, operating systems, networks, cyber-physical systems, cryptography, even work safety), motivations (almost always the negation: insecure defaults), and design principles. The identification of newer principles—off-by-default, zero trust, fallback/termination, guiding developers—is genuinely useful. The problems section (emergency break-glass, predictions becoming false, developer workarounds, trade-offs) maps open issues well. And the connection to the EU Cyber Resilience Act's \"secure by default\" requirement gives it practical relevance for regulators and practitioners.\n\nSoft spots: the ACM/IEEE-only search is the biggest one. The author flags it in Section 4.2 and again in 4.3, and the conclusions are explicitly scoped as exploratory. But the trend figure (Fig. 2) and the domain table (Table 2) are descriptive of that sample, not of the whole literature. That's a moderate limitation, not a deal-breaker, because the qualitative claims don't depend on statistical representativeness. A follow-up searching Scopus or Web of Science, or HCI venues like CHI and SOUPS, would be the natural next step. The single-author thematic analysis with no inter-rater reliability is a minor issue for a scoping review, and it's acknowledged. The bibliometric bits (bigrams, year counts) are basic but not load-bearing. On circularity: the handful of self-citations are background context and don't feed back into the conclusions.\n\nBottom line: this deserves a serious referee. It's a solid foundation for anyone working on security design principles or on regulatory implementation like the CRA. I'd send it out with minor revision requests, mainly pushing the discussion of what the library restriction means for each major claim—which the author already partially does.","headline":"Useful, honest first systematic map of the safe/secure-defaults paradigm; the ACM/IEEE-only sample is a real limitation but not a deal-breaker.","tokens_in":29281,"tokens_out":2825,"would_cite":true,"duration_ms":28476,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This systematization of knowledge argues that the safe-and-secure-defaults paradigm, rooted in Saltzer and Schroeder's 1975 fail-safe defaults principle, has been extensively discussed, applied, and extended across computing domains since…","keywords":["safe and secure defaults","fail-safe defaults","security design principles","systematic mapping study","scoping review","Internet of Things security","Saltzer and Schroeder","default settings"],"falsifier":"Run the same Boolean query and inclusion criteria in Scopus, Web of Science, or domain-specific venues such as SOUPS, CHI, and safety-engineering journals and compare the corpus. If substantial primary studies on safe and secure defaults appear before the late 1990s, if the mid-2010s acceleration disappears outside ACM/IEEE, or if a well-developed body of empirical work on default-setting behavior shows up, the review's claims about the paradigm's timeline, IoT-driven growth, and the folklore of human factors would be contradicted.","tokens_in":28295,"feed_emoji":"🔐","tokens_out":8240,"duration_ms":73831,"temperature":0.7,"pith_summary":"This paper is a systematization of knowledge that maps 148 peer-reviewed papers discussing safe and secure defaults, the design rule that systems should start in a locked-down, safe state rather than an open one. The review shows that the paradigm, born as the fail-safe-defaults access-control principle in Saltzer and Schroeder's 1975 paper, has been continuously discussed and extended since the late 1990s, with publication volume accelerating from the mid-2010s onward as Internet-of-Things insecurity made the topic urgent. The same survey catalogs how the paradigm has expanded beyond access control into networking, cryptography, operating systems, cyber-physical systems, and even work safety, and how newer principles such as off-by-default, automated default generation, and zero-trust verification have been added to the original eight. A sympathetic reader would care because the review gives researchers, practitioners, and regulators a working map of a doctrine that is now written into recent cyber-security regulation, while also exposing how little rigorous empirical evidence backs the behavioral assumptions attached to defaults.","feed_headline":"148 papers chart the rise of the 'safe defaults' design rule","feed_subtitle":"The principle born in 1975 now spans IoT, zero trust, and new EU law — but evidence for its human side stays thin.","key_machinery":"The load-bearing apparatus is the review protocol itself: a systematic mapping study and scoping review run against ACM's and IEEE's electronic libraries using the Boolean query $(\\text{safe AND default}) \\lor (\\text{secure AND default})$, filtered by four exclusion criteria (primary studies only; substantive discussion of defaults with definition, example, rationale, or elaboration; peer-reviewed journals and conference proceedings; no finance papers where 'default' means non-payment), yielding a corpus of 148 papers analyzed thematically. The conceptual backbone that organizes the findings is Saltzer and Schroeder's 1975 fail-safe-defaults principle, generalized by replacing 'access' with any resource so that the paradigm covers safety as well as security; the review's tables of contextual domains, motivating themes, design principles, and problems carry the argument that the paradigm has both spread and evolved.","core_discovery":"On the paper's own terms, the central discovery is that the safe-and-secure-defaults paradigm is not a static historical footnote but a live, expanding design doctrine. Reviewing 148 primary studies gathered from the ACM and IEEE digital libraries with the query $(\\text{safe AND default}) \\lor (\\text{secure AND default})$ and strict inclusion criteria, the author concludes that the paradigm has been extensively discussed, used, and developed further since the late 1990s; that its growth accelerated from roughly the mid-2010s, driven substantially by the perceived insecurity of IoT devices; and that it has been applied across a wide range of computing domains while being extended with principles the originators did not consider, including off-by-default, turning security features on by default, overriding and fallback designs, clean-up and leak-prevention routines, automated generation of secure defaults, and zero-trust assumptions. The review also documents recurring problems, such as emergency access overrides, the argument that today's secure default becomes tomorrow's vulnerability, developer workarounds, and security-performance trade-offs, and it notes that most claims about human behavior attached to defaults are speculative folklore rather than robust empirical findings.","pith_inferences":["One extension the paper leaves implicit is that the ACM/IEEE restriction may explain the absence of organizational and social-science perspectives: a reader shopping for literature on default settings would find a large body of work in management, psychology, behavioral economics, and policy journals that this sample simply does not include.","Because the review treats defaults as largely static, a natural testable extension is to give defaults a temporal dimension—sunset clauses, dynamic re-defaulting, or periodic re-evaluation—so that 'today's secure default becomes tomorrow's vulnerability' becomes a design requirement rather than a critique.","The folklore gap in human-factor claims suggests a concrete research program: instrument real systems to measure what happens when users and developers actually encounter secure defaults, using A/B experiments of the kind one paper in the sample used for product features, to replace speculation with effect estimates.","A further extension: since the paradigm is now encoded in regulation, one could audit shipped products for default settings that violate the cataloged principles (deny-by-default, off-by-default, least privilege), turning the SoK's taxonomy into a compliance checklist."],"forward_implications":["If the review is right, the EU Cyber Resilience Act's 'secure by default configuration' requirement has a concrete body of engineering knowledge behind it, from off-by-default networking to automated credential generation, that implementers and auditors can draw on.","If the review is right, designers in domains far from 1970s access control—IoT, cyber-physical systems, web security, cryptography—can legitimately treat safe and secure defaults as a general engineering doctrine rather than a specialized historical principle.","If the review is right, the paradigm's expansion means a full catalog of security design principles, with overlapping terms unified (psychological acceptability as least surprise, isolation as compartmentalization), would be a high-value next step for research and practice.","If the review is right, the empirical gap matters: regulations and designs that assume users and developers stay with defaults out of inertia or out of information conveyed by defaults rest on claims the literature has not yet tested rigorously.","If the review is right, configuration vulnerabilities in cloud computing and other newer domains are under-covered, and secure defaults are the natural starting point for mitigating that class of failures."],"supporting_citations":[{"why":"Supplies the original fail-safe-defaults principle and the eight design principles that define the paradigm's starting point.","marker":"[1]"},{"why":"Closest prior systematic review of security design notations, used to mark the gap the SoK fills.","marker":"[2]"},{"why":"Mirai botnet case, the anchor for the review's claim that IoT insecurity drove the mid-2010s acceleration of the literature.","marker":"[70]"},{"why":"Survey of IoT vulnerabilities and insecure default configurations, load-bearing for the IoT domain's centrality in the sample.","marker":"[73]"},{"why":"Policy-through-software-defaults paper that grounds the review's connection between the paradigm and regulation.","marker":"[76]"},{"why":"The C-language 'disciplined opt-in, no safe default' argument that defines the opt-in motivation category in the review.","marker":"[127]"},{"why":"Provides the 'today's secure default becomes tomorrow's vulnerability' argument used in the problems-with-predictions theme.","marker":"[134]"},{"why":"Explicitly questions whether 'deny access' still counts as a fail-safe default, one of the few papers directly discussing the paradigm.","marker":"[135]"},{"why":"Unhelpful-assumptions argument that the review aligns with in criticizing the folklore status of human-factor claims.","marker":"[169]"}],"fun_headline_variants":["Safe defaults doctrine expands: 148 studies map its growth","From 1975 to IoT: 148 papers on secure defaults","Secure defaults evolve: zero trust and off-by-default rise","148 studies: secure defaults doctrine grows, evidence thin"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole picture depends on the search being run only in ACM's and IEEE's electronic libraries; if much of the research on safe and secure defaults lives in other venues, such as human-computer interaction, social science, or safety engineering, the identified timeline, principles, and gaps would be different.","fun_headline_variants_meta":{"raw":{"variants":["Safe defaults doctrine expands: 148 studies map its growth","From 1975 to IoT: 148 papers on secure defaults","Secure defaults evolve: zero trust and off-by-default rise","148 studies: secure defaults doctrine grows, evidence thin"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000578,"raw_usage":{"total_tokens":2732,"prompt_tokens":956,"completion_tokens":1776,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":1708}},"tokens_in":572,"tokens_out":1776,"duration_ms":11169,"temperature":1.0,"reasoning_tokens":1708,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:34:27.522002+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same Boolean query and inclusion criteria in Scopus, Web of Science, or domain-specific venues such as SOUPS, CHI, and safety-engineering journals and compare the corpus. If substantial primary studies on safe and secure defaults appear before the late 1990s, if the mid-2010s acceleration disappears outside ACM/IEEE, or if a well-developed body of empirical work on default-setting behavior shows up, the review's claims about the paradigm's timeline, IoT-driven growth, and the folklore of human factors would be contradicted.","supporting_citations":[{"cited_title":"Some Were Meant for C: The Endurance of an Unmanageable Language,","cited_arxiv_id":null,"evidence_quote":"The C-language 'disciplined opt-in, no safe default' argument that defines the opt-in motivation category in the review."},{"cited_title":"Developer-Centered Security and the Symmetry of Ignorance,","cited_arxiv_id":null,"evidence_quote":"Provides the 'today's secure default becomes tomorrow's vulnerability' argument used in the problems-with-predictions theme."},{"cited_title":"Is “Deny Access","cited_arxiv_id":null,"evidence_quote":"Explicitly questions whether 'deny access' still counts as a fail-safe default, one of the few papers directly discussing the paradigm."},{"cited_title":"Unhelpful Assumptions in Software Security Research,","cited_arxiv_id":null,"evidence_quote":"Unhelpful-assumptions argument that the review aligns with in criticizing the folklore status of human-factor claims."}],"review_version":1}