{"id":"e0ec7c7f-3419-4cdb-8bf0-e2dd1d35fa9a","arxiv_id":"2506.16848","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In Czech nursery schools, drilling at least annually is associated with fewer reported evacuation issues, and slow movement by children needing help is the most common problem.","lead":"A survey of 1,151 Czech nursery schools maps how preschools run fire drills: most drill about once a year, and slow movement because children need help is the most common problem. Schools that drill annually or more often report fewer evacuation issues, which gives safety planners a concrete argument for more frequent practice.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The frequency–issues association (p=0.023) does not survive multiple-testing correction and is driven mainly by a small 'less often' group, so the abstract's causal wording outruns the cross-sectional evidence.","rationale":"Stress-testing this paper in good faith: the survey is unusually large (1,151 respondents, 21.5% of registered Czech nursery schools), the instrument is fully included, and the authors transparently list limitations in Section 2.3. The descriptive results (e.g., 76.5% perform drills; slow movement is the most common issue) are credible and useful. The central inferential claim, however, is more fragile than the abstract suggests. The reader focused on non-response/response bias; I agree that this is a general threat, but the more decisive, internally checkable problem is that the only significant test supporting the 'more frequent drills → fewer issues' claim (Table 9, p=0.023) is unadjusted for multiple comparisons and is driven by a small 'less often' group. The reader's rationale does flag multiple testing, so we partially agree. My proposed check—correcting for multiplicity, testing the ordered trend, and excluding rare categories—would settle whether the association survives. If it does not, the abstract and Section 4.1 must be reworded. Since the paper's value is primarily descriptive and the recommendations can be softened without losing the contribution, the appropriate verdict remains conditional; hence UNCHANGED relative to the reader.","tokens_in":23230,"tokens_out":5717,"duration_ms":56485,"concrete_test":"Re-run the Section 3.3.2 analysis using the raw data from Table 9 with (1) a Bonferroni or Benjamini-Hochberg correction applied to all p-values in Tables 7–11, (2) a Jonckheere-Terpstra trend test across the ordered categories 'less often < annually < more often', and (3) a sensitivity analysis dropping the 3 schools reporting 'more than 2x per year' and/or merging 'less often' with 'annually'. If the corrected test, the trend test, or the sensitivity variant loses significance at α=0.05, the abstract and Discussion (Section 4.1) must replace causal wording with 'schools that drill less than annually report more issues—an association that does not survive adjustment.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result is the Kruskal-Wallis comparison in Table 9 (Section 3.3.2): schools drilling less than annually report 1.45 issues on average versus 1.03 (annual) and 1.01 (more often), p=0.023. This p-value is unadjusted despite dozens of tests across Tables 7–11; a Bonferroni correction over the categorical tests in Tables 8 and 9 alone would require p<0.0025, making the result non-significant. The 'less often' group is small (about 58 of 872 schools, ~6.7%), and the annual and more-frequent groups are nearly identical, so the data do not show a dose-response for more-than-annual drilling. Because drill frequency and issue counts are self-reported by the same respondents, common-method and social-desirability bias could create or inflate the association; Section 2.3 acknowledges response and non-response bias but does not quantify the bias magnitude needed to explain the observed gap. The abstract's 'led to' wording goes beyond what a cross-sectional, self-report survey with a 23.5% response rate can support. The descriptive mapping of Czech nursery-school drill practices remains valuable, but the causal/inferential claim should be demoted unless the analysis survives adjustment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents an online survey of 1,151 Czech nursery schools (23.5% response rate) on fire-drill practices, operational conditions, and reported evacuation problems. The authors describe drill frequency, announcement practices, warning signals, movement organization, and issues such as slow movement and frightened children. They test associations between school characteristics and drill outcomes, finding that schools drilling less than annually report more issues (mean 1.45 vs 1.03 and 1.01), that ground-floor schools report fewer issues, and that specific issues (e.g., waiting for a free staircase) relate to building height and escape-route availability. The paper concludes with recommendations for more frequent, purpose-designed, and child-adapted fire drills. The abstract states that 'more frequent fire drills led to significantly fewer issues during evacuation.'","tokens_in":23489,"tokens_out":5821,"duration_ms":59061,"significance":"If the descriptive findings stand, the paper is a useful contribution to the limited empirical literature on fire evacuation in preschools, particularly for Czechia. The large sample, regional coverage, and transparent reporting of the survey instrument and limitations are strengths, and the non-parametric methods are appropriate for the count and categorical data. However, the headline causal claim is not supported by the cross-sectional, self-report design; the descriptive mapping is the more solid and valuable part of the paper.","major_comments":[{"comment":"The abstract states that 'more frequent fire drills led to significantly fewer issues during evacuation,' and Section 4.1 repeats this as 'our results confirmed that nursery schools that performed fire drills annually or more often encountered significantly fewer problems during the drills.' These causal statements are not warranted by the study design: the data are a cross-sectional survey of self-reported drill frequency and self-reported past issues, collected at a single time point from the same respondents. The limitations section (Section 2.3) explicitly acknowledges non-response and response bias, which makes a causal reading even less defensible. I request that all causal wording be replaced with associational wording (e.g., 'was associated with,' 'reported by schools with'), or that the authors provide a specific design-based argument for causal identification, which is not currently present.","section":"Abstract; Sections 3.3.2, 4.1, 5"},{"comment":"The headline result—that schools with less frequent drills report more issues—rests on a single Kruskal-Wallis test with p=0.023. The paper performs many such tests without any adjustment for multiple comparisons. For example, correcting only for the categorical comparisons in Tables 8 and 9 (12 tests) would require p<0.0021, and correcting across all tests in Tables 6-11 would make p=0.023 clearly non-significant. In addition, the effect is driven by the small group of schools that drill less than once a year (n≈58, 6.7% of Part II respondents); the annual and more-than-annual groups have nearly identical means (1.03 vs 1.01), so there is no evidence of a dose-response gradient. The authors should either apply a multiple-testing correction (or a preregistered family of tests), explicitly label the analyses as exploratory, and/or present effect sizes with confidence intervals. The current formulation overstates the evidential strength.","section":"Table 9 (Section 3.3.2); also Tables 6-11"},{"comment":"Both the predictor (frequency of fire drills, Question #11) and the outcome (number of issues, Question #20) are self-reported by the same respondent in the same survey, and the outcome is a count of predefined issue categories rather than an objective measure. This invites common-method bias—for example, respondents with a more systematic approach to fire safety might report both more frequent drills and a higher number of noticed issues, or social desirability might lead to underreporting of issues. Section 2.3 lists response bias as a general limitation but does not discuss its likely direction or magnitude for the frequency–issues association. The authors should add a sensitivity discussion, or a simple tipping-point analysis, and should explicitly frame the association as self-reported and subject to unmeasured confounding.","section":"Sections 3.2.2 and 3.3.2; Questions #11 and #20"}],"minor_comments":[{"comment":"Typos appear in the text and figure captions, including 'T op left' in Section 3.2.1, 'W arning' in Table 5 variable labels, and 'frightene' in Section 3.3.3; a full proofread is recommended.","section":"Section 3.2.1 and Section 3.3.3"},{"comment":"Reference [52] is a placeholder ('J. Doe') and lacks a title and source; this should be replaced with the actual citation for early childhood education in Czechia.","section":"References"},{"comment":"Table 9 uses the labels 'Less often,' 'Annually,' and 'More often' that are less specific than the categories in Table 5; the grouping rule (e.g., 'Less often' means less than once a year, 'More often' means twice a year or more) should be stated in the table note or text.","section":"Tables 5 and 9"},{"comment":"The abstract's statement of '23.5% of nursery schools invited to participate' is ambiguous because the methods clarify that this is the percentage of successfully delivered invitations (4,903); consider saying 'of invited schools that received the invitation.'","section":"Abstract and Section 2.1"},{"comment":"The repeated header note 'This is a preprint version of the article accepted to Safety Science with DOI...' appears at the top of the abstract and again at the end; in the final published version this should be standardized or removed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's descriptive survey data are likely of interest to the Safety Science readership, and the authors are commendably transparent about limitations. My main concern is the gap between the causal language in the abstract/conclusions and the associational evidence. The revision requests are achievable without new data collection, so I see this as a major revision rather than a rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this paper is worth having on file for its descriptive dataset, but the headline claim about drill frequency causing fewer evacuation problems is not supported by its own analysis. The dataset—1,151 Czech nursery schools, 23.5% response—is genuinely new and the first national mapping of drill practices in this occupancy type. The methods are clearly documented, the tables are legible, and Section 2.3 candidly lists non-response and response bias. The descriptive findings (annual drills dominate, slow movement is the top issue, ground-floor schools report fewer issues) are useful to practitioners and researchers.\n\nThe soft spot is the inferential framing. The abstract says more frequent drills 'led to' fewer issues, but this is a cross-sectional, self-report survey. The Kruskal-Wallis result in Table 9 (p=0.023) compares a small 'less often' group (58 of 872 schools) against annual and more-frequent groups that are nearly identical (1.03 vs 1.01 issues). There is no dose-response beyond the annual threshold. And that p-value is unadjusted among dozens of tests across Tables 7–11; a Bonferroni correction would sink it. The same applies to the ground-floor association (p=0.010) in Table 8. These are plausible patterns worth reporting as descriptive trends, but not as tested hypotheses. The discussion section also stretches into recommendations—gamification, escape-route training, staff education—that may be sensible but go beyond what this survey can evaluate.\n\nI wouldn't reject the paper because of this; the dataset earns its place. But the causal wording should be revised and the multiple-testing issue addressed before I'd cite the statistical claims. For your own use, treat the descriptive statistics as the product and skip the causal sentence in the abstract.","headline":"A valuable national survey dataset undercut by a causal claim that the cross-sectional, self-report design cannot support.","tokens_in":23957,"tokens_out":2476,"would_cite":false,"duration_ms":26688,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Surveying 1,151 Czech nursery schools, the paper argues that schools which drill less than once a year report more evacuation problems than schools that drill at least annually, and that slow movement because children need help is the…","keywords":["fire safety","fire drills","nursery schools","preschool children","evacuation","survey research","Czechia","human behavior"],"falsifier":"Compare the survey's self-reported drill frequencies and issue counts against official fire-safety documentation or school drill logs for a random sample of both respondents and non-respondents. If non-respondent schools or logged records show no gap in problems between less-than-annual and annual-or-more drillers, or reveal that reported frequencies are inaccurate, the central association would not survive.","tokens_in":23061,"feed_emoji":"🚸","tokens_out":6066,"duration_ms":66346,"temperature":0.7,"pith_summary":"This paper sets out to establish what fire-drill practice actually looks like in Czech nursery schools and whether drill frequency is tied to how smoothly evacuations go. Drawing on an online survey of 1,151 nursery-school leaders (23.5% of those invited), it claims that schools drilling less than once a year report on average 1.45 distinct problems during drills, against about 1.0 for schools drilling annually or more often (p=0.023). It also argues that ground-floor-only schools encounter fewer problems, that slow movement because children need assistance is the single most common issue, and that drills should be embedded in age-specific fire-safety education rather than treated as isolated events. A careful reader would care because preschool children are a vulnerable population whose evacuation depends on adult decisions, and there is little standardized guidance for drills in such settings.","feed_headline":"Fewer fire drills, more evacuation trouble in nursery schools","feed_subtitle":"Survey of 1,151 Czech nursery schools links sub-annual drills to more problems and flags stairs as the toughest spot.","key_machinery":"The load-bearing object is the survey instrument: a 20-question anonymous online questionnaire in two parts, the first probing building, occupancy and operational conditions of 1,151 Czech nursery schools and the second asking the 880 schools that do drill about frequency, announcement, warning signals, movement organization, route use, evaluation and a fixed list of ten possible issues. The argument is carried by a count of issues per school and by non-parametric comparisons (Wilcoxon, Kruskal-Wallis, chi-square and Spearman tests) of that count and of individual issues against drill frequency, floor level, staff-to-child ratio and other indicators. That machinery turns self-reported practice into a quantitative frequency-issues association.","core_discovery":"The paper's central claim is that drill frequency predicts evacuation problems in nursery schools: schools that held drills less often than once a year reported a mean of 1.45 issues during drills, compared with 1.03 for annual drills and 1.01 for more frequent drills (Kruskal-Wallis p=0.023). The same pattern appears for children's fear, with frightened children reported as an issue in 37.9% of schools drilling less than annually, 23.5% of annual-drill schools and 17.6% of more-frequent ones (p=0.015). The paper also reports that slow movement because children required assistance was the most common problem (41% of respondents), that children often needed help on stairs, and that schools with all children's spaces on the ground floor had fewer issues (mean 0.94) than schools with spaces on the first floor (1.10) or higher (1.26, p=0.010).","pith_inferences":["A direct test would link self-reported drill frequency to independently timed evacuations or drill logs; the survey's reliance on self-report is the main open flank.","If the frequency-issues relationship generalizes, policy bodies could pilot different drill frequencies across comparable schools and compare issue counts or evacuation times.","The announcement effect (unannounced drills showing less slow movement) may reflect that unannounced drills are run by more experienced or better-prepared schools; this confound is not resolved by the survey."],"forward_implications":["If drill frequency is genuinely protective, annual drills should be a floor, not a ceiling, for nursery schools; the paper recommends going beyond the common annual schedule.","Buildings with children's rooms above the ground floor should expect more problems and likely need additional training on stair evacuation and stricter design requirements.","Because slow movement due to child assistance is the most frequent problem, drills should specifically rehearse stair movement and assistance strategies for younger children.","Standardized post-drill evaluation and reporting should be part of the drill routine, since only about 61.6% of schools evaluate in writing.","Fire drills should be one component of a systematic, age-appropriate fire-safety education program rather than standalone events."],"supporting_citations":[{"why":"Defines the two core purposes of drills (training occupants and evaluating performance) and the design and evaluation pitfalls the paper's recommendations build on.","marker":"[2]"},{"why":"Documents how routine drills differ from real evacuations, serving as the basis for the paper's call for realistic and varying drill scenarios.","marker":"[8]"},{"why":"Supplies the international table of required drill frequencies and multi-story fire-safety considerations that frame the Czech comparison.","marker":"[20]"},{"why":"Shows that stair descent is the most challenging task for preschool children, matching the paper's slow-movement and stair-assistance finding.","marker":"[27]"},{"why":"Provides an earlier preschool and day-care survey whose drill-frequency and staff-ratio questions this study extends.","marker":"[28]"},{"why":"Describes limits and future directions for evacuation drills, referenced when the paper argues drills must be integrated into broader fire-safety education.","marker":"[12]"}],"fun_headline_variants":["Rare fire drills mean more nursery evacuation problems","Annual drills cut evacuation issues in Czech nursery schools","Nursery fire drill frequency predicts evacuation trouble","Stairs, rare drills pile up nursery evacuation problems","Preschool evacuations suffer when fire drills are infrequent"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that the 23.5% of invited nursery-school leaders who answered are representative of all Czech nursery schools and that their self-reported drill frequencies and problem lists are accurate enough for the comparisons to reflect real practice, not who chose to respond or how they wished to appear.","fun_headline_variants_meta":{"raw":{"variants":["Rare fire drills mean more nursery evacuation problems","Annual drills cut evacuation issues in Czech nursery schools","Nursery fire drill frequency predicts evacuation trouble","Stairs, rare drills pile up nursery evacuation problems","Preschool evacuations suffer when fire drills are infrequent"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000608,"raw_usage":{"total_tokens":2819,"prompt_tokens":918,"completion_tokens":1901,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":1827}},"tokens_in":534,"tokens_out":1901,"duration_ms":13782,"temperature":1.0,"reasoning_tokens":1827,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:37:20.928678+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the survey's self-reported drill frequencies and issue counts against official fire-safety documentation or school drill logs for a random sample of both respondents and non-respondents. If non-respondent schools or logged records show no gap in problems between less-than-annual and annual-or-more drillers, or reveal that reported frequencies are inaccurate, the central association would not survive.","supporting_citations":[{"cited_title":"Gwynne, E","cited_arxiv_id":null,"evidence_quote":"Defines the two core purposes of drills (training occupants and evaluating performance) and the design and evaluation pitfalls the paper's recommendations build on."},{"cited_title":"Kinateder, C","cited_arxiv_id":null,"evidence_quote":"Documents how routine drills differ from real evacuations, serving as the basis for the paper's call for realistic and varying drill scenarios."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the international table of required drill frequencies and multi-story fire-safety considerations that frame the Czech comparison."},{"cited_title":"Najmanov´ a, E","cited_arxiv_id":null,"evidence_quote":"Shows that stair descent is the most challenging task for preschool children, matching the paper's slow-movement and stair-assistance finding."}],"review_version":1}