{"id":"c94b5d95-6eb8-4ace-a447-fc727f6c29f4","arxiv_id":"2506.15525","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A participatory design workshop with 17 US high school students yields six student-derived guidelines for designing generative AI tools and policies in high schools.","lead":"This paper reports a participatory design workshop in which 17 US high school students designed GenAI tools and school policies to address bias, misinformation, cheating, and over-reliance. It proposes six actionable design guidelines for EdTech developers based on those student perspectives.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The six guidelines are only as strong as the sample: 17 participants recruited via AI/robotics mailing lists, 16 with prior GenAI experience, 58.8% from private schools. A diverse replication is needed before treating these as broadly actionable EdTech guidelines.","rationale":"I considered challenging the grounding of individual guidelines (e.g., multilingual support appears in Guideline 3 without a corresponding student quote in §4), but this is a lesser issue because the paper frames guidelines as researcher translation of insights, not verbatim student output. Similarly, the lack of raw data is a transparency concern, not a logic flaw. The sample-transferability question is the one assumption whose failure would invalidate the central contribution, since the guidelines are the paper's main deliverable. The reader's conditional verdict is appropriate: the study is methodologically careful, the limitations section is honest, and the findings are credible for the studied group, but the guidelines require replication before being adopted as general standards. Therefore no verdict change is needed; the existing CONDITIONAL designation already captures the risk.","tokens_in":22685,"tokens_out":4807,"duration_ms":63525,"concrete_test":"Run a replication of the same seven-step workshop protocol with a stratified sample (e.g., at least 30-50 students) drawn from public schools, students with no prior GenAI interest, and at least two non-U.S. regions. Compare the emergent problem spaces and map the resulting design recommendations against the six published guidelines. If the same six problem spaces and six guidelines emerge unchanged, the transferability objection is answered; if new themes appear or priorities shift, the published guidelines should be re-scoped as context-specific rather than generalizable EdTech guidance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that high school students' participatory-design insights yield actionable guidelines for GenAI EdTech designers (Abstract; §5.2). For that claim to hold, the concerns expressed by the 17 participants must transfer to high school students broadly. That transferability is not established. Recruitment was through AI and robotics interest mailing lists, all participants were U.S.-based, 58.8% attended private schools (versus an 18.1% national average), and 16 of 17 already used GenAI text tools (§3.1, Table 1, §6). The workshop itself was also shaped by themes from prior teacher interviews and a GenAI primer (§3.2), so the problem spaces students brainstormed may reflect a self-selected, framer-shaped sample. The authors acknowledge these limits in §6 and frame the work as providing 'deep, nuanced insights' rather than universal findings. However, the abstract and the guidelines in §5.2 are presented as designer-facing guidance for high-school EdTech generally, with no scope delimiter like 'for AI-interested U.S. students.' If a broader sample produces different priorities or guidelines, the claimed gap-filling contribution would not hold as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a participatory design workshop with 17 U.S. high school students (ages 14–17), in which participants brainstormed concerns about GenAI in education, designed GenAI tools, and developed school policies. From the workshop data, the authors identify three themes (AI tool features, school/classroom use, regulations) and four student-designed policy proposals, and they derive six guidelines for educational technology designers. The paper argues that these guidelines are more actionable than existing UNESCO and U.S. Department of Education frameworks and that student voices should be included in school AI policy development. The study is positioned as addressing the underrepresentation of high school students in participatory design research on GenAI.","tokens_in":22866,"tokens_out":2476,"duration_ms":30440,"significance":"If the findings are transferable, the paper makes a useful contribution by foregrounding high school students' perspectives on GenAI, an underrepresented group in EdTech participatory design. The workshop design is well documented, the data analysis follows standard inductive coding with two independent coders and discrepancy resolution, and the paper transparently acknowledges its sample limitations in Section 6. The proposed guidelines are concrete and could provide a starting point for designers. However, the significance is tempered by the narrow, self-selected sample and by the fact that several guideline recommendations go beyond what the student data directly support.","major_comments":[{"comment":"The abstract and Section 5.2 present the six guidelines as general designer-facing guidance for GenAI EdTech in high schools, without a scope delimiter such as 'for AI-interested U.S. students.' Yet Section 6 acknowledges that participants were recruited through AI/robotics mailing lists, 58.8% attended private schools, 16 of 17 had prior GenAI experience, and all were U.S.-based. This is load-bearing because the transferability of the guidelines to the broader high school population is not established. The claims should be tempered to reflect the sample's scope, or the paper should provide a clearly argued basis for why these specific concerns are likely to generalize.","section":"Abstract and §5.2"},{"comment":"The workshop structure was shaped by themes from prior teacher interviews and included a GenAI presentation that covered example uses, benefits, risks, and strategies for exploring bias before the problem-space brainstorming. While facilitators avoided guiding discussion, this prior framing may have primed students to focus on particular problem spaces. More importantly, Guideline 4 recommends a peer-to-peer 'AI Ambassadors' model in which students help teachers learn AI, but the student data in Section 4.2 only show that students worried about teacher AI literacy and suggested that teachers take time-consuming classes; they did not propose peer teaching. This guideline appears to over-reach the data and needs either additional support from the transcripts or a clearly labeled inferential step.","section":"§3.2 and §5.2.4"},{"comment":"Guideline 1 recommends that GenAI tools include 'educational modules on data privacy' and 'regular prompts about data privacy.' In the findings, students emphasized consent, data deletion after graduation, and limiting data access (Section 4.3), but there is no quoted or reported student suggestion for educational modules or regular privacy prompts. If this recommendation is an author inference rather than a direct student value, it should be presented as such, with reasoning, rather than as something 'our findings revealed.' The same issue occurs in Guideline 3, where 'minimal internet requirement' and 'multilingual support' are recommended; Section 4.2 mentions cross-device accessibility but not offline functionality or multilingual support. These guidelines mix data-driven insights with designer judgment without separating the two.","section":"§5.2.1"},{"comment":"The paper describes an inductive coding process with two independent coders and discrepancy resolution, but it reports no inter-rater reliability metrics, such as Cohen's kappa or percent agreement. For a study whose central contribution is a set of themes and guidelines, the absence of any agreement measure makes it difficult to assess coding reliability. The survey results in Section 4.5 are also purely descriptive, with no statistical tests; this is acceptable for a small qualitative sample, but the paper should explicitly state that quantitative claims are limited to descriptive summaries and should avoid implying that pre-post changes are statistically significant.","section":"§3.4 and §4.5"}],"minor_comments":[{"comment":"In Guideline 1, 'contexual knowledge' is a typo; it should read 'contextual knowledge.'","section":"§5.2.1"},{"comment":"In Table 1, the 'Previous AI Experience' row lists 'No Experience 1 5.9%' even though the text in Section 3.1 says '16 students had prior experience with GenAI,' so the table is internally consistent; however, the percentages in the table do not sum to 100% because respondents could select multiple options, which should be noted directly in the table caption.","section":"Table 1"},{"comment":"The sub-theme is inconsistently named 'The Roles of Teachers & Parents' in Table 2 but 'The Role of Teachers & Parents' in Section 4.4.2 and Section 5.1.4; please standardize the terminology.","section":"§4.4.2 and §5.1.4"},{"comment":"Guideline 6 introduces a 'Transparent AI Certification' program and an 'AI Accountability Dashboard' as concrete mechanisms, but the student data support a general desire for external oversight and distrust of centralized control. The leap from these preferences to a specific certification program should be more carefully flagged as a design proposal developed from, but not directly generated by, the student input.","section":"§5.2.6"},{"comment":"The limitations section is candid but could also mention that the workshop was a single-session event, so the durability of students' stated preferences and the stability of the derived guidelines over time are unknown.","section":"§6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a good fit for IDC and the participatory design community. The author team includes researchers from MIT RAISE, and some prior work on K-12 AI guidelines is cited with author overlap; this is not improper, but the framing that existing frameworks were not co-designed with students should be checked against the cited RAISE report to ensure the contribution is clearly distinguished. The main revision needed is to align the scope of the claims with the sample and to separate data-derived findings from author-proposed design recommendations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a well-run participatory design study with an honest methods section, and the six guidelines are worth taking seriously as hypotheses. The main limitation is exactly the one the authors name: a small, self-selected, U.S.-centric sample, 58.8% private school, all recruited through AI/robotics lists, 16 of 17 already using GenAI. That doesn't sink the paper, because the authors frame it as deep insight rather than universal truth, but it does mean the headline guidelines are not yet established for high-school students broadly.\n\nWhat's genuinely new: high school students are rarely included in EdTech PD, and almost never for GenAI policy. The workshop structure is thoughtful: affinity diagramming, tool design, skits, policy design. The qualitative analysis is standard inductive coding with multiple coders and discrepancy resolution. The guidelines (transparent consent, collaborative integrity, adaptive accessibility, teacher PD via student AI ambassadors, balanced integration, third-party certification) are concrete and clearly traceable to student quotes. Good credit: the paper doesn't oversell. Section 6 is direct about recruitment bias, and the discussion explicitly frames the contribution as insights, not generalizable findings.\n\nSoft spots in proportion: the main one is the scope mismatch between cautious limitations and the abstract/presentation of guidelines as designer-facing for high school generally. A scope qualifier in the abstract would fix that. Also, the workshop was shaped by preceding teacher interviews and an AI primer; the paper says facilitators avoided steering, but you can't fully rule out framing effects. Minor: no inter-rater reliability metric, but that is common in qualitative HCI and not a fatal gap. Survey data are descriptive, no statistical tests, which is fine given the n.\n\nOne thing I'd push back on relative to the stress-test note: the transferability worry lands, but the authors already flag it. It is not a hidden flaw. The central empirical claim—these students, through PD, generated these concerns and guidelines—holds up for the participants studied. Whether the guidelines generalize is a validation question, not a correctness question.\n\nWho this is for: EdTech designers, HCI researchers working on PD with teens, school policy people. It deserves a serious referee. I would send it out, with the expectation that revision tightens the scope language. I'd cite it if I were working on student-centered AI policy.","headline":"A candid, well-run PD study with six useful but not yet generalizable guidelines; send it to review, but ask for scope qualifiers.","tokens_in":23429,"tokens_out":1731,"would_cite":true,"duration_ms":21517,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Seventeen high school students designed GenAI tools and policies, and their ideas form six concrete guidelines for educational technology builders.","keywords":["Generative AI","Participatory Design","High School Students","AI in Education","Qualitative Study","Educational Technology","AI Policy","Academic Integrity"],"falsifier":"Run the same workshop with a larger, demographically representative sample of public high school students, including students with no prior AI interest; if their problem spaces and tool and policy proposals diverge substantially from the six guidelines, the guidelines' generalizability claim is not supported.","tokens_in":22480,"feed_emoji":"🎓","tokens_out":7419,"duration_ms":77488,"temperature":0.7,"pith_summary":"Seventeen high school students took part in a participatory design workshop where they named the problems generative AI creates in school and designed tools and policies to address them. The paper argues that their self-identified concerns—bias and misinformation, crime and plagiarism, over-reliance on AI, and false accusations of academic dishonesty—translate into six actionable guidelines for educational technology designers. Those guidelines cover consent-based data practices, transparent collaboration instead of surveillance, adaptive accessibility, teacher AI literacy built with students, balanced AI use that protects foundational skills, and third-party accountability. A sympathetic reader would care because students are the primary users of these tools yet are rarely included in their design or in school AI policy.","feed_headline":"High schoolers' designs yield 6 guidelines for classroom AI","feed_subtitle":"Seventeen teens want AI to cite sources, ask consent, and give hints rather than surveil them.","key_machinery":"The load-bearing mechanism is a seven-step participatory design workshop: an introduction and pre-survey, a GenAI primer, problem-space brainstorming with affinity diagramming (grouping sticky-note concerns into themes), guided GenAI tool design worksheets with role-played skits, a benefits-and-drawbacks discussion, school policy design, and a post-survey. The workshop carries the argument by generating student-created artifacts and transcripts that are then coded inductively, turning student voices into design guidelines rather than imposing researcher priorities.","core_discovery":"The authors claim that high school students, a group largely absent from prior generative-AI design research, can produce concrete and actionable design guidance when given a structured participatory space. In the workshop, student groups developed tools such as a browser extension that cross-checks AI answers against trusted sources, a personalized tutor that gives incremental hints rather than direct answers, and systems jointly regulated by libraries and government bodies, plus four school policies covering source legitimacy, teacher AI use, parental consent, and data deletion. From these artifacts and discussions the authors derive six guidelines for educational technology designers and argue that schools should formally involve students in AI policy development.","pith_inferences":["An implication the authors leave implicit is that the same preference for system-facing fixes, if stable across larger samples, would shift the academic-integrity debate away from policing students and toward changing how tools are built.","The six guidelines could be operationalized as a scored certification checklist; a natural next study would test whether independent raters, teachers, and students interpret each guideline consistently enough for the proposed 'Transparent AI Certification' to be enforceable.","Because the workshop's problem-space brainstorming was deliberately open-ended, re-running the workshop with the GenAI primer shortened or removed would isolate how much of the students' framing was shaped by the primer itself.","A testable extension is that students' demand for source citation can be turned into a product requirement: an AI tool that retrieves and displays verifiable sources alongside each claim, which could be compared with current AI-tool designs that do not provide citations."],"forward_implications":["GenAI tools built for high schools would include built-in source citations and user-controlled data deletion as standard features rather than optional extras.","Schools would rely less on AI detectors and more on transparent, collaborative tools that let students disclose their AI use, reducing false accusations of cheating.","EdTech developers would prioritize accessibility across low-end devices and limited internet connections over raw computational power.","Teacher AI literacy programs would pair formal training with student 'AI Ambassador' co-educators, sharing the burden of keeping up with the technology.","AI tutoring tools would give incremental hints and reflection prompts instead of direct answers, preserving foundational skills while building AI proficiency."],"supporting_citations":[{"why":"Supplies the existing international guidance for GenAI in education that this paper extends with student-derived, actionable steps.","marker":"[46]"},{"why":"Supplies the developer-facing U.S. guidance the paper contrasts with its high-school-specific, student-co-designed guidelines.","marker":"[99]"},{"why":"Defines participatory design as involving affected stakeholders in the design process, the method the workshop is built on.","marker":"[11]"},{"why":"Documents that K-12 students, particularly high schoolers, are rarely included in participatory design research.","marker":"[91]"},{"why":"Documents that AI content detectors are unreliable, grounding students' concerns that feed Guideline 2.","marker":"[18]"},{"why":"Shows that over-reliance on AI can reduce independent thinking, supporting Guideline 5's balanced-use recommendation.","marker":"[15]"},{"why":"Presents student data-privacy concerns that motivate Guideline 1's consent-based data practices.","marker":"[49]"},{"why":"Supplies the digital-equity evidence behind Guideline 3's adaptive accessibility features.","marker":"[64]"},{"why":"Provides a co-design precedent showing students are primary stakeholders in educational technology.","marker":"[115]"}],"fun_headline_variants":["Teens co-design AI tools, yield 6 classroom guidelines","17 high schoolers rewrite AI rules for schools","Student-led AI design: 6 guidelines from teens","High schoolers pitch AI fixes: cite, consent, hints","Teen participatory design shapes 6 AI classroom rules"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the concerns and design preferences voiced by these 17 U.S. students—recruited through AI-interest mailing lists, mostly from private schools—represent high school students broadly, and that the workshop's GenAI primer and teacher-interview-derived scaffolding did not materially steer those concerns.","fun_headline_variants_meta":{"raw":{"variants":["Teens co-design AI tools, yield 6 classroom guidelines","17 high schoolers rewrite AI rules for schools","Student-led AI design: 6 guidelines from teens","High schoolers pitch AI fixes: cite, consent, hints","Teen participatory design shapes 6 AI classroom rules"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000112,"raw_usage":{"total_tokens":1003,"prompt_tokens":828,"completion_tokens":175,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":444,"completion_tokens_details":{"reasoning_tokens":96}},"tokens_in":444,"tokens_out":175,"duration_ms":2968,"temperature":1.0,"reasoning_tokens":96,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:53:14.325721+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same workshop with a larger, demographically representative sample of public high school students, including students with no prior AI interest; if their problem spaces and tool and policy proposals diverge substantially from the six guidelines, the guidelines' generalizability claim is not supported.","supporting_citations":[{"cited_title":"2023.Guidance for generative AI in education and research","cited_arxiv_id":null,"evidence_quote":"Supplies the existing international guidance for GenAI in education that this paper extends with student-derived, actionable steps."},{"cited_title":"Department of Education, Office of Educational Technology","cited_arxiv_id":null,"evidence_quote":"Supplies the developer-facing U.S. guidance the paper contrasts with its high-school-specific, student-co-designed guidelines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents that K-12 students, particularly high schoolers, are rarely included in participatory design research."},{"cited_title":"InProceedings of the 52nd ACM technical symposium on computer science education","cited_arxiv_id":null,"evidence_quote":"Supplies the digital-equity evidence behind Guideline 3's adaptive accessibility features."}],"review_version":1}