{"id":"0652d6e0-137a-45b1-86af-6f0fd0f78e3b","arxiv_id":"2607.24387","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"SE PhD students want to communicate for collaboration, recognition, and impact, but social anxiety, channel fragmentation, and missing guidance keep practice far behind aspiration.","lead":"Interviews with 18 software engineering PhD students show strong motivation to communicate research, blocked by anxiety, unclear audiences, weak feedback, and thin institutional support. The gap matters for how SE doctoral training and research impact are designed.","discovery_kind":"new_application","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The 15% volunteer response rate within the already-selected symposium pool likely over-samples communication-engaged students, inflating the \"strong motivation\" side of the aspiration–practice tension the paper's central claim rests on.","rationale":"The reader identified sample selection via elite doctoral symposia as the weakest assumption; I agree with the family of concern but locate the sharper instrument one level down: the 16/108 volunteer rate within that pool, which the paper does not discuss at all. This is not a manufactured worry — the paper's own numbers (§3.2) document it, and its concluding claim (\"strong motivation already in place,\" §6) rests on motivation prevalence estimates drawn from a self-selected 15% of an already-selected pool. That said, this concern does not overturn the verdict: the reader's CONDITIONAL already frames the recommendations as hypothesis-generating for a selected subgroup, and §5.6 explicitly disclaims universal representativeness. My concern refines *which* subgroup the findings describe (not merely \"DS-accepted students\" but \"communication-engaged volunteers among DS-accepted students\") and identifies the specific claim component (motivation prevalence, hence the tension framing) most exposed. The qualitative themes themselves remain credible as existence proofs of motivations and barriers; what is at risk is their prevalence and the community-level design implication. The proposed test is feasible with public data and would directly estimate the direction and rough magnitude of the bias. Verdict: UNCHANGED — the concern sharpens the conditional framing rather than demanding rejection or acceptance.","tokens_in":30257,"tokens_out":2035,"duration_ms":84664,"concrete_test":"Quantify the selection direction directly. From the 108 contacted DS authors, draw the 16 interviewees plus a matched random sample of ~30 non-respondents, and code observable public communication footprints (LinkedIn posts about their research, personal websites, blog posts, Twitter/X activity) — all publicly available. If interviewees show markedly higher baseline communication activity than non-respondents, the reported motivation prevalence and the strength of the aspiration–practice tension are inflated by self-selection, and the support recommendations should be scoped accordingly. A cheaper partial check: re-tabulate the results at the participant level (motivation × barrier matrix) to confirm the tension holds within individuals, not only in aggregate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a tension: participants are \"highly motivated\" (motivation themes supported by 16–17/18, §4.2) yet constrained by barriers, warranting tailored support systems. The reader flagged the recruitment pool (ICSE/FSE Doctoral Symposium acceptees). I think the sharper, less-addressed layer is the within-pool self-selection: 108 eligible students were contacted and only 16 agreed to a 40-minute interview explicitly about science communication (§3.2). People who volunteer for a study on a topic systematically differ from non-volunteers on engagement with that topic — this is one of the most robust findings in survey methodology. The paper's mitigation argument (§3.2, §3.4) addresses pool-level selection (\"relatively high acceptance rates\" of DS tracks) but says nothing about the ~85% who declined, who plausibly include the least motivated, most anxious, and most time-pressed students — exactly the population for whom the barrier findings and support recommendations would matter most. Consequences: (1) the near-universal motivation prevalence (16–17/18) is likely inflated, so the \"strong aspiration\" pole of the tension may be much weaker in the broader SE PhD population; (2) the composition of motivations (e.g., intrinsic advocacy vs. career calculus) may skew toward enthusiasts; (3) the policy conclusion — \"with strong motivation already in place, fostering the right conditions\" (§6) — depends precisely on motivation being widespread, which is the quantity most threatened by volunteer bias. Note the bias could cut the other way on barriers (anxious students may avoid such interviews too), but the motivation prevalence and the \"aspiration–practice tension\" framing are the load-bearing casualties.","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper reports a semi-structured interview study with 18 software engineering PhD students (16 recruited via ICSE/FSE 2024–25 Doctoral Symposium author lists, 2 pilots from personal networks), analyzed with hybrid deductive–inductive thematic analysis. It addresses three RQs — motivations, channels, barriers — and produces thematic maps: eight motivation themes (51 codes), nine barrier themes (45 codes), and a three-space channel taxonomy (scholarly/professional, public/informal, institutional/community). The central interpretive claim is an aspiration–practice tension: participants are broadly motivated (most themes supported by 16–17/18) yet constrained by audience uncertainty, channel fragmentation, mental-wellbeing and socio-cultural barriers, and limited institutional guidance. The discussion frames this as a 'visibility–vulnerability tension' and 'invisible labor,' and derives implications (mentorship, training, recognition mechanisms, psychological safety). Methodologically the study is competent for its genre: dual coding with iterative reconciliation, member checking (7/18 respondents), a Zenodo replication package, saturation discussion, and a reasonably candid threats-to-validity section.","tokens_in":30636,"tokens_out":2313,"duration_ms":83503,"significance":"If the findings hold, this is a useful empirical baseline: it is, as the authors argue, the first interview study of how early-career SE researchers perceive and practice science communication, and it produces actionable input for doctoral training, mentorship design, and recognition mechanisms. The visibility–vulnerability tension (§5.2) and the three-space channel taxonomy (§5.3) are plausible analytic contributions that later work can test. Strengths worth naming: a public replication package on Zenodo, member checking with participants, dual independent coding with reconciliation for the first eight interviews, explicit threats-to-validity discussion including the pilot-asymmetry issue (§3.4 Construct Validity), and appropriately hedged language about exploratory scope in §5.6. The main risk to significance is not internal inconsistency but sampling: the empirical base is 18 self-selected volunteers from an already research-visible pool, which bounds how far the community-level framing (and especially the §6 premise that 'strong motivation [is] already in place') can travel.","major_comments":[{"comment":"The concluding sentence — 'With strong motivation already in place, fostering the right conditions can help PhD students more fully realize their potential as communicators' — depends on motivation being widespread in the SE PhD population. But the evidence for that comes from 16 volunteers out of 108 contacted Doctoral Symposium authors (a ~15% response rate) for an interview explicitly about science communication, plus 2 network-recruited pilots. Volunteer self-selection on topic engagement is a well-documented mechanism in survey and interview methodology, and the ~85% who declined plausibly include the least motivated and most time-pressed students — precisely the population the barrier findings and support recommendations concern. §3.4 addresses pool-level selection (acceptance into DS tracks) but never discusses within-pool self-selection, and the response rate is not reported anyw","section":"§3.2, §3.4 (External Validity), §6 (Conclusion)"},{"comment":"The mitigation offered for selection effects is that ICSE/FSE Doctoral Symposia have 'relatively high acceptance rates,' but no numbers or citations are given, and high acceptance of submissions does not speak to acceptance into the interview study. Relatedly, the two pilots were recruited from the authors' personal networks (§3.1), a different sampling frame, yet are pooled into all prevalence counts (§4) on the grounds that 'no substantial changes were made to the study design.' The manuscript itself notes (§3.4, Construct Validity) that pilots were not exposed to two questions added afterward, which asymmetrically depresses prevalence for at least the Career Advancement theme. Please either report the DS acceptance-rate evidence, present the prevalence counts with and without pilots, or present the pilots' contribution qualitatively separately. This is fixable without new data collect","section":"§3.1–3.2 (recruitment and pilots), §4 (pooled counts)"}],"minor_comments":[{"comment":"Figures 3 and 5 report participant support per theme, but Figure 4 (channels) gives no prevalence counts, making RQ2's evidence harder to weigh than RQ1/RQ3. Adding per-channel participant counts (or noting why they are not meaningful) would help.","section":"§4.3 / Fig. 4"},{"comment":"After interview 8, the remaining interviews were coded independently by one author each with theme reconciliation afterward (§3.3). That is defensible for reflexive thematic analysis, but the paper would be stronger if it stated this choice explicitly (e.g., citing Braun & Clarke's position on inter-rater reliability) rather than leaving readers to infer why no agreement statistic is reported.","section":"§3.3 (coding procedure)"},{"comment":"Member checking drew responses from only 7/18 participants — itself a self-selected subset (§3.3.1). The text acknowledges this; one additional sentence noting that non-responders may have felt the maps did not represent them would make the limitation concrete rather than pro forma.","section":"§3.3.1 (member checking)"},{"comment":"Several typos in §4: '9 put out of our 18 participants' (§4.4.5, Language barriers), 'Skills & Compentencies' (§4.2.8), heading 'A version to social media' should read 'Aversion to social media' (§4.4.8), 'yong researcher' (§4.4.8), 'yong'/'Phd Year' in Fig. 2 axis. Please proofread §4 carefully.","section":"§4.4.5, §4.4.8, §4.2.8, Fig. 2"},{"comment":"The opening of §4.4 says barriers were reported 'as mentioned by most participants,' but the immediately preceding counts (10, 11, 7 of 18) mix majority and minority; tighten the wording to match the numbers.","section":"§4.4 (intro paragraph)"},{"comment":"§4.4.5 reports '10 out of 18 participants talked about various spectrum of cultural barriers' while §4.4's intro says 'Ten out of the 18 participants mentioned language, cultural differences, and social anxiety' — it is unclear whether these are the same ten or overlapping sets. Clarify the counting basis.","section":"§4.4.5 vs. §4.4 intro"},{"comment":"Related work would benefit from situating the findings against the broader ECR science-communication literature (e.g., Mason & Merga [23] is cited but the comparison of findings — e.g., self-efficacy sources vs. your visibility–vulnerability tension — is thin). A short comparative paragraph in §5.2 would sharpen the contribution claim.","section":"§2/§5.2"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is the first real interview study of how SE PhD students experience science communication—motivations, channels, and barriers together—rather than another advocacy piece or LinkedIn scrape. That gap is real, and they fill it cleanly.\n\nWhat they did well: hybrid thematic analysis with dual coding, iterative reconciliation, thematic maps, lots of quotes, a Zenodo package, and an honest threats section. The channel typology (scholarly/professional, public/informal, institutional/community) and the barrier set (audience uncertainty, channel fragmentation, language/culture, mental wellbeing, thin institutional support) are useful and grounded. The visibility–vulnerability framing in the discussion is a fair analytic label for what the transcripts show, not a gimmick.\n\nSoft spots, in proportion. The load-bearing claim is a tension: near-universal high motivation (16–17/18 on several themes) constrained by barriers, so “motivation is already there—fix the conditions.” Recruitment was ICSE/FSE Doctoral Symposium acceptances, then ~16/108 volunteers for a 40-minute interview about communication. They discuss pool bias; they barely touch within-pool volunteer bias. People who agree to talk about outreach are not a random draw from SE PhDs. Motivation prevalence is the quantity most at risk of inflation; barrier composition may skew too. Member checking was 7/18. Implications for “tailored support across diverse environments” should stay hypothesis-generating for a visible, conference-active subgroup—not community-wide policy. That does not sink the paper; it bounds it.\n\nMath/data/citations: no quantitative overclaim; related work is appropriate (general sci-comm plus their own SE outreach line); self-cites are background, not circular. No formal results to check.\n\nWho it’s for: people who supervise SE PhDs, design doctoral training, or care about research–practice transfer and research culture. Not for someone hunting a causal or field-wide prevalence estimate.\n\nI’d send it to referees. Treat the support recommendations as starting points, not proven interventions. Worth engaging if you work on SE education, impact, or early-career practice.","headline":"Solid first empirical map of how SE PhDs experience science communication; the aspiration–practice story is real in this sample, but volunteer bias likely inflates how universal the “strong motivation” pole is.","tokens_in":31225,"tokens_out":552,"would_cite":true,"duration_ms":13803,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Software engineering PhD students want to communicate their research, but anxiety, unclear audiences, and weak institutional support keep most of that motivation from turning into practice.","keywords":["science communication","software engineering","early-career researchers","PhD students","motivations","barriers","communication channels","collaboration"],"falsifier":"Interview or survey a broader SE PhD sample that includes students who never appear at top doctoral symposiums and check whether the same high motivation plus the same barrier profile (audience uncertainty, mental wellbeing, weak institutional guidance) still dominates; if motivation collapses or barriers look qualitatively different, the central aspiration–practice claim does not generalize.","tokens_in":31116,"feed_emoji":"📣","tokens_out":898,"duration_ms":20007,"temperature":0.7,"pith_summary":"This paper asks how early-career software engineering researchers actually experience science communication—not as a slogan, but in day-to-day doctoral life. Through interviews with 18 PhD students from many countries, it maps three linked dimensions: why they want to share work, which channels they use, and what blocks them. The central pattern is an aspiration–practice gap. Students are driven by collaboration, visibility and career prospects, topic advocacy, skill-building, and social encouragement, yet they stall on social anxiety, uncertain audiences and venues, thin feedback, little guidance, and a fragmented media landscape. The authors argue that the SE community needs tailored, balanced support so doctoral students can communicate with confidence across different cultures and institutions.","feed_headline":"SE PhDs want to share research—barriers block them","feed_subtitle":"Interviews show strong motivation stalled by anxiety, unclear audiences, and weak support","key_machinery":"Hybrid deductive–inductive thematic analysis of 18 semi-structured interviews, organized into three interconnected thematic maps—motivations (eight themes), communication channels (scholarly/professional, public/informal, institutional/community), and barriers (nine themes)—that jointly surface the aspiration–practice tension.","core_discovery":"Early-career software engineering researchers are strongly motivated to communicate—for collaboration and networking, professional recognition, societal impact and topic advocacy, skill development, training, practical resources, and social-emotional support—but translating that motivation into action is routinely constrained by audience uncertainty, channel fragmentation, social and cultural dynamics, mental-wellbeing pressures, negative experiences, personal disposition, and limited institutional guidance. The result is a persistent tension between aspiration and practice that calls for context-sensitive support rather than one-size-fits-all outreach advice.","pith_inferences":["If the visibility–vulnerability tension is structural, interventions that only teach ‘how to post’ without reducing social risk will underperform.","SE’s industry-facing, conference-heavy culture may make practitioner feedback scarcity a sharper barrier than in fields with clearer public audiences.","Recruiting mainly symposium-accepted students may understate barriers for less visible PhDs, so the real support gap could be larger than reported.","Making communication count in hiring and funding decisions would test whether the aspiration–practice gap shrinks when incentives align."],"forward_implications":["Doctoral education in software engineering should treat communication training—audience adaptation, storytelling, online engagement—as core skill development, not an optional extra.","Supervisors and communities need structured mentorship on when, where, and how to share research so students are not left to learn by trial and error.","Institutions and evaluation systems should recognize communication labor as real scholarly work rather than invisible add-on effort.","Support must address psychological safety and cultural differences, not only technical writing or platform tips.","Channel strategy should be designed around distinct spaces (scholarly, public, institutional) instead of assuming one platform fits all audiences."],"fun_headline_variants":["SE PhDs eager to share research—anxiety and weak support block them","Motivation high, action low: why SE doctoral researchers struggle to communicate","SE early-career researchers want impact—fragmented channels stall them","Aspiration vs practice: barriers holding back SE PhD science communication","Interviews reveal SE PhDs blocked by unclear audiences and thin guidance"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That patterns from 18 PhD students mostly recruited via major conference doctoral symposiums are a solid enough base to speak for early-career software engineering researchers and to design support across diverse settings.","fun_headline_variants_meta":{"raw":{"variants":["SE PhDs eager to share research—anxiety and weak support block them","Motivation high, action low: why SE doctoral researchers struggle to communicate","SE early-career researchers want impact—fragmented channels stall them","Aspiration vs practice: barriers holding back SE PhD science communication","Interviews reveal SE PhDs blocked by unclear audiences and thin guidance"]},"model":"grok-4.5","effort":"low","cost_usd":0.001745,"raw_usage":{"total_tokens":890,"prompt_tokens":790,"num_sources_used":0,"completion_tokens":80,"cost_in_usd_ticks":17448000,"prompt_tokens_details":{"text_tokens":790,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":20,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":790,"tokens_out":80,"duration_ms":2575,"temperature":1.0,"reasoning_tokens":20,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T16:08:25.322411+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Interview or survey a broader SE PhD sample that includes students who never appear at top doctoral symposiums and check whether the same high motivation plus the same barrier profile (audience uncertainty, mental wellbeing, weak institutional guidance) still dominates; if motivation collapses or barriers look qualitatively different, the central aspiration–practice claim does not generalize.","supporting_citations":[],"review_version":1}