REVIEW 4 cited by
Software Engineering User Study Recruitment on Prolific: An Experience Report
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Online participant recruitment platforms such as Prolific have been gaining popularity in research, as they enable researchers to easily access large pools of participants. However, participant quality can be an issue; participants may give incorrect information to gain access to more studies, adding unwanted noise to results. This paper details our experience recruiting participants from Prolific for a user study requiring programming skills in Node.js, with the aim of helping other researchers conduct similar studies. We explore a method of recruiting programmer participants using prescreening validation, attention checks and a series of programming knowledge questions. We received 680 responses, and determined that 55 met the criteria to be invited to our user study. We ultimately conducted user study sessions via video calls with 10 participants. We conclude this paper with a series of recommendations for researchers.
Forward citations
Cited by 4 Pith papers
-
When AI Joins the Team! A Model of How AI Adoption Relates To Social Patterns in Software Engineering Teams
AI adoption is associated with community smells through two distinct paths: indirectly through more peer consultation in specialization work, and directly through better communication quality in coordination work.
-
Exploring Dependence, Overreliance, and Addiction Related Behaviors Associated with Large Language Model Use Among Software Engineers
Software engineers report habitual, functional dependence on LLMs and some overreliance, with addiction-like behavior appearing marginal.
-
What Motivates Whom? A Survey of Newcomers to OSS and Experienced OSS Practitioners
Demographics and motivations correlate with OSS project-selection preferences, with distinct patterns for newcomers versus experienced practitioners in a 208-person survey.
-
The Influence of Fraudulent AI-Generated Responses on Software Engineering Surveys
Suspicious or AI-assisted survey answers leave most SE quantitative distributions stable but materially change qualitative framing, code prominence, and interpretive evidence.
Discussion (0). Continue with ORCID to comment.