Recognition: unknown
Online word-of-mouth in West Africa: the effects of snowball sampling on completion rate, respondent demographics, and survey responses
Pith reviewed 2026-05-09 14:14 UTC · model grok-4.3
The pith
Snowball sampling via word-of-mouth yields higher survey completion rates, more new users and women, and shorter responses than river sampling via direct ads.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
In a survey distributed through geo-targeted Facebook ads, many respondents arrived instead through snowball sampling via shared posts and word-of-mouth referrals. Snowball-sampled respondents completed the survey at higher rates and included a larger fraction of new users and women than river-sampled respondents. Snowball respondents also produced shorter answers and took less time to finish the survey.
What carries the argument
The contrast between river sampling (direct clicks from targeted ads) and snowball sampling (arrivals through referrals and shares) as distinct routes to the same survey.
If this is right
- Snowball sampling can increase the total number of completed surveys.
- Snowball sampling can shift the respondent pool toward more new users and women.
- Snowball respondents tend to give shorter answers and finish more quickly.
- Mixed sampling routes require researchers to separate or control for arrival method when interpreting survey data.
Where Pith is reading between the lines
- Researchers could deliberately combine both routes to balance high completion with more detailed responses.
- Shorter answers from snowball users may reduce the depth of qualitative data collected.
- The pattern of faster but less complete responses might appear in surveys on other platforms or regions where word-of-mouth spreads widely.
Load-bearing premise
Observed differences in completion rates, demographics, and response behavior are caused by the sampling route rather than unmeasured factors such as motivation or self-selection into sharing.
What would settle it
A controlled experiment that randomizes whether people receive an ad or a referral invitation and finds no difference in completion rates or demographics would falsify the claim that the sampling route itself drives the observed effects.
Figures
read the original abstract
We place geo-targeted advertisements on Facebook to encourage users to fill out an online survey, following a process known as river sampling. We discovered a large number and variety of users also came to our survey through snowball sampling, including shared social media posts and other word-of-mouth referral methods. In this article, we analyze the differences between the respondents from river and snowball sampling. We present evidence that the respondents obtained by snowball sampling are more likely to complete the survey and contain a higher fraction of new users and women than those obtained by river sampling. Additionally, the evidence indicates that users from snowball sampling give shorter responses and take less time on the survey than users from river sampling. We hope these findings provide insight for other researchers who incorporate social media strategies when fielding surveys.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript describes an online survey in West Africa fielded via geo-targeted Facebook advertisements (river sampling) that unexpectedly also attracted respondents through word-of-mouth referrals (snowball sampling). It reports that snowball respondents exhibit higher survey completion rates, a higher share of new users and women, shorter open-ended responses, and less time spent on the survey than river-sampled respondents, and suggests these patterns offer practical insights for social-media-based survey design.
Significance. If the reported differences can be shown to arise from the sampling route rather than from self-selection or unmeasured respondent traits, the work would supply useful descriptive evidence for researchers conducting surveys in West Africa or similar settings. The observational comparison is straightforward and the topic is timely for digital survey methodology, but the absence of controls or modeling for selection processes limits the strength of any causal interpretation.
major comments (2)
- [Abstract] Abstract and title: the language of 'effects of snowball sampling' on completion rate, demographics, and response behavior is not supported by the described design. The comparison is post-hoc between two self-selected streams; respondents who arrive via referral must both receive a share and choose to participate, so differences are consistent with selection on unmeasured traits (motivation, network position, survey interest) that also affect the measured outcomes. No randomization, matching, referral-chain covariates, or selection model is mentioned.
- [Methods and Results] Methods/results: the abstract presents clear directional claims yet provides no sample sizes, completion-rate percentages, statistical tests, or controls for confounders. Without these details it is impossible to assess whether the reported differences are statistically reliable or robust to alternative explanations.
minor comments (1)
- The manuscript would benefit from explicit discussion of how 'new users' and 'completion' are operationalized and from any available platform metadata on referral chains.
Simulated Author's Rebuttal
We thank the referee for the detailed and constructive report. We agree that the observational, post-hoc nature of the comparison precludes causal claims about the 'effects' of snowball sampling. We will revise the title, abstract, and discussion to frame the results as descriptive differences between self-selected respondent streams, while noting potential selection biases. We address the major comments below.
read point-by-point responses
-
Referee: [Abstract] Abstract and title: the language of 'effects of snowball sampling' on completion rate, demographics, and response behavior is not supported by the described design. The comparison is post-hoc between two self-selected streams; respondents who arrive via referral must both receive a share and choose to participate, so differences are consistent with selection on unmeasured traits (motivation, network position, survey interest) that also affect the measured outcomes. No randomization, matching, referral-chain covariates, or selection model is mentioned.
Authors: We accept this criticism. The title and abstract currently use 'effects,' which overstates the evidence. We will change the title to 'Differences in completion rates, demographics, and responses between river and snowball sampling in West Africa' and revise the abstract to describe 'differences' and 'associations' rather than effects. We will add explicit language noting that the streams are self-selected and that unmeasured traits may drive the patterns. No randomization or matching is possible given the design, but we will highlight this limitation. revision: yes
-
Referee: [Methods and Results] Methods/results: the abstract presents clear directional claims yet provides no sample sizes, completion-rate percentages, statistical tests, or controls for confounders. Without these details it is impossible to assess whether the reported differences are statistically reliable or robust to alternative explanations.
Authors: The full manuscript reports the relevant sample sizes, completion-rate percentages, and results of statistical tests (chi-squared for demographics, t-tests and regressions for response length and time). We will update the abstract to include these key figures and p-values for transparency. As this is a purely observational comparison with no additional covariates collected, we cannot add controls or selection models; we will expand the limitations section to discuss robustness to alternative explanations such as motivation or network position. revision: partial
Circularity Check
No circularity: purely observational comparison with no derivations or models
full rationale
The paper conducts a post-hoc observational split of survey respondents into river-sampling (geo-targeted Facebook ads) and snowball-sampling (referrals) streams, then reports raw differences in completion rates, demographics, response length, and time spent. No equations, fitted parameters, predictions, uniqueness theorems, or self-citations appear in the derivation chain; the claims are direct empirical contrasts without any reduction of outputs to inputs by construction. Methodological limitations around self-selection exist but are unrelated to circularity.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Standard assumptions for comparing independent groups (e.g., no unmeasured confounding between sampling routes)
Reference graph
Works this paper leans on
-
[1]
Summary report of the AAPOR task force on non-probability samplin g
Baker R, Brick JM, Bates NA, Battaglia M, Couper MP, Dever JA, G ile KJ, Tourangeau R. Summary report of the AAPOR task force on non-probability samplin g. Journal of survey statistics and methodology. 2013;1(2)
2013
-
[2]
The en d of the (research) world as we know it? Understanding and coping with declining response rate s to mail surveys
Stedman RC, Connelly NA, Heberlein TA, Decker D, Allred SB. The en d of the (research) world as we know it? Understanding and coping with declining response rate s to mail surveys. Society & Natural Resources. 2019;32(10)
2019
-
[3]
Snowball sampling: Problems and techniqu es of chain referral sampling
Biernacky P, Waldorf D. Snowball sampling: Problems and techniqu es of chain referral sampling. Sociological methods & research. 1981;10(2)
1981
-
[4]
Information overload an d misinformation sharing behaviour of social media users: Testing the moderating role of cog nitive ability Journal of Information Science
Apuke OD, Omar B, Tunca EA, Gever CV. Information overload an d misinformation sharing behaviour of social media users: Testing the moderating role of cog nitive ability Journal of Information Science. 2024;50(6)
2024
-
[5]
Total survey error: Past, present, an d future
Groves RM, Lyberg L. Total survey error: Past, present, an d future. Public opinion quarterly. 2010;75(5)
2010
-
[6]
Heckathorn, D. D. Respondent-Driven Sampling II: Deriving Valid Population Estimates from Chain-Referral Samples. Social Problems. 2020;49:11-34. doi:10.15 25/sp.2002.49.1.11
2020
-
[7]
Online surveys and digital d emography in the developing world: Facebook users in Kenya
Pham KH, Rampazzo F, Rosenzweig LR. Online surveys and digital d emography in the developing world: Facebook users in Kenya. arXiv preprint arXiv:191 003448. 2019
2019
-
[8]
Can Citizen Science using social media inform sanitatio n planning? Journal of environmental management
Freihardt J. Can Citizen Science using social media inform sanitatio n planning? Journal of environmental management. 2020;259:110053
2020
-
[9]
Characterizing low effort responding among youn g African adults recruited via Facebook advertising
Olamijuwon EO. Characterizing low effort responding among youn g African adults recruited via Facebook advertising. Plos one. 2021;16(5):e0250303. May 5, 2026 16/31
2021
-
[10]
Survey sampling in the Global South using Facebook advertisements
Rosenzweig LR, Bergquist P, Pham KH, Rampazzo F, Mildenberge r M. Survey sampling in the Global South using Facebook advertisements. Political Science Res earch and Methods; 2025;13(4):781-797. doi:10.1017/psrm.2025.18
-
[11]
HIV knowledge and ass ociated factors among internet-using men who have sex with men (MSM) in South Africa and t he United States
Wagenaar BH, Sullivan PS, Stephenson R. HIV knowledge and ass ociated factors among internet-using men who have sex with men (MSM) in South Africa and t he United States. PloS one. 2012;7(3):e32915
2012
-
[12]
The mobile phone revolut ion, the Internet and rural electricity: What are the implications for food security in Africa? Inf ormation Development
Ezeoha AE, Obi A, Igwe A, Ezeruigbo C. The mobile phone revolut ion, the Internet and rural electricity: What are the implications for food security in Africa? Inf ormation Development. 2020 Nov;36(4):603-22
2020
-
[13]
Methods to recruit hard-to-reach groups comparing two chain referral sampling methods of recruiting injecting drug users across nine studies in Russia and Est onia
Platt L, Wall M, Rhodes T, Judd A, Hickman M, Johnston LG, Rent on A, Bobrova N, Sarang A. Methods to recruit hard-to-reach groups comparing two chain referral sampling methods of recruiting injecting drug users across nine studies in Russia and Est onia. Journal of Urban Health. 2006 Nov;83:39-53
2006
-
[14]
Formulas for Bayesian A/B Testing; 2015 [cited 2024 April 25]
Miller E. Formulas for Bayesian A/B Testing; 2015 [cited 2024 April 25]. Available from: https://www.evanmiller.org/bayesian-ab-testing.html. Appendix Ads fielded during the campaigns. We list the ads along with their text and a description of the accompan ying image in Table 20. We cannot show the images because they are copyrighted. We obtaine d permiss...
2015
-
[15]
To show our appreciation w e will randomly select 400 participants to receive $5 (USD) worth of mobile data
We would like to ask you some questions. To show our appreciation w e will randomly select 400 participants to receive $5 (USD) worth of mobile data. {Research participant information and consent form. } Do you want to continue?
-
[16]
What is your age? Please provide your age in years; example 25
-
[17]
What is your gender?
-
[18]
What is the name of the village, town, or city in which you live?
-
[19]
Is your residence located in an area that is urban or rural?
-
[20]
What is your religious background?
-
[21]
How many children do you have? If you have no children, answer 0 ( zero)
-
[22]
What is your occupation?
-
[23]
What is the price of 1 cup of local rice (sold loose) in your local mark et? May 5, 2026 17/31
2026
-
[24]
In 6 weeks, how much do you think the price of 1 cup of local rice ( sold loose) will be at the market?
-
[26]
What is the price of 1 liter of cooking oil in your market?
-
[27]
In 6 weeks, how much do you think the price of 1 liter of cooking oil will be at the market?
-
[28]
You answered with {response to last question } Why?
-
[29]
What is the price of 1 kilo of beef (bone-in) in your market?
-
[30]
In 6 weeks, how much do you think the price of 1 kilo of beef (bone -in) will be at the market?
-
[31]
Would you agree to participate in future research with us?
-
[32]
Please enter your email address below for future opportunitie s to receive rewards for participating in other surveys
-
[33]
Include the country code
Please provide your phone number below to qualify for mobile data . Include the country code. (If you are not interested, leave this field blank and click ”Submit”). The survey in Hausa is available upon reasonable request. Graphics for response time and length. We show all the histograms for the response time and length broken up by month and sex. The da...
2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.