REVIEW 4 major objections 4 minor 152 references
Digital Labor: Challenges, Ethical Insights, and Implications
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A decade of digital labor research chased productivity, not workers' rights.
desk verdict Useful but unverifiable review of digital labor scholarship whose central worker-voice gap claim contradicts its own methods section. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying apparatus is a structured review protocol: a literature search with six terms ('Crowdwork,' 'Ghost work,' 'Crowdsourcing,' 'Crowdsource,' 'Crowdplatform,' 'Ghost Workers') across a single search source, manual screening that kept 143 peer-reviewed papers, and a six-dimension coding rubric (when, where, who, what, why, how) applied to each paper. The rubric does the work of turning a pile of studies into distributions by year, region, stakeholder, theme, motivation, and method, so that absences like missing worker voices or missing policy work become visible as counts rather than impressions.
What would settle it
A reader could settle the central claim by conducting a broader multi-source search of the 2015–2024 literature and counting how many digital labor papers include direct interview or survey data from workers about rights, pay, and grievance procedures. If a substantial share (for example, more than a quarter) of that larger corpus does, the review's 'noticeable negligence of workers' rights' finding would be an artifact of what was retained rather than a property of the field.
Extended reading notes
Core claim
The central claim is that, over the ten years from 2015 to 2024, research on digital gig labor has a dominant focus on improving laborers' performances, with noticeable negligence of workers' rights. The same mapping shows that few studies include direct input from workers, that the unintended and unpaid labor platforms impose on users (ad-watching, captchas, forced tasks) is rarely studied, and that cross-border labor policy, geography-based AI policy, and remote-work statutes are almost absent from the conversation. Recurring topics such as power asymmetries, data invisibility, and platform accountability appear frequently but have not been converted into practical changes in the market. These patterns, the paper argues, are properties of the corpus that a systematic review can surface and that interventions must address.
Load-bearing premise
The load-bearing premise is that the 143 papers kept after a six-term search in one literature database, with manual exclusion of design, algorithm, and theory papers, fairly represent what digital labor scholarship has actually done since 2015.
Editorial extensions
If this is right
- Research should shift from describing tasks and productivity toward studies that collect primary data from workers on pay, rights, and grievances.
- Platform designers should add features that give workers control over task selection, communication, and evaluation, and should involve workers in co-design.
- Policymakers should create legal categories and cross-border standards tailored to platform work, since traditional employment law does not cover most digital workers.
- Industry should make invisible labor visible and compensate for unpaid time such as waiting for tasks and learning tools.
- Scholars should investigate unpaid user labor, like ad-viewing and captcha completion, including how that data feeds algorithms and how burdens differ across regions.
Reading between the lines
- The gap findings are, strictly, claims about what a single-search-source, six-term retrieval surfaces; a broader multi-source search might relocate some gaps, so they are best read as hypotheses about the field's shape.
- If the review is right that rights and voice are neglected, then AI systems trained on crowd work carry an unexamined ethical vulnerability that standard fairness metrics will not capture, because the labor conditions are invisible to the model.
- A cheap test of the central claim would be to count, across the same years and venues, the share of papers whose primary data come from workers themselves; the review implies that share is small.
- The paper's own limitations suggest a testable extension: including design, algorithm, and theory papers in a future corpus could reveal whether rights-oriented work is hiding in technical interventions rather than missing from the literature.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a systematic literature review of 143 papers published between 2015 and 2024 on digital gig labor, aiming to map the field's growth, geographical distribution, stakeholders, research themes, and methods, and to identify gaps. The central synthesis claims that the scholarship overweights improving laborers' performance and neglects workers' rights, that unpaid and unintended digital labor is an understudied concern, that few studies include direct input from workers, and that the literature lacks pluralistic ethical analysis; it then derives implications for industry, policy, platform design, and future research.
Significance. If the synthesis is accurate, the paper would provide a useful map of a decade of digital-labor scholarship and actionable implications for HCI, AI-ethics, and policy audiences. The paper's strengths include the disclosure of search parameters (Table 2), the internal consistency of the period counts (34, 42, and 67 summing to 143 in Section 4.2.1), the explicit inclusion and exclusion criteria (Section 4.1), and a candid limitations section (Section 5.3). However, the contribution is currently qualified by an internal inconsistency in the paper's own methods summary, the absence of an archived corpus or coding dataset, and several statements that go beyond what the reported evidence can support; these issues bear directly on the paper's headline gap findings.
major comments (4)
- [Section 5.1 and Section 4.2.6(a)] The third gap claim, that "few studies include direct input from the workers" (citing [43,76,77,91,110]), is contradicted by the paper's own methods summary, which states that most reviewed papers use qualitative approaches, including interviews with crowdworkers, and cites [77,110] as examples. Section 4.2.3 also identifies 54 crowdworker-focused papers, several of which (e.g., [30,53,77,110,114,150]) are described as involving direct worker participation through interviews, ethnography, or co-design. Because the 143-paper corpus and the coding data are not published, readers cannot determine whether the "few direct input" code was applied inconsistently or whether the supporting citations were miscoded; this uncertainty directly undermines a headline finding and must be resolved by reporting the coding rubric, the coded values for the cited papers, and the full corpus.
- [Section 4.1 and Section 5.1] The first gap claim, that the scholarship has a "dominant focus on improving laborers' performances, with noticeable negligence of workers' rights," is a distributional assertion that lacks quantitative support in the manuscript. Section 4.1 describes a six-category rubric (Table 3), but the paper does not report inter-rater reliability, code frequencies, or operational definitions distinguishing "focus on performance" from "workers' rights." Without such evidence, the claim is an interpretation of selected examples rather than a synthesis finding; please report the coding frequencies for all gap-related categories and make the codebook and coded dataset available, or soften the claim to match the narrative evidence presented.
- [Section 5.1, second gap] The "unintended and unpaid digital labor" gap is presented as a finding of the literature review, but the text explicitly states that "the writers of this manuscript investigated five dominant gig-labor platforms" and observed forced ad-views. This is author-conducted platform inspection, not a synthesis of the 143-paper corpus, and it is partly entailed by the paper's own expanded definition of digital labor in Section 1. Please clearly position this as a proposed new research direction grounded in the authors' own observations, rather than as a gap in the reviewed scholarship, or substantiate it with citations from the selected corpus.
- [Section 4.1 and reference list] Section 4.1 states that "we retained only peer-reviewed papers," but the reference list includes multiple items that are not peer-reviewed, including arXiv preprints and SSRN working papers (e.g., [19], [68], [76], [80], [123]). This inconsistency affects the credibility of the corpus description and must be corrected either by applying the peer-review criterion strictly and removing non-peer-reviewed items, or by revising the stated criterion to something like "academic publications, including preprints and working papers." The authors should also state which of the 143 papers were preprints or not peer-reviewed.
minor comments (4)
- [Section 4.2.2] The phrase "Unites States" appears twice in Section 4.2.2 and should be corrected to "United States."
- [Figure 1] Figure 1 would be easier to interpret and more aligned with PRISMA-style reporting if it included the number of papers excluded at each stage of the selection process, such as the number excluded for domain-specific applications, design/algorithm focus, or failure to meet the peer-review criterion.
- [Section 5.1] The word "commensurable" in the sentence about forced ad-views appears to mean "comparable" or "consistent with"; consider revising for clarity.
- [Section 4.1 and Table 3] The rubric definitions in Table 3 are very broad (e.g., "What" is defined as "the primary focus of the research, such as labor dynamics and systemic challenges"); providing more specific coding instructions for each category would strengthen the reproducibility of the coding process.
Circularity Check
No significant circularity; the gap findings are literature-corpus readings, not definitional entailments, and limitations are explicit.
full rationale
This is a literature review rather than a derivation chain, and its central claims are about the 143-paper corpus it assembled. The selection used neutral search terms (Table 2) plus an inclusion criterion requiring papers to discuss worker experiences or labor conditions (Section 4.1). The headline findings in Section 5.1—dominant focus on improving performance, negligence of workers' rights, the unpaid-labor gap, and few direct worker inputs—are readings of that corpus. None of these findings is equivalent by construction to the corpus definition: the inclusion filter is a relevance screen, and the coding rubric in Table 3 records when/where/who/what/why/how without encoding the reported gap categories. The broadened definition of digital labor in Section 1 (including forced ad-views) is a scope choice; reporting that the literature under-studies that scope is a normal review outcome, not a self-definitional reduction. Section 5.3 explicitly discloses the main limitations: reliance solely on Google Scholar, no direct input from workers, and no new empirical findings. These are honest scope restrictions rather than circular moves. There are no self-citations, no fitted parameters, no imported uniqueness theorems, and no equations that reduce to their own inputs. One internal tension exists: Section 4.2.6(a) says most reviewed papers use qualitative approaches including interviews with crowdworkers, citing [77,110], while Section 5.1 says few studies include direct worker input, citing [77,110] among others. That inconsistency is a coding-reliability or evidence-quality concern about a distributional claim, not a circularity, because the claim is not entailed by the search and selection procedure. Accordingly, no circular step is identified.
Assumptions & free parameters
free parameters (3)
- Review window 2015-2024 =
2015-2024
- 143-paper corpus cutoff =
143 of 300+
- Search term set =
Six terms
assumptions (4)
- domain assumption The selected corpus stands in for the whole field of digital labor scholarship.
- domain assumption Manual screening and coding by the authors yields reliable categories.
- ad hoc to paper Digital labor includes unintended tasks imposed on users, such as forced ad-views.
- domain assumption Peer-reviewed status and sufficient depth are adequate quality filters.
Cite this review
Pith. "Pith review of Digital Labor: Challenges, Ethical Insights, and Implications." pith.science (2026). https://pith.science/paper/XJGAUTCM
@misc{pith2026250611788,
author = {Pith},
title = {Pith review of: Digital Labor: Challenges, Ethical Insights, and Implications},
year = {2026},
howpublished = {\url{https://pith.science/paper/XJGAUTCM}},
note = {Machine review of arXiv:2506.11788}
}
read the original abstract
Digital workers on crowdsourcing platforms (e.g., Amazon Mechanical Turk, Appen, Clickworker, Prolific) play a crucial role in training and improving AI systems, yet they often face low pay, unfair conditions, and a lack of recognition for their contributions. To map these issues in the existing literature of computer science, AI, and related scholarship, we selected over 300 research papers on digital labor published between 2015 and 2024, narrowing them down to 143 on digital gig-labor for a detailed analysis. This analysis provides a broad overview of the key challenges, concerns, and trends in the field. Our synthesis reveals how the persistent patterns of representation and voices of gig workers in digital labor are structured and governed. We offer new insights for researchers, platform designers, and policymakers, helping them better understand the experiences of digital workers and pointing to key areas where interventions and future investigations are promptly needed. By mapping the findings from the past ten years' growth of the domain and possible implications, this paper contributes to a more coherent and critical understanding of digital labor in contemporary and future AI ecosystems.
Figures
Reference graph
Works this paper leans on
-
[134]
Lindi van Niekerk, Arturo Ongkeko, Rachel Alice Hounsell, Barwani Khaura Msiska, Don Pascal Mathanga, Josselyn Mothe, Noel Juban, Phyllis Awor, and Dina Balabanova. 2020. Crowdsourcing to identify social innovation initiatives in health in low-and middle-income countries.Infectious Diseases of Poverty9 (2020), 1–12
work page 2020
-
[19]
Alan Chan, Chinasa T Okolo, Zachary Terner, and Angelina Wang. 2021. The limits of global inclusion in AI development.arXiv preprint arXiv:2102.01265(2021)
arXiv 2021
-
[68]
Jane Hsieh, Angie Zhang, Mialy Rasetarinera, Erik Chou, Daniel Ngo, Karen Lightman, Min Kyung Lee, and Haiyi Zhu. 2024. Supporting Gig Worker Needs and Advancing Policy Through Worker-Centered Data-Sharing.arXiv preprint arXiv:2412.02973(2024)
arXiv 2024
-
[76]
Lilly Irani. 2023. Algorithms of suspicion: authentication and distrust on the Amazon mechanical Turk platform.A vailable at SSRN 4482508(2023)
work page 2023
-
[80]
Pyeonghwa Kim, Charis Asante-Agyei, Isabel Munoz, Michael Dunn, and Steve Sawyer. 2024. Decoding the Meaning of Success on Digital Labor Platforms: Worker-Centered Perspectives.arXiv preprint arXiv:2411.14298(2024)
work page Pith review arXiv 2024
-
[123]
Jack Stilgoe, Cornelia Evers, Jake Love Soper, Maya Khurana, and Tiago Mata. 2022. Ghost Work: Empowering the invisible labour behind artificial intelligence. A vailable at SSRN 4891800(2022)
work page 2022
-
[1]
Tahir Abbas and Ujwal Gadiraju. 2022. Goal-setting behavior of workers on crowdsourcing platforms: An exploratory study on mturk and prolific. InProceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 10. 2–13
2022
-
[2]
Eman Salem Alashwali, Pawel Szalachowski, and Andrew Martin. 2020. Exploring HTTPS security inconsistencies: A cross-regional perspective.Computers & security97 (2020), 101975
2020
Show all 152 references
-
[3]
Ali Alkhatib, Michael S Bernstein, and Margaret Levi. 2017. Examining crowd work and gig work through the historical lens of piecework. InProceedings of the 2017 CHI conference on human factors in computing systems. 4599–4616
2017
-
[4]
Moritz Altenried. 2020. The platform as factory: Crowdwork and the hidden labour behind artificial intelligence.Capital & Class44, 2 (2020), 145–158
2020
-
[5]
Moritz Altenried. 2024. Mobile workers, contingent labour: Migration, the gig economy and the multiplication of labour.Environment and Planning A: Economy and Space56, 4 (2024), 1113–1128
2024
-
[6]
Juan Carlos Alvarez de la Vega, Marta E Cecchinato, John Rooksby, and Joseph Newbold. 2023. Understanding platform mediated work-life: a diary study with gig economy freelancers.Proceedings of the ACM on Human-Computer Interaction7, CSCW1 (2023), 1–32
2023
-
[7]
Mohammad Amir Anwar and Mark Graham. 2020. Hidden transcripts of the gig economy: labour agency and the new art of resistance among African gig workers.Environment and Planning A: Economy and Space52, 7 (2020), 1269–1291
2020
-
[8]
Mohammad Amir Anwar and Mark Graham. 2021. Between a rock and a hard place: Freedom, flexibility, precarity and vulnerability in the gig economy in Africa. Competition & Change25, 2 (2021), 237–258
2021
-
[9]
Payal Arora. 2016. Bottom of the data pyramid: Big data and the global south.International Journal of Communication10 (2016), 19
2016
-
[10]
Natã M Barbosa and Monchu Chen. 2019. Rehumanized crowdsourcing: A labeling framework addressing bias and ethics in machine learning. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12
2019
-
[11]
Matthew Barker, Emma Kallina, Dhananjay Ashok, Katherine Collins, Ashley Casovan, Adrian Weller, Ameet Talwalkar, Valerie Chen, and Umang Bhatt. 2023. FeedbackLogs: Recording and incorporating stakeholder feedback into machine learning pipelines. InProceedings of the 3rd ACM C...
2023
-
[12]
Oleg Bazaluk, Muhammad Ataur Rahman, Nurul Mohammad Zayed, Md Faisal-E-Alam, Vitalii Nitsenko, and Lesia Kucher. 2024. Crowdsourcing review: the crowd workers’ perspective.Journal of Industrial and Business Economics(2024), 1–20
2024
-
[13]
Janine Berg. 2015. Income security in the on-demand economy: Findings and policy lessons from a survey of crowdworkers.Comp. Lab. L. & Pol’y J.37 (2015), 543
2015
-
[14]
Janine Berg, Miriam A Cherry, and Uma Rani. 2019. Digital labour platforms: a need for international regulation?Revista de Economía Laboral16, 2 (2019), 104–128
2019
-
[15]
Janine Berg, Marianne Furrer, Ellie Harmon, Uma Rani, and M Six Silberman. 2018. Digital labour platforms and the future of work.Towards decent work in the online world. Rapport de l’OIT(2018)
2018
-
[16]
Shahzad Sarwar Bhatti, Xiaofeng Gao, and Guihai Chen. 2020. General framework, opportunities and challenges for crowdsourcing techniques: A comprehensive survey.Journal of Systems and Software167 (2020), 110611
2020
-
[17]
Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022. Power to the people? Opportunities and challenges for participatory AI. InProceedings of the 2nd ACM Conference on Equity and Access in Algorithms, ...
2022
-
[18]
Alice M Brawley and Cynthia LS Pury. 2016. Work experiences on MTurk: Job satisfaction, turnover, and information sharing.Computers in Human Behavior54 (2016), 531–546
2016
-
[20]
Stevie Chancellor, Eric PS Baumer, and Munmun De Choudhury. 2019. Who is the" human" in human-centered machine learning: The case of predicting mental health from social media.Proceedings of the ACM on Human-Computer Interaction3, CSCW (2019), 1–32
2019
-
[21]
Jianyu Chen. 2024. Are we only all by ourselves? A double-level perspective to cope with the insecurity of the nonstandard gig work model.International Journal of Manpower45, 4 (2024), 733–753
2024
-
[22]
Daniel Chuene and Jabu Mtsweni. 2015. The adoption of crowdsourcing platforms in South Africa. In2015 IST-Africa Conference. IEEE, 1–9
2015
-
[23]
Lorenzo Cini. 2023. Resisting algorithmic control: Understanding the rise and variety of platform worker mobilisations.New Technology, Work and Employment 38, 1 (2023), 125–144
2023
-
[24]
Eric Corbett, Emily Denton, and Sheena Erete. 2023. Power and public participation in ai. InProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–13
2023
-
[25]
Are we all in the same boat?
Esra Cemre Su de Groot and Ujwal Gadiraju. 2024. " Are we all in the same boat?" Customizable and Evolving Avatars to Improve Worker Engagement and Foster a Sense of Community in Online Crowd Work. InProceedings of the CHI Conference on Human Factors in Computing Systems. 1–26
2024
-
[26]
2022.Understanding the impacts of gig economy platforms on freelancers’ work practices
Juan Carlos Alvarez De La Vega. 2022.Understanding the impacts of gig economy platforms on freelancers’ work practices. University of Northumbria at Newcastle (United Kingdom)
2022
-
[27]
Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The participatory turn in ai design: Theoretical foundations and the current state of practice. InProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–23
2023
-
[28]
Nicola Dell and Neha Kumar. 2016. The ins and outs of HCI for development. InProceedings of the 2016 CHI conference on human factors in computing systems. 2220–2232
2016
-
[29]
Xuefei Deng, Kshiti D Joshi, and Robert D Galliers. 2016. The duality of empowerment and marginalization in microtask crowdsourcing.MIS quarterly40, 2 (2016), 279–302
2016
-
[30]
Kimberly Do, Maya De Los Santos, Michael Muller, and Saiph Savage. 2024. Designing Gig Worker Sousveillance Tools. InProceedings of the CHI Conference on Human Factors in Computing Systems. 1–19
2024
-
[31]
Sarah A Donovan, David H Bradley, and Jon O Shimabukuru. 2016. What does the gig economy mean for workers? (2016)
2016
-
[32]
Stephan Drechsler and Christian Harteis. 2025. A Systematic Review of Learning Opportunities Within the Crowdworkers’ Workplace.Journal of Education and Learning14, 2 (2025)
2025
-
[33]
Shuili Du, Mayowa T Babalola, Premilla D’cruz, Edina Dóci, Lucia Garcia-Lorenzo, Louise Hassan, Gazi Islam, Alexander Newman, Ernesto Noronha, and Suzanne van Gils. 2024. The Ethical, Societal, and Global Implications of Crowdsourcing Research.Journal of Business Ethics(2024), 1–16
2024
-
[34]
David Durward, Ivo Blohm, and Jan Marco Leimeister. 2016. Is there papa in crowd work?: A literature review on ethical dimensions in crowdsourcing. In2016 Intl IEEE Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Commu...
2016
-
[35]
David Durward, Ivo Blohm, and Jan Marco Leimeister. 2020. The nature of crowd work and its effects on individuals’ work perception.Journal of Management Information Systems37, 1 (2020), 66–95
2020
-
[36]
Senjuti Dutta, Rhema Linder, Doug Lowe, Richard Rosenbalm, Anastasia Kuzminykh, and Alex C Williams. 2022. Mobilizing crowdwork: A systematic assessment of the mobile usability of hits. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–20
2022
-
[37]
Carsten Eickhoff. 2018. Cognitive biases in crowdsourcing. InProceedings of the eleventh ACM international conference on web search and data mining. 162–170
2018
-
[38]
Nancy Ettlinger. 2016. The governance of crowdsourcing: Rationalities of the new exploitation.Environment and Planning A: Economy and Space48, 11 (2016), 2162–2180
2016
-
[39]
Shaoyang Fan. 2023. Improving the work environment and data quality in micro-task crowdsourcing. (2023)
2023
-
[40]
Molly Q Feldman and Brian James McInnis. 2021. How we write with crowds.Proceedings of the ACM on Human-Computer Interaction4, CSCW3 (2021), 1–31
2021
-
[41]
Claudia Flores-Saviaga, Yuwen Li, Benjamin Hanrahan, Jeffrey Bigham, and Saiph Savage. 2020. The challenges of crowd workers in rural and urban America. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 8. 159–162
2020
-
[42]
Robert C Ford, Brendan Richard, and Michael P Ciuchta. 2015. Crowdsourcing: A new way of employing non-employees?Business Horizons58, 4 (2015), 377–388
2015
-
[43]
Sarah E Fox, Vera Khovanskaya, Clara Crivellaro, Niloufar Salehi, Lynn Dombrowski, Chinmay Kulkarni, Lilly Irani, and Jodi Forlizzi. 2020. Worker-centered design: Expanding HCI methods for supporting labor. InExtended abstracts of the 2020 CHI conference on human factors in co...
2020
-
[44]
Gerald Friedman. 2014. Workers without employers: shadow corporations and the rise of the gig economy.Review of keynesian economics2, 2 (2014), 171–188
2014
-
[45]
Alessandro Gandini. 2019. Labour process theory and the gig economy.Human relations72, 6 (2019), 1039–1056
2019
-
[46]
Christine Gerber. 2021. Community building on crowdwork platforms: Autonomy and control of online workers?Competition & Change25, 2 (2021), 190–211
2021
-
[47]
Antonio Ghezzi, Donata Gabelloni, Antonella Martini, and Angelo Natalicchio. 2018. Crowdsourcing: a review and suggestions for future research.International Journal of management reviews20, 2 (2018), 343–363
2018
-
[48]
Thuy Giang Thi Giang, Luu Tien Dung, Huynh Thuy Tien, and Chung Tu Bao Nhu. 2024. Corporate social responsibility and gig worker commitment: Empowerment and trust as mediators.Journal of Global Responsibility(2024)
2024
-
[49]
Elham Shafiei Gol, Mari-Klara Stein, and Michel Avital. 2019. Crowdwork platform governance toward organizational value creation.The Journal of Strategic Information Systems28, 2 (2019), 175–195
2019
-
[50]
Is your gig any good?
Caleb Goods, Alex Veen, and Tom Barratt. 2019. “Is your gig any good?” Analysing job quality in the Australian platform-based food-delivery sector.Journal of Industrial Relations61, 4 (2019), 502–527
2019
-
[51]
Mark Graham, Isis Hjorth, and Vili Lehdonvirta. 2017. Digital labour and development: impacts of global digital labour platforms and the gig economy on worker livelihoods.Transfer: European review of labour and research23, 2 (2017), 135–162
2017
-
[52]
2019.Ghost work: How to stop Silicon Valley from building a new global underclass
Mary L Gray and Siddharth Suri. 2019.Ghost work: How to stop Silicon Valley from building a new global underclass. Eamon Dolan Books
2019
-
[53]
Rafael Grohmann and Willian F ARAÚJO. 2021. Beyond Mechanical Turk: the work of Brazilians on global AI platforms.VERDEGEM, Pieter. AI for everyone (2021), 247–266
2021
-
[54]
Rafael Grohmann, Gabriel Pereira, Abel Guerra, Ludmila Costhek Abilio, Bruno Moreschi, and Amanda Jurno. 2022. Platform scams: Brazilian workers’ experiences of dishonest and uncertain algorithmic management.New Media & Society24, 7 (2022), 1611–1631
2022
-
[55]
Guy Grossman, Melina R Platas, and Jonathan Rodden. 2018. Crowdsourcing accountability: ICT for service delivery.World Development112 (2018), 74–87. 16 Digital Labor: Challenges, Ethical Insights, and Implications Conference ’25, November 5–7, 2025, PA
2018
-
[56]
Yasamin Hadavi, Eric Villafranca, and Stacie Petter. 2022. The Duality of Empowerment and Marginalization in Microtask Crowdsourcing: A Replication.AIS Transactions on Replication Research8, 1 (2022), 2
2022
-
[57]
Karin Hansson. 2014. A micro-democratic perspective on crowd-work. InGroup 14, ACM International Conference on Supporting Groupwork, Sanibel Island, Florida, USA, November 9–12, 2014
2014
-
[58]
Karin Hansson, Michael Muller, Tanja Aitamurto, Lilly Irani, Athanasios Mazarakis, Neha Gupta, and Thomas Ludwig. 2016. Crowd dynamics: Exploring conflicts and contradictions in crowdsourcing. InProceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Comp...
2016
-
[59]
Kotaro Hara and Yudai Tanaka. 2022. Understanding crowdsourcing requesters’ wage setting behaviors. InCHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–6
2022
-
[60]
Giannis Haralabopoulos, Christian Wagner, Derek McAuley, and Ioannis Anagnostopoulos. 2019. Paid crowdsourcing, low income contributors, and subjectivity. InArtificial Intelligence Applications and Innovations: AIAI 2019 IFIP WG 12.5 International Workshops: MHDW and 5G-PINE 2...
2019
-
[61]
Ellie Harmon and M Six Silberman. 2019. Rating working conditions on digital labor platforms.Computer Supported Cooperative Work (CSCW)28, 5 (2019), 911–960
2019
-
[62]
David Hesmondhalgh. 2010. User-generated content, free labour and the cultural industries.ephemera10, 3/4 (2010), 267–284
2010
-
[63]
James Hickson. 2024. Freedom, domination and the gig economy.New Political Economy29, 2 (2024), 321–336
2024
-
[64]
Lars Hornuf and Daniel Vrankar. 2022. Hourly wages in crowdworking: A meta-analysis.Business & Information Systems Engineering64, 5 (2022), 553–573
2022
-
[65]
Sarah B Horton. 2016. Ghost workers: the implications of governing immigration through crime for migrant workplaces.Anthropology of Work Review37, 1 (2016), 11–23
2016
-
[66]
Mahmood Hosseini, Alimohammad Shahri, Keith Phalp, and Raian Ali. 2016. Crowdsourcing transparency requirements through structured feedback and social adaptation. In2016 IEEE Tenth International Conference on Research Challenges in Information Science (RCIS). IEEE, 1–6
2016
-
[67]
Debra Howcroft and Birgitta Bergvall-Kåreborn. 2019. A typology of crowdwork platforms.Work, employment and society33, 1 (2019), 21–38
2019
-
[69]
Srihari Hulikal Muralidhar, Sean Rintel, and Siddharth Suri. 2022. Collaboration, invisible work, and the costs of macrotask freelancing.Proceedings of the ACM on Human-Computer Interaction6, CSCW2 (2022), 1–25
2022
-
[70]
Kashif Imteyaz, Claudia Flores Saviaga, and Saiph Savage. 2024. Human Computation, Equitable, and Innovative Future of Work AI Tools. InProceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 12. 155–156
2024
-
[71]
Kori Inkpen, Stevie Chancellor, Munmun De Choudhury, Michael Veale, and Eric PS Baumer. 2019. Where is the human? Bridging the gap between AI and HCI. InExtended abstracts of the 2019 chi conference on human factors in computing systems. 1–9
2019
-
[72]
Panagiotis G Ipeirotis. 2010. Analyzing the amazon mechanical turk marketplace.XRDS: Crossroads, The ACM magazine for students17, 2 (2010), 16–21
2010
-
[73]
Lilly Irani. 2015. The cultural work of microwork.New media & society17, 5 (2015), 720–739
2015
-
[74]
Lilly Irani. 2015. Difference and dependence among digital workers: The case of Amazon Mechanical Turk.South Atlantic Quarterly114, 1 (2015), 225–234
2015
-
[75]
Design thinking
Lilly Irani. 2018. “Design thinking”: Defending Silicon Valley at the apex of global labor hierarchies.Catalyst: Feminism, Theory, Technoscience4, 1 (2018)
2018
-
[77]
Lilly C Irani and M Six Silberman. 2016. Stories We Tell About Labor: Turkopticon and the Trouble with" Design". InProceedings of the 2016 CHI conference on human factors in computing systems. 4573–4586
2016
-
[78]
Emma Kallina and Jatinder Singh. 2024. Stakeholder Involvement for Responsible AI Development: A Process Framework. InProceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–14
2024
-
[79]
Anne Kaun, Julia Velkova, Salla-Maaria Laaksonen, Alessandro Delfanti, Alexis Logsdon, Fredrik Stiernstedt, Tuukka Lehtiniemi, and Minna Ruckenstein. 2020. In the Shadows of the Digital Economy: The Ghost Work of Infrastructural Labor.AoIR Selected Papers of Internet Research(2020)
2020
-
[81]
Reilly Kincaid and Jeremy Reynolds. 2024. Unconventional Work, Conventional Problems: Gig Microtask Work, Inequality, and the Flexibility Mystique.The Sociological Quarterly65, 2 (2024), 246–268
2024
-
[82]
Aniket Kittur, Jeffrey V Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013. The future of crowd work. InProceedings of the 2013 conference on Computer supported cooperative work. 1301–1318
2013
-
[83]
Nikos Koutsimpogiorgos, Jaap Van Slageren, Andrea M Herrmann, and Koen Frenken. 2020. Conceptualizing the gig economy and its regulatory problems. Policy & Internet12, 4 (2020), 525–545
2020
-
[84]
Ketana Krishna. 2024. Artificial Intelligence & the Global South: Bridging or Exacerbating the Digital Divide?A vailable at SSRN 4866372(2024)
2024
-
[85]
Kristine M Kuhn, Jeroen Meijerink, and Anne Keegan. 2021. Human resource management and the gig economy: Challenges and opportunities at the intersection between organizational HR decision-makers and digital labor platforms.Research in personnel and human resources management3...
2021
-
[86]
Laura Lascau, Sandy JJ Gould, Duncan P Brumby, and Anna L Cox. 2022. Crowdworkers’ temporal flexibility is being traded for the convenience of requesters through 19 ‘invisible mechanisms’ employed by crowdworking platforms: A comparative analysis study of nine platforms. InCHI...
2022
-
[87]
Hanlin Li, Nicholas Vincent, Stevie Chancellor, and Brent Hecht. 2023. The dimensions of data labor: A road map for researchers, activists, and policymakers to empower data producers. InProceedings of the 2023 ACM conference on fairness, accountability, and transparency. 1151–1161
2023
-
[88]
Brian McInnis, Alissa Centivany, Juho Kim, Marta Poblet, Karen Levy, and Gilly Leshed. 2017. Crowdsourcing law and policy: a design-thinking approach to crowd-civic systems. InCompanion of the 2017 ACM conference on computer supported cooperative work and social computing. 355–361
2017
-
[89]
Brian McInnis, Dan Cosley, Chaebong Nam, and Gilly Leshed. 2016. Taking a HIT: Designing around rejection, mistrust, risk, and workers’ experiences in Amazon Mechanical Turk. InProceedings of the 2016 CHI conference on human factors in computing systems. 2271–2282
2016
-
[90]
Brian McInnis and Gilly Leshed. 2016. Running user studies with crowd workers.Interactions23, 5 (2016), 50–53
2016
-
[91]
Brian James McInnis, Elizabeth Lindley Murnane, Dmitry Epstein, Dan Cosley, and Gilly Leshed. 2016. One and Done: Factors affecting one-time contributors to ad-hoc online communities. InProceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Comp...
2016
-
[92]
Mareike Möhlmann, Lior Zalmanson, Ola Henfridsson, and Robert Wayne Gregory. 2021. Algorithmic management of work on online labor platforms: When matching meets control.MIS quarterly45, 4 (2021). 17 Conference ’25, November 5–7, 2025, PA Rahman et al
2021
-
[93]
2015.Crowdsourcing mental health and emotional well-being
Robert Robert Randall Morris. 2015.Crowdsourcing mental health and emotional well-being. Ph. D. Dissertation. Massachusetts Institute of Technology
2015
-
[94]
James Muldoon and Paul Apostolidis. 2023. ‘Neither work nor leisure’: Motivations of microworkers in the United Kingdom on three digital platforms.new media & society(2023), 14614448231183942
2023
-
[95]
Wim Naudé, Amy Bray, and Celina Lee. 2021. Crowdsourcing Artificial Intelligence in Africa: Findings from a Machine Learning Contest. (2021)
2021
-
[96]
Wim Naudé, Amy Bray, and Celina Lee. 2022. Crowdsourcing Artificial Intelligence in Africa: Analysis of a Data Science Contest.A vailable at SSRN 4076351 (2022)
2022
-
[97]
Gemma Newlands. 2021. Algorithmic surveillance in the gig economy: The organization of work through Lefebvrian conceived space.Organization Studies42, 5 (2021), 719–737
2021
-
[98]
Chinasa T Okolo. 2023. Addressing global inequity in AI development. InHandbook of Critical Studies of Artificial Intelligence. Edward Elgar Publishing, 378–389
2023
-
[99]
Chinasa T Okolo and Hongjin Lin. 2024. Explainable AI in Practice: Practitioner Perspectives on AI for Social Good and User Engagement in the Global South. In Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–16
2024
-
[100]
2023.100 Lives, Crowdwork and the Gig Economy
International Labor Organization. 2023.100 Lives, Crowdwork and the Gig Economy. https://www.ilo.org/resource/article/100-years-100-lives-crowdwork-and- gig-economy
2023
-
[101]
Marie-Therese Png. 2022. At the tensions of south and north: Critical roles of global south stakeholders in AI governance. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 1434–1445
2022
-
[102]
Jennifer Preece, Carol Boston, Mary Lou Maher, Kazjon Grace, and Tom Yeh. 2016. From Crowdsourcing Technology Design to Participatory Design and Back Again!. InProceedings of The European Conference on Social Media 2016/3rd European Conference on Social Media, Caen, France, 12...
2016
-
[103]
Hatim A Rahman. 2021. The invisible cage: Workers’ reactivity to opaque algorithmic evaluations.Administrative Science Quarterly66, 4 (2021), 945–988
2021
-
[104]
Uma Rani and Marianne Furrer. 2021. Digital labour platforms and new forms of flexible work in developing countries: Algorithmic management of work and workers.Competition & Change25, 2 (2021), 212–236
2021
-
[105]
Veronica A Rivera and David T Lee. 2021. I want to, but first I need to: Understanding crowdworkers’ career goals, challenges, and tensions.Proceedings of the ACM on Human-Computer Interaction5, CSCW1 (2021), 1–22
2021
-
[106]
Alex Rosenblat and Luke Stark. 2016. Algorithmic labor and information asymmetries: A case study of Uber’s drivers.International journal of communication10 (2016), 27
2016
-
[107]
Gobinda Roy and Avinash K Shrivastava. 2020. Future of gig economy: opportunities and challenges.Imi Konnect9, 1 (2020), 14–27
2020
-
[108]
Mayra Russo, Mackenzie Jorgensen, Kristen M Scott, Wendy Xu, Di H Nguyen, Jessie Finocchiaro, and Matthew Olckers. 2024. Bridging Research and Practice Through Conversation: Reflecting on Our Experience. InProceedings of the 4th ACM Conference on Equity and Access in Algorithm...
2024
-
[109]
Dina Sabie, Cansu Ekmekcioglu, and Syed Ishtiaque Ahmed. 2022. A decade of international migration research in HCI: Overview, challenges, ethics, impact, and future directions.ACM Transactions on Computer-Human Interaction (TOCHI)29, 4 (2022), 1–35
2022
-
[110]
Niloufar Salehi, Lilly C Irani, Michael S Bernstein, Ali Alkhatib, Eva Ogbe, Kristy Milland, and Clickhappier. 2015. We are dynamo: Overcoming stalling and friction in collective action for crowd workers. InProceedings of the 33rd annual ACM conference on human factors in comp...
2015
-
[111]
Joni Salminen, Ahmed Mohamed Sayed Kamel, Soon-Gyo Jung, Mekhail Mustak, and Bernard J Jansen. 2023. Fair compensation of crowdsourcing work: the problem of flat rates.Behaviour & Information Technology42, 16 (2023), 2871–2892
2023
-
[112]
Shruti Sannon, Billie Sun, and Dan Cosley. 2022. Privacy, surveillance, and power in the gig economy. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–15
2022
-
[113]
Mou Rani Sarker, Tanjum Afrin Taj, Md Abdur Rouf Sarkar, Md Fuad Hassan, Andrew M McKenzie, Md Abdullah Al Mamun, Dweep Sarker, and Humnath Bhandari. 2024. Gender differences in job satisfaction among gig workers in Bangladesh.Scientific Reports14, 1 (2024), 17128
2024
-
[114]
Saiph Savage, Chun Wei Chiang, Susumu Saito, Carlos Toxtli, and Jeffrey Bigham. 2020. Becoming the super turker: Increasing wages via a strategy from high earning workers. InProceedings of The Web Conference 2020. 1241–1252
2020
-
[115]
Daniel Schlagwein, Dubravka Cecez-Kecmanovic, and Benjamin Hanckel. 2019. Ethical norms and issues in crowdsourcing practices: A Habermasian analysis. Information Systems Journal29, 4 (2019), 811–837
2019
-
[116]
Yuya Shibuya, Andrea Hamm, and Teresa Cerratto Pargman. 2022. Mapping HCI research methods for studying social media interaction: A systematic literature review.Computers in Human Behavior129 (2022), 107131
2022
-
[117]
M Six Silberman, Bill Tomlinson, Rochelle LaPlante, Joel Ross, Lilly Irani, and Andrew Zaldivar. 2018. Responsible research with crowds: pay crowdworkers at least minimum wage.Commun. ACM61, 3 (2018), 39–41
2018
-
[118]
Elena Simperl. 2015. How to use crowdsourcing effectively: Guidelines and examples.LIBER Quarterly: The Journal of the Association of European Research Libraries25, 1 (2015), 18–39
2015
-
[119]
Mona Sloane, Emanuel Moss, Olaitan Awomolo, and Laura Forlano. 2022. Participation is not a design fix for machine learning. InProceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–6
2022
-
[120]
Gretchen M Spreitzer, Lindsey Cameron, and Lyndon Garrett. 2017. Alternative work arrangements: Two images of the new world of work.Annual Review of Organizational Psychology and Organizational Behavior4, 1 (2017), 473–499
2017
-
[121]
Andrew Stewart and Jim Stanford. 2017. Regulating work in the gig economy: What are the options?The Economic and Labour Relations Review28, 3 (2017), 420–437
2017
-
[122]
Rebecca E Stewart, Nathaniel Williams, Y Vivian Byeon, Alison Buttenheim, Sriram Sridharan, Kelly Zentgraf, David T Jones, Katelin Hoskins, Molly Candon, and Rinad S Beidas. 2019. The clinician crowdsourcing challenge: using participatory design to seed implementation strategi...
2019
-
[124]
Norman Makoto Su, Amanda Lazar, and Lilly Irani. 2021. Critical affects: Tech work emotions amidst the techlash.Proceedings of the ACM on Human-Computer Interaction5, CSCW1 (2021), 1–27
2021
-
[125]
Zhi Ming Tan, Nikita Aggarwal, Josh Cowls, Jessica Morley, Mariarosaria Taddeo, and Luciano Floridi. 2021. The ethical debate about the gig economy: A review and critical analysis.Technology in Society65 (2021), 101594
2021
-
[126]
Udayan Tandon, Vera Khovanskaya, and Lilly Irani. 2022. Barriers to Community-Led Real Utopias: A Case Study of Taxis in San Diego. In34th Annual Meeting. SASE. 18 Digital Labor: Challenges, Ethical Insights, and Implications Conference ’25, November 5–7, 2025, PA
2022
-
[127]
Navid Tavanapour and Eva AC Bittner. 2019. Towards supportive mechanisms for crowd collaboration–design guidelines for platform developers. InLearning and Collaboration Technologies. Ubiquitous and Virtual Environments for Learning and Collaboration: 6th International Conferen...
2019
-
[128]
Nguyen Hoang Thuan, Pedro Antunes, and David Johnstone. 2016. Factors influencing the decision to crowdsource: A systematic literature review.Information Systems Frontiers18 (2016), 47–68
2016
-
[129]
Carlos Toxtli, Christopher Curtis, and Saiph Savage. 2024. A Culturally-Aware AI Tool for Crowdworkers: Leveraging Chronemics to Support Diverse Work Styles.Proceedings of the ACM on Human-Computer Interaction8, CSCW2 (2024), 1–34
2024
-
[130]
Carlos Toxtli, Siddharth Suri, and Saiph Savage. 2021. Quantifying the invisible labor in crowd work.Proceedings of the ACM on human-computer interaction5, CSCW2 (2021), 1–26
2021
-
[131]
Paola Tubaro, Antonio A Casilli, and Marion Coville. 2020. The trainer, the verifier, the imitator: Three ways in which human platform workers support artificial intelligence.Big Data & Society7, 1 (2020), 2053951720919776
2020
-
[132]
Steven Vallas and Juliet B Schor. 2020. What do platforms do? Understanding the gig economy.Annual review of sociology46, 1 (2020), 273–294
2020
-
[133]
Niels Van Doorn. 2017. Platform labor: on the gendered and racialized exploitation of low-income service work in the ‘on-demand’economy.Information, communication & society20, 6 (2017), 898–914
2017
-
[135]
Ward van Zoonen and Anu E Sivunen. 2024. Understanding the viability conundrum in online crowdwork: The costs of unprecedented autonomy.Journal of Business Research185 (2024), 114912
2024
-
[136]
Rama Adithya Varanasi, Divya Siddarth, Vivek Seshadri, Kalika Bali, and Aditya Vashistha. 2022. Feeling proud, feeling embarrassed: Experiences of low-income women with crowd work. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–18
2022
-
[137]
PJ Wall, D Saxena, and S Brown. 2021. Artificial intelligence in the Global South (AI4D): potential and risks. ArXiv 2021.arXiv preprint arXiv:2108.10093(2021)
2021 arXiv
-
[138]
Gwendolyn Paige Watson, Lauren D Kistler, Baylor A Graham, and Robert R Sinclair. 2021. Looking at the gig picture: Defining gig work and explaining profile differences in gig workers’ job demands and resources.Group & Organization Management46, 2 (2021), 327–361
2021
-
[139]
Mark E Whiting, Grant Hugh, and Michael S Bernstein. 2019. Fair work: Crowd work minimum wage with one line of code. InProceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 197–206
2019
-
[140]
Cedric Deslandes Whitney and Justin Norman. 2024. Real risks of fake data: Synthetic data, diversity-washing and consent circumvention. InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. 1733–1744
2024
-
[141]
David Gray Widder. 2024. Epistemic power in AI ethics labor: Legitimizing located complaints. InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. 1295–1304
2024
-
[142]
Leslie Willcocks. 2019. Hidden in plain sight: The ghost in the automation and future of work debate.LSE Business Review30 (2019)
2019
-
[143]
Alex C Williams, Gloria Mark, Kristy Milland, Edward Lank, and Edith Law. 2019. The perpetual work life of crowdworkers: How tooling practices increase fragmentation in crowdwork.Proceedings of the ACM on Human-Computer Interaction3, CSCW (2019), 1–28
2019
-
[144]
2021.Algorithmic management consequences for work organisation and working conditions
Alex J Wood. 2021.Algorithmic management consequences for work organisation and working conditions. Technical Report. JRC Working Papers Series on Labour, Education and Technology
2021
-
[145]
Alex J Wood, Mark Graham, Vili Lehdonvirta, and Isis Hjorth. 2019. Good gig, bad gig: autonomy and algorithmic control in the global gig economy.Work, employment and society33, 1 (2019), 56–75
2019
-
[146]
Alex J Wood, Vili Lehdonvirta, and Mark Graham. 2018. Workers of the Internet unite? Online freelancer organisation among remote gig economy workers in six Asian and African countries.New Technology, Work and Employment33, 2 (2018), 95–112
2018
-
[147]
Dongyuan Wu and Jason L Huang. 2024. Gig work and gig workers: An integrative review and agenda for future research.Journal of Organizational Behavior45, 2 (2024), 183–208
2024
-
[148]
Yuan Xiang, Jing Du, Xue Ni Zheng, Li Rong Long, and Huan Yan Xie. 2024. Judging in the Dark: How Delivery Riders Form Fairness Perceptions Under Algorithmic Management.Journal of Business Ethics(2024), 1–18
2024
-
[149]
Forouzan Yazdanipoor, Hady Faramarzi, and Abdollah Bicharanlou. 2022. Digital labour and the generation of surplus value on Instagram.tripleC: Communication, Capitalism & Critique. Open Access Journal for a Global Sustainable Information Society20, 2 (2022), 179–194
2022
-
[150]
Angie Zhang, Alexander Boltz, Chun Wei Wang, and Min Kyung Lee. 2022. Algorithmic management reimagined for workers and by workers: Centering worker well-being in gig work. InProceedings of the 2022 CHI conference on human factors in computing systems. 1–20
2022
-
[151]
Jing Zhang, Victor S Sheng, Qianmu Li, Jian Wu, and Xindong Wu. 2017. Consensus algorithms for biased labeling in crowdsourcing.Information Sciences382 (2017), 254–273
2017
-
[152]
Jun Zou, Bin Ye, Lie Qu, Yan Wang, Mehmet A Orgun, and Lei Li. 2018. A proof-of-trust consensus protocol for enhancing accountability in crowdsourcing services.IEEE Transactions on Services Computing12, 3 (2018), 429–445. Received 20 February 2007; revised 12 March 2009; accep...
2018
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.