Pith. sign in

REVIEW 3 major objections 4 minor 13 references

Content Moderation Futures

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that content moderation fails because platforms structurally prioritize growth and novelty over user safety, and argues that worker solidarity is the path to change.

desk verdict A transparent, richly detailed qualitative study whose central 'necessarily' claim overreaches its network-recruited, Western-skewed sample, but which deserves serious peer review and revision, not rejection. read the letter →

arxiv 2509.09076 v1 pith:CBJGI7CJ submitted 2025-09-11 cs.HC cs.CY

Lindsay Blackwell This is my paper · ORCID
classification cs.HCcs.CY
keywords contentmoderationplatformgovernanceTrust&Safetyworkersolidarityparticipatorydesignsocialworldscareworkbusinessincentives
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that social media platforms fail at content moderation not because the technical problem is unsolvable, but because their business model makes safety subordinate to growth and novelty. The evidence comes from six participatory design workshops with 33 professionals across industry and civil society, who agreed that successful moderation is principled, consistent, contextual, proactive, transparent, and accountable—and that current companies miss all six. The author concludes that the binding constraint is structural misalignment between corporate incentives and public interest, and that worker solidarity among governance professionals is the main lever for change.

What carries the argument

The analytic engine is the social worlds map, which organizes platform governance into four groups—business, state, public, and working class—with business dominant. This framework demonstrates that governance outcomes reflect power imbalances among these worlds, not technical choices, and it grounds the argument that only collective worker action can shift the balance. The future workshop method supplies the empirical data by having participants define an ideal governance future and compare it to present failures.

What would settle it

A counterexample: a major investor-owned platform that maintains high safety investment and measurable harm reduction while deliberately forgoing profitable growth opportunities would contradict the claimed necessity. Alternatively, a representative study of outsourced moderators in the Majority World attributing failures to technical or operational issues rather than corporate incentives would weaken the structural argument.

Watch

Extended reading notes

Core claim

The central claim is that successful governance is undermined by the pursuit of technological novelty and rapid growth, so platforms necessarily prioritize innovation and expansion over public trust and safety. The paper supports this through consensus among participants that good moderation is principled, consistent, contextual, proactive, transparent, and accountable, alongside documentation of how business incentives, chronic underinvestment, scale, and exploitative labor produce failure on every dimension. It positions content moderation as essential but undervalued care work, arguing that organized solidarity across governance workers is the foundation for meaningful reform.

Load-bearing premise

The claim that platforms 'necessarily' prioritize growth over safety rests on interviews with 33 professionals who are largely Western, white, and highly educated; the author herself notes that more precarious commercial moderators in the Majority World are absent, so their experiences could qualify or falsify the structural diagnosis.

Editorial extensions

If this is right

  • If platforms necessarily prioritize growth over safety, then incremental policy tweaks inside companies will not fix moderation; structural change is required.
  • Worker solidarity across industry, civil society, and outsourced moderators becomes the primary mechanism for improving governance.
  • The paper's care-work framing implies moderation should be treated as permanent, skilled labor with fair pay and support, not outsourced and hidden.
  • Because scale flattens context, effective moderation may require localized, community-governed alternatives to uniform global policies.
  • Regulatory pressure alone is unlikely to be sufficient, since the paper shows companies can resist or loophole around accountability mechanisms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: compare platforms with alternative ownership models (non-profits, co-ops) against investor-owned ones to see whether the 'necessarily' claim holds or whether ownership structure is the true variable.
  • The paper's absence of Majority World moderator voices leaves open the possibility that the growth-vs-safety trade-off is experienced differently in outsourced settings, where the immediate problem may be wages and working conditions rather than corporate strategy.
  • The author's solidarity prescription implies unionization or professional associations could be a measurable lever: one could track whether platforms with unionized Trust & Safety workforces show different moderation outcomes.
  • If the care-work framing is right, worker burnout and turnover rates could serve as leading indicators of governance quality—a proxy regulators might adopt.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper reports a qualitative study of content moderation professionals' lived experiences and visions for platform governance. Six participatory design workshops were conducted with 33 professionals recruited via judgment sampling emphasizing expertise and relational trust. The author presents a thematic analysis with 36 codes, organized around participants' agreement that successful moderation is principled, consistent, contextual, proactive, transparent, and accountable; their diagnosis that current platforms fail on these dimensions because of vague Western-centric policies, the flattening effects of scale, 'growth at all costs' business incentives, chronic underinvestment, and exploitative labor arrangements; and their proposed alternatives including proactive design, responsive regulation, user education, and worker solidarity. The paper argues that platforms 'necessarily prioritize innovation and expansion over public trust and safety' and frames content moderation as undervalued care work that should be re-centered through collective worker action.

Significance. If the central claim is accepted, the paper makes a meaningful contribution by bringing the perspectives of a broad range of Trust & Safety professionals—from policy, operations, research, and civil society—into a literature that has largely focused on frontline commercial moderators or external critiques. The methodological reporting is unusually detailed: recruitment criteria, workshop structure, the 36-code thematic analysis, participant tables, and a positionality statement are all provided, and quotations are used throughout to support the findings. The author also explicitly acknowledges limitations, including the absence of precarious Majority-World workers and the Western, well-educated skew of the sample. These are real strengths. The weakness is the gap between the situated, interpretive evidence and the universal, necessity-framed conclusion in the abstract and discussion. The paper is best read as a rich professional-world diagnosis rather than a proof that all platforms under all conditions must behave this way; the revision should close that gap by qualifying the claim or adding independent institutional evidence.

major comments (3)
  1. [Abstract; §1.6; §1.4.1.2] The central claim that platforms 'necessarily prioritize innovation and expansion over public trust and safety' is not supported by the sampling frame. The evidence comes from 33 professionals recruited through judgment sampling with 'relational trust' as an explicit criterion (§1.4), and the author's own limitations paragraph (§1.4.1.2) concedes that the sample is 'largely (though not exclusively) Western, white, and highly educated' and that 'more precarious technology governance workers' and Majority-World perspectives are absent. This absence is load-bearing because 'necessarily' asserts a structural property of the entire platform economy, not merely the perceptions of this participant group. As reported, the data can sustain a situated diagnosis: these professionals experienced and perceived a systematic misalignment. They cannot, by themselves, establish necessity. I recommend ref
  2. [§1.5.2.4; §1.6.3] The labor argument and the worker-solidarity prescription rely heavily on secondhand accounts of outsourced commercial moderators. P2, P3, P22, and P31 speak about content moderators in the third person, and no participant is identified as a currently employed commercial content moderator in a BPO in the Majority World. The paper acknowledges this at §1.4.1.2. Since the paper argues that organizing care labor is the principal lever for change, the absence of the very workers who are to be organized is not an incidental diversity gap. Either direct recruitment of such workers, or a more central engagement with published first-person accounts of commercial moderation (e.g., Roberts 2019), should be incorporated; alternatively, the solidarity claim should be explicitly limited to the professional groups actually represented in the workshops.
  3. [§1.5.1–§1.5.2; §1.4] The workshop design combines a future-workshop 'magic wand' phase with the KJ method, which is well suited to surfacing shared ideal values. However, the paper moves from participants' agreement on the six criteria to treating those criteria as an objective benchmark against which platforms 'fail.' Because recruitment prioritized relational trust and expertise, the group may share a professional worldview; consensus among such a group is evidence about that professional social world, not independent validation of the criteria. The analysis would be strengthened by explicitly treating the six criteria as a collectively produced normative vision and by reporting points of disagreement or contestation rather than only convergence. This does not invalidate the findings, but it sharpens the epistemic weight placed on them.
minor comments (4)
  1. [§1.4] The phrase 'future workshop' is capitalized inconsistently. Also, the description of the 'magic wand' approach could state more explicitly whether participants were asked to suspend business-model constraints only, or all constraints including legal and technical ones.
  2. [§1.5.3.4] Figure 1.2 is described as 'shared by P33' and based on Braithwaite's responsive regulation; please clarify whether the figure is reproduced from workshop artifacts or redrawn by the author, and ensure permission/attribution if needed.
  3. [§1.3–References] Some reference titles have inconsistent capitalization and formatting (e.g., 'Techlash or Tech Change?' and 'New Worlds Arise'); please conform to the target journal's style. A few references are cited as 2025 works that may still be in press; mark them as such.
  4. [§1.4.1.1] The positionality statement is a strength, but it does not address how the author's industry roles and network may have shaped which participants felt safe to participate or how they framed their accounts during workshops. A sentence on this would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; qualitative study is self-contained, with primary workshop data supporting interpretive claims.

full rationale

The paper's central argument—that platform governance fails because corporate incentives prioritize innovation and growth over safety—is an interpretive synthesis of participant testimony and cited literature, not a formal derivation. No parameter is fitted and then predicted; no equation or definition makes the conclusion equivalent to its inputs by construction. Participants' stated criteria for successful moderation (principled, consistent, contextual, proactive, transparent, accountable) and their accounts of corporate failure are independent data points that the author organizes into themes. The author's self-citations (e.g., Blackwell et al. 2017; Schoenebeck & Blackwell 2021) appear in related work and discussion but are not load-bearing premises that force the conclusion. The sampling limitations conceded in §1.4.1.2 (Western, white, highly educated; absence of precarious Majority-World workers) bear on generalizability of the 'necessarily' claim, but that is a validity/evidence-strength issue, not circularity. No self-definitional reduction, fitted-input-called-prediction, imported uniqueness theorem, or ansatz-smuggling-via-citation is present. The findings are grounded in directly reported workshop data, and the analysis does not reduce to its inputs by construction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No numeric free parameters apply: this is a qualitative study with no fitted quantities; the 36 analytical codes are researcher-created categories, not parameters fit to data. The paper introduces no invented entities; the social worlds map (Figure 1.1) and the responsive-regulation pyramid (Figure 1.2) are analytic devices drawn from existing frameworks (Clarke; Braithwaite). The load-bearing assumptions are methodological: self-reported testimony, sample representativeness, single-researcher coding, workshop facilitation neutrality, and imported interpretive frameworks, each detailed above.

assumptions (5)
  • domain assumption Participant statements in workshops are treated as truthful, accurate descriptions of platform governance practice and company behavior.
    The entire results section (§1.5) rests on self-reports by 33 practitioners; claims about underinvestment, outsourcing conditions, executive influence, and company motives are not independently verified with company records, audits, or observational data.
  • domain assumption The recruited sample reaches 'saturation' and is adequate to support generalizations about content moderation professionals and platform behavior.
    Recruitment combined judgment sampling with relational trust (§1.4); the author's own Limitations section notes the sample skews Western, white, and highly educated and excludes precarious Majority World moderators, which limits generalization of claims like 'current technology companies fail to achieve these goals'.
  • domain assumption Single-researcher inductive thematic analysis yields themes that faithfully represent participant views.
    Coding was performed by hand by the author with one pilot-coding discussion with a research assistant; no inter-rater agreement statistics, full codebook, or member checking is reported (§1.4, 'I conducted a thematic analysis').
  • domain assumption The future workshop protocol surfaces participants' genuine collective visions rather than facilitator-shaped outputs.
    The author designed and facilitated all six workshops and disclosed her industry positionality (§1.4.1.1); the method assumes the three-stage structure (inspiration, ideation, implementation) elicits participant priorities without steering them toward the reported conclusions.
  • domain assumption The interpretive lenses of care work, invisible labor, and social worlds (from Star and Strauss; Fraser; Irani; Clarke) validly apply to content moderation.
    The Discussion (§1.6.3) imports these frameworks from prior literature to reinterpret the findings; the paper does not independently justify their applicability to this setting beyond asserting it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Content Moderation Futures." pith.science (2026). https://pith.science/paper/CBJGI7CJ

@misc{pith2026250909076,
  author       = {Pith},
  title        = {Pith review of: Content Moderation Futures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CBJGI7CJ}},
  note         = {Machine review of arXiv:2509.09076}
}
read the original abstract

This study examines the failures and possibilities of contemporary social media governance through the lived experiences of various content moderation professionals. Drawing on participatory design workshops with 33 practitioners in both the technology industry and broader civil society, this research identifies significant structural misalignments between corporate incentives and public interests. While experts agree that successful content moderation is principled, consistent, contextual, proactive, transparent, and accountable, current technology companies fail to achieve these goals, due in part to exploitative labor practices, chronic underinvestment in user safety, and pressures of global scale. I argue that successful governance is undermined by the pursuit of technological novelty and rapid growth, resulting in platforms that necessarily prioritize innovation and expansion over public trust and safety. To counter this dynamic, I revisit the computational history of care work, to motivate present-day solidarity amongst platform governance workers and inspire systemic change.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 1 linked inside Pith

  1. [1]

    Ultimately, the communication flows one way—it comes top-down,

    “I think this is also something that is hard on ICs. I worked on escalations for a couple of years when I first started. A lot of times, we’d get escalations from foreign governments asking us to take down ‘fake accounts’ that were critical of the government. There were numerous examples of situations where leadership—I’ll be intentionally vague here—woul...

    2002

  2. [4]

    I Got Flagged for Supposed Bullying, Even Though It Was in Response to Someone Harassing Me About My Disability

    Keller, D., & Leerssen, P. (2020). Facts and Where to Find Them: Empirical Research on Internet Platforms and Content Moderation. In N. Persily & J. Tucke (Eds.), Social Media and Democracy: The State of the Field and Prospects for Reform, 220–251. Cambridge University Press. 72 Kiesler, S., Kraut, R., Resnick, P., & Kittur, A. (2012). Regulating Behavior...

    2020

  3. [7]

    73 Meta. (2025a). More Speech and Fewer Mistakes. Newsroom. http://about.fb.com/news/2025/01/ meta-more-speech-fewer-mistakes/ Meta. (2025b). Regulation (EU) 2022/2065 Digital Services Act Transparency Report for Facebook. Transparency Center. http://transparency.meta.com/reports/regulatory-transparency-reports/ Mezrich, B. (2023). Breaking Twitter: Elon ...

    2025

  4. [8]

    At the End of the Day Facebook Does What It Wants

    U.S. Equal Employment Opportunity Commission. (2024, September 10). High Tech, Low Inclusion: Diversity in the High Tech Workforce and Sector, 2014-2022. Vaccaro, K., Sandvig, C., & Karahalios, K. (2020). “At the End of the Day Facebook Does What It Wants”: How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on Human-Com...

    2024

  5. [13]

    Thach, H., Mayworm, S., Delmonaco, D., & Haimson, O. (2024). (In)visible Moderation: A Digital Ethnography of Marginalized Users and Content Moderation on Twitch and Reddit. New Media & Society, 26(7), 4034–4055. Trust & Safety Professional Association (2025). About Us. http://www.tspa.org/about-tspa Tyler, T. R. (2006). Why People Obey the Law. Princeton...

    2024

  6. [217]

    Tyler, T., Katsaros, M., Meares, T., & Venkatesh, S. (2021). Social Media Governance: Can Social Media Companies Motivate Voluntary Rule Following Behavior Among Their Users? Journal of experimental criminology, 17(1), 109–127. Tyler, T., Meares, T., & Katsaros, M. (2025). New Worlds Arise: Online Trust and Safety. Annual Review of Criminology,

    2021

  7. [1844]

    Massanari, A. (2017). #Gamergate and The Fappening: How Reddit’s Algorithm, Governance, and Culture Support Toxic Technocultures. New Media & Society, 19(3), 329–346. Maxim, K., Parecki, J., & Cornett, C. (2022). How to Build a Trust and Safety Team In a Year: A Practical Guide From Lessons Learned (So Far) at Zoom. Journal of Online Trust and Safety, 1(4...

    2017

  8. [1984]

    I truly believe that Facebook chose to go to end-to-end encryption to avoid regulation,

    But at the same time, I look at countries that do have some restrictions in place, and their CSAM risk is almost eliminated in some cases—so I do think that some government intervention might be necessary. I worry that tech companies are sacrificing the well-being of society for greed; for making money. I want more government intervention and more laws, w...

    2020

Show all 13 references
  1. [2010]

    colonial impulse

    while accumulating massive amounts of valuable user data (Zuboff, 2018), which they use to optimize engagement, predict behavior, and deliver targeted advertising in increasingly automated ways. These logics of extraction result in platform designs that tolerate or even encour...

    2018

  2. [2012]

    little old ladies

    and further marginalizing vulnerable users by reproducing global power asymmetries and entrenching structural harms. Through the lens of scalability, social media platforms become sites of contested governance, where decisions about speech, visibility, and harm are shaped by c...

    2019

  3. [2016]

    sociotechnical stack

    is the organization of social media governance labor, resisting institutional atomization and centering the workplace as 66 an essential location for contesting the possibilities of governance. Aligned with recent calls for the promotion of a worker-centered HCI (Irani, 2015b;...

    2017

  4. [2019]

    new and flashy

    to disguise the power dynamics that determine who will succeed and who fails (Becerra & Thomas, 2023). Innovation promises novel solutions to human problems, but also the production of wealth for individual innovators, complicating distinctions between public good and private ...

    2023

  5. [2024]

    Abhorrent Violent Material

    offers a pathway toward redistributing power and normalizing the value of care. Ultimately, while improving the future state of scaled content moderation will require a host of systemic changes, worker solidarity is foundational to achieving equitable governance and broader so...

    2020 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.