Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Position: The Most Expensive Part of an LLM should be its Training Data

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that the human labor behind LLM training data is worth 10 to 1,000 times more than the compute, energy, and engineering used to train the models, so compensating data creators should be the largest cost of building an LLM.

desk verdict A transparent, honestly-scoped position paper whose headline 10-1000x claim needs a sensitivity analysis before it can carry the weight it is asked to carry. read the letter →

arxiv 2504.12427 v1 pith:EGM6MDHF submitted 2025-04-16 cs.CL cs.AIcs.CYcs.LG

Nikhil Kandpal , Colin Raffel This is my paper · ORCID
classification cs.CLcs.AIcs.CYcs.LG
keywords largelanguagemodelstrainingdatacostlaborreplacementvaluationcompensationcopyrightandfairuselicensingAIeconomics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models are usually priced by the cost of compute, energy, and engineering, but this paper argues that the largest hidden cost is the human effort that produced the words they learn from. To measure that effort, the authors estimate what it would cost to pay writers to recreate the training corpora of 64 LLMs released between 2016 and 2024, using a deliberately low typing speed of 30 words per minute and a $3.85 per hour median global minimum wage. Across every model, the replacement labor cost of the training data is 10 to 1,000 times larger than the estimated training-run cost, with ratios such as about 300 times for GPT-4 and 6,000 times for DeepSeek-V3. The paper concludes that fair compensation for data producers should be the dominant expense in LLM production, and that current web-scraping practices leave LLM providers carrying an enormous financial and ethical liability.

What carries the argument

The argument runs on a deliberately conservative word-to-dollar conversion: a training dataset is valued by dividing its word count by 30 words per minute and multiplying by $3.85 per hour, the median statutory minimum wage across 167 countries. This replacement-cost formula converts an unmeasured stock of writing labor into a dollar amount that can be set directly against hardware, energy, and engineering costs from existing training-cost estimates. The machinery intentionally ignores text quality, expertise, research, and editing, which means every ratio it produces is meant to be a lower bound on the true gap.

What would settle it

For one named frontier model, tally every actual or legally required payment to training-data creators, including licensing fees, opt-in program payouts, and court-ordered settlements, and compare that total with the publicly reported training-run cost; the paper's claim that data compensation should be the dominant cost fails if the realized total is smaller than the training cost.

Watch

Extended reading notes

Core claim

The central discovery is a quantitative mismatch between what it costs to run an LLM training job and what it would cost to produce the text that makes the model useful. Taking a replacement-cost view of data, the authors compute dataset value as the total number of words times the time needed to type them at 30 words per minute at the median global minimum wage, and compare that figure with prior estimates of training cost for the same models. The comparison shows data labor exceeding training cost for all 64 models, by two to three orders of magnitude in the most extreme recent cases. From this the authors draw the positional claim: because the human labor in training data is an order-of-magnitude larger than every other expense combined, compensation to data creators should be the most expensive part of producing an LLM, even though it is almost never paid.

Load-bearing premise

The load-bearing premise is that a training corpus is worth what it would cost to type every word of it from scratch at a very low wage, even though most of that text was already produced for other purposes and a replacement cost is not the same thing as the compensation actually owed to its creators.

Editorial extensions

If this is right

  • Fair compensation at the paper's deliberately low rates would make frontier LLM training unaffordable for most current providers, with data costs exceeding the total annual revenue of 3 of the 10 major companies examined.
  • The implicit labor debt backed by training data is growing, roughly doubling every eight months, so the gap between data value and data compensation will widen unless data collection or algorithmic efficiency changes.
  • LLM providers would need to restructure data collection around permissively licensed and public-domain text, opt-in contribution, and verified authorship metadata to make any compensation scheme practical.
  • Research should shift from compute-optimal to price-optimal language models, allocating a fixed financial budget between acquiring data and training rather than treating scraped data as free.
  • Sustainable compensation is more likely to come from royalty or revenue-sharing structures than from upfront payment, because upfront payment at these rates is infeasible for all but the wealthiest organizations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same word-to-dollar formula could be standardized as a labor-intensity metric on model cards, letting researchers, regulators, and consumers compare the hidden human cost of different models alongside FLOPs and parameter counts.
  • If training data were priced in, the economically rational scaling strategy might favor far smaller, higher-value datasets or synthetic-data-heavy pipelines, meaning current web-scale scraping may be inefficient as well as uncompensated.
  • The paper's numbers provide a lower-bound anchor for legal settlements and licensing negotiations: actual fair-use rulings or copyright settlements could convert this implicit labor debt into a real balance-sheet liability for LLM providers.
  • Using professional writer wages or realistic research-and-editing speeds would push the reported ratios even higher, so the 10-to-1,000-fold gap is best read as a floor on the true disparity.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper argues that the monetary value of the human labor required to produce LLM training datasets dwarfs the cost of training the models themselves. It estimates dataset production cost as the number of word units in each training set multiplied by an assumed hourly wage divided by an assumed writing speed, using $3.85/hour and 30 words per minute as deliberately conservative point estimates. Applying this valuation to 64 LLMs released between 2016 and 2024, the authors report that dataset costs are 10–1000x larger than training costs, that several recent datasets would be valued above $10 billion, and that compensating data creators at even these low rates would be unaffordable for most of the ten companies examined. The paper concludes with research directions on data collection, data efficiency, and compensation structures.

Significance. If the quantitative claim is robust, the paper makes an important contribution to the policy and economics of AI training data, connecting labor valuation, copyright debates, and affordability in a way that few prior works do. Its strengths include a transparent closed-form calculation that is not fitted to a target outcome, an explicit intent to bias assumptions downward, and a fair presentation of opposing views in Section 5. The paper is also careful to describe its valuation as a deliberate lower bound rather than a market-price estimate. The main risk is not circularity but parameter sensitivity: the headline 10–1000x range depends on two point estimates and an undefined token-to-word conversion, and the paper does not currently provide enough data or sensitivity analysis for the quantitative claim to be independently verified. The directional claim that dataset labor value greatly exceeds training cost is likely to persist under many alternative assumptions, but the specific magnitudes and the phrase “even under highly conservative estimates” are not yet fully established.

major comments (4)
  1. [Section 2.2 and Section 3.1] The headline „10–1000x“ claim rests on two point estimates, $3.85/hour and 30 words per minute, with no sensitivity analysis. The calculation is simply dataset words × wage ÷ speed, so a wage of $1/hour and a speed of 60 wpm would reduce estimated dataset costs by a factor of about 7.7 relative to the paper’s numbers. Because the paper asserts that the result holds even under conservative assumptions, the authors should report the ratio range across a plausible grid of wage and speed values, and should state the minimum ratio across the 64 models under each parameter choice.
  2. [Section 2.2 and Section 3.1] The conversion from tokens to word units is never defined. Training dataset sizes are typically reported in tokens, while Section 2.2 refers to “word units,” and treating tokens as words can inflate English text cost by roughly 1.3–1.5x. The paper should state explicitly whether dataset sizes are token counts, and if so provide the token-to-word conversion used and justify it, since this choice directly scales every dataset cost estimate.
  3. [Section 3.1] The per-model data underlying Figure 1 and the 10–1000x range are not included. Without a table of each model’s training cost, dataset size, and computed ratio, the claim that “training data costs are at least an order of magnitude larger than training costs” for all 64 models cannot be checked, and the minimum ratio cannot be evaluated under alternative parameter choices. An appendix with these values would make the central quantitative claim reproducible.
  4. [Section 2.2] The paper equates the value of a training dataset with its replacement cost, and then uses that value as an estimate of compensation owed to training data producers. This conflates the cost of re-creating text from scratch with the amount owed to the original authors of text that already exists for other purposes. The authors should either reframe the analysis as a hypothetical replacement-cost valuation, clearly separated from a claim about legal or moral compensation owed, or add a discussion of why replacement cost is the right basis for compensation. The direction of the gap would likely persist under this reframing, but the direct policy meaning changes.
minor comments (4)
  1. [Throughout] There are several typographical errors, including “heteregeneous” (Section 2.2), “the the optimal balance” (Section 4.2), inconsistent spelling of “Britannica” as “Brittanica” (Table 1 and Section 3.1), and “V o” in the Vox Media reference.
  2. [Figure 1] The figure lacks clear axis labels and a visible scale; the annotated ratios 300x and 6000x are not evident from the plotted points. The y-axis should be labeled with units and the annotations should be tied to specific points.
  3. [Section 3.2] The ten organizations in Figure 3 are described in the text, but the figure does not state the revenue year or the source for each company’s revenue; adding these details would make the affordability comparison easier to audit.
  4. [Section 2.2] The wage and speed assumptions are described qualitatively as “conservative,” but no evidence is given that 30 wpm is a lower bound for coherent text production rather than a typical value; a reference for adult typing speeds and for the $1–$50 annotation wage range would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the dataset-cost estimate is an explicit closed-form calculation with no fitted inputs or load-bearing self-citations.

full rationale

The paper's central quantitative claim is not derived from its conclusion. Section 2.2 defines dataset value as replacement cost, asking 'How much would it cost to pay workers to write a collection of coherent text equal in size to a given training dataset?' and then sets two explicit exogenous parameters: 30 words per minute and $3.85/hour, justified by external references (typing-speed assumptions and ILO wage data). The per-model dataset cost is a closed-form product of these parameters and reported dataset sizes, while training costs are imported from Cottier et al. (2024) and Epoch AI rather than fitted to reproduce the headline ratio. There is no fitted parameter later relabeled as a prediction, no equation that presupposes the conclusion, and no load-bearing self-citation: the Raffel-coauthored references (Muennighoff et al. 2023; Penedo et al. 2024) appear only as supporting context for data-efficiency research and data-quality trends, not as justification for the cost comparison. The normative conclusion that compensation 'should be' the most expensive part is an argument built on the chosen valuation approach, not a mathematical consequence of it. The main vulnerability is sensitivity of the 10-1000x range to the wage/speed and token-to-word assumptions, but sensitivity and assumption criticism are correctness concerns, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The core estimate is a direct multiplication of dataset size, writing speed, and wage rate. The two free parameters are deliberately chosen to underestimate costs. The largest conceptual assumption is that replacement labor cost is the appropriate measure of compensation owed, which is a normative choice.

free parameters (2)
  • Writing speed = 30 words per minute
    Assumed in Section 2.2 as a lower-end typing speed; directly determines per-word labor cost.
  • Hourly wage = $3.85 USD per hour
    Median minimum wage across 167 countries, used in Section 2.2; deliberately conservative.
assumptions (4)
  • domain assumption Training data value equals the cost to reproduce it from scratch.
    Section 2.2 introduces replacement cost as the valuation basis.
  • domain assumption All text can be valued uniformly by word count, regardless of quality or type.
    Section 2.2 states text quality is ignored and only word units are used.
  • domain assumption Training cost estimates from Cottier et al. (2024) are accurate for hardware, energy, and engineering labor.
    Section 2.1 adopts this methodology without independent verification.
  • ad hoc to paper Creators of training data are morally entitled to compensation.
    The paper's normative stance, stated in the abstract and conclusion; not derivable from the cost estimate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Position: The Most Expensive Part of an LLM should be its Training Data." pith.science (2026). https://pith.science/paper/EGM6MDHF

@misc{pith2026250412427,
  author       = {Pith},
  title        = {Pith review of: Position: The Most Expensive Part of an LLM should be its Training Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EGM6MDHF}},
  note         = {Machine review of arXiv:2504.12427}
}
read the original abstract

Training a state-of-the-art Large Language Model (LLM) is an increasingly expensive endeavor due to growing computational, hardware, energy, and engineering demands. Yet, an often-overlooked (and seldom paid) expense is the human labor behind these models' training data. Every LLM is built on an unfathomable amount of human effort: trillions of carefully written words sourced from books, academic papers, codebases, social media, and more. This position paper aims to assign a monetary value to this labor and argues that the most expensive part of producing an LLM should be the compensation provided to training data producers for their work. To support this position, we study 64 LLMs released between 2016 and 2024, estimating what it would cost to pay people to produce their training datasets from scratch. Even under highly conservative estimates of wage rates, the costs of these models' training datasets are 10-1000 times larger than the costs to train the models themselves, representing a significant financial liability for LLM providers. In the face of the massive gap between the value of training data and the lack of compensation for its creation, we highlight and discuss research directions that could enable fairer practices in the future.

Figures

Figures reproduced from arXiv: 2504.12427 by the authors.

Figure 1
Figure 1. Estimated costs for LLMs’ training datasets surpass the costs of training by 1-3 orders of magnitude. Above, we plot the training costs of 64 language models released between 2016 and 2024 along with the estimated cost of their training datasets. lions of US dollars (Cottier et al., 2024) and billion-dollar training runs projected in the near future (Morales, 2024). Amid these rapidly escalating costs, one major exp… view at source ↗
Figure 2
Figure 2. Over time the estimated labor costs to produce the con￾tent of LLM training datasets has increased, with numerous recent models having been trained on datasets that we conservatively estimate to have implicitly cost over $10 billion USD. ement. The HTML standard (W3C, 2025) defines one use of this element as providing authorship information for a document. However, this tag is not widely used for this purpose as aut… view at source ↗
Figure 3
Figure 3. The costs of training datasets used by major LLM companies make up a significant fraction of these companies’ revenues. Above we visualize training data cost as a percentage of each company’s annual revenue for ten recent LLM training runs. We denote non-publicly traded companies denoted with * to indicate that for these companies we use third-party reported revenue rather than revenues reported in official financia… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation

    cs.CR 2026-03 unverdicted novelty 6.5 of 10

    Page-granular Flex-Mem and switchable Flex-NPU cut TrustZone LLM TTFT by ~10× vs a CMA strawman and ~2.4× vs a pipelined secure-NPU strawman on RK3588.

  2. The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A new 8TB openly-licensed text corpus trains 7B LLMs that are competitive with Llama 1/2, showing that performant models need not depend on unlicensed web data.

Reference graph

Works this paper leans on

48 extracted references · 22 canonical work pages · cited by 2 Pith papers

  1. [1]

    https://huggingface.co/collections/r-three/common-pile-665a13e48528df6b00416dc0, 2025

    Common Pile . https://huggingface.co/collections/r-three/common-pile-665a13e48528df6b00416dc0, 2025

    work page 2025

  2. [2]

    A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al

    Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A. A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024 a

    arXiv 2024

  3. [3]

    J., Javaheripi, M., Kauffmann, P., et al

    Abdin, M., Aneja, J., Behl, H., Bubeck, S., Eldan, R., Gunasekar, S., Harrison, M., Hewett, R. J., Javaheripi, M., Kauffmann, P., et al. Phi-4 technical report. arXiv preprint arXiv:2412.08905, 2024 b

    arXiv 2024

  4. [4]

    Alden newspapers v

    Alden Newspapers . Alden newspapers v. openai. https://assets.law360news.com/1831000/1831297/complaint.pdf, 2024. Ongoing legal case. Related to copyright issues in generative AI training

    work page arXiv 2024

  5. [5]

    M., and Weber, G

    Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F. M., and Weber, G. Common voice: A massively-multilingual speech corpus. In Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pp.\ 4211--4215, 2020

    2020

  6. [6]

    Authors guild v

    Authors Guild . Authors guild v. openai. https://authorsguild.org/app/uploads/2023/12/Authors-Guild-OpenAI-Microsoft-Class-Action-Complaint-Dec-2023.pdf, 2023. Ongoing legal case. Related to copyright issues in generative AI training

    work page 2023

  7. [7]

    K., Ahn, H., Bae, J., Zhao, K., Kang, M., Chung, Y., Pratapa, A., Neiswanger, W., Strubell, E., Mitamura, T., Schneider, J., Hovy, E., Grosse, R., and Xing, E

    Choe, S. K., Ahn, H., Bae, J., Zhao, K., Kang, M., Chung, Y., Pratapa, A., Neiswanger, W., Strubell, E., Mitamura, T., Schneider, J., Hovy, E., Grosse, R., and Xing, E. What is your data worth to gpt? llm-scale data valuation with influence functions, 2024. URL https://arxiv.org/abs/2405.13954

    arXiv 2024

  8. [8]

    Common crawl dataset

    Common Crawl . Common crawl dataset. https://commoncrawl.org/. Accessed: 2025-01-20

    work page 2025

Show all 48 references
  1. [9]

    Concord music group v

    Concord Music Group . Concord music group v. anthropic. https://fingfx.thomsonreuters.com/gfx/legaldocs/znvnklkgwpl/UNIVERSAL Ongoing legal case. Related to copyright issues in generative AI training

  2. [10]

    The rising costs of training frontier ai models, 2024

    Cottier, B., Rahman, R., Fattorini, L., Maslej, N., and Owen, D. The rising costs of training frontier ai models, 2024. URL https://arxiv.org/abs/2405.21015

    2024 arXiv

  3. [11]

    Ai is a lot of work

    Dzieza, J. Ai is a lot of work. The Verge, Jun 2023

    2023

  4. [12]

    Encyclopedia britannica 15th edition

    Encyclopedia Britannica . Encyclopedia britannica 15th edition. URL https://www.britannica.com/topic/Encyclopaedia-Britannica-English-language-reference-work/Fifteenth-edition

  5. [13]

    Data on notable ai models, 2024

    Epoch AI . Data on notable ai models, 2024. URL https://epoch.ai/data/notable-ai-models. Accessed: 2025-01-16

    2024

  6. [14]

    Gadre, S. Y., Ilharco, G., Fang, A., Hayase, J., Smyrnis, G., Nguyen, T., Marten, R., Wortsman, M., Ghosh, D., Zhang, J., Orgad, E., Entezari, R., Daras, G., Pratt, S., Ramanujan, V., Bitton, Y., Marathe, K., Mussmann, S., Vencu, R., Cherti, M., Krishna, R., Koh, P. W., Saukh,...

    2023 arXiv

  7. [15]

    Gadre, S. Y., Smyrnis, G., Shankar, V., Gururangan, S., Wortsman, M., Shao, R., Mercat, J., Fang, A., Li, J., Keh, S., Xin, R., Nezhurina, M., Vasiljevic, I., Jitsev, J., Soldaini, L., Dimakis, A. G., Ilharco, G., Koh, P. W., Song, S., Kollar, T., Carmon, Y., Dave, A., Heckel,...

    2024 arXiv

  8. [16]

    and Zou, J

    Ghorbani, A. and Zou, J. Data shapley: Equitable valuation of data for machine learning, 2019. URL https://arxiv.org/abs/1904.02868

    2019 arXiv

  9. [17]

    Evaluation of similarity-based explanations, 2021

    Hanawa, K., Yokoi, S., Hara, S., and Inui, K. Evaluation of similarity-based explanations, 2021. URL https://arxiv.org/abs/2006.04528

    2021 arXiv

  10. [18]

    A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Rae, J

    Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Rae, J. W., Vin...

    2022 arXiv

  11. [19]

    Statistics on wages

    International Labor Organization . Statistics on wages. https://ilostat.ilo.org/topics/wages/, 2025

    2025

  12. [20]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models, 2020. URL https://arxiv.org/abs/2001.08361

    2020 arXiv

  13. [21]

    Koh, P. W. and Liang, P. Understanding black-box predictions via influence functions, 2020. URL https://arxiv.org/abs/1703.04730

    2020 arXiv

  14. [22]

    M., Stanley, O., Nagyfi, R., ES, S., Suri, S., Glushkov, D., Dantuluri, A., Maguire, A., Schuhmann, C., Nguyen, H., and Mattick, A

    Köpf, A., Kilcher, Y., von Rütte, D., Anagnostidis, S., Tam, Z.-R., Stevens, K., Barhoum, A., Duc, N. M., Stanley, O., Nagyfi, R., ES, S., Suri, S., Glushkov, D., Dantuluri, A., Maguire, A., Schuhmann, C., Nguyen, H., and Mattick, A. Openassistant conversations -- democratizin...

    2023 arXiv

  15. [23]

    Releasing Common Corpus: the largest public domain dataset for training LLMs

    Langlais, P.-C. Releasing Common Corpus: the largest public domain dataset for training LLMs . HuggingFace, Mar 2024

    2024

  16. [24]

    Datacomp-lm: In search of the next generation of training sets for language models

    Li, J., Fang, A., Smyrnis, G., Ivgi, M., Jordan, M., Gadre, S., Bansal, H., Guha, E., Keh, S., Arora, K., et al. Datacomp-lm: In search of the next generation of training sets for language models. arXiv preprint arXiv:2406.11794, 2024

    2024 arXiv

  17. [25]

    M., Zhou, X., Li, Y., Xiong, C., Villa, L., Biderman, S., Li, H., Ippolito, D., Hooker, S., Kabbara, J., and Pentland, S

    Longpre, S., Mahari, R., Lee, A., Lund, C., Oderinwale, H., Brannon, W., Saxena, N., Obeng-Marnu, N., South, T., Hunter, C., Klyman, K., Klamm, C., Schoelkopf, H., Singh, N., Cherep, M., Anis, A., Dinh, A., Chitongo, C., Yin, D., Sileo, D., Mataciunas, D., Misra, D., Alghamdi,...

    2024 arXiv

  18. [26]

    A., and Zettlemoyer, L

    Min, S., Gururangan, S., Wallace, E., Shi, W., Hajishirzi, H., Smith, N. A., and Zettlemoyer, L. Silo language models: Isolating legal risk in a nonparametric datastore, 2024. URL https://arxiv.org/abs/2308.04430

    2024 arXiv

  19. [27]

    AI models that cost \ 1 billion to train are underway, \ 100 billion models coming — largest current models take “only” \ 100 million to train: Anthropic CEO

    Morales, J. AI models that cost \ 1 billion to train are underway, \ 100 billion models coming — largest current models take “only” \ 100 million to train: Anthropic CEO . Tom’s Hardware, Jul 2024

    2024

  20. [28]

    M., Barak, B., Scao, T

    Muennighoff, N., Rush, A. M., Barak, B., Scao, T. L., Piktus, A., Tazi, N., Pyysalo, S., Wolf, T., and Raffel, C. Scaling data-constrained language models, 2023. URL https://arxiv.org/abs/2305.16264

    2023 arXiv

  21. [29]

    New york times v

    New York Times . New york times v. openai. https://nytco-assets.nytimes.com/2023/12/NYT_Complaint_Dec2023.pdf, 2023. Ongoing legal case. Related to copyright issues in generative AI training

    2023

  22. [30]

    M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A

    Park, S. M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A. Trak: Attributing model behavior at scale, 2023. URL https://arxiv.org/abs/2303.14186

    2023 arXiv

  23. [31]

    Pasquale, F. A. and Sun, H. Consent and compensation: Resolving generative ai’s copyright crisis. SSRN Electronic Journal, 2024. URL https://api.semanticscholar.org/CorpusID:269814777

    2024

  24. [32]

    Multiple ai companies bypassing web standard to scrape publisher sites, licensing firm says

    Paul, K. Multiple ai companies bypassing web standard to scrape publisher sites, licensing firm says. Reuters, Jun 2024

    2024

  25. [33]

    B., Lozhkov, A., Mitchell, M., Raffel, C., Werra, L

    Penedo, G., Kydlíček, H., allal, L. B., Lozhkov, A., Mitchell, M., Raffel, C., Werra, L. V., and Wolf, T. The fineweb datasets: Decanting the web for the finest text data at scale, 2024. URL https://arxiv.org/abs/2406.17557

    2024 arXiv

  26. [34]

    Estimating training data influence by tracing gradient descent, 2020

    Pruthi, G., Liu, F., Sundararajan, M., and Kale, S. Estimating training data influence by tracing gradient descent, 2020. URL https://arxiv.org/abs/2002.08484

    2020 arXiv

  27. [35]

    Reddit and openai build partnership

    Reddit. Reddit and openai build partnership. https://redditinc.com/blog/reddit-and-oai-partner, 2024

    2024

  28. [36]

    Thomson reuters' adjusted eps beats expectations, ai boosts results

    Reuters . Thomson reuters' adjusted eps beats expectations, ai boosts results. https://www.reuters.com/business/media-telecom/thomson-reuters-reports-higher-fourth-quarter-revenue-2024-02-08/, 2024

    2024

  29. [37]

    Shutterstock expands partnership with openai, signs new six-year agreement to provide high-quality training data

    Shutterstock . Shutterstock expands partnership with openai, signs new six-year agreement to provide high-quality training data. https://www.prnewswire.com/news-releases/shutterstock-expands-partnership-with-openai-signs-new-six-year-agreement-to-provide-high-quality-training-...

    2023

  30. [38]

    H., Kumar, S., Lucy, L., Lyu, X., Lambert, N., Magnusson, I., Morrison, J., Muennighoff, N., Naik, A., Nam, C., Peters, M

    Soldaini, L., Kinney, R., Bhagia, A., Schwenk, D., Atkinson, D., Authur, R., Bogin, B., Chandu, K., Dumas, J., Elazar, Y., Hofmann, V., Jha, A. H., Kumar, S., Lucy, L., Lyu, X., Lambert, N., Magnusson, I., Morrison, J., Muennighoff, N., Naik, A., Nam, C., Peters, M. E., Ravich...

    2024 arXiv

  31. [39]

    The atlantic announces product and content partnership with openai

    The Atlantic . The atlantic announces product and content partnership with openai. https://www.theatlantic.com/press-releases/archive/2024/05/atlantic-product-content-partnership-openai/678529/, 2024

    2024

  32. [40]

    Llama: Open and efficient foundation language models, 2023 a

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. Llama: Open and efficient foundation language models, 2023 a . URL https://arxiv.org/abs/2302.13971

    2023 arXiv

  33. [41]

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N....

    2023 arXiv

  34. [42]

    Will we run out of data? limits of llm scaling based on human-generated data, 2024

    Villalobos, P., Ho, A., Sevilla, J., Besiroglu, T., Heim, L., and Hobbhahn, M. Will we run out of data? limits of llm scaling based on human-generated data, 2024. URL https://arxiv.org/abs/2211.04325

    2024 arXiv

  35. [43]

    Vox media and openai form strategic content and product partnership

    Vox Media . Vox media and openai form strategic content and product partnership. https://www.voxmedia.com/2024/5/29/24166483/vox-media-openai-strategic-content-and-product-partnership, 2024

    2024

  36. [44]

    Html standard

    W3C . Html standard. https://html.spec.whatwg.org, 2025

    2025

  37. [45]

    Call for papers -- the babylm challenge: Sample-efficient pretraining on a developmentally plausible corpus, 2023

    Warstadt, A., Choshen, L., Mueller, A., Williams, A., Wilcox, E., and Zhuang, C. Call for papers -- the babylm challenge: Sample-efficient pretraining on a developmentally plausible corpus, 2023. URL https://arxiv.org/abs/2301.11796

    2023 arXiv

  38. [46]

    Breaking down emerging segments in the ai content licensing landscape

    Wiley . Breaking down emerging segments in the ai content licensing landscape. https://www.wiley.com/en-us/network/trending-stories/breaking-down-emerging-segments-in-the-ai-content-licensing-landscape, 2024

    2024

  39. [47]

    Articulating value from data

    World Economic Forum . Articulating value from data. https://www3.weforum.org/docs/WEF_Articulating_Value_from_Data_2021.pdf, 2021

    2021

  40. [48]

    Wildchat: 1m chatgpt interaction logs in the wild, 2024

    Zhao, W., Ren, X., Hessel, J., Cardie, C., Choi, Y., and Deng, Y. Wildchat: 1m chatgpt interaction logs in the wild, 2024. URL https://arxiv.org/abs/2405.01470

    2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.