Pith. sign in

REVIEW 3 cited by

Is ChatGPT Transforming Academics' Writing Style?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.08627 v2 pith:BCAJI7BD submitted 2024-04-12 cs.CL cs.AIcs.DLcs.LG

Mingmeng Geng , Roberto Trotta This is my paper · ORCID
classification cs.CLcs.AIcs.DLcs.LG
keywords abstractsanalysischatgptstylewritingacademicsarxivfrequency
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Based on one million arXiv papers submitted from May 2018 to January 2024, we assess the textual density of ChatGPT's writing style in their abstracts through a statistical analysis of word frequency changes. Our model is calibrated and validated on a mixture of real abstracts and ChatGPT-modified abstracts (simulated data) after a careful noise analysis. The words used for estimation are not fixed but adaptive, including those with decreasing frequency. We find that large language models (LLMs), represented by ChatGPT, are having an increasing impact on arXiv abstracts, especially in the field of computer science, where the fraction of LLM-style abstracts is estimated to be approximately 35%, if we take the responses of GPT-3.5 to one simple prompt, "revise the following sentences", as a baseline. We conclude with an analysis of both positive and negative aspects of the penetration of LLMs into academics' writing style.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Human-LLM Coevolution: Evidence from Academic Writing

    cs.CL 2025-02 conditional novelty 6.0 of 10

    After ChatGPT-style words were publicly flagged in early 2024, their frequency in arXiv abstracts dropped, while other common LLM-favored words kept rising, suggesting authors are adapting their writing to avoid detection.

  2. Who Gets Seen in the Age of AI? Adoption Patterns of Large Language Models in Scholarly Writing and Citation Outcomes

    cs.CY 2025-09 reject novelty 5.0 of 10

    Analyzing 98,000 Scopus computer science papers, the paper finds a global rise in AI-like writing after ChatGPT and reports regional differences in citation returns, but the key differential-gain result is statistical...

  3. Humans Coexist, So Must Embodied Artificial Agents

    cs.LG 2025-02 conditional novelty 5.0 of 10

    Coexistence, defined as sustained meaningful and reciprocal interaction among an agent, humans, and environment, is presented as a necessary design goal for embodied AI.

Pith tools