Pith. sign in

REVIEW 3 major objections 6 minor 89 references

Stylometry recognizes human and LLM-generated texts in short samples

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Stylometric fingerprints of words, grammar, and punctuation separate human from LLM-written ten-sentence encyclopedic texts, reaching 0.98 accuracy in the Wikipedia-versus-GPT-4 case and 0.87 Matthews correlation across seven classes.

desk verdict Solid, well-controlled stylometric benchmark for short Wikipedia-style LLM detection, with a real but fixable human-baseline contamination issue. read the letter →

arxiv 2507.00838 v2 pith:ES7FUWPL submitted 2025-07-01 cs.CL cs.AIcs.LG

Karol Przystalski , Jan K. Argasiński , Iwona Grabska-Gradzińska , Jeremi K. Ochab This is my paper
classification cs.CLcs.AIcs.LG
keywords stylometrymachine-generatedtextdetectionlargelanguagemodelsauthorshipattributionLightGBMSHAPexplanationsWikipediabenchmarkclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that stylometry—the quantitative study of linguistic style—can identify machine-generated text even in samples as short as ten sentences, at least for the well-defined text type of encyclopedic Wikipedia introductions. The authors built a benchmark of 2,424 human term summaries from a 2022 Wikipedia snapshot and comparable ten-sentence descriptions produced by GPT-3.5/4, LLaMa 2/3, Orca, and Falcon, then classified them with tree-based models on two kinds of interpretable features: 195 handcrafted StyloMetrix features and thousands of frequency-based n-gram features. Cross-validated results reached 0.87 Matthews correlation coefficient in seven-class attribution and binary accuracy up to 0.98 for Wikipedia versus GPT-4, with SHAP explanations showing markers such as overused words, punctuation patterns, and more standardized grammar in LLM text. The authors deliberately limit the claim: their own cross-domain test on the AuTexTification benchmark scored macro-F1 0.54 with frequency features, so the method's strength is tied to the specific genre, not to all text. If the claim holds, it gives a cheap, explainable detection route that needs no watermark and no access to the generating model.

What carries the argument

The load-bearing machinery is a feature-engineering pipeline that converts short texts into normalized frequency vectors over linguistically annotated units: lemmas (uni- to trigrams), part-of-speech tags, dependency bigrams, and morphological annotations, alongside StyloMetrix's 195 engineered stylometric features. These vectors feed LightGBM, a gradient-boosted tree classifier, under 10-fold group cross-validation that keeps a given term's topic out of both training and test folds. Shapley Additive Explanations (SHAP), a per-prediction attribution method, then links each classification decision back to individual features, exposing that whitespace tokens, punctuation, proper nouns, dates, and part-of-speech bigrams carry much of the discriminative signal. This combination is what makes the approach explainable and cheap: no watermark, no access to model logits, and no neural classifier is needed.

What would settle it

Audit the 2,424 term summaries by tracing edit histories or applying an independent detector, and count any machine-written or machine-edited passages; even a small contamination rate would require recomputing the reported 0.98 accuracy. Cleaner still: rerun the Wikipedia-versus-GPT-4 binary experiment on a corpus verifiably written and edited before 2018; if accuracy falls well below 0.98, the original separation leaned on the unverified baseline.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that LLM prose carries a stable, measurable stylistic signature that a gradient-boosted tree classifier can exploit. Using ten-sentence Wikipedia-style samples, frequency-based features—normalized counts of lemmas, part-of-speech uni- to trigrams, dependency bigrams, and morphological tags—achieve near-perfect binary separation between Wikipedia summaries and each of the six LLMs (up to 0.98 for Wikipedia versus GPT-4, with several pairs at 1.00), and a 0.87 Matthews correlation coefficient separating all seven sources at once. SHAP attributions show the signal is distributed: Wikipedia samples are richer in proper nouns and numerals, while LLM outputs show standardized part-of-speech sequences, distinctive punctuation and whitespace behaviour, and overuse of particular words such as 'significant', 'notable', and 'despite'. The detector also generalizes to unseen LLMs and survives single-step paraphrase attacks with recall at or above the unparaphrased validation level in nearly all cases, the main exceptions being GPT-4 and LLaMa 3 when first encountered. The paper is explicit that these results demonstrate machine-versus-human detection for a defined text type, not a universal detector.

Load-bearing premise

The 2022 Wikipedia snapshot used as the human baseline is assumed to contain no text written or edited by a large language model; if any of the 2,424 summaries were machine-assisted, the reported human-versus-machine separation is inflated.

Editorial extensions

If this is right

  • Ten-sentence encyclopedic text can be separated human-from-machine with near-perfect binary accuracy and attributed to a specific LLM at 0.87 MCC using only interpretable frequency features and boosted trees.
  • One-step paraphrase attacks with DIPPER and Parrot do not break detection: recall stays at 98.8% or higher for every LLM, so surface rewording is not a sufficient evasion for this text type.
  • Held-out LLMs are detected with high recall—88.2% validation recall for the hardest case, GPT-4, and above 94% for LLaMa 3—indicating some generalization across model families rather than memorization of the six training generators.
  • Cross-domain transfer remains a bottleneck: on AuTexTification the frequency features reach macro-F1 0.54 and StyloMetrix 0.48, below baselines, so accuracy is tied to the encyclopedic genre.
  • SHAP explanations turn detection into auditable claims—overused words like 'significant' and 'notable', lower punctuation variability, standardized grammatical sequences—that a human reviewer can check directly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because SHAP exposes the exact words and grammar patterns that separate LLM output from Wikipedia, the same tables could be read as a recipe for making future models less detectable: diversify part-of-speech sequences and suppress overused words. The paper does not test this, so it is an inference.
  • The near-total survival of paraphrase attacks suggests the discriminative signal sits in generation-level habits rather than in surface wording alone; a direct test would be recursive or iterative paraphrasing, which the authors explicitly leave to future work.
  • The closed-set seven-class experiment would become an open-set problem as soon as a new, unseen model appears; a practical detector would need an 'unknown model' output or a separate novelty-detection step, extending the paper's own acknowledgment that unseen models change the multiclass task.
  • Since the authors note Wikipedia samples are multi-author and multi-editor, the 'human' class may actually be a mixture of many styles; retraining the same pipeline on single-author human corpora would probably tighten the human class and could change which features dominate the explanations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper reports a controlled study of stylometric classification of short (10-sentence) Wikipedia-style texts into human-written and LLM-generated classes. The dataset combines human Wikipedia summaries from a 2022 snapshot and a Wikipedia-API collection, LLM outputs from six models (GPT-3.5, GPT-4, LLaMa 2/3, Orca, Falcon), and outputs of four summarizers. Using StyloMetrix and a frequency-based CLARIN-PL pipeline with decision trees and LightGBM under group cross-validation, the authors report a multiclass MCC up to 0.87 and binary accuracies up to 1.00, with the Wikipedia-vs-GPT-4 case at 0.98. They also present robustness tests on unseen models, paraphrased texts, and the AuTexTification benchmark, plus SHAP-based explanations of discriminating features such as grammatical standardization and overused words. The central claim is that stylometry can distinguish machine- from human-generated text for this well-defined text type.

Significance. If the central claim holds, the paper provides a useful, interpretable, and inexpensive complement to neural MGT detectors: it shows that boosted trees on linguistic frequency features can give high accuracy on short encyclopedic texts, with explainable feature attributions. The study merits credit for a controlled dataset, group cross-validation to prevent topic leakage, explicit tests on unseen LLMs, and honest reporting of a cross-domain benchmark where the frequency pipeline underperforms a logistic-regression baseline. The released code and data (OSF and GitHub) support reproducibility. The main significance is therefore not a new detector architecture but a careful demonstration of the strengths and limits of explainable stylometry in a narrow domain.

major comments (3)
  1. [Section 3.1.1] The human baseline is not verifiably pre-LLM. Only the HuggingFace snapshot 20220301.simple is dated, while the 1,048 Wikipedia-API samples have no reported retrieval date, revision IDs, or as-of parameter. Since these samples constitute roughly 43% of the human class, the binary and multiclass results (e.g., Table 5's wiki accuracy of 0.98-1.00) cannot be interpreted as human-versus-machine unless this subset is shown to predate GPT-3.5 or to be free of LLM edits. The concern is directly checkable from the released code and data; please report the retrieval metadata and either stratify the analysis by source or verify the absence of AI-assisted content.
  2. [Abstract and Tables 4-5] The abstract's statement of 'accuracy between .79 and 1. in binary classification' is not supported by the reported experiments. Table 4 includes decision-tree accuracies as low as 0.53 (prompt #2, Orca vs. LLaMa 2), and the StyloMetrix panel of Table 5 includes 0.77 (LLaMa 2 vs. Orca). If the intended scope is only the LGBM frequency-based pipeline, the abstract should say so explicitly; otherwise the stated range misrepresents the full set of binary results.
  3. [Section 4.4.2 and Table 8] The paraphrase robustness test is presented as showing no collapse, but the paper does not address the possibility that the high recall on Dipper- and Parrot-paraphrased texts is driven by stylistic fingerprints of the paraphrase models themselves rather than by persistence of the original LLM's signals. The training set contains only unparaphrased texts, so a positive test could mean that the classifier recognizes 'was paraphrased by Dipper/Parrot' rather than 'is a paraphrased LLM output'. Please add a control with human texts passed through the same paraphrasers, or at least discuss this confound explicitly.
minor comments (6)
  1. [Section 3.1.1] The text first states that the final dataset consists of 2,439 terms and then says that after removing outlying texts it consists of 2,424 terms; please clarify which number is used for the reported experiments.
  2. [Section 3 and elsewhere] There are repeated typos, including 'Metholodogy' in section headings and 'dynamicaly' in the Introduction; a careful proofread is needed.
  3. [Table 4] The caption says each entry corresponds to a task with class 1 and 2 as column and row labels, but the table appears to show only one triangle of a symmetric matrix; please clarify the layout.
  4. [Section 3.3.2] The feature description says lemmas and POS tags exclude named entities, but the list later refers to 'excluding named entities and punctuation' without specifying which feature groups exclude punctuation; please make the definitions precise.
  5. [References] Several references contain incomplete author lists with '...' placeholders (e.g., Hu et al. 2023, Liu et al. 2023a, Mikros et al. 2023); these should be completed before publication.
  6. [Section 5.2 and Table 11] The text refers to 'noa (2025)' but the table caption names GPTZero; please correct the in-text citation or clarify the entity being referenced.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is supported by fresh-data experiments and an external benchmark, not by a self-referential derivation.

full rationale

The paper is an empirical classification study rather than a formal derivation, so there is no equation chain in which a predicted quantity is defined as its own input. The only author-tied component is the CLARIN-PL stylometric pipeline, which is cited from the authors' prior work ('As in previous works (Argasiński et al., 2024; Ochab & Walkowiak, 2024), we decided to use (i) tree models... (ii) feature engineering approach'). This reuse is not load-bearing: the features are described explicitly in the paper, they come from public NLP libraries (spaCy, StyloMetrix), and the pipeline is code-reproduced and evaluated on a newly assembled dataset (2424 Wikipedia-derived samples) as well as on the external AuTexTification benchmark ('cross-domain detection is tested on an existing benchmark by Sarvazyan et al. (2023a)'). The reported accuracies and MCC values are measured through grouped 10-fold cross-validation, so the central claim that stylometry distinguishes LLM-generated from human-written short encyclopedic texts does not reduce to a fitted parameter or a self-citation. The noted concern about the human class mixing a 2022 snapshot with live Wikipedia-API data is a data-purity limitation, not a circularity: it does not make the result true by construction. No circular step is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central result rests on standard machine-learning tools and a carefully constructed dataset. The main unverified assumption is the purity of the human baseline; no new entities are introduced, and the only hand-set values are generation and classifier hyperparameters.

free parameters (5)
  • Temperature for LLM generation = 0.7
    Section 3.1.3: set to 0.7 for all six models as default/common value; affects lexical variation and therefore separability of generated texts.
  • LGBM hyperparameters = max_depth=5, num_leaves=5, learning_rate=0.5, bagging_freq=3, bagging_fraction=0.8
    Section 3.4: chosen without hyperparameter optimization; the reported performance depends on these choices.
  • Inclusion thresholds for human texts = >=1100 alphanumeric chars, >=10 sentences, max 18 sentences
    Section 3.1.1: these cutoffs determine the dataset and hence the benchmark difficulty.
  • Summarizer parameters = max chars 1000 for T5/BART, length penalty 2.0, beams 4; Sumy 10 sentences
    Section 3.1.2: parameters tuned to produce about 10-sentence summaries; affect summarizer stylometric statistics.
  • Feature set for frequency pipeline = ~3000 features (lemma, POS, dependency, morphological n-grams); 196 StyloMetrix features
    Section 3.3: feature composition is a modeling choice; the abstract's .79-1.00 range applies to the frequency-based feature set only.
assumptions (4)
  • domain assumption The 20220301.simple Wikipedia snapshot predates widespread LLM contamination and is treated as purely human text.
    Section 3.1.1: the authors select this snapshot to avoid MGT, but do not verify absence of AI-assisted edits; contamination would bias the human baseline.
  • domain assumption Two fixed prompts and temperature 0.7 produce representative samples of each LLM's style.
    Section 3.1.3: broader prompt variation or different sampling parameters could change the stylistic distributions.
  • domain assumption Normalized frequencies of lemmas, POS tags, dependency bigrams, and morphological annotations capture authorship style rather than topic.
    Section 3.3.2: group cross-validation prevents topic overlap between train and test, but topic-correlated features still exist within groups.
  • standard math Standard cross-validation and SHAP provide unbiased estimates and explanations.
    Section 3.4: standard tools; feature selection was performed before CV on the same dataset, so estimates may be slightly optimistic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stylometry recognizes human and LLM-generated texts in short samples." pith.science (2026). https://pith.science/paper/ES7FUWPL

@misc{pith2026250700838,
  author       = {Pith},
  title        = {Pith review of: Stylometry recognizes human and LLM-generated texts in short samples},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ES7FUWPL}},
  note         = {Machine review of arXiv:2507.00838}
}
read the original abstract

The paper explores stylometry as a method to distinguish between texts created by Large Language Models (LLMs) and humans, addressing issues of model attribution, intellectual property, and ethical AI use. Stylometry has been used extensively to characterise the style and attribute authorship of texts. By applying it to LLM-generated texts, we identify their emergent writing patterns. The paper involves creating a benchmark dataset based on Wikipedia, with (a) human-written term summaries, (b) texts generated purely by LLMs (GPT-3.5/4, LLaMa 2/3, Orca, and Falcon), (c) processed through multiple text summarisation methods (T5, BART, Gensim, and Sumy), and (d) rephrasing methods (Dipper, T5). The 10-sentence long texts were classified by tree-based models (decision trees and LightGBM) using human-designed (StyloMetrix) and n-gram-based (our own pipeline) stylometric features that encode lexical, grammatical, syntactic, and punctuation patterns. The cross-validated results reached a performance of up to .87 Matthews correlation coefficient in the multiclass scenario with 7 classes, and accuracy between .79 and 1. in binary classification, with the particular example of Wikipedia and GPT-4 reaching up to .98 accuracy on a balanced dataset. Shapley Additive Explanations pinpointed features characteristic of the encyclopaedic text type, individual overused words, as well as a greater grammatical standardisation of LLMs with respect to human-written texts. These results show -- crucially, in the context of the increasingly sophisticated LLMs -- that it is possible to distinguish machine- from human-generated texts at least for a well-defined text type.

Figures

Figures reproduced from arXiv: 2507.00838 by the authors.

Figure 1
Figure 1. Explanations for binary classification between the Wikipedia and GPT-4 using (a) StyloMetrix and (b) frequency-based features. Only the first 10 most [PITH_FULL_IMAGE:figures/full_fig_p012_1.png] view at source ↗
Figure 2
Figure 2. General explanations for multiclass classification. The first 10 most [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Local explanations of 10 most important StyloMetrix (a-c) and frequency features (d-f) in multiclass classification for text samples describing the term [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Text sample from LLaMa 2 with highlighted important frequency [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 4
Figure 4. Figure 4: Text sample from the the Wikipedia with highlighted text spans cor [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

89 extracted references · 48 canonical work pages

  1. [1]

    title GPTZero

    ( year 2025 ). title GPTZero . https://gptzero.me/ note Accessed : 2025-05-20

    2025

  2. [2]

    author Al-Shaibani, M. S. , & author Ahmed, M. ( year 2025 ). title The arabic ai fingerprint: Stylometric analysis and detection of large language models text . journal arXiv preprint arXiv:2505.23276 \/ ,

    work page Pith review arXiv 2025

  3. [3]

    , author Alobeidli, H

    author Almazrouei, E. , author Alobeidli, H. , author Alshamsi, A. , author Cappelli, A. , author Cojocaru, R. , author Debbah, M. , author Étienne Goffinet , author Hesslow, D. , author Launay, J. , author Malartic, Q. , author Mazzotta, D. , author Noune, B. , author Pannier, B. , & author Penedo, G. ( year 2023 ). title The falcon series of open langua...

    arXiv 2023

  4. [4]

    GPT4All: An Ecosystem of Open Source Compressed Language Models

    author Anand, Y. , author Nussbaum, Z. , author Treat, A. , author Miller, A. , author Guo, R. , author Schmidt, B. , author Community, G. , author Duderstadt, B. , & author Mulyar, A. ( year 2023 ). title Gpt4all: An ecosystem of open source compressed language models . https://arxiv.org/abs/2311.04931. arXiv:2311.04931 http://arxiv.org/abs/2311.04931

    work page Pith review arXiv 2023

  5. [5]

    ( year 2018 )

    author Argamon, S. ( year 2018 ). title Computational forensic authorship analysis: Promises and pitfalls . journal Language and Law/Linguagem e Direito \/ , volume 5 \/ , pages 7--37

    2018

  6. [6]

    author Argasiński, J. K. , author Grabska-Gradzińska, I. , author Przystalski, K. , author Ochab, J. K. , & author Walkowiak, T. ( year 2024 ). title Stylometric analysis of large language model-generated commentaries in the context of medical neuroscience . journal International Conference … \/ , (pp. pages 281--295 ). https://link.springer.com/chapter/1...

    doi:10.1007/978-3-031-63775-9_20 2024

  7. [7]

    , author Wang, Y

    author Bevendorff, J. , author Wang, Y. , author Karlgren, J. , author Wiegmann, M. , author Tsivgun, A. , author Su, J. , author Xie, Z. , author Abassy, M. , author Mansurov, J. , author Xing, R. , author Ta, M. N. , author Elozeiri, K. A. , author Gu, T. , author Tomar, R. V. , author Geng, J. , author Artemova, E. , author Shelmanov, A. , author Habas...

    2025

  8. [8]

    Voight - Kampff

    author Bevendorff, J. , author Wiegmann, M. , author Karlgren, J. , author Dürlich, L. , author Gogoulou, E. , author Talman, A. , author Stamatatos, E. , author Potthast, M. , & author Stein, B. ( year 2024 ). title Overview of the " Voight - Kampff " Generative AI Authorship Verification Task at PAN and ELOQUENT 2024 . In editor G. Faggioli , editor N. ...

    2024

Show all 89 references
  1. [9]

    author Bhat, M. M. , & author Parthasarathy, S. ( year 2020 ). title How Effectively Can Machines Defend Against Machine - Generated Fake News ? An Empirical Study . In editor A. Rogers , editor J. Sedoc , & editor A. Rumshisky (Eds.), booktitle Proceedings of the First Worksh...

    2020 doi

  2. [10]

    , author Kumarage, T

    author Bhattacharjee, A. , author Kumarage, T. , author Moraffah, R. , & author Liu, H. ( year 2023 ). title ConDA : Contrastive Domain Adaptation for AI -generated Text Detection . In editor J. C. Park , editor Y. Arase , editor B. Hu , editor W. Lu , editor D. Wijaya , edito...

    2023

  3. [11]

    , author Roten, C.-A

    author Bozza, S. , author Roten, C.-A. , author Jover, A. , author Cammarota, V. , author Pousaz, L. , & author Taroni, F. ( year 2023 ). title A model-independent redundancy measure for human versus chatgpt authorship discrimination using a bayesian probabilistic approach . j...

    2023

  4. [12]

    , author Afroz, S

    author Brennan, M. , author Afroz, S. , & author Greenstadt, R. ( year 2012 ). title Adversarial stylometry: Circumventing authorship recognition to preserve privacy and anonymity . journal ACM Transactions on Information and System Security (TISSEC) \/ , volume 15 \/ , pages 1--22

    2012

  5. [13]

    , author Eggert, S

    author Brooks, C. , author Eggert, S. , & author Peskoff, D. ( year 2024 ). title The Rise of AI - Generated Content in Wikipedia . In editor L. Lucie-Aimée , editor A. Fan , editor T. Gwadabe , editor I. Johnson , editor F. Petroni , & editor D. van Strien (Eds.), booktitle P...

    2024

  6. [14]

    , author Kang, H

    author Chen, Y. , author Kang, H. , author Zhai, V. , author Li, L. , author Singh, R. , & author Raj, B. ( year 2023 ). title Token Prediction as Implicit Classification to Identify LLM - Generated Text . In editor H. Bouamor , editor J. Pino , & editor K. Bali (Eds.), bookti...

    2023

  7. [15]

    , author Japkowicz, N

    author Crothers, E. , author Japkowicz, N. , author Viktor, H. , & author Branco, P. ( year 2022 ). title Adversarial Robustness of Neural - Statistical Features in Detection of Generative Transformers . In booktitle 2022 International Joint Conference on Neural Networks ( IJC...

    2022

  8. [16]

    , author Japkowicz, N

    author Crothers, E. , author Japkowicz, N. , & author Viktor, H. L. ( year 2023 ). title Machine- Generated Text : A Comprehensive Survey of Threat Models and Detection Methods . journal IEEE Access \/ , volume 11 \/ , pages 70977--71002 . https://doi.org/10.1109/ACCESS.2023.3...

    2023

  9. [17]

    ( year 2021 )

    author Damodaran, P. ( year 2021 ). title Parrot: Paraphrase generation for NLU . note Version Number: v1.0

    2021

  10. [18]

    , author Baglivo, F

    author De Angelis, L. , author Baglivo, F. , author Arzilli, G. , author Privitera, G. P. , author Ferragina, P. , author Tozzi, A. E. , & author Rizzo, C. ( year 2023 ). title Chatgpt and the rise of large language models: the new ai-driven infodemic threat in public health ....

    2023

  11. [19]

    , author Poelman, W

    author Dhaini, M. , author Poelman, W. , & author Erdogan, E. ( year 2023 ). title Detecting chatgpt: A survey of the state of detecting chatgpt-generated text . journal arXiv preprint arXiv:2309.07689 \/ ,

    2023 arXiv

  12. [20]

    author Ding, S. H. , author Fung, B. C. , author Iqbal, F. , & author Cheung, W. K. ( year 2017 ). title Learning stylometric representations for authorship analysis . journal IEEE transactions on cybernetics \/ , volume 49 \/ , pages 107--121

    2017

  13. [21]

    ( year 2011 )

    author Eder, M. ( year 2011 ). title Style- Markers in Authorship Attribution : A Cross - Language Study of The Authorial Fingerprint . journal Studies in Polish Linguistics \/ , (pp. pages 101--116 )

    2011

  14. [22]

    , author Kestemont, M

    author Eder, M. , author Kestemont, M. , & author Rybicki, J. ( year 2016 ). title Stylometry with R : A Package for Computational Text Analysis . journal The R Journal \/ , volume 8 \/ , pages 1--15 . :10.32614/RJ-2016-007

    2016 doi

  15. [23]

    , author Proisl, T

    author Evert, S. , author Proisl, T. , author Jannidis, F. , author Reger, I. , author Pielström, S. , author Schöch, C. , & author Vitt, T. ( year 2017 ). title Understanding and explaining Delta measures for authorship attribution . journal Digital Scholarship in the Humanit...

    2017 doi

  16. [24]

    , author Falchi, F

    author Fagni, T. , author Falchi, F. , author Gambini, M. , author Martella, A. , & author Tesconi, M. ( year 2021 ). title TweepFake : About detecting deepfake tweets . journal PLOS ONE \/ , volume 16 \/ , pages e0251415 . https://journals.plos.org/plosone/article?id=10.1371/...

    2021 doi

  17. [25]

    , author Huang, S.-j

    author Geng, C. , author Huang, S.-j. , & author Chen, S. ( year 2020 ). title Recent advances in open set recognition: A survey . journal IEEE transactions on pattern analysis and machine intelligence \/ , volume 43 \/ , pages 3614--3631 . note Publisher: IEEE

    2020

  18. [26]

    , author Zhang, X

    author Guo, B. , author Zhang, X. , author Wang, Z. , author Jiang, M. , author Nie, J. , author Ding, Y. , author Yue, J. , & author Wu, Y. ( year 2023 ). title How Close is ChatGPT to Human Experts ? Comparison Corpus , Evaluation , and Detection . journal CoRR \/ , volume a...

    2023 doi arXiv

  19. [27]

    , author Yang, W

    author Guo, L. , author Yang, W. , author Ma, L. , & author Ruan, J. ( year 2024 a ). title BLGAV : Generative AI Author Verification Model Based on BERT and BiLSTM . In editor G. Faggioli , editor N. Ferro , editor P. Galušč 'a kov 'a , & editor A. G. S. Herrera (Eds.), bookt...

    2024

  20. [28]

    , author Han, Z

    author Guo, M. , author Han, Z. , author Chen, H. , & author Peng, J. ( year 2024 b ). title A Machine - Generated Text Detection Model Based on Text Multi - Feature Fusion . In editor G. Faggioli , editor N. Ferro , editor P. Galusc 'a kov 'a , & editor A. G. S. d. Herrera (E...

    2024

  21. [29]

    , & author Mimno, D

    author Hicke, R. , & author Mimno, D. ( year 2023 ). title T5 meets tybalt: Author attribution in early modern english drama using large language models . journal arXiv preprint arXiv:2310.18454 \/ ,

    2023 arXiv

  22. [30]

    ( year 2025 )

    author HIX A.I. ( year 2025 ). title HIX AI Detector . https://bypass.hix.ai/ai-detector note Accessed : 2025-05-20

    2025

  23. [31]

    , author Ou, W

    author Hu, X. , author Ou, W. , author Acharya, S. , author Ding, S. , author D'Gama, R. , & author ... ( year 2023 ). title Tdrlm: Stylometric learning for authorship verification by topic-debiasing . journal Expert Systems with Applications \/ ,

    2023

  24. [32]

    , author Chen, C

    author Huang, B. , author Chen, C. , & author Shu, K. ( year 2024 a ). title Can large language models identify authorship? journal arXiv preprint arXiv:2403.08213 \/ ,

    2024 arXiv

  25. [33]

    , author Ruan, W

    author Huang, X. , author Ruan, W. , author Huang, W. , author Jin, G. , author Dong, Y. , author Wu, C. , author Bensalem, S. , author Mu, R. , author Qi, Y. , author Zhao, X. et al. ( year 2024 b ). title A survey of safety and trustworthiness of large language models throug...

    2024

  26. [34]

    , author Hu, Z

    author Hung, C.-Y. , author Hu, Z. , author Hu, Y. , & author Lee, R. K.-W. ( year 2023 ). title Who wrote it and why? prompting large-language models for authorship verification . journal arXiv preprint arXiv:2310.08123 \/ ,

    2023 arXiv

  27. [35]

    , author Meng, Q

    author Ke, G. , author Meng, Q. , author Finley, T. , author Wang, T. , author Chen, W. , author Ma, W. , author Ye, Q. , & author Liu, T.-Y. ( year 2017 ). title Lightgbm: A highly efficient gradient boosting decision tree . journal Advances in neural information processing s...

    2017

  28. [36]

    , author Song, Y

    author Krishna, K. , author Song, Y. , author Karpinska, M. , author Wieting, J. , & author Iyyer, M. ( year 2023 ). title Paraphrasing evades detectors of AI -generated text, but retrieval is an effective defense . journal arXiv preprint arXiv:2303.13408 \/ ,

    2023 arXiv

  29. [37]

    , author Garland, J

    author Kumarage, T. , author Garland, J. , author Bhattacharjee, A. , author Trapeznikov, K. , author Ruston, S. W. , & author Liu, H. ( year 2023 ). title Stylometric Detection of AI - Generated Text in Twitter Timelines . journal CoRR \/ , volume abs/2303.03697 \/ . https://...

    2023 doi arXiv

  30. [38]

    , & author Liu, H

    author Kumarage, T. , & author Liu, H. ( year 2023 ). title Neural authorship attribution: Stylometric analysis on large language models . In booktitle 2023 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC) \/ (pp. pages 51--54 )....

    2023

  31. [39]

    , author Liu, Y

    author Lewis, M. , author Liu, Y. , author Goyal, N. , author Ghazvininejad, M. , author Mohamed, A. , author Levy, O. , author Stoyanov, V. , & author Zettlemoyer, L. ( year 2019 ). title Bart: Denoising sequence-to-sequence pre-training for natural language generation, trans...

    2019 arXiv

  32. [40]

    , author Villanova del Moral, A

    author Lhoest, Q. , author Villanova del Moral, A. , author Jernite, Y. , author Thakur, A. , author von Platen, P. , author Patil, S. , author Chaumond, J. , author Drame, M. , author Plu, J. , author Tunstall, L. , author Davison, J. , author S a s ko, M. , author Chhablani,...

    2021 arXiv

  33. [41]

    , author Li, Q

    author Li, Y. , author Li, Q. , author Cui, L. , author Bi, W. , author Wang, Z. , author Wang, L. , author Yang, L. , author Shi, S. , & author Zhang, Y. ( year 2024 a ). title MAGE : Machine -generated Text Detection in the Wild . In editor L.-W. Ku , editor A. Martins , & e...

    2024

  34. [42]

    , author Shi, Y

    author Li, Z. , author Shi, Y. , author Liu, Z. , author Yang, F. , author Payani, A. , author Liu, N. , & author Du, M. ( year 2024 b ). title Quantifying Multilingual Performance of Large Language Models Across Languages . https://arxiv.org/abs/2404.11553. :10.48550/ARXIV.24...

    2024 doi arXiv

  35. [43]

    , author Diab, M

    author Liu, A. , author Diab, M. , & author Fried, D. ( year 2024 a ). title Evaluating Large Language Model Biases in Persona - Steered Generation . http://arxiv.org/abs/2405.20253. :10.48550/arXiv.2405.20253 note arXiv:2405.20253 [cs]

    2024 doi arXiv

  36. [44]

    , author Han, T

    author Liu, Y. , author Han, T. , author Ma, S. , author Zhang, J. , author Yang, Y. , author Tian, J. , & author ... ( year 2023 a ). title Summary of chatgpt/gpt-4 research and perspective towards the future of large language models . journal arXiv preprint arXiv … \/ ,

    2023

  37. [45]

    , author Zhang, Z

    author Liu, Y. , author Zhang, Z. , author Zhang, W. , author Yue, S. , author Zhao, X. , author Cheng, X. , author Zhang, Y. , & author Hu, H. ( year 2023 b ). title ArguGPT : evaluating, understanding and identifying argumentative essays generated by GPT models . journal CoR...

    2023 doi arXiv

  38. [46]

    , author Yao, Z

    author Liu, Z. , author Yao, Z. , author Li, F. , & author Luo, B. ( year 2024 b ). title On the Detectability of ChatGPT Content : Benchmarking , Methodology , and Evaluation through the Lens of Academic Writing . http://arxiv.org/abs/2306.05524. :10.48550/arXiv.2306.05524 no...

    2024 doi arXiv

  39. [47]

    , author Aygüler, F

    author Lorenz, L. , author Aygüler, F. Z. , author Schlatt, F. , & author Mirzakhmedova, N. ( year 2024 ). title BaselineAvengers at PAN 2024: Often - Forgotten Baselines for LLM - Generated Text Detection . In editor G. Faggioli , editor N. Ferro , editor P. Galusc 'a kov 'a ...

    2024

  40. [48]

    , author Liu, S

    author Lu, N. , author Liu, S. , author He, R. , author Ong, Y.-S. , author Wang, Q. , & author Tang, K. ( year 2024 ). title Large Language Models can be Guided to Evade AI -generated Text Detection . journal Transactions on Machine Learning Research \/ , volume 2024 \/ . htt...

    2024

  41. [49]

    author Lund, B. D. , author Wang, T. , author Mannuru, N. R. , author Nie, B. , author Shimray, S. , & author Wang, Z. ( year 2023 ). title Chatgpt and a new academic reality: Artificial intelligence-written research papers and the ethics of the large language models in schola...

    2023

  42. [50]

    author Lundberg, S. M. , author Erion, G. , author Chen, H. , author DeGrave, A. , author Prutkin, J. M. , author Nair, B. , author Katz, R. , author Himmelfarb, J. , author Bansal, N. , & author Lee, S.-I. ( year 2020 ). title From local explanations to global understanding w...

    2020

  43. [51]

    , & editor Kytö, M

    editor Lüdeling, A. , & editor Kytö, M. (Eds.) ( year 2008 ). title Corpus linguistics: an international handbook \/ volume volume 1 of series Handbücher zur Sprach - und Kommunikationswissenschaft \/ . address Berlin New York : publisher Walter de Gruyter

    2008

  44. [52]

    , author Moro, R

    author Macko, D. , author Moro, R. , author Uchendu, A. , author Lucas, J. , author Yamashita, M. , author Pikuliak, M. , author Srba, I. , author Le, T. , author Lee, D. , author Simko, J. , & author Bielikova, M. ( year 2023 ). title MULTITuDE : Large - Scale Multilingual Ma...

    2023

  45. [53]

    , author Koursaris, A

    author Mikros, G. , author Koursaris, A. , author Bilianos, D. , & author ... ( year 2023 ). title Ai-writing detection using an ensemble of transformers and stylometric features. journal IberLEF … \/ ,

    2023

  46. [54]

    , author Martín, A

    author Miralles, P. , author Martín, A. , & author Camacho, D. ( year 2024 ). title Team aida at PAN : Ensembling Normalized Log Probabilities . In editor G. Faggioli , editor N. Ferro , editor P. Galušč 'a kov 'a , & editor A. G. S. Herrera (Eds.), booktitle Working Notes Pap...

    2024

  47. [55]

    , author Honnibal, M

    author Montani, I. , author Honnibal, M. , author Honnibal, M. , author Boyd, A. , author Landeghem, S. V. , & author Peters, H. ( year 2023 ). title explosion/spaCy: v3.7.2: Fixes for APIs and requirements . https://doi.org/10.5281/zenodo.10009823. :10.5281/zenodo.10009823

    2023 doi

  48. [56]

    ( year 1968 )

    author Mosteller, F. ( year 1968 ). title Association and estimation in contingency tables . journal Journal of the American Statistical Association \/ , volume 63 \/ , pages 1--28

    1968

  49. [57]

    , author Mitra, A

    author Mukherjee, S. , author Mitra, A. , author Jawahar, G. , author Agarwal, S. , author Palangi, H. , & author Awadallah, A. ( year 2023 ). title Orca: Progressive learning from complex explanation traces of gpt-4 . journal arXiv preprint arXiv:2306.02707 \/ ,

    2023 arXiv

  50. [58]

    , author Sundararajan, K

    author Neal, T. , author Sundararajan, K. , author Fatima, A. , author Yan, Y. , author Xiang, Y. , & author Woodard, D. ( year 2017 ). title Surveying stylometry techniques and applications . journal ACM Computing Surveys (CSuR) \/ , volume 50 \/ , pages 1--36

    2017

  51. [59]

    author Ochab, J. K. , author Argasiński, J. , author Grabska-Gradzińska, I. , & author Przystalski, K. ( year 2025 a ). title Repository for: Stylometry recognizes human and LLM -generated texts in short samples , . https://osf.io/dfz6k/. :10.17605/OSF.IO/DFZ6K. note Publisher: OSF

    2025 doi

  52. [60]

    author Ochab, J. K. , author Matias, M. , author Boba, T. , & author Walkowiak, T. ( year 2025 b ). title StylOch at PAN: Gradient-boosted trees with frequency-based stylometric features . In editor J. C. de Albornoz , editor J. Gonzalo , editor L. Plaza , editor A. G. S. de H...

    2025

  53. [61]

    author Ochab, J. K. , & author Walkowiak, T. ( year 2024 ). title Implementing interpretable models in stylometric analysis . In booktitle Digital Humanities 2024: Conference Abstracts \/ . address Washington, D.C. : publisher George Mason University (GMU)

    2024

  54. [62]

    , author Stetsenko, D

    author Okulska, I. , author Stetsenko, D. , author Kołos, A. , author Karlińska, A. , author Głąbińska, K. , & author Nowakowski, A. ( year 2023 ). title Stylometrix: An open-source multilingual tool for representing stylometric vectors . journal arXiv preprint arXiv:2309.12810 \/ ,

    2023 arXiv

  55. [63]

    title OpenAI API Reference Documentation : Create chat completion

    author OpenAI ( year 2025 ). title OpenAI API Reference Documentation : Create chat completion . https://platform.openai.com/docs/api-reference/chat/create note (last accessed on 2025-06-30)

    2025

  56. [64]

    , author Rao, D

    author Patel, A. , author Rao, D. , author Kothary, A. , author McKeown, K. , & author Callison-Burch, C. ( year 2023 ). title Learning interpretable style embeddings via prompting llms . journal arXiv preprint arXiv:2305.12696 \/ ,

    2023 arXiv

  57. [65]

    , author Varoquaux, G

    author Pedregosa, F. , author Varoquaux, G. , author Gramfort, A. , author Michel, V. , author Thirion, B. , author Grisel, O. , author Blondel, M. , author Prettenhofer, P. , author Weiss, R. , author Dubourg, V. , author Vanderplas, J. , author Passos, A. , author Cournapeau...

    2011

  58. [66]

    , author Argasiński, J

    author Przystalski, K. , author Argasiński, J. K. , author Lipp, N. , & author Pacholczyk, D. ( year 2025 ). title Building Personality - Driven Language Models : How Neurotic is ChatGPT \/ . Synthesis Lectures on Engineering , Science , and Technology . address Cham : publish...

    2025 doi

  59. [67]

    , author Shazeer, N

    author Raffel, C. , author Shazeer, N. , author Roberts, A. , author Lee, K. , author Narang, S. , author Matena, M. , author Zhou, Y. , author Li, W. , & author Liu, P. J. ( year 2020 ). title Exploring the limits of transfer learning with a unified text-to-text transformer ....

    2020

  60. [68]

    , & author Sojka, P

    author R eh u r ek, R. , & author Sojka, P. ( year 2010 ). title Software Framework for Topic Modelling with Large Corpora . In booktitle Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks \/ (pp. pages 45--50 ). address Valletta, Malta : publisher ELRA

    2010

  61. [69]

    author Sadasivan, V. S. , author Kumar, A. , author Balasubramanian, S. , author Wang, W. , & author Feizi, S. ( year 2025 ). title Can AI - Generated Text be Reliably Detected ? Stress Testing AI Text Detectors Under Various Attacks . journal Transactions on Machine Learning ...

    2025

  62. [70]

    author Sarvazyan, A. M. , author Gonz 'a lez, J. A. , author Franco-Salvador, M. , author Rangel, F. , author Chulvi, B. , & author Rosso, P. ( year 2023 a ). title Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine - Generated Text in Multiple D...

    2023

  63. [71]

    author Sarvazyan, A. M. , author Gonz 'a lez, J. A. , author Rosso, P. , & author Franco-Salvador, M. ( year 2023 b ). title Supervised Machine - Generated Text Detectors : Family and Scale Matters . In editor A. Arampatzis , editor E. Kanoulas , editor T. Tsikrika , editor S....

    2023

  64. [72]

    , author Schuster, R

    author Schuster, T. , author Schuster, R. , author Shah, D. J. , & author Barzilay, R. ( year 2020 ). title The Limitations of Stylometry for Detecting Machine - Generated Fake News . journal Computational Linguistics \/ , volume 46 \/ , pages 499--510 . https://doi.org/10.116...

    2020 doi

  65. [73]

    ( year 2009 )

    author Stamatatos, E. ( year 2009 ). title A survey of modern authorship attribution methods . journal Journal of the American Society for information Science and Technology \/ , volume 60 \/ , pages 538--556

    2009

  66. [74]

    , & author Johansson, F

    author Stiff, H. , & author Johansson, F. ( year 2022 ). title Detecting computer-generated disinformation . journal International Journal of Data Science and Analytics \/ , volume 13 \/ , pages 363--383 . https://doi.org/10.1007/s41060-021-00299-5. :10.1007/s41060-021-00299-5

    2022 doi

  67. [75]

    , author Wu, X

    author Su, Z. , author Wu, X. , author Zhou, W. , author Ma, G. , & author Hu, S. ( year 2024 ). title HC3 Plus : A Semantic - Invariant Human ChatGPT Comparison Corpus . http://arxiv.org/abs/2309.02731. :10.48550/arXiv.2309.02731 note arXiv:2309.02731 [cs]

    2024 doi arXiv

  68. [76]

    , & author Lv, Z

    author Sun, J. , & author Lv, Z. ( year 2025 ). title Zero-shot detection of llm-generated text via text reorder . journal Neurocomputing \/ , volume 631 \/ , pages 129829

    2025

  69. [77]

    , author Lavril, T

    author Touvron, H. , author Lavril, T. , author Izacard, G. et al. ( year 2023 ). title Llama: Open and efficient foundation language models . arXiv:2302.13971 http://arxiv.org/abs/2302.13971

    2023 arXiv

  70. [78]

    , author Ma, Z

    author Uchendu, A. , author Ma, Z. , author Le, T. , author Zhang, R. , & author Lee, D. ( year 2021 ). title TURINGBENCH : A Benchmark Environment for Turing Test in the Age of Neural Text Generation . In editor M.-F. Moens , editor X. Huang , editor L. Specia , & editor S. W...

    2021

  71. [79]

    , author Han, K

    author Vaze, S. , author Han, K. , author Vedaldi, A. , & author Zisserman, A. ( year 2022 ). title Open- Set Recognition : A Good Closed - Set Classifier is All You Need . In booktitle International Conference on Learning Representations \/ . https://openreview.net/forum?id=5...

    2022

  72. [80]

    , author Cui, S

    author Wang, S. , author Cui, S. , author Zhang, C. , author Zhang, Z. , author Wang, J. , & author Liu, T. ( year 2024 a ). title Towards Persona - Oriented LLM - Generated Text Detection : Benchmark Dataset and Method . In editor M. Wand , editor K. Malinovsk 'a , editor J. ...

    2024 doi

  73. [81]

    , author Mansurov, J

    author Wang, Y. , author Mansurov, J. , author Ivanov, P. , author Su, J. , author Shelmanov, A. , author Tsvigun, A. , author Whitehouse, C. , author Mohammed Afzal, O. , author Mahmoud, T. , author Sasaki, T. , author Arnold, T. , author Aji, A. F. , author Habash, N. , auth...

    2024

  74. [82]

    , author Yang, S

    author Wu, J. , author Yang, S. , author Zhan, R. , author Yuan, Y. , author Chao, L. S. , & author Wong, D. F. ( year 2025 ). title A Survey on LLM - Generated Text Detection : Necessity , Methods , and Future Directions . journal Computational Linguistics \/ , (pp. pages 1--...

    2025 doi

  75. [83]

    , author Zhang, K

    author Xu, Z. , author Zhang, K. , & author Sheng, V. S. ( year 2024 ). title Freqmark: Frequency-based watermark for sentence-level detection of llm-generated text . journal arXiv preprint arXiv:2410.10876 \/ ,

    2024 arXiv

  76. [84]

    , author Kalita, D

    author Yadagiri, A. , author Kalita, D. , author Ranjan, A. , author Bostan, A. K. , author Toppo, P. , & author Pakray, P. ( year 2024 ). title Team cnlp-nits-pp at PAN : Leveraging BERT for Accurate Authorship Verification : A Novel Approach to Textual Attribution . In edito...

    2024

  77. [85]

    , author Duan, J

    author Yao, Y. , author Duan, J. , author Xu, K. , author Cai, Y. , author Sun, Z. , & author Zhang, Y. ( year 2024 ). title A survey on large language model (llm) security and privacy: The good, the bad, and the ugly . journal High-Confidence Computing \/ , (p. pages 100211 )

    2024

  78. [86]

    , author Chen, J

    author Yu, P. , author Chen, J. , author Feng, X. , & author Xia, Z. ( year 2025 ). title CHEAT : A Large -scale Dataset for Detecting CHatGPT - writtEn AbsTracts . journal IEEE Transactions on Big Data \/ , (pp. pages 1--9 ). https://ieeexplore.ieee.org/abstract/document/1085...

    2025

  79. [87]

    , author Holtzman, A

    author Zellers, R. , author Holtzman, A. , author Rashkin, H. , author Bisk, Y. , author Farhadi, A. , author Roesner, F. , & author Choi, Y. ( year 2019 ). title Defending Against Neural Fake News . In editor H. Wallach , editor H. Larochelle , editor A. Beygelzimer , editor ...

    2019

  80. [88]

    author Zhao, W. X. , author Zhou, K. , author Li, J. , author Tang, T. , author Wang, X. , author Hou, Y. , author Min, Y. , author Zhang, B. , author Zhang, J. , author Dong, Z. et al. ( year 2023 ). title A survey of large language models . journal arXiv preprint arXiv:2303....

    2023 arXiv

  81. [89]

    2015, , 579, A101

    Aladro, R., Martín, S., Riquelme, D., et al. 2015, , 579, A101

    2015

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.