Interpretable Text Classification Applied to the Detection of LLM-generated Creative Writing

Andrea McGlinchey; Mattias Wahde; Minerva Suvanto; Peter J Barclay

REVIEW 3 major objections 4 minor 2 references

Interpretable Text Classification Applied to the Detection of LLM-generated Creative Writing

T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Simple unigram statistics can tell LLM-rewritten fiction from human prose at 98 percent accuracy, while human readers score at chance.

desk verdict Useful interpretability analysis of LLM-rewrite detection, but the ungrouped split and the 'structurally different' prompt mean the headline accuracy numbers need re-analysis before I'd trust them. read the letter →

arxiv 2601.07368 v1 pith:HXFIQRSN submitted 2026-01-12 cs.CL

Minerva Suvanto , Andrea McGlinchey , Mattias Wahde , Peter J Barclay This is my paper · ORCID

classification cs.CL

keywords LLM-generatedtextdetectioninterpretableclassificationcreativefictionunigramentropysynonymvarietybag-of-wordsfeatureshumanversusAIdetective

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that human readers are almost unable to tell an LLM's rewrite of a classic detective novel from the original prose, while even very simple machine-learning classifiers do it nearly perfectly. Using short excerpts from twelve human-written novels and GPT-rewritten versions, the authors report test accuracies between 0.93 and 0.98 for a range of classifiers, with an interpretable linear classifier at the top; 119 human evaluators, by contrast, score 297/595, essentially a coin flip. The central explanatory claim is that the language model consistently uses a wider range of synonyms and slightly modernised phrasing, which raises the unigram entropy of the generated texts (9.164 vs 8.875 bits) and gives classifiers a stable statistical fingerprint. The paper argues that because this signal is spread across many small word-level cues, it is not something a human can consciously perceive and is hard to remove by post-editing. If correct, this would give editors and publishers a practical, explainable tool for flagging disguised AI-authored fiction.

What carries the argument

The load-bearing object is a linear classifier trained on bag-of-words unigram features, whose weights are directly inspectable; the paper uses those weights, plus a parallel analysis of grammatical features, to trace classification to specific word-replacement patterns. The companion identity is the 0.29-bit unigram-entropy gap between the two classes, which quantifies the model's greater synonym variety and serves as a single-number summary of the detection signal.

What would settle it

Inspect the released DET12 data: if any test-set sample's paired counterpart appears in the training set, the core accuracy claim collapses to memorization. Alternatively, on a held-out corpus of contemporary fiction and rewrites, compute the per-class unigram entropy gap; if classifiers still reach 0.98 while the gap disappears, synonym variety is not the operative cue.

Watch

Extended reading notes

Core claim

The paper claims that on the DET12 data set — 8,068 short excerpts from a dozen human-written detective novels plus 8,068 GPT-rewritten versions — a linear classifier with bag-of-words unigram features reaches 98.14% test accuracy, while 119 human evaluators score 49.9%, essentially chance. The decisive property is that the rewrites use a wider vocabulary: the unigram entropy of the generated class is 9.164 bits versus 8.875 for the human class, about 22% more word-choice variation. Manual annotation of the highest-weighted features attributes this to rephrasing (especially replacement of simple verbs, adjectives, and nouns by less common synonyms), plus secondary effects of temporal drift,

Load-bearing premise

The paper does not state that each original and its rewrite were always placed in the same data split, so if a test-set rewrite has its source human text in training, the 'unseen' test set is not unseen and 98% accuracy could reflect content memory rather than style discrimination.

Editorial extensions

If this is right

An editorial workflow could flag likely AI-rewritten fiction automatically, because the signal survives on short 100-word samples.
The detection does not hinge on artefacts of punctuation, casing, or layout; normalising these did not change accuracy, so the signal lies in word choice.
Post-processing that simply edits the most salient words would need to remove roughly 80 to 100 heavy-weight features to drop accuracy from 0.98 to around 0.88, and even then the classifier remains far above human performance.
Adding n-grams, TF-IDF, or grammatical features does little beyond unigram counts, suggesting the information is already present in single-word frequencies.
Preliminary evidence from the authors indicates that generating from an outline rather than rewriting still leaves the text detectable, though this is explicitly presented as needing further work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A clean way to test the synonym-variety mechanism is to instruct the model to avoid synonyms and to match the source text's vocabulary distribution; if the entropy gap narrows and classifier accuracy falls, the mechanism is confirmed rather than merely correlated.
The temporal-drift and Americanism cues are probably specific to this mid-century British corpus; contemporary human fiction would not provide the same contrast, so the 0.98 accuracy should not be expected to transfer across eras or genres without recalibration.
The human near-chance result is a comment on task setting as much as on human ability: evaluating side-by-side pairs under time pressure may imperfectly reflect real-world detection, so the human–machine gap deserves a more ecologically valid follow-up.
Because the split procedure is not fully specified, the first thing a reader should check is whether any original-rewrite pair straddles train and test; this single detail determines whether the paper's accuracy numbers are about style or about content memory.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a decent, honest paper that explains why simple unigram classifiers can tell human-written 1920s detective fiction apart from GPT-4.1 rewrites of the same passages. The genuinely new parts are the manual feature annotation (E1–E5) and the entropy analysis, and the 119-participant human evaluation is a nice addition. The authors also ship the tokenized data on Zenodo, which is good practice. The claim that humans are at chance while simple ML beats them is solid for this specific setup.

But two issues keep me from fully trusting the 0.98 number. First, the train/validation/test split is described as a random split of 'labelled original and rewritten text samples' without stating that each original–rewrite pair was kept together. If any pairs straddle train and test, the test set is not genuinely unseen — the classifier could pick up content cues from the original in training and apply them to the rewrite in test. That is load-bearing, and the paper needs a re-analysis with pair-grouped splits (or at least whole-book held-out sets). Second, the prompt explicitly says 'appears structurally different to the original,' which invites synonym substitution and other variation. So the main explanation (E1, larger synonym variety) is at least partly a consequence of the generation protocol, not a general property of LLM-written fiction. To their credit, the authors acknowledge this in §6, but they only cite unpublished preliminary experiments — no data — and then claim robustness to evasion based on Figure 5, which is essentially a feature-ablation study. That robustness claim is not supported.

The feature annotation is careful and the categories are plausible; the entropy result (0.29 bits higher for rewrites) is a clean summary of the same effect. But the paper overreaches when it generalizes beyond the 'rewrite to be different' paradigm. Also, the high-accuracy detection was already established in McGlinchey and Barclay (2025) by two of the same authors, so the classification numbers themselves are not new.

Bottom line: this deserves a serious referee, mainly to force a re-analysis with pair-grouped splits and an honest discussion of prompt-dependence. The interpretability framework is worth engaging with. I'd bring it to a reading group if you work on AI-text detection; otherwise it's a maybe. I'd cite it for the feature categories, with a caveat, once the split issue is resolved.

Referee Report

3 major / 4 minor

Summary. The paper constructs DET12, a corpus of 8,068 short excerpts from twelve classic detective novels by Agatha Christie and Dorothy L. Sayers, together with 8,068 GPT-4.1 rewrites generated under a prompt that asks for the same key details but an "appears structurally different" text. Using only unigram features, five machine-learning classifiers are trained on a 70/15/15 split and achieve 0.93--0.98 test accuracy, with the best linear classifier at 0.9814. In the same task, 119 human evaluators perform at chance (297/595 correct). The authors then use the linear classifier's weights and a manual annotation of 190 frequent, class-selective unigrams to argue that the main cue is the LLM's greater synonym variety, supplemented by temporal drift, Americanisms, foreign-language removal, and loss of colloquialisms. A unigram-entropy analysis reports higher entropy for the LLM class (H_1=9.164 vs H_0=8.875). The paper concludes that detection is robust and not easy to circumvent.

Significance. If the test set is genuinely unseen in the sense that no original--rewrite pair straddles the train/test boundary, the 0.98 accuracy of a simple linear classifier on short fiction excerpts is a practically valuable result, and the interpretability analysis is a useful template for understanding classifier decisions. The study has notable strengths: the data and tokenized datasets are released on Zenodo; the artefact checks for punctuation, casing, and letter distributions are thoughtful; and the human evaluation is substantially larger than previous work (595 judgments). The visualisation in Figure 3 is a nice illustration of an inherently interpretable model. However, two load-bearing issues prevent me from accepting the paper in its current form: the train/test split is not stated to keep original--rewrite pairs together, and the central explanation (synonym variety) is substantially built into the generation prompt, so the generalisation and robustness claims go beyond the evidence. These are fixable with additional experiments or by reframing the claims.

major comments (3)

[Section 3.1 / Table 3] The split description says only that "labelled original and rewritten text samples" were split into 70/15/15 sets; it does not state that each original--rewrite pair was kept in the same split. With 8,068 pairs and a sample-level random split, a rewritten test sample will very likely have its source human text in the training set (probability ≈0.7). Because the LLM was prompted to preserve key details, a test rewrite and its training-set original share proper nouns and many content words; a unigram classifier can exploit that overlap. The reported 0.93--0.98 accuracies could therefore reflect content memory rather than style discrimination. Please state the grouping explicitly, and, if the split was not pair-grouped, re-run the experiments with a grouped split (e.g., GroupShuffleSplit) and report the resulting accuracies.
[Section 3.1 / §5.1 / §5.2 / §6] The generation prompt explicitly instructs GPT-4.1 to "Create a new text excerpt which contains the same key details but appears structurally different to the original." This instruction directly invites synonym substitution and greater lexical variation. The paper's main explanation category E1 (rephrasing, Table 4) and the higher unigram entropy reported in §5.2 are therefore measures of behaviour elicited by this specific prompt, not established properties of LLM-generated fiction in general. The authors acknowledge this in §6 ("the methodology may have introduced greater variation ... by requesting rewording") and cite "preliminary experiments" without data, but then still conclude that the classification "appears robust, and therefore not easy to circumvent by malicious actors." This robustness claim is unsupported: a different generation protocol (e.g., neutral continuation or same
[Section 5, Figure 5 / §6] The robustness argument uses an ablation in which learned feature weights are set to zero one by one and accuracy is recomputed on the test set without retraining the classifier. This shows how much the fitted model relies on individual features, but it does not simulate a malicious actor who rewrites text to evade detection. It also does not account for correlated features that a retrained model could exploit after a feature is removed. The conclusion in §6 that "it is difficult to make automatically generated detective fiction harder to detect ... whether by removal or by rewriting" therefore overinterprets the experiment. I recommend either reframing Figure 5 as a descriptive sensitivity analysis or adding experiments that actually modify generated text (e.g., synonym substitution or prompting with different instructions) and measure evasion.

minor comments (4)

[Section 5.2] The line "As 29.164/28.875 ≈1.22" appears to be missing superscripts; it should read 2^9.164 / 2^8.875 ≈ 1.22.
[Section 5, Figure 5 / §6] There is an inconsistency between the text of Figure 5, which describes removing the first 100 features, and the conclusion in §6, which refers to "the 80 features which most help the classifiers." Please align these numbers.
[Table 2] Typo in the caption: "Cspecifices" should be "C specifies".
[Section 5.1] The manual annotation was performed by hand without reporting inter-annotator agreement or multiple annotators; the authors note this limitation, but it would be helpful to state the number of annotators explicitly.

Circularity Check

1 steps flagged · score 4.0 of 10

Synonym-variety 'explanation' restates the prompt's rewording instruction; held-out accuracy is genuine but the explanatory/generalization claim is prompt-bound.

self definitional [Abstract; §3.1 (generation prompt); §5.1 (E1/Table 4); §5.2 (entropy); §6 (limitation)]
"Abstract: 'the LLM tends to use a larger variety of synonyms...' §3.1 prompt: 'Create a new text excerpt which contains the same key details but appears structurally different to the original.' §6: 'the methodology may have introduced greater variation in the LLM-generated texts by requesting rewording; such an approach may invite use of synonyms, for example.'"

The paper's central explanatory finding—that LLM text has greater synonym variety (E1) and higher unigram entropy (H=9.164 vs 8.875)—is the direct operationalization of the prompt's instruction to make the output 'structurally different' and to reword the original. The 'discovery' is therefore a restatement of the data-generation input, not an independent property of LLM fiction. The authors concede this confound in §6 and support generalization only via unreported 'preliminary experiments'. This makes the robustness claim ('not easy to circumvent') unsupported outside the rewrite-to-be-different paradigm. However, the 0.93–0.98 test accuracies are measured on a held-out test set, so the core detection result is not itself circular.

full rationale

Most of the pipeline is externally validated: classifiers are evaluated on a held-out test set, human performance is freshly measured, and the unigram classifiers are standard. Self-citations (McGlinchey & Barclay 2025; Wahde et al. 2024) supply a baseline and a classifier, but are not load-bearing for the new measurement. The main circularity risk is in the explanation, not the detection: Class 1 was generated with an explicit instruction to make text 'structurally different', which invites synonym substitution; the paper then 'identifies' synonym variety as the main cue and entropy as corroboration. §6 admits the prompt may have introduced the variation, and the robustness extension relies on Fig. 5's weight-removal exercise and on unreported preliminary experiments. These are generalization/correctness limitations rather than a circular derivation of the test accuracy. The train/test split's failure to state whether original–rewrite pairs were kept together is a data-hygiene risk, not a circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or linguistic entities are postulated. The analysis relies on standard bag-of-words features and entropy; the main load-bearing choices are the rewrite prompt, the arbitrary feature-selection thresholds, and the assumption that rewritten excerpts represent LLM fiction. These choices, not fitted parameters, carry most of the explanatory weight.

free parameters (3)

Manual annotation feature-selection thresholds = fmax ≥ 100, R_i ≥ 0.75
Chosen thresholds in §5.1 to select 190 features for annotation; affect which explanation categories are observed, but not the trained classifier.
Ablation weight threshold = |w| ≥ 0.1
Acknowledged in Fig. 5 description as 'somewhat arbitrary'; used to compare frequent vs high-weight feature removal and underpins the robustness discussion.
LLM temperature = 0.7
Temperature setting for GPT-4.1 rewrites in §3.1; influences vocabulary diversity and therefore the magnitude of the synonym-variety effect.

assumptions (4)

domain assumption Class 1 (LLM-rewritten excerpts) is representative of LLM-generated creative fiction more broadly
The paper frames the task as detecting LLM-generated text, but Class 1 is produced by rewriting human excerpts with a fixed prompt (§3.1), not by generation from an outline or continuation.
ad hoc to paper The instruction to make the rewrite 'appear structurally different' does not itself cause the synonym diversity attributed to the LLM
This prompt instruction is in §3.1; the paper's main explanation (E1) is the larger variety of synonyms, which may be a response to that instruction rather than an intrinsic LLM property.
domain assumption Unigram frequencies are sufficient to capture the human/AI distinction
The paper asserts in §4.2 that n-grams, TF-IDF and Biber features add little; this is only checked on DET12 (one genre, one LLM).
standard math Shannon entropy over token frequencies measures meaningful vocabulary variation
Standard formula from Shannon (1948) used in §5.2; no issue with the math itself, but the '22% greater variation' interpretation rests on exponentiating a small entropy difference.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Interpretable Text Classification Applied to the Detection of LLM-generated Creative Writing." pith.science (2026). https://pith.science/paper/HXFIQRSN

@misc{pith2026260107368,
  author       = {Pith},
  title        = {Pith review of: Interpretable Text Classification Applied to the Detection of LLM-generated Creative Writing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HXFIQRSN}},
  note         = {Machine review of arXiv:2601.07368}
}

read the original abstract

We consider the problem of distinguishing human-written creative fiction (excerpts from novels) from similar text generated by an LLM. Our results show that, while human observers perform poorly (near chance levels) on this binary classification task, a variety of machine-learning models achieve accuracy in the range 0.93 - 0.98 over a previously unseen test set, even using only short samples and single-token (unigram) features. We therefore employ an inherently interpretable (linear) classifier (with a test accuracy of 0.98), in order to elucidate the underlying reasons for this high accuracy. In our analysis, we identify specific unigram features indicative of LLM-generated text, one of the most important being that the LLM tends to use a larger variety of synonyms, thereby skewing the probability distributions in a manner that is easy to detect for a machine learning classifier, yet very difficult for a human observer. Four additional explanation categories were also identified, namely, temporal drift, Americanisms, foreign language usage, and colloquialisms. As identification of the AI-generated text depends on a constellation of such features, the classification appears robust, and therefore not easy to circumvent by malicious actors intent on misrepresenting AI-generated text as human work.

Figures

Figures reproduced from arXiv: 2601.07368 by the authors.

**Figure 2.** Analysis of linguistic features on our DET12 training data set. The left panel shows the 10 features that had the [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Visualisation of the classification result using the interpretable classifier, where one can inspect the precise impact [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Drop in test set accuracy when features are re [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: Shannon entropy over increasing sample size. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 1 linked inside Pith

[1]

Bentz, C., Alikaniotis, D., Cysouw, M., and Ferrer-i-Cancho, R. (2017). The entropy of words—learnability and expressivity across more than 1000 languages.Entropy, 19(6):275. Biber, D. (1991).Variation across speech and writing. Cambridge university press. Cabezas-Clavijo, ´A., Magad ´an-D´ıaz, M., Rivas-Garc ´ıa, J. I., and Sidorenko-Bautista, P. (2024)....

arXiv 2017
[30]

Verma, V ., Fleisig, E., Tomlin, N., and Klein, D. (2023). Ghostbuster: Detecting text ghostwritten by large lan- guage models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies (Volume 1: Long Papers), pages 1702–1717. Wahde, M., Della Vedova, M. L., Virgolin...

2023

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

Bentz, C., Alikaniotis, D., Cysouw, M., and Ferrer-i-Cancho, R. (2017). The entropy of words—learnability and expressivity across more than 1000 languages.Entropy, 19(6):275. Biber, D. (1991).Variation across speech and writing. Cambridge university press. Cabezas-Clavijo, ´A., Magad ´an-D´ıaz, M., Rivas-Garc ´ıa, J. I., and Sidorenko-Bautista, P. (2024)....

arXiv 2017

[2] [30]

Verma, V ., Fleisig, E., Tomlin, N., and Klein, D. (2023). Ghostbuster: Detecting text ghostwritten by large lan- guage models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies (Volume 1: Long Papers), pages 1702–1717. Wahde, M., Della Vedova, M. L., Virgolin...

2023