Pith. sign in

REVIEW 4 major objections 5 minor 32 references

Measuring Human Involvement in AI-Generated Text: A Case Study on Academic Writing

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A continuous human-involvement score, defined by BERTScore recall between prompt and generated text, can be predicted from the generated text alone by a dual-head RoBERTa model.

desk verdict A useful continuous task formulation and a solid model, but the BERTScore-based label is only weakly validated by a small, outlier-filtered human study, so the headline MSE should be read as predicting BERTScore rather than true human involvement. read the letter →

arxiv 2506.03501 v1 pith:PRPEXJSY submitted 2025-06-04 cs.CL cs.AI

Yuchen Guo , Zhicheng Dou , Huy H. Nguyen , Ching-Chun Chang , Saku Sugawara , Isao Echizen This is my paper
classification cs.CLcs.AI
keywords Human-AIcollaborationLLMdetectoracademiccheatingBERTScorecontinuoushuman-involvementmetricRoBERTaregressiontokenclassificationparticipationdetectionobfuscation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Binary detectors answer the wrong question: most real academic writing is a mix of human and machine contributions, so asking 'is this AI or human?' is unhelpful. The paper proposes replacing that binary with the degree of human involvement, defined as how much of the prompt's content appears in the generated abstract as measured by BERTScore recall. It then builds a RoBERTa-based dual-head model that predicts this score from the generated text alone and simultaneously marks which words came from the person. On a new continuous dataset built from real abstracts, the model reaches a token-classification F1 of 0.9423 and a regression mean squared error of 0.004, while existing binary detectors fail to track involvement levels.

What carries the argument

Two pieces carry the argument. First, BERTScore recall: a token-level semantic similarity between the prompt (reference) and the generated text (candidate), max-min normalized to [0,1], which the paper defines as the human-involvement label. Second, a dual-head RoBERTa encoder: a regression head outputs the normalized involvement score from the generated text alone, while a token-classification head labels each token as human-contributed or not, and the two heads are trained together so that word-level attribution improves the score estimate.

What would settle it

A direct test: log the actual human contributions in a set of collaborative abstracts (e.g., track the writer's drafts and edits), compute BERTScore recall between the true human-supplied text and the final abstract, and measure the rank correlation against the logged contribution. If the correlation is much lower than the reported 0.52–0.64 on realistic collaborations where the prompt is not a verbatim subset of the final text, the metric and the regressor trained on it would not generalize.

Watch

Extended reading notes

Core claim

On the paper's own terms: human involvement in AI-generated academic text is a continuous quantity, not a category, and the recall component of BERTScore computed between the user's prompt and the generated text is a valid proxy for it. The authors construct the CAS-CS dataset by feeding an LLM prompts made of randomly selected sentences from real human abstracts, then normalize BERTScore recall to [0,1] as the regression label and mark overlapping tokens as the token-classification label. Their dual-head RoBERTa model, trained jointly on both labels, predicts involvement from the generated text alone; the token head also identifies human-contributed words. The paper reports that on this continuous dataset all four tested binary detectors hover near chance accuracy under different binarization thresholds, while the proposed regressor achieves MSE 0.004 and the token head an F1 of 0.9423, with human evaluation giving a Spearman correlation of 0.52 (0.64 after outlier removal) for the metric itself.

Load-bearing premise

The entire pipeline assumes BERTScore recall between the prompt and the generated text captures how much a human actually contributed; the only evidence is a 55-text human evaluation with Spearman 0.52 (0.64 after outlier removal), so if that proxy fails in real collaborative writing, the regression target and the trained model are misaligned with true human involvement.

Editorial extensions

If this is right

  • Instructors could set graded thresholds, flagging only texts whose predicted involvement falls below a chosen cutoff, instead of relying on a single AI/human label.
  • Because the model needs only the generated text at inference time, it can score submissions without knowing the prompts students used.
  • Binary detectors will keep failing on collaborative text, since the same output can sit at any point along the involvement spectrum.
  • Joint training of the regression and token-attribution heads improves both, implying that explaining human contributions is not separate from estimating them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the metric is validated on real collaborative writing, the same dual-head design could be extended to other collaborative artifacts such as reports or code, whenever a prompt-to-output similarity proxy can be defined.
  • The moderate human correlation (0.52 raw, 0.64 denoised) suggests the label is noisy; a direct validation against logged human edits, rather than prompts drawn from the same abstract, would be the stronger test.
  • The token-attribution head could be turned into a formative tool that highlights which sentences a student drafted, potentially shifting the conversation from policing to teaching.
  • Cross-model results (worse on Claude and Falcon than on GPT models) imply the detector's calibration depends on the generating model, so deployment would need per-model recalibration or uncertainty reporting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper addresses the limitation of binary AI-text detectors in scenarios where humans collaborate with LLMs. It proposes a continuous 'human involvement' score defined as the min-max normalized BERTScore recall between the prompt and the generated text (Eq. 1), constructs a synthetic dataset (CAS-CS) by randomly sampling sentences from real human-written abstracts as prompts and generating LLM outputs, and trains a RoBERTa-based dual-head model that simultaneously predicts this regression score and performs token-level classification of human-contributed words. The authors report strong performance on their dataset (regression MSE 0.004, token F1 0.9423) and some generalization across LLMs and prompt templates. The central claim is that the proposed metric and regressor can estimate the degree of human involvement in academic AI-assisted writing from text alone.

Significance. If the proposed metric were valid, the work would be a meaningful step beyond binary detection by offering a continuous, interpretable measure of human involvement in collaborative writing. The paper provides a new dataset, an open-source model, and a dual-head architecture that yields both a score and token-level explanations, and the authors honestly include limitation statements. However, the significance is conditional on the validity of BERTScore recall as a proxy for human involvement, which rests on a small and methodologically limited human evaluation. The extent to which the paper advances the field therefore depends on whether the metric can be validated in more realistic collaboration scenarios.

major comments (4)
  1. [Sec. IV.A.2 and Sec. V.B] The load-bearing assumption that BERTScore recall between prompt and generated text measures human involvement is validated only by a human evaluation of 55 texts with 10 testers, reporting Spearman correlations of 0.52 raw and 0.64 after post hoc removal of outliers (residuals > 2 SD). No inter-rater agreement is reported, and the evaluation protocol asked testers to mark human-contributed words, which is essentially the same operationalization as the token-level labels in Eq. (2). This does not independently validate the regression target against a holistic notion of human involvement. The paper even acknowledges in the Conclusion that 'the current measurement of human involvement relies on a model-based score.' Given that all training labels and evaluation metrics derive from this proxy, the claimed MSE of 0.004 mainly demonstrates prediction of BERTScore, not of true human involvement. This needs either a substantially stronger human validation (larger sample, pre-registered analysis, multiple annotators, and an independent definition of human involvement) or a careful reframing of the contribution as 'measuring prompt-to-text lexical/semantic overlap' rather than human involvement.
  2. [Sec. IV.B] The CAS-CS dataset construction selects Z entire sentences from a real human-written abstract and uses them as the prompt. This creates a strong lexical and semantic overlap between the prompt and the generated text, which BERTScore recall is specifically designed to capture. The regression label (Eq. 1) and the token labels (Eq. 2, tokens appearing in both prompt and generated text) are both derived from this same overlap, so the model is trained to learn a function of that overlap. The experimental results therefore show that the model can predict the overlap signal from text alone, but they do not establish that this signal corresponds to human involvement in realistic settings where a human's contribution may be paraphrased, restructured, or expressed as high-level ideas. The paper would be strengthened by a discussion of this circularity and by an evaluation on naturally occurring human-AI collaborative texts where the level of human involvement is known or independently assessed.
  3. [Sec. V.B] The removal of outlier data points with residuals greater than two standard deviations is performed post hoc and is not justified by any principled criterion. Reporting the correlation after denoising (0.64) as the primary evidence of credibility, while also reporting the raw correlation (0.52), does not resolve concerns about selective reporting. The raw correlation is moderate, and with only 55 texts and no inter-annotator reliability measure, the evidence for the metric's validity is weak. A sensitivity analysis (e.g., reporting correlations for several outlier thresholds, or using robust correlation measures) and a pre-registered analysis plan would be needed to support the conclusion that the regression label 'is significantly positively correlated with human judgment.'
  4. [Sec. V.D, Table II] The regression model converted to a binary classifier achieves a perfect accuracy of 1.00 on the PAS-CS dataset, which is suspicious and likely reflects a distributional artifact rather than a genuine ability to distinguish human-authored from AI-generated text. Since the regression model was trained to predict BERTScore-like overlap, its binary decisions at a threshold may be exploiting the fact that PAS-CS human texts have high self-overlap while AI texts have low prompt-overlap. A more detailed analysis of the predictions (e.g., score distributions, error analysis, out-of-distribution robustness) is needed to support the claim that the method outperforms existing binary detectors on polarized benchmarks. Without such analysis, the perfect accuracy in Table II may overstate the practical utility of the approach.
minor comments (5)
  1. [References] Reference [7] is truncated as 'Survey: 30' and appears incomplete; the full bibliographic details should be provided.
  2. [Fig. 6] The vertical axis label contains a typo: 'BERTSscore' should be 'BERTScore'.
  3. [Table V] The header formatting of Table V is confusing: it lists 'Training set Testing set GPT-4 Claude-3' but the rows do not clearly indicate which columns correspond to training and testing sets. Please clarify the table structure.
  4. [Sec. IV.C.1] The sentence 'The logits of each token are then sent to the regression head' is ambiguous, as the regression head operates on a pooled representation, not individual token logits. Reword for clarity.
  5. [Sec. IV.A.1] The paper does not specify which underlying BERT/RoBERTa model is used for the 'en-sci' BERTScore backbone; please cite the model and note its source.

Circularity Check

2 steps flagged · score 5.0 of 10

Human-involvement labels are defined as BERTScore recall, so the regressor's success largely demonstrates prediction of that same metric; the only external human evaluation uses a near-duplicate prompt-overlap protocol.

  1. self definitional [Section III.B (Problem Formulation), Eq. (1)]
    "First, we compute the recall score between P and G using BERTScore to reflect how much P is involved in G. This recall score is then normalized using min-max normalization to obtain regression label y reg, which represents the human involvement in the generated text. yreg = BERTScore(P, G)−min(BERTScore) max(BERTScore)−min(BERTScore) (1)"

    Human involvement is defined as the normalized BERTScore recall between the prompt and the generated text. The dual-head model is trained on this same y_reg and evaluated against it, so the reported MSE of 0.004 certifies how well the model predicts the BERTScore-based label from G alone, not how well that label corresponds to an independently measured human-involvement quantity. The definition makes the target and the metric equivalent by construction; an external validation step is needed to break the circle.

  2. other [Section V.B (Regression Label Credibility), Fig. 6]
    "The horizontal axis shows the tester-assessed human involvement values, which were obtained by dividing the number of words identified by the tester as human contributed by the total number of words in the text."

    The human evaluation quantifies human involvement as the fraction of words a tester marks as human-contributed. In the dataset, the token label y_cls is defined by whether a token in the generated text appears in the prompt (Eq. 2), and the prompt is built by copying whole sentences from a real abstract (Sec. IV.B). Testers marking human-contributed words are therefore performing essentially the same prompt-overlap judgment that BERTScore recall operationalizes. The reported Spearman correlation of 0.52 (0.64 after outlier removal) thus mainly shows consistency between two versions of the same overlap construct, not independent confirmation that the metric tracks human involvement in a broader sense.

full rationale

The central circularity is that the paper's regression target is constructed as BERTScore recall (Eq. 1), and the model's headline performance is measured against that same target. The claimed capability to 'measure human involvement' therefore reduces, at the label level, to the ability to predict the chosen metric. The only external anchor is the human evaluation in Sec. V.B, but that evaluation uses a token-overlap protocol that mirrors the dataset's own definition of human-contributed tokens, so it does not provide a fully independent validity check; the conclusion concedes that the measurement is model-based and that survey-based validation is future work. There is a nontrivial element: the regressor must infer the label from the generated text alone, without access to the prompt, and the paper shows some generalization across models and templates. However, because the ground-truth construct is defined by the metric, the central claim remains partially circular. No load-bearing self-citation issue was found; the cited prior work by the authors is used for prompt-template design and background, not to justify the metric's validity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The approach rests on defining human involvement as BERTScore recall between prompt and text, and on a synthetic dataset where prompts are random sentence subsets of real abstracts. No new physical entities are introduced; the continuous involvement score is a new measurement construct with limited independent validation.

free parameters (4)
  • BERTScore normalization bounds (min and max in Eq. 1) = Not reported
    The regression labels are normalized using max-min mapping over the dataset; the exact min and max are data-dependent and affect all labels.
  • Token CE loss weights (Eq. 8) = 1.0 and 1.2
    Hand-chosen hyperparameters to account for token label imbalance.
  • Human-evaluation outlier removal threshold = 2 standard deviations
    The paper removes outliers with residuals greater than two standard deviations to improve the Spearman correlation from 0.52 to 0.64; post hoc exclusion criterion.
  • BERTScore backbone 'en-sci' = Model name only
    Choice of embedding model for BERTScore affects all labels; the specific model is not clearly identified.
assumptions (3)
  • domain assumption BERTScore recall between prompt and generated text is a valid proxy for human involvement
    Stated in Section IV.A.2 and validated only weakly in Section V.B (Spearman 0.52 raw, 0.64 after outlier removal).
  • ad hoc to paper Randomly sampling Z sentences from the original abstract and using them as the prompt simulates real human-AI collaborative academic writing
    Dataset construction in Section IV.B; this is a specific modeling choice that may not generalize to real collaboration patterns.
  • domain assumption Human involvement can be estimated from the generated text alone without access to the prompt
    Task formulation in Section III.B; presupposes that human involvement leaves a detectable trace in the text.
invented entities (1)
  • Continuous human involvement score (normalized BERTScore recall) independent evidence
    purpose: To quantify the degree of human contribution to AI-generated text in an academic context.
    Defined in Section IV.A.2 and used as the regression target. Independent support is limited to a 55-text human evaluation with moderate correlation; the score is a new measurement construct rather than a physical entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Measuring Human Involvement in AI-Generated Text: A Case Study on Academic Writing." pith.science (2026). https://pith.science/paper/PRPEXJSY

@misc{pith2026250603501,
  author       = {Pith},
  title        = {Pith review of: Measuring Human Involvement in AI-Generated Text: A Case Study on Academic Writing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PRPEXJSY}},
  note         = {Machine review of arXiv:2506.03501}
}
read the original abstract

Content creation has dramatically progressed with the rapid advancement of large language models like ChatGPT and Claude. While this progress has greatly enhanced various aspects of life and work, it has also negatively affected certain areas of society. A recent survey revealed that nearly 30% of college students use generative AI to help write academic papers and reports. Most countermeasures treat the detection of AI-generated text as a binary classification task and thus lack robustness. This approach overlooks human involvement in the generation of content even though human-machine collaboration is becoming mainstream. Besides generating entire texts, people may use machines to complete or revise texts. Such human involvement varies case by case, which makes binary classification a less than satisfactory approach. We refer to this situation as participation detection obfuscation. We propose using BERTScore as a metric to measure human involvement in the generation process and a multi-task RoBERTa-based regressor trained on a token classification task to address this problem. To evaluate the effectiveness of this approach, we simulated academic-based scenarios and created a continuous dataset reflecting various levels of human involvement. All of the existing detectors we examined failed to detect the level of human involvement on this dataset. Our method, however, succeeded (F1 score of 0.9423 and a regressor mean squared error of 0.004). Moreover, it demonstrated some generalizability across generative models. Our code is available at https://github.com/gyc-nii/CAS-CS-and-dual-head-detector

Figures

Figures reproduced from arXiv: 2506.03501 by the authors.

Figure 1
Figure 1. Text can be entirely authored by a person, generated by an LLM using [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example text reflecting low and high human involvement. The same [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Method for generating continuous dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (7 more)
Figure 3
Figure 3. Figure 3: Relationships between metrics. To verify the credibility of BERTScore-based label, we also performed human judgment evaluation, as described in Sec V-B. B. Continuous CAS-CS Dataset, Polarized Dataset PAS-CS Due to text length limitations, we utilized abstracts from ac…
Figure 5
Figure 5. Figure 5: Structure of dual-head model. credibility of the detector results and reduce users’ concerns about reliability, we designed a human evaluation experiment to ascertain whether our definition of human involvement in Sec. IV-A2 is consistent with human judgment. We descri…
Figure 6
Figure 6. Figure 6: Results of human evaluation. Spearman rank correlation coefficients [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Abstract entirely written by ChatGPT. Write an abstract for an academic paper about: “ Effective planning in model based reinforcement learning (MBRL) and model predictive control (MPC) relies on the accuracy of the learned dynamics model. In many instances of MBRL and…
Figure 8
Figure 8. Figure 8: Abstract written on basis of human-informed prompt. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Abstract written by a person [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Result of regression with ChatGPT (upper left), GPT4 (upper [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 18 canonical work pages

  1. [1]

    ChatGPT: Applications, op- portunities, and threats,

    A. Bahrini, M. Khamoshifar, H. Abbasimehr, R. J. Riggs, M. Esmaeili, R. M. Majdabadkohne, and M. Pasehvar, “ChatGPT: Applications, op- portunities, and threats,” in2023 Systems and Information Engineering Design Symposium (SIEDS). IEEE, 2023, pp. 274–279

    work page 2023

  2. [2]

    GPT-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “GPT-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Security and privacy challenges of large language models: A survey,

    B. C. Das, M. H. Amini, and Y . Wu, “Security and privacy challenges of large language models: A survey,”arXiv preprint arXiv:2402.00888, 2024

    arXiv 2024

  4. [4]

    Abstracts written by chatgpt fool scientists,

    H. Else, “Abstracts written by chatgpt fool scientists,” Nature, January 2023, accessed: 2024-12-14. [Online]. Available: https://www.nature.com/articles/d41586-023-00056-7

    work page 2023

  5. [5]

    University policies on ai writing tools — overview & list,

    J. Caulfield, “University policies on ai writing tools — overview & list,” https://www.scribbr.com/ai-tools/chatgpt-university-policies/, 5 2024

    work page 2024

  6. [6]

    Fake academic papers may have been created with ai; japanese researcher falsely cited as author on overseas ‘predatory journal’ website,

    Y . Nagasawa and T. K. . Y . Shimbun, “Fake academic papers may have been created with ai; japanese researcher falsely cited as author on overseas ‘predatory journal’ website,” The Japan News, November 2024, accessed: 2024-12-14. [Online]. Available: https://japannews.yomiuri.co.jp/society/general-news/20241121-223696/

    work page arXiv 2024

  7. [7]

    Kyaw, “Survey: 30

    A. Kyaw, “Survey: 30

    work page

  8. [8]

    Balancing ai and academic integrity: what are the positions of academic publishers and universities?

    B. H. Gulumbe, S. M. Audu, and A. M. Hashim, “Balancing ai and academic integrity: what are the positions of academic publishers and universities?”AI & SOCIETY, pp. 1–10, 2024

    work page 2024

Show all 32 references
  1. [9]

    A comprehensive AI policy education framework for university teaching and learning,

    C. K. Y . Chan, “A comprehensive AI policy education framework for university teaching and learning,”International journal of educational technology in higher education, vol. 20, no. 1, p. 38, 2023

    2023

  2. [10]

    CHEAT: A large-scale dataset for detecting ChatGPT-written abstracts,

    P. Yu, J. Chen, X. Feng, and Z. Xia, “CHEAT: A large-scale dataset for detecting ChatGPT-written abstracts,”arXiv preprint arXiv:2304.12008, 2023

    2023 arXiv

  3. [11]

    Enhancing robustness of LLM-synthetic text detectors for academic writing: A comprehensive analysis,

    Z. Dou, Y . Guo, C.-C. Chang, H. H. Nguyen, and I. Echizen, “Enhancing robustness of LLM-synthetic text detectors for academic writing: A comprehensive analysis,” inInternational Conference on Advanced Information Networking and Applications. Springer, 2024, pp. 266– 277

    2024

  4. [12]

    BERTScore: Evaluating text generation with BERT,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “BERTScore: Evaluating text generation with BERT,” inInternational Conference on Learning Representations, 2020

    2020

  5. [13]

    Unsupervised and distributional detection of machine-generated text,

    M. Gall ´e, J. Rozen, G. Kruszewski, and H. Elsahar, “Unsupervised and distributional detection of machine-generated text,”arXiv preprint arXiv:2111.02878, 2021

    2021 arXiv

  6. [14]

    Detection of chatgpt fake science with the xfakesci learning algorithm,

    A. A. Hamed and X. Wu, “Detection of chatgpt fake science with the xfakesci learning algorithm,”Scientific Reports, vol. 14, no. 1, p. 16231, 2024

    2024

  7. [15]

    Ghostbuster: Detecting text ghostwritten by large language models,

    V . Verma, E. Fleisig, N. Tomlin, and D. Klein, “Ghostbuster: Detecting text ghostwritten by large language models,” inProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long...

    2024

  8. [16]

    Raidar: generative ai detection via rewriting,

    C. Mao, C. V ondrick, H. Wang, and J. Yang, “Raidar: generative ai detection via rewriting,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  9. [17]

    Classification of human-and ai-generated texts: Investigating features for chatgpt,

    L. Mindner, T. Schlippe, and K. Schaaff, “Classification of human-and ai-generated texts: Investigating features for chatgpt,” inInternational Conference on Artificial Intelligence in Education Technology. Springer, 2023, pp. 152–170

    2023

  10. [18]

    Origin tracing and detecting of llms,

    L. Li, P. Wang, K. Ren, T. Sun, and X. Qiu, “Origin tracing and detecting of llms,”arXiv preprint arXiv:2304.14072, 2023

    2023 arXiv

  11. [19]

    How close is ChatGPT to human experts? comparison corpus, evaluation, and detection,

    B. Guo, X. Zhang, Z. Wang, M. Jiang, J. Nie, Y . Ding, J. Yue, and Y . Wu, “How close is ChatGPT to human experts? comparison corpus, evaluation, and detection,”arXiv preprint arXiv:2301.07597, 2023

    2023 arXiv

  12. [20]

    ArguGPT: evaluating, understanding and identifying argumenta- tive essays generated by GPT models,

    Y . Liu, Z. Zhang, W. Zhang, S. Yue, X. Zhao, X. Cheng, Y . Zhang, and H. Hu, “ArguGPT: evaluating, understanding and identifying argumenta- tive essays generated by GPT models,”arXiv preprint arXiv:2304.07666, 2023

    2023 arXiv

  13. [21]

    GPT-Sentinel: Distinguishing human and ChatGPT generated content,

    Y . Chen, H. Kang, V . Zhai, L. Li, R. Singh, and B. Raj, “GPT-Sentinel: Distinguishing human and ChatGPT generated content,”arXiv preprint arXiv:2305.07969, 2023

    2023 arXiv

  14. [22]

    Implementing BERT and fine- tuned RobertA to detect AI generated news by ChatGPT,

    Z. Wang, J. Cheng, C. Cui, and C. Yu, “Implementing BERT and fine- tuned RobertA to detect AI generated news by ChatGPT,”arXiv preprint arXiv:2306.07401, 2023

    2023 arXiv

  15. [23]

    ConSERT: A contrastive framework for self-supervised sentence representation trans- fer,

    Y . Yan, R. Li, S. Wang, F. Zhang, W. Wu, and W. Xu, “ConSERT: A contrastive framework for self-supervised sentence representation trans- fer,” inProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference...

    2021

  16. [24]

    SimCSE: Simple contrastive learning of sentence embeddings,

    T. Gao, X. Yao, and D. Chen, “SimCSE: Simple contrastive learning of sentence embeddings,” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 6894– 6910

    2021

  17. [25]

    Dual contrastive learning: Text classification via label-aware data augmentation,

    Q. Chen, R. Zhang, Y . Zheng, and Y . Mao, “Dual contrastive learning: Text classification via label-aware data augmentation,”arXiv preprint arXiv:2201.08702, 2022

    2022 arXiv

  18. [26]

    On the detectability of ChatGPT content: Benchmarking, methodology, and evaluation through the lens of academic writing,

    Z. Liu, Z. Yao, F. Li, and B. Luo, “On the detectability of ChatGPT content: Benchmarking, methodology, and evaluation through the lens of academic writing,”arXiv e-prints, pp. arXiv–2306, 2023

    2023

  19. [27]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,”arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  20. [28]

    BERT: Pre- training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understanding,” inProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologi...

    2019

  21. [29]

    Distinguishing fact from fiction: A benchmark dataset for identifying machine-generated scientific papers in the LLM era

    E. Mosca, M. H. I. Abdalla, P. Basso, M. Musumeci, and G. Groh, “Distinguishing fact from fiction: A benchmark dataset for identifying machine-generated scientific papers in the LLM era.” inProceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 20...

    2023

  22. [30]

    Release strategies and the social impacts of language models,

    I. Solaiman, M. Brundage, J. Clark, A. Askell, A. Herbert-V oss, J. Wu, A. Radford, G. Krueger, J. W. Kim, S. Krepset al., “Release strategies and the social impacts of language models,”arXiv preprint arXiv:1908.09203, 2019

    1908 arXiv

  23. [31]

    Machine- generated text-detection by fine-tuning of language models,

    Nicolai Thorer Sivesind and Andreas Bentzen Winje, “Machine- generated text-detection by fine-tuning of language models,” 2023. [Online]. Available: https://huggingface.co/andreas122001/roberta- academic-detector

    2023

  24. [32]

    Detectgpt: Zero-shot machine-generated text detection using probability curvature,

    E. Mitchell, Y . Lee, A. Khazatsky, C. D. Manning, and C. Finn, “Detectgpt: Zero-shot machine-generated text detection using probability curvature,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 24 950–24 962. VII. SUPPLEMENTARYMATERIAL A. More Example Applic...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.