Pith. sign in

REVIEW 4 major objections 5 minor 64 references

LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxification

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM pipeline can replace human annotators for hate speech detoxification data, and the resulting 8,276-pair benchmark trains models that beat existing baselines.

desk verdict The ParaDeHate dataset is a genuine new resource, but the paper's central claim that an LLM replaces human annotators at comparable quality is not supported by the evidence — the same model generates and judges the data. read the letter →

arxiv 2506.01484 v2 pith:YNW5DLYB submitted 2025-06-02 cs.CL

Shuzhou Yuan , Ercong Nie , Lukas Kouba , Ashish Yashwanth Kangen , Helmut Schmid , Hinrich Schütze , Michael Färber This is my paper
classification cs.CL
keywords hatespeechdetoxificationparalleldatasetconstructionLLM-in-the-loopannotationGPT-4o-ministyletransfercontentpreservationtoxicityfilteringbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a single large language model can take over the human side of building parallel detoxification datasets: one model rewrites hateful text, checks that the rewrite keeps the original meaning, and checks that the rewrite is no longer toxic. To support this, the authors replicate the ParaDetox pipeline with GPT-4o-mini in place of crowd workers and report that BART-large, a pre-trained text-to-text transformer, trained on the resulting 19,726 pairs performs about as well as BART-large trained on the human-built original. They then apply the same pipeline to 11,905 hate speech samples from four existing datasets, producing ParaDeHate, a released benchmark of 8,276 hate/non-hate pairs. Fine-tuning BART-large on ParaDeHate improves style accuracy, fluency, and BLEU over existing zero-shot and unsupervised detoxification baselines. The practical point is that human annotation, the main bottleneck for detoxification data, may be replaceable by a cheap, scalable LLM loop.

What carries the argument

The load-bearing mechanism is a three-task self-annotation loop. One model, GPT-4o-mini, is prompted to rewrite a hateful sentence into a neutral version, then to answer yes or no whether the rewrite preserves the original meaning, and finally to answer yes or no whether the rewrite still contains toxicity; only pairs that pass both checks enter the dataset. Two automatic cross-checks monitor the LLM's judgments: sentence-transformer cosine similarity with a threshold of 0.70 for content preservation, with Cohen's kappa 0.55 against the LLM, and a RoBERTa-based toxicity score with a threshold of 0.9 for toxicity, with kappa 0.72. The argument works by showing that models trained on data passed through this loop match models trained on crowd-validated data, so the loop itself is the replacement for human validation.

What would settle it

Audit a random sample of the 8,276 ParaDeHate pairs with independent human raters, asking two questions: does the rewrite preserve the original meaning, and does it still count as hate speech? If a substantial fraction, say more than 10%, fail either check, the claim that the LLM self-check loop produces human-comparable detoxification data is undermined.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that LLM-generated parallel detoxification data is of human-comparable quality and sufficient to train competitive detoxification models. The evidence is a direct A/B test: retraining ParaDetox with the LLM as annotator yields a model with style accuracy 0.98 versus 0.96 for the human-annotated version, fluency 0.91 versus 0.71, and content preservation 0.70 versus 0.85, which the paper reads as comparable overall with different strengths. The same three-step procedure applied to hate speech yields ParaDeHate, whose LLM reference outputs score 0.98 style accuracy, 0.74 content preservation, and 0.76 fluency, and a BART-large model fine-tuned on it outperforms all evaluated baselines on style accuracy, fluency, and BLEU. The paper concludes that LLM-in-the-loop data creation is a scalable alternative to human annotation for hate speech detoxification.

Load-bearing premise

The whole result rests on GPT-4o-mini being a reliable judge of its own rewrites, because the same model decides both content preservation and remaining toxicity, and no human verifies the final 8,276 pairs.

Editorial extensions

If this is right

  • If the central claim is right, parallel detoxification data can be generated at near-zero marginal cost; the full ParaDeHate build cost $3.39 in API fees, so the bottleneck that limited supervised style transfer to a few high-resource settings largely disappears.
  • Hate-speech-specific training data matters: existing unsupervised and zero-shot baselines either keep toxicity in the output, as with Duplicate and BART-zero-shot, or destroy fluency and meaning, as with DRG and CondBERT, while BART fine-tuned on ParaDeHate reaches 0.95 style accuracy, 0.78 fluency, and 0.31 BLEU.
  • Replacing human annotators with an LLM preserves downstream performance: BART-large trained on ParaDetox-LLM is comparable to BART-large trained on ParaDetox-Human across the three quality axes, with a trade-off between content preservation and fluency.
  • ParaDeHate can serve as a benchmark for hate speech detoxification because it provides a test set with reference rewrites and results for a wide range of existing baseline methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the paper leaves implicit is that the dataset's quality depends on GPT-4o-mini not being systematically lenient about its own outputs, since the same model generates and validates every rewrite; the moderate kappa of 0.55 against embedding similarity on content preservation makes an independent human audit the natural next check.
  • The pipeline's cost and speed suggest it could be re-run on emerging hate speech domains or other languages as soon as new data appears, but the paper's English-only evaluation leaves open whether the self-check loop holds up under cross-cultural variation in what counts as hate speech.
  • A testable extension is to swap the validator for a different model or a human judge; if agreement drops sharply, the loop's reliability is model-specific rather than a general property of LLM self-annotation.
  • Because the reported BLEU scores compare generated outputs against the LLM's own reference text, part of the measured gain may reflect learning the LLM's rewriting style rather than true detoxification; human judgments of meaning preservation would clarify what the benchmark actually rewards.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an LLM-in-the-loop pipeline for constructing parallel detoxification datasets, using GPT-4o-mini as both generator and validator. The authors first claim to replicate the ParaDetox pipeline by replacing human annotators with the LLM, and report comparable downstream performance to the human-annotated version (Table 1). They then extend the pipeline to hate speech, producing ParaDeHate, a dataset of 8,276 hate/non-hate text pairs, and show that a BART-large model fine-tuned on this dataset outperforms several baseline detoxification methods (Table 3). The paper releases the dataset and reports cost details.

Significance. If the central claim were established, the work would be significant: it would demonstrate a scalable, low-cost alternative to human annotation for detoxification data and would provide a new benchmark for hate speech rewriting. The authors are transparent about the pipeline, release the dataset, and include cost estimates, which are strengths. However, the current validation is insufficient to support the claim that LLM annotation is 'comparable to human annotation,' and the dataset quality is not independently verified. The significance of the contribution therefore hinges on additional experiments and analyses that are not present in this manuscript.

major comments (4)
  1. [§3.4, Table 1] The comparison between ParaDetox-Human and ParaDetox-LLM is confounded: each BART model is evaluated on its own dataset's test set, so the style accuracy, content preservation, fluency, and BLEU values are not directly comparable. BLEU is computed against different reference sets, and the other metrics are measured on different inputs. This design does not support the abstract's claim that 'the LLM performs comparably to human annotation.' To support that claim, the authors should evaluate both models on a common human-annotated test set (e.g., the original ParaDetox test set) or perform cross-evaluation (train on one dataset, test on the other).
  2. [§3.2–§3.3, §4.2] The same GPT-4o-mini model is used for all three roles: Task 1 (generation), Task 2 (content preservation check), and Task 3 (toxicity check). The final 8,276 ParaDeHate pairs are therefore accepted or rejected by the model that generated them. The automatic thresholds (0.70 cosine similarity, 0.9 toxicity score) are not applied as independent filters; they are only used to compute Cohen's kappa against the LLM's own labels. There is no human verification of the final dataset. This circularity means the dataset could contain meaning-changing or still-toxic pairs while appearing clean on automatic metrics. The Limitations section does not mention this absence of human validation, which is a central gap for a dataset paper.
  3. [§5.2, Table 3] The evaluation of models on ParaDeHate uses LLM-generated references as ground truth, and the test set itself is produced by the same pipeline. No human evaluation of the references is reported. The content preservation metric for BART fine-tune (0.78) is computed as the LaBSE cosine similarity between the model output and the original toxic input, not between the output and a gold detoxified reference, so it does not measure how well the model reproduces the reference detoxification. Consequently, the headline result that BART fine-tuned on ParaDeHate outperforms all baselines rests entirely on automatic metrics whose validity depends on the unvalidated quality of the LLM-generated test set.
  4. [§3.4, Table 1 (also §4.2)] The paper does not report any inter-annotator agreement for the human-annotated ParaDetox data, nor does it compare the LLM's judgments against human judgments at the pair level. The Cohen's kappa values in §3.2 and §3.3 measure agreement between the LLM and heuristic thresholds, not agreement with humans. Without a human-LLM agreement study, the claim that the LLM 'replaces' human annotators is not directly tested.
minor comments (5)
  1. [§4.1] The dataset is referred to as 'HateExplain' in the text, but the cited dataset and the rest of the paper use 'HateXplain.' Please correct the spelling.
  2. [§5.1, Table 3] The header 'P ARADEHATE' contains a spurious space; it should read 'ParaDeHate.'
  3. [§3.4, Table 1] The caption for Table 1 states that BLEU is computed against 'the reference text from ParaDetox-Human and ParaDetox-LLM respectively,' which confirms that the comparison is not apples-to-apples. A sentence in the main text explaining this caveat would help readers interpret the table.
  4. [§5.2, Table 3] The 'LLM-reference' row is the reference itself (BLEU = 1.00); it would be clearer to label this row as 'Reference' or 'Upper bound' rather than implying it is a model to compare against.
  5. [Limitations] The Limitations section lists model choice, closed-source dependency, and English-only data, but conspicuously does not mention the lack of human validation of the generated dataset. Adding a discussion of this limitation and its implications would strengthen the paper's scientific honesty.

Circularity Check

2 steps flagged · score 5.0 of 10

Partial circularity: the same GPT-4o-mini generates and validates ParaDeHate, and Table 1's BLEU comparison scores each model against its own dataset's references, so the 'human-comparable' claim is not independently tested.

  1. self definitional [Section 3.2–3.3; Figure 2 caption; Section 4.2]
    "An LLM acts as the annotator, performing three tasks: rephrasing hate speech, verifying content preservation, and evaluating toxicity. Texts that pass all three checks are considered detoxified and are included in the resulting parallel dataset."

    The inclusion criterion for the dataset is the same LLM's own 'Yes'/'No' judgments on content preservation (Task 2) and toxicity (Task 3) for its own rewrites (Task 1). Thus the dataset is, by construction, the set of texts that GPT-4o-mini considers meaning-preserving and non-toxic. When Section 5.2 later reports 'LLM-generated text in PARADEHATE demonstrates high quality with 0.98 style accuracy, 0.74 preservation,' the preservation and non-toxicity properties are partly restatements of the filtering rule that created the dataset. No human gold standard checks these properties on the final 8,276 pairs. External classifiers (LaBSE, Detoxify, CoLA) provide some independent signal, so the circularity is partial rather than total.

  2. other [Section 3.4, Table 1 caption]
    "BLEU denotes the BLEU score of BART-large generated text and the reference text from ParaDetox-Human and ParaDetox-LLM respectively."

    The headline comparison used to conclude 'LLMs can generate parallel detoxification datasets with quality on par with human annotations' includes a BLEU comparison in which each model is scored against its own dataset's references: LLM-generated references for PARADETOX-LLM and human references for PARADETOX-HUMAN. The two BLEU scores are therefore not measurements of the same target; the LLM-trained model is compared against the LLM's own outputs, not against human detoxifications. This makes the human-comparability conclusion partially self-referential. The other metrics in Table 1 come from external classifiers and provide independent evidence, which is why this is a partial circularity rather than a complete reduction.

full rationale

The paper's core construction is a dataset-generation pipeline, not a mathematical derivation, and much of the downstream evaluation is genuinely external: style accuracy uses the Detoxify/RoBERTa classifier, content preservation uses LaBSE embeddings, fluency uses a CoLA-trained RoBERTa, and BART is fine-tuned on held-out splits and compared against several baselines. Those results have independent content. No load-bearing self-citation chain is present, and no uniqueness theorem is imported from the authors' prior work. The partial circularity resides in the validation of the headline claim that the LLM performs comparably to human annotation. Section 3.2–3.3 uses the same GPT-4o-mini model that generated each rewrite to decide whether the rewrite preserves meaning and is non-toxic; the final 8,276 ParaDeHate pairs are therefore, by construction, the set the LLM judged acceptable. Section 5.2 then reports the LLM-reference row as 'high quality' on those same properties. Table 1's BLEU comparison compounds this: PARADETOX-LLM BLEU is computed against LLM-generated references while PARADETOX-HUMAN BLEU is computed against human references, so the two BLEU scores are not measuring the same target. The Limitations section acknowledges model choice, closed-source, and English-only scope, but not the absence of human validation of the final pairs, which is the central gap. I therefore rate the circularity as partial (5), not total: the automatic external classifiers and the held-out training/evaluation protocol prevent the entire contribution from reducing to the LLM's own judgments.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The three free parameters are all hand-set or data-derived choices that control which pairs survive the pipeline. The two thresholds are fitted to small subsamples and directly determine the dataset composition. The axioms reflect the paper's reliance on existing dataset labels, automatic metrics, and the self-validating LLM, none of which are independently verified for the final ParaDeHate corpus.

free parameters (3)
  • Cosine similarity threshold for content preservation = 0.70
    Set empirically based on observations from a subsample of text pairs (Section 3.2). It affects which pairs are labeled as meaning-preserving.
  • Toxicity score threshold = 0.9
    Set empirically from subsamples (Section 3.3). Samples above threshold are treated as toxic and discarded. This is a hand-tuned cut-off on a data-driven score.
  • Sampling temperature for GPT-4o-mini = 0.6
    Chosen by hand for generation (Appendix A); affects the diversity and quality of rewrites.
assumptions (4)
  • domain assumption The merged hate speech datasets (CreHate, HateXplain, Davidson, Founta) provide reliable hate speech labels, and their 'hate speech' categories are comparable across datasets.
    Section 4.1 aggregates samples using the hate speech labels from each source; if labels are noisy or inconsistent, the input set to the pipeline is contaminated.
  • ad hoc to paper GPT-4o-mini's judgments of content preservation and toxicity are valid and consistent with human judgments.
    Tasks 2 and 3 use the same model that generated the rewrites; the paper reports kappa against automatic thresholds but no human agreement on the final dataset.
  • domain assumption Automatic metrics (LaBSE cosine similarity, Detoxify/Unitary toxicity classifier, CoLA-based fluency) are adequate proxies for the qualities they claim to measure.
    All evaluations in Sections 3.4 and 5 rely on these automatic metrics; no human evaluation is reported for ParaDeHate.
  • domain assumption The behavior of GPT-4o-mini is stable across API versions and over time.
    The dataset is generated once with a closed model; the paper does not track versioning, so reproduction may differ.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxification." pith.science (2026). https://pith.science/paper/YNW5DLYB

@misc{pith2026250601484,
  author       = {Pith},
  title        = {Pith review of: LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YNW5DLYB}},
  note         = {Machine review of arXiv:2506.01484}
}
read the original abstract

Detoxification, the task of rewriting harmful language into non-toxic text, has become increasingly important amid the growing prevalence of toxic content online. However, high-quality parallel datasets for detoxification, especially for hate speech, remain scarce due to the cost and sensitivity of human annotation. In this paper, we propose a novel LLM-in-the-loop pipeline leveraging GPT-4o-mini for automated detoxification. We first replicate the ParaDetox pipeline by replacing human annotators with an LLM and show that the LLM performs comparably to human annotation. Building on this, we construct ParaDeHate, a large-scale parallel dataset specifically for hatespeech detoxification. We release ParaDeHate as a benchmark of over 8K hate/non-hate text pairs and evaluate a wide range of baseline methods. Experimental results show that models such as BART, fine-tuned on ParaDeHate, achieve better performance in style accuracy, content preservation, and fluency, demonstrating the effectiveness of LLM-generated detoxification text as a scalable alternative to human annotation.

Figures

Figures reproduced from arXiv: 2506.01484 by the authors.

Figure 1
Figure 1. An example of a hate speech input and its [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pipeline for constructing PARADEHATE. We begin by collecting hate speech texts from widely used datasets. An LLM acts as the annotator, performing three tasks: rephrasing hate speech, verifying content preservation, and evaluating toxicity. Texts that pass all three checks are considered detoxified and are included in the resulting parallel dataset. general forms of toxicity (e.g., offensive or profane language), wh… view at source ↗
Figure 3
Figure 3. Prompt for Task 1: Generation of Paraphrases. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Prompt for Task 3: Toxicity Check. score above 0.9 are labeled as still containing toxic content, while those below 0.9 are labeled as non-toxic. We again compute Cohen’s kappa coefficient to assess the inter-annotator agreement between the LLM’s judgments and the toxi…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 18 canonical work pages

  1. [1]

    Eleftheria Briakou, Di Lu, Ke Zhang, and Joel Tetreault. 2021. https://doi.org/10.18653/v1/2021.naacl-main.256 Ol \'a , bonjour, salve! XFORMAL : A benchmark for multilingual formality style transfer . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 31...

    doi:10.18653/v1/2021.naacl-main.256 2021

  2. [2]

    Keith Carlson, Allen Riddell, and Daniel Rockmore. 2018. Evaluating prose style transfer with the bible. Royal Society open science, 5(10):171920

    work page 2018

  3. [3]

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. 2024 a . https://openreview.net/forum?id=dbFEFHAD79 MLLM -as-a-judge: Assessing multimodal LLM -as-a-judge with vision-language benchmark . In Forty-first International Conference on Machine Learning

    work page 2024

  4. [4]

    Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024 b . https://doi.org/10.18653/v1/2024.emnlp-main.474 Humans or LLM s as the judge? a study on judgement bias . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8301--8327, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.emnlp-main.474 2024

  5. [5]

    David Dale, Anton Voronov, Daryna Dementieva, Varvara Logacheva, Olga Kozlova, Nikita Semenov, and Alexander Panchenko. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.629 Text detoxification using large pre-trained neural models . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7979--7996, Online and Pun...

    doi:10.18653/v1/2021.emnlp-main.629 2021

  6. [6]

    Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the 11th International AAAI Conference on Web and Social Media, ICWSM '17, pages 512--515

    2017

  7. [7]

    Daryna Dementieva, Nikolay Babakov, and Alexander Panchenko. 2024. https://doi.org/10.18653/v1/2024.naacl-short.12 M ulti P ara D etox: Extending text detoxification with parallel data to new languages . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume...

    doi:10.18653/v1/2024.naacl-short.12 2024

  8. [8]

    Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. https://doi.org/10.18653/v1/2022.acl-long.62 Language-agnostic BERT sentence embedding . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878--891, Dublin, Ireland. Association for Computational Linguistics

    doi:10.18653/v1/2022.acl-long.62 2022

Show all 64 references
  1. [9]

    Paula Fortuna, Monica Dominguez, Leo Wanner, and Zeerak Talat. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.809 Directions for NLP practices applied to online hate speech detection . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing...

    2022 doi

  2. [10]

    Antigoni Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of twitter abusive behavior. In Proceedings of th...

    2018

  3. [11]

    Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. https://doi.org/10.1609/aaai.v32i1.11330 Style transfer in text: Exploration and evaluation . Proceedings of the AAAI Conference on Artificial Intelligence, 32(1)

    2018 doi

  4. [12]

    Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. https://doi.org/10.1073/pnas.2305016120 Chatgpt outperforms crowd workers for text-annotation tasks . Proceedings of the National Academy of Sciences, 120(30):e2305016120

    2023 doi

  5. [13]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  6. [14]

    Laura Hanu and Unitary team . 2020 a . Detoxify. Github. https://github.com/unitaryai/detoxify

    2020

  7. [15]

    Laura Hanu and Unitary team . 2020 b . Detoxify. Github. https://github.com/unitaryai/detoxify

    2020

  8. [16]

    Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. https://doi.org/10.18653/v1/2022.acl-long.234 T oxi G en: A large-scale machine-generated dataset for adversarial and implicit hate speech detection . In Proceedings of the 60th A...

    2022 doi

  9. [17]

    Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick. 2020. https://openreview.net/forum?id=HJlA0C4tPS A probabilistic formulation of unsupervised text style transfer . In International Conference on Learning Representations

    2020

  10. [18]

    Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. https://doi.org/10.1162/coli_a_00426 Deep learning for text style transfer: A survey . Computational Linguistics, 48(1):155--205

    2022 doi

  11. [19]

    Youngwook Kim, Shinwoo Park, and Yo-Sub Han. 2022. https://aclanthology.org/2022.coling-1.579/ Generalizable implicit hate speech detection using contrastive learning . In Proceedings of the 29th International Conference on Computational Linguistics, pages 6667--6679, Gyeongju...

    2022

  12. [20]

    Abdullatif K \"o ksal, Timo Schick, Anna Korhonen, and Hinrich Schuetze. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.414 L ong F orm: Effective instruction tuning with reverse instructions . In Findings of the Association for Computational Linguistics: EMNLP 2024, pa...

    2024 doi

  13. [21]

    Yevhen Kostiuk, Atnafu Lambebo Tonja, Grigori Sidorov, and Olga Kolesnikova. 2023. Automatic translation of hate speech to non-hate speech in social media texts. arXiv preprint arXiv:2306.01261

    2023 arXiv

  14. [22]

    Meltem Kurt Pehlivano g lu, Robera Tadesse Gobosho, Muhammad Abdan Syakura, Vimal Shanmuganathan, and Luis de-la Fuente-Valent \' n. 2024. Comparative analysis of paraphrasing performance of chatgpt, gpt-3, and t5 language models using a new chatgpt generated dataset: Paragpt....

    2024

  15. [23]

    Wen Lai, Viktor Hangya, and Alexander Fraser. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.745 Style-specific neurons for steering LLM s in text style transfer . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13427--13443,...

    2024 doi

  16. [24]

    L \'e o Laugier, John Pavlopoulos, Jeffrey Sorensen, and Lucas Dixon. 2021. https://doi.org/10.18653/v1/2021.eacl-main.124 Civil rephrases of toxic texts with self-supervised transformers . In Proceedings of the 16th Conference of the European Chapter of the Association for Co...

    2021 doi

  17. [25]

    Dong-Ho Lee, Jay Pujara, Mohit Sewak, Ryen White, and Sujay Jauhar. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.948 Making large language models better data creators . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15349-...

    2023 doi

  18. [26]

    Nayeon Lee, Chani Jung, Junho Myung, Jiho Jin, Jose Camacho-Collados, Juho Kim, and Alice Oh. 2024. https://doi.org/10.18653/v1/2024.naacl-long.236 Exploring cross-cultural differences in E nglish hate speech annotations: From dataset construction to analysis . In Proceedings ...

    2024 doi

  19. [27]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. https://doi.org/10.18653/v1/2020.acl-main.703 BART : Denoising sequence-to-sequence pre-training for natural language generation, translatio...

    2020 doi

  20. [28]

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, and 1 others. 2024 a . From generation to judgment: Opportunities and challenges of llm-as-a-judge. arXiv preprint arXiv:2411.16594

    2024

  21. [29]

    Juncen Li, Robin Jia, He He, and Percy Liang. 2018. https://doi.org/10.18653/v1/N18-1169 Delete, retrieve, generate: a simple approach to sentiment and style transfer . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Li...

    2018 doi

  22. [30]

    Wong, and Min Yang

    Renhao Li, Minghuan Tan, Derek F. Wong, and Min Yang. 2024 b . https://doi.org/10.18653/v1/2024.emnlp-main.271 C o E vol: Constructing better responses for instruction finetuning through multi-agent cooperation . In Proceedings of the 2024 Conference on Empirical Methods in Na...

    2024 doi

  23. [31]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  24. [32]

    Varvara Logacheva, Daryna Dementieva, Sergey Ustyantsev, Daniil Moskovskiy, David Dale, Irina Krotova, Nikita Semenov, and Alexander Panchenko. 2022. https://doi.org/10.18653/v1/2022.acl-long.469 P ara D etox: Detoxification with parallel data . In Proceedings of the 60th Annu...

    2022 doi

  25. [33]

    Fuli Luo, Peng Li, Jie Zhou, Pengcheng Yang, Baobao Chang, Xu Sun, and Zhifang Sui. 2019. https://doi.org/10.24963/ijcai.2019/711 A dual reinforcement learning framework for unsupervised text style transfer . In Proceedings of the Twenty-Eighth International Joint Conference o...

    2019 doi

  26. [34]

    Eric Malmi, Aliaksei Severyn, and Sascha Rothe. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.699 Unsupervised text style transfer with padded masked language models . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages ...

    2020 doi

  27. [35]

    Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 14867--14875

    2021

  28. [36]

    Ercong Nie, Bo Shao, Zifeng Ding, Mingyang Wang, Helmut Schmid, and Hinrich Sch \"u tze. 2024. Bmike-53: Investigating cross-lingual knowledge editing with in-context learning. arXiv preprint arXiv:2406.17764

    2024 arXiv

  29. [37]

    Cicero Nogueira dos Santos, Igor Melnyk, and Inkit Padhi. 2018. https://doi.org/10.18653/v1/P18-2031 Fighting offensive language on social media with unsupervised text style transfer . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (...

    2018 doi

  30. [38]

    Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2024. https://doi.org/10.1162/tacl_a_00660 Automatically correcting large language models: Surveying the landscape of diverse automated correction strategies . Transactions of the Associ...

    2024 doi

  31. [39]

    Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020. Automatically neutralizing subjective bias in text. In Proceedings of the aaai conference on artificial intelligence, volume 34, pages 480--489

    2020

  32. [40]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(1)

    2020

  33. [41]

    Sudha Rao and Joel Tetreault. 2018. https://doi.org/10.18653/v1/N18-1012 Dear sir or madam, may I introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association ...

    2018 doi

  34. [42]

    Nils Reimers and Iryna Gurevych. 2019. https://doi.org/10.18653/v1/D19-1410 Sentence- BERT : Sentence embeddings using S iamese BERT -networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference...

    2019 doi

  35. [43]

    Paul R \"o ttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. https://doi.org/10.18653/v1/2024.naacl-long.301 XST est: A test suite for identifying exaggerated safety behaviours in large language models . In Proceedings of the 2024 Co...

    2024 doi

  36. [44]

    Paul R \"o ttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. https://doi.org/10.18653/v1/2021.acl-long.4 H ate C heck: Functional tests for hate speech detection models . In Proceedings of the 59th Annual Meeting of the Associatio...

    2021 doi

  37. [45]

    Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. Advances in neural information processing systems, 30

    2017

  38. [46]

    Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.54 Large language models for data annotation and synthesis: A survey . In Proceedings of...

    2024 doi

  39. [47]

    Cagri Toraman, Furkan S ahinu c , and Eyup Yilmaz. 2022. https://aclanthology.org/2022.lrec-1.238/ Large-scale hate speech detection with cross-domain transfer . In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2215--2225, Marseille, France....

    2022

  40. [48]

    Minh Tran, Yipeng Zhang, and Mohammad Soleymani. 2020. https://doi.org/10.18653/v1/2020.coling-main.190 Towards a friendly online community: An unsupervised style transfer framework for profanity redaction . In Proceedings of the 28th International Conference on Computational ...

    2020 doi

  41. [49]

    Nafis Irtiza Tripto, Saranya Venkatraman, Dominik Macko, Robert Moro, Ivan Srba, Adaku Uchendu, Thai Le, and Dongwon Lee. 2024. https://doi.org/10.18653/v1/2024.acl-long.357 A ship of theseus: Curious cases of paraphrasing in LLM -generated texts . In Proceedings of the 62nd A...

    2024 doi

  42. [50]

    Advaitha Vetagiri, Eisha Halder, Ayanangshu Das Majumder, Partha Pakray, and Amitava Das. 2024. https://aclanthology.org/2024.icon-1.34/ MULTILATE : A synthetic dataset on AI -generated MULTI moda L h ATE speech . In Proceedings of the 21st International Conference on Natural ...

    2024

  43. [51]

    Somin Wadhwa, Silvio Amir, and Byron Wallace. 2023. https://doi.org/10.18653/v1/2023.acl-long.868 Revisiting relation extraction in the era of large language models . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2023 doi

  44. [52]

    Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019. https://doi.org/10.1162/tacl_a_00290 Neural network acceptability judgments . Transactions of the Association for Computational Linguistics, 7:625--641

    2019 doi

  45. [53]

    Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.210 Challenges in detoxifying language models . In Findings...

    2021 doi

  46. [54]

    Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. 2024. Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge. arXiv preprint arXiv:2407.19594

    2024 arXiv

  47. [55]

    Xing Wu, Tao Zhang, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. https://doi.org/10.24963/ijcai.2019/732 Mask and infill: Applying masked language model for sentiment transfer . In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence,...

    2019 doi

  48. [56]

    Ifeoluwa Wuraola, Nina Dethlefs, and Daniel Marciniak. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.869 Understanding slang with LLM s: Modelling cross-cultural nuances through paraphrasing . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language P...

    2024 doi

  49. [57]

    Xinli Yu, Zheng Chen, and Yanbin Lu. 2023. https://doi.org/10.18653/v1/2023.emnlp-industry.69 Harnessing LLM s for temporal data - a study on explainable financial time series forecasting . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processi...

    2023 doi

  50. [58]

    Shuzhou Yuan, Antonis Maronikolakis, and Hinrich Sch \"u tze. 2022. https://doi.org/10.18653/v1/2022.woah-1.1 Separating hate speech and offensive language classes via adversarial debiasing . In Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), pages 1--10, S...

    2022 doi

  51. [59]

    Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. https://doi.org/10.18653/v1/S19-2010 S em E val-2019 task 6: Identifying and categorizing offensive language in social media ( O ffens E val) . In Proceedings of the 13th Inte...

    2019 doi

  52. [60]

    Ruoyu Zhang, Yanzeng Li, Yongliang Ma, Ming Zhou, and Lei Zou. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.872 LLM a AA : Making large language models as active annotators . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13088--13103,...

    2023 doi

  53. [61]

    Yi Zhang, Tao Ge, and Xu Sun. 2020. https://doi.org/10.18653/v1/2020.acl-main.294 Parallel data augmentation for formality style transfer . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3221--3228, Online. Association for Com...

    2020 doi

  54. [62]

    Gonzalez, and Ion Stoica

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. https://openreview.net/forum?id=uccHPGDlao Judging LLM -as-a-judge with MT -bench and chatbot ...

    2023

  55. [63]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  56. [64]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.